diff --git a/exchange/bitswap/README.md b/exchange/bitswap/README.md index bfa0aaa868b7173680a07d68fc71d0fcc70954ac..cfdbd27e08e599034f333d5094787a6819176bff 100644 --- a/exchange/bitswap/README.md +++ b/exchange/bitswap/README.md @@ -1,47 +1,37 @@ -#Welcome to Bitswap -###(The data trading engine) +# Bitswap + +## Protocol +Bitswap is the data trading module for ipfs, it manages requesting and sending +blocks to and from other peers in the network. Bitswap has two main jobs, the +first is to acquire blocks requested by the client from the network. The second +is to judiciously send blocks in its posession to other peers who want them. + +Bitswap is a message based protocol, as opposed to response-reply. All messages +contain wantlists, or blocks. Upon receiving a wantlist, a node should consider +sending out wanted blocks if they have them. Upon receiving blocks, the node +should send out a notification called a 'Cancel' signifying that they no longer +want the block. At a protocol level, bitswap is very simple. + +## go-ipfs Implementation +Internally, when a message with a wantlist is received, it is sent to the +decision engine to be considered, and blocks that we have that are wanted are +placed into the peer request queue. Any block we possess that is wanted by +another peer has a task in the peer request queue created for it. The peer +request queue is a priority queue that sorts available tasks by some metric, +currently, that metric is very simple and aims to fairly address the tasks +of each other peer. More advanced decision logic will be implemented in the +future. Task workers pull tasks to be done off of the queue, retreive the block +to be sent, and send it off. The number of task workers is limited by a constant +factor. + +Client requests for new blocks are handled by the want manager, for every new +block (or set of blocks) wanted, the 'WantBlocks' method is invoked. The want +manager then ensures that connected peers are notified of the new block that we +want by sending the new entries to a message queue for each peer. The message +queue will loop while there is work available and do the following: 1) Ensure it +has a connection to its peer, 2) grab the message to be sent, and 3) send it. +If new messages are added while the loop is in steps 1 or 3, the messages are +combined into one to avoid having to keep an actual queue and send multiple +messages. The same process occurs when the client receives a block and sends a +cancel message for it. -Bitswap is the module that is responsible for requesting and providing data -blocks over the network to and from other ipfs peers. The role of bitswap is -to be a merchant in the large global marketplace of data. - -##Main Operations -Bitswap has three high level operations: - -- **GetBlocks** - - `GetBlocks` is a bitswap method used to request multiple blocks that are likely -to all be provided by the same set of peers (part of a single file, for example). - -- **GetBlock** - - `GetBlock` is a special case of `GetBlocks` that just requests a single block. - -- **HasBlock** - - `HasBlock` registers a local block with bitswap. Bitswap will then send that -block to any connected peers who want it (with the strategies approval), record -that transaction in the ledger and announce to the DHT that the block is being -provided. - -##Internal Details -All `GetBlock` requests are relayed into a single for-select loop via channels. -Calls to `GetBlocks` will have `FindProviders` called for only the first key in -the set initially, This is an optimization attempting to cut down on the number -of RPCs required. After a timeout (specified by the strategies -`GetRebroadcastDelay`) Bitswap will iterate through all keys still in the local -wantlist, perform a find providers call for each, and sent the wantlist out to -those providers. This is the fallback behaviour for cases where our initial -assumption about one peer potentially having multiple blocks in a set does not -hold true. - -When receiving messages, Bitswaps `ReceiveMessage` method is called. A bitswap -message may contain the wantlist of the peer who sent the message, and an array -of blocks that were on our local wantlist. Any blocks we receive in a bitswap -message will be passed to `HasBlock`, and the other peers wantlist gets updated -in the strategy by `bs.strategy.MessageReceived`. -If another peers wantlist is received, Bitswap will call its strategies -`ShouldSendBlockToPeer` method to determine whether or not the other peer will -be sent the block they are requesting (if we even have it). - -##Outstanding TODOs: -- [ ] Ensure only one request active per key -- [ ] More involved strategies -- [ ] Ensure only wanted blocks are counted in ledgers