GitLab

This addr manager should seriously help with the addrsplosion
problem.

http://gifs.gifbin.com/012011/1295375531_cat-jump-fail.gif

About to allow dht to start without local addresses. this is
so that we can initialize the dht and sign it up to listen
on the muxer, before our node starts accepting incoming connections.
otherwise, we lose some (we're observing this happening already).

I looked through the dht's use of the peerstore, and the check here
doesnt seem to be as important as the panic implies. I believe the
panic was used for debugging weird "dont have any address" conditions
we had earlier.

Network now signals when it successfully listens on some address
or when an address shuts down. This will be used to establish and
close nat port mappings. It could also be used to notify peers
of address changes.

the debug log is flooded with pages upon pages of...

we've gotta be more judicious with our use of console logs. i'm sure
there's interesting actionable information in here. let's use the
console logging more like a sniper rifle and less like birdshot.

feel free to revert if there are specific critical statements in this
changeset

03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) QUERY worker for: <peer.ID QmSoLp> - not
found, and no closer peers. prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) completed prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLp>) finished prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) 0 provider entries prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) 0 provider entries decoded prefixlog.go:107
03:05:24.096 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) got closer peers: 0 [] prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID QmWGN3>)
FindProviders(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK)
Query(<peer.ID QmSoLn>) end prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) query finished prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) QUERY worker for: <peer.ID QmSoLn> - not
found, and no closer peers. prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) completed prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
queryPeer(<peer.ID QmSoLn>) finished prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
all peers ended prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
spawnWorkers end prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
failure: %s routing: not found prefixlog.go:107
03:05:24.097 DEBUG        dht: dht(<peer.ID
QmWGN3>).Query(QmXvrpUZXCYaCkf1jfaQTJASS91xd47Yih2rnVC5YbFAAK).Run(3)
end prefixlog.go:107

more races 2015-01-24

+ longer timeout

Many times, a node will start up only to shut down immediately.
In these cases, reproviding is costly to both the node, and the
rest of the network. Also note: the probability of a node being
up another minute increases with uptime.

TODO: maybe this should be 5 * time.Minute

Moved it to its own package to isolate scope.

s/kademlia calls for makign sure to query all peers we
have in our routing table, not just those closest. this
helps ensure most queries resolve properly.

When some queries finished, but we got no result, it should
be a simple NotFoundError. Only when every single query ended
in error do we externalize those to the client, in case
something major is going wrong

This commit moves the record validation/verification
from dht/ into the new record/ packaage. Validator object
-- which is merely a map of ValidatorFuncs -- with a
VerifyRecord

cc @whyrusleeping

this time, without loading the private key on every startup

mitigated adding our own addresses where received
from peers

see #573