More networking fixes (#384)

* Don't connect to ourselves for Kademlia

* Improve log messages

* Manually drop connections

* Kademlia is now 32 seconds

* Fix potential panic

* Fix other potential panic

* No longer pass the endpoint everywhere