Creating a toy cryptocurrency - part 2b: Finishing our P2P node

Hi everyone, welcome to part 2b of my series on creating a toy cryptocurrency. For those who haven't read previous posts, please start here:

https://steemit.com/cryptocurrency/@garethnelsonuk/creating-a-toy-cryptocurrency-part-1

Now, let's continue!

Previously on steemit.....

Last time we discussed the basics of P2P networking and built the world's simplest P2P node using python and msgpack. As it stands, this program is hardly worthy of the name "P2P node" and just about establishes how we connect and what format our messages are in at the most basic level. We need to expand on this to build an actual network of nodes and we need to elaborate on the messages. Let's begin then.

What does a node need to do?

Before we can look into finishing off the node we need to figure out exactly what a node needs to do, this actually maps quite readily to the types of message it needs to exchange with other nodes. Our basic node needs 2 basic types of messages: those that any P2P protocol requires and those specific to the application - we'll define them as python data structures that can be encoded using msgpack. We also need to specify a unique ID for each message, so replies can be tracked.

Below, we assume 2 nodes are communicating - node A is talking to node B

Request peer list

After connecting to a seed node we need a list of other peers to connect to, this is how the network is built. We'll discuss methods of finding seed nodes later, for now let's focus on the messages.

{'msgtype':'getpeerlist','msgid':1234} # this is sent from node A to node B
{'msgtype':'peerlist','replyto':1234,'peers':((10.0.0.1,12036),(10.0.0.2:12456))} # this is sent from node B back to node A - as you can see, the replyto field indicates the message we're replying to
# in the above, node B replied with 2 peers it's aware of, it does not of course include node A in the reply

Search

This is an interesting one, and it is the flaw in our first basic design - basically to find a file, our protocol will simply broadcast this message to all the peers it knows about and wait for replies. See if you can spot the flaw in this design - i'll reveal it after we code the implementation, but it should be fairly obvious if you ponder it for a bit. Some kind of search procedure is needed by all P2P protocols - they all need to locate files, or users, or transactions and blocks......

{'msgtype':'search','msgid':1234,'filename':'coolstuff.txt'} # node A sends this to node B, and to node C (an assumed other node)
# upon receiving this message, a node should either respond if it has the file or knows where to find it, or it should rebroadcast the message
# a node that knows the location of the file should reply with the below
{'msgtype':'searchresult','replyto':1234,'atpeer':(10.0.0.1,12523)} # the "atpeer" field is just an IP address and port of the peer that has the file

Get file contents

In a real and proper P2P protocol, this would actually be rather complex to implement on top of UDP, so for our toy protocol we'll instead cheat by just using really small "files". A real-world solution would instead make use of TCP sockets (which cause issues due to NAT and firewalls) or would essentially rebuild TCP's reliability features on top of UDP. Both these approaches are beyond the scope of this article, but interested readers are encouraged to look into a block-based solution (as bittorrent and others use), or something equivalent to the "circuits" concept used by the virtual world Second Life - a client/server protocol that nonetheless offers many lessons for UDP networking in general.

{'msgtype':'getfile','msgid':1234,'filename':'coolstuff.txt'} # node A sends this to node B
{'msgtype':'filecontents','replyto':1234,'filecontents':'SGV5LCB0aGlzIGlzIHRoZSBjb250ZW50cyBvZiBhIGNvb2wgdGV4dCBmaWxl'}
# node B responds with the above, the filecontents field is base64-encoded - this is not strictly necessary when dealing with plain text and in msgpack even with binary, but it makes things easier to debug as we can actually output it to a terminal

Implementing our basic P2P node

Now we know the message types we can implement our file transfer node. To make things simple, we'll turn it into a python module so we can interact with it on the terminal and manipulate state directly. First, let's put all our socket code and state into a class and make it operate in a thread so we can run it in a background thread. We'll also remove the dict containing the file contents that we added in the last article so we can test nodes that don't have all the files. To make things simple, our class constructor will start the thread immediately.

First, let's have a basic skeleton that only binds the socket and receives packets. We'll drop a lot of the extra debugging output from the program as we now know it all works and we'll add a special handler that can dispatch messages based on the type. This uses a rather cool feature of Python that you won't find in non-dynamic languages: dynamic method invocation.

Essentially, we can add several handle_MSGTYPE functions and have our code dynamically call the relevant method. For the sake of keeping the code simple we'll also make a few silly assumptions and not bother with error checking - in a real system we'd naturally want to error check everything.

I've also added a simple "ping" message type:

import socket
import msgpack
import pprint
import thread

class FileShareNode:
   def __init__(self,files):
       self.file_content = files
       self.sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
       self.sock.bind(('',0))
       print 'Listening for peers on UDP port %s' % self.sock.getsockname()[1]
       thread.start_new_thread(self._sock_thread,())
   def handle_ping(self,data,addr):
       reply_data = msgpack.packb({'msgtype':'pong','replyto':data['msgid'])
       self.sock.sendto(reply_data,addr)
   def _sock_thread(self):
       while True:
         in_data,in_addr = sock.recvfrom(65536) 
         data = msgpack.unpackb(in_data)
         pprint.pprint((in_addr,data))
         getattr(self,'handle_%s' % data['msgtype'])(data,in_addr)

Let's test this out then, save the code to a file and open 2 terminals, run python in each and get ready to test, see my screenshot below to see how:

(Note - you may need to right click and open in a new tab if this image is too small to read)

You'll notice I haven't put anything into the files for now - this is just a basic networking test.

What i'm doing here in the screenshot is setting up 2 instances of the basic node and sending a message of type ping from one to the other by manually manipulating the socket object. The node in the first terminal sends a ping message to the node in the second terminal, and gets back a "pong" message - then promptly throws an exception because we have no handler for the pong type.

Your homework assignment is to implement a pong handler and a ping() method. This will be on the exam.

Unfortunately to implement the rest of the messages i'll need to post another article as this is running into steem's limits - but as always if you're getting value from these posts, please upvote and tip me in bitcoin at 14vL5HUzqTLfFv9Pf9Vy7naKhJb35HQ6Mn