Creating a toy cryptocurrency - Part 2a: P2P Networking

Hi everyone, this is the 2nd part of a series of posts on creating a cryptocurrency, for an overview of the series see part 1 here:

https://steemit.com/cryptocurrency/@garethnelsonuk/creating-a-toy-cryptocurrency-part-1

Now let's get to it!
This article will go over the basics of a P2P network and end with building a very simple peer-to-peer node in Python. If you're following along, it's assumed that you know how to code in Python and have at least some basic understanding of networking and sockets. If not, it is advisable to catch up a bit on these subjects if you intend to follow the series and implement your own node.

The basics - P2P vs Centralised client/server

Imagine you're reading a random web page, perhaps the wikipedia article on lolcats (a highly recommended article by the way). This article is sent to you across the internet from wikipedia's servers. Your web browser running on your computer is the client.
This is the client/server model that is used in many applications across the internet: one machine is a client, and another machine is the server. Servers serve multiple clients and perform services for those clients.

Historically, to pay money across the internet you needed to use a centralised service of some kind such as paypal or your bank's online banking service. The centralised party keeps track of account balances and clients talk to that centralised party to move funds around. This is the only way you can deal with funds stored by a central party and until Satoshi invented bitcoin it was the only possible way to move funds online at all.

This applies for other forms of communication too - there exist centralised vs P2P versions of chat/IM clients, video streaming and file transfers and all sorts of other services. There also exist "hybrid" systems where a centralised party uses a P2P network to offload traffic - services such as Spotify use this model for example - the music i'm listening to right now may come from another random spotify user's computer or from spotify's central servers depending on network conditions.

P2P network structure and topology

Now we know the basic difference between P2P and client/server let's look at how such networks are structured.
As shown above, this is the traditional client/server model where multiple clients talk to one server:

A P2P model on the other hand has every machine acting as both a client and server (literally so as we'll see when it comes to implementation):

(With much thanks to wikipedia and others for these images)

At the most basic level, a P2P network must perform a few basic functions:

1. Connecting together peers that are not directly able to reach each other
2. Responding to requests to perform whatever function the network is designed for (whether that be a cryptocurrency, filesharing, music/video streaming or whatever else)
3. Enabling new users to connect to the network and get started

In the case of a P2P network for file sharing (the most common usage of P2P networks still) this means being able to locate data stored by the peers and downloading it. Whether the target file is cut up into blocks or stored as just one file does not matter to the structure of the network - what matters is each node must be able to locate the data in other nodes in a search operation.

Now let's look at a basic and naive implementation of a filesharing P2P node - bonus points to anyone who spots the immediate flaw.

Implementing a filesharing node

As above, we'll now begin implementation of a simple filesharing node in python. We'll choose a dynamic UDP port to listen on, as opposed to choosing one that may already be registered to IANA - this also has another useful feature in that some firewalls and NAT devices work better with high port ranges.

Why use UDP you ask? UDP is naturally message-orientated and much more widespread than nicer protocols like SCTP (which in all likelihood you've never actually used - it's not widely supported and known).

One of the first questions we must ask is "how are we going to format the actual protocol messages?" and to keep things simple we'll use the wonderful msgpack library. Beyond that we won't use anything outside the standard python library in Python 2.7 - i'm using a Linux box, but it should be possible to follow along on windows (just don't ask me how), OS X or any of the BSDs.

Let's get started then!

First, open yourself a terminal and install msgpack:
pip install msgpack-python
If you don't have pip installed, go install it from https://pip.pypa.io/en/stable/installing/ and then come back here.
Now give it a quick test, go start up python in a terminal and try it out:

Python 2.7.9 (default, Mar  1 2015, 12:57:24) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import msgpack
>>> msgpack.packb({'testing':1234})
'\x81\xa7testing\xcd\x04\xd2'

Ok, we now have all the tools we need to implement a very basic filesharing node. For the sake of simplicity we will use a python dict to store the "files", though obviously a real node would use the filesystem. Let's write the code, look through it and then figure out the flaws and then figure out both how to do it properly AND what on earth this has to do with cryptocurrency.

First, we bind a socket, listen for packets and unpack the msgpack format:

import socket
import msgpack
import pprint

the_files = {'coolstuff.txt':'Hey, this is the contents of a cool text file',
             'epic awesome music.mp3':'Not an actual MP3, but you get the idea'}

# Create our UDP socket and bind it to a random high port on all interfaces
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.bind(('',0))

# Tell the user what port it's on
print 'Listening for peers on UDP port %s' % sock.getsockname()[1]

# Listen for packets and then do stuff with them
while True:
   in_data,in_addr = sock.recvfrom(65536) # get a packet and get data
   print 'Got a packet from %s' % str(in_addr)

  # Try and decode the packet's data and then print the contents
   try:
      data = msgpack.unpackb(in_data)
      pprint.pprint(data)
   except:
      print 'Could not unpack data from peer'
~                                                 

Go ahead and save that code into a file (name it "filesharenode.py") and run it, you should get the following output:

$ python filesharenode.py 
Listening for peers on UDP port 39944

Now let's give it a quick test, open another terminal and send a string to the socket:

$ python
Python 2.7.9 (default, Mar  1 2015, 12:57:24) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
>>> s.bind(('',0))
>>> s.sendto('this is not valid',('127.0.0.1',39944))
17

This is fairly simple stuff if you're familiar with the sockets API, we're just opening another UDP socket, binding to any available port and sending the string "this is not valid" to the socket binded by filesharenode.py. On my machine that resulted in this output:

Got a packet from ('127.0.0.1', 58599)
Could not unpack data from peer

You should get the same output with just a different hostname. Now let's encode something in msgpack and send it.
From the same python instance you used to send the invalid string, simply import msgpack and pack any data structure you want before sending it:

>>> import msgpack
>>> valid_data = msgpack.packb({'whatever':1234})
>>> s.sendto(valid_data,('127.0.0.1',39944))
13

In my example, I packed a dict with one key: {'whatever':1234} and it appeared at filesharenode.py like this:

Got a packet from ('127.0.0.1', 58599)
{'whatever': 1234}

So we now have a simple P2P node that is capable of unpacking messages in msgpack format, woohoo - give yourself a pat on the back and check out part 2b of this series and we'll continue on to actually serving files and then get to how this ties in with cryptocurrency.

As before, if you're getting value out of my posts please upvote them and tip me in bitcoin using 14vL5HUzqTLfFv9Pf9Vy7naKhJb35HQ6Mn

Part 2b will be coming very soon - just didn't want to make this post TOO big.