Creating a simple cryptocurrency: part 10

The latest repository.

Consensus determination (continued)

Consensus determination has been enhanced so that even if consensus has been lost, it can be recovered without a full manual restart if the consensus loss was caused by too many servers going offline and enough of those return.

Consensus must be reached by the end of the beta period. At the beginning of the next beta period, the following algorithm assesses the situation:

if (myBalHash !== null) {
    if (consensus === null) {
      'no consensus, loading saved balances'.log('cr')
      bl.load()
    } else if (consensus.equals(myBalHash)) {
      'consensus reached and i agree, saving balances'.log('cr')
      bl.save(myBalHash)
    }
  }

myBalHash is the hash of the local balances database and consensus is the consensus hash. Both of these variables can be null: the former if the local server recently started and the latter if consensus was not reached.

Testing limits

A balances database was created with 50,000 random accounts to determine whether the program can keep it all in memory and process it within the current one minute beta period, including sending the entire database to a restarted out-of-sync server. Processing took only six seconds, so there appeared to be room to increase the the number of accounts. But when 100,000 accounts was tried, servers frequently froze.

Some research revealed that the problem was due to using a file system directory as a database, for the sake of simplicity. Evidently, according to this report there is a work-around to make this approach work, but it involves direct access to the operating system. Instead, another approach was used; see below.

With the new database, 2,000,000 random accounts required about 30 seconds to process, including sending the database to a restarted out-of-sync server. Half of the beta period was used, so to allow for a safe margin, the server consensus interval should be increased to two minutes or more for greater numbers of accounts.

The above tests were performed with three servers running on a single laptop (and in one test one of the servers ran on another laptop on the local network).

The largest number of payments that could be processed was about 800 in the 45 second alpha period. The bottleneck seems to be the UDP socket, not CPU speed. This amount of data is small compared to the potential size of the balances database, and is easily handled aside from the bottleneck. But even including the 15 second blackout period, more than 10 transactions per second can be handled, surpassing Bitcoin's 7 transactions per second. Certainly adequate for a local community.

New balances database

The new database is simply all accounts packed into a single block. It is loaded from a single file on disk and kept in memory, in its entirety. It is updated with incoming transactions and saved to disk each time consensus is reached. If consensus fails, the disk file is reloaded and consensus is attempted again. Each version of the disk file is saved in an archive folder.

Because the database file is packed raw data, it can not be manually manipulated as easily as the former database composed of hex-string files in a directory. But the program automatically initializes the database with the largest possible balance in a single account belonging to the rank zero server. From this account, coins can be distributed to a million or more accounts.

String methods moved into program

To reduce dependencies, all String methods were moved from the npm string-methods package to the beginning of the program in an immediately executing function. The opportunity was taken to add some useful Buffer and Number methods, to make the code more consistently styled.

< part 9 | part 11 >