ShipChain Validator Architecture guide

ShipChain Validator Architecture guide

This article is based on BlockMatrix's deleted Loom Validator Architecture medium article. Because ShipChain is based on Loom, this might be of use for validators of ShipChain. Credits of the idea go to BlockMatrix.

This article is intended as a guide for a proposed optimal safe setup for validators of ShipChain. However, it is in no way the only way or standard to do things. Rather, the more different setups, the better.

Keep the validator node secure
The main design consideration for a validator on any blockchain network is this: protecting the validating node at all costs!

A validator node should not be exposed to the public internet. It should remain hidden, only contactable by your own trusted nodes through a private (preferably encrypted) network, coupled with aggressive firewall settings, access policies and monitoring.

Sentry nodes
In ShipChain’s mainnet, the way to achieve this protection is through the use of sentry nodes. These are public nodes, they peer with other nodes in the network, offer APIs to access the underlying chain data and they can elastically scale based on demand. Another reason for using sentries is extra protection against DDoS attacks, therefore, the sentries should be located in various geographical regions.

An example of cloud-based sentry node configuration with connectivity to a private validator node.

It is optional whether you want to interconnect the Sentry nodes within each Virtual Private Cloud (VPC) region, however, they should all peer privately with the validator node. There are many ways to do this, from Direct Connect in AWS, Partner Interconnect in GCP, and many vendor-agnostic solutions such as a VPN, IPSec tunnel or P2P mesh via Wireguard.

ShipChain Configuration

The nodes need to be configured so that the validator node only speaks to the sentry nodes on the internal network, and the sentry nodes do not gossip the validator information out to the external network. To achieve this, a few settings in the config.toml configuration file must be modified.

The first thing we need to do is to grab the node keys. To do this, run hydra client cat-key nodekey on each node and note them down along with the internal IP address. To get your IP-addresses, enter the following command and get your ipv4 address (hint: not 127.0.0.1 or 192.168.x.x ):
$ ip -4 addr | grep -oP '(?<=inet\s)\d+(\.\d+){3}'

The node keys and IP-addresses of other validators which you are connected to can be found in ./shipchain-mainnet/addrbook.json

Validator Configuration

• The first step is controlling connections through a whitelist of peers, the first thing we can do is disable the peer exchange reactor: pex = false

• As all communication is over a local and private network, the strict address book must be disabled: addr_book_strict = false

• Finally, the sentry nodes can be defined (and backup/failover validator nodes) with a permanent list of peers which follow the pattern of nodekey@local_ip:46656 For example: persistent_peers = "[email protected]:46656,[email protected]:46656,..."

Sentry Configuration

• As sentry nodes need to communicate with other public peers, the peer exchange reactor must be enabled: pex = true

• Communication over a local and private network must also be allowed: addr_book_strict = false

• To ensure that the validator details are not leaked through the network, add their node keys to: private_peer_ids = "2137216ef21653126ab21736128ea76712,1762176e98a90b98c98982398e98f98"

• For our peering list, we want to add the validator node(s) and a selection of other sentries: persistent_peers = “[email protected]:46656,[email protected]:46656,…”

Scaling
Currently, it is not possible to dynamically update the persistent peer list for nodes already online. Therefore, to bring in additional sentry nodes, the existing node configs need to be updated and the hydra service restarted. hydra client restart-service

Relay Nodes

There is a third node type, called a relay node. The idea with these nodes is to achieve better peering for the validators, whilst removing the dependency on the local sentry nodes being available 100% of the time (this is an obvious attack vector). These tend to be lightweight nodes, connected to external, trusted peers over a private connection. They are set up similar to the sentry nodes, however, the peer exchange reactor is off: pex = false

The reason for these node types is to ensure you never connect external peers directly to your validator, therefore these relays act like hidden peering nodes where one end is connected to a list of trusted peers, and the other is connected to your validator on a separate, private connection.

Conclusion

There are obvious tradeoffs in the above examples (like extra latency), this setup has been battle-tested and performed well in both simulated and production environments.

The goal is to keep the validator node sufficiently protected, whilst maintaining a varied peering network to ensure latency is kept to a minimum and to have no dependency on a single node, service, or region in order to remain in sync with the network.

A highly recommended security addition would be a hot failover node. Another article in the future might be published about that.