EOS Tribe Infrastructure Upgrade

in eos •  3 months ago

infrastructure-upgrade-Q4.jpg

Since our last Infrastructure upgrade EOS Tribe have been busy experimenting with various server configurations. Our goal was to build Enterprise level reliable and scalable EOS API infrastructure.

We have quickly discovered that IO operations and not processor speed that is the most critical component in maintaining a healthy cluster of EOS API nodes as well as producer node.

Thanks to EOS Tribe partnership with our Data Center partner Syptec we were able to build and tear down a complete server infrastructure based on stress test results.

We have rebuilt our entire infrastructure 3 times trying various IO technologies.

At the end we have stopped on bare metal servers with attached NVMe SSD drives by Samsung that deliver consistent top performance with heavy random read/write operations characteristics of Nodeos process.

Currently we are dedicating 3 physical servers to EOS api1, api2 and api3 load balanced on api.eostribe.io and 1 physical server for Worbli API node. And one specialized physical server with server grade 4.0 Ghz processor to Producer Node.

You can see specification for each server on diagram below:

EOS Tribe Servers.png

We are running 2 nodeos instances on each server with one using history api plugin. And one nodeos instance on api3 server is also synchronizing with MongoDB plugin while we are working on alternative history plugin initiative :

Below is EOS Tribe overall EOS Infrastructure deployment diagram supported by above servers:

EOS Tribe Infrastructure v2.png

We are running our Primary and Backup Block Producer on secured Wireguard Mesh network directly peered with other top Block Producer nodes (via seed node).

EOS Tribe block producing node is also running latest version of Heartbeat plugin. As one could see from CPU statistics reported by Heartbeat monitor EOS Tribe producer is running on one of the fastest CPU on the network:

Screen Shot 2018-11-07 at 5.17.20 PM.png

EOS Tribe servers are hosted in secure Data Center in Bluffdale, UT on a fast low latency 10G connection. We are running in the same facility Facebook is running it's servers.

It requires multiple biometrics access checkpoints for access to the server room. Photographs are usually forbidden however with permission of our provider we can provide a sneak peak into the facility:

20181107_143331_1541626425897.jpg

1541625852427001_1541625864096.jpg

1541625928677001_1541625931054.jpg

1541625939388001_1541625942598.jpg

20181107_143434_1541626481792.jpg

20181107_143554_1541626565744.jpg

The power generators you see on a last picture along with battery backup power inside a facility is capable to power entire Data Center for up to 36 days in case of total power loss to facility.

EOS Tribe goal is to provide enterprise level support for EOS network as a block producer and provide stable and scalable API to development community.



Connect with us! We're building a better future on EOSIO.

Website | Medium | Github | Telegram | FB | Twitter | Discord


Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

That is fantastic and exactly what block producers should be competing on. Other value adds welcome but this is why I vote for you guys. Great work

Some serious work guys! Good job.