Here at Cypherglass we are a bit old-school on certain things. One is software updates. We go with an enterprise grade/financial institute philosophy of thorough testing prior to production release, rather than an Agile practice of releasing code straight into the wild. Our belief is that we aren’t running the latest version of Candy Crush or Words with Friends, but that we are operating software responsible for billions of currency transactions on which people’s livelihoods are dependent.

Our underlying principle of always protecting the EOS network and token-holder is incorporated into deployment of new Mainnet software releases.

1. We thoroughly test any version in our lab using the Jungle Testnet before deployment into our production environment. This process typically takes a day or two before we are satisfied, but can take longer if we see something we don’t like.
2. Once thoroughly tested in the lab, we upgrade our production nodes in a the following specific hierarchical order based on system criticality...
   * Internal Blockchain Backup Node
   * Internal Proxy Nodes
   * Internal Full Nodes
   * Public Full Nodes
   * Production Backup Block Producer Node
   * Production Block Producer Node

Recently while testing version 1.0.8 of the Mainnet code, we encountered a bug related to resource settings. We tested different configuration settings, and eventually figured out what would work with the new code. During this time, several other BPs encountered the error on their production nodes causing an outage of their production Block Producer nodes, and required Standby Block Producers to step in and produce blocks.

While we were testing the new 1.0.9 Mainnet code, Block.One released a 1.0.10 version to fix bugs in the 1.0.9 code. Glad we hadn’t deployed it already.

These are just a couple examples of why Cypherglass would rather err on the side of caution when upgrading our production environment. No code is perfect, it all has bugs, so being cautious is something we have built into our culture.

Cheers from the Cypherglass Infrastructure team and go EOS!!!!

Cypherglass.com
SteemIt
Telegram
Twitter
Facebook