Efficient steemd initial sync and blockchain replay

I was quite busy last days and was trying to run my own dockerized steemd on the side. I learnt how long does it take to sync and replay the blockchain and do the initial sync.

I wasted a lot of time but in the end I found an efficient way to do initial startup of a fresh steemd node.

Note: This is for someone that wants to quickly set up a steemd node for development and/or Piston usage on a local blockchain copy.

Most problably this is already mentioned somewhere else, but I need the information in one place and here it goes:

• Build steemd in a RELEASE mode for the initial sync. You can do the sync, then switch to debug version if needed. The sync on debug takes ages, at least for my i5 and i7 (8GB RAM on both laptops + fast SSD drives EVO 850 pro). I set the following cname flags for the build and this significalty speeded up the sync:

cname_flags=-DCMAKE_BUILD_TYPE=Release \
            -DENABLE_CONTENT_PATCHING=OFF \
            -DLOW_MEMORY_NODE=ON

• Use an extremely short config.ini for sync and replay. I just enabled witness plugin. Account_history is a major culprit for slowing sync down.

$ cat custom/config.ini 
seed-node = 52.4.250.181:39705
checkpoint = [1782161, "001b3191f02df6e842b701b22b0ffacdc3c1d33e"]
rpc-endpoint = 127.0.0.1:8090
enable-stale-production = false
required-participation = false
enable-plugin = witness

• Use steemd develop branch for development. I encountered several issues on the latest tag, also master is currently 2 weeks delayed. I had some weird errors due to follower plugin and I was advised by @str11ngfello on the chat to cherry pick a bug fix for the follow plugin. I decided to go with the bleeding edge develop branch instead.

git checkout develop && git submodule update --init --recursive

• If you see 'last block ID does not match current chain state' error just call steemd with --replay-blockchain option. Complete wipe of the db is not needed.

steemd --data-dir /steemdata/witness_node_data_dir --replay-blockchain

• Always try to gracefully close the steemd server. Press ctrl+C and wait. Otherwise you will need to replay the whole blockchain again which currently takes ~10 mins with my minimal config.
• Use preloaded blockchain. I created a dockerized microcontainer holding the fresh snapshot that can be built on a Docker HUB and converted it into a named volume container. I will try to describe the approach in my next post.