Complete Witness Installation Guide

in #witness-category7 years ago (edited)

Welcome to a step-by-step "foolproof" guide to manually install a witness and seed node.

All you have to do is type these commands one after the other. If something goes wrong, it's not likely covered here, sorry. Everything works for me after installing several times. You will have to do some research to resolve your issue, or ask for help in the witness steemit.chat channel.


Prerequisites

Servers

You will need two servers, one for the Witness Node, and one for the Seed Node. This is up to you to figure out what to get, like two VPSs, or a dedicated host that you run as VPSs, etc.

Depending on your witness position, and the size of the blockchain (currently at 7GB), you may need to upgrade. It's up to you to choose to go smaller and then bigger, or start big at first and plan ahead.

A recommend a minimum to start off of 2 CPUs at 2GHz, with 4GB RAM (and 8GB swap virtual RAM) for your witness node that will be signing blocks. Similar specs should be used for the seed node. If you can do better, great! I have not had issues with my specs so far at my position.

OS

I recommend choosing Ubuntu 16.04.1 as your OS, but you can choose any OS and figure out the equivalent commands if the ones I provide don't work. Up to you.

Securing Your Server Tutorials

I wrote some tutorials for securing your VPS if you are unsure of what some security measures you should consider taking.

Tutorials to help users secure their server for a Witness:

Do not skimp on the security measures.

With that introduction done, lets begin!



Witness Installation


1) Install Steem

sudo apt-get update

sudo apt-get install -y autoconf automake cmake g++ git libssl-dev libtool make pkg-config

sudo apt-get install -y libboost-chrono-dev libboost-context-dev libboost-coroutine-dev libboost-date-time-dev libboost-filesystem-dev libboost-iostreams-dev libboost-locale-dev libboost-program-options-dev libboost-serialization-dev libboost-signals-dev libboost-system-dev libboost-test-dev libboost-thread-dev

sudo apt-get install -y doxygen libncurses5-dev libreadline-dev perl
git clone https://github.com/steemit/steem
cd steem
git checkout v0.16.2

Use the latest version. If ever you install another version, change it in the checkout command above.

Now make and compile the source:

git submodule update --init --recursive
mkdir build
cd build
cmake -DLOW_MEMORY_NODE=ON -DCMAKE_BUILD_TYPE=Release ..

The commands below can take a bit of time.

make -j$(nproc) steemd
make -j$(nproc) cli_wallet

When that is done, install it:

sudo make install

A extra step I take for clarity and later upgrades, is to move the programs folder into another folder for production running. In my home directory I make a folder for the version, like mkdir steem0162, cd steem0162, then I move the programs folder with mv steem/build/programs .


2) Create and edit the config.ini

Startup the steemd client for a few seconds, then ctrl+c to kill it:

cd programs/steemd
./steemd

Kill it with ctrl+c.

This creates the config.ini file which we now edit:

nano witness_node_data_dir/config.ini

You will need to modify the file on each server, both the Witness Node and the Seed Node.

Witness Node config.ini

Lets do the witness config first.

Find these sections, and modify the values accordingly.

Make sure the p2p-endpoint is not commented out with a hash sign #. If it is, remove the hash.

p2p-endpoint=0.0.0.0:2001

Below is a list of seed nodes you can add to the confing. Copy and paste them into the rpc-endpoint section.

seed-node = 149.56.108.203:2001         # @krnel (CA)
seed-node = anyx.co:2001                # @anyx (CA)
seed-node = gtg.steem.house:2001        # @gtg (PL)
seed-node = seed.jesta.us:2001          # @jesta (US)
seed-node = 212.117.213.186:2016        # @liondani (SWISS)
seed-node = seed.riversteem.com:2001    # @riverhead (NL)
seed-node = seed.steemd.com:34191       # @roadscape (US)
seed-node = seed.steemnodes.com:2001    # @wackou (NL)
seed-node = 192.99.4.226:2001
seed-node = 45.55.217.111:2001
seed-node = 104.236.82.250:2001
seed-node = steem.clawmap.com:2001
seed-node = seed.steemnodes.com:2001
seed-node = seed.steemwitness.com:2001

Ping each of the seed nodes to find the best speeds to use, comment out the one's that don't work. You can place them in any order you want. Pick the best ones for your location.

Next, find the the sections for rpc-endpoint, public-api, enable-plugin and make them like this:

rpc-endpoint = 0.0.0.0:8090
public-api = database_api login_api
enable-plugin = witness

The extra api and plugins are not required for now.

Seed Node config.ini

Do the same as above. You can copy and paste the config from the witness node so far, except don't put rpc-endpoint, comment it out if you did copy/paste from the witness node:

#rpc-endpoint = 0.0.0.0:8090

For the seed, add an extra api here that was in the default config.ini. If you didn't copy/paste, then this is already here:

public-api = database_api login_api account_by_key_api

The seed node has just one thing to not put (rpc-endpoint), and one additional api, that's it.

Add your witness name if you want, or not and wait until later when we get the WIF private key.


3) Download existing blocks (optional, but recommended)

This can speed up the loading of the blockchain the first time. I recommend it.

You can go to the folder the blockchain needs to be in, or move it later after you download:

cd witness_node_data_dir/blockchain

Download the file (currently 7GB):

wget https://gtg.steem.house/get/blockchain/block_log

That will download the existing blockchain and save you time.


4) Sync up the blockchain

Get blockchain data (if skipped above step)

Start a screen if you want to be able to do something else with your terminal session:

screen -S steem

Navigate to the steemd folder.
If you did the download above, just do cd ../.., otherwise find your way to programs/steemd/.

You have two options, depending on if you downloaded the blocklog data:

a) No download of the blockchain above:

Do a resync if it's your first time.

./steemd --resync-blockchain

This takes time to download and sync the blocks.

b) Downloaded the blockchain above:

If you already have the blockchain block_log data from above, do a replay:

./steemd --replay-blockchain

This also takes time, but a lot less time than if you didn't download the blockchain before hand.

If at any time you want to leave this screen window and come back later, just do ctrl+a+d. Then you can move around the file system again. To get back into the screen, do screen -r steem

After this runs and shows something similar to below, specifically "Got # transactions from...", it's done and you can close steemd for now with ctrl+c, and then exit the screen for now with ctrl+a+d, or close the screen completely with exit.

396166ms th_a       application.cpp:491           handle_block         ] Got 4 transactions from network on block 8959960
399071ms th_a       application.cpp:491           handle_block         ] Got 4 transactions from network on block 8959961
402192ms th_a       application.cpp:491           handle_block         ] Got 8 transactions from network on block 8959962

Do all of this for the Seed Node as well. Your seed node taks end here. Once the seed node is up and giving the above results, you're done for the seed node. For the seed node, please also consider the section "Some additional steps to consider", starting at step 14.


5) Use the cli_wallet to get the WIF private key

Note, if at any time you get an error running a command inthe cli_wallet, you need to ctrl+c kill the wallet, then restart it. Don't waste your time after getting an error: close and reopen.

You can do this on the witness node, on the seed node, or on your own computer (that is, if your computer is running linux).

Start the wallet

Go to the cli_wallet, open it in a screen if you want to exit and leave it open:

cd programs/cli_wallet
screen -S wallet
./cli_wallet -s wss://node.steem.ws/

Set a password for the wallet

Then you reach the CLI and a prompt: new >>>. Now we are in the CLI Wallet and can continue.

Pick a nice crazy "cli_password" and write it down, then set that password in your cli_wallet:

set_password cli_password

Then unlock the wallet with your password:

unlock cli_password

Get your WIF Private Key

Get your ACTIVE_PRIVATE_KEY from steemit.com/@yourname/permissions, and click "login to show" your active key, then put it below in this command:

import_key ACTIVE_PRIVATE_KEY

Now we need to get the WIF Private Key

Input this command:

suggest_brain_key

Result looks something like:

suggest_brain_key
{
"brain_priv_key": "FFDJKH CYRHJA DFJJF YBFJIJ BHVVE TSDCBJK OGENN DBHBHJJ BFDUKWF MJKLHF VGBFN FGNGM JDFKHF KDFJDFH SPKASNF FJFDHF",
"wif_priv_key": "5Hw38jSDKJFGkFSJHD7sfKJFDK72nf0h7hJNFJNFkj7nln08uBFJBKJ",
"pub_key": "STM6jhgj67KJDJGFJHBFJ6767KHGVWSPOD897FBhbwl9y8bKFBlBVF87"
}

Save this data as well. Don't lost it.

If you want to close the cli_wallet in a screen or terminal, do ctrl+c.


6) Edit the config.ini again

We are going to add the witness name and WIF private key.

Return to the config.ini and set our witness name and private key:

nano witness_node_data_dir/config.ini
witness = "krnel"
private-key = wif_priv_key_from_suggest_brain_key

Make sure your name is in quotes, and not the WIF private key.


7) First a quick intro on screen:

Screen allows you to run a program without using the terminal itself to run it which would otherwise leave you unable to use the terminal. It's handy.

  • Resume if only one screen, otherwise a list of screens appears to choose from:
    screen -r

  • Create a new screen with a name to reference it:
    screen -S NAME

  • Create a new screen from your current scree:
    ctrl + a+c

  • Switch between screens:
    ctrl + n

  • Exit screen, but keep it alive:
    ctrl + a+d

  • Resume a named screen
    screen -r NAME

  • Kill a screen that you aren't in (this will kill steemd, cli_wallet, or any other program that is running inside the screen):
    screen -X -S #### quit

To get the number, use screen -r when you have more than one screen.


8) Start steemd up with screen

screen -r steem
./steemd --replay-blockchain

If you didn't previously use screen -S steem, then do that instead of the "-r" parameter.

After a bit of time, you will get the " Got 10 transactions from network on block ..." displaying again. Now your witness node is up again.

b) Optional for cli_wallet

If you want to do the same screen setup with your cli_wallet:

screen -S wallet
./cli_wallet -s wss://node.steem.ws/
unlock cli_password


9) Update Witness

Test the "update_witness" command to make sure everything is working.

Using the cli_wallet, either in a screen, or in the terminal itself, run this command after unlocking it and importing your active key (it won't work if you don't import your acative key):

Example

screen -S wallet
./cli_wallet -s wss://node.steem.ws/
set_password cli_password
unlock cli_password
import_key ACTIVE_PRIVATE_KEY

update_witness "<your-witness-name>" "<your-witness-post>" "<your-public-brain-key-from-above>" {"account_creation_fee":"10.000 STEEM","maximum_block_size":65536,"sbd_interest_rate":1000} true

  • your witness post
  • your brain public key from above
  • account creation fee (varies)
  • don't touch the blocksize
  • sbd interest (varies)

Here is what I did (replace the data as you need to):

update_witness "krnel" "https://steemit.com/ANY_POST_YOU_MADE" "STM6jhgj67KJDJGFJHBFJ6767KHGVWSPOD897FBhbwl9y8bKFBlBVF87" {"account_creation_fee":"20.000 STEEM","maximum_block_size":65536,"sbd_interest_rate":400} true

I do this just to make sure everything works, and then we can declare being a witness and update it with the real post link afterwards.

Remember, if you get an error, close the cli_wallet by killing it with ctrl+c, then open it again and try another time. Also make sure your wallet is "unlocked" unlocked >>> before typing in a command, or else it won't work.

If this work without errors, set your price feed.


10) Publish the Price Feed

Check here for other price feeds to get an idea of where other witnesses are going, until you can understand how things work more: https://steemd.com/witnesses and https://steemdb.com/witnesses

Here is the command, replace my name with your own:

publish_feed krnel { "base":"0.164 SBD", "quote":"1.020 STEEM"} true

There are also auto price feeds if you want to use those, such as @someguy123's Steemfeed-JS - A NodeJS price feed for witneses.

You will need to update the price feed accordingly, every few days at least. @clayop and other witnesses have explanation on the price feed mechanism: https://steemit.com/steemit/@clayop/the-impact-of-price-feed-discount-on-the-market.


11) Verify Witness Status

In the cli_wallet do:

get_witness your_name

ex: "get_witness krnel"

The output should end with something similar to this:

",
  "running_version": "0.16.2",
  "hardfork_version_vote": "0.0.0",
  "hardfork_time_vote": "2016-03-24T16:00:00"
}
unlocked >>> 

Everything works.

If you try to get_witness before you update, it wont work, and give a null result.

You're good to finalize everything by declaring your witness intentions on Steemit, then update your witness with that post.


12) Post Your Witness Declaration

Here is what my Witness Declaration looks like: KrNel's Declaration of Witness


13) Update Witness with Witness Thread

After that is posted, use the cli_wallet to post an update again, this time with your witness thread in it:

update_witness "krnel" "https://steemit.com/witness-category/@krnel/krnel-s-declaration-of-witness" "STM6jhgj67KJDJGFJHBFJ6767KHGVWSPOD897FBhbwl9y8bKFBlBVF87" {"account_creation_fee":"20.000 STEEM","maximum_block_size":65536,"sbd_interest_rate":400} true

And check your status again:

get_witness krnel

Success!



Some additional steps to consider


14) Linux tweaks (recommended)

From Steem 0.16.0 Release Notes:

For those running steemd on linux there are several virtual memory configuration options we recommend. The Linux virtual memory writes dirty pages of the shared file out to disk more often than is optimal which results in steemd being slowed down by redundant IO operations. These settings were chosen to optimize reindex time.

echo    75 | sudo tee /proc/sys/vm/dirty_background_ratio
echo  1000 | sudo tee /proc/sys/vm/dirty_expire_centisecs
echo    80 | sudo tee /proc/sys/vm/dirty_ratio
echo 30000 | sudo tee /proc/sys/vm/dirty_writeback_centisecs

Just put each of those commands in and that's it.

Do this tweaking on the Witness and Seed Nodes.


15) Shared Memory Improvements (optional)

If you have issues with disk I/O writing, you might want to switch to RAM or VRAM swap for better performance of the shared memory file and writing.

Increase your swap memory as shown by @abit's post here, and run steemd with that new shared-file-dir location:

Change the size of /dev/shm

sudo mount -o remount,size=10G /dev/shm

Modify config.ini shared-file-dir

shared-file-dir = /shm/

Or run steemd with the location specified:

./steemd --shared-file-dir /dev/shm/ --replay

If you already have the shared memory files created previously, you need to move them to the /dev/shm:

cp witness_node_data_dir/blockchain/shared_memory.* /dev/shm/

Do this on the Witness and Seed Nodes as you see fit.


16) Autorun (optional)

Use supervisor to auto run the witness and seed nodes on restart of the server:

Install supervisor

sudo apt-get install supervisor
sudo systemctl enable supervisor

Set a config for steem:

nano /etc/supervisor/conf.d/steem.conf

Add this to the file and change the "/path/to/steemd" for the actual path in your system (use pwd to find the absolute path to your steemd file:

[program:steemd]
command=steemd --replay-blockchain
directory=/path/to/steemd
user=steem
startsecs=1800
autostart=true
autorestart=true
stdout_logfile=AUTO
stderr_logfile=AUTO

After that, you can restart supervisor to pick up the new config:

sudo service supervisor restart

If you need to stop or start steemd, use:

sudo supervisorctl stop steemd
sudo supervisorctl start steemd

To check the status, use:

sudo supervisorctl status


Congratulations! You're a witness!


Thank you to those who came before and helped put out information to become a witness.

I would like to specifically thank this guide by @bitcalm: How to become a witness, as well as thank @bitcoinparadise, @teamsteem and @instructor2121 for their help in assisting me setup my witness and seed nodes.


Thank you for your time and attention! I appreciate the knowledge reaching more people. Take care. Peace.


If you appreciate and value the content, please consider:

Upvoting ,    Sharing or   Reblogging below.

Follow me for more content to come!


Please also consider supporting me as a Steem Witness by voting for me at the bottom of the Witness page; or just click on the upvote button if I am in the top 50:



@krnel
2017-01-31, 10:16am

Sort:  

Hello @krnel,

Congratulations! Your post has been chosen by the communities of SteemTrail as one of our top picks today.

Also, as a selection for being a top pick today, you have been awarded a TRAIL token for your participation on our innovative platform...STEEM.
Please visit SteemTrail to get instructions on how to claim your TRAIL token today.

If you wish to not receive comments from SteemTrail, please reply with "Stop" to opt out.

Happy TRAIL!

for the commands:
make -j$(nproc) steemd
make -j$(nproc) cli_wallet

you need to do sudo or they fail:

sudo make -j$(nproc) steemd
sudo make -j$(nproc) cli_wallet

@krnel LOVE YOUR GUIDE HERE thank you very much because after reading SO MANY other guides, this provides still 5 months later the best combination of an updated guide combined with a comprehensive list of what needs to be done from first downloading putty to securing the server to getting the witness and node running. I voted you up 83M as a witness today and followed you to help me see your newest posts! With me not being able to contribute anything on this post, I voted up a couple of your newer posts!

Sweet, thanks for the appreciation and support, I appreciate it ;) Glad it's still of use to others :)

this is a good guide. thank you!

You're welcome.

I'm glad I have been able to help you. It's an honor. I really love your work here on Steem. Keep it up! This is another awesome post!

thanks for the tutorial. i'm interested in being a witness and this info is well needed.
one question. is there a minimum spec for internet connection. i have access to a server and could run it from my home, however i only have a mediocre internet provider (standard not fibre) which currently gives me a download speed of 8.05mb/s and upload of 2.59 Mb/s
I realise i would need a 2nd server for seed node which i may consider renting online. does 1 server require higher bandwith than the other? could i run 1 from home or would i need to upgrade my internet?

Welcome. You need a dedicated line.

Thanks
Ive just been looking online at server providers, i notice there are different services i.e. KVM, OpenVZ, Openstack KVM, are they all suitable? also i followed a link in 1 of your previous posts to https://www.ovh.co.uk they have 3 options for vps, SSD, CLOUD RAM, and CLOUD with Ceph. are they all suitable?

The more pricey the better usually.

You can also look for local companies, with hardware hosted in your area, or your country, if you want to support a local node for faster access to users in your own country.

Awesome work @krnel. This is greatly appreciated! A solid resteem.

Welcome :)

This information seriously needed an update. Nice job krnel.

Great info Krnel, filling in a few blanks for me 😁 resteemed

Welcome :)

Just thought I'd note, for posterity as I'm sure this post will get looked at a lot as time goes by, there is no v0.16.2 branch

git checkout v0.16.2

It looks like the stable branch is the best one to go for, at time of writing they just merged in a pull request for v0.16.3, so we've already moved on. So should be

git checkout stable

Ah thanks! That's a better one so people don't have to keep track of the version each time.

Thank you for highlighting the seed node list and their owners. Will reuse for steemdata node :)

Coin Marketplace

STEEM 0.31
TRX 0.11
JST 0.034
BTC 64852.72
ETH 3178.07
USDT 1.00
SBD 4.20