Your guide to setting up a Witness Server (STEEM-in-a-box HF19)

4 months ago

NOTE: This article contains both upgrade instructions, AND install-from-fresh instructions.

This article is designed for those running a previous version of my Steem-in-a-box software, or for those looking to set up a new witness server.

If you want to learn more about witnesses and what they do, check out my article here: https://steemit.com/witness-category/@someguy123/seriously-what-is-a-witness-why-should-i-care-how-do-i-become-one-answer

NEW FEATURES

new features include:

  • dlblocks (downloads blocks from @gtg's server, decompresses them into the right folder)
  • install_docker (downloads and installs docker. if not running as root, auto adds your current user to the docker group)
  • remote_wallet (runs cli_wallet against steemd.steemit.com)

UPGRADING FROM HF18

If you have a previous installation of Steem-in-a-box, simply do the following:

Update your steem-in-a-box scripts:

git pull

Disable your witness, or change it to your backup key

./run.sh wallet

locked>> unlock "supersecurepassword"
unlocked>> update_witness "someguy123" "yoururl" "STM1111111111111111111111111111111114T1Anm" {"account_creation_fee":"0.200 STEEM","maximum_block_size":131072,"sbd_interest_rate":0} true

Now stop the witness server

./run.sh stop

Now, make sure you delete the /dev/shm files, as they will not be compatible with HF19 and need to be re-built

rm -rf /dev/shm/*

Finally, update your STEEM image, then start it with replay

./run.sh install
./run.sh replay

You can now check the logs with ./run.sh logs and see that it's rescanning the blockchain.

Once it's fully replayed, you can re-broadcast your witness :)

 update_witness "someguy123" "yoururl" "STM1abcdabcdabcdabcdabcdbdcabacbdcab" {"account_creation_fee":"0.200 STEEM","maximum_block_size":131072,"sbd_interest_rate":0} true

Installing a NEW witness

To become a witness, you'll require a server with at LEAST 8GB of RAM, 100GB disk, and a fast, reliable network. I recommend @privex - they accept STEEM, are affordable, and sell servers with a Zero Block Miss SLA

I personally use @privex for my own witness, as proof of their quality - I haven't missed a block since moving to them months ago, and I get almost 60 per day. (DISCLAIMER: I'm the CEO of Privex Inc. however you'll find plenty of people here on STEEM unaffiliated with us, praising our servers and network)

You're free to use any other server provider. I don't recommend trying to run a witness on your home internet, as it is likely not reliable enough for a witness.

Be aware that certain server providers have poor networks or hardware, and may cause missed blocks, so you may be hopping around a few until you can find one that's suitable for a witness node.

If you want to learn more about witnesses and what they do, check out my article: https://steemit.com/witness-category/@someguy123/seriously-what-is-a-witness-why-should-i-care-how-do-i-become-one-answer

First, we install some basic dependencies (assuming you're on Ubuntu).

sudo apt update
sudo apt install git curl wget

Next, we download Steem-in-a-box, created by myself.

git clone https://github.com/Someguy123/steem-docker.git
cd steem-docker

Now that it's downloaded, and we're in the right folder, we can install Docker.

./run.sh install_docker

NOTE: If you're running this without root, you'll need to log out and back in again after it's done. Remember to cd steem-docker after logging back in.

Now, we download the pre-compiled STEEM image from DockerHub:

./run.sh install

A new feature in the HF19 Steem-in-a-box, automatically downloads blocks from @gtg's server, extracts them, and puts them in the right folder. This will take a while, but will dramatically speed up your setup time.

./run.sh dlblocks

This next part is for witnesses, if you're wanting to run a seed, don't touch the config!

Open the wallet using a public server with the following command:

./run.sh remote_wallet

Create a key pair for your witness.

suggest_brain_key

You should get something similar to this:

  "wif_priv_key": "5xxxxxxxxxxxxxxxxxxx",
  "pub_key": "STMxxxxxxxxxxxxxxxxx"

Put the keys in notepad or something similar for the moment so you don't lose them.

Press CTRL-D to exit the wallet.

Now, you need to decide on how much "shared memory" to give the server. Right now, 8GB is the bare minimum, however 12GB is recommended. DO NOT GIVE MORE SHARED MEMORY THAN YOU HAVE ACTUAL RAM AVAILABLE. e.g. if you have an 8GB VPS, only use 8G for your shared memory.

sudo ./run.sh shm_size 8G

Open up the config file in your favourite text editor, nano is good for beginners:

nano data/witness_node_data_dir/config.ini

You should not run a seed if you are a witness, so change

p2p-endpoint = 0.0.0.0:2001

to

# p2p-endpoint = 0.0.0.0:2001

This disables the seed.

On a blank line, put the following:

witness = "YOUR NAME GOES HERE WITHOUT THE @ SIGN"
private-key = 5xxxxxxxxxxxxx

Put in your witness name, and your private key from earlier. Be aware the name needs quotes, while the private key doesn't. You use the private key (wif_priv_key) you generated earlier and put into notepad.

You will also need to adjust shared-file-size to whatever size you decided earlier for the "shared memory". There should already be a line with 12G, adjust that, if you need to do not make a new line for this.

shared-file-size = 8G

Save and close the file (if using nano, press CTRL-X, say yes to saving)

Next, we adjust the steem-in-a-box settings so that it's named correctly, and to disable port forwarding for seeds:

nano .env

The file will be blank. In the file put the following:

PORTS=
DOCKER_NAME=witness

Again, save and close the file with CTRL-X.

Now, we can start up the witness server

./run.sh replay

Check the logs with this command:

./run.sh logs

You should see something similar to this:

344773ms th_a       application.cpp:297           startup              ] Replaying blockchain on user request.
344774ms th_a       database.cpp:151              reindex              ] Reindexing Blockchain
344823ms th_a       block_log.cpp:130             open                 ] Log is nonempty
344823ms th_a       block_log.cpp:139             open                 ] Index is nonempty
344823ms th_a       database.cpp:159              reindex              ] Replaying blocks...
344834ms th_a       database.cpp:2571             show_free_memory     ] Free memory is now 11G
   0.77369%   100000 of 12925066   (12282M free)

If you see lots of red error messages, something went wrong. You can ask for help debugging it in the witness channel on
STEEMIT.CHAT
.

If it appears to be working, leave it for an hour or so. Check the logs every 10 minutes until you see something like this:

1299055ms th_a       application.cpp:507           handle_block         ] Got 14 transactions on block 12928269 by pharesim -- latency: 55 ms
1302427ms th_a       application.cpp:507           handle_block         ] Got 18 transactions on block 12928270 by xeldal -- latency: 426 ms
1305291ms th_a       application.cpp:507           handle_block         ] Got 26 transactions on block 12928271 by arhag -- latency: 291 ms
1308045ms th_a       application.cpp:507           handle_block         ] Got 20 transactions on block 12928272 by pfunk -- latency: 45 ms
1311092ms th_a       application.cpp:507           handle_block         ] Got 23 transactions on block 12928273 by bhuz -- latency: 92 ms

This means your witness is now fully synced.

Go to your profile on Steemit, click Wallet, then Permissions. Next to "Active Key" you should see "Login to show". Click that button and log in. Then press "Show private key", to reveal it. Make sure it begins with a "5".

Copy the key.

Now open the wallet with:

./run.sh wallet


(Image courtesy of @klye, copied from his old guide)

Set a password, unlock the wallet with the same password, then import your ACTIVE private key from steemit (NOT the one in notepad):

set_password "mysupersecurepass"
unlock "mysupersecurepass"
import_key 5zzzzzzzzzzzz

Finally, we can update/create the witness on the network. Replace the STMxxxx key with the public key you saved earlier in notepad, and replace YOURNAME with your witness name on Steemit.

update_witness "YOURNAME" "https://steemit.com/witness-category/@YOURNAME/my-witness-thread" "STMxxxxxxx" {"account_creation_fee":"0.200 STEEM","maximum_block_size":131072,"sbd_interest_rate":0} true

Assuming there are no big red messages, you're now a witness! :)

Go ahead, vote for yourself (and @someguy123).

How do I know what rank I am?

Check SteemDB: https://steemdb.com/witnesses or steemd: https://steemd.com/witnesses

What about a price feed?

As a witness, you're expected to run a price feed.

I recommend Steemfeed-JS, as it goes nicely with the docker set up.


Do you like what I'm doing for STEEM/Steemit?

Vote for me to be a witness - every vote counts.

Don't forget to follow me for more like this.


Have you ever thought about being a witness yourself? Contact me on STEEMIT.CHAT, or just ask in the witness channel. I'm happy to guide you! Join in shaping the STEEM economy.

Are you looking for a new server provider? My company @privex offers highly-reliable and affordable dedicated and virtual servers for STEEM, LTC, and BTC! Contact me on STEEMIT.CHAT for more information.

Are you looking for a Graphics designer for your project? Logos, PSD mockups, banners? Check out @MaliceRed - he posts about his life as a Freelance Graphics Designer, and is looking for work.

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  trending

Nice sharing! But it just too complicated for me for those technical terms and command XD. And I think maybe someday I will be interested in doing so..but The first point is I need to be good enough XD. Anyway, thanks so much for your sharing! It give me some idea about it.

Very cool and would definitely try this when i get the gist of it all.
It looks confusing but i will get the understanding or maybe someone can help me do this faster?

This is likely the next step for me as I spend every day on steem or working on something steem related.... thanks for the guide man! Bookmarked!

You are doing great work on Steem brother witness @someguy123, teaching others is very noble, keep it up, Steem needs knowledge.

Thanks for a very informative guide. I am just in the process of setting up a witness node, so this came at a perfect time... Still trying to figure out where to host it, so I will take a good look at @privex

Thank you for your help and support
We surely need you
Please follow @someguy123 he is great
I am your new follower
Best Regards Andrei

i really want bookmarking to come back to this. think i might take you up on a virtual server at some point!

upvoted because you mention this. Making it easier to know where to go to spend steem as well.

I recommend @privex - they accept STEEM, are affordable,

Priceless your Job here, thanks for doing that! Upvote! /wave

i was waiting for this :D awesome stuff man

upgrading tonight!

Thank you for this updated version and thank you for everything @someguy123! Seriously! I wouldn't be where I am if it wasn't from you.

Dear @someguy123 - I would like to be a witness. I have some problems and so I would like to ask you if you think I could.

  1. Do not have electricity all day
  2. Have intermittent phone and internet
  3. Am a Windows developer not linux
    a) Although I know how to use Ming and such it is not my specialty nor is it my focus
  4. Can people witness, get blocks without being a voted witness? I thought you had to be a witness in order to get the blocks
  5. Today, I am in the process of transferring steem and sbd to poloniex and instead of it being quick, as fast or faster than Bitcoin, it is past two hours now and the transfer is not complete. That tells me there is not enough witnessess to handle the network

http://privex.io/ not work. What is this? Have you change your provider?

That's one heck of a disclaimer! ;)

Thanks for this. I have been wanting to set up a witness the last week but didnt really know where to start. I am going to try this locally first on my dev box and then if all goes well.. set up on a dedicated server. Looking forward to be able to give back to the steem community.

I had a problem when running the seed. I couldn't telnet to it from an external box. I tried to play with the firewall but that wasn't it, the culprit was in the run.sh script:

PORTS="2001"
if [[ -f .env ]]; then
source .env
fi

What's happening is, after PORT is set to 2001, it gets overridden by the value from the .env file. In this case, it's empty (as instructed in this post).

PORTS=
DOCKER_NAME=witness

So when running the docker container, the resulting command that starts the docker is:
docker run -v /dev/shm:/shm -v /home/user/steem-docker/data:/steem -d --name witness -t steem

The fix is to set PORTS=2001 in the .env file. And we end up with the proper command being executed, with an open port allowing public access to the seed node:
docker run -p0.0.0.0:2001:2001 -v /dev/shm:/shm -v /home/user/steem-docker/data:/steem -d --name witness -t steem

·

That's the point. It's designed to be a witness.

All you have to do is just remove PORTS= from the .env file and it would work like normal, with the default port of 2001 being opened.

·
·

Thanks so much for this post. I had good experiences using this service. I love reading the tips on how to maximize long term gains@yash0108

·
·

I am using the "Your guide to setting up a Witness Server (STEEM-in-a-box HF19)" guide to set up my witness account and have run into the following issue: "DOCKER_NAME=witness File Name to Write: .envPORTS= " I am not sure what to put here based on the guide. Any help would be appreciated.

Congratulations @someguy123!
Your post was mentioned in my hit parade in the following category:

  • Pending payout - Ranked 3 with $ 1240,38

It is one of the popular pairs. If you want huge %, try a unpopular pair and be patient.

Although at this point in time I have no desire to be a witness, I appreciate the info contained in this post as it explains (at least to My ability to understand it) quite a few things about what's going on in the background to keep Steemit running.

This has brought quite a few questions that I'd like to pursue, so I appreciate the info on checking Steemit-chat witness channel.

Hopefully I won't be asking to many stupid questions.


SDG

Thanks for sharing this one.
very helpful for us.
i have followed you mr.
i like your creation so much.
vry motivate me.

Is there a way to run it on something else than ubuntu?

·

Yep. But you'll need to install docker by hand. install_docker runs various apt commands.

Once you've installed docker, continue with the standard ./run.sh install, editing config etc. and it should work as normal.

Thanks for the detailed post

THANKS for this post, I was discovering recently the witness aspect of Steem, and I was looking for information on how to set up a server. I will read this with great care.

How much monthly bandwidth does a witness require?

Ah you're the CEO of @privex? Nice.

·

For a witness, I'm not entirely sure. For a seed the bandwidth requirements are higher because you're distributing blocks.

Since I set up my witness a few months ago, here's what my network stats look like:

RX bytes:114048278566 (114.0 GB) TX bytes:99253351124 (99.2 GB)

So it could require 20-30GB a month, I really don't know.

Thanks for the detailed instructions! Will definitely be using this

Forkelicious.

it looks really cool, thanks i'mma try it

Nice one, thanks for sharing @someguy123

that's great help from you
thank you very much

Buen post

Thank you for your good post. I understand little English and only understand what Google translates for me, but I liked this post.

I wish the team good luck. I hope that these changes will only improve the platform and there will be no reverse effect.

Great information @someguy123! Will greatly help somebody who wants to become a witness.

Thanks for the helpful article. I will try and set it up as being instructed. some how complicating but i know you will assist if i go wrong.

Is this the software like a mining software or block chain?

·

It's similar to mining, except instead of it being who has the most hash power, it's who has the most votes.

·
·

Ok Thanx , now I got it...

Ho boy, Thx far all your works !
I have created my witness (for curiosity purpose) on another account.
In fact, 8Gb is not enough to replay the blockchain. 16 Gb is required.

Now, I would like to host my own instance of steemit.

Cool! I would give it a try if I had a computer connected full time to the internet.

Interesting info indeed!, always been curious about witness, probably not getting into that any time soon but in the future, maybe...

Steemit will, give back the rewards to the people and not to the big Companies !

Very good and helpful post, following you for more and thanks for sharing

Very useful information. Keep posting the latest information ... I'm very happy if you want to resteem some of my articles ....

Hosting your own node or server on any network can be heaps of fun and pretty rewarding.

steemit hard fork in 1 day what will happen

Nice post.....I appreciate your efforts on this collection!!
How can I make followers on steemit community

Hey, a quick question. Who would you say should consider becoming a witness? Anyone? Those with a lot of investment in the platform? Etc.

Thanks for sharing!

·

There are a variety of different backgrounds as witnesses.

Some are highly technical, e.g. @smooth, @abit, @gtg, myself etc. - those with history in development and system administration are more likely to be trusted by the other witness whales. A vote from @abit for example, could throw you up 10-20 ranks if you're low down, or 5-10 ranks if you're in the top 50.

Some people like @charlieshrem never even operated his own witness, he paid someone else to manage it, and he was #1 witness for a while. This shows even without a developer/sysadmin background, you can still make it to the top 20 if you know the right people.

A large part of your success as a witness depends on how much the community likes you. If you're well known and liked by the developers, the whales, and the normal users, then you're bound to have great success as a witness.

Sometimes, even if you're practically a famous Steemian, you'll still be stuck at a low rank. Example: @teamsteem is one of the most well known and loved Steemians, with posts constantly breaching $1000, and yet he's struggling to make it into the top 50 after running for months.

The only way to find out is to try it. Setup costs for a witness are low, and you can always stop if it's not working out for you.

Thanks for the information :)

Thanks for a very informative guide. I am just in the process of setting up a witness node, so this came at a perfect time... Still trying to figure out where to host it, so I will take a good look at @privex

great post, but im new here

very informative guide

We (Steemit) strongly recommend using the official docker images found at steemit/steem on Docker Hub. They are compiled by us and we use them in production to run steemit.com.

Beyond that - great work @someguy123!

I'm really tempted to try and setup my own witness, just to see if I can actually get it done.

nice ..
i followed your steemit account because of your supportiveness in steemit community please help me raise my steem power thanks ..

Excellent work, Thanks for sharing someguy123. To speed up the sync process, I think we can skip the replay by saving and sharing the shared memory file. It will be 14G though and growing.

Cannot connect to the docker demon at unix:///var/run/docker.sock. is the docker daemon running?

I'm hating this OS

·

I bet you're using something different than Ubuntu? If so (like on Fedora/Centos), you need to start the docker service first.

·
·

Nope, Ubuntu 16

·
·
·

Check its status: systemctl status docker | grep running
If it's not running, start it: systemctl start docker

I hate Ubuntu too, I'm more used to Fedora.

How much bandwidth does running a witness consume?

thank you very much for detailed guide. i will try

Realy nice tutorial, but one thing is not clear to me: Why should I run a witness? Just to support the community or will I receive some kind of income for doing it?

thanks the helpful guide!

a -f for the logs would be awesome ^^

·

You can follow in realtime with:
docker logs --tail=30 --follow witness
or
docker logs --tail=30 --f witness
(assuming your container's name is "witness", the default in @someguy123's installation)

·
·

Oh I figured out. I started to deal directly with the container itself after some time. Thanks anyway.

How do I use docker to interact with the blockchain?

Work for me, and your server just fly! thanks for your help btw!

Finished! Outstanding guide!

The first few steps wok fine for me but then I get some errors. I am root on a fresh ubuntu 16. can someone help?

After ./run.sh install I am getting :

"Error response from daemon: No such image: someguy123/steem:latest
Installation completed. You may now configure or run the server"

After ./run.sh remote_wallet I am getting Unable to find image 'steem:latest' locally
docker: Error response from daemon: repository steem not found: does not exist or no pull access.

Thank you for the good Guide, but i can no update_witness in the wallet. i receive always this message. somebody can help me?

10 assert_exception: Assert Exception
!ec: websocket send failed: invalid state
{"msg":"invalid state"}
th_a websocket.cpp:164 send_message

·

This happens because of the cli_wallet timeout. Exit the wallet (Ctrl-D) then restart it, but this time execute your commands quick, don't wait longer than a minute or you will timeout again.

·
·

thank you, that can be the Issue.

I have needed really to much time to insert the update_witness command. If i insert the update-witness command over my terminal in the ssh connection, the inserted command always was changed or some code was missing, it need time to correct the command.

now have usesd the alternativ Guide to install a witness https://steemit.com/witness-category/@drakos/witness-tools-conductor-docker

I don't know much about this, but what are witnesses?

·

@arckrai witnesses are people who keep the steem blockchain going, its important to vote for them.

·

At the start of the post I wrote:

If you want to learn more about witnesses and what they do, check out my article here: https://steemit.com/witness-category/@someguy123/seriously-what-is-a-witness-why-should-i-care-how-do-i-become-one-answer

Thank you for your information

Hi! How much does this cost you per month to run and how much do you make back in SP? I am interested in doing this but trying to figure whether its worth investing my time to learn how to do it! I want to make the most out of steem and this is another option! Thanks for any replys!

i am just new here in steemit , i liked you artilce and will consider it in my next steps , i voted and followed you
thanks for sharing ! 👏🏻👏🏻👏🏻

So it can be done only on linux systems, and not on Windows? LOL, sometimes when I think to become a witness, these technical details show me the right path of not to become a witness!!!!

Great post! upvoted!

Thx I am gonne try this soon...

God job men you are really got a posting and design your logo and post i really like steemit and i'm new here