Let's build Bots for STEEM - Part 01: The Server
Hi Steemians, let's build BOTS!
On this blog I want to document my progress in implementing bots for the STEEM blockchain.
What is a "Bot"?
A bot is a piece of code running on a computer and doing some stuff, for instance communicating with the STEEM blockchain, where already a lot of bots are running.
The first toolkit for building bots was the PISTON library from @xeroc. I've build some bot's with PISTON, but since steemit inc has disabled their own wss endpoint it seems to be urgent to migrate to steem-python, the official successor of piston.
New Server
Since I don't want to kill my running PISTON based bots, I rented a new virtual server with a naked Ubuntu 16 from @privex.
After one day I got a mail with the login credentials.
First Contact
Now it's time to open a terminal and to connect to the new server. Just enter
$ ssh [email protected]
into the terminal, answer with yes and enter your password (see mail).
The authenticity of host '178.XXX.XXX.XXX (178.XXX.XXX.XXX)' can't be established.
ECDSA key fingerprint is SHA256:IjtacL87I6xxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '178.XXX.XXX.XXX' (ECDSA) to the list of known hosts.
[email protected]'s password:
After login with the correct password we are asked to change the password.
You are required to change your password immediately (root enforced)
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-81-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
89 packages can be updated.
0 updates are security updates.
*** System restart required ***
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
WARNING: Your password has expired.
You must change your password now and login again!
Changing password for ubuntu.
Enter the old password from the mail and then enter your new password twice.
(current) UNIX password:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Connection to 178.XXX.XXX.XXX closed.
After entering the password twice, it is changed and the SSH session to the server is terminated.
Second Login
Now we can login to our server with our new password. Type again
$ ssh [email protected]
and enter your password.
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-81-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
89 packages can be updated.
0 updates are security updates.
*** System restart required ***
Last login: Wed Jan 10 06:00:00 2018 from XXX.XXX.XXX.XXX
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@xxxx:~$
We are now ready to start.
The Hardening
It's important to harden the server to make it as difficult as possible for attackers to gain access to the server. @privex recommends the following steps
fail2ban
The installation of fail2ban requires root access. We can use the sudo command to get temporarily root rights.
ubuntu@xxxxxxxx:~$ sudo apt install fail2ban
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python python-minimal python2.7 python2.7-minimal python3-pyinotify whois
Suggested packages:
mailx monit python-doc python-tk python2.7-doc binutils binfmt-support python-pyinotify-doc
The following NEW packages will be installed:
fail2ban libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python python-minimal python2.7 python2.7-minimal python3-pyinotify whois
0 upgraded, 10 newly installed, 0 to remove and 89 not upgraded.
Need to get 4,194 kB of archives.
After this operation, 18.1 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Answer Y for Yes and the installation will start.
Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-minimal amd64 2.7.12-1ubuntu0~16.04.2 [338 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7-minimal amd64 2.7.12-1ubuntu0~16.04.2 [1,294 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-minimal amd64 2.7.11-1 [28.2 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-stdlib amd64 2.7.12-1ubuntu0~16.04.2 [1,880 kB]
Get:5 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7 amd64 2.7.12-1ubuntu0~16.04.2 [224 kB]
Get:6 http://archive.ubuntu.com/ubuntu xenial/main amd64 libpython-stdlib amd64 2.7.11-1 [7,656 B]
Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 python amd64 2.7.11-1 [137 kB]
Get:8 http://archive.ubuntu.com/ubuntu xenial/universe amd64 fail2ban all 0.9.3-1 [227 kB]
Get:9 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-pyinotify all 0.9.6-0fakesync1 [24.7 kB]
Get:10 http://archive.ubuntu.com/ubuntu xenial/main amd64 whois amd64 5.2.11 [34.0 kB]
Fetched 4,194 kB in 0s (11.3 MB/s)
Selecting previously unselected package libpython2.7-minimal:amd64.
(Reading database ... 81808 files and directories currently installed.)
Preparing to unpack .../libpython2.7-minimal_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking libpython2.7-minimal:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package python2.7-minimal.
Preparing to unpack .../python2.7-minimal_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking python2.7-minimal (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package python-minimal.
Preparing to unpack .../python-minimal_2.7.11-1_amd64.deb ...
Unpacking python-minimal (2.7.11-1) ...
Selecting previously unselected package libpython2.7-stdlib:amd64.
Preparing to unpack .../libpython2.7-stdlib_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package python2.7.
Preparing to unpack .../python2.7_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking python2.7 (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package libpython-stdlib:amd64.
Preparing to unpack .../libpython-stdlib_2.7.11-1_amd64.deb ...
Unpacking libpython-stdlib:amd64 (2.7.11-1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for mime-support (3.59ubuntu1) ...
Setting up libpython2.7-minimal:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Setting up python2.7-minimal (2.7.12-1ubuntu0~16.04.2) ...
Linking and byte-compiling packages for runtime python2.7...
Setting up python-minimal (2.7.11-1) ...
Selecting previously unselected package python.
(Reading database ... 82554 files and directories currently installed.)
Preparing to unpack .../python_2.7.11-1_amd64.deb ...
Unpacking python (2.7.11-1) ...
Selecting previously unselected package fail2ban.
Preparing to unpack .../fail2ban_0.9.3-1_all.deb ...
Unpacking fail2ban (0.9.3-1) ...
Selecting previously unselected package python3-pyinotify.
Preparing to unpack .../python3-pyinotify_0.9.6-0fakesync1_all.deb ...
Unpacking python3-pyinotify (0.9.6-0fakesync1) ...
Selecting previously unselected package whois.
Preparing to unpack .../whois_5.2.11_amd64.deb ...
Unpacking whois (5.2.11) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu17) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Setting up python2.7 (2.7.12-1ubuntu0~16.04.2) ...
Setting up libpython-stdlib:amd64 (2.7.11-1) ...
Setting up python (2.7.11-1) ...
Setting up fail2ban (0.9.3-1) ...
Setting up python3-pyinotify (0.9.6-0fakesync1) ...
Setting up whois (5.2.11) ...
Processing triggers for systemd (229-4ubuntu17) ...
Processing triggers for ureadahead (0.100.0-19) ...
ubuntu@xxxxxxxx:~$
Alternative SSH Port
In order to change the SSH port we need to change a config file and restart the SSH daemon. Enter
ubuntu@xxxxxxxx:~$ sudo vi /etc/ssh/sshd_config
to open the vi editor. The fifth row of the config file contains the currently configured SSH port: 22, the default value. In case you have no experience with vi, just click the i key to switch to insert mode and use the arrow keys to navigate to the port number.
# Package generated configuration file
# See the sshd_config(5) manpage for details
# What ports, IPs and protocols we listen for
Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0
Protocol 2
Delete the 22 and insert your new port number, for instance 12345.
# What ports, IPs and protocols we listen for
Port 12345
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
...
-- INSERT --
Finally we need to save the changes. First hit the ESC key to leave the insert mode and than type
:wq
to save the changes and close the editor.
Now we need to restart the SSH process (daemon). Enter the following command:
ubuntu@xxxxxxxx:~$ sudo /etc/init.d/ssh restart
[ ok ] Restarting ssh (via systemctl): ssh.service.
Seems that the SSH port change was succesfully.
To test if the port configuration works, open a new terminal and enter
$ ssh [email protected] -p 12345
[email protected]'s password:
Note that we now always need to provided the port number with the -p option.
After entering your password you are logged in.
System Updates
We need again the sudo command to run an apt update
$ sudo apt update
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Get:5 http://security.ubuntu.com/ubuntu xenial-security/main Sources [104 kB]
Get:6 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [48.9 kB]
Get:7 http://archive.ubuntu.com/ubuntu xenial-updates/main Sources [290 kB]
Get:8 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [419 kB]
Get:9 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [184 kB]
Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/universe Sources [186 kB]
Get:11 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [7,224 B]
Get:12 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [7,968 B]
Get:13 http://security.ubuntu.com/ubuntu xenial-security/restricted Translation-en [2,152 B]
Get:14 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [698 kB]
Get:15 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [194 kB]
Get:16 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [100 kB]
Get:17 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3,208 B]
Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [291 kB]
Get:19 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [7,588 B]
Get:20 http://archive.ubuntu.com/ubuntu xenial-updates/restricted Translation-en [2,272 B]
Get:21 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [571 kB]
Get:22 http://archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [230 kB]
Get:23 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.2 kB]
Get:24 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8,052 B]
Fetched 3,678 kB in 0s (4,755 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
93 packages can be upgraded. Run 'apt list --upgradable' to see them.
We see that 93 packages can be upgraded. Now we run
sudo apt upgrade -y
to update the system.
Reboot the system
Finally we need to reboot our new server, because we see the System restart required message after login.
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-81-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
0 packages can be updated.
0 updates are security updates.
*** System restart required ***
Last login: Wed Jan 10 05:40:08 2018 from XXX.XXX.XXX.XXX
Just type
$ sudo reboot
to reboot your server.
Connection to 178.XXX.XXX.XXX closed by remote host.
Connection to 178.XXX.XXX.XXX closed.
The server shuts down and closes all open connections. After the reboot we need to login again.
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-109-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
0 packages can be updated.
0 updates are security updates.
Congratulations
Our server is now up and running.
In my next post I will install steem-python from the scratch. Stay tuned!
Thank you very much for this initial tutorial. I am really looking forward to the next parts. Especially, I am currently looking for a way to read out my wallet history, so that I can automatically find out who has sent me money and up vote their most recent post. Unfortunately, I have not been able to find the piston documentation for this part yet. If you would cover that, it would be highly appreciated.
Congratulations @botsultant, you have decided to take the next big step with your first post! The Steem Network Team wishes you a great time among this awesome community.
The proven road to boost your personal success in this amazing Steem Network
Do you already know that awesome content will get great profits by following these simple steps, that have been worked out by experts?