Zero to Witness: Part 1: We need a big computer!

Welcome to my brand new series!

Since starting to write my Shader Programming series, I've really come to love the concept of Steem.

It makes sense to start thinking about the next step; in this series, I'm going to take you along with me on my own journey to become a Steem Witness.

There are a lot of guides around, but most are either a little outdated, or at least assume some level of familiarity with Linux and the command line.

Across this whole series, I'll try not to assume any particular knowledge, and will explain technical terms and other lingo as it's introduced. Hopefully, I can make it relatively painless for you to join me as we explore this path together. Let's go!

The basic idea: A big computer!

Steem witnesses secure the blockchain by running Witness Nodes, and can be rewarded by receiving block rewards when new blocks are created that they have "witnessed".

In practical terms, that means we're going to need a big computer that we can leave running 24/7.

Although the process of witnessing blocks is how new Steem coins are minted, and is analogous to "mining" on other blockchains, a Witness Node doesn't need to perform any proof-of-work task and so the machine we build will be fairly power efficient. It won't need a particularly huge PSU, and it won't need an expensive graphics card. In fact, it won't need a graphics card at all.

PC-under-a-desk vs "proper" hosting

Certainly, one way we could approach this would be to build a custom PC for the task, which would need to be left running all the time and connected to the internet via our usual domestic router.

However, that means a large initial financial outlay, and our node would be reliant on domestic power and internet connectivity -- not ideal when you're aiming for "five nines" uptime (99.999%), which we definitely want!

A much better idea is to rent a computer at a datacentre. If you're not used to doing this and the idea sounds daunting, it can be a lot cheaper than you might initially think, so read on.

By using a hosting company, they are responsible for the hardware, the power, and the network connectivity.

Sounds good! But having decided to use a hosted service, we still have another choice to make. Do we want a virtual machine, or a dedicated server? By the way, "server" is simply a word for any computer that will be providing a service, but the words "computer", "machine" and "server" are largely interchangeable.

Virtual machines

Under the virtual machine (VM) scenario, we don't actually rent a physical computer at all; instead, a particularly powerful machine hosts many emulated computers, and it's one of those emulated machines that we rent.

Virtual machines are sometimes also referred to as "cloud servers" or "virtual private servers", but these are all exactly the same thing.

Because of economies of scale, VMs are quite cheap.

From a software perspective, a VM is completely equivalent to a normal computer. We can install the operating system of choice that we want on it, we get full administration-level ("root") access, and can run whatever we want.

A VM hosting package will typically include a guaranteed amount of RAM on the host computer that's only available to our own VM. The package may also promise a given number of guaranteed CPU cores -- but the sad fact is that your VM still ends up competing for resources with all the other virtual machines running on the big physical machine.

A particular resource contention is often when doing disk access (ie, loading or storing data). In our case, handling a large blockchain, that can be a bit of a problem.

Dedicated server

The other option when using a hosting company is to rent a dedicated machine. That's exactly what it sounds like; somewhere in a given rack in a given datacenter, there's an actual, physical, 19" rack-mounted computer that is exclusively for your use. It might look a bit like the one in the pic at the top of the article, which is part of my home lab network, but you'll never get to actually see your dedicated machine.

Unsurprisingly, compared to a VM this option costs a little more. However, all those juicy resources are ours and ours alone. We get to use all the CPU cores, all the RAM, all the diskspace and bandwidth, all at once and all the time. And we don't need to buy the machine, we're only renting it!

If we wanted, we could even create multiple virtual machines running on our big beast of a box.

I think it's clear which option I'm steering us towards here. But let's make a direct comparison.

Where's the sweet spot?

For probably 90% of applications, if you were in the market for your first server, I'd tell you to grab a cheap and cheerful generic VM for 10 EuroDollarPounds (also known as international beer tokens) a month.

That would be ideal if you were looking to set up a few websites, or run a private Minecraft server.

However, our use-case is a little bit special. Steem has been running for several years, and generates a new block every 3 seconds. That adds up to a LOT of data; in fact, we can assume we're going to need close to 700 gigabytes of disk space for our Witness Node.

So that Steem can hold as much data in memory as possible to keep everything nice and snappy (remember, 3 second block times!), we're also going to need a decent amount of RAM - let's say 32Gb at minimum.

Let's take these specs to a popular European hosting company: Hetzner. Their reviews are a little spotty, but their prices are great. Personally I have found them to be reliable, and on the infrequent times I have needed support they have been efficient and courteous. Note that I don't work for them and there are no affiliate links in my articles.

To look at their VM offering first (they call it "cloud") take a look here. Oops -- you'll see that you can scroll all the way to the bottom before hitting our RAM requirement, and even then we would be severely lacking in storage and would need to add an extra disk; an additional expense.

This is very typical no matter which provider you look at. Here's OVH's VM prices as another common example.

So in our case, this is actually an easy choice. For the amount of grunt we need, we're going straight to dedicated. But here's the shock: Check out Hetzner's dedicated server prices for their AX line and you'll see that they're really not as expensive as you might expect. In fact, we can meet our spec while not spending much more at all than we would on a VM with upgraded storage.

The AX51-NVMe looks ideal for us:

CPU: AMD Ryzen 7 3700X (beast mode activated!)
RAM: 64Gb DDR4 ECC (double our spec; DDR4 means it's fast; ECC means it's proper, error-correcting, datacenter-quality)
DRIVES: 2 x 1Tb NVMe SSD

A note about those drives: They use the latest NVMe interface, so they're very fast. However, don't be thinking we're going to end up with 2Tb of usable space like I expected the first time I ordered an AX51. These drives are locked into RAID 1 configuration to guard against data loss; each drive mirrors the other, so we see 1Tb of usable space.

Let's buy it!

The AX51-NVMe is ideal for us, and at 64 international beer tokens a month, is affordable. I've gone ahead and ordered one! I didn't take any additional upgrades on the package at this point, as I think we can get our Witness Node running in 1Tb of disk. I picked Helsinki as the location, which was a little cheaper.

For the pre-installed operating system, I chose "Ubuntu 20.04.3 LTS Base". With Ubuntu, the release number gives the date the version was released; so 20.04 is from April 2020. "LTS" means it's a long-term support version, but in fact there is a fresh LTS version out now, 22.04.

I'm just sticking with 20.04 for now in case of any software incompatibilities with the new version; we may end up doing an in-place upgrade to 22.04 later.

NOTE: Ubuntu is a distribution of Linux. You need a few things besides the Linux kernel to run a full Linux system, and various different full Linux operating systems are based around the kernel. Ubuntu is simply my current favourite, and you're welcome to use another if you follow along; however, if you're not already pretty experienced with Linux I do recommend sticking as close as possible to my specific choices.

What's next?

Feel like joining me on the road to witnessdom? Order yourself an AX51-NVMe, or an equivalent machine elsewhere, and join me next time as we start to set it up :)

Alternatively, if you have enough spare parts to assemble most of the machine yourself physically, and maybe just need to order a nice big disk and some extra RAM -- and if you have somewhere to put it where fans won't annoy you -- consider building a PC just for this purpose, and installing Ubuntu on it.

Next episode, we'll learn how to connect to our server using a terminal program.