Blockchain 101 - Why double producing is bad
Background
A blockchain is a type of distributed database, where all of the nodes in the network use a set of rules to agree on what data should be in the database. These rules are called "consensus rules". These rules are important, because they prevent the nodes in the blockchain from having inconsistent sets of data.
A common example of why inconsistent data would be bad is called "double spending". If user A sent 1 token to user B, and then user A tried to send the same token to user C - it would be really bad if the nodes did not all agree on who had the token. Both user B and C could be tricked into thinking they had the token.
Blockchains are state-based databases. A "block" is a set of transactions that update the state of the database. The data in the database only changes when a new block is produced. During the time between blocks, the data in the database is static (it doesn't change). Whenever a new block is produced, it gets distributed to all the nodes in the network - which then validate whether or not the block is 'valid'. If the nodes consider the block valid, then they update their state.
Every new block that is produced is connected to all the previous blocks. This is why blockchains are called blockchains (chain of blocks). Whenever the nodes in the network validate a new block, they also validate that the previous blocks in the chain are valid too.
Forks
When two (or more) nodes in a blockchain do not agree on which data is correct, this is called a fork. In order to preserve the integrity of the network's data, the nodes must reach an agreement on which set of data is correct.
Typically the way this is handled is by a method called "longest chain". Each version of the inconsistent data will have a set of one or more blocks that makes it different than the other version(s). The next time a new block gets produced, the block producer will have to choose which of the chains to append the next block to. Eventually one chain will become longer than the rest. The longest chain becomes the 'valid' chain, and the rest are considered invalid.
Miners vs. Witnesses
Different blockchains have different protocols to determine who produces the next block.
In Proof of Work (PoW) blockchains (such as Bitcoin and Ethereum), the block producers are called "miners". Miners compete to solve complicated math problems that can only be solved with heavy computing power. Whoever solves the math problem first, gets to produce the next block.
Steem uses a protocol called Delegated Proof of Stake (DPoS), which is much more efficient than PoW. In DPoS, the token-holders (users who hold Steem Power) get to elect the block producers. They do this by voting. The block producers are called witnesses.
Steem Block Production Assignments
The Steem blockchain automatically assigns blocks to the witnesses. Whichever witness is assigned the block is responsible for producing it within the given time window. Blocks are spaced 3 seconds apart. Every 63 seconds, the Steem blockchain will produce a "round" of 21 blocks.
20 of the 21 blocks are produced by the 20 witnesses who have the most votes (measured in Steem Power). These witnesses are called the "top 20 witnesses".
The remaining block is randomly assigned to one of the remaining witnesses (called "backup witnesses"), with a probability equal to their percentage of votes (again measured in Steem Power) relative to the other backup witnesses.
Double Producing
The term "double producing" refers to a block producer creating two different blocks, and then distributing them both to the nodes in the network.
Double producing is really bad, because it creates a fork. Until all of the nodes in the network can agree on which of the two blocks is to become the 'longest chain', there will be two different versions of data floating around.
Double producing may also cause other witnesses to miss blocks, if the block they produce ends up on what becomes the shorter chain.
If an application or end-user makes a decision based on data that ends up being in the shorter chain, that data may become invalid after the longest chain is formed. This can be really bad if important decisions get made based on this invalid data.
Malicious users could potentially exploit this by attempting to double spend.
Punishment for Double Producing
In the 'early days' of Steem, the consequences for doing this were extremely severe. Anyone who caught another witness double producing could literally take all of the Steem Power from the double-producer's account.
The method that allowed this ended up getting turned off because it turned out that there was a potential exploit, but the fact that it was there should indicate how serious this is.
Also, witnesses should be extra vigilant when doing maintenance, migrating/adding servers, generating new key pairs. Although they may not be malicious, the common mistake is to use the same key pair on two servers, which results in double producing.
So, always disable your witness when doing maintenance by broadcasting this key
STM1111111111111111111111111111111114T1Anm, and triple check that the keys are different on every node. Re-enable the witness with the main server node's key when all is good and keep an eye on the logs until the next produced block is OK.Here's an example of a double production (collision):
1560475ms th_a database.cpp:538 _maybe_warn_multiple ] Encountered block num collision at block 22296845 due to a fork, witnesses are: [["steem-bounty","2018-05-10T03:26:00"],["steem-bounty","2018-05-10T03:26:00"]]Note that if the collision happens between two witnesses, it's often due to a temporary lag, for example:
28220ms th_a database.cpp:538 _maybe_warn_multiple ] Encountered block num collision at block 22214744 due to a fork, witnesses are: [["netuoso","2018-05-07T07:00:24"],["xeldal","2018-05-07T07:00:27"]]Notice the timestamps with exactly 3 seconds difference.
This is an excellent explanation of "Double Block Producing" and much easier to understand now..
I learned something new from this also, I never knew in the early days that SP could be taken from a witness for double producing.. Very interesting..
I can see how that illustrates just how important it is to not double produce blocks.
Thanks for this @timcliff
First of all, I have to say that I am extremely happy that I decided to follow you. The first time that I met you was during your post on raising the dust vote amount. So, I'll just come out and tell you the truth, I really only followed you so that I can keep up with any more crazy ideas that you might have. Now, however over the course of the last week or so, I am really beginning to realize that you're doing the best you can try to think of ways to solve problems here. Not only that, but you're probably one of the smartest guys and I've met here and my short time. So, I would like to thank you once again for what you're doing. And like I said last time, you deserve to be paid for your work, so I am hoping that you will accept payment for this post. And I have been thinking about this since the last time. You could take, say half, or any amount you decide, and just hand it out to people in the community who you deem Worthy or that need it. Or however you would seem fit, it's your money. Just an idea to throw out there. Now, on to this post. This is probably the best, simple, explanation of the blockchain that I have ever heard. And I appreciate you for the simplified explanation that you offer. I also think that it is a great thing that you explain the potential issues with double producing. And as far as how easy it is to be able to become a witness due to the simplified processes, I never realized that this was the case. To me, I agree with you, and I think that this is a serious potential for abuse. And while they removed the old punishment, have they not installed any new form of punishment for double producing? And also, is there any way that you guys(witnesses) keep track of this, so that if it does begin to happen it can be stopped quickly? Or is it something that potentially could be hidden until it caused a serious Fork? I ask you these questions because lately as I have gotten deeper into the steemit "behind the scenes" there are a lot of things that concern me. When I first arrived here, I thought everything was happy-go-lucky. I now know that this is not the case. I have enough concern that I have stopped investing my own personal money, and have only been gaining money and SP here through authoring and curation. I appreciate any response that may be given in regards to my questions. And I would like to tell you to keep up all of your great work. I think that you were doing good stuff here and I believe that you deserve encouragement. So thank you so much again for what you're doing and I hope that you have a wonderful week. As always I send you and all of yours all my love until we meet again.
Haha, thanks :)
As far as the punishment for double producing, it is an easy way for a witness to loose witness votes. It is pretty easy for other witnesses (and anyone else who is running a node) to detect it, as it shows up in everyone's logs.
The majority of the time, it just ends up with a "micro fork" - which is basically a fork that lasts one block. As soon as it is the next witness's turn, they pick one of the blocks, and everyone follows. While it is still possible for someone to get tricked in this short period of time, it is a fairly common practice in cryptocurrency to at least wait a few block confirmations before considering something "final" - which should protect against this as long as users/apps are following this best practice. (Some still might not though, which is why it is still best to avoid even a micro-fork as much as possible.)
Thank you sir.
The whole concept of Witness work involves not only votes and SP weights, but some common ethics to help the platform rather then acting maliciously. The unfortunate event of 'trading / begging' etc for Votes makes a ground for ill intended people to achieve high positions in Witness list.
Lately I noticed many votes coming towards my Witness, that simple got removed after a couple of days. The way i see it, is because of not approving back. (and I am not approving back, because too many missed blocks / compared to position as a main reason).
I would only say it's our mistake for offering tons of simplified solutions suck as docker allowing everyone to become a witness, then execute different strategies (of questionable ethics) to claim high.
As simple as that, someone with enough knowledge to compile and run nodes from scratch is assumed to have enough knowledge to make money outside the scope of abusing the platform.
Witness work is not for everyone. Making it easy for everyone to join the race will only bring trouble.
I would agree in a sense, being a witness is not for everyone but it should not only be for the "tech savvy either" we just recently set up our witness as a project because we are 2 steemians and someone technologically inclined and we are a fan of steem... do we really know the ins and outs? no we do not but we are learning... We had our first block yesterday, and missed it but I do not know if it was due to this double producing. So now we have shut off the server and rebuilt the whole build from scratch instead of following a manual.
So I agree for you there... the votes should come from community work, and running a good service and not from tit for tat...
We are learning on the way. But that would also be why we start lower and then have a chance to build up votes
our witness is @swisswitness
Well, that's something totally different and totally ok, if someone is learning with a goal to achieve full understanding of the platform and provide great service.
Few months ago, I offered "paid witness setup", but these who contacted me willing to pay for such work, got it totally fee (along with support). I am glad some of them are successful, still doing their work. The 'payment' was just a trick to distinguish who is serious and who is not. They got a full training and explanation to any questions they had.
I have nothing against a total newbie who is willing to learn, and I will always support that by all means and sure others will too. But the real problem is that some people are pushing way more into campaigns / trading votes / begging for votes, rather then understanding technology. At the end of the day, they do rank better, and platform suffers.
I don't think anyone needs to have PhD in computer science to be a witness, the only requirements is to take it seriously, act ethically, learn new things, and support others.
Lucky enough most of witnesses we have are great people. But if you want to distinguish others, just check how many of them become inactive whenever SBD goes down on the market, and then get back once it raises.
Woah ! I definitely have to read this more than once. Cue airplane over my head :) So much to learn. What a layered beast all of this is.
hi @timcliff, as a new witness I have so much to learn and thankfully people like you educating. found the post very informative and easy to understand (for the non tech. like me) and some of the comments also added value
That was somehow a good recap of the blockchain consensus and double spending problem, and same time a good lesson of history of Steem.
I just tried to contribute to Steem github and saw your comment, thanks for the review! If you have time, could you give me some information about how (or if) should jenkins be ran on external branches?
Sorry to bother you, and thank you so much!
Replied to your question in a different thread, but that is not something I am familiar with.
Oh, thank you so much, sorry for asking twice, I didn't thought you will be checking that comment with all the activity here :D
No worries :)
Anyway, I have updated the PR after your comment, if there is anything you see wrong about it, I appreciate your input!
And since in your PR about guidelines you introduce the branch stable, I wonder if the new PR should be opened against develop or master.
It is a good question, and I’m not entirely sure. I always submit my PRs to master, but maybe I’ve been doing it wrong all this time :)
Okay, I will follow your example for the moment, if I will be able to open any other PRs and same time I will keep an eye on the guidelines. Thanks a lot for your answers, I was worried about both of these things pretty much, now I feel better!
In time I will ask you other questions about how do you test the developed features, but I don't want to take all of your day/night so for the moment, thank you again!
I should mention, I'm not one of the 'official devs'. You may want try asking your questions in here: https://discord.gg/B29Bbng. I know some of the Steemit developers hang out there and answer questions on a periodic basis.
Wow... I just can’t hold that wow. This is amazing , this is comprehensive , It felt like reading a white paper . Thanks for this much education and information.
I like the sound of this. I look forward to further courses
Thank you @timcliff, you have described a very important issue in a way that everyone understands, it is very valuable. I would like to ask for more ;-)
Thank you @timcliff for such a comprehensive explanation focussed on Double Producing.
Few questions,
There is a method for it in the code. It is deprecated though.
I believe PoW has the same problem, but it is less of a big deal because pretty much everyone waits for multiple confirmations with PoW tokens.
Each node just doesn’t make any changes to the data in its own database until it receives the next block.
Thank You @timcliff,
Sorry, an off the topic question,
what does it mean by single-threaded and multi-threaded in blockchain and how it affects the tps?
In simple terms - single threaded means it can only do one thing at a time. Multi threaded means it can do multiple things at once. Multi threaded obviously allows more transacting.
Thank you and please keep on writing more about blockchain, especially about graphene and DPOS .. I have set GINA to stalk you, looking forward to more info @timcliff :)
And I have gone through your blog, you seem to have very active participation in STEEM, I just wanna know if 'one account, one vote' will be a thing just for SMT or we will be seeing a fork for STEEM. Cause if it's just limited to SMT one of my posts becomes obsolete :)
Currently it is just a concept being discussed for SMTs.