The Day The BlockChain Stood Still - Witness Update from @derangedvisions and @c0ff33a
By now you will have seen a 100 or more posts on what happened on the 17th September 2018 when the blockchain stood still so I will spare you the details, what I will say is it is thanks to the excellent work of our Top Witnesses and multiple developers working on code patches all that we lost was a little Steemit time, once the problem was solved everything restarted without transaction loss.
I held out on my paused v0.20.00 Witness server in case there was any chance to restart it without a reply - the end solution however required replacing the block log and replaying which takes my server 6 hours. And at that time everyone was asked to revert to v0.19.12 , v0.20.00 has been fixed now though and we are seeing some Witnesses on it - I think I will hold out until next Monday night to suffer another long replay.
So last night I finally went to bed just after 2am, setting an alarm for 4am to see if the replay had done. It hadn't of course so a 6am alarm and at that point the replay had finished and there were patch instructions so a quick stop, build and config edit and start got me running and back processing transactions - and a block a few minutes after they started!
Irony of the day was 45 minutes after getting that block with my Witness running smoothly on v0.19.12 conductor caught up with the 6 blocks I missed when the blockchain was not running and promptly disabled my perfectly working Witness. I noticed quickly and was able to manually reactivate - nothing works when the blockchain stops!
So the Blockchain restarted at 1am my time, and my server started processing transactions again at 7am after the replay - not bad going really and even now many Witnesses are still out of action. I have been producing blocks all day and everything is smooth again, the bug that caused it all has been fixed and we can all carrying on looking forward to HF20 on the 25th.
Some of my previous posts you might have missed
Witness Update by @c0ff33a v0.20.00 and giving something back to voters
|
The C0ff33 Vlog 25th August 2018 |
Sublime Sunday by @c0ff33a ☕️
|
Steemd v0.19.12
Kind thanks to @enginewitty for employing his Ninja graphic design skills and producing this 🔥 banner
If you enjoyed my post don’t forget to leave an Upvote, feel free to comment on what you liked or where I can do better even. Follow @c0ff33a for regular and varied posts, photography, my weekly VLOG, posts on coffee roasting and brewing - just follow and see.
To grow your presence and following on Steemit, be active in the community Discords that offer the chance to learn, network, share and be found. The #SteemitRamble discord https://discord.gg/HZYzEry hosts a range of shows where you can join in and get your name shown, every Thursday there are two Pimp My Post Thursdays which are hugely popular and give you an opportunity to get your posts seen.
For fun and entertainment then #SteemStar is the discord to be in, https://discord.gg/ZDANjzy with regular shows during the week it’s the best place to relax, and also network the friends you make could be your future biggest upvoters.
All photos are my own, taken with my iPhone. All my Steemit posted pictures can be freely used by other Steemit users in their own blog posts - all I ask is that you follow me and credit @c0ff33a linking to my post you took it from. You can also follow me on Twitter
Initiative Q is building a new payment network and giving away significant sums of their future currency to early adopters. It’s by invite only and I have a limited number of invites. My personal invite link: https://initiativeq.com/invite/Sz3OZG0SX
Sign up for the Xaya crypto early and get some free coin!
https://growapi.refereum.com/xaya/share/8169/community/212
The picture to the right kindly sketched by the wonderful @bridget.art , be sure to check her incredible art out https://steemit.com/@bridget.art
The greatest challenge we face on Steemit is getting our posts seen, I have always believed the simplest and best value way of doing this is by commenting on the blog posts of others, the more you engage people and interact with them the greater the chance they will visit your own blog and return the favour. Other then spending all my free time commenting on Steemit posts, I also distribute coffee roasting machines and also espresso coffee machines and roast my own Speciality Coffee Range. being one of the premium coffee suppliers Yorkshire including a wide range of filter coffee sachets
!steemitworldmap 53.766547 lat -1.801500 long Bradford, West Yorkshire, United Kingdom. d3scr
This awesome animated Banner has been kindly been created by the wonderful @snook
Don’t forget to vote for Witnesses https://steemit.com/~witnesses
And you can vote for the Witness team c0ff33a and @derangedvisions - click below to vote easily
Pretty good accounting of it all!
https://steemit.com/steem/@steemitblog/steem-status-100-operational
Thank you, that is of course the experts view - I wasn't too far off for an amateur though! It's certainly been an interesting week, and I seem to learn new things every day about the workings of the blockchain.
I was really impressed how it was handled, the full range of Witnesses pulled together to contribute to solve the problem and also help each other out - an still are right now.
Thanks for all you do for us, every day! 😊
Thank you - but it's not just me I am quite sure, it's every one of the active engaging community members like yourself that keep this blockchain alive - keeping transactions being processed with servers is one thing - but the people actually creating the transactions and engaging / sharing make those transactions to be processed !
I don't understand any of that, but thank you for being someone who does!
Thank you very much, a much more educated answer is linked above by @bluefinstudios - but one thing I did take away from the whole experience is how many people work together from around the world to contribute to keep everything running.
Yes, there does seem to be a lot of teamwork showing up on Steemit lately. And I love the international aspect of Steemit, getting a glimpse into the very different lives of people on the other side of the planet.
I don't really understand the tech side so thank you for explaining what happened!
Thank you, the link @bluefinstudios shared is far more technical then I could manage - it is reassuring to know that so many people work in the background to keep this blockchain running - not only professionals but backup Witnesses so many people around the world contributed to help each other get back on track.
everything restored back to normal that's more than a sigh of relief pheww everyone worked hard
Thank you, I think we were all very relieved when it finally came back to life again!
Well done on getting around this hiccup. My understanding of blockchain forks is that both can run concurrently until one fork is favored over another?
How is it possible that they both fell over together?
How do you fancy getting together sometime and do some videos on the whole witness process. Maybe more people would vote for witnesses if it was demystified a bit?
We could do a hangout or something? :-)
Thank you, I'm still trying to catch up the down time on Monday and subsequent lack of sleep through the night wiped me out last night so effectively I'm running two days behind on posts and comments!
Basically a fork is as you say when then blockchain divides and the newer version renders the older one incapable. In this case the problem was we are preparing for a fork on the 25th for V20, but this error caused it to happen early and without consensus - so it could not be allowed. When it occured only the Witnesses running V20 could sign blocks, but with so many still on V19 who were trying but failing to sign blocks it caused the blockchain to become unmanageablely unstable. It stopped to protect itself, a solution to the problem was found and it restarted.
I think a video hangout on the whole Witness process would be awesome, we definitely should try to do that. If you have Discord grab me on there c0ff33a / Deranged [Witness]#7845
i remember now that you mention it that the blockchain will stop if it runs into anomalies. That makes sense.
Get some much needed rest and we will set up something on the video idea. I'll hit you up on discord.
Michael
Thanks for explaining the situation, Rob. I’m grateful you and dedicated witnesses and developers were working tirelessly to get things running smoothly again.
I did get a message across the screen, when I tried to post, there was a temporary problem.
Thank you, if you check out the post @bluefinstudios linked it gives a far fuller explanation then I could - but the message is the same - many people including Witnesses of all ranks put in a great deal of effort to not only solve the problem, but help each other getting up and running again.
Thanks for the explanation. Now thanks to you I know what happened even though I had an idea but I was confident that it was going to be back in no time. I was trying to post about my new Challenges Project but when steemit.com came back then I finished the process of posting it. Thank you for the info one more time.
Thanks for the update! Let's move it again :)
Posted using Partiko Android
Thanks for all the hard work you and the other witnesses put in to get the system up and running again. I was a bit confused about what was happening. I thought Steemit was blocking me because of my time away! LOL! I did find out on Discord though, and was grateful when all was up and running again yesterday!
I have to be honest I just sat and watched the Witness Chat awaiting instructions and did as I was told, but certainly I can say there is a reason the Top 20 are at the position they are because a great deal was put into resolving this by many of them. While my investment in Steem is small in comparison to others, it was really reassuring for me to see the work put in by our top Level Witnesses - throughout the outage there was constant information relayed and instructions passed to help all Witnesses get back running again.