Troubles in paradise

in #blog7 years ago

So yesterday I was enjoying a bit of well deserved rest after the guests that came to @mlvanderwaal's birthday party had left, when I got a notification via Steemify regarding a message of fellow-witness @rival. Our server had started to miss blocks, which means it had failed.

Quickly I flipped open the MacBook to see what was the cause of this. As I couldn't figure that out straight away, I disabled the witness account (to avoid missing any more blocks) and started working through the logs. The moment of failure was easily spotted, but the cause is still a mystery. My best guess at the moment is network issues, as memory/cpu/diskspace/etc. were all within limits.
I will submit an issue on the Steemit github just in case it could be related to a bug in the code.

I cleaned the data and restarted the steemd (node) software, to sync it back up to the blockchain. As the backup nodes seemed unaffected, I switched the witness signing key to our main backup and I could finally go to sleep after a very long day.

image.png


The witness tool SteemTurbine I'm developing would have prevented missing any blocks, but currently it is running in debug-mode on our test witness servers, so unfortunately it didn't spot the issue on our main server. I actually planned to put SteemTurbine into production this week, but ofcourse Murphy with his dreaded law came around just before that...

Until SteemTurbine is running in production, I will be checking on the server manually several times a day to catch any issues as soon as possible.


Special thanks to @rival for watching my back. Working together with this awesome community can and will make this platform thrive!


Me and my fellow blockbrothers are a witness as @blockbrothers. If you want to support us we would appreciate your vote here. Or you could choose to set us as your proxy.

Vote for @blockbrothers via SteemConnect
Set blockbrothers as your proxy via SteemConnect

As blockbrothers, we build Steemify, a notification app for your Steemit account for iOS.
Get it Here:


Sort:  

thx for mentioning me, if we all help each other this will be a great platform..

@bennierex Thank You for sharing with the Community about this event and I am glad that people like yourself are doing everything they can to make STEEMIT the Best in Social Media Sites for all...................

Coin Marketplace

STEEM 0.19
TRX 0.15
JST 0.029
BTC 63592.23
ETH 2551.58
USDT 1.00
SBD 2.75