@themarkymark witness update and incident response

in witness •  5 months ago


Hi Steemit!

What a night, all three witness nodes and all four full nodes updated to 0.19.5 smooth as silk. This is about 4AM in the morning. When bringing up the witnesses with 0.19.5 we had to run a temporary parameter until everything stabilized.

Once the blockchain stabilized, I shut down my witnesses as suggested and removed the temporary parameter and BAM all witnesses nodes are in full replay. This takes them offline for about 3-5+ hours depending on the hardware. I waited about 2 hours and went to bed for an hour for the last hour or so while they replayed. Got up an hour later than I intended and primary witness is all online and enabled again.

I was the earliest non-top 20 witnesses to produce blocks but ended up having to replay all my witnesses due to a bug. I filed an issue on github for the bug to hopefully prevent it in the future.

No blocks were missed, and all witness nodes are online.

What happened in the first place

I won't get into details of what happened and leave that for Steemit Inc to report, but everything is stable now and back on track.

And now something completely different

As always, I like to talk about something unrelated at the end of my witness updates. I recently hit my 1 year anniversary (cleverly called Steemiversary) a couple of days ago and wrote a post with some details of my journey so far. Feel free to check it out.






X48EJ

Why you should vote me as witness

Witness & Administrator of four full nodes

Active, Present, Passionate

My recent popular posts

STEEM, STEEM Power, Vests, and Steem Dollars. wtf is this shit?
The truth and lies about 25% curation, why what you know is FAKE NEWS
WTF is a hardware wallet, and why should you have one?
GINABOT - The Secret to your Sanity on Steemit
How to calculate post rewards
Use SSH all the time? Time for a big boy SSH Client
How to change your recovery account
How curation rewards work and how to be a kick ass curator
Markdown 101 - How to make kick ass posts on Steemit
Work ON your business, not in your business! - How to succeed as a small business
You are not entitled to an audience, you need to earn it!
How to properly setup SSH Key Authentication - If you are logging into your server with root, you are doing it wrong!
Building a Portable Game Console



Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

One thing it does bring to light is how responsive the top witnesses are to this sort of thing. I cannot imagine this kind of quick turnaround for an issue in the sector in which I work as my full-time job - banking. There would have been meetings and phonecalls to arrange more meetings and phonecalls. It's encouraging to see everyone working together to protect service.

Thanks for the update man, i know my proxy vote is in good hands.

Was morning in Thailand when I couldn't log on! Was not panicking at all lol !! Any chance Steemit inc. could tell its users in advance when there is going to be downtime please :-)

·

It wasn't planned, but I am sure there will be an announcement soon describing what happened. We are in better shape now than we were yesterday.

·
·

Cool, keep up the good work fella and have a great day :-)

Is there any possibility I would understand what happened when they describe it ? lol....doubtful :-) some things are best left unsaid!!

·
·
·

Yeah, it isn't too complex, just unexpected.

Thanks for getting us thought the great steem blackout of 2018

Thanks @themarkymark for the update. Have a nice day

@themarkymark, We appreciate you staying up so late to get the nodes updated and back online.

You already had my witness vote. Thank you for being a reliable witness.

Why you should vote me as witness

Although you presently still have my support for witness this could soon come to an end due to the voting bots issue.

My position as to why voting bots are bad for the steem community were laid out in a recent blog post, whose link is included for your reading pleasure.

https://steemit.com/steem-network/@novacadian/where-is-the-trust-in-proof-of-brain-mining

A solution was suggested in a more recent post, whose link is included for your reading pleasure.

https://steemit.com/steem/@novacadian/a-decentralized-capcha-annonymous-proof-of-brain-verification

My question is, would you support my solution outlined in the link above?

·

I am not a fan of the idea of captcha and I don't think voting bots are the issue, trending sucked before voting bots, it sucked during, and it will suck after unless other things are fixed.

The problem is 80% of Steem users shit post and put no effort into their content and there is not enough manpower or stake being used to counter their actions.

The steem power used by voting bots offers a fair and transparent promotion service for all Steemians. Remove them, and it will go underground and will be similarly used but will not be transparent, fair, and limited to the select few in the good 'ol boys clubs.

·
·

Guess you didn't read the first link that explains how non-proof of brain voting is producing counterfeit steem.

Appreciate all that you do for the community and genuinely like you from our interactions on the platform yet my new policy of supporting only witnesses that support abating the creation of counterfeit steem will mean my support for you as witness will be given elsewhere.

P.S. You will likely not notice my minnow sp.

·
·
·

I did read it, but it creates no more steem than if the original SP (delegator or bot owner) used if voted. The same amount of steem is used, it is just distributed more across many users.

·
·
·
·

It is not meeting the criteria of proof of brain. It would be like a proof of work miner changing the algorythm. Such coins are counterfeit. That is, made under false pretence.

·
·
·
·
·

Have a look at New and tell me how much proof of brain is going on.

The fact of the matter is Bid Bots are here, like every media platform on the planet there is promotion. I would rather see it run fair and transparently than underground and black market.

·
·
·
·
·
·

A CAPCHA would sort the wheat from the chaff for those that choose to use it.. Even if only as a tool for SMT Distributors who wish to only promote their coins to real viewers.

One's settings could have an option to use the CAPCHA or not. The rewards from those who use it should be greater than those that don't do as to encourage users to comply to steem's white paper.

·
·
·
·
·
·
·

Have a look at New and tell me how much proof of brain is going on.

You are proving my point.

P.S. An even better example is Trending.

Nice post
Good luck

Thanks for being vigilant on this. I just changed my witness voting from a proxy to manual and you for sure have one of my votes. Thanks for the update!

·

Thank you for providing this info... I often feel like there is nobody explaining these outages. From my user perspective, Steemit seems to have abysmal uptime. Would love to see some stats on this.

·

Once there is an official announcement, I can break it down into laymen's terms if need be, but I don't want to be the one to announce it.

·
·

Thank you, that would be appreciated.

interesting and useful information

I have a little note, maybe you are interested to read it. I do not expect an upvote from you if it is not interesting. But if you are willing to read it I am very happy. Hope you always suskes.
https://steemit.com/indonesia/@catataniranda/15-steemian-world-with-the-highest-reputation-one-from-indonesia-bilingual-f64ccc89132f1

Could not log in to my account. Was not scared though. The team must inform about this in advance.

·

No one knew it was going to happen, there was no way to notify in advance.

·
·

Ahh thats okay, I voted for you in the witness post.

·
·

Really bad timing...lol, taking more backlash from stuff said a couple weeks ago, in the comment there was "she could have flagged your account down to nothing" to which I replied I would say the same thing again...next thing I know everything goes down...lol, now that is truly what you call bad timing. I thought I was banished into some never land.

Loading...

Thanks for the help during that day and congrats on your 1st Steemiversary!

Shocked how you're not one of the top20 witnesses -- you seem to be doing substantially more for the community (at least in a visual way) than most.

Big ups. Keep up the good work.

·

Thanks, I'm trying but doesn't feel like that will ever happen.

Loading...

Are you on discord?

Guestimate confirmed.
25 to 30%. Add favored apps, and ~15k authors get to split ~55% of the pool.

Can i get an opinion on this?

·

I'm on @buildawhale's Discord as well as many others.

Loading...