Steemit.com and Nodes Operational
Hello Steemians, steemit.com is back up as are all of Steemit’s nodes. Over the weekend the chain halted due to a bug in the curations reward curve that was unrelated to the previous halt or the delegation bug. While all services have been restored, and operations appear stable, we wanted to take this opportunity to fill you in on what occurred. A more thorough investigation will be conducted and more information will be shared after our next retrospective.
We were able to develop a patch which did not necessitate a hardfork, however, it did require a replay of our nodes which are used by steemit.com and other Steem applications like Steem Monsters. Witnesses upgraded using the patch and resumed block production on September 2nd at 06:26:42 UTC. Around 13:00:00 UTC on September 3rd Steemit and the Steemit API services were restored after replays were completed.
The Bug
The bug was especially resistant to detection because the problem was not with the mathematics backing the curation reward curve, but subtleties in the integer based approximation of a square root. The bug was the result of a subtle behavior in the new curation rewards curve that lead to votes being able to have a curation reward weight of -1, which when stored in an unsigned field resulted in a computed value of 2^64 - 1.
When paid out, these votes were attempting to receive trillions in STEEM. This was caught by other assertions which instructed the blockchain to halt block production and prevent the unintended behavior from impacting any reward distribution.
The Fix
The new release creates a floor for the curation reward weight in such cases to 0. This can only happen when votes are allocating very few rshares and would not receive a reward, so forcing the value to 0 will not impact how rewards are allocated.
We apologize for the inconvenience. We will be conducting a retrospective and sharing the results of that retro on @steemitblog, so if you would like to learn more about this incident and what we have learned from it, be sure to follow this account.
The Steemit Team
OldSteem went out fighting, much how it lived.
It's NewSteem time! Let's thrive.
!BEER cheers to the new STEEM :))
mister whatsup , in new steem whales are downvoting even good post who not agree. not everyone's thinking is same. every people have a own definition of good post. That's why they downvoting own way. some days ago my friend get downvote and when she asked to downvoter that why you downvoted me then he answered that your post is not related to crypto , post only crypto related topics LOL.
Your friend was likely using the wrong tags and misunderstood the flags. :)
This is in the nature of downvotes: they are misleading and are usually perceived as offensive and unfair. Nobody likes to see a red flag or disappear his comment half invisible. This is always followed by an emotional reaction. If you want it to be emotional, this is of course the best method.
Instead of misunderstanding, one could also choose an alternative method. Usually it costs more time and energy. If you want to be fast, you won't take the time to look for alternatives.
However, I would think there is no such thing as "wrong tags", because the categories are freely selectable. I can decide which tag I take for what. Only because someone is perhaps cheeky and places his topic to adopt the methods of advertising for himself, he does not deserve a downvote. Is my opinion. Those who are bothered by someone using a "wrong tag" can safely ignore the content after all. The policy at @steemstem says exactly this: You can post under the "steemstem" or "science" tag, but whether you are perceived or read is a completely different matter.
I think everyone has the intelligence to recognize when a tag is a phoney and decide whether or not to pay attention to the content.
If you would like to use a platform that has eliminated downvotes you really should go check out whaleshares.com
If you want to earn Steem, we have to have a way to curb abuse. So, if the downvotes are too much for everyone to handle I guess we should just pay all the inflation back to stakeholders and take the financial end out of this entirely.
I still haven't made up my mind what I use this platform for and if money making is my first goal (guess not). The place here has many different faces and they appear according to what the people make of it. I came here for blogging purposes and thought that being voted on my blogs are an interesting thing to experience. I heard a lot about decentralization and saw it probably as an inspiring and interesting field of experimentation. Went through the usual ups and downs. Learned a lot.
I've got some ideas how abuse was rising within a system in the first place and how it could be lessened in the long term. I favor an indirect approach which I explain on my blog.
Never been on whaleshares.com - so there is no downvoting?
They eliminated downvotes entirely. It's been messy for them
Very appropriate. This ought to and should be done expeditiously!!
Amen....
Someone pretty smart worked this out!(hope they got a beer)...Still, it does seem the kind of thing that should have been picked up on the testnet?
-is there a key reason it wasn't spotted, are you improving the test environment to pick this kind of thing up moving forward?
This is very necessary! Thanks for bringing it to their notice!!
Thanks for digging the Steem Grave a bit dipper. Great Job Team!
Thank you for the update. I am glad the new steem is back
glad too
me too
Thx for the update. I think the next hardfork needs a better Testnet testing and also the time for a complete replay should be optimized, maybe by introducing checkpoints that can be loaded more quickly than replaying the whole blockchain.
After having some replays I'd love to see some way to get faster replays. Great suggestion :)
100% right!
Impressed by how quickly the bug was detected and a fix provided considering the time of the week this happened. Creds to Justin and Vandeberg in particular for providing a solution.
After sleeping till 3pm today to catch up after a marathon of ensuring our own witness got up and running asap, I can't help but respect the shit out of everyone else who was present over the last 48hours.
They really did work their butts off! Steemit is so fortunate to have such a dedicated team!
We shall all respect the hard-working and firm people in our lives.
Someone had to frick up at first - testing is not a light task! Testing in important projects can take up to 40% of the time, not in this last HF. Don't do that again!
I read your comment and it's right to cheer at firefighter, but in this case... some of them might have also been the ones seeding the fire! That is not ok.
I'm ready for #newsteem to take my body.
wuhhuhhh nice
HF21 sucks bigtime, but was somewhat expected. HF22 seemed okay, but was unexpected downtime. What are we calling this latest one? Not HF23, but... "the 48 hour can't-divide-by-zero surprise patch"?
Really not on a roll, from what I've seen. A lot of people seem pretty ticked off at how things continue to go here. Unexpected downtime is a definite hard limit for so many people. Investors are mad, content-creators are mad, developers are mad, and the public is avoiding this place like the plague. We're developing a really terrible reputation, and I kind of understand why. This place may have been able to rest on its groundbreaking programming in 2016, but the years are ticking by and we've gone nowhere but downhill (and slightly sideways) since then. We need to do better. We had such potential!
That madness you talk about is from my point of view not necessary. Systems do change on long terms and that needs patience and calmness. Seems, you think that, too?
I tried several times to propose alternative mechanisms - still do :) If you would like to have a look, read here: https://steemit.com/steemit/@erh.germany/the-usage-of-voting-buttons-reload
I'd like you to participate in that experiment I proposed.
Not sure where you've been, @drutter, but you must have missed MIRA's release. That little thing from STINC that dealt with STEEM scalability. Something that virtually no other crypto has done, so that STEEM continues to be just about the fastest and cheapest blockchain out there. Oh by the way the testnet is open to all. Did you do much testing before complaining like someone around here owes you something?
As you know, I was here for MIRA. But as you forget, I have zero understanding of pretty much anything you just said. I'm a content-creator and that's pretty much it around here. Cheer up, you might not be underwater forever.
Is that some kind of investment lingo?
For transparency sake, my investment approach is that of a HODLer. My investment cryptfolio is all in PoW coins, as DPoS seems too susceptible to corruption to be considered for investment, in my humble opinion. My interest in STEEM is
for non-investment reasons.
My big buy in to STEEM was at 3275 BTC satoshi. It brought my account up to Dophin status and left 1000 STEEM which was gifted to my daughter and Son-In-Common-Law to start their STEEM journey as Minnows.
That buy in cost approximately 0.18 BTC or approximately $1800 at today's trading value. Through the magic of inflation and stake based voting rewards, this account is now showing an Estimated Account Value in US dollars of $1,449.55 even though STEEM's value in the BTC pairing has dropped and 1000 STEEM was given away.
The biggest buy in to my Bitcoin investments was done in the bear market of 2015 at $330 CAD. Prior to that doubling down all my other coins were obtained when one could still mine BTC using a graphic card.
Hope that clarifies my cryptfolio so that you can, at least, not make idiotic innuendos.
So you do understand what I meant by underwater, then. You got REALLY mad. Good thing you're rich and awesome though, carry on.
Gonna be hard to recover after this one. Reliability is extremely important in my opinion when you messing with peoples income. Isn't this what testnets are for so you don't mess up the production chain and end up with egg all over your face?
When you talk about "reliability" some thoughts were coming up, considering "liability":
I think the problem is probably due to the fact that Steemit has been advertising profitably from the beginning on an extremely unstable platform that is still in the testing phase. If you like, the test isn't over yet, I'd rather think in years, if not decade. Until then, many hardforks will follow.
However, it is difficult to express reliability and responsibility at the same time in speculative transactions. If the price hadn't fluctuated so much and gone down to the basement, fewer people might be bothered by the changes.
It is difficult to tell where the dissatisfaction comes from: From the decisions of the company, witnesses or a general depression in the crypto market and the Bitcoin dropped off. Much is mixed up. Causes and effects are understood causally and linearly, but they are not. It may well be that Steemit Inc. only wants to keep the store running, but is not interested in innovation. Who really knows? Difficult to say. Corporate communication doesn't point to it, many things seem to be quite unprofessional and with very little investment in personnel costs. But that's a guess. You would have to see an annual report. I don't know what it's like in the USA, but you can look into it in Germany.
The Federal Ministry of Justice publishes not only court notices and commercial register entries in the Federal Gazette, but also company data such as the annual financial statements.
Companies with a legal form as "GmbH" or "AG" even have a duty to publish their balance sheets in the Federal Gazette. Interested business people certainly like to read annual reports in order to estimate the willingness of a company on spendings both structural and Human Resources wise.
Okay, #newsteem...
Kudos for getting back up and functioning on a holiday weekend.
Side-eye for not updating that Twitter post so we knew what progress was being made all day long.
I was contemplating making a bigger investment into Steem. I'm definitely going to wait on that now...