RE: Steemit Retro: August & HF21/22
As a professional who ran a software development firm for 20 years and has been in the field consulting to hundreds of companies large and small on Mission Critical Systems.
My Assessment from the outside without even having to be in the design, planning, development and business meetings was this migration was a complete and abject failure.
You have a run or catch fire attitude towards development with out the infrastructure to support an AGILE Development infrastructure.
Your methodologies are those of hackers, with no adherence to ISO test standards.
I don't know who is in charge of QA there if anyone.
There obviously was not a well thought out test plan.
There obviously was no regression testing, a little unit testing maybe.
You don't have an adequate Test environment.
You had no bridge back and store and forward plan.
No one is looking at the overall system integration because things like reindexing times, can easily be calculated and planned for. And knowing how your intial new account staking works was not thought of.
The worst part was because you had no falll back procedures you implemented another fork patching stuff, instead of rolling back and testing.
This cost your user base lose of time and money.
On top of it all your communications continues to suck and your flagship product on the blockchain SteemIt.com has not had a single needed MAJOR enhancement in 2 years. Jeez you can't even change the color of your background.
And to top it off your stuff not working and you complain that the "abnormal, and unacceptable, burden on engineers" ...... Which I might point out was caused by your own actions.
If you had slowed down and tested enough. Released incrementally and had a temporary fall-back to regroup planned in case of a major SNAFU (which in this case happened). YOUR ENGINEERS WOULD NOT BE OVERWORKED AND COMPLAINING.
Whoever the managers and upper level decision makers on this blockchain are obviously not knowledgeable in what it takes in a full SDLC.
You are in a mature running system with real peoples money involved here.
This is unprofessional and unacceptable.
I could send you simple admins from companies in the 200-300 person level who don't know squat about this or any other blockchain, who would have had a better migration and they earn only about 120k a year!
This is your retrospective... YOU just had an unmitigated disaster on your hands and your white washing it with this post!
I have no idea if the things you listed are accurate, but can you get a job at steemit team in Texas?
I think you have to be more careful, for once, as @justinw pointed out, rolling back is just not an option on the blockchain. On the other side, if you ran such a firm for such a long time, you should know that you're never able to catch 100% of the bugs before you deploy it in practice, the most important thing, though, is to make sure you don't make the same mistake twice and be prepared for the future for those tests.
As you pointed out in your comment and as Steemit pointed out in their post, in terms of re-indexing after a failure they should've been more prepared and should've had more recent backups on their hand to be able to react more quickly.
Similarly, as also Steemit wrote, the environment on the test servers should map more closely the production environment.
I do agree on the Steemit updates though, as the flagship of the Steem apps there should be more focus on its development. Currently they have their devs working on communities which is probably why they don't have any free hand for the Steemit main page.
Unfortunately, most of these things should've happened already years ago, but because a certain CEO spent more time with his hair than with his leadership nothing got done.
This year I'm more positive since Steemit Inc seems finally to be learning from the mistakes and are trying strongly to do a better job, include the community and communicate.
You are delegating to a downvotebot which downvotes me for nothing! Thats why i have to downvote your comments! Best regards!
I delegate my steempower over dlease.io. Just downvoting my comment to revenge yourself against this downvote bot without considering the quality of my comment, without mentioning who this downvotebot is and without trying to contact me first is a pretty slick move. I hope you continue like this, we need more of this on this platform.
Yep-we need definitly more of that ones who lease their SP to downvote bots for something like 19% via dlease, hope also you continue like this, we need more of this on this plattform.
And he still didn't tell me who this alleged downvote bot is =D
Should know who you are delegating to, i am not your nanny.
Didn't even know how terrible downvote bots are, they must be making a lot of money on this platform nowadays.
It is pretty simple-You delegate SP for something like 19% return of your invest to a downvote bot. This bot steals a -small- part of my rewards. So you get -small- downvotes from me. Pretty simple. Best regards.
Not a complete whitewash. I feel this one provides hope for the willingness to learn from HF21 things they failed to learn from HF20.
"The challenges that have arisen out of hardforks has placed an abnormal, and unacceptable, burden on engineers. This is not only unfair to the engineers, but also leads to fear and anxiety about future hardforks. While Steem’s facility with respect to system upgrades is a feature we believe should be exploited, we must dedicate more effort to ensuring that this can be done in a way that sufficiently considers the psychological well being of not just engineers, but community members, stakeholders, users, exchanges and Witnesses."
As @justinw states "Mistakes can happen, but it is very important to make sure you aren't making the same mistakes twice.". While we could claim they did make the same mistake twice now, with both HF20 and HF21 failing to do sufficiently thorough testing and resulting in massive down time (from a HE perspective), you could also look at it that the mistake was that they aparently learned the wrong lesson(s) from HF21, and are now ready to learn the right lesson.
As I wrote in my response to @justinw, I think Steemit Inc should go visit a few modern HE shops and see how they deal with agile, testing and a CICD in a HE-DTAP setting. Part of their problem I think is that they believe they are somehow in close-to virgin territory when it comes to finding the golden balance between HE and agile paradigms because they are a blockchain shop. I think once they figure out they are not, they should be ready to lose what from a distance pretty much resembles high-school science project approach to deployment.