Steemwallet.com: How to improve security in a least authority way AND how "NOT" to roll-out such security measures.
Steemit Inc is getting into an escalating habit of disruptive and destructive updates.
We all remember the HF20 disaster, and hopes were Steemit Inc would have gotten a major wake-up call and would start to apply lessons learned from the HF that nearly killed the platform. Maybe try and be a little more controlled, a little less disruptive when planning to roll out new stuff.
Sadly that didn't happen. With the API rate limit that killed off API intensive statistics scripts like my daily flag-war report on my @pibarabot account, Steemit Inc showed once more it had zero problems breaking stuff. But then, that is just statistics scripts. People may miss the drama from my visualization of the flag wars on the platform, and for me as a dev it sucks they broke a script that took weeks to develop, the rate limit wasn't all that disruptive to the platform and its users. What it was though was a big warning sign denoting Steemit Inc hadn't learned much from the HF20 disaster. That Steemit Inc still has zero reservations in rolling out potentially disruptive changes without thinking through or checking out what they might break.
Today they did just that, and in a way that I feel gives HF20 a good run for its money as potentially being the most dangerous roll-out for the platform since inception.
At first glance it might seem like a good idea. I'm a big fan of least authority solutions, that is why I wrote this proposal for faccet contracts for the STEEM blockchain. Least authority design is good. It is important. But if you migrate from a sub-optimal solution to a better solution closer to least authority, there are multiple steps that should be thought out meticulously. If you don't, as the steemwallet solution shows, you create a gaping hole for people with bad intentions to jump into.
So why am I making so much fuss about the split ? To understand that, it is important to understand how Phishing works, it is important to understand what phishing mails look like.
Basically the problem with the split is all about trust relationships. We as users of the steemit site have over the years build up a trust relationship with steemit.com as a domain. Likely many of us have installed some kind of petname extension into our browser. So if we look at the different domain names steemit uses and/or could have used (or that a phisher could hurry and claim and use) and how they relate to trust levels we get something like this:
- steemit.com: High
- wallet.steemit.com: Medium
- social.steemit.com: Medium
- steemitwallet.com: Low
- steemitwa11et.com: Low
- steemwallet.com: Low
Now what Steemit INC is basically saying:
- Don't trust the steemit.com domain that you feel you can trust as much as you did (only trust it with your posting key).
- Trust this new random named domain that any phisher could have claimed instead. Trusted with at least your active key. Oh, and only trust this claim from us because we posted about it on @steemitblog that everyone should know is really us.
This is basically the worst option they could have chosen from the splitting option at their disposal. A proper role-out IMHO would have looked something like this:
- Keep trusting steemit.com like you were trusting it before. As you were probably already trusting it with your active key, we'll keep that the same and make steemit.com the new wallet site.
- Trust the site social.steemit.com with just your posting key. It's the same domain so partial trust is implied if you already trusted steemit.com, but you don't need to trust the new site with your active key.
If Steemit Inc had picked that road, everything would have been perfectly fine and the split wouldn't have opened up the social gap it is opening up right now. If Steemit INC had run their plans by someone with a bit of a security background. Anyone with a bit of a security background. Then this new intrinsically dangerous update role-out could have been a seamless improvement in overall security.
As they didn't though, I feel this roll out is just one in a never ending cycle of ill prepared role outs. Moving to an other frontend for posting and using a local wallet would be an option if sites like busy didn't use steemconnect, a service that basically demands you delegate your active key to it in order to delegate posting rights to any of the alternate frontends.
So for now steemit.com is probably still the best option for posting purposes, but for wallet purposes, the best option from now on is probably using either a local wallet or using an interactive Python shell with @holger80's Beem library.
Ideally though, Steemit INC should IMHO take an example from witnesses like @reggaemuffin and create and create and post a statement with Adoption Requirements, not just for hard forks but also for things like this new web-site split or the recent implementation of rate limiting.
So while I can really appreciate the ideas behind the split into two websites, as many times before, Steemit INC has shown to be brilliant as visionaries but not so brilliant as for the quality control of the roll out of their visionary features. We really need to devise some kind of community QA think tank that both Steemit INC and the Witnesses could be provisioned by or with @steemalliance that could provide Steemit INC with the proper skill and knowledge set to minimize the impact of roling out visionary new features.
Please note that while I'm being to the point, this post is in no way meant to tear down Steemit INC. It is meant as constructive criticism that I hope might help Steemit INC and related entities such as the (top 20) witnesses and the @steemalliance to step up their game. I seriously wouldn't have made my capability-based facet-contract implementation for the STEEM blockchain proposal if I didn't believe in the potential Steemit INC has. As I've said before, I would gladly donate up to a workday a week of my time to any coordinator witness approved by @steemalliance to work on contingency control and least authority geared reviews or infrastructure. I think there will be many others who would be happy to do the same.
I agree with many things in your article, but c'mon, if you use an rpc heavy script, deploy your own node. Nodes are expensive and steemit rightfully limited their api so that their funds are not wasted on unoptimized bots.
For example buildteam runs their own private nodes that costs about 1k per month or more. We use rpc nodes a lot. We pay for our use.
If your stats gain you money or are worth it, use that money to pay for rpc nodes. Same as you pay for the computer doing the calculation.
What my script did was:
The result would then get processed into something like this.
The stats were made more in support of the work being done by the @freezepeach initiative than for making money. Most posts made just a few cents, half of what I had set @freezepeach as beneficiary for, so that wasn't the point. The point was making people who were being structurally targeted by high powered flagging for disputable reasons to be made aware of the @freezepeach initiative.
Like most stats script, there would be a large amount of API calls done in a short time and the rest of the day zero. There are reasons to do rate limiting, proper rate limiting with window sizes, flow control, etc, you likely know the routine. That however isn't hos it was implemented. They just made up a number and a timeframe. Hit the number of API calls they had set as upper limit within 5 minutes and get 4xx errors for a while after that. No library, including my own is capable of working that into a decent client side rate limit implementation that way.
So it is not the rate limit itself that is the problem, just like it wasn't the HF20 features that were the problem, it is the implementation and lack of adoption requirements that could allow for gracefull degradation of the service instead of a disruptive one.
Hope I'm at least partially making sense.
Hmmm if you do something like that, maybe try steemsql? That can run aggregate queries.
Allowing such bursts would allow apps to basically DDOS the api, especially with the expensive history lookups you do.
If you want to improve your design, stream blocks, then don't collect posts but flags and then put them in a queue and query one of these posts per 3 seconds or something. And at the end of the day do one query of the data you collected over the day.
No real world api allows such expensive bursts without limiting you very quickly.
Possibly valid for my script, but not really the point why I brought up the rate limit example.
Other API's that implement rate limits tend to use custom HTTP headers like:
Or simply:
As a library author, these headers are something you can work with to implement proper client side rate limiting. You can run statistics scripts that run at exactly the server defined rate-limit speed. That means a script that used to run for an hour might run for 15 hours now, but it will still run. Graceful degradation.
There are established best practices for implementing rate limiting for a web API. Just picking a limit size and period that the client would normally be unable to guess and then start responding with 4xx errors for a period of time the client couldn't reasonably guess either is in no way graceful degradation. It is a disruptive roll-out.
Yeah I can agree with that. They probably needed something that stops the flood quickly and did not do any play nice stuff.