SteemReports - The New Anatomy of Steemit
A few months ago, I presented the The Anatomy of Steem and Steemit, and I suggest you might like to have a look at that if you haven't previously.
There have recently been some changes to the architecture, so in the process of trying to understand them myself, I made another diagram which might also help people who are less technical, or haven't had time to investigate.
The new elements are shown in the yellow area, and are centered around Jussi, which is a newly developed reverse proxy server. Instead of contacting the RPC servers directly, the web server software, now requests content from Jussi and Jussi is responsible for deciding the most efficient way to provide the requested information.
If somebody requests the list of trending posts under the category 'life' for example, Jussi will check to see whether it has recently stored that data in its fast RAM cache. If not, it will request that from the RPC server, but if so, it can send it straight away without asking the RPC server again. This reduces the number of requests going to the RPC server, meaning more requests can be handled which helps Steemit to scale to support more users.
Jussi also sends some queries to the SBDS server, where that is more efficient than asking the RPC server, and in future will also be able to use the Hivemind database too. This will further reduce the load on the RPC servers, but Hivemind is not yet operational.
When you post, comment, vote or transfer funds, these are called 'write operations' and involve data being added to the blockchain. This can still only be triggered by the RPC servers, but with Jussi delegating most of the 'read operations' to the new services, the RPC server software itself can be streamlined to do its specific jobs more efficiently, which can also reduce its hardware requirements.
Most third-party apps such as busy.org, chainbb.com and steemreports.com will likely also migrate to using Jussi in the next few weeks.
You can find the Github code repositories for the new components here if you are interested:
https://github.com/steemit/jussi
https://github.com/steemit/sbds
https://github.com/steemit/hivemind
Disclaimer: The diagram is slightly simplified, and if anyone with more knowledge has any corrections or clarifications to make regarding what I've explained here, please let me know, as I'm still learning myself!
This is all very helpful. Thanks for the clear diagrams.
Jussi is a great addition to the infrastructure.
This wonderful comment has received a bellyrub 0.73 % upvote from @bellyrub.
Nice overview. Where would alternative apps and sites fit into this picture?
I think that currently, many alternative apps and sites will be connecting directly to the RPC servers (either steemd.steemit.com or public (or private) community nodes).
Steemit have said that over the next few weeks third-party developers using steemd.steemit.com should redirect their API requests to the new Jussi node(s) (at api.steemit.com), and that they will stop supporting direct access to the RPC.
There may be a few compatibility issues in this process, as the interfaces are not 100% compatible, but probably nothing severe. Most community RPC nodes will probably transition over a few months to be using the Jussi proxy layer in front of them too.
I'd think in the bottom two niches, as they're not hosted on the Steemit servers (or not the same instances, anyway).
Very interesting. What is the difference with the hivemind and the SBDS?
I'm not yet exactly sure what SBDS provides, I'm hoping to have play around with it in the next few days. I do know that Hivemind's DB schema includes some tables for Steem communities data though, which I guess SBDS doesn't.
I love seeing developments on steem.
This will bring steemians that will invest big time.
Joining steemit is one of the best decisions I've made in 2018
wow
so great information about "how steemit works" I really need and like to know more! thanks for sharing
Its good to know how steemit works...
And i love the innovations and uniqueness of steemit.
I'm glad i found myself here.
Awesome ....
Thanks for sharing and i will follow you and you also follow me brother
Hi steemreports. "steemgigs" is my witness and there is currently another full public rpc node on wss://steemd.steemgigs.org, i don't know to publicize its existence beyond doing a post, however, i am thinking you can add it to steemreports.com. I will be grateful. I didn't know how to contact you in chat
Cool bits of info here, thanks!
Hey, slighty off topic - for the past 24-hours or so I've been getting this error message Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application. for http://www.steemreports.com/steemit-chart/openmic/?show=new. Any idea's what's up there or when it might be fixed? Thanks much!
Sorry, I didn't see this until now, but luzcypher got me on chat, and I fixed it. Good luck with the judging!
Thanks, man! Yeah, I rely upon that page for so many things it's insane! Thanks again!