DEVELOPERS: 100% Free use of a real-time database for STEEM apps
Ingestron is a tool for ingesting the blockchain's data into rethinkdb:
Real-time web apps are subtly different. In a real-time app, there's no pressing refresh or reload. Instead, new content just streams into the user's browser over a websocket. This way, interactions feel faster and more natural. Comments on a post would slide into view about 4 seconds after being written to the blockchain. In the photo above, you can see ingestron eating the blockchain and spitting it into rethinkdb.
Ingestron can be found here: https://github.com/faddat/ingestron
Its existence owes many thanks to the go-steem team, who wrote the library that enabled ingestron and the example app that underpins its design as well.
Some might question the necessity of this, but I'm not aware of any blockchain that can deliver the data it stores quickly enough for real-time analysis from its daemon application. As always, STEEM is far, far faster than most chains, but sites like steemd, steemdb, and steemshovel all use some form of intermediary database. In fact, steemit.com does, as well. All I'd like to do with ingestron and the rethinkdb cluster I set up is grow out our ecosystem and maybe get a sense of how my servers perform with some load on them .
Never fear, though: This is to be an entirely open source design. If you're a little batty like me, and want to run the whole architecture for yourself, please know that you can contact me for help any time using google hangouts at [email protected] or you can read my forthcoming articles, where I will be going over:
- Setting up a steemit.com development environment including:
- STEEM
- Steemit.com
- Mysql
- Caddy web server for automatic https
- Steem's config.ini: Which settings to use for what and why, laid out in an easy to read table
- Using Hugo to generate HTML from the STEEM blockchain
- Setting up a CockroachDB cluster for holding STEEM blockchain data
- Setting up Rethinkdb in a Clustered, Sharded, Replicated fashion
oh that's awesome! I was going to do the same thing and you saved me the trouble.
:D.
So, I think that this post's proceeds are going to go to a dedicated cluster just for DB's with steem. Tell me, have you looked at crate.io?
I have not seen crate.io before. Looks very interesting though. Going to have to read more on it.
Thank you for your work, I will probably use it! Keep going!
Here's a better photo showing the replicated, sharded nature of the cluster. While I'm busy writing connecty-doodads, I'm going to write a cockroachdb one as well.
Re: Rethink: I come strongly from the ops side of the house. Anyone know how to pull a nested array and make that into a new table?
If not, that's okay; I'm sure I will eventually figure it out. The big prize is in that nested array, though: The transactions...
I upvoted. I would be interested to see a bigger version of this picture.
Ask, and you shall receive.... but you're going to need to click the link:
Awesome! Thanks!
The free database software like MySQL and others are very useful too and come real handy. So do check them out.
dickbutt
https://steemit.com/dickbutt/@skeptic/dickbutt-has-taken-over-steemit