I Am Now A Customer of @steemsql and @arcange - Exciting Steem Apps Soon to Come!
I finally found some time recently to look at @steemsql, the database service provided by @arcange that converts the Steem Blockchain into a usable Database structure. Let the data manipulation commence!
Since I posted recently of my idea for what I called a generosity rank algorithm for Steemit to help improve post quality and reward pool usage, I decided to start coding a proof of concept using @steemsql. After a couple of days @arcange announced that the service would no longer be free to use as it requires considerable hardware and time to keep running - so I have become one of his first customers, playing 10SBD for one month's access to the data - so I'd best make good use of it.
So far I have got about halfway through coding the vote rank index algorithm (no longer named generosity rank algorithm) and I aim to have a usable page online within the next week - I am very busy at the moment with other work.
Observations of note so far:
- Indexing the entire blockchain will take days of processing, so initially I will just use data from the last 7 days of paid out posts to calculate the index values and ranks for each profile.
- I need to take account bot profiles since they are currently topping the rank table (utopian in particular). If we compare everyone's vote behaviour to utopian or another bot, we will have skewed results. I might make an option to either include or exclude bots on the finished rank checker page.
- The general concept of the algorithm appears to work ok. I will adjust it further once I am able to complete the first full indexing. Even with a greatly reduced range of data to process, the processing time is still quite long.
- I can see interesting accounts being highlighted by the code, at least one of which is actively advertising that they are 'raping the rewards pool' for fun.
- Since I am calculating the value of historic vote payouts, I am able to see some stats that I haven't seen on other sites - it's pretty interesting. Maybe I will make some graphs too just for a source of additional statistics.
More to come soon!
Wishing you well,
Ura Soul
View My Witness Application Here
Shouldn't the STEEM blockchain already be indexed? This seems like if it is not it should be a part of HF20 or at least HF20.1 if it cannot included in the primary fork. This actually leaves me a bit dismayed.
While witness servers are looking at an ever increasing RAM allocation one would think that indexing is something that could reduce the need for RAM. What I am hoping is that the indexing is only missing from the SteemSQL interface and not the actual STEEM BC.
I like your proposal and I hope it gets serious consideration. Your proof of concept will go a long way in getting that added to the upcoming HardForks. All I know is that something has to be done.
Return Of The Flag Wars ...
... was the name of one of my more recent post about a war most of us are familiar with.
Reward Pool Rape, be it deliberate or unintentional is a problem. The Flaggers are filled with hate and vitriol and it is directed at the ones on the receiving end of the rewards and more unfortunately, their followers. It is directed at both parties with extreme vitriol and what appears to be malice.
The language is abusive and demeaning, and this for a flaw that is entirely system based and was only uncovered when a particular user became famous over night for what in my opinion are in fact not only valid post but generally post that have great value.
Looking forward to seeing you get this proof of concept launched.
I want to see a civilized STEEMtopia going forward.
NOTE: you got my last open Witness vote.
I don't really know much about how the blockchain and steem software operates behind the scenes, I imagine it is indexed in useful ways to allow nodes to run efficiently. Steemsql, by it's nature is a third party service and SQL is not inherently part of the BC structure - so it seems Steemsql might be a bit limited in comparison to what a full relational database version of the BC could be.
The index I am creating is more of an index in the sense that google indexes pages on the web, rather than in the context of a database index for performance - I'm just running a test of my script now on one of my servers - I might even get a full index kicked off tonight, but it depends on how much time and resources it's going to take as to whether i attempt that without any optimisations made.
fingers and toes crossed! thanks for your witness vote :)
Keep at it my friend. That generosity algorithm is a great idea, and needs to be used a bit more. Have you seen the steemworld data that gives a community score, CS. It’s pretty good, just gives you an idea on how much self voting you are doing. Not too complex but a good start
Thanks, yes, I have seen that - although I had never heard of steemworld before. I'm aiming to make mine more useful - we'll see how it comes out soon. :)
That’s awesome. Looking forward to seeing your finished result. Steemworld is pretty awesome. I’ve almost completely stopped using steemnow because steemworld offers several Moreno features, just don’t tell @penguinpablo haha
Thank you so much for all your observations and reasearch! I came into steemit not too long ago and I didn't know a thing about the platform. I always read up and look for people like you for help! I greatly appreciate it God bless !
You are welcome!
Who is sexually assaulting our rewards pool for fun?
case in point: @rewardpoolrape
Thank you so much for your valuable post, i appreciate your technology...
best of luck my dear, carry on your creativity........
Great, I hope you can share more posts like this
First of all good luck for your new achievements. wish you all the very best. wish you will help me buddy
Good post, hopefully your work is useful for everyone
nice post and best of luck
I'm very new on Stemit, but can already tell from your post that you are working on something big! I like your ideas very much and it would be great that the quality material somehow comes to the daylight. I will have an eye on your further developments and look forward to any new info ;)