Block.ops - An Analysis Tool - Tags, Links, Transfers, Delegations and some New Analyses!
Three block operations were tackled this week as I work down the list; each adds new analysis capabilities to the database.
A host of new analysis functions were constructed based on the additional data.
Block operations added
- Transfers.
- Delegations.
- Tags and links to allow community analyses (update to "comment" block operation)
Development of analyses
- New Utopian analysis! A breakdown of Utopian-io voting by type (steemstem, utopian contribution, mspwaves, comment etc), by contribution type, and by author.
- Bid-bot profitability analysis: Links transfers (in transfers collection) to bid-bot votes (in comments collection). Enables profitability of vote at vote purchase and vote at post payout to be calculated.
- Transfer summary: Analysis which summarises transfers made to and from account, sorted by party and/or time.
- Delegation summary: Analysis which summarises delegations made to and from account in specific time/block range.
For the last year I have carried out a monthly analysis of the Steem blockchain activity by application (i.e. by the different websites and applications through which you can post to the Steem blockchain - often termed dApps).
My aim is now to build a tool that can automate such complex analyses of Steem data, providing both historic time series and rapidly updatable real-time results. In addition to the dApp analysis there are many other projects for which such a system could be useful.
This tool is Block.ops.
You can read all about block.ops (including the project aims, technology stack, roadmap and how to contribute) in the introductory post here:
https://steemit.com/utopian-io/@miniature-tiger/block-ops-an-analysis-tool-1537300276791
Repository
https://github.com/miniature-tiger/block.ops
New Features
Tags and links and Utopian analysis
Post / comment tags can allow analyses to be carried out for different Steem communities. Comment links can provide useful additional details on the posts (for example Utopian moderator comments contain specific links). These elements are now added as arrays into the "comments" documents collection of Mongo.
Using this new information I created an analysis for the Utopian community. The Utopian analysis aggregates Utopian-io voting percent by:
- Type of vote (contribution, moderator comment, steemstem, mspwaves, steemmakers, other)
- Contribution type (analysis, translation etc)
- Author
Below are some example charts from the analysis, showing the overall breakdown of Utopian-io voting and a further separation of contributions by type (1000 on the y-axis is 1000% i.e. a day's full 10 votes).
The code changes are here:
https://github.com/miniature-tiger/block.ops/commit/84996ad18518e4f7eb61a761f5d9e53bc64c2efc
Transfer operations and summary of transfers
Transfer operations cover movement of SBD and liquid Steem between Steem user accounts. These operations are now captured by the data loading process for each block.
Transfers are stored within a separate "transfers" collection. Succesful data capture is logged in the blocksProcessed collection.
I also created an analysis which summarises transfers made to and from a chosen account, sorted by party and/or time. This can serve as a starting point for more complex analyses involving transfers.
The code changes are here:
https://github.com/miniature-tiger/block.ops/commit/bd71beefb24468f78d8deafdd4ab3e5d59d070b7
Bid-bot profitability analysis
Bid-bots allow Steem users to purchase upvotes based on an auction system, typically with maximum and available minimum returns.
An analysis of bid-bot profitability (for users) requires linking the transfer payments (in the transfers collection of the MongoDB) to the bid-bot votes (found in the comments collection using the author/permlink in the transfer memo and filtered out from other votes) and the fx rates (in the prices collection).
The analysis allows the ratio of the vote value on the post (after curation) to the transfer payment amount to be calculated.
Separate fx rates are applied to captures the vote value at vote purchase and the vote value at post payout to illustrate differences in that arise due to price movements and reward pool movements. The following chart shows a day of vote purchasing which was followed by falls in the value of rshares (reward pool emptying) leading to unprofitable purchases:
The code changes are here:
https://github.com/miniature-tiger/block.ops/commit/e0c22c3f930a98a30d00698fc4e3ca5a234991b9
Delegation operations and summary of delegations
Delegation operations cover "loans" of Steem Power between Steem user accounts. These operations are now captured by the data loading process for each block.
Delegations are stored within a separate "delegation" collection. Succesful data capture is logged in the blocksProcessed collection.
I also created an analysis which summarises delegations made to and from an account in a specific time/block range. This can serve as a starting point for more complex analyses involving delegations.
The code changes are here:
https://github.com/miniature-tiger/block.ops/commit/0e4949efd17e005c78450b5ecf11cf401d01313e
Progress!
GitHub Account
My account on github is here:
https://github.com/miniature-tiger
Thanks for the contribution, @miniature-tiger! Some very cool features, especially the overall breakdown of Utopian-io voting is of course interesting to me, haha. Not very surprising to see the translations category using such a huge part of the voting power. Hopefully with the new bot it will also be a flat 1000 on the y-axis each day!
It's definitely a very interesting project and I'm looking forward to seeing what other features you add in the future. Although I guess this means you won't be working on Archipelago any time soon? :(
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Unlike other categories, translations have had the highest number of active people. So if there were as many active people in other categories, I would say the distribution would be different.
This is true. Although I think that there are actually lots of active development contributors, but mostly they tend to contribute infrequently due to the length of time required to add new features.
I'd add that it's not a full analysis, just an illustration of the functionality, so it's not something to draw firm conclusions from.
Thanks Amos!
Yes, the Utopian analysis was something I started looking into based on the discussion on discord of the new voting bot and scaling. I was trying to see what the scaling might look like and whether any problems could arise, particularly with the timing. The analysis needs a bit more work on that front but the breakdown itself is interesting.
I had hoped to get back into Archipelago more heavily this week! I've done some work on it off github already. But I need to add a few more operations to block.ops in preparation for loading in a big chunk of data for some analyses I'm looking into. So hopefully within the next week or so!
Thanks for the review!
Thank you for your review, @amosbastian!
So far this week you've reviewed 3 contributions. Keep up the good work!
Hi! This looks pretty cool! Do you plan on running a live demo?
Thanks @stoodkev!
Currently it's all local on my laptop. I'm still trying to work out whether there's a good way to host something, given the potential size of the DB and the volume of calls and data transfer required to fill the DB. The node/Mongo/hosting side of things is all new to me so it's a bit of a journey of discovery!
Got it ! Looking forward to see where it goes! Good luck on journey then ;)
Very impressive work here @miniature-tiger!
If only I could find some time to dive in :D
As a SteemSTEM manager, thanks for this. This consists in very useful pieces of information. :)
Congratulations! Your post has been selected as a daily Steemit truffle! It is listed on rank 10 of all contributions awarded today. You can find the TOP DAILY TRUFFLE PICKS HERE.
I upvoted your contribution because to my mind your post is at least 18 SBD worth and should receive 79 votes. It's now up to the lovely Steemit community to make this come true.
I am
TrufflePig
, an Artificial Intelligence Bot that helps minnows and content curators using Machine Learning. If you are curious how I select content, you can find an explanation here!Have a nice day and sincerely yours,
TrufflePig
I upvoted your post.
Keep steeming for a better tomorrow.
@Acknowledgement - God Bless
Posted using https://Steeming.com condenser site.
Hi @miniature-tiger!
Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server