Will HF20 improve curation and what if it were already active?

in utopian-io •  7 months ago

scooter400.jpg
Image: Fancycrave, CC0, modified

Repository

https://github.com/steemit/steem

Details

Steem as it is now is criticized for its low rewards for curators, making it financially unattractive to curate content manually. SP delegations to bots or dapps typically bring higher rewards without any work involved. Steem Hardfork 20 (HF20) is announced for a while now. Several details about it were revealed with @steemitblog's "Hardfork 20 (“Velocity”) development update" in December 2017 and there is first code in the GitHub project. One aspect of HF20 are changes to the Steem curation system. Among other things, HF20 is aiming to improve the situation for curators with the following changes:

  • Reducing the reverse auction time from 30 to 15 minutes
  • Leave unclaimed curation rewards from votes within the reverse auction time in the reward pool instead of giving them to the author.
  • Only partly related: Removing the dust vote threshold and downshifting each vote value by around 1.2 SP.

Outline

  • Curation 101
  • How to calculate the author and curator reward share?
  • Current HF19 author/curator share
  • What if the same votes were done with HF20?
  • Moving voting times towards a more realistic scenario
  • How much of the rewards remain unclaimed and go back into the pool?
  • Summary

Scope of the analysis and tools

In this post I'm looking into how the content rewards are distributed between authors and curators in the current HF19 and how they may look if the HF20 changes are active. The analysis is based on close to 4 weeks of top-level posts that fulfill the following criteria:

  • only top-level posts, no comments
  • the posts are payed out with a non-zero author reward
  • no posts that declined the payout (this was not necessarily needed, but having the author and curator rewards stored with the post simplified the verification of my calculations)
  • only posts with curation rewards enabled (to have meaningful author/curator share rations)
  • no posts with beneficiaries set (beneficiaries shift the ratio between author and curator rewards, because parts of the author rewards are payed out to the beneficiaries)
  • no posts with changed votes (changing a vote overwrites the initial rshare/weight table, making it hard to reconstruct the author/curator share retrospectively)
  • Filter out posts where the correct author/curator share for HF19 could not be reconstructed

The data from April 1st to April 26th 2018 contains 326421 posts that match those criteria. The data was fetched from the SteemSQL database maintained by @arcange. The data was queried and processed with python and plotted with matplotlib. The queries and all data processing scripts are on my Github.

Curation 101

Heavily simplified, the Steem curation system could be summarized as: You get rewarded if you are among the first that vote for a soon-to-be-popular contribution. Your share of the curation rewards depends on the value of the post before your vote in relation to the value of your vote and the sum of all votes to come after you. The number of votes before and after is irrelevant. Since being the first to vote on a post is a thing a human voter can never win against a bot, there is the "reverse auction time": The curation share you get for votes within the first 30 minutes is discounted. If you vote at the same time a post is created, 100% of your vote value will go to the authors, nothing to the curators. If you vote at 30 minutes or later, 75% of your vote value will go the author and 25% is distributed between all curators. For more details on the curation system and sample calculations, please see my previous post Effective Curation - Analysis of Curation Rewards in Theory.

How to calculate the author and curator reward share?

The value of each vote is internally calculated with rshares, a unit depending on the voter's SP, VP and vote percentage. This rshares value is assigned a weight as a relation of its value with respect to all previous vote rshares and the voting time if within the reverse auction time. See vote_evaluator::do_apply() for details about the calculations. The payout of the curators is done based on their vote weight in relation to the sum of all maximum vote weights (excl. the reverse auction deduction). See database::cashout_comment_helper() for details. With the list of votes and rshares on each post, these calculations can (mostly) be repeated, both with the HF19 as well as with the HF20 curation rules.

In this work, I repeat the calculations for the vote weight, the total_weight and the reverse auction system to determine the payout share of the content rewards between authors and curators. The total_payout_value and the curator_payout_value serve as a reference to confirm that my calculations for the current HF19 state are correct. In the same way, I can apply the HF20 changes to the algorithms and get results as if HF20 was active. One thing that is not available retrospectively is the reward fund and the amount of active rshares at payout time. However, the STEEM value of the author share is stored with the post in the author_rewards field as multiples of 0.001 STEEM. This makes it possible to reconstruct the total number of tokens distributed between author and curators, and allows me to "virtually" distribute the same amount of tokens based on the HF20 rules.

All query, processing and calculation scripts are on GitHub. I'm iterating over all votes of each post, repeating the actual and maximum weight calculations in order to get the author and curator share of the tokens. By modifying the calculation parameters the changed situation with HF20 can be simulated.

Changes from HF19 to HF20

The changes from HF19 to HF20 that I'm considering here are from the Steem 0.20.0 github project. Some of them are merged already, others are still in progress. There is no guarantee that HF20 will go live with exactly these variants.

  • Reduce reverse auction time: The reverse auction system was initially introduced to give humans a chance for curation rewards against bots who can vote much quicker. With more "short-form content", the 30 minutes are considered to be too long and the window is planned to be reduced to 15 minutes -> Github issue #1878

  • Lost curation rewards from the reverse auction should go back to rewards pool: In the current HF19 system, a self-vote at the time a post is created gives the author 100% of the vote value as author rewards plus a share of the curation rewards from all following votes within the reverse auction time. This is considered an unfair advantage for the author against other curators. For that reasons, the rewards that cannot be claimed by curators due to the reverse auction system should now go back into the reward pool instead of going to the author -> Github issue #1877.

  • Remove vote dust threshold: In the current HF19 system you need at least around 1.2 SP to vote. If your vote is worth less you cannot vote. You'll see an error message and your vote will not make it to the blockchain. With HF20 this system is planned to be changed in a way that everybody can vote as long as the account has enough bandwidth. This is motivated with user experience improvements. However, all votes are shifted down by around 1.2 SP in value. A vote the was "worth" 1.2 SP in the current system will be worth nothing now, but go to the blockchain anyway. Tiny votes below the value of a 1.2SP value will have zero value. This is intended to make it less attractive to cast lots of low value votes -> Github issue #1764.

Current HF19 author/curator reward ratios

With the current system of giving unclaimed curation rewards from votes within the reverse auction time to the author, the ration between author and curator rewards is shifted away from the target 75%/25% author/curator share towards higher percentages for the author.

Example: An author upvotes his own post at creation time to $10. Other voters add another $10 after the reverse auction time. The first $10 will fully go the author due to the reverse auction system. From the second $10, $7.5 go to the author and $2.5 to all curators. By summing those up, this makes $17.5 for the author and $2.5 for the curators. This corresponds to 87.5% for the author, 12.5% for the curators.

hf19_author_curator_shares.png

This graphs shows the curator shares of all posts that matched the selection criteria above. The histogram on the left contains one entry per post with the ratio of the curator share w.r.t. to the total payout. The mean curator share on this data set is 14.6%, the median share is 17.6%. The pie chart on the right sums up all tokens payed to authors and curators, giving a share between authors and curators by value. The curator share for this data set makes up 18.4% of the total payed out content rewards.

What if the same votes were done with HF20?

Now let's repeat the same calculations with the reduced reverse auction time, the unclaimed curation rewards going back to the pool and with the dust vote shift applied:

hf20_author_curator_shares.png

We can see that the exact same voting behavior with HF20 would give a mean curator share of 16.6% and a median share of 20.6% by number. Summing up the rewards now makes 20.9% for the curators. That's 2.5% more than with HF19!

However, assuming the exact same voting behavior with HF20 can be very misleading. The following graph shows how often and with how much value votes to these posts were done within the first hour after the post was created:

hf19_voting_times.png

The histogram on the left shows the number of votes in relation to the post age. The histogram on the right shows the same by value (rshares). In both cases we see clear peaks at "common" post age times. The first and strongest peak at 0 minutes are votes that are cast immediately when the post is created, typically by the author or an associated account. The second strongest peak is at 30 minutes, exactly at the end of the current HF19 reverse auction time. Both histograms show another peak at 15 minutes and a general increase in the number and value of votes between 20 and 25 minutes. The by-value histogram on the right has another peak at around 28 minutes. These voting signatures before or at 30 minutes are likely auto-votes. With a reduction of the reverse auction time from 30 to 15 minutes, those are very likely to change as well. Everybody who has an autovote set to 30 minutes now will certainly change this to 15 minutes with HF20, and earlier votes accordingly.

Moving voting times towards a more realistic scenario

For a more realistic HF20 scenario, I'm dividing now all vote times by 2. A vote that was at 30 mins before (at the end of the HF19 reverse auction time) is now at 15 mins (at the end of the HF20 reverse auction time). A vote that was at 20 mins with HF19 before is now calculated as if it were done at 10 minutes with HF20. I think this is a reasonable assumption, because hardly anybody will leave autovotes later than 15 minutes with HF20.

Now the picture looks different:

hf20v2_author_curator_shares.png

The mean curator share is now down to 15.5% and the median value to 19.1%. Looking at the distribution by value gives 19.7% of the post rewards to the curators. That's only slightly more than the current HF19 distribution.

How much of the rewards remain unclaimed and go back into the pool?

Unclaimed curation rewards in HF19 go to the author. This means that the full "post value" is payed out. With HF20, parts of the curation rewards remain unclaimed and are not payed out. The following pie chart shows the share of unclaimed rewards for the current HF19 and the HF20 case with the voting times divided by 2:

unclaimed_tokens.png

The HF19 chart on the left shows that all tokens are payed out. The share between author and curator for the HF19 case is the same as shown above. With HF20, parts of the curation share is unclaimed. Summing those up make around 6.6% of all tokens. You can see that this is exactly the share that goes to the authors in HF19. This basically means that in the HF20 scenario around 93.4% of the pending payout as shown in condenser or busy is actually payed out. The remaining part remains in the reward pool, increasing the overall payout and vote value for everybody in the same way.

Summary

The following table summarizes the numbers presented above:

ScenarioMean curator share per postMedian curator share per posttotal curator share by value
Current system (HF19)14.6%17.6%18.4%
HF20 with the same votes16.6%20.6%20.9%
HF20 with reduced voting times15.5%19.1%19.7%

With the current HF19 system, curators get between 14-18% of the post rewards depending on which metric is used. Calculating the curator share with the same posts and votes as if HF20 were active increases the curator share by around 2-3%. However, with the changes in the reverse auction time the voting behavior will change as well. A reasonable assumption is that all votes within the reverse auction time will be done a factor 2 earlier. This increases the curator share to only slightly more than 1% compared to the current state. So even though around 6% of the post rewards might remain unclaimed in HF20, the share of the curators from the total payout increases only slightly. Does it change anything for curation bots? To my understanding not a single thing. Curation bots will then wait 10-15 minutes before a vote instead of 20-30 minutes and probably even have more of an edge against humans, because nobody is online 24/7 to wait for the perfect post.

This analysis assumes that the amount of self-votes at post creation time remains the same as now. That's pretty lucrative for authors now. With HF20, authors would "sacrifice" 25% of their vote value with an insta-self-vote. A later self-vote would also give 75% to the author, but makes the author eligible for curation rewards as well. Analyzing the most profitable (self-)vote time with HF20 would easily be a post on its own and depends on all other voters as well. There might be behavioral changes involved with HF20 that increase the curator share. If however the author becomes it's own best curator, this obviously is not a gain to the current system.

Hooray HF20!

...won't change much for curators. Happy vote & delegation selling.

Proof of Work

https://github.com/crokkon/steem-analyses/tree/master/1805_hf20_curation

N.B.: I'm still hoping that I somehow messed up the calculations and the situation with HF20 will actually become much better for curators compared to now. If you find any issues, please let me know in the comments!

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

This is fantastic analysis, @crokkon. I agree with @abh12345; I'd hoped for more of an improvement for curators relative to authors, but that's no reflection on the rigour of your work.
The elephant in the room is the SBD price. Until that's back down around a dollar, we're comparing apples and oranges.

·

Thanks @mattclarke, also for the RS! With the "elephant in the room" I assume you mean that author rewards are payed out 50/50, while curation rewards are SP only? That actually comes on top. The numbers shown here as author and curator rewards are in STEEM as taken from the reward pool, before the split.

·

I think that this is the key point regarding curation vs vote selling.

Wouldn't it be simplest just to switch curation payouts to the same currency as post payout? i.e. 50/50 SP/SBD in most cases. Then we have apples and apples.

Waiting for the SBD price to change its behaviour may not happen - and there are also benefits of a SBD price above $1 for many people.

·
·

Without comprehending the reasoning behind the SBD pump, it's really hard to know how to approach it.
It's also incredibly difficult explaining the architecture of the place to noobs, when they can't understand why we'd have two currencies which are worth roughly the same.
Simplicity is it's own reward.

·
·

It will come in handy to say thanks for giving

·

What's wrong with more than $1 sbd price?

·
·

SBD's were designed to be a hedge, pegged to the USD; so if you had some gains on steem and wanted to lock them in, you could buy SBDs instead of having to cash out via an exchange or bank account.
You'd know that even if the price dropped, you'd still own that amount of US dollars worth of steem.
All of the mechanisms the team built to keep the price around a dollar were designed to push it up if its value crept down below a dollar.
Since they're designed to sit at 1 dollar, it didn't occur to anyone that somebody might buy them for $5 (who would do that?) and nobody built in a mechanism to pull the price back down if it went higher.
It's all very strange :)

·
·
·

Pesky free markets meeting planners plans.
They just never do as they are told!
😂

( I know nothing of this, ...just my 2 cents. lol)

·
·
·
·

Plan your plans and scheme your schemes.
They're flat ash under the wheel of progress.

·
·
·
·
·

Yeah, but socialism will work next time..........honest...
😂

·
·

The higher the SBD price, the higher the likelihood of self-voting for a 'cash' payout, over curation as an 'invested' payout.

Hi @crokkon

Thanks for looking into this one. I had hoped for a better 'result' for the curators following HF20 to be honest.

Chopping the vote times in half (as the reverse auction time is being halved) and factoring in the 'return to pool' of the early self-vote makes total sense, and sooo yeah, not too pleased with the numbers, but still an awesome piece of work on your part.

Cheers!

·

Thanks @abh12345, I was hoping for 'better' results as well. The insta-votes or possibly the lack thereof on HF20 may still improve the situation slightly with HF20, but I guess we'll only know when it's active.

·
·

Indeed. I trust you'll be crunching the numbers as soon as this is the case :)

·
·
·

I'm ready, only Steemit seems to hesitate with a release date :)

Thank you for coming up with this analysis. This is a good benchmark or prediction of the possible results once HF20 is deployed.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

Nice, thorough analysis.

Just skimming yr post following way too much whisky on a Wednesday.

Nothing intelligent to add into the mix ATM, instead I'll just state the obvious and say that it seems that the (potential next) HF will just be an easily avoidable redistributive tweak, albeit in the right direction.

Then again, you never know the effects of small changes.... data analysis can't predict that, as you're well aware of course!

Glad to see your getting rewarded for such great analysis!

Okay. First off, thank you for this. You answered the question I had regarding where the auto self-upvoting rewards actually go after HF 20 if they're no longer going to the author. They end up back in the rewards pool, rather than with the curators.

Which seems like a wasted opportunity, since so many people who self-upvote claim to be doing it to sweeten the curator pot when they've actually been sweetening their own author's pot with the self-upvote. It seems like the curator rewards would be significantly impacted if the auto self-upvoting actually folded into the curator rewards.

So, the status quo is more or less maintained. I guess we might as well keep HF 19 (aside from Velocity, maybe).

·

Thank you! To be clear, the author gets 75% of the vote value in both HF19 and HF20. In HF19, the remaining 25% are distributed between the author and the curators depending on the voting time, giving the author a part of that pot on top. In HF20, those 25% go to the curators or stay in the pool depending on the voting time.

Self-voting at post creation time in HF19 gives 100% to the author, while reducing the curation share for every other voter. Even if a self-vote is done after the reverse auction time when 25% of the vote value goes to the curators, there are still 75% of it going to the author. Arguing with "sweetening the curator pot" is questionable even in this case, but that's a topic on it's own ;)

If the reverse auction part of the curation share would remain with the curators, we'd have a fixed 75/25% distribution of the post rewards. This would actually be possible with changing a single line of code. Seems like this is not wanted by those in charge...

·
·

Self-voting at post creation time in HF19 gives 100% to the author, while reducing the curation share for every other voter. Even if a self-vote is done after the reverse auction time when 25% of the vote value goes to the curators, there are still 75% of it going to the author. Arguing with "sweetening the curator pot" is questionable even in this case, but that's a topic on it's own ;)

You were talking about a potential 1-2.5% increase in curator rewards between HF 19 and HF20 depending on when everyone voted. I was thinking that if that 6.6% that might go back to the rewards pool in your example actually ended up with the curators, it would be better to get the 25% distributed among them than just the 18-20%. Sweetening the pot might not be the right phrase, but potentially having the entire 25% to work with rather than only 18-something percent would be better than nothing.

As it is, I'm not sure why a 65-35, or 60-40 split hasn't been proposed, discussed, implemented, what have you. I understand the 50-50 split that was tried went away, not sure why, but guessing author's didn't like sharing so much with the curators. Which is fine. I would hope that the author is putting in more time to create a post than a curator is spending to read the post and then decide whether or not to upvote it. Or for that matter, simply auto voting, so I'm in agreement that the majority of the rewards should go with the author. There's still a range between 75-25 and 50-50 that could be explored, but as you say, that's a whole different topic.

If the reverse auction part of the curation share would remain with the curators, we'd have a fixed 75/25% distribution of the post rewards. This would actually be possible with changing a single line of code. Seems like this is not wanted by those in charge...

So, this begs the question, "Why?"

Hey @crokkon

We're already looking forward to your next contribution!

Contributing on Utopian

Learn how to contribute on our website or by watching this tutorial on Youtube.

Utopian Witness!

Vote for Utopian Witness! We are made of developers, system administrators, entrepreneurs, artists, content creators, thinkers. We embrace every nationality, mindset and belief.

Want to chat? Join us on Discord https://discord.gg/h52nFrV

It should be more profitable to curate than to just upvote your own comments.

A bit of a tangent here, but just today I've come across a reason why the current dust limit could be a good thing. I'm guessing this is a bot, because I'm pretty sure you can't manually post every 5 minutes, but @adsofficial18 seems to be earning something from it. Most of the posts are only getting the odd cent from @cheetah though.
This also gives a case for limiting daily blog posts. Do you think HF20 would have any effect on discouraging this sort of behaviour?

·

Wow, please report that account to @steemcleaners, that's clearly automated spam: http://steemcleaners.com/reports/new
The dust limit has no influence here. The votes are big enough that they easily exceed the HF19 and the HF20 dust vote limits. What is there already is the bandwidth limit. If you look up that account on steemd now, you'll see that it actually ran out of bandwidth right now and is currently not able to post:
adsofficial.png
However, as you can see in the authors posting history, it doesn't really prevent that kind of content...

·
·

I reported it. I forgot about bandwidth limit. I'm surprised they managed the amount of posts they did before it ran out!

As always: brilliant work! Unfortunately the result is disappointing for us small fish. This again helps to better understand where we stand and where we might head to. Thanks a lot!

Offtopic:

changing a vote overwrites the initial rshare/weight table, making it hard to reconstruct the author/curator share retrospectively

Does changing a vote % set its weight to zero in reality? I see in the rshares table it is zero'ed, eg. at steemd.com. Is it just UI simplification or true?

·
·
·

shhhh :P

·
·
·

MY GAHD!

·
·
·
·

Here come the gentlebot fans!

·
·
·

No you :)

·
·
·
·

@gentlebot likes peace and quiet 😂

great analysis, but fucking dammit.

Very well done, informative post, @crokkon. Thanks for the info. Is there a projected date of release for the HF20? To be honest, I have doubts about improvements until I actually see them, given I've never seen more than 1% paid for curation.

·

I'm sorry, I'm not aware of a projected release date. There is a lot more than 1% curation rewards in total already now, but as an individual you really have to know the game to earn a share of it also with low SP.

·
·

Yeah, I pulled my hair out trying to figure it out when I started a few months ago, but never made a cent until I invested in SP. It seems the main way to make anything is to invest thousands or more. Thanks for the info...

You've raised some good points, however this will also increase bots takings. What are your thoughts on bots on the platform and how will it impact new users?

·

I don't think Steem without bots will happen in the near future. I could image that curation bots might profit from HF20 and I don't think that it will make a big difference for bid-bots. For new users it probably doesn't make a difference at all if we're at HF19 or HF20 curation.

WARNING - The message you received from @wande is a CONFIRMED SCAM!
DO NOT FOLLOW any instruction and DO NOT CLICK on any link in the comment!

For more information, read this post:
https://steemit.com/steemit/@arcange/phishing-site-reported-steemautobot-dot-ml

If you find my work to protect you and the community valuable, please consider to upvote this warning or to vote for my witness.

Loading...

Thanks for this detailed post. I hope it will make Steemit better.

To the question in your title, my Magic 8-Ball says:

It is certain

Hi! I'm a bot, and this answer was posted automatically. Check this post out for more information.