Steem Analytics: Studies in Mass Adoption - Retention

in #utopian-io5 years ago (edited)

SteemAnalyticsSolo.png

At the height of the bull market in January 2018 there were 130k authors active on Steem. Fast forward one year to the bear market conditions of January 2019 and this number had fallen to only 40k.

There is a strong correlation between the price of Steem and the number of active authors. This could be due to a general rise and fall in interest for all things crypto when the markets wax and wane but a more likely cause is the rise in dollar rewards available as the Steem price increases.

When considering the path to mass adoption, retaining existing users is as important as attracting new ones. An analysis of the relationship between earnings levels and retention seems like a good place to start.


1. Retention by earnings level: January 2019 to February 2019

The basic questions I am investigating here are:

  • Is there a relationship between retention levels and earnings levels?
  • At what level of earnings do we see a difference in retention?

This first study considers all accounts that posted or commented in the first two weeks of January 2019. The accounts are aggregated into bands of earnings and the number of accounts within each band is determined. Retention rates are then calculated based on whether each account made a post or comment in the first two weeks of February 2019, i.e. after an interval of one month.

For the purposes of all this analysis, earnings are defined as author rewards excluding those rewards purchased from vote bots. No allowance is made for curation or beneficiary rewards since these earnings sources are not directly related to posting. In any case these elements are more of a concern for larger SP holders and this study is focused on smaller accounts. The earnings are total rewards over the two-week period and are quoted in Steem (not STU). Note that the earnings levels are based on the first two weeks of January and take no account of earnings in February or earnings over the intermediate period.

The overall retention rate using this definition was 53%, i.e. 53% of authors posting/commenting in the first two weeks of January also posted in the first two weeks of February, whilst 47% did not. How does this break down by earnings level?

On the chart:

  • The blue bars show the number of accounts at each earnings level.
  • The orange diamonds show the percentage of the January accounts that also posted in the first two weeks of February, i.e. the retention level.

RetJan1-1420191mthAll.png

So there is good news and bad news here:

  • Good news: Retention is actually quite sticky for authors earning one Steem or more. We have retention of over 70% at 1 Steem, increasing towards 90%-100% by the time we reach 10 Steem earned over the two weeks.
  • Bad news: Retention for the first earnings band, i.e. less than 1 Steem, is 40%. And 70% of authors / commenters fall into this first band.

The overall retention level of 53% is thus heavily influenced by this first earnings bracket. So let's look at the low earnings results at a more granular level.

Here is the same study but with smaller buckets and lower earnings levels:

  • Smaller buckets: 0.1 Steem rather than 1 Steem.
  • Lower earnings levels: from 0 Steem - 5 Steem rather than 0 Steem - 50 Steem.

RetJan1-1420191mthLow.png

A similar pattern to the first chart and, again, there is good news and bad news:

  • Good news: Retention levels improve at very low earnings levels - we can see an impact from as low as 0.1 Steem.
  • Bad news: Retention for the first earnings band, i.e. less than 0.1 Steem, is close to 30%. And 53% of authors / commenters (across all earnings) fall into this first band.

2. Retention by earnings level: Looking at the full month of January

At this point I wanted to do a quick check on whether the 14 day period was having an effect. Is it too short? For this next study I lengthened the earnings period to the full month of January and the retention period to the full month of March, retaining a gap of one month between the two periods.

The two charts below compare the retention rates from the 14 day period (in orange) to the full month study (in red):

Earnings: 0 Steem - 50 Steem, 1 Steem buckets

RetJan1-3120191mthAll.png

Low earnings: 0 Steem - 5 Steem, 0.1 Steem buckets

RetJan1-3120191mthLow.png

The patterns are really very similar, so that's all good. Fourteen days looks like a reasonable length of time for the study.


3. Retention over the following months

Finally a look at how the retention levels progress over a longer period. Here we have:

  • Orange diamonds - retention from January 1-14 to February 1-14, i.e. 1 month gap (as in section 1).
  • Red diamonds - retention from January 1-14 to March 1-14, i.e. 2 month gap.
  • Purple diamonds - retention from January 1-14 to April 1-14, i.e. 3 month gap.

I have not re-evaluated the earnings distribution for each month, so the classification of each account into an earnings bracket is maintained from January. It is likely that the accounts that remain pass into different earnings bands for each month. But I think that the initial distribution is a suitable proxy for authors at different earnings levels and sufficient for the conclusions drawn.

Earnings: 0 Steem - 50 Steem, 1 Steem buckets

RetJan1-1420193mthAll.png

Low earnings: 0 Steem - 5 Steem, 0.1 Steem buckets

RetJan1-1420193mthLow.png

In both charts retention levels fall by around 5% for each additional month. Retention is pretty sticky in subsequent months in comparison to the first month of the study.

I have checked that studies starting in February and March (rather than January) show a similar pattern of lower retention in the first month (so it is not just a case of March / April retention being stronger). This may suggest that the population splits into:

  • A central core of accounts that post regularly, with long-term retention rates that are actually quite high.
  • A second group of accounts, perhaps new accounts or returning accounts or occasional posters, for which retention rates are low, and many of which stop posting after the first month.

Further study would be required to confirm this.


4. Conclusions

The charts show a clear pattern:

  • Retention increases with earnings level.
  • Retention increases markedly between zero earnings and low earnings levels.
  • Retention rates are surprisingly high (given my initial low expectations) starting from only a few Steem of earnings (e.g. 80% at 2 Steem).

Based on these studies it could be theorised that accounts are more likely to stop posting if they only earn small amounts of Steem and that they are much more likely to stop posting if they earn no Steem at all. This does sound pretty logical.

However, correlation does not imply causation. In fact it is possible that the cause and effect are completely reversed: New accounts arrive on Steem but quickly decide it is not for them. They leave the blockchain after making only a few posts or comments and earn no Steem in the process. Or to put it more succinctly: Accounts earn no Steem because they leave, rather than they leave after earning no Steem.

It is also possible that accounts do not leave at all but simply stop posting and commenting. Users could remain as consumers of content only or they could be busy playing NextColony or MagicDice.

Further study would be required to provide evidence for each of these possible conclusions.


5. Thoughts:

I'll admit that I have not concretely demonstrated a causal link between earnings and retention. More work is required. But it wouldn't surprise me if this turned out to be the case. Steem/Steemit adverts have traditionally highlighted the rewards as a main selling point of the blockchain. Users may become disillusioned if they do not obtain the earnings they were anticipating.

So is the answer to channel a greater share of rewards towards promising new accounts and low earners? It might help to improve retention rates. But it feels like a slow and tortuous path to mass adoption, rather than an organic solution.

The reward pool paid out approximately 1.3m Steem (or 450k STU) in author rewards in January 2019. This amount is significantly reduced once purchased votes are removed. The top 10% of authors gained 90% of the rewards; a pattern which is unlikely to change given the nature of stake-based reward voting. This leaves a very small pot with which to retain smaller accounts and keep new users happy.

It seems unlikely that the reward pool mechanism will drive us to mass adoption.

Instead I think that the answer is to look at other social networks and content creation sites, the facebooks, instagrams, and youtubes of this world, and to understand their paths to mass adoption without the use of crypto rewards. There is an opportunity with the arrival of Communities to shift the emphasis away from the promise of rewards towards a rounder offering, treating Steem/SMTs as one element rather than the main focus.

Early thoughts in any case. I have a few more studies planned that relate to mass adoption. Assuming I can get them to work, I'll add more thoughts as I go.

Thanks for reading!


Repository:

https://github.com/steemit/steem

This analysis is of data from the Steem blockchain which is an open source project.


Tools and scripts:

gears_blockops_green.jpg

I used the block.ops analysis system to produce this study. Block.ops is an open-source analysis tool designed for heavy-duty analyses of the Steem blockchain data.

You can find the repository for block.ops here:
https://github.com/miniature-tiger/block.ops

The analysis used all the Steem blocks from the period analysed.

The study can be recreated by (once I upload the new analyses to github!):

  • Loading the data for the relevant time period into block.ops.
  • Using the earningsdistribution command from the command line, for example:
    $ node blockOps earningsdistribution "2019-04-01" "2019-04-15"

Thanks for reading!

Sort:  
Loading...

I fully agree on not making rewards the main selling point of Steem. It clearly isn't working because what the median user is earning is pennies if not even that. I commend your post. You have made it clearer than ever to me that having SMTs or Steem-Engine tokens as the main rewarding mechanism is key. Make STEEM the main trading pair of all the tokens and Steem Power the infrastructure token. Let rewarding users be up to each app. The free market will sort out the resulting token ecosystem. Reserve STEEM and Steem Power rewards for Steem Power holders and witnesses alone. Proof-of-Brain using STEEM, SBD and SP has clearly failed. It's better to have an evolutionary approach to the problem by having larger number of token economies. That's the decentralized way as per the original vision and the white paper.

I think it will depend on what "Communities" offers.

But if we can reach out to existing groups / teams / organisations / institutions with the offer that they can have:

  • Their own decentralised space which only their members can post to.
  • Their own fee-less cryptocurrency (SMT) with their own rules for how this is distributed and access to an exchange.
  • Control over their membership to avoid spam / abuse / trolling / downvotes from other groups.
  • Modular rules on how posts are displayed allowing different ways to vote and promote the community favourite posts (there should be fewer sybil issues with closed communities).
  • Magazine-style formats / newsletter / pinned posts etc to attract different types of community (a UI issue).
  • Ownership of advertising revenues with modular rules for splitting between the community / individual authors / funding the SMT.
  • The potential to integrate / link to / wrap around the blockchain games that Steem is starting to lead on.
  • Access to the existing central Steem space (i.e. the current offering).
  • Fast 3-second posting.

It's a really strong offering. And one that can be advertised through focusing on the community aspects and the technology rather than the rewards. Existing communities should hopefully be stickier than individuals, particularly if the offer is a good one.

For Steem I prefer the "curation slider at the discretion of the voter" approach:
https://steemit.com/utopian-io/@miniature-tiger/curation-earnings-analysis-with-stats-and-thoughts-on-50-50-curation-system-proposal-1556541324925

  • Flexible for different dApps and users preferences.
  • Retains the current abilities for SP holders to distribute Steem inflation - important for users who want to continue with the current Steem ethos as well as dApps like DTube and their existing membership and concepts like SBI.
  • Gives SP holders control over the inflation generated by their holdings without the need to use complex workarounds (vote-bots, circular voting, black-market vote-swapping etc).
  • Removes the funding for vote-bots.

But I also appreciate the need to move Steem back towards the top of the crypto listings so it is taken more seriously. I hope to look at this if I can get the data together.

Im fully on board with this analysis. I recently wrote a post about how we are crying out for engagement on the platform, we don't have a front end that allows us to deep dive into content into author profiles or a culture of engaging without rewards

We dont engage on old posts, we don't engage after the first 2 or 3 comments, we don't post just for the sake of sharing an idea. Rewards have too long been the focus and its only got us this far, now its time to adapt the tool and the messaging to take us to the next step.

There is a big problem of way too many content creators and very few content consumers. I wish they had kept the little ticker that told you how many views a blog got.

From memory it didn't produce robust numbers, which is why they removed it. But we should definitely have something similar. Ideally distinct viewers / how long they spent on the page etc.

I'm hoping that "communities" will help with engagement. People engage naturally, without the need for rewards, given the right environment.

That said, I think that rewards are still important but as one element of a wider offering and not as the main focus for recruiting people to the site.

I have been feeling a bit confused on what is going on with steem lately since there is so much development going on, so much new stuff being built but it does not seem to translate in more users or better community. it felt like just the opposite actually.

so to see these numbers is encouraging and uplifting. Now we just need to onboard a bit more. It would be good to see around 100k active accounts here even if that would mean splitting the pot and earning less

I think that the retention levels above 1 Steem are really positive and also the idea that there is a central core of accounts with long-term retention rates that are actually quite high. On the other hand the retention rates for the lowest earnings bracket are really not good.

It would be good to see around 100k active accounts here even if that would mean splitting the pot and earning less.

It would be great to get back to those numbers!

The dilution of rewards as the number of users rises illustrates one problem with obtaining mass adoption. If retention is linked to earnings then we become limited by the reward pool as user numbers grow, unless the Steem price rises as a result of new users.

I think (somewhat contradictorily) that we should:

  • Change the focus away from rewards to a wider offering, hopefully based around communities.
  • Add other revenue streams for users (such as sharing of advertising revenues) to provide greater incentive / competition with other content creation sites.

It was an interesting study to read, one thing about retention that no one really wants to talk about or even discuss is the effect downvotes or flags have on accounts.

Most of the downvotes and flags go to these smaller earners, (accounts that earn less than 50 steem a week), as has been admitted by several flag/groups because they can do nothing against the large stake holders.

So perhaps in the next study include downvotes and removal of rewards in the evaluation to see if it has an effect on retention. This should be done before the implementation of the EIP and downvote pool takes effect in my opinion so that people can see if flags do contribute to the retention problem.

Unfortunately it wouldn't be a particularly robust study. Currently the number of downvotes is relatively tiny due to the cost. The population downvoted is dominated by those targeted by steemcleaners etc.

The EIP proposal would see:
(a) Downvotes become more of a normal occurrence - so there would be some change in perception and sentiment towards being downvoted.
(b) A wider population affected with potentially very different characteristics to the current downvoted population.

It probably wouldn't be appropriate to extrapolate across from pre-EIP retention impact from downvoting to post-EIP.

That said, I do share the concern that the EIP-proposed downvote pool could be significantly abused. An abusive whale could cause significant toxicity across the platform if they decided to target hundreds or thousands of low earning accounts. It would be possible for a competitor crypto to buy Steem specifically for this purpose.

I would be interested to hear if there is any proposed solution to such an occurrence. Probably users would organise to counter but this does assume a fair degree of cooperation and altruism.

It may be better to wait for Communities to brought in first, including SMTs. Ideally each community would have the potential to protect its members against downvotes, at least on SMT rewards rather than Steem rewards. There are a number of ways this could be implemented.

It would be possible for a competitor crypto to buy Steem specifically for this purpose.

Indeed. This would be pretty low of a competitor or a disgruntled whale, but it's not out of the question at all.

Hi @miniature-tiger!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

Hey, @miniature-tiger!

Thanks for contributing on Utopian.
Congratulations! Your contribution was Staff Picked to receive a maximum vote for the analysis category on Utopian for being of significant value to the project and the open source community.

We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

Congratulations @miniature-tiger! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

You got more than 1250 replies. Your next target is to reach 1500 replies.

You can view your badges on your Steem Board and compare to others on the Steem Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Vote for @Steemitboard as a witness to get one more award and increased upvotes!

Coin Marketplace

STEEM 0.19
TRX 0.15
JST 0.029
BTC 63968.82
ETH 2633.99
USDT 1.00
SBD 2.84