When do whales upvote?

in #programming8 years ago

Ever wondered when whales are upvoting? I was wondering the same thing. I thought: perhaps some times are better to post than others.

So I wrote a program in Python to plot the upvote times for July 2016 for a random selection of whales. Here are the results:

The founders

@ned and @dan appear very limited in the time they have to upvote. They're both similar in the number of votes they give.

If you get upvoted by @dan or @ned, you've done well. Not only because the founders like your work, but also because they aren't the most active upvoters.

Whale bots

The accounts @steemed, @itsascam, and @steemroller are all bots run by the same whale. These graphs illustrate bot behaviour: the distribution of upvotes is more even, and of course, they never seem to sleep.

All these graphs really tell us are when the authors in the bot's author list have published an article.

Others

Here are a bunch of other whales. Maybe some of them are bots, let me know if that's the case. It's fun making assumptions about the whales, like when they sleep.

berniesanders

Between 07:00 UTC and 14:00 UTC @berniesanders is mostly inactive. At the other times though he's a generous upvoter and quite consistent.

blocktrades

@blocktrades is very likely asleep between 06:00 UTC and 14:00 UTC. After this time there's a bit of voting, but not a great deal.

complexring

@complexring is either or bot or someone who has trouble sleeping. While upvoting between 06:00 and 12:00 UTC is significantly lower, there are still some votes taking place. @complexring is much more active in upvoting than @berniesanders.

nextgencrypto

@nextgencrypto does most upvoting during the weekend with some activity in the evening (their local time), assuming the gaps indicate sleep.

pharesim

@pharesim's behaviour resembles @complexring. Perhaps a bot. There are lower periods of activity which could indicate sleep. Friday seems to be the most stable day where there's a clear pickup in voting after 12:00 UTC.

rainmain

It's clear that @rainman is sleeping between around 22:00 UTC and 04:00 UTC. Another low upvoting whale, @rainman is pretty consistent during the day.

smooth

@smooth's behaviour is similar to rainmain in that it's consistent during voting times, but there's no clear downtime, which raises the suspicion of bot use. @smooth is a slightly more active upvoter during the weekend.

tombstone

@tombstone's graph looks empty, but that's caused by the bizarre outlier on Friday at 10:00 UTC. I have no idea what @tombstone was doing then, but it was an upvote frenzy.

The rest of the time @tombstone is quite consistent with voting, with slightly more activity between 02:00 UTC and 08:00 UTC.

When should I post?

I've intentionally not drawn too many conclusion from these graphs. Perhaps the most useful information is knowing which whales upvote the most and when, so you can time your publication correctly.

Do you want to increase the chance that @berniesanders will upvote you? Don't post at 07:00 UTC because for the next 8 hours he's unlikely to see it; better to post around 03:00 UTC on a Wednesday.

With so many whales you're more or less covered whenever you post. What you're posting is far more important than when you post it.

Show me the code

Feel free to use and adapt the code below as you like. Any bugs, please let me know in the comments. Read @furion's post for more information on parsing the blockchain.

The program requires Python 3 and the following libraries, which you can install with pip:

  • matplotlib
  • numpy
  • pandas
  • seaborn
  • steem

The program runs on the command-line. Provide a list of whale username and optionally, either the start and end blocks to analyze or start and end dates. The latter will convert the dates into the appropriate block numbers (this isn't fast - there's probably a better way to do it).

python3 vote_dist.py dan ned smooth berniesanders -f 2016-07-01 -t 2016-07-31

Enjoy!

import argparse
import datetime
import math
from collections import defaultdict

import pandas as pd
import seaborn as sns
import numpy as np
from steemapi.steemnoderpc import SteemNodeRPC


def get_stats(rpc, users, beg_block, end_block):
    print("Getting stats for {} from block {} to block {}".format(
          users, beg_block, end_block))

    stats = defaultdict(list)
    current_block = beg_block
    while current_block < end_block:
        block = rpc.get_block(current_block)

        if "transactions" not in block:
            continue

        for tx in block["transactions"]:
            for op in tx["operations"]:
                op_type = op[0]
                op_data = op[1]

                timestamp = pd.to_datetime(tx['expiration'])

                if op_type == "vote":
                    author = op_data['author']
                    voter = op_data['voter']
                    weight = op_data['weight']
                    if voter in users and weight > 0:
                        stats[voter].append((timestamp, weight))
        current_block += 1
    return stats


def get_block_num_for_date(rpc, date):
    """
    Gets the first block number for the given date.

    This is anything but fast. There's probably a better way, but this was the
    first thing that came to mind.
    """
    print("Getting block num for {}".format(date))

    block = rpc.get_block(1)
    bc_date = pd.to_datetime(block['timestamp'])

    SECONDS_IN_DAY = 24 * 60 * 60
    NUM_BLOCKS_PER_DAY = math.floor(SECONDS_IN_DAY / 3)

    # Estimate the block number
    block_num = (date - bc_date).days * NUM_BLOCKS_PER_DAY

    # Use estimation to find the actual block number
    best_block_num = block_num
    best_block_diff = None
    while True:
        block = rpc.get_block(block_num)
        block_date = pd.to_datetime(block['timestamp'])
        diff = (date - block_date).total_seconds()

        if best_block_diff:
            if abs(diff) > abs(best_block_diff):
                break

        best_block_num = block_num
        best_block_diff = diff
        if diff > 0:
            block_num += 1
        elif diff < 0:
            block_num -= 1
        else:
            break
    return best_block_num


def create_plot(user, votes):
    print("Creating plot for {}".format(user))

    df = pd.DataFrame.from_records(votes, columns=['time', 'weight'])
    df['day'] = df['time'].dt.weekday_name
    df['hour'] = df['time'].dt.hour

    col_order = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday",
                 "Saturday", "Sunday"]

    plot = sns.FacetGrid(df, col="day", col_order=col_order, col_wrap=3)
    plot = plot.map(sns.plt.hist, "hour", bins=np.arange(0, 23),
                    color="c").set_titles("{col_name}")
    plot.set(xticks=np.arange(23, step=2), xlim=(0, 23))
    plot.set_axis_labels('Hour', 'Upvotes')
    plot.fig.suptitle(user, size=16)
    plot.fig.subplots_adjust(top=.9)

    return plot


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('usernames', nargs='*',
                        help='Usernames to show statistics for')
    parser.add_argument('-s', '--server', default='ws://localhost:8090',
                        dest='server',
                        help='Address of the steem JSON-RPC server')
    block_group = parser.add_argument_group('blocks')
    block_group.add_argument('-b', '--begin-block', default=1, type=int,
                        dest='beg_block',
                        help='The block to begin on.')
    block_group.add_argument('-e', '--end-block', default=None, type=int,
                        dest='end_block',
                        help='The block to end on. Default is last_irreversible_block_num.')
    date_group = parser.add_argument_group('dates')
    date_group.add_argument('-f', '--from-date', type=str, dest='from_date',
                        help='The date to end on.')
    date_group.add_argument('-t', '--to-date', type=str, dest='to_date',
                        help='The date to end on.')
    args = parser.parse_args()

    # Connect to steem rpc server
    rpc = SteemNodeRPC(args.server, "", "")

    # Get the block numbers
    if args.from_date and args.to_date:
        args.from_date = pd.to_datetime(args.from_date)
        args.to_date = pd.to_datetime(args.to_date)
        args.beg_block = get_block_num_for_date(rpc, args.from_date)
        args.end_block = get_block_num_for_date(rpc, args.to_date + datetime.timedelta(days=1)) - 1
    else:
        props = rpc.get_dynamic_global_properties()
        if args.end_block:
            if args.end_block > props['last_irreversible_block_num']:
                args.end_block = props['last_irreversible_block_num']
        else:
            args.end_block = props['last_irreversible_block_num']

    # Validate block numbers
    if args.beg_block < 0:
        print("begin-block must be greater than 0")
        sys.exit(1)

    if args.end_block < args.beg_block:
        print("end-block must be greater than beg-block")
        sys.exit(1)

    # Get the stats
    stats = get_stats(rpc, args.usernames, args.beg_block, args.end_block)
    for user, votes in stats.items():
        plot = create_plot(user, votes)
        plot.savefig('{}.png'.format(user))

    if len(stats) == 0:
        print("Nothing found for the given parameters")

Like my post? Don't forget to follow me!

Sort:  

Oh, man, @bitcalm. Your posts never cease to amaze me. I don't know if it's just because you're posting topics that I can relate to or are interested in, but you get all my upvotes. I've been meaning to code something like this for a while now, but opted to experiment posting on different times with different kinds of content. This post is very insightful and is shaping up to be one of my most favorite posts in Steemit so far. I guess an improvement to this is to factor in the tags or types of post they upvote along with their voting time. This really has a potential to analyze whale voting behavior.

By the way, @complexring is 100% human, and an accomplished mathematician. As to why his voting behavior looks like that, I'm not sure. But, he's a cool guy who has an inclination to fiction posts.

@bitcalm nailed it on this post. I would love to see this trend month over month!

Right you are!

Good to know @complexring is human. Hope he figure out the sleep problem ;) Actually, it'd be cool to have a mathematician weigh in on this. Can you pass the link on?

👍nice showing graphics to know @bitcalm

100% Proof Pokemon Go End Game Connection to DARPA CIA Google NSA Skynet Illuminati Fema Spy Grid

https://steemit.com/cia/@cryptocurrency1/100-proof-pokemon-go-end-game-connection-to-darpa-cia-google-nsa-skynet-illuminati-fema-spy-grid

I was giggling as I read, wondering if any of the possibly-a-bot-but-really-a-human whales were rethinking their sleep schedules.

The more you know :)

Sure thing! I hope he replies on this thread with his thoughts

What @jedau said. I've been posting whenever and wherever I can since I've got to wait about 2 weeks to get internet at my house, so parking lot hopping with free wifi has been my life the last few days since I moved. Yay Me! Anyway - I saw your list of bots and I'm thinking, "Well, I'm a frikken idiot" because I've been seeking those usernames out and trying to engage...maybe get on their radar...and I might as well be talking to my stereo. sigh I kind of have to laugh at myself. LOL

It's good you were notified they were bots before you got emotionally invested in them LOL :D seriously though, great effort trying to find places to post from. I really hope that my upvote for you was worth more but sadly it isn't.

That made me laugh.

You're welcome. I'm glad you enjoy my posts. This one took a bit more effort than the others, but it was a lot of fun. It's only touching the surface of the kind of statistical analysis you can do on the blockchain.

Good for you for blazing the trail. Personally, I've been mostly scared of making any analysis out of fear of making a mistake and being criticized for it. I really hope posts like this gain some traction to increase the confidence of people like myself. I agree that there are tons of insights to be mined.

Everyone makes mistakes. Hopefully if there's a mistake someone spots it, and I learn from it.

Yeah, I guess so. Maybe it comes down to strength of character regarding that aspect. Though I do appreciate criticism directed my way, using them as a means to improve.

Do you think you could find out the general topics they upvote on? Not really because I want to posting about those topics but more to see the direction in which steem it might be heading in. After all the whales are the curators of steem.

I think there was already a post that covered this. I can't remember who it was by. It covered the tags they upvote in most.

Producing stats for topics based on the content would be a bit harder. You could try a keyword based approach with clustering, but that'd be a lot of work for some shaky results. Fun to do if you have the time for it :)

Data is beautiful.

I agree with this wholeheartedly. If only data wasn't shared by all, I would've put a ring on it instantly. That's how much I like it.

Wow that was way to dam helpful !! OK if anyone agrees smash dat mfcking up vote! Also how has this helped others??

The upvote button for this post has been so smashed, I don't know if we could even resuscitate it.

I can assure you that I am not a bot ... I just happen to enjoy most posts I read.

Hehe my analysis was anything but in depth, and I'm no statistician. Hope you weren't offended :D I hear you're a mathematician. Any tips for future stats posts?

Absolute friggin gold.

Your perverted, you know that! lol

I see what you did there...

Oh Man, you stole my thunder! http://catchawhale.com
Was going to publish my results soon, but looks like you beat me to the chase.
I will still of course be publishing my findings but this looks pretty damn good.

But you made a cool web app out of it. I was too lazy to do that (well actually it's cause I'm going on holiday soon and wouldn't finish it in time).

Looking forward to your post. Oh and I'm following you now :D

As am I! We should collaborate in the near future!

Interesting research, but people can still post everyday, this is not a reason to just post on those specific days in those specific hours.

I don't have time for all that. Steemit is just one of the many things I do, I can't be up all in this bizness 24/7 I have other things to do. Money doesn't run my life :)

Now?

Some fantastic work by you here. It is a real pity there are so few whales - there just arn't enough to go around.

Thanks! They say this will change as time goes on and power is more distributed. We'll see.

Good stuff, I was wondering what best post times would be. Steemians, it's wise to do your best with the knowledge @bitcalm just posted!

This was actually much more interesting that I expected, good job :)

I can confirm I don't use a bot to vote for me; it's all done manually by yours truly, at least for now. I'm a bit surprised that I'm a "low upvoting whale" since I feel like I vote a lot, but as I don't use bots or have a team of people curating for me I suppose that's normal.

I'm also trying to refrain from voting on posts that already have $1000+ payouts in order to balance things out a little, so I guess that factors in as well.

Hi Rainman,

its great to see you communicate with us. i think it doesn't matter, if one uses a bot or votes himself.
through you own reading , you will catch more valuable stuff than the bots, the bots on the other hand, will be able to spread more earnings to many authors. so both sides have good and bad aspects.

maybe you could do me a favor and check this out,
https://steemit.com/food/@knozaki2015/britafilters-hacked-read-the-whole-hacking-story-steemit-exclusive

i co-authored it with a friend (100% of the Steem $ going to him) , and this is my piece i think was my best post of all in my Steemit history.

thanks !

Nice to hear you don't use a bot. It was difficult to tell with you because it could have just been (and probably was) the way things averaged out....or you're telling porkies ;)

It makes sense that the bot powered whales are upvoting more. Perhaps that's a good metric to determine if someone is a bot - I can imagine bots upvote more than regular users (and also never post articles). At the same time, I literally just looked at the images to compare them. I wanted to add the total count to each plot, but I'd already generated them and took 30 mins - laziness won. :)

Nice to hear that you hold back on large reward posts. I think a lot of users will be happy to hear that.

You did get my vote nevertheless ;)

I could be telling porkies of course but that doesn't come natural to me so I don't :) My curation rewards probably would've been way higher had I used a bot in any way, and being sick like I was for the last 3-4 days would not have made such a big impact.

I'm fine with the lesser rewards though, and to be honest I feel the system is a bit too skewed towards whale curators at the moment. Us whales working hard to optimize our curation rewards will not help with the distribution issue, so I prefer to take a relaxed attitude towards it.

Your power has grown! Impressive. Most impressive.

This is cool. Great you made this statestic. ^^

Coin Marketplace

STEEM 0.20
TRX 0.13
JST 0.030
BTC 65702.61
ETH 3485.24
USDT 1.00
SBD 2.51