Steem API Basics - Part 1: Parsing Post Payloads with Ruby

The Steem API returns a data payload for posts that is big, long, and hard to handle. I'm here to help you master this data in order to make the most out of Steem.

Understanding the post payload can be a lot to take in at once, so, as part of the @steemsmarter ongoing effort of Making Sense Of Steem, I'm breaking this post into a five-part series that will go deep into how we can dominate the data that we receive from this API endpoint.

Where we last left off

In my previous post, we looked at finding posts published in a time range from the Steem blockchain, during which I used Ruby on Rails and the awesome Radiator gem from @inertia to create a a lib service class named SteemPostFinder. This new class let us run queries such as:

SteemPostFinder.new('crypto').posts_in_time_range('2018-02-14','2018-02-15')

This query returns an array of Discussion payloads hashes pulled directly from the blockchain via the Steem API. Sweet! We now have a huge package of Discussion payloads... but what good is this if we don't know how to handle a single payload?

Here we go again

If you haven't seen my previous post, you should probably check it out. It's pretty awesome in my super non-biased opinion.

We're going to be building on top of the SteemPostFinder class from that post to add some easy ways to grab individual posts so that we can probe their inner workings. Either way, it will probably help you to review the gist of the previous post's final results to get the most of this post.

Parsing a single payload

We're going to use @inertia's Radiator gem to grab the payload for a single discussion from the Steem API. I'm wrapping it in a new Rails method so we can resuse this logic in our SteemPostFinder class (see the updated gist):

def get_post(author, permlink)
  params = { tag: @tag_name, limit: 1, start_author: author, start_permlink: permlink }
  response = api.get_discussions_by_created(params)
  response.blank? ? [] : response['result'].try(:first).try(:with_indifferent_access)
end

From here we can use our SteemPostFinder class to get the post payload for any discussion

post = SteemPostFinder.new('steem').get_post('thescubageek','steemsmarter-finding-daily-posts-with-ruby')

At this point we have a post variable containing the payload from the Steem API for a single post. It's a huge payload, but I bet you can handle it. Let's go a little deeper.

Disclaimer

I refer to posts in a loose sense in this article, but what I'm really talking about is considered to be discussions under the hood. A discussion is a parent object that includes both posts and comments, both of which are really instances of discussions at the end of the day.

Confused? No worries -- it is confusing -- and this will be covered in a future article.

The Payload

Here's an example of the mess that a single payload dumps on us:

{
  "id": 33672515,
  "author": "thescubageek",
  "permlink": "steemsmarter-finding-daily-posts-with-ruby",
  "category": "steem",
  "parent_author": "",
  "parent_permlink": "steem",
  "title": "SteemSmarter: Finding Daily Posts with Ruby",
  "body": "Steem is awesome. The Steem API documentation, however, is definitely not. I want to help change that.",
  "json_metadata": {
    "tags": [ "steem", "steemdev", "ruby", "blockchain", "steemit" ],
    "users": ["steemsmarter"],
    "image": [
      "https://ws1.sinaimg.cn/large/006tNc79gy1folmvqw02ej30m80ek773.jpg",
      "https://ws2.sinaimg.cn/large/006tNc79gy1folmvpf65lj30m80bfq4j.jpg"
    ],
    "links": [
      "https://steemit.com/@ashe-oro",
      "https://steemit.com/@crypthoe",
      "https://steemit.com/@steemsmarter",
      "https://steemit.com",
      "https://github.com/inertia186/radiator",
      "https://steemit.com/@inertia"
    ],
    "app": "steemit/0.1",
    "format": "markdown"
  },
  "last_update": "2018-02-19T04:36:15",
  "created": "2018-02-19T04:30:03",
  "active": "2018-02-22T01:01:24",
  "last_payout": "1970-01-01T00:00:00",
  "depth": 0,
  "children": 5,
  "net_rshares": "5637031107296",
  "abs_rshares": "5637031107296",
  "vote_rshares": "5637031107296",
  "children_abs_rshares": "6144117672855",
  "cashout_time": "2018-02-26T04:30:03",
  "max_cashout_time": "1969-12-31T23:59:59",
  "total_vote_weight": 2392548,
  "reward_weight": 10000,
  "total_payout_value": "0.000 SBD",
  "curator_payout_value": "0.000 SBD",
  "author_rewards": 0,
  "net_votes": 21,
  "root_comment": 33672515,
  "max_accepted_payout": "1000000.000 SBD",
  "percent_steem_dollars": 10000,
  "allow_replies": true,
  "allow_votes": true,
  "allow_curation_rewards": true,
  "beneficiaries": [],
  "url": "/steem/@thescubageek/steemsmarter-finding-daily-posts-with-ruby",
  "root_title": "SteemSmarter: Finding Daily Posts with Ruby",
  "pending_payout_value": "38.348 SBD",
  "total_pending_payout_value": "0.000 STEEM",
  "active_votes": [
    {
      "voter": "ashe-oro",
      "weight": 2158,
      "rshares": "430002590491",
      "percent": 10000,
      "reputation": "14758152785384",
      "time": "2018-02-19T04:30:09"
    },
    {
      "voter": "lukestokes",
      "weight": 719260,
      "rshares": "3016796311432",
      "percent": 10000,
      "reputation": "221773617070505",
      "time": "2018-02-19T23:41:30"
    }, {
      "voter": "inertia",
      "weight": 663067,
      "rshares": "1390080324192",
      "percent": 10000,
      "reputation": "122595430429474",
      "time": "2018-02-19T05:24:21"
    }
  ],
  "replies": [],
  "author_reputation": "92507853048",
  "promoted": "0.000 SBD",
  "body_length": 17125,
  "reblogged_by": []
}

Nasty eh? Quite a lot to swallow at once... for today, let's get a feel of most important fields: author and permlink.

Author and Permlink

author and permlink are the two unique key-pairs upon which all discussions (posts + comments) are referenced. There can only be exactly one pair of author/permlink in the entire Steem blockchain.

For posts, the permlink defaults to the dasherized title of the article. For example, "SteemSmarter: Finding Daily Posts with Ruby" becomes steemsmarter-finding-daily-posts-with-ruby.

For comments, the permlink is the same as the parent post/comment, except that the permlink is:

1. Prefixed with re- followed by the parent posts/comment's author, ie re-thescubageek-
2. Suffixed with the the timestamp of the comment, ie -20180226t231257609z
   For example, a reply to the post above may look like: re-thescubageek-steemsmarter-finding-daily-posts-with-ruby-20180226t231257609z

author and permlink payload parameters are significant for:

• Looking up an individual discussion on the Steem blockchain
• Referencing a starting point for recursive time-range API lookups (see my last post)
• Identifying parent/child relationships in the post/comment hierarchy (more on this later in a future post)

The gist is this: if you don't know the author and permlink for a discussion, you're gonna have a bad time.

What's Next?

In my next post, we will pick up by probing deeper into the the parameters related to content, author profile, and post metadata in the discussion payload.

If you find this post helpful please resteem and upvote, and remember to follow @thescubageek and @steemsmarter to stay on top of our latest Steem projects.

Until next time, keep on Steemin' Steemians!