Steem for script kiddies: paginated get_account_history calls

Today, @ety001 published the post, The official API node will decrease the get_account_history upper limit. The post said,

We will decrease the upper limit of get_account_history API from 10000 to 100.

Please add pagination function into your program if you are using this API.

It happens that I've been meaning to learn about implementing pagination of API calls in shell scripts, but I never took the time... until today. After reading account_history_api.get_account_history, I was able to determine that I need to control the two parameters: start and limit.

Unsurprisingly, start tells it where to start, and limit tells it how many to grab (moving forward in time). It's clear from the documentation that you can start by setting start to some value, and then incrementing it in chunks of limit to get multiple pages.

What wasn't immediately obvious to me from the documentation, however, is how to get the pages at the end of the transaction list. I hacked around the shell prompt, and it turned out to be simple enough. Step 1 is to get the most recent transaction number, and step 2 is to calculate the other page starting numbers by decrementing from there.

So, here's the little toy script that I scraped together to accomplish that (using @social as an example account):

#!/bin/bash

STEEM_API="https://api.steemitdev.com"
PG_LIM=1  # Remember this starts counting at 0
STEEM_ACCT="social"
export STEEM_API PG_LIM STEEM_ACCT

### Get most recent transaction id
START=$(curl -s --data '{
  "jsonrpc": "2.0",
  "method": "account_history_api.get_account_history",
  "params": {
    "account": "'${STEEM_ACCT}'",
    "start": -1,
    "limit": 0
  },
  "id": 1
}' ${STEEM_API} | jq -S '.result[][][0]')

echo "Most recent transaction: "${START}

### List the most recent transactions in pages
for LCV in {2..0}
do
   FROM=$(( ${START} - ${LCV} - ( ${LCV} * ${PG_LIM} )  ))
   echo "New Page: (back ${LCV} pages)"
   curl -s --data '{
     "jsonrpc": "2.0",
     "method": "account_history_api.get_account_history",
     "params": {
       "account": "'${STEEM_ACCT}'",
       "start": "'${FROM}'",
       "limit": '${PG_LIM}'
     },
     "id": 1
   }' ${STEEM_API} | jq -S '.result[][][0]'
done

And here's some sample output, with 3 pages and page limit set to "1" (which gives back two results):

Most recent transaction: 47431
New Page: (back 2 pages)
47426
47427
New Page: (back 1 pages)
47428
47429
New Page: (back 0 pages)
47430
47431

Of course, this just prints out the label number for the transactions. Additional machinations would be needed with "jq" in order to extract other information from the transaction history. This is left as an exercise for the reader. ;-)

One point of note is that both start and limit start counting at 0. So if I tell it to start at transaction 1, it actually starts at the second transaction, and if I tell it to use 100 for the limit, it actually returns 101 results.

Incidentally, if I'm not misunderstanding something, some of the documentation for "get_account_history" seems to contain erroneous explanations for start and limit (i.e. here).

If I understand correctly, start actually has the following meanings:

Start value	Meaning
-1	The most recent transaction in the account's history
0	The first (earliest) transaction in the account's history
N	The (N+1) transaction in the account's history (Starting at 0 and moving forward in time.)

And - as noted above - limit starts counting at zero, so where the above document says: "1,000 results or 10,000 results", that should be "1,001" or "10,001". (And those will both be invalid after the coming change to 100 err... 101 that prompted this post. ;-)

That's all for now. Happy scripting and thanks for reading!

---

Visit the /promoted page and #burnsteem25 to support the inflation-fighters who are helping to enable decentralized regulation of Steem token supply growth.