was a bit of a mission to find out the right way to do this, but as I wrote previously, I am going through a couple of processes in order to be able to convert the steem block_log into JSON format (the raw format provided by full steemd RPC node), and from there it is simple to import it to any other database format.

Without any further ado, here is the python script, it depends on piston-lib's module pistonapi from @xeroc.

#!/usr/bin/python3
import os
import piston
from pprint import pprint
from pistonapi import SteemNodeRPC

rpc = SteemNodeRPC("ws://127.0.0.1:8090", "", "")

props = rpc.get_dynamic_global_properties ( )
current_block = 0
last_confirmed_block = props [ 'last_irreversible_block_num' ]

while current_block < last_confirmed_block:
  current_block += 1
  os.write ( 2, str.encode( "Block #" + str ( current_block ) + '\r' ) )
  block = rpc.get_block(current_block)
  pprint(block)

This puts out the raw JSON output to stdout and prints a nice friendly progress message to stderr to tell you how far it has got to. It will continue to run until the latest confirmed block at the time you start the script.

From this, we will be feeding it into another program that takes this data in and populates some other type of database structure, such as another blockchain format.

PS

In the process, I discovered what kind of load the RPC puts on my server. Here is a screencap of my netdata monitoring as this script runs and pulls it off the full node running on it.

First thing that jumped out at me was the RPC, which is processing hundreds, maybe thousands of requests per second, is only using one of the 10 cores. Second, the RPC is only feeding the data at a rate of about 1 megabyte per second (the ipv4 traffic, which is just for the loopback device, 'localhost')

So, I am not concerned at all about opening this node up to the public, the caveat is that it is not encrypted so it is not advisable for you to use it to broadcast transactions (well, if they have sensitive data, though there is not many that have that, for example the create_account function, which is currently broken anyway) if you are concerned about revealing to those outside the steem network what you are putting up to it. So use it at your own risk:

ws://projectinception.lt:8090

---

We can't code here! This is Whale country!

Vote #1 l0k1

_{Go to steemit.com/~witnesses to cast your vote by typing l0k1 into the text entry at the bottom of the leaderboard.}

_{(note, my username is spelled El Zero Kay One or Lima Zero Kilo One, all lower case)}