I am currently working on a little asynchronous Twisted based library for using API steem nodes from python. As you may know, I've been running a number of bots that were based on steem-python. For convenience, I started to build a little wrapper for the stream_from API call, that made it a bit easier to build simple bots. Especially, given a number of issues with how steem-python handles run-time errors, the wrapper library tries to allow even crashing bots to pick up streaming blocks where the left of.

But the more I started depending on steem-python for my code and the bots I've been running, the more I started realizing that my steempersist wrapper would probably be much better off not being a wrapper for steem-python at all. I started to think what a proper python library for the steem json-RPC API shoud look like.

Please note, the functionality for my work in progress asyncsteem is still far from complete, but the core is coming together right now, and this post is meant to show that things are starting to look promising.

So what features do I plan to give the asyncsteem python library, compared to what you get with steem-python

• Asynchronous, based on defacto asynchronous framework for Python: Twisted
• Initially targeted at Python 2.7(Python 3 support is planned post v1.0)
• Propper error handling.
• Centered around the idea of event streaming
• Limited parallel JSON-RPC queries for performance reasons
• Support for throttling

While many features are currently still missing, documentation is still non existance, and the code is in desperate need of comments to explain what is going on, the first, the step I want to report on here, is a little benchmark I managed to run, comparing how steem-python and my work in progress asyncsteem perform in the simple scenario of streaming operations using the steem-python stream_from API and its asyncsteem counterpart.

The scripts truly do nothing else but use the base API and calling time.time() whenever a set number of blocks have been processed to see how many seconds have passed. The runs were done on the same machine running on the same network, both supplied with the same set of nodes, and both set to use https JSON-RPC.

steem-python

Ok, so here is the log from running the steem_python based script for a little while

• 100 blocks processed in 18.696368932724 seconds
• 100 blocks processed in 38.46126675605774 seconds
• 100 blocks processed in 16.9816734790802 seconds
• 100 blocks processed in 171.23247861862183 seconds
• 100 blocks processed in 100.00816512107849 seconds
• 100 blocks processed in 100.00731372833252 seconds
• 100 blocks processed in 100.00825309753418 seconds
• 100 blocks processed in 100.01019859313965 seconds
• 100 blocks processed in 99.99268531799316 seconds
• 100 blocks processed in 100.05622410774231 seconds
• 100 blocks processed in 100.02814364433289 seconds
• 100 blocks processed in 99.95744681358337 seconds
• 100 blocks processed in 100.01225447654724 seconds

We see there is some fluctuation in the performance. They start off relatively fast for the first 300 blocks, then stabalize at about one block per second.

asyncsteem

Now for the results from asyncsteem.

• Starting off with node rpc.buildteam.io
• Switching to node steemd.minnowsupportproject.org : CancelledError
• Switching to node steemd.pevo.science : Non-JSON response from server
• Switching to node rpc.steemviz.com : Non-JSON response from server
• 1000 blocks processed in 113.636193991 seconds.
• Switching to node seed.bitcoiner.me : Non-JSON response from server
• Switching to node rpc.steemliberator.com : Non-JSON response from server
• 1000 blocks processed in 141.749578953 seconds.
• Switching to node api.steemit.com : Non-JSON response from server
• 1000 blocks processed in 156.835906982 seconds.
• 1000 blocks processed in 76.7560169697 seconds.
• 1000 blocks processed in 72.6464269161 seconds.
• 1000 blocks processed in 68.1995720863 seconds.
• 1000 blocks processed in 76.6237959862 seconds.
• 1000 blocks processed in 77.0866971016 seconds.
• 1000 blocks processed in 80.9138948917 seconds.
• 1000 blocks processed in 75.990858078 seconds.
• 1000 blocks processed in 76.8726959229 seconds.
• 1000 blocks processed in 79.1454079151 seconds.
• 1000 blocks processed in 76.5809831619 seconds.
• 1000 blocks processed in 75.8056778908 seconds.
• 1000 blocks processed in 75.8016791344 seconds.
• 1000 blocks processed in 78.1602649689 seconds.
• 1000 blocks processed in 78.1862130165 seconds.
• 1000 blocks processed in 75.3016858101 seconds.
• 1000 blocks processed in 76.0714540482 seconds.
• 1000 blocks processed in 76.8744580746 seconds.
• 1000 blocks processed in 75.0945250988 seconds.
• 1000 blocks processed in 74.8684568405 seconds.
• 1000 blocks processed in 76.5565879345 seconds.
• 1000 blocks processed in 78.8888401985 seconds.
• 1000 blocks processed in 78.9060978889 seconds.
• 1000 blocks processed in 77.9604611397 seconds.
• 1000 blocks processed in 78.8955488205 seconds.

Notice how the library starts off with a bit of problems, it starts off at about six blocks per second, then, after having found a well-behaving node, doubles its speed to about double that.

Conclusions

While work on asyncsteem is still far from done, I believe the preliminary benchmark I just ran shows it is already quite promising. Once more of the library is completed, even if the full order of magnitude speed difference ends up being half what I've seen today, there are still many scenario's where today steem_python's performance has shown prohibitive, asyncsteem might just be fast enough to implement them.

Comming up

There is still a lot of work to do before asyncsteem can be deamed production stable, but the core is starting to come together quite nicely at this moment. I hope to soon to be porting the @pibarabot Watching The Watchers script to asyncsteem and let it run in paralel to the current script to see how stable everything runs. The first version won't be able to post anything yet (sigining operations is further down on my list than stability, error handling and logging), but as soon as things run stable enough as shadow service, I'll probably move on to working on the signing bit and porting Watching The Watchers completely to asyncsteem. Then once @croupierbot is also ported and has proven to run stable it will become time to phase out steempersist by merging that project into asyncsteem.