Part 23: Retrieve And Process Full Blocks From The Steem Blockchain
This tutorial is part of a series where different aspects of programming with steem-python
are explained. Links to the other tutorials can be found in the curriculum section below. Every action performed on the Steem Blockchain
is registered in blocks
. These blocks
are the building stones for the Steem Blockchain
and this tutorial will explain how to retrieve and process individual blocks
.
Repository
https://github.com/steemit/steem-python
What will I learn
- Stream full blocks from the blockchain
- What do blocks look like?
- What in a transaction?
- What is an operation?
- Filter for different kind of operations
Requirements
- Python3.6
steem-python
Difficulty
- basic
Tutorial
Setup
Download the file from Github. There is 1 file get_block.py
which contains the code. The file takes 3 arguments from the command line which sets the starting_block
, block_count
and what operation
to filter for.
Run scripts as following:
> python get_block.py 22224731 10 transfer
Stream full blocks from the blockchain
Tutorials until now used the function stream
to stream operations
as they appeared on the Steem blockchain
from the current head block
. This is pretty close to a live stream as long as there is no backlog. This works perfect however it is susceptible to crashes and it is also not possible to backtrack previous blocks
.
By taking full control over which blocks to retrieve from the blockchain the user will have more control and the software becomes more resistant to crashes. The Blockchain
class holds the function stream_from
which can be used to achieve this. The start_block
must be set to the preferred block to starting streaming from, block_count
is used to calculate the end_block
and full_blocks
must be set to True
in order to retrieve the entire block.
from steem.blockchain import Blockchain
stream = self.b.stream_from(start_block=self.block,
end_block=self.end_block,
full_blocks=True)
for block in stream:
//do something with the block
Note: For this tutorial end_block is set for testing purposes. Leaving this blank will keep the stream alive and wait for new blocks
What do blocks look like?
Blocks
are the building stones for the blockchain
and in essence contain all the information that is stored on Steem
. Blocks
are linked in a chain hence the name blockchain
. Each block
itself is not that large and new blocks
are created every 3 seconds. Only information that is added or changed gets put into new blocks
. The basic structure of a block
looks as follows:
{
'previous': '01531f5aded177b75bbe144f44958c01e8061890',
'timestamp': '2018-05-07T15:20:03',
'witness': 'jesta',
'transaction_merkle_root': '01fbed04e90d4b19569788f2f34467d2de2edb40',
'extensions': [],
'witness_signature': '204cb7d24d1b3b440768873d3bd5fd29dd35925e38508346e998a486a83164b8aa7be6d748fdc4433231275b2caad3fe9400b850ac8656dc4565409115563d9a8b',
'transactions': []
'block_id': '01531f5ba3f99e0e10a6015491fa6f09bb3501ac',
'signing_key': 'STM8MFSHJA2DsjmEUBsf8JYdhm25YghdEUeDEMqNVMrWMCgHErDaL',
'transaction_ids': []
}
The timestamp
is useful to validate the age of the data, transactions
and transaction_ids
are left blank in this example as they contain most of the data. The transaction
and the transaction_id
are associated with each other based on their index
. So the first transaction
correlates with the first transaction_id
. Every action on the Steem
network registered into a transaction
.
What is a transaction?
Every transaction
has the following structure:
{
'ref_block_num': 8004,
'ref_block_prefix': 1744181707,
'expiration': '2018-05-07T15:29:54',
'operations': [
['vote', {
'voter': 'acehjaya',
'author': 'zunaofficial79',
'permlink': 'kabut-pagi',
'weight': 10000
}]
],
'extensions': [],
'signatures': ['1f205086670c27365738697f0bc7cfde8b6e976608dc12b0d391b2b85ad7870a002313be4b09a358c30010be2a09bebacead866ba180afc348ce40c70566f4ed88']
}
Found under operations
are alterations made to the Steem Blockchain
by its users. Here common operations
like voting, transfers and comments are found.
What is an operation
Every action that is registered on Steem
is a operation
which fits in one of the following categories:
comment, delete_comment, vote
account_create, account_update, request_account_recovery, recover_account
limit_order_create, limit_order_cancel
transfer, transfer_to_vesting, withdraw_vesting, convert
pow, feed_publish, witness_update, account_witness_vote
custom, custom_json
Source
Each operation
has its own structure and requires unique handling for processing. A transfer looks as follows:
{
'from': 'yusril-steem',
'to': 'postpromoter',
'amount': '5.000 SBD',
'memo': 'https://steemit.com/life/@yusril-steem/our-challenges-and-trials-in-life'
}
Filter for different kind of operations
Each block
is filled with transactions
in which an operation
is embedded. Filtering for operations
is done by checking its type which is located under ['operations'][0][1]
. The transaction_index
is tracked to know the specific transaction
in which the operation
took place.
for transaction in block['transactions']:
if transaction['operations'][0][0] == self.operation:
print(transaction_index,
transaction['operations'][0][1])
transaction_index += 1
Running the script
Running the script will print operations
, for which the filter is set, to the terminal ,with their corresponding transaction_index
, starting from the starting_block
for block_count
amount of blocks
. Feel free to test out the different operation
types to get a feeling how they are stored in the blocks
.
python get_block.py 22224731 10 transfer
Booted
Connected to: https://rpc.buildteam.io
Block: 22224731
44 {'from': 'hottopic', 'to': 'jgullinese', 'amount': '0.001 SBD', 'memo': 'Hello Friend, Your post will be more popular and you will find new friends.We provide "Resteem upvote and promo" service.Resteem to 18.000+ Follower,Min 45+ Upvote.Send 1 SBD or 1 STEEM to @hottopic (URL as memo) Service Active'}
...
...
...
Curriculum
Set up:
- Part 0: How To Install Steem-python, The Official Steem Library For Python
- Part 1: How To Configure The Steempy CLI Wallet And Upvote An Article With Steem-Python
Filtering
- Part 2: How To Stream And Filter The Blockchain Using Steem-Python
- Part 6: How To Automatically Reply To Mentions Using Steem-Python
Voting
- Part 3: Creating A Dynamic Autovoter That Runs 24/7
- Part 4: How To Follow A Voting Trail Using Steem-Python
- Part 8: How To Create Your Own Upvote Bot Using Steem-Python
Posting
- Part 5: Post An Article Directly To The Steem Blockchain And Automatically Buy Upvotes From Upvote Bots
- Part 7: How To Schedule Posts And Manually Upvote Posts For A Variable Voting Weight With Steem-Python
Constructing
Rewards
- Part 9: How To Calculate A Post's Total Rewards Using Steem-Python
- Part 12: How To Estimate Curation Rewards Using Steem-Python
- Part 14: How To Estimate All Rewards In Last N Days Using Steem-Python
Transfers
- Part 11: How To Build A List Of Transfers And Broadcast These In One Transaction With Steem-Python
- Part 13: Upvote Posts In Batches Based On Current Voting Power With Steem-Python
Account Analysis
- Part 15: How To Check If An Account Is Following Back And Retrieve Mutual Followers/Following Between Two Accounts
- Part 16: How To Analyse A User's Vote History In A Specific Time Period Using Steem-Python
- Part 18: How To Analyse An Account's Resteemers Using Steem-Python
Wallet
The code for this tutorial can be found on GitHub!
This tutorial was written by @juliank.
Thank you for your contribution.
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Thanks for the work and feedback, what would you like to see in these screenshots that are not already in the commandline outputs I shared above?
Thank you. That was very useful!
Your links to tutorials before Part 16 show me a blank page.
Tutorial 15 is at:
https://steemit.com/utopian-io/@steempytutorials/part-15-how-to-check-if-an-account-is-following-back-and-retrieve-mutual-followers-following-between-two-accounts
Your link points at:
https://utopian.io/utopian-io/@steempytutorials/part-15-how-to-check-if-an-account-is-following-back-and-retrieve-mutual-followers-following-between-two-accounts
Thanks for mentioning, will update that later
You have a minor grammatical mistake in the following sentence:
It should be its own instead of it's own.Thanks bot!
Hey @steempytutorials
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!
Contributing on Utopian
Learn how to contribute on our website or by watching this tutorial on Youtube.
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!