What Will I Learn?

In this part of the series we will learn how:

• the perl global namespace works
• dynamically modify the namespace and extend it with your own methods
• explore the steemit api code
• toy around with various steemit apis to create a more or less useful application

Requirements

• A modern perl environment
  you can follow my earlier post on how to write a simple steemit client
  Or how to setup a modern perl development environment
• you may also want to check out the related git repo since we will continue to change the example 2 branch
• if you come directly from the example 2 please note that there are 2 new requirements:

sudo apt-get install libssl1.0-dev zlib1g-dev
cpanm IO::Socket::SSL

this is to allow us to use https to connect to steemit nodes.

Difficulty

• Intermediate

Tutorial Contents

The Perl symbol table

First a small introduction into perl namespaces. The relevant documentation can be found on the perldoc website . But i will still show you the basics here that we need later on.

Every package in perl is denoted by a notation like this Foo::Bar. So let's assume we have this module. In there we have all the variables defined with our and subroutines accessible. For each name we have several types of fields:

• Scalar: $
• Hash: %
• Array: @
• Subroutine: &

Normally we use this by defining things in our packages like this:

package Foo;

our $VERSION = 5;
sub foo {
   return “foo”
} 

Then we can access the things we defined by:

use Foo;
print Foo::VERSION;
print Foo->foo;

This is nice and in 99% of all cases what we want. But what of we want to dynamically change things, like adding new subroutines.

For this we can use a simple trick that looks like this;

{
   no strict ‘refs’;
   my $methodname = ‘Foo::bar’;
   *$methodname = sub {
         Return “bar”
    };
}

Which is exactly the same as defining our code statically in the package. Sometimes this can even be useful to override methods in other modules for debug purposes. But be careful to only use this feature in a clear structured way since you will have a hard time debugging things when you are not careful.

The steemit API

We are focusing here still only on the database api. Since we will focus on the signing and cryptography part later on. The most up to date information on the api can of be found in the [source code] (https://github.com/steemit/steem/blob/master/libraries/app/include/steemit/app/database_api.hpp)

At the bottom we have the api method definitions which we can call.

FC_API(steemit::app::database_api,
   // Subscriptions
   (set_block_applied_callback)

   // tags
   (get_trending_tags)
   (get_tags_used_by_author)
   (get_discussions_by_payout)
   (get_post_discussions_by_payout)

And more… if we now look at the definition of each method we can find more details about it.

      /** This API will return the top 1000 tags used by an author sorted by most frequently used */
      vector<pair<string,uint32_t>> get_tags_used_by_author( const string& author )const;

We see here for example that we can give into it as parameter the author name and it will return us a list of pairs tag => <number used>

Or this one:

vector<discussion> get_discussions_by_trending( const discussion_query& query )const;

This will accept a query element and return us a list of discussions. This query discussion element is probably one noteworthy thing to talk about. Its definition is here:

/**
 *  Defines the arguments to a query as a struct so it can be easily extended
 */
struct discussion_query {
   void validate()const{
      FC_ASSERT( filter_tags.find(tag) == filter_tags.end() );
      FC_ASSERT( limit <= 100 );
   }

   string           tag;
   uint32_t         limit = 0;
   set<string>      filter_tags;
   set<string>      select_authors; ///< list of authors to include, posts not by this author are filtered
   set<string>      select_tags; ///< list of tags to include, posts without these tags are filtered
   uint32_t         truncate_body = 0; ///< the number of bytes of the post body to return, 0 for all
   optional<string> start_author;
   optional<string> start_permlink;
   optional<string> parent_author;
   optional<string> parent_permlink;
};

This data structure is used by many calls which give us a discussion and allows us to define what we are searching for. We will see in the next chapter how this is useful

I won't go over each method. My best tip is to try around some different methods. Look at the results and find interesting ways to use it.

extending the perl example with all database methods

Now we want to use the knowledge we gained so far and extend our example from the previous exercise with all database methods.

To recap we had this general method for executing calls:

sub _request {                                                                                                           
   my( $self, $api, $method, @params ) = @_;                                                                             
   my $response = $self->ua->get( $self->url, json => {                                                                  
      jsonrpc => '2.0',                                                                                                  
      method  => 'call',                                                                                                 
      params  => [$api,$method,[@params]],                                                                               
      id      => int rand 100,                                                                                           
   })->result;                                                                                                           
                                                                                                                         
   die "error while requesting steemd ". $response->to_string unless $response->is_success;                              
                                                                                                                         
   my $result   = decode_json $response->body;                                                                           
                                                                                                                         
   return $result->{result} if $result->{result};                                                                        
   if( my $error = $result->{error} ){                                                                                   
      die $error->{message};                                                                                             
   }                                                                                                                     
   #ok no error no result                                                                                                
   require Data::Dumper;                                                                                                 
   die "unexpected api result: ".Data::Dumper::Dumper( $result );                                                        
}                                                                                                                        

And this method as one example call:

sub get_accounts {                                                                                                       
   my( $self, @params ) = @_;                                                                                            
   return $self->_request('database_api','get_accounts',@params);                                                        
}   

Now since we have > 80 calls we don't want to repeat the same pattern again and again. On the other hand it would still be nice to have a dedicated method call for each one.

So what do we do? First we introduce a helper method:

install_methods();
                                                                                                                         
sub install_methods {                                                                                                    
   my %definition = _get_api_definition();                                                                               
   for my $api ( keys %definition ){                                                                                     
      for my $method ( $definition{$api}->@* ){                                                                          
         no strict 'subs';                                                                                               
         no strict 'refs';                                                                                               
         my $package_sub = join '::', __PACKAGE__, $method;                                                              
         *$package_sub = sub {                                                                                           
            shift->_request($api,$method,@_);                                                                            
         }                                                                                                               
      }                                                                                                                  
   }                                                                                                                     
}                                                                                                                        
                                                                                                                         
                                                                                                                         
sub _get_api_definition {                                                                                                
                                                                                                                         
   my @database_api = qw(                                                                                                
      verify_account_authority                                                                                           
      get_liquidity_queue                                                                                                
      get_discussions_by_feed                                                                                            
      get_discussions_by_cashout                                                                                         
      get_content_replies                                                                                                
      lookup_accounts                                                                                                    
      get_state                                                                                                          
      get_withdraw_routes                                                                                                
   );                                                                                                                    
                                                                                                                         
   return (                                                                                                              
      database_api          => [@database_api],                                                                          
   )                                                                                                                     
}         

Please note i have shortened the actual methods here for readability purposes. They are all included in the github branch for the example3

Now we defined all out api calls in a tidy list and dynamically add all methods to the namespace of our package. This will yield us now with all the database api methods that exist in the database api.

example usage

We now bring everything together in a new script “example3.pl”

#/usr/bin/env perl                                                                                                       
use Modern::Perl '2017';                                                                                                 
use Data::Dumper;                                                                                                        
use FindBin;                                                                                                             
use lib "$FindBin::Bin/../lib";                                                                                          
use Steemit;                                                                                                             
                                                                                                                         
my $steem = Steemit->new;                                                                                                
say "Initialized Steemit client with url ".$steem->url;                                                                  
                                                                                                                         
#get the last 99 discussions with the tag utopian-io                                                                     
#truncate the body since we dont care here                                                                               
my $discussions = $steem->get_discussions_by_created({                                                                   
      tag => 'utopian-io',                                                                                               
      limit => 99,                                                                                                       
      truncate_body => 100,                                                                                              
});                                                                                                                      
                                                                                                                         
#extract the author names out of the result                                                                              
my @author_names = map { $_->{author} } @$discussions;                                                                   
say "last 99 authors: ".join(", ", @author_names);                                                                       
                                                                                                                         
#load the author details                                                                                                 
my $authors = $steem->get_accounts( [@author_names] );                                                                   
#say Dumper $authors->[0];                                                                                               
                                                                                                                         
#calculate the reputation average                                                                                                                             
my $reputation_sum = 0;                                                                                                  
for my $author ( @$authors ){                                                                                            
   $reputation_sum += int( $author->{reputation} / 1000_000_000 );                                                       
}                                                                                                                        
                                                                                                                         
say "Average reputation of the last 99 utopian authors: ". ( int( $reputation_sum / scalar(@$authors) )  / 100 );        
 

It will result in this:

~/git/perlSteemit/bin$ perl example3.pl 
Initialized Steemit client with url https://rpc.steemliberator.com
last 99 authors: theoutspokenking, mwfiae, tobias-g, fachrurrazi, killerfreak, dpyroc, azwarrangkuti, dimasputra, orelmely, taylangkcn, earlpayton, goalsetter, streetdealin, abriella, faisalazmi, quiva, apocz, theoutspokenking, reshmira, whatsapps, schamangerbert, joelsteem, omeratagun, onurkahveci, malikaja, brainz, cricketsport, chicoou, goalsetter, izhaaan, omeratagun, buyapungoe, josue33, lucyexactly12345, chicoou, earlpayton, josue33, realinfo, omur61, orkutyorulmaz, escorn, gifmaker, iamankit, mirhimayun, omeratagun, omeratagun, musangprik, realinfo, steemitstats, hackspoiler, codygee237, juviemaycaluma, godfish, maarian, simpleawesome, evansbankx, tngflx, beyonddisability, cryptopimp, cutkhanza, nehomar, charansai612, abuthalib, faisalamin, ferizal, tarikhakan55, rizkythamrin, kodeblacc, buckydurddle, shoganaii, beforandafter, sogata, ammarraisafti, faisalamin, rdvn, semasping, an0na, dpyroc, laxam, tobaloidee, neokuduk, oups, rooneey, amaliatul, yandaalpiansyah, wens, apadet90, pakwarazik1990, kaking, sahmmie, hendrisaputra, yandot, khaled-dz, fatimatul, abhishekjanu, lucymar, riskaanis833, abhishekjanu, azkaalqhifari
Average reputation of the last 99 utopian authors: 36.25

Of course it will look different on your screen ;)

recap

So we have learned how to:

• Manipulate the perl symbol table to dynamically install methods
• Explore the steemit api
• Calculate the average reputation of the last 99 steemit users
• examples are uploaded to the github branch for the example3

Curriculum

write a basic per package to interact with steemit

---

Posted on Utopian.io - Rewarding Open Source Contributors

---