So you wanna build a bot from scratch? Part 1: Getting Setup Step by StepsteemCreated with Sketch.

in steemit •  3 months ago

robot1.jpg
Can you build a bot take your account from minnow to whale?

Today I want to start the first in a series on how to build your own steembot from scratch.
I'm sure there are as many ways to build bots as there are people who want to build them.

This tutorial is NOT for everyone. You need to have some coding experience and it's extremely helpful if you know your way around AWS. If Cloud9, Lambda, Dynamo, Lex & CloudWatch sound like foreign concepts to you, then this is not the tutorial for you. But if those words have you chomping at the bit, then this might be your big chance to level up with a minimum of cost and effort.

Keep in mind that I am not saying my way is the easiest or even the best. If you want to go low effort you don't even have to build your own, there are 1001 bot services out there that will let you upload your keys and automate pretty much any task you could ever think of. However if you go that route you really lose control of your keys and at the same time you learn nothing.

Beyond the pre-made bots, there are also many bot building toolkits, I really like Steem Bot which makes the most common tasks very easy. We won't be using it here because I want to teach you how to build a bot from scratch and that toolkit is a literal kitchen sink. Lambda has a max size constraint and the steembot toolkit is just a bit too much for it. We need lightweight and as close to the metal as possible.

To build our bot, we are going to go through several iterations before we come to the final iteration.
Here are the iterations...

  1. Maintainer, this bot does general maintenance tasks on your account. Specifically ours will launch about once a day, check the account for rewards balances, claim them and then power up any rewards. This will accelerate compounding during the remaining phases.

  2. Autofollow, this bot will automatically follow anyone who follows you

  3. Autovote, this bot will automatically upvote new posts from anyone you are following.

  4. Autocomment, this bot will leave behind a message letting people know you voted for them. You see all the major bots doing this. It looks like "You've received a 10% upvote from @somebot!"

  5. Autopay, this bot is the final iteration, it takes into account delegated SP and pays the people who delegated steem to you, proportional to their delegation amount.

While working through #3 & #4 I will also be showing you the ways I use Lex, SageMaker & Comprehend to add a bit of understanding to the bot about what I'm about to upvote and to try and find something insightful to say. This part hasn't worked out so well, because I like pretty pictures and photoblogs currently boggle sarabot. She knows I like photoblogs, but she can't figure out what they're about. I'm hoping to resolve that over the coming month as I build out these tutorials.

Ok so on to the first steps.

#1 you need an AWS account. I won't be getting you into anything that isn't free for at least a year and I'm trying to favor things that are free for life under the AWS free tier. This will give you plenty of flexibility in the future.

#2 once your AWS account is created you will need an IDE. I used to use VS Code and serverless, but I've found Cloud9 to provide a superior workflow for the way we are building this. So go ahead and spin up a Cloud 9 instance.

If you don't have a Cloud 9 account already, follow these pictures...
step1.png
Give it a name, something you can remember because at some point you're going to walk away and wonder what the heck this thing was.
step2.png
We're building Lambdas and only using this environment to test if they will run. Lambdas are pretty resource constrained so let's just go with defaults here, they're more than enough for our purposes.
step3.png
As a final step, I also change the theme to night in order to save wear and tear on my eyeballs.
step4.png
step5.png

#3 In your Cloud 9 instance you will want to configure a new Lambda function. You start this by clicking the "AWS Resources Tab"
step6.png
Lambdas are small, lightweight, special purpose functions. The reason we build this as a collection of Lambdas instead of setting up nodejs on a standalone server is simple. Lambdas are free for life whereas running a server, even a micro instance is only free for a year. After that year, you must begin paying and if your bot isn't making a healthy profit, this could be a drain on resources quickly. Lambdas remain free and once deployed they are essentially maintenance free.

Here I've clicked on the Lambda logo to create a new application and a new function within that application. Applications are just convenient ways to group functions. You can name yours anything you want.
step7.png
Here I've chosen the app name of basicbots and the function name of maintainer, click Next when done

step8.png
In the image above it can be easy to get lost in the options. Furthermore, AWS has had support for node 8.10 since April, yet the templates have not been updated to reflect this. Go ahead and just choose the first option here, empty-nodejs. I'm aware it says nodejs 6.10 but we will update it to be node 8.1 by the time we're done

step9.png
Here it's asking for a function trigger. We have no triggers setup yet. So for now we select none. Do NOT select API Gateway, this will allow you to put an API on top of your function, but it also adds cruft we won't be using because we won't be using the API gateway for this.

step10.png
Pay attention here, you want 1 GB (1024 MB) of RAM available to your Lambda, don't go any higher or lower. Higher will cost money and lower could cause your bot to crap out at an inopportune time.

step11.png
Final review, make sure everything looks correct because this is your last chance! (not really, you can change any of these settings and I'll show you how in a minute)

#4 Once you have your Lambda created, you will want to make sure to update the template.yaml to node 8.1, this is because we use async/await and also promisify to make the code cleaner and to avoid falling into callback hell.

step12.png
Here is my lambda fresh from the template system. We are going to need to change a few things. Open the template.yaml file

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An AWS Serverless Specification template describing your function.
Resources:
  maintainer:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: maintainer/index.handler
      Runtime: nodejs6.10
      Description: ''
      MemorySize: 1024
      Timeout: 15

Normally these defaults would be ok. 15s is more than enough time for about anything, but this is only going to run once a day we don't want it to bail out in the middle. So we change the timeout and we also add some more descriptive text. Furthermore, we need to change the runtime to nodejs8.10. The final version looks like...

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: A function to handle daily maintenance tasks for steemit.
Resources:
  maintainer:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: maintainer/index.handler
      Runtime: nodejs8.10
      Description: 'This function handles claiming rewards and powering up any loose steem we have clanging around in our pockets'
      MemorySize: 1024
      Timeout: 300

Now save it

Once the template is updated, you also need to update your local environment to node 8.1 you can do this from the command line with...
step13.png

nvm install 8 && nvm use 8

#5 Now you have everything in place you need to make your bot. So let's get cracking on the code.

Open up the file index.js and it will look like this...

exports.handler = (event, context, callback) => {
    // TODO implement
    callback();
};

Not really helpful is it? Of these incoming parameters the only one we will use is the event parameter because we will pass in our account name and our posting key. For now let's just focus on getting our prerequiste steem library in.
Back to the command line

cd basicbots/maintainer
npm init
npm install --save steem

Now, go back to index.js and require steem. May as well make that entrance function async as well

const steem = require('steem');

exports.handler = async (event, context, callback) => {
    
    callback();
};

This is good, but isn't terribly useful.
Let's get our account info first...

If you're like me, you hate promises because they cause code to quickly devolve into an unreadable mess. But the only thing I hate more than promises are so called nodebacks This is the practice of doing callbacks the node way. Fortunately nodejs 8 gave us a new utility function called promisify. This will turn just about any nodeback into a promise. Also they gave us async/await which allows us to deal with asynchronous functions in the most logical way, by simply waiting for them to finish before proceeding.

So instead of

steem.api.getAccounts(['saragarmee'], function(err, result) {
    console.log(err, result);
});

We can do this

let accountInfo = await getAccounts([saragarmee]);
accountInfo = accountInfo[0];

All together the code looks like this...

const steem = require('steem');
const {promisify} = require('util');

const getAccounts = promisify(steem.api.getAccounts);
exports.handler = async (event, context, callback) => {
    let accountInfo = await getAccounts(['saragarmee']);
    accountInfo = accountInfo[0];
    console.log("accountInfo for saragarmee: ",accountInfo);
    callback();
};

Protip: You should obviously change the name saragarmee to your own account name.

Now you can run this by right clicking on the "maintainer" lambda in the lambdas window on the right and then selecting "Run" -> "Run Local"

This will pop open a new view with a green button labeled "Run". go ahead and click it and the output will be something like...

Function Logs
2018-08-05 09:14:47.488 accountInfo for saragarmee:  { id: 1064920,
  name: 'saragarmee',
  owner: 
   { weight_threshold: 1,
     account_auths: [],
     key_auths: [ [Array] ] },
  active: 
   { weight_threshold: 1,
     account_auths: [],
     key_auths: [ [Array] ] },
  posting: 
   { weight_threshold: 1,
     account_auths: [ [Array] ],
     key_auths: [ [Array] ] },
  memo_key: 'STM62oYBzJPjJBq94Z6GFHw9vvrzmYDk6ub4hBA55Vs6We9gyqf8G',
  proxy: '',
  last_owner_update: '1970-01-01T00:00:00',
  last_account_update: '2018-07-08T11:49:21',
  created: '2018-07-05T10:45:42',
  mined: false,
  recovery_account: 'blocktrades',
  last_account_recovery: '1970-01-01T00:00:00',
  reset_account: 'null',
  comment_count: 0,
  lifetime_vote_count: 0,
  post_count: 147,
  can_vote: true,
  voting_power: 8075,
  last_vote_time: '2018-08-05T09:07:42',
  balance: '0.000 STEEM',
  savings_balance: '0.000 STEEM',
  sbd_balance: '0.004 SBD',
  sbd_seconds: '32223855',
  sbd_seconds_last_update: '2018-08-05T05:42:39',
  sbd_last_interest_payment: '2018-07-13T18:22:27',
  savings_sbd_balance: '0.000 SBD',
  savings_sbd_seconds: '0',
  savings_sbd_seconds_last_update: '1970-01-01T00:00:00',
  savings_sbd_last_interest_payment: '1970-01-01T00:00:00',
  savings_withdraw_requests: 0,
  reward_sbd_balance: '0.000 SBD',
  reward_steem_balance: '0.000 STEEM',
  reward_vesting_balance: '4.052593 VESTS',
  reward_vesting_steem: '0.002 STEEM',
  vesting_shares: '39493.051621 VESTS',
  delegated_vesting_shares: '0.000000 VESTS',
  received_vesting_shares: '1047361.470606 VESTS',
  vesting_withdraw_rate: '0.000000 VESTS',
  next_vesting_withdrawal: '1969-12-31T23:59:59',
  withdrawn: 0,
  to_withdraw: 0,
  withdraw_routes: 0,
  curation_rewards: 10,
  posting_rewards: 441,
  proxied_vsf_votes: [ 0, 0, 0, 0 ],
  witnesses_voted_for: 1,
  last_post: '2018-08-05T09:07:45',
  last_root_post: '2018-07-29T22:08:39',
  average_bandwidth: '51690465331',
  lifetime_bandwidth: '212692000000',
  last_bandwidth_update: '2018-08-05T09:07:45',
  average_market_bandwidth: 2533719816,
  lifetime_market_bandwidth: '16520000000',
  last_market_bandwidth_update: '2018-08-05T06:58:51',
  vesting_balance: '0.000 STEEM',
  reputation: '32694079099',
  transfer_history: [],
  market_history: [],
  post_history: [],
  vote_history: [],
  other_history: [],
  witness_votes: [ 'yabapmatt' ],
  tags_usage: [],
  guest_bloggers: [] }

The values we are most interested in right now are...

  reward_sbd_balance: '0.000 SBD',
  reward_steem_balance: '0.000 STEEM',
  reward_vesting_balance: '4.052593 VESTS',

As you can see I don't have a lot, but the function call to claim them requires we list them all correctly and it looks like...

steem.broadcast.claimRewardBalance(privatePostingWif, 'username', '0.000 STEEM', '0.000 SBD', '123.093932 VESTS', function(err, result) {
  console.log(err, result);
});

First off, let's promisify this function. Then claim it...

let result = await claimRewardBalance(wif, 'saragarmee', accountInfo.reward_steem_balance, accountInfo.reward_sbd_balance,accountInfo.reward_vesting_balance);

console.log("Result from claiming rewards: ",result);

But wait? WTH is this wif parameter?

This is your posting key in wif format. To find it go to your wallet, then go to permissions. Find the posting key. Then show the private key. It will start with a 5.

We obviously don't want to paste that into our source code. So we need to make sure this is given as a parameter to the function. So go back to the run screen.
Change the payload from...

{}

to...

{
"account" : "saragarmee",
"wif" : "5somethingIcopiedfrommyownaccount"
}

Now you're just about all set except for one thing.
You don't want this key ending up on github, but that payload will end up there if you don't take the time to create a .gitignore file for the lambda-payloads.json and you may as well add a line for node_modules as well.
If you don't know how to create a .gitignore file, follow this link for more information...

You'll notice that I added my account name to make it easier to change things later.
The final code now looks like...

const steem = require('steem');
const {promisify} = require('util');

const getAccounts = promisify(steem.api.getAccounts);
const claimRewardBalance = promisify(steem.broadcast.claimRewardBalance);

exports.handler = async (event, context, callback) => {
    let account = event.account;
    let wif = event.wif;
    let accountInfo = await getAccounts([account]);
    accountInfo = accountInfo[0];
    console.log("accountInfo for "+account+": ",accountInfo);
   
    let result = await claimRewardBalance(wif, account, accountInfo.reward_steem_balance, accountInfo.reward_sbd_balance,accountInfo.reward_vesting_balance);

    console.log("Result from claiming rewards: ",result);
    callback();
};

Final output looks like this

Function Logs
2018-08-05 09:40:51.146 accountInfo for saragarmee:  { id: 1064920,
  name: 'saragarmee',
  owner: 
   { weight_threshold: 1,
     account_auths: [],
     key_auths: [ [Array] ] },
  active: 
   { weight_threshold: 1,
     account_auths: [],
     key_auths: [ [Array] ] },
  posting: 
   { weight_threshold: 1,
     account_auths: [ [Array] ],
     key_auths: [ [Array] ] },
  memo_key: 'STM62oYBzJPjJBq94Z6GFHw9vvrzmYDk6ub4hBA55Vs6We9gyqf8G',
  proxy: '',
  last_owner_update: '1970-01-01T00:00:00',
  last_account_update: '2018-07-08T11:49:21',
  created: '2018-07-05T10:45:42',
  mined: false,
  recovery_account: 'blocktrades',
  last_account_recovery: '1970-01-01T00:00:00',
  reset_account: 'null',
  comment_count: 0,
  lifetime_vote_count: 0,
  post_count: 147,
  can_vote: true,
  voting_power: 8075,
  last_vote_time: '2018-08-05T09:07:42',
  balance: '0.000 STEEM',
  savings_balance: '0.000 STEEM',
  sbd_balance: '0.004 SBD',
  sbd_seconds: '32223855',
  sbd_seconds_last_update: '2018-08-05T05:42:39',
  sbd_last_interest_payment: '2018-07-13T18:22:27',
  savings_sbd_balance: '0.000 SBD',
  savings_sbd_seconds: '0',
  savings_sbd_seconds_last_update: '1970-01-01T00:00:00',
  savings_sbd_last_interest_payment: '1970-01-01T00:00:00',
  savings_withdraw_requests: 0,
  reward_sbd_balance: '0.000 SBD',
  reward_steem_balance: '0.000 STEEM',
  reward_vesting_balance: '4.052593 VESTS',
  reward_vesting_steem: '0.002 STEEM',
  vesting_shares: '39493.051621 VESTS',
  delegated_vesting_shares: '0.000000 VESTS',
  received_vesting_shares: '1047361.470606 VESTS',
  vesting_withdraw_rate: '0.000000 VESTS',
  next_vesting_withdrawal: '1969-12-31T23:59:59',
  withdrawn: 0,
  to_withdraw: 0,
  withdraw_routes: 0,
  curation_rewards: 10,
  posting_rewards: 441,
  proxied_vsf_votes: [ 0, 0, 0, 0 ],
  witnesses_voted_for: 1,
  last_post: '2018-08-05T09:07:45',
  last_root_post: '2018-07-29T22:08:39',
  average_bandwidth: '51690465331',
  lifetime_bandwidth: '212692000000',
  last_bandwidth_update: '2018-08-05T09:07:45',
  average_market_bandwidth: 2533719816,
  lifetime_market_bandwidth: '16520000000',
  last_market_bandwidth_update: '2018-08-05T06:58:51',
  vesting_balance: '0.000 STEEM',
  reputation: '32694079099',
  transfer_history: [],
  market_history: [],
  post_history: [],
  vote_history: [],
  other_history: [],
  witness_votes: [ 'yabapmatt' ],
  tags_usage: [],
  guest_bloggers: [] }
2018-08-05 09:40:54.397 Result from claiming rewards:  { id: '0bde32fb548ee9b444e036a42a07f0c5e7e55b49',
  block_num: 24797836,
  trx_num: 22,
  expired: false,
  ref_block_num: 25205,
  ref_block_prefix: 1553222321,
  expiration: '2018-08-05T09:50:48',
  operations: [ [ 'claim_reward_balance', [Object] ] ],
  extensions: [],
  signatures: 
   [ '1f05ee76655cd4a75e8c013c05372bbd4c24a9966044e9cbdacf861180755358c04530653d9170203f2bfd269d41f7e7e813caef96ea7fa3f7695815c39f134815' ] }

Request ID
259ae5b3-b1a1-1a29-ac92-3420bcb751b8

So now we've verified it works in testing, we just need to deploy it and add a cloudwatch rule to call it once a day.
To deploy, just right click on the function and select "Deploy" it will spin for a moment.

Now proceed to cloudwatch

step14.png

Rules here are really easy to create...
step15.png
Click on Create Rule, then tick the "schedule" radio button. Next enter 1 in the field and use the drop down to select Day.

Now we just need to set the target which is our lambda function
step16.png

We also want the payload we used in cloud 9, this contains our account name and our posting wif key.
So check the radio button that says "Constant (JSON Text)" and paste it in there.

step17.png

And that's it! You've now built a bot to handle daily maintenance. In my next posting I will show you how to extend this to autopower up any loose steem it finds in your account, but for now I'm out of time.

Thanks for reading this and please consider tossing me some upvote love if this was helpful in any way.

As always this posting is 100% steem powered up!

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

steem-js already has bluebird promisified functions. Append Async to any function, e.g. getAccounts becomes getAccountsAsync. Use it within an async function, or with a callback().

·

Thanks for the tip. I did not know this and unfortunately I didn't see it anywhere in the documentation.
https://github.com/steemit/steem-js/tree/master/doc

However now you mention it I do see it as a dependency in the node_modules directory.
I'm going to keep using promisify because it's working (so far), and I have all the code already written and don't want to have to refactor it. I'll keep it in mind for future projects though.

Thank you so much!

·
·

Sadly it's not well documented. I discovered it by chance when someone commented about it. And now I transmit the same tip whenever I can.

Hi. I featured your blog:
https://steemit.com/payitforward/@alexbiojs/pay-it-forward-hello-world-19-week / or you can search for "Pay it forward: "Hello World!" (19 week)", my post.
This is a contest - "Pay it forward". The purpose is promotion of people with reputation 55 or lower with big potential.
Be ready to reply comments under your post from judges or other participants.
There will be some little prizes for the winners :)

·

So many uses for bots on a platform like steemit. This is really interesting and something I'll need to file away for future use as I will need to start automating some parts of @pifc as we grow.

Thanks for featuring this post @alexbiojs.

·
·

indeed, I also wanna automate some tasks in the future to save some time )

I'd love to play with this stuff, but haven't found the time yet. I can see some uses for a bot I control. I'd be careful about automatic comments as they can be seen as spam.

·

I agree with that, but it runs deeper too. People don't want to feel like they've been tricked by a machine. It's unnerving. This is why even with my own bot I'm probably going to switch her over to the "you got an upvote from me!" and leave it at that. My initial thinking was that she could leave an insightful comment during the time when it's most critical to get a comment. But I'm seeing replies, especially from people who I've told specifically "I had my bot upvote you", where I can just feel the faux paus oozing from the egg on my face. This is why I always read everything she upvotes and comments on and if someone responds, I personally carry on the conversation. However a simple "I upvoted you!" would probably serve much better in most cases.

I'm trying to respond to you over on the other thread too, but it's being slow and I don't want to spam the blockchain with the same thing over and over again.

Congratulations @saragarmee! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

Award for the number of upvotes

Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word STOP

Support SteemitBoard's project! Vote for its witness and get one more award!

Interesting! That is very patient of you to be presenting the steps in details. I hope I can find time to play around this.

Good going... Keep blogging! 😊

I came to your post because @alexbioijs featured you in his entry to our Pay if Forward Curation Contest.

I'd actually love to build a bot, but I do not know my...
"...way around AWS. (AND) If Cloud9, Lambda, Dynamo, Lex & CloudWatch (DO) sound like foreign concepts..." to me...
So if I shared with you what I wanted it to do (I think it's somewhat original, and based on my About Me) what might it take to get you to set it in motion for me? -- i.e., to build me a bot?

·

I can't really answer that without a lot more information. Some things are easy, others are hard. Right now I'm contracted to build a couple of bots to manage some miniwhales. Price is going to depend on 2 major things. Scope of work, meaning what does this bot need to do. Also "management interface", the shinier the interface the more it would cost. You can email me saragarmin@protonmail.com and tell me the details though.

·
·

Thanks, will do!

This is interesting; I'd be cautious of the auto comment however because you don't want it looking "spammy" :)
I found your post because @alexbiojs featured you in an entry to our Pay it Forward contest. You are more than welcome to join us next week with an entry of your own :)

Congratulations @saragarmee! You have completed the following achievement on Steemit and have been rewarded with new badge(s) :

Award for the number of upvotes

Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word STOP

Do you like SteemitBoard's project? Then Vote for its witness and get one more award!

Congratulations @saragarmee! You have completed the following achievement on Steemit and have been rewarded with new badge(s) :

Award for the number of upvotes

Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word STOP

Do you like SteemitBoard's project? Then Vote for its witness and get one more award!

Congratulations @saragarmee! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

Award for the number of upvotes

Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word STOP

Support SteemitBoard's project! Vote for its witness and get one more award!

You are indeed a smart lady and I am glad to read this post about bot creation. More success