Steem Bot Tracker - Major Code Refactor

in utopian-io •  7 months ago

The Steem Bot Tracker website was the first Steem-related development project I've ever done. I built it in a couple days back in early October as an experiment and learning experience using the Steem APIs and to see if I could make something to help me better utilize the voting bots to promote my posts.

At the time I never imagined it would grow to be what it is today, and I also had a lot of learning to do on the best and most efficient ways of utilizing the Steem APIs. Over the past 5 months I added loads of new features on top of a messy and inefficient code base and it was finally time to go back and fix it.

What could have been done better

Previously, the entire site ran in the browser. It was just an HTML page with some JavaScript and there was no server-side component at all. This meant that all of the Steem API calls were made from each client that was viewing the site separately.

When the site first launched not many people used it and there were about 5 or 6 bid-based voting bots in total, so having each client viewing the page make its own API requests wasn't really a big deal. But now approximately 2,500 people use the site each day, with around 30 on at any given time, and there are almost 100 bid-based voting bots (which is kind of ridiculous, but we'll save that for another post).

The site reloads the voting bot info every 30 seconds, so that means that all 30 people with the site open at one time were making API calls to get information for up to 100 voting bots every 30 seconds!

While this is still probably inconsequential relative to the total traffic to the api.steemit.com node, it needed to be fixed. Additionally, when the api.steemit.com node has issues - like it did last Friday - the site wasn't able to load data which caused a number of problems for bot users and owners and there was no easy way for me to fix that other than to wait for the node to be fixed.

The fix

I have been working for the past couple weeks on updating the site to pull data from a central server instead of directly connecting to a full Steem node. This creates a number of advantages. Now the site can get nearly all of the data it needs about all of the bots from one API call to the server, as compared to potentially hundreds of calls before.

Additionally should the api.steemit.com node have issues again the server can easily switch over to another node that is working and the site will be unaffected.

On top of that, the site should now work better for the users. The list of bid bots should load nearly instantly and also update nearly instantly as you adjust the filters since what was previously a large number of Steem API calls has been reduced to one call to get data pre-loaded on a central server.

The code has been generally re-factored and cleaned up as well which should lead to fewer bugs and faster turnaround time for new features as well as making it much easier for others to contribute to the development of the project.

For bot owners

For bid-bot owners, one thing to note is that from now on the site will require that bots implement the bids API in order to display detailed bid information. Previously for bots that did not provide an API the site would try to figure out the bid details from the bot's account history, but that was never very accurate and caused too many problems by displaying incorrect data.

The following are the list of bid-bots which currently do not provide an API:

@boomerang, @minnowhelper, @discordia, @mrswhale, @hellowhale, @moneymatchgaming, @levitation, @yourwhale, @sleeplesswhale, @voterunner, @anadolu, @chronocrypto, @puppybot, @zapzap, @payforplay, @lightningbolt, @thebot

If the owners of any of those bots need help setting up the API please get in touch with me.

For bot users

This was a mostly behind the scenes change to make the site run better and faster, so for those of you who were hoping for some new cool features - sit tight, they will come soon! These changes were necessary to make it easier to develop new features and keep the site working when the SteemIt node has issues.

That being said, hopefully you notice the site running better and faster now! If you have any new features you would like to see on the site please let me know in the comments.

Thanks for your support!

As always I want to thank everyone who has helped and supported me in creating this tool. I love that it has been able to help so many people learn about and use vote selling services to promote their content and grow their following and earnings on the Steem platform. Once again if you have a suggestion or feature request I encourage you to submit an issue on the GitHub repo for the project. Please stay tuned for more updates in the coming weeks!

For those of you who are interested in looking at the code, there are too many commits to list here, but the updated code is in the "big_changes" branch which you can see here: https://github.com/MattyIce/bottracker/tree/big_changes



Posted on Utopian.io - Rewarding Open Source Contributors

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Thank you for the contribution. It has been approved.

You can contact us on Discord.

[utopian-moderator]

·

Hey @ms10398, I just gave you a tip for your hard work on moderation. Upvote this comment to support the utopian moderators and increase your future rewards!

Hey, @yabapmatt, so nice to have you on our side!

Don't you think about storing data in the database for bidbot analytics?

Thanks for the good job @yabapmatt :)
I've got one question, why those voting bots allows negative ROI? The main purpose of the bot is to help user to get traction and in the same time make a few buck or at least not loose their capital... That's how most bot owners are advertising their bots and most of the time it's completely wrong:
yesterday I saw a -72% ROI ... An other exemple here:
Screen Shot 2018-03-22 at 21.43.02.png
My suggestion is to ad one more button on the top of the site to show only non negative ROI bots, then some people can votre for them first and more bot owner might be interested to setup the non negative ROI setup on their bot.

·

That's what the ROI Limit is used for.

·
·

Thanks for your reply @therealwolf, I like to use your website, very professional and clear. When you advertise something it's not fake and the return is really good. Your bot is one of the few that actually have a limitation in terms of negative ROI, kudos for that.

·

Hey! I am the owner of the bidbot, @fishbaitbot . The problem with having a fixed non negative ROI is that people will then start using bid bots for earning instead of getting earning through good quality posts. The only aim of BidBots is to provide exposure. Paid profits on steemit is detrimental to steemit!

·
·

Hello @omstavan thank you very much for your reply. I just think that voting bots should have at least a negative ROI limitation. That's my personal opinion, just to get a system more fair :)

·
·
·

And i agree, this is kind of advertising, so there should be an upper limit. There shouldn't be a very high cost for exposure for platforms like steemit. A negative ROI limit would ensure that it doesn't cost too much!

·
·
·

Many voting boys implement that. My bot has a -10% limit on ROI. Some others also have similarly set values. These values are mentioned in the Steembottracker. You can select which bout to use from there!

·
·

This from someone who sells his vote to the highest bidder. This is comedy.

·

His site does show when the bots are showing a positive ROI. In the final ten minutes they turn green. Also popups come up for profitable bots. But I don't believe there is a filter to filter out neg ROI bots. You can however see which ones are neg as max suggested bid is above 0.

Trouble with having only positive ones show is that pretty much all the bots that are more than say 20 mins from voting show a positive ROI and then people come in toward bid time and reduce your profitability.

In the actual bot he created he put in a min ROI setting where the bot owner could stop taking bids below a certain ROI.

Some say the bots are to gain visibility. Ie upvotes = more chance of being seen. Not to make a profit. Ultimately the culture of STEEM is about creating quality content over simply going for cash.

In any case the bots face the issue of if their vote pool doesn't fill they ... don't lose money per se but they get less than their vote value

·
·

Hello peter thanks for your reply. as @therealwolf mention there is a column where you can see negative ROI limit and some boots have a 0%, I guess if you bid you are certain to receive a fair vote back.
If the bots are fair with their user they will be used more, if it's the opposite people will stop using them. You can see on the Steem bot tracker page, most voting bot have no negative RIO limitation.

Thank you again @yabapmatt. I've been on your site pretty much daily as I try to understand voting and bots.

Interestingly I did 3 tests just over a week ago where I bought votes. Over the week the estimated payout according to steemit (which I assume is in SBD) dropped by more than I would expect inflation to be. I have not worked out why. Perhaps there is an acceleration in the amount of content being produced. Certainly the new posts page is showing that.

https://steemit.com/test/@petervroom/test-1-bot-vote
https://steemit.com/test/@petervroom/test-2-bot-vote
https://steemit.com/test/@petervroom/test-3-bot-vote

·

The payout shown on steemit.com is based on the market price of STEEM so it will go down as the price of steem goes down and vice versa.

·
·

You should talk to the owner of the bot to increase the percentage of the bot since lately they are giving a 150% refund and that brings an incredible loss. Invert 5 sdl and return 7 or $ 7.5 in voting .. for that reason stop using bot ..
previously they were super profitable, they returned 200% up to 245%, but that changed.

·
·

Thank you for your generous vote @yabapmatt!

I'm so confused with the currencies! So on steemit the vote value is in STEEM not SBD? In any case the payment is in SBD/SP/STEEM and they did not change much in relationship to each other so shouldn't the the amount listed on steemit stay the same pretty much?

If it were in US$ that would be different.

Confused!

·
·
·

The payout value shown on steemit.com and other front end sites shows the current USD value of the Steem Power you will get from the post if you have chosen to receive 100% of the rewards as Steem Power.

So if the upvotes on a particular post would cause it to receive 1 Steem Power, and the market price of STEEM is currently at $2, then the payout for the post would show $2. If the price of STEEM goes up then the payout value shown goes up and vice-versa.

If you choose to take 50% of your post payout in SBD (which you always should right now) then you get half of the payout in SBD where the system assumes that 1 SBD is worth 1 USD. The fact that the actual market value of SBD is significantly different than what the Steem blockchain assumes it to be is the cause for all of this confusion.

·
·
·
·

Interesting, thank you. I will suggest they post this to the faq.

·
·
·
·

@yabapmatt are you sure steemit shows the payout in USD?

Have a look at these posts where I recorded what your site says will be paid out in SBD (ie selected SBD in the dropdown) and then what steemit said the payout would be. I think the USD to SBD/STEEM just over a week ago was something like USD $2 per Steem. Using the 30 day average it would be even higher. Yet the expected steemit payout and your calculated SBD payout were roughly the same.

https://steemit.com/test/@petervroom/test-1-bot-vote
https://steemit.com/test/@petervroom/test-2-bot-vote
https://steemit.com/test/@petervroom/test-3-bot-vote

Best

·
·
·
·
·

No, I think you're misunderstanding. What is shown on steemit.com is what I call "POST REWARDS" on the bot tracker site. If you select that from the drop down then it should match what is shown on steemit.com.

Steemit.com assumes 1 SBD is worth 1 USD, the bot tracker site values SBD at the actual market value, so that's why the numbers will be different.

We as bot owners and authors are extremely grateful for your service here Matt in helping us find the bots available and promote our posts!

Resteeming soon to help more of us using https://steembottracker.com/ to get to know you!

·

Jerry - I think we all appreciate the contribution you are doing to this platform as well.
Nice to see so many involved in making Steemit, the best social platform out there.

There was a recent change on the gui, for the "action" part, I prefer the old way with one button for bid and another one for details.
I am lazy, I prefer to do only one click :)

Can you add the column with the current ROI ?

·

Yes, you are correct - this was necessary to make room for new information that has or will soon be added to the table which is getting quite large. I will consider adding the current ROI if I can fit it. Stay tuned for more new updates coming soon!

so i just need to run a bot, to clone this repo to any server and fit the config?

If it requires Javascript and overloads the CPU in a machine that lacks a GPU, it probably mines some Proof Of Work currency.
I still believe so.
I might be wrong.
Do API calls consume so much CPU capacity?
The Javascript could be spared by loading the newest image/frame/state every time the HTML page is reloaded.

In your defense, loading a Steemit page, and scrolling in it beyond a few pages overloads the CPU too.
steemit.com is shit, and in the past filling the memo part in the wallet felt like POW too.

·

It didn’t overload the CPU and it’s certainly not mining cryptocurrency, it was just inefficient and needed a refactor.

·
·

It didn’t overload the CPU

It did.
Not as much as steem.supply and some other websites unrelated to STEEM that I know, but it did.
We may regard the word overload in different ways.
I did not mean that it clogged it completely like some of those websites.

and it’s certainly not mining cryptocurrency, it was just inefficient and needed a refactor.

I saw your acknowledgement of the inefficiency in your initial post, and added another suggestion for improvement in my comment after I edited it.

JS is evil.
I try to avoid JS demanding websites as much as possible.
I will try your website again, but compulsory JS is always bad.
My suggestion in my previous comment applies to other real time needs as well.

·
·
·

This comment has received a 24.88 % upvote from @steemdiffuser thanks to: @stimialiti. Steem on my friend!

Above average bids may get additional upvotes from our trail members!

Get Upvotes, Join Our Trail, or Delegate Some SP

·
·
·

You got a 7.25% upvote from @adriatik courtesy of @stimialiti!

·

This comment has received a 50.00 % upvote from @steemdiffuser thanks to: @stimialiti. Steem on my friend!

Above average bids may get additional upvotes from our trail members!

Get Upvotes, Join Our Trail, or Delegate Some SP

·

You got a 50.00% upvote from @greengrowth thanks to @stimialiti! You too can use @GreenGrowth by sending your post URL in the memo field to the bot. Minimum bid is 0.01.

If you feel this post is spammy or not worthy of @Greengrowth you can contact a moderator in our Discord Channel https://discord.gg/6DhnVTQ.

·

You got a 6.94% upvote from @adriatik courtesy of @stimialiti!

you are on the right track my friend. getting bots on your side is what will keep you around. your hard work is paying of as you can see your post is on trending. quite a big deal. i wish you the very best of luck

Thanks Matt, can I ask a question..

How come the Bid History doesnt seem to work anymore?

·

Does it work for anyone else?

Such a great job. Thank you so much for your contribution to Steemit!

Thanks for the good job!

I love the steemit dynamics and people like you make it worthwhile. Tanx for all your efforts @yabapmatt.

Congratulations, your post received one of the top 10 most powerful upvotes in the last 12 hours. You received an upvote from @utopian-io valued at 129.40 SBD, based on the pending payout at the time the data was extracted.

If you do not wish to receive these messages in future, reply with the word "stop".

Good post.
Thanks for share my friend

You know that's a remarkable achievement in the field of steemit. Mostly for the minnows like myself it could come real handy. Like you know it's really difficult to get going at the start, so we can use the bots for powering up. I haven't used it till now but I am thinking of trying the bot in the near future. Thank you. @yabapmatt

Truly you are a professional programmer who seems to be performing useful
Thanks for the wonderful information

i am regular visitor of steembottracker to boost my post

that's a great concept of API, you get lot of benefits, hope the mentioned bots will get API soon

Hey @yabapmatt, great post! I enjoyed your content. Keep up the good work! It's always nice to see good content here on Steemit! :)

hi i m new in steemit, can you guide me to increase my steem power

I think the only project that is really worth till now.....it really helps alot ....

Hey @yabapmatt I am @utopian-io. I have just upvoted you!

Achievements

  • WOW WOW WOW People loved what you did here. GREAT JOB!
  • Seems like you contribute quite often. AMAZING!

Community-Driven Witness!

I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!

mooncryption-utopian-witness-gif

Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x

Congratulations @yabapmatt, this post is the eighth most rewarded post (based on pending payouts) in the last 12 hours written by a Hero account holder (accounts that hold between 10 and 100 Mega Vests). The total number of posts by Hero account holders during this period was 639 and the total pending payments to posts in this category was $10585.39. To see the full list of highest paid posts across all accounts categories, click here.

If you do not wish to receive these messages in future, please reply stop to this comment.

@yabapmatt, aprecio tu trabajo por la comunidad y esta herramienta grande Steembottracker

Very good work @yabapmatt. Now I know why my tool nextvote is not working... I need to get the new bidbot database: https://steembottracker.net/bid_bots.

Thanks for the update, really needed.

WARNING! The comment below by @nirayma leads to a known phishing site that could steal your account.
Do not open links from users you do not trust. Do not provide your private keys to any third party websites.

Thanks @yabapmatt , the API setup really tough for me, but I do learn a lot while setting up..... thanks again.

Hey how are you. I somehow transferred again wrong to you. @minnowvotes can you please refund @johnskotts as well can I have your discord handle to join group. Thank you

I would like to thank all @bid-bots:
@boomerang, @minnowhelper, @discordia, @mrswhale, @hellowhale, @moneymatchgaming, @levitation, @yourwhale, @sleeplesswhale, @voterunner, @anadolu, @chronocrypto, @puppybot, @zapzap, @payforplay, @lightningbolt, @thebot
that has helped me so far. @bot has been able to help so many people learn about and use voice sales services to promote their content and grow the followers and their income in platform Steem and thanks to @yabapmatt have been here, good work, hopefully you always success and lucky
Thanks for this @yabapmatt

This has been happening for the last few days....

Screenshot_2018-03-23-20-33-55.png

Just wondering on a time frame for the fix?

·

I fixed that earlier today, sorry for the issue!

·
·

Ohhh, cuz i just got it before this posting. I will retry 😁

Edit* IT WORKS! Just had to reload the page Thanks!

That's great work you've done my friend...
It is so much easier to use bots now with the help of this website.
I haven't used much but I will surely when I get done SBD

Thanks for your update about Steem Bot Tracker
Very valuable info. Will continue checking your content

upvoted (not that my vote really matters that much)

cheers, Piotr

Really grateful for this service, as a newcomer to Steemit this safe and secure way of upvoting has meant I have been able to strive forward with my account to a point where I can now help others with upvoting on their work, it all comes back around in the end and hats off for taking the time out to develop and improve this site.

Hiii @yabapmatt, very impressive and helpful project you are running. I have issue that about 12 plus hours ago I send a bid of 4.510 SBD to @boomerang. but still not received Upvote or Refund. I sent several message but no reply. So now I decided to take help from you because boomerang is working under your steem bot tracker. So Plz help me to solve this problem .I shall be much obliged.
Thanks.
Here is the proof of Transfer.
Boomerang Transfer.jpg

·

I will try to get in touch with the owners of @boomerang to let them know about this and hopefully get you a refund.

·
·

Thanks a lot @yabapmatt. I got it.

Nice words. I admire how you always post very positive things. I salute you!

Thanks for your excellent work. It is certainly a great help for new users

Nice piece of work, it's helping all of us minnows get their content out.

This post has been just added as new item to timeline of Steem Bot Tracker on Steem Projects.

If you want to be notified about new updates from this project, register on Steem Projects and add Steem Bot Tracker to your favorite projects.

How can i get one?

WARNING - The message you received from @nirayma is a CONFIRMED SCAM!
DO NOT FOLLOW any instruction and DO NOT CLICK on any link in the comment!
For more information, read this post: https://steemit.com/steemit/@arcange/phishing-site-reported-www-steemitfollowup-dot-cf
Please consider to upvote this warning if you find my work to protect you and the platform valuable. Your support is welcome!

We as bot owners and authors are extremely grateful for your service here Matt in helping us find the bots available and promote our posts!

Resteeming soon to help more of us using https://steembottracker.com/ to get to know you!

It's a very engaging platform. It's almost like a game. with the dip, things have been less interesting but I'm sure we'll be back to bigger numbers :)

Thanks for this, I’m relatively new and am not getting traction. Didn’t understand bots until reading some of your posts. I was against them, but it may be useful for newbies until getting noticed. Resteeming...

Awesome stuff! Im totally new to this promotion on steemit, just testing and see how it works. The website is very helpful. Thank you for your time putting this all together!

Thanks for such an outstanding service of steembottracker. I have used many times this service.

Thank you very much for the information, I am new to steemit, and this helps me to understand more how the page works. Grateful and I will follow you to see your next publications. regards

I really need your assistance.

guys can i get upvotes because i am a new user i really need help to grow guys

Its a great start and hope it continues to raise high in ststus

Hello

How are you?
i want to sell my upvote in steembottracker
What should I do?

Pleasa vote my post