Scraping and Process Improvements

in #utopian-io6 years ago (edited)

Link to Pull Request

closes #48

New and Improved Features

Scraping process

This pull request packages up everything involved with scraping into one function. Previously one would have to run the scrape function to read new posts on the blockchain. Then run comment() to comment and vote() to vote. With the need for an approve() function it made more sense for these to be all in the same function. The scrape function now looks like:

const allPosts = await this._blockchainAPI.getPosts(this.communityName)
const results = await this._database.writePosts(allPosts)

await this.approve()
await this.comment()
await this.vote()

Now the process is synchronous so everything can be done in one iteration unlike before when you may have approved something after you ran the comment function, and thus have to wait until the next iteration to comment on the newly approved post.

Some logging was added to make things cleaner:
image

Approval Function

The approval function has been needed for a while now that we need some sort of spam filter. The approve function checks that the author of the post has voted on this week's weekPost. The rules state that you must like this post. The is_approved flag in the database is switched which tells the vote method that it should vote on this post.

Now Playing

Check out the #nowplaying community and bot in action here



Posted on Utopian.io - Rewarding Open Source Contributors

Sort:  

Thank you for the contribution. It has been approved.

You can contact us on Discord.
[utopian-moderator]

Hey @walnut1 I am @utopian-io. I have just upvoted you!

Achievements

  • You have less than 500 followers. Just gave you a gift to help you succeed!
  • Seems like you contribute quite often. AMAZING!

Community-Driven Witness!

I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!

mooncryption-utopian-witness-gif

Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x

Coin Marketplace

STEEM 0.27
TRX 0.12
JST 0.031
BTC 68870.08
ETH 3734.86
USDT 1.00
SBD 3.73