Steemauto and its problems - Solved!

GitHub
PR

Summary

We had a bad week!
Our main server got an advanced DDoS attack!
Our websites and our main RPC node went down!
...
and now, after a few days, the main features are up!

Explanation

_{source: pixabay.com}
After noticing that our server is under high DDoS attack, I paused that server and moved all data to another server.
I started websites and features (curation trail, fanbase, and etc) on our backup server and backup RPC node! We experienced some new errors (related to libraries like steem-js and dsteem)!
I tried almost any method to fix these problems, and I got nothing in the result!
Today I decided to develop our customized methods to interact with steem blockchain instead of using steem-js and dsteem (thanks to the developers of these tools;) )

Our backup RPC node is appbase (v0.19.10) and needs new configurations to interact with that server directly without any middleware like Jussi (for example Jussi is installed on https://api.steemit.com to interact with appbases)

_{source: pixabay.com}
I started this Pull Request for fixing common issues we had in these days. Right now, must of the tasks completed and I'm working on the remaining tasks.
I will add more tasks to the list (if needed) to solve completely RPC node errors which are happening because of the high load of Steemauto!

Clarification

During these problems, all critical data was safe! (actually, we don't have such critical data in our servers)
I bought another server (256GB RAM) and I moved websites and backend apps to that server. Also, I started a local RPC node (v0.19.5) which is reliable than appbase (v0.19.10) for our use case!

Now, all the main services (curation trail, fanbase, schedule posts, and etc ) should work without any downtime. Enjoy using our free and unlimited services.

---

Development

I started by adding streaming methods, which are the important part of Steemauto. We should stream all latest blocks to detect recent upvotes by trails and recent posts by fans! Our recent method for streaming blocks was from steem-js which can be paused after any network error!

_{source: pixabay.com}
By adding our customized methods for streaming blocks (block numbers and block operations) streaming methods will not fail, in any case with help of our call() method, the streaming methods will try to stream the latest block.

const streamBlockNumber = async (cb) => {
  let lastBlock = 0
  setInterval(async () => {
    const result = await call(
      config.steemd,
      'condenser_api.get_dynamic_global_properties',
      []
    )
    if (result && result.head_block_number && !isNaN(result.head_block_number)) {
      if (result.head_block_number > lastBlock) {
        lastBlock = result.head_block_number
        cb(lastBlock)
      }
    }
  }, 500)
}

We used a call() method to make jsonrpc 2.0 calls directly to the our appbase. This call method will just return the result or null! by ignoring errors in the call method, we can easily retry another call by checking just returned result.

This is our call() method:

const call = async (steemd, method, params) => {
  try {
    const body = JSON.stringify({
      id: 0,
      jsonrpc: '2.0',
      method,
      params
    })
    const res = await fetch(
      steemd,
      {
        method: 'POST',
        body
      }
    )
    if (res.ok) {
      const result = await res.json()
      return result.result
    } else {
      return null
    }
  } catch (e) {
    return null
  }
}

Maybe it is not a good solution to ignore errors, but in our use case, this is the best solution! If the main node was down, we can add some extra code to change config.steemd and use another RPC node. (maybe in another contribution)

Then, we used these methods inside our apps instead of libraries like steem-js and dsteem.
By using these methods, our apps are working very well! (community confirmed)

Also, we exported similar parts of our apps to the extra files to reduce duplicate codes. For example, broadcasting upvotes and checking the voting power limit are the most used parts in all apps.
(still is not implemented in the apps)

Now, we can expect a better Steematuo!

Steem on and support Steemauto with your upvotes, resteems, donations, and witness votes ;)

_{source: pixabay.com}

Thanks for your great support

---

This post is submitted to the https://utopian.io

Regards,
2018-07-30