DBlog.io - Decentralized Blogging Platform - Backend improvement

_{source: pixabay.com}

GitHub repo: https://github.com/mahdiyari/dblog-backend
Commit1: link
Commit2: link
Commit3: link
Commit4: link
Small fix: link

Recently I announced a new opensource project: https://steemit.com/onsteem/@mahdiyari/announcement-onsteem-a-better-interface-on-the-steem-blockchain

I started developing back-end for this project called Dblog.io (new name)!
We will have a back-end here and a front-end here

A demo will be available here: https://dev.onsteem.com (back-end and front-end connected)

---

Development

_{source: pixabay.com}

We need to keep all posts and comments from our users in the database to provide faster experience to the visitors!
After the first login, a function will be called to start syncing user comments and posts. This function will make POST requests to an RPC node then it will insert all posts and comments to the database. We will keep all the posts in the posts table and all comments in the comments table in the MySQL database.
Of course, we can't insert all post's data into the database because of storage limits, we will keep only constant information which is not changeable! like permlink, author, id, category, created, and etc

MySQL structure is also updated: link to the commit

_{source: pixabay.com}

Later we can sort posts by timestamp and categoryand detect similar posts by using category!

After syncing all posts and comments from any user, we will need to stream blocks for the coming posts and comments. This job is done by /stream_sync.js

We used a simple array for saving recent posts and comments to prevent from adding duplicate values to the database!
Because MySQL INSERT is not fast enough. If a user broadcasts multiple edit posts, we may add them multiple times to the database.
I explained much about this in the codes:

// we will use this function to save recent posts in the 'posts[]'
// if a user broadcasts multiple 'edited posts' to the blockchain
// we may add same post, multiple times to the database
// because INSERT in MySQL is not fast enough to insert data
// and that may process multiple INSERTs at the same time
// we can check posts against 'posts[]' variable
// to ensure there will not be any duplicate posts in the database
// this function will return a number from 0 to 500 as index (posts[index])
// to create a pool of posts to keep recent posts and delete older posts
let index = 0
let maxIndex = 500
const postsIndex = async () => {
  if (index < maxIndex) index += 1
  else index = 0
  return index
}

Then we can use following code to save post's id in the posts array:

let index = await postsIndex()
posts[index] = comment.id // or post.id

Recent posts and comments will be added to the posts array from index 1 to the 500, then index will be changed to 0, and we will replace old posts with newer posts!
We can check for duplicate values by using indexOf() method which is much faster than checking database.

I used an ExpressJS server for receiving new registered users by a post request which is faster than checking the database every second!

I will work on the new APIs to display posts in the front-end with some extra features!

---

_{source: pixabay.com}
This contribution submitted to the https://utopian.io
Support this new project with your contributions, upvotes, and witness votes. Thanks for your great support:)

Regards,
2018-07-16