DBlog.io - Decentralized Blogging Platform - Backend improvement

in utopian-io •  2 months ago

diary-968603_640.jpg
source: pixabay.com

GitHub repo: https://github.com/mahdiyari/dblog-backend
Commit1: link
Commit2: link
Commit3: link
Commit4: link
Small fix: link

Recently I announced a new opensource project: https://steemit.com/onsteem/@mahdiyari/announcement-onsteem-a-better-interface-on-the-steem-blockchain

I started developing back-end for this project called Dblog.io (new name)!
We will have a back-end here and a front-end here

A demo will be available here: https://dev.onsteem.com (back-end and front-end connected)


Development

programming-1873854_640.png
source: pixabay.com

We need to keep all posts and comments from our users in the database to provide faster experience to the visitors!
After the first login, a function will be called to start syncing user comments and posts. This function will make POST requests to an RPC node then it will insert all posts and comments to the database. We will keep all the posts in the posts table and all comments in the comments table in the MySQL database.
Of course, we can't insert all post's data into the database because of storage limits, we will keep only constant information which is not changeable! like permlink, author, id, category, created, and etc

MySQL structure is also updated: link to the commit

ring-binders-aligned-2654130_640.jpg
source: pixabay.com

Later we can sort posts by timestamp and categoryand detect similar posts by using category!

After syncing all posts and comments from any user, we will need to stream blocks for the coming posts and comments. This job is done by /stream_sync.js

We used a simple array for saving recent posts and comments to prevent from adding duplicate values to the database!
Because MySQL INSERT is not fast enough. If a user broadcasts multiple edit posts, we may add them multiple times to the database.
I explained much about this in the codes:

// we will use this function to save recent posts in the 'posts[]'
// if a user broadcasts multiple 'edited posts' to the blockchain
// we may add same post, multiple times to the database
// because INSERT in MySQL is not fast enough to insert data
// and that may process multiple INSERTs at the same time
// we can check posts against 'posts[]' variable
// to ensure there will not be any duplicate posts in the database
// this function will return a number from 0 to 500 as index (posts[index])
// to create a pool of posts to keep recent posts and delete older posts
let index = 0
let maxIndex = 500
const postsIndex = async () => {
  if (index < maxIndex) index += 1
  else index = 0
  return index
}

Then we can use following code to save post's id in the posts array:

let index = await postsIndex()
posts[index] = comment.id // or post.id

Recent posts and comments will be added to the posts array from index 1 to the 500, then index will be changed to 0, and we will replace old posts with newer posts!
We can check for duplicate values by using indexOf() method which is much faster than checking database.

I used an ExpressJS server for receiving new registered users by a post request which is faster than checking the database every second!

I will work on the new APIs to display posts in the front-end with some extra features!


calligraphy-2658504_640.jpg
source: pixabay.com

This contribution submitted to the https://utopian.io
Support this new project with your contributions, upvotes, and witness votes. Thanks for your great support:)

Regards,
2018-07-16

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  
·

I wonder to see what will happen if I upvote that bot!

Thank you for your contribution.

  1. magic numbers such as if (nextPosts.length === 100) { startPermlink = nextPosts[99].permlink; startAuthor = nextPosts[99].author }
  2. why not index = (index + 1) % maxIndex instead of if (index < maxIndex) index += 1 ; else index = 0
  3. why not use something like mongodb for better scaleablity. I am sure the table will become very huge and there will be performance issues later.
  4. What is the innovation for this project? I have seen similar projects.

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

I want to know what time will you add voting according to Tag.

·

I'm working on that

hello sir I'm following you and upvoting every post of your please make me 1 upvote please thank you

wow great sir..

Plz chek my new post in my blog..thanks for support advance.

Nice blog

Hey @mahdiyari
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

Quick question on SteemAuto for you: Is there a way to block SteemAuto from voting on an author's own post? I don't want to be self-voting, but I would like to follow a trail that potentially will vote for me.

hello bro i want to contact you so please msg me on telegram https://t.me/Jagsarathore