Introducing: Steem Engine QL - An open source GraphQL layer for Steem Engine

in #utopian-io5 years ago (edited)

Repository

https://github.com/Vheissu/SteemEngineQL

Steem Engine QL

If you're familiar with Steem Engine, it is a Steem blockchain sidechain that allows the creation of custom tokens. While Steem Engine itself provides an official library for working with Steem Engine in the form of sscjs and a few other non-official options, admittedly it gets quite chatty when you start making requests.

This GraphQL layer aims to make it easier to request Steem Engine data in bulk without the need for constant requests causing a chatty application.

Even if you're not familiar with GraphQL, it offers an alternative means of interacting with the Steem Engine platform, querying information about tokens and orders. At present, it only allows read-only operations, in future additional mutations support will be added. If you're curious, the GraphQL website offers a great explainer and examples of what GraphQL is.

If you're an application developer and you're not a fan of having to dig into the codebase to see how SSCJS calls can be made and prefer a more natural object-like query language which results in faster requests, this is the package for you.

Under the hood

The codebase is simplistic and easy to extend. If you have your own custom smart contracts, you can easily add in resolvers, schemas and other types of GraphQL construct in to support them.

Steem Engine QL is actually comprised of quite a few libraries all put together to create a cohesive GraphQL solution for working with Steem Engine.

Steem Engine QL under-the-hood utilises some of these popular libraries:

The primary motivation behind creating this was seeing how "chatty" Steem Engine applications can get, especially if you're making more than a couple of calls. If you go to the market page on Steem Engine, you might have noticed it takes upwards of 7-10 seconds to load the page.

The reason for this is because this page specifically requires a lot of information, it needs the buy and sell books, it needs user specific pieces of information and more. Compounded by the fact these requests are HTTP/1.1 instead of HTTP/2 and you get bottlenecks.

image.png

In the process of writing this post, I actually discovered there is a bug in Steem Engine's UI itself and it is doubling the same request in some cases. However, of those requests, there are twelve unique requests for data to populate this view.

  • sscstore params
  • tokens params
  • tokens (limit 1000 and offset 0)
  • market metrics (limit 1000 and offset 0)
  • steem-peg balance call
  • balances for logged in user (limit 1000 and offset 0)
  • buy book (symbol "MARKET", limit 200 and offset 0)
  • sell book (symbol "MARKET", limit 200 and offset 0)
  • trades history (symbol "market", limit 30 and offset 0)
  • user buy book (symbol "MARKET", account "beggars", limit 100 and offset 0)
  • user sell book (symbol "MARKET", account "beggars", limit 100 and offset 0)
  • user token balances (symbol "MARKET", account "beggars", limit 2 and offset 0)

To get that same amount of information, you could run the following queries (in the playground linked below) and it's much faster, it's also one network request. I apologise for the wall of text you're about to see. There are ways you could make this smaller using fragments and other GraphQL features.

{
  sscstore {
    priceSBD,
    priceSteem,
    quantity,
    disabled
  },
  
  tokenParams {
    tokenCreationFee
  },
  
  tokens(limit: 1000, offset: 0) {
    issuer,
    symbol,
    name,
    metadata {
        url,
      icon,
      desc
    },
    precision,
    maxSupply,
    supply,
    circulatingSupply
  },
  
  metrics(limit: 1000, offset: 0) {
    symbol,
    volume,
    volumeExpiration,
    lastPrice,
    lowestAsk,
    highestBid,
    lastDayPrice,
    lastDayPriceExpiration,
    priceChangeSteem,
    priceChangePercent
  },
  
  steempBalance {
    account,
    symbol,
    balance
  },
  
  balances(account: "beggars", limit: 1000, offset: 0) {
    account,
    symbol,
    balance
  },
  
  buyBook(symbol: "MARKET", limit: 200, offset: 0) {
    txId,
    timestamp,
    account,
    symbol,
    quantity,
    price,
    tokensLocked,
    expiration
  },
  
  sellBook(symbol: "MARKET", limit: 200, offset: 0) {
    txId,
    timestamp,
    account,
    symbol,
    quantity,
    price,
    expiration
  },
  
  tradesHistory(symbol: "MARKET", limit: 30, offset: 0) {
    type,
    symbol,
    quantity,
    price,
    timestamp
  },
  
  userBuyBook: buyBook(symbol: "MARKET", account: "beggars", limit: 100, offset: 0) {
    txId,
    timestamp,
    account,
    symbol,
    quantity,
    price,
    tokensLocked,
    expiration
  },
  
  userSellBook: sellBook(symbol: "MARKET", account: "beggars", limit: 100, offset: 0) {
    txId,
    timestamp,
    account,
    symbol,
    quantity,
    price,
    expiration
  },
  
  tokenBalance(symbol: "MARKET", account: "beggars") {
    account,
    symbol,
    balance
  }
}

The above queries which replace all 12 of those market page requests result in just one single request which returns all of the data for me in under 2 seconds. Combined with caching and other strategies, you could theoretically make this number a lot faster. Right now, calls are just being mapped behind-the-scenes to the SSC SDK itself.

image.png

Demo and Playground

The linked GitHub repository has all available GraphQL query methods documented, which you can try out in the playground here: https://graphql.steem.services - all you need to do is simply copy and paste one of the example queries in the GitHub repository into the query environment to see real data come back.

All queries are operating on the Steem Engine mainnet. The source code is on GitHub here.

Roadmap

I plan on adding in support for Steem itself as well, and the ability to perform specific market actions like buying and selling through the GraphQL layer. Mutations have security considerations, so the initial release you see does not have them, but they are eventually planned.

Another planned feature is to add in support for caching and batching resolver calls to SSC. At present, if you make a lot of calls, it can still take a while for them to resolve. Some of the calls being made can be cached for a short period of time to reduce network latency.

Issues/Bugs/Feedback

As always, if you encounter any issues or have feedback, please create an issue on the repository https://github.com/Vheissu/SteemEngineQL/issues and it will be looked at. Forks and contributions to make this library even better are also greatly appreciated and welcomed.

GitHub Account

https://github.com/Vheissu

Sort:  

I love how graphql simplies the traditional complex/multi request flows in a single payload. I will be playing with this tool for the upcoming bot ideas running on steem-engine.

The post is detailed and the problem and solution is well-defined. Hope to see you more in the development category :)


Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Chat with us on Discord.

[utopian-moderator]

Love your work @emrebeyler cheers for the great review. I am hoping to expand the scope of this in the coming week or so. I'll be using it on my Steem Marketplace idea as well.

Magic Dice has rewarded your post with a 43% upvote. Thanks for playing Magic Dice.

Thank you for your review, @emrebeyler! Keep up the good work!

Hey, @beggars!

Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

Hi @beggars!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

i dont know what all this means but i support anything that makes things easier for people involving large projects such as steem-engine

Magic Dice has rewarded your post with a 94% upvote. Thanks for playing Magic Dice.

Congratulations @beggars! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

You made more than 800 comments. Your next target is to reach 900 comments.

You can view your badges on your Steem Board and compare to others on the Steem Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

Do not miss the last post from @steemitboard:

The Steem blockchain survived its first virus plague!
Vote for @Steemitboard as a witness to get one more award and increased upvotes!

Coin Marketplace

STEEM 0.18
TRX 0.13
JST 0.028
BTC 57346.65
ETH 3107.45
USDT 1.00
SBD 2.40