A Look At The Upcoming New Steem Engine Dex

in #steem-engine7 years ago (edited)

Hi there. As some of you know, I have been leading the effort on the new Steem Engine DEX for the past few months. If you have ever used the current Steem Engine exchange, then you would be aware of its shortcomings and missing features.

One of the biggest complaints about the current DEX is speed. When you actually look underneath the hood of the current DEX, you will see that there is a lot of data needed to render some of the views. Opening up developer tools on the exchange view, for example, you will see request after request.

Other annoyances include having to click twice when signing in to the exchange (this annoys me too) and other various little caveats which sodden the experience a little bit.

Everything Reimagined

While the current exchange has served us well, Steem Engine 2.0 is totally rebuilt from the ground up. At a glance, this is what is different on a technical level.

  • Developed using the Aurelia Javascript Framework, using TypeScript and modern web API's
  • A contributor friendly codebase with a neatly organised structure, naming and conventions to find parts of the application.
  • ESLint for code linting (ensure contributions are all to the same standard)
  • Prettier for code formatting and standardisation
  • New JSON Web Token (JWT) authentication layer (you actually log in to the DEX now)
  • State management for handling client-side data
  • HTTP/2 enabled API and resource requests to take advantage of multiplexing and other features in the updated HTTP specification which speed up web application performance.
  • GraphQL for smarter data fetching. The site will make a smaller amount of requests for needed data. A data fetching cache layer where possible has been introduced to lessen the roundtrip latency of some queries.
  • Eager and optimistic loading. Instead of making users wait for data to load, views will intelligently load data in a micro-app fashion where individual components on the screen load independently of one another.
  • CSS Modules for safer styling of UI components
  • Covered in unit tests (this is an ongoing effort) to produce smarter and safer deploys

Speed

We touched upon this in the opening paragraph, but the current DEX admittedly is slow at times. In the new DEX, we focused on consolidating requests and leveraging the power of GraphQL for instances where multiple sets of data are needed.

In most screens (especially the request heavy exchange view) we have been able to reduce the number of requests upwards of 80%. One such improvement is on the exchange view, two requests were being made; one for every single token with a limit of 1000 and one for every single token metric with a limit of 1000.

These two requests were absolutely massive from not only a payload perspective, but they're not cacheable because the SSC Javascript package uses POST requests to make RPC calls to the API. They contributed to the slowdown, so I set out to rework this in the new DEX.

Instead of making every token request, it should only request the current token it is on as well as the token it is paired with. At present, we only have STEEMP as the main pairing token, but there will be other pairs in the future. Why load all of those hundreds of other tokens if you only need one? In this one area, the speed of the data fetching request was increased 100x, a fraction of the payload size.

The general approach is less is more. Only querying for data that we need, when we need it. Instead of doing the filtering on the client, leveraging the API and its support for querying parameters to only request and filter what we need, passing it back to the API.

This is still ongoing work, but early results are already impressive.

100% TypeScript

While TypeScript does not guarantee complete safety, it offers some type of safety when it comes to working in a growing and distributed codebase. As the number of contributors grows, it is important that contributors are able to find their way around the codebase and the code documents itself.

Thanks to intellisense and types, anyone working on the Steem Engine codebase can be confident they're providing the right values to functions, get documented for arguments on functions and other features that TypeScript offers.

JSON Web Token (JWT) authentication layer

With the current DEX, you do not technically log in to the application. As many discovered, the username is just stored in localStorage and calls are made through Keychain or Steem Connect. In the new DEX we needed a way to verify users through a stronger authentication process, so JWT authentication was introduced creating authentication tokens using a series of token swaps and handshakes to ensure a user is who they say they are.

The reason for this change is KYC. While KYC is not a requirement, some upcoming features of the site will require users undergo KYC and we needed a safe way for users to upload KYC documents securely and verify that they are the only ones uploading their documents.

The introduction of JWT auth does not mean all operations will not ask for your permission. The Steem Keychain or Steem Connect service will still be used to authorise each and every single blockchain action, keeping you in control of your account and funds.

State management

When you are building large-scale web applications, client-side state management becomes crucial. When you are dealing with the troves of data that Steem Engine is, even more so. The new DEX is utilising state management in all areas from tokens to metrics and even the logged in user.

Data freely moves around the application in a safer manner, with the ability to trace the path of data and determine how, when and why it changed.

GraphQL

At present, the DEX uses a series of RPC calls via the sscjs client. While this works fine, it has limitations which prevent basic things such as caching being implemented. It also offers up challenges in terms of the number of requests being made to get data.

With GraphQL, we can query the Steem blockchain and Steem Engine specific data and bring it back in as many requests as we want. As mentioned in the speed section above, we are leaning on it for the exchange view to speed things up quite considerably.

GraphQL is the future of API's and integrating it with Steem and Steem Engine seemed like a worthy task. If you want to run your own GraphQL server for Steem and Steem Engine, the code is available for free here.

Disclaimer

Before getting into the UI parts of the new app, I just want to point out all of this is still a work in progress. While technically the exchange is quite complete now, there are still some UI parts that need some more thought and tweaking. What you see here might not look the same as the final version.

Localisation Localization 현지화 本土化

Some of you might know, but I am Australian and in Australia we speak British English. So, unlike our American friends we use "s" a lot in place of "z". The same thing goes for those in New Zealand, Great Britain and other Commonwealth countries that speak English.

Understandably, not everyone speaks English and not everyone likes being forced to use British or American style English. So, in the new DEX we've completely localised the UI so you get to decide. At launch we are going to have support for the following languages:

  • American English
  • British English
  • Korean
  • Chinese

Here is what the exchange UI looks like in Korean.

How about the tokens screen in Chinese (Simplified)?

The localisations in the app are not perfect, at times they might be wrong. This is where YOU come in. Do you speak a language other than English and would you like to see it supported in the new Steem Engine? We want your translations and improvements.

The process has been made as simple as possible for contributing locales. It's a simple JSON file with the key (original spelling and then the translation: https://github.com/steem-engine-exchange/steem-engine-dex/tree/master/src/locales

User Settings Screen

This is a feature that has been a long time coming, a user settings panel for Steem Engine. This area will allow you to manage personal information (part of the newly introduced KYC process), as well as various settings on the Steem Engine exchange relating to your account.

You will be able to hide zero balances from your wallet. You will also be able to toggle a mode which only shows your favourited tokens on the wallet screen. If you're like me, you're not really concerned with most tokens except for a select few, it will make things easier. These settings can also be changed on the wallet screen itself.

Favouriting tokens is as easy as typing the token name into the input area and selecting it. You can choose multiple tokens that you want to mark as favourites. You can also favourite tokens from the tokens page as well as the wallet screen as well.

New Exchange UI

If you watched Aggroed's Steemfest talk, then this screen will already be familiar to you. It is the most significant part of the new DEX where most of the work has been done on a technical level. The new exchange screen allows you to buy and sell Steem Engine tokens faster and with more information than ever before.

The biggest addition here is candlestick charts allowing you to visualise the rise and fall of your favourite Steem Engine tokens. The data only gives us a daily glance for now, but work is being done to allow for smaller intervals such as 5 minute, 15 minute and 1-hour charts.

There is still some work to be done on this screen in terms of colours, typography and spacing, so excuse the UI quirks that are visible for now.

New Wallet UI

The new wallet UI looks familiar as it is based on the old DEX design, only it has been given a refresh and had some improvements made. The icons now all display correctly and even on smaller screens.

Other Changes

I could go on all day talking about the new changes in the upcoming DEX, but we'll summarise here with a few other smaller (but still important points).

No test? No merge

The previous DEX had no tests or process for handling user contributions, so there were instances where contributions would sometimes break something on the DEX. While these contributions were tested, it can be impossible to know of side-effects in your application if there are no tests.

Going forward, while we still invite the community to make contributions to the DEX and we hope that the new codebase makes this not only easier but more straightforward, we will have a "no test, no merge" policy where all changes require accompanying tests and ensure that existing tests still pass.

This is an essential change as we have moved to a continuous deployment process where work is managed via GitFlow and then automatically released through Travis CI off of the master branch.

Continuous Deployment

Leading off of the last point, the new DEX is continuously deployed through Travis CI. Now that we have a proper testing process in place, we can confidently (to a certain degree) be sure that the DEX is in a functional and working state if the tests are passing.

The previous DEX required manual deployment, and admittedly, the process was flawed, allowing for potential errors in the deployment process.

Want to Contribute?

The code is public here on GitHub and anyone who would like to contribute, please reach out to me on Discord because the more the merrier. My Discord handle is beggars aka T-RexOfBinance#4504.


Stay tuned for part two where I will dive into some deeper technological aspects of the new Steem Engine.

Sort:  

There were a lot of words and hard ones.
But I like the picture of the new dex! :D

I AM REALLy the first comment?!?!?!

Instead of making every token request, it should only request the current token it is on as well as the token it is paired with. At present, we only have STEEMP as the main pairing token, but there will be other pairs in the future. Why load all of those hundreds of other tokens if you only need one? In this one area, the speed of the data fetching request was increased 100x, a fraction of the payload size.

YES PLEASE! I asked for this for BTCP THANK YOUI!!! I cannot WAIt to use BTCP as a base pair :D Like DNA man, AND I CAN ACTUALLY do a deposit of BTCP and trade it for a token, and withdraw it, and i can use a token to buy btc and withdraw BTC directly imagine... im also getting BTCP on EOS so our steem engine tokens are pegged on eos as well, with natural human arbitrage pegging the price thanks to steem engine smart contracts

I cant wait to do a SAND token apir for all my local tokens :D

I cannot wait to see stable coin pairs

stable coin would be awesome :)

More coin pairs is going to be a huge addition to the site, 100%. I am excited to see this thing really expand beyond its current scope. We are only at the start right now, things are going to get huge soon.

Looks great! Can't wait to see and use it live :D Keep up the good work!

Thank you for sharing DEX update news. I'm the one of staff member in sct community and have created PR for korean translaiton improvement. https://github.com/steem-engine-exchange/steem-engine-dex/pull/117 It would be helpful to integrate with crowdin platform keeping maintain localization

I have already commented somewhere about new S-E, but I will ask again... Is it possible to integrate the part where we can see our history of trades for selected token pair? That would be awesome!
Thanks!

Wow what I have been waiting for so we could get good insight of what is actually Going on in the market. Nice update especially with smt at the side train, massive development on the steem blockchain and hoping for price to moon in the shortest period on time ☺

Absolutely. There are a lot of good things happening on Steem right now. Steem Engine is going to be getting NFT's shortly as well, so that will open up the flood gates even more. It is definitely the right time to try and snag yourself some cheap ENG tokens off of the market, they're going to have way more utility soon once we start opening up the new platform.

Nice will add it to my todo list hoping to shot for 2000eng token

You will be able to hide zero balances from your wallet. You will also be able to toggle a mode which only shows your favourited tokens on the wallet screen. If you're like me, you're not really concerned with most tokens except for a select few, it will make things easier. These settings can also be changed on the wallet screen itself.

This is my favourite part of the upcoming update -- looking forward to being able to hide those zero balances and select some favourites!! :D

I though we were going to be stuck with Version 1.0 forever. It has served us well despite its many quirks that irritate me so much. Loving the new UI.

Nice. It has been really annoying trading without seeing historical price history via candlesticks. The order book depth chart is not enough to trade with 🤣🙈

Some great updates to the dex. Hiding the tokens and upping the speed are massive in my opinion.

Coin Marketplace

STEEM 0.04
TRX 0.33
JST 0.105
BTC 64684.13
ETH 1913.44
USDT 1.00
SBD 0.37