ARK Utilities — Making the ARK Ecosystem Faster

ARK Utilities is a separate library providing common functions for working
with data (sorting, filtering, …) and performing common tasks. All functions in
ARK Utilities will make use of our custom designed benchmark framework to ensure
it is the best candidate in each respective area.

ARK Utilities significantly increases speed and performance across the entire
ARK Core, and this post will go into technical details while also showing actual
test percentage increases.

ARK Core has come a long way over the past year. With the new codebase securely
in place and a proper foundation set for our future development efforts, we can
now focus on optimizing some of its functionality for better overall
performance. As such, we have created a new dedicated library named ARK
Utilities with the aim to provide a wide selection of commonly used algorithms
tailored to performance.

**ARK Utilities is available at:
**https://github.com/ArkEcosystem/utils

By introducing this new layer of dependency, we are able to replace many third
party dependencies in Core with @arkecosystem/utils as those will now be part
of the ARK Utilities library. Additionally, we can now provide a common API.
This allows us to swap out or make changes to the implementations as we identify
performance issues and other shortcomings, without affecting the API.

ARK Utilities is written in TypeScript and exposes commonly used functions while
hiding the concrete implementation. It will fully be utilized with our upcoming
Core v2.1 update, and will gradually get additional utilities as we introduce a
series of optimized functions. This will also help other (non-ARK) projects to
make use of this library as it will be independent from our Core code.

We would like to invite any and all projects who want to help improve the
overall scope of commonly used crypto functions and libraries to suggest
improvements, open GitHub issues, or provide pull-requests if they have ideas
for how we can improve the library. We will be laying the first bricks with our
upcoming release and we hope others in the greater blockchain community will
help us continue to build ARK Utilities into a powerful resource for the entire
industry.

What are some of these functions?

A lot of our code is using the popular lodash library. As many of you may
already know, lodash is not exactly known for having the best performance, so we
started by replacing a few functions with alternatives which proved to be faster
in our benchmarks. Here are a few examples which will be part of the new ARK
Utilities:

• ARK Utilities leverages a few algorithms from the
  Fast.JS library which provides higher
  performance versions of many common JavaScript functions (map, filter, …).
• For sorting we switched from Lodash’s sortBy to Fast
  Sort. As we expose the same API as
  before, we only have to change the import statement to benefit from it.

While we are still benchmarking and measuring, we can say that so far everything
is faster than lodash.

That sounds great in theory, but give us some actual numbers!

Concrete numbers can better demonstrate the actual level of improvement so here
are some of the performance results we have seen. These numbers show the
increase in several different functions and were taken from a real working
environment, and data from the actual ARK Network. We would like to note that
these improvements become even more evident with larger data sets and the %
increase in performance in some instances goes up by as much as 12x.

✓ utils x 20,166,554 ops/sec ±2.98% (78 runs sampled)

✓ lodash x 402,498 ops/sec ±1.33% (88 runs sampled)



✓ utils x 1,842,682 ops/sec ±1.73% (81 runs sampled)

✓ lodash x 760,114 ops/sec ±1.73% (82 runs sampled)



✓ utils x 410,181 ops/sec ±1.23% (87 runs sampled)

✓ lodash x 151,037 ops/sec ±1.58% (85 runs sampled)



✓ utils x 52,281 ops/sec ±0.68% (93 runs sampled)

✓ lodash x 17,441 ops/sec ±1.55% (89 runs sampled)



✓ utils x 5,282 ops/sec ±0.48% (91 runs sampled)

✓ lodash x 1,530 ops/sec ±2.43% (83 runs sampled)



✓ utils x 538 ops/sec ±0.80% (89 runs sampled)

✓ lodash x 80.41 ops/sec ±1.65% (64 runs sampled)



✓ utils x 1,921,146 ops/sec ±1.33% (87 runs sampled)

✓ lodash x 723,453 ops/sec ±2.61% (78 runs sampled)



✓ utils x 325,372 ops/sec ±0.52% (92 runs sampled)

✓ lodash x 153,786 ops/sec ±1.49% (87 runs sampled)



✓ utils x 34,702 ops/sec ±0.92% (91 runs sampled)

✓ lodash x 16,023 ops/sec ±1.37% (86 runs sampled)



✓ utils x 3,562 ops/sec ±0.46% (93 runs sampled)

✓ lodash x 1,529 ops/sec ±2.14% (83 runs sampled)



✓ utils x 363 ops/sec ±0.47% (88 runs sampled)

✓ lodash x 71.07 ops/sec ±2.94% (58 runs sampled)



✓ native x 23,282,872 ops/sec ±1.55% (80 runs sampled)

✓ utils x 41,298,044 ops/sec ±1.95% (81 runs sampled)

✓ lodash x 6,841,156 ops/sec ±1.17% (87 runs sampled)



✓ native x 2,411,309 ops/sec ±1.09% (89 runs sampled)

✓ utils x 5,051,863 ops/sec ±4.03% (84 runs sampled)

✓ lodash x 751,676 ops/sec ±1.61% (87 runs sampled)



✓ native x 245,745 ops/sec ±3.92% (77 runs sampled)

✓ utils x 458,229 ops/sec ±2.00% (85 runs sampled)

✓ lodash x 80,584 ops/sec ±1.19% (90 runs sampled)



✓ native x 28,034 ops/sec ±1.44% (87 runs sampled)

✓ utils x 45,515 ops/sec ±1.35% (86 runs sampled)

✓ lodash x 8,094 ops/sec ±0.95% (90 runs sampled)

Here is another example to show this increase at scale. We increased the sample
number to higher numbers and tested sorting 100,000 wallets in descending order.
In this case, the utilities function is **639.44% faster than lodash. As
**we continue to grow, the performance gains will only get more noticeable.

    ✓  utils x 533 ops/sec ±0.48% (87 runs sampled)
    ✓  lodash x 72.12 ops/sec ±2.40% (59 runs sampled)

How do I use it?

**ARK Utilities is available at :
**https://github.com/ArkEcosystem/utils

Install as a dependency:

yarn add @arkecosystem/utils

And import a function from @arkecosystem/utils:

import { sortBy } from “@arkecosystem/utils”;

---

With ARK Core now forming a solid foundation to build on, we are excited to
start introducing a series of key optimizations that will increase efficiency
and improve performance levels, making the ARK Platform even more powerful.
While these are just the initial improvements, we have already seen a drastic
increase in performance and intend to continue to improve our ARK Utilities
library.

If you are an ARK community developer or a blockchain enthusiast, head over to
our Github and start contributing now to help us improve these tools. Don’t
forget to check out our newly updated Github Bounty
Program
to find out how you can earn ARK by supporting our development efforts and
contributing to the code!

---

ARK.io | Github | Facebook | Twitter | Forum | Blog | Explorer | Shop | Discord | YouTube