RE: Using Intel Optane for STEEM blockchain seed node

You are viewing a single comment's thread from:

Using Intel Optane for STEEM blockchain seed node

in steemdev •  last month

I have heard about Intel Optane a lot also I tested the Boot Time of a system, its really fast. The test was good and nicely documented, but considering 7 hours is still a lot of time (though a very less compared to the SSDs), what are the options we still have to reduce it further.

I would like to see a witness-server running on Optane Memory, to see how it goes. If the tests are successful, then I guess every blockchain project should use Optane, what you think?

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  
·

I am aware of @anyx ‘s setup and the read performance from optane , ie rocksdb is on optane is what made me explore this in detail. I have doubts whether we should be keeping the rocksdb on Optane or not. May be we should keep block_log and index on optane and rocksdb in SSD.

·
·

The block log can be kept on hdd or ssd, unless you plan on offering API access to blocks. The block log is written sequentially when live, and read sequentially during replay.

The index/shared memory file should be kept fully in ram for best performance. It works on optane with some patience; specifically but the ratio of ram to optane or other disk will matter a lot. 32gb of ram and the rest on optane presently only takes around 3h40m to replay consensus state, versus 3h10m or so if you had 64gb of ram.

History (via rocksdb) could definitely be kept on something like an SSD. However, note that that for both reads and writes for both replays and live, it is random access. If you plan on offering API access to it, that further pressures the random access. Keeping it on optane improves performance quite a bit for API usage.

·
·
·

Agree on block log.

On the rocksdb part on optane, IMHO more rests are needed as tests by LMDB shows slightly surprising results. See here : https://news.ycombinator.com/item?id=18403637

·
·
·
·

Interesting, it seems like either rocksdb has a pretty big bug, or their testing method was flawed. I don't see similar problems with optane for other databases and the testing that I've done.

·
·
·
·
·

Exactly - so an extensive testing is required to validate what is going on. In our case we have the block_log's I/O performance handled by the blockchain code and the rocksdb's performance. ie 2 scenarios:

  1. How block_log I/O fares in Optane
  2. How RocksDB performance is on Optane in comparison to SSD and if its low, can we make it better by tweaking it.
·
·
·
·
·
·

API access to the block log you mean?
Location of the block log will have effectively zero impact on performance for replays, and even if there is an impact on performance for replays it would be due to a poor implementation and can be brought to exactly zero with a 5 minute tweak to the code to add a prefetch.

·
·
·
·
·
·
·

API access to the block log you mean?
No - the writes during witness/seed mode. But as you rightly said I think there won't be much of an impact as the writes are sequential ? (sequential I/O).

Doubt: There will be a I/O cache present, that should be better in the case of Optane than SSD right ?

Yea, Optane is really fast!

I would like to see a witness-server running on Optane Memory, to see how it goes. If the tests are successful, then I guess every blockchain project should use Optane, what you think?

IMHO we need to do extensive tests before deciding. For example for the full node we need a lot more testing as the RocksDB performance looks little weird to me as per the results here : http://www.lmdb.tech/bench/optanessd/ So in the case of witness nodes also, there may be surprises that we have to test extensively. But I feel if we test for STEEM, it can be generalized for other Graphene based blockchains too.

I have requested for access from Packet + Intel & if we can get some hardware, extensive tests can be done. I think we will have to do

  1. Tests with witness and run extensive tests
  2. Tests in the fullRPC node and benchmark the RocksDB storage (engine)