SteemDB.com - Migration complete and a few thoughts on it's future.

in #steemdev6 years ago

Well, it took longer than I anticipated, but SteemDB.com is slowly coming back to life.

The data migration I started a couple days ago completed earlier today and I'm bringing the services back online. If you still see the "DB Migration" message still (on every page), your DNS has not updated to the new server. The DNS should update in the coming hours and the site will then become available once more.

If you're very impatient, you can always access the site via it's IP address: http://144.76.176.150

New Server

The new server SteemDB is hosted on is now provided by @privex, and I've had great support from @someguy123 getting this all setup.

SteemDB went from a 96gb RAM server with 8 cores to a 256gb RAM server with 12 cores. I've optimized and upgraded all of the software to the latest versions, configured the filesystem for high IO throughput, and should have plenty of room to grow over the next year or so.

Current Issues and Status

A few notes about this new deployment and issues surrounding the migration:

  • The database sync is currently catching up, as I write this it was processing block 18636474 (2018-01-02). So currently the 9 most recent days of data are missing from some sections. This new server is processing these blocks at a rate of about 3-6 blocks/second. We are currently at a height of 18892572, which means there's about 250k blocks that need to be processed. At a rate of 3 blocks per second (to be conservative), it will catch up to the current block height and finish processing in about 24 hours.
  • The witness pages and other sources that pull data live are unaffected by the sync process, but pages like an accounts transfers/votes will be impacted by the sync.
  • The account history module is disabled for the next 24 hours, so no new snapshots of accounts will be created until the sync is completed.
  • The apps page is missing data and somewhat broken, and I will investigate what's happening here. It's most likely due to the inactive history module, which also creates snapshots of app usage.
  • The homepage is no longer cached, so it's somewhat slow. I'll work on caching this again in the future.
  • Any missed blocks by witnesses during this migration were not recorded unfortunately.

Where SteemDB goes from here

Over the past month I've been doing a lot of thinking about the various projects I maintain in regards to the Steem ecosystem, and one reoccurring thought I have is that I have too many active projects. I am still in the planning phases, but one of my goals in 2018 is to shed or offload some of this workload so I can focus on the projects that actually matter to me, as well as the ecosystem as a whole.

With that being said, SteemDB isn't one of the projects that I want to personally stop maintaining. For months now in casual conversations I've mentioned that I really want to rebuild SteemDB and create a v2 of my block explorer code, and I am hoping to find the time to do so this year. SteemDB was my 2nd Steem related project (created Aug 2016) and I've learned so much since it's creation.

In 2018, I would very much like to take the time to sit down and rebuild SteemDB into something better. Improve the technology, the scalability, it's responsiveness, and usefulness. These foundational tools (wallets, blockexplorers, etc) are crucial to on-boarding new users and developers alike, while providing external transparency into the operations happening on the blockchain.

Luckily, this new server has room to grow, and I can fit a v2 right alongside v1 until it's ready to replace it :)

Sort:  

Awesome, now lets get some content to the top thats has some artistic nature to it! Right now its all dominated by memes and blockchain news. Please check out my new grand canyon time lapse films if you have a moment. https://steemit.com/video/@cinemaninja/bmvu7-grand-canyon-a-4k-time-lapse-film-ninja-cut Thanks!

I couldn't agree more. Once we start having less of a crypto focus and more of a general focus... that's when Steemit will really start to take off.

Good luck taking this app to the next level.

Hi @Jesta,

I heard you during the meeting on Discord in December. I use this comment section as an opportunity to thank you for the work you are carrying out for the community. I discovered Steemit about 3 months ago, am now completely immersed into it and am pretty amazed about what is going on here!

And a lot of this originates from people like you.

So, warm thanks :-)

It's a pretty deep rabbit hole :)

Thanks!

@jesta my Dear friend Your Such Great posts Check My Blogs Did You like Support Me Thank you bro

yes its great.
thanks for nice words.
@qami

we're pioneers ;)

Best of luck for your data migration work, I hope you did it very well and in the future you will do better work.

That's a monster server!

Out of curiosity is the data stored on SSD drives?

Yup, all stored on SSDs, and the entire DB is kept loaded in RAM for even faster access.

Checking it now, very cool!

so interesting!

Never knew this wesite and I know almost all the tools. Thanks man, I'll check it out!

Hey all!
I have put together some picks. They are low market cap coins that I believe have great potential for profit. Obviously there is risk and do your own diligence before buying. I am not a financial planner or advisor.
In no particular order:

  1. AirSwap AST (Binance)
  2. Lamden TAU (EtherDelta)
  3. Loopring LPC (Binance)
  4. aelf ELF (Binance)
  5. Luxcoin LUX (Cryptopia)

(Follow me on my new Steemit for daily picks, and have an excellent day!)

Absolutely. All the analytics sites that are being built are a huge help to managing accounts.

You are right bro, the system needs more tools to gain control of our accounts.
@racoo

I'm learning about new tools as I don't know about more #steemit tools.

Yes, I've the same minings as you, shohana1

wow we both are learning, nice to meet you new friend

I also will check this out! Thank you 👍🏽
Upvoted 👍🏽👍🏽👍🏽

Thanks!

How did I not know about this, the website looks amazing man! Glad someone tipped me off to follow you, please keep improving the site, I think it's looking clean and professional already, and so much info, this is an awesome find for me.

Voted for you as witness, really appreciate the work!

true, totally agree with you! there needs to be more awareness of the site!

Coin Marketplace

STEEM 0.28
TRX 0.12
JST 0.033
BTC 71443.06
ETH 3660.17
USDT 1.00
SBD 3.73