SteemData Update

in steemdata •  11 months ago

Running full steemd nodes /w reduced RAM requirements

After a handful of naive attempts and hardware configurations, I have converged onto a stable setup. It is probably not optimal and I believe @gtg is doing extensive research in this domain, so unless in a hurry, it is worthwhile to wait for his post.

In an effort to start moving towards HA, while keeping the costs reasonable, I've started looking into a hybrid solution for steemd nodes. Rather than keeping the entire shared memory file in RAM, some of it is being swapped onto very fast NVMe drives. So far it seems like the concept works well enough for practical purposes.

Servers

I'm currently using SP-128's /w NVMe servers by OVH.

CPU:  Intel  Xeon E5-1650v3 - 6c/12t - 3.5GHz /3.8GHz
RAM:  128GB DDR4 ECC 2133 MHz
Storage: SoftRaid 2x 1.2TB NVMe

Configuration

A very basic initial config is available on GitHub. I'm still running things in Docker for ease of deployment/upgrades.

SteemData Migration

SteemData is currently being migrated to more powerful hardware in a new datacenter (OVH - Gravelines, France). OVH is quite a fair bit more expensive, however they do have excellent connectivity, and a fair amount of bandwidth between EU/US which allows for future geo-expansion. Once the migration is complete, the switch will happen on DNS level, so no changes are necessary for end users.

An apology for taking so long

Whenever SteemData experiences interruptions, I'm reminded by how many people depend on it.

I'm expecting the SteemData upgrade to complete by the end of January. I apologize for slow progress - due to my full-time commitment @ view.ly, I am only able to work on it in evenings and on weekends.

SteemData Donation

@transisto has recently surprised me with a fundraiser campaign for SteemData. Time and time again, I'm humbled by fellow community members offering care and support.

The price of STEEM has gone up significantly in the past 2 weeks, and as of right now at least, the witness pay is more than sufficient to cover the expansion costs. As such, I strongly feel like right thing to do is to pass the funds to other Steem projects in need.

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Thank you very much furion.
SteemData is a very useful service, your efforts make it more stable and make steem more valuable.

Haven’t heard from you in a while , hope everything goes well and keep us posted on the updates

Does the combination "steemd nodes" mean that steemd.com is decentralized, or is roadscape still the only one to operate it?

If steemd.com is centralized, what are steemd nodes?

·

This comment has received a 17.98 % upvote from @steemdiffuser thanks to: @stimialiti. Steem on my friend!

Above average bids may get additional upvotes from our trail members!

Get Upvotes, Join Our Trail, or Delegate Some SP

·

You got a 2.86% upvote from @upyou courtesy of @stimialiti!

Much appreciated!!

Upvote please

its great that you have this fixed, will be looking forward to @gtg post on this

Thanks for all your effort! I hope, in time, there will be a solution for the massive RAM requirements, because this will become a problem for the Steem network sooner or later.

props for putting in the work, well done for being a standup person in the community.

Thanks for this update about steem... But how do I convert my sbd to steem power? I will love to support this project too

I'm learning about this platform, thanks @furion for helping people like us to know more about this and make Steemit a wonderful place were we can share and learn new and interesting things worldwide and make money with it.

Thank you for your input.
Best regards.....

Just saw this belatedly. Thanks for the update. Let us know if you need any tech help. Over our time relying on SteemData the dev on our team got to know it fairly well and if there's anything we can contribute to lighten your workload we'd be happy to do so.

Way to go! THANK YOU for contributing your valuable skills, time, and efforts to make this community a better place, @furion! Classy move to pay forward @transisto's fundraiser too.

·

P.S. Looking forward to an update about @viewly!

I am very interested in your post

Sounds like you've put a lot of work in. I'm trying to figure out how it all works from an incentive standpoint. Can you tell me how much witnesses get paid? Does the pay rise and fall with the price of Steem?

Thanks in advance for your answers.