RocksDB and SMTs Announcement

in #steemit3 years ago (edited)

Dear Steemians,

We have estimated and prepared the detailed versions of our short- and long-term roadmap for RocksDB and SMTs with their target completion dates. To make everything come in shorter bursts and as the community has requested, we’ve broken out SMTs into SMT-lite, which is paired down to include the core functions of issuance and transfer of tokens. Additional roadmapped SMTs features are also listed.

Below is the spreadsheet outlining the different tasks for RocksDB and SMT-lite, as well as their ETA, points and status.

Screenshot 2018-12-05 at 4.38.04 PM.png

For RocksDB, our plan is to write an adapter that uses the BMIC (Boost Multi-Index Container) interface and replaces the behavior with a RocksDB backend. This will allow steemd to support both the existing BMIC/Chainbase implementation and RocksDB as a compile option without needing to change the consensus logic of steemd. Additionally, by creating the adapter to use the same interface as BMIC, we can leverage the existing BMIC unit tests to ensure that we replicate the expected behavior of BMIC, but backed by RocksDB. This will result in a 95%+ memory consumption reduction by moving our storage from memory to disk. Ultimately our target is to have Economic nodes (Full Nodes by a Bitcoin standard) running on commodity hardware with 4GB of RAM and a standard SSD.

The scaled-down SMTs scope includes these features: claiming names, tokens, issuance of tokens to one account, transfers between accounts, immutability (meaning that once a token is launched none of its original parameters can be altered) and a testnet.

Next iterations of SMTs will add back in these original functions:

Screenshot 2018-12-05 at 4.25.20 PM.png

Again, I will discuss these topics and your questions tomorrow during a livestream at 11am CST. See my post from Monday regarding submitting questions.

Best regards,


man....why don't you get along with Dan again and you guys could merge EOS with Steem......otherwise he creates steem 2.0 and we all get fucked up......doesn't matter who is the boss, better to be number 2 with Dan in charge and become a billionaire than be the number 1 and go room for small coins....EOS and Steem must merge......he already proved he was right when he created a new coin from scratch and it got much higher market capital than steem....sell the control of Steemit Inc to assure yourself finantially and assure the success of Steem. Telling this not to put you down in any way but to genuinely help you.....All the best

You got a 89.48% upvote from @brupvoter courtesy of @christianchannel!

I agree with you my friend!!

Sorry who is this Dan everyone is talking about? And what platform is he creating?

He was the architect of steem, he created steem.

Dan Larimer. He is now (or maybe he's already left it, too?) behind the project EOS. Steemit pretty much fell apart after he left last year. Though, as many of us hopefully know, correlation isn't nec. causation... but in this case, there seems to be a strong relationship nonetheless.., imo.

Upvoted for the commitment to getting a simple token delivered; thank you. The greater concern here is probably the period of time proposed for the RocksDB work, since technical people seem to doubt it can be done that quickly. So I wonder at the actual price difference between what you are doing and what Blocktrades proposed to handle. But again, we need tokens and thank you for that commitment.

Commitment? The only thing delivered is a little beer mat with some writings on it.

30 days, what are the deliverables? Are you serious? We should be happy in 30 days he finished his powerdown, how many mind-fucks can you undergo?

"Ultimately our target is to have Economic nodes (Full Nodes by a Bitcoin standard) running on commodity hardware with 4GB of RAM and a standard SSD."
This is fantastic news and will benefit the Steem ecosystem enormously.
If Crypto prices had stayed high this would never have been a priority.
This is a perfect example of how going through tough times creates a much stronger & better result than being on easy street.
Well done and good luck in getting these initiatives completeted.

As the saying goes


Hopefully the 3rd & 4th panel won't apply to us.

100% correct and nice graphic.

Posted using Partiko iOS

lol, you wished him good luck... you're evil.... :)

"Ultimately our target is to have Economic nodes (Full Nodes by a Bitcoin standard) running on commodity hardware with 4GB of RAM and a standard SSD"

You just got trolled, or better, scammed... how does that feel... fantastic news, or ? fake news?

This is absurd... that doesn't help the "ecosystem"...

It's far closer to repainting a house that's partially collapsed from termite damage and calling it a repair.

yes, agree more

Thanks for the update. If you make it possible to run a full node on a 4gb server I promise to throw in some ;)

This right here is why optimizing steem's core technology is so important

Server? Servers are in centralized datacenters. Shouldn't the end goal be the actual end users running nodes on their android phones? And on their gaming PC's. Bitcoin isn't mined only in datacenters. Steem nodes shouldn't witness blocks in only those locations either. That is a recipe for disaster.

Well I totally agree. The thing why I did say server is because I got a internet connection that is unreliable at the best and simply disfunctional at the worst of times.
If I would have something decent I would throw in additional pcs at my home.

A gaming PC can be a server and indeed a good one for blockchain applications with excellent single core speed. Servers don’t have to be I data centers. Once upon a time every business had them in their office.

Posted using Partiko iOS one runs any nodes on android phones bro sorry...
Just no.. LOL

"SERVERS?? SERVERS?? WHY arent we running steem on Potatoes and calculators?!?!

Why, I propose we Hand calculate the hash's and mail them my carrier pidgeon!

You have just described Holochain. You can thank me later ;PP

Thanks for the update!

It would be great if you and the dev team would consider adding native support for NFTs based on the SIP001 spec here, right after the work on basic tokens is complete:

Edit: please see my reply to smooth deeper in this thread for short overview

Hi @upheaver - I just wanted to say thank you for contributing to STEEM ecosystem, I think it's very impressive that you just voluntarily devote probably tons of your time to open this platform to new possibilities. !tip 100 hide

Thank you, highly appreciated

NFTs are accomplished in SMTs by issuing SMT-lite of 1(one) token and 0(zero) divisibility. What are the differences in the approaches? Thanks.

My proposal allows for storing of custom json metadata inside each individual token by 3 different parties: the token issuer, the current owner and the dapp interacting with the token.

The specification document contains a walk-through example

Thanks. I'm fairly sure we'll achieve the goals (e.g. Support Steem Monsters Card Issuance) supported by those reqs with SMT-lite. If I'm incorrect, let us know where.

Using SMTs for NFT functionality would be a hack - while it might cover the very basics, the native enhancements I proposed would provide new capabilities and in turn enable new kinds of businesses/ecosystems to emerge.

What is functionally different besides the name? If there are missing features, particularly small ones like json fields, now is the time to hash that out with the developers and get them incorporated.

It's explained better when reading the spec, in short there are a few things:

  • native apis for nfts (querying, issuing, burning)
  • separate storage inside each token for issuer's, owner's and dapp's data - they can write only to their own section.
  • a new mechanism for users to send NFT tokens to third party dapps (similar to a smart contract)
  • for those apps to consume/process token interactions asynchronously using queueing

This enables powerful new ways to build apps.

For example you could send your steemmonster cards to a real time strategy game dapp built by a 3rd party, play with them,earn custom points, and then convert those points to native experience points by steemmonsters - all data logged nativelly inside the token

Yes I understand your spec does things in a different way. Since you worked on your own to design something, and there are always different ways to get to the same goal line, it will clearly be different in terms of details such as the names of the APIs.

What I'm trying to drill down to are the core functional differences that can bridge the gap because I think there is near zero probability that the Steemit devs are going to implement your spec, but actually decent probability they might be convinced to make some small functional changes to enable the same or similar use cases.

For example, having data storage areas seems like something might be possible to go into even simple SMTs. If it goes in, many of the uses cases you have in mind will be enabled, if not using the same APIs that you specified, etc. If not then those use cases won't be able to use SMTs (or certainly not as easily)

Thank you for your explanation here. I didn't quite get the benefit when I read your post and glanced over the documentation.

Good job @ned . I really like it, how fast you react and how you start communicate with the community!

Ned I love the work you are doing and especially that you are listening to the community. That allows everyone to advance.

Ultimately our target is to have Economic nodes (Full Nodes by a Bitcoin standard) running on commodity hardware with 4GB of RAM and a standard SSD.

I am SO excited by this. Increased decentralisation!

Rise of the many nodes! Can't wait. Any word on CPU requirements / core count targets? I'm not about to run out and buy hardware today, but it would stop me from selling some other hardware I have lying around. :)

You can run the smoke blockchain on 4gb Ram.. :) Perhaps steemit inc takes note and removes the cancer that is SBD.

This is huge news for our #OneLoveDTube community, a few of our RPC node dependant projects are having a hard time currently and with those resource requirements we could run our own node off the servers we have already 💪

Posted using Partiko iOS

Good job @ned . I really appreciate how hard the changes must have been this close to Christmas, but I understand critical situations call for serious solutions. I am very happy to see how fast things are moving the and increased focus on deliverables, communication and the discourse in the comments.

When you hang around people all day that talk like this you start to think all this jargon is normal. There were so many acronyms in this post I thought you were speaking another language. You need like a glossary for your posts perhaps. haha

But that said i'm happy to hear SMT is coming along we'll see to it that we can support SMT projects on @steempeak

lol - I just read many things I do not understand once again. My accounting degree, MBA and CPA are useless here.

This place never fails to amuse and I hope it survives. I just pray all the big guys get it together. I will keep posting weight loss work and helping tiny fish as long as I can.

Someone walks in and wonders why the heck you're helping "tiny fish" hahaha

Lol .. you are funny.

Posted using Partiko iOS

Why thank you! I amuse myself :)

The amused survivor :)

The way I look at it this that SteemInc may well speak their own language, but at least now they are attempting to use it to communicate to others outside of SteemInc!

Is someone outside of steem gonna understand all those very very steem acronyms?

Exactly my thoughts @jarvie - wouldn't a glossary under the main dropdown menu on everyone's account page a be a good idea? Or is there perhaps already one somewhere amongst the FAQs items?

At least we have @upheaver to explain it to us!

He's much easier to understand. haha

I have never been more excited about Steemit then I am right now. Thank you Ned for your decision making in the past few weeks.

Thank you for your continued best efforts @Ned!

First of all, this update made my day!

Second, I know very little about development so this question is from a layman's point of view. There are many of us here, so I hope my question will help my fellow novices to understand this process better.

I am thrilled that the two most essential elements are going to be addressed. Is there any way to work on them at the same time so they can both launch asap? Could one of the two elements be outsourced to experts in the community or industry so they can be completed more quickly?

ammm i didn't know @ned can do pin posts ....
is there an option to do it in my profile ?

No. Anything goes for the management. Fuck any sort of system or order.

ammm hope they will add it to users soon

They won’t. It was just done so “Ned” can keep everyone from leaving, I suspect.

How is this post different from the last 50 times promises and dates were made and failed. You know, like "live stream tomorrow" and "post and make money"

Yet here we are, with all the usual hypnotized, kool-aid drinking, cult members kissing @Ned's ass as usual. And such an ass he is.

soon (tm)

"these are goals, not promises" - @Ned Scott.

Was wondering same thing. That is no bueno. Everytime I pop back into my account, shit is just getting worse, not better :(( I am migrating everything over to Holo/ Holochain, for they are doing what I envisioned everyone else to be trying to do, yet are actually doing it. Feel free to join me; thank me later.

Are you considering @blocktrades offer to take the RocksDB implementation? The above seems to point to a 'no, thank you'.


Seems that Ned wants his scaled-down company to do it all...since they continue to deliver in such a stellar manner.

Upvoted, a real example of using reasoning and not wishful thinking.

Ah right, thanks for the link.

Well I hope this doesn't put other people off offering to to help, I thought this one might be a goer :/

that issue is at least a year old, and no one stops them to develop it and have a PR.
Like everybody else.

They'd like to get paid for that work.
Like everybody else.

Steemit burnt millions failing this far. Nobody is gonna do free work for them now, when compensation in turn is unlikely at this point. Developers of this caliber charge significant consulting rates. Ned had a bunch on staff, which is ordinarily cheaper than consultants. By a magnitudinal scale, but he fucked that all up, and nobody is going to do his homework for him now for free. He still has millions - he better hire somebody that can fix his fuckups, because most of the people who would have contributed for free, left last year or the year before, after witnessing the centralized incompetence.

To be honest, Blocktrades should develop it and release it at a public repo.

That’d decentralization.

Looks like Steemit Inc listened to my advice 🙃 I was one of the few telling them a no thanks to the expensive BT deal.

That advice came out of nowhere.
They should do it like everybody else, PR and have the consensus decide, what is best.
At least, they could still vote against SMTs, if they are not happy with the current solution, but they didn't do it in the past, so I doubt they will do it

Yes I saw you were not keen.

Only Steemit Inc can accurately assess the offer against doing it themselves - let's hope the right decision was made.

Why is @berniesanders downvoting this? What the hell is going on over here? Why is this post hidden due to low ratings? WTF is going on the Steemit Blockchain? I am shocked

Keep up the communication Ned. We’re mostly very supportive of the hard work putnin by you and your team.

@Ned have you also considered FPGA acceleration of the blockchain replay? It is heavily CPU single core bound and likely involved many steps that would benefit from FGPA acceleration. I have these $275 Acorn 215 FPGAs from Squirrels research based on Xinlix Artix 7 FPGA that I'm interested in seeing whether they can be used to accelerate Steem node replay times. The main tech guy there @gpuhoarder on Discord thinks its possible.

I don't suppose it matters if I ask for the third or fourth time for things to be explained in terms for regular users to understand instead of techno jargon alone, so... jazz hands?

My understanding is a leaner version of SMTs is under development and a new database system (rocksDB) is to be deployed which will hold a lot of stuff that had been floating around in RAM to be stored on disk. This should allow full witness like nodes to run on machines that will be much more affordable; which should help to further decentralize the network and make everything much cheaper to manage.

[NOTE: The above interpretation is open for debate and likely mis-interpreted in many areas. It is not meant as financial advice but Ned's post is making me feel very bullish. 😎]

Thank you for the clarification.

Posted using Partiko Android

Thanks! 😊

I agree I don't understand any of this...
@ned can you please put this in terms us regular guys will understand?

In short I ned laid off 70% of staff so they would no longer delay progress and we have viable ideas as to how to solve sustainability for nodes and we are working on SMT-lite So tokens can start to be distributed 😅

Posted using Partiko iOS

Thanks. I saw the 70% thing but I didn't understand the whole SMT thing. Thanks for clearing that up for me @d00k13. As always I can count on you!

Posted using Partiko Android

People, people, people... This is the lay of the land for the time being... We are all relatively young here... As is the space itself.. In other words: It's the Wild West. You want gold, you're gonna have to go diggin for it; and ain't nobody liekly to tell ya where it's hidin... Do your own research!! If you don't know a word, look it up!! All this "Wahhh I don't get IT!!" Imo, just means you aren't ready for whatever it is you're attempting to "get" to begin. Nothing here will be spoonfed... that's YEARS down the line, and much much more money behind/ into this before you can have the ease and comfort you apparently seek.

It's great to see these updates @ned.

Communication is really important both in the good and in the bad times. I wouls say that it's much more important in the bad times. Thank you for doing this.

Also, SMTs, Simple SMTs, SMT-lite, it doesn't matter as long as we get them. So this is great news!

Steem on!!!

Posted using Partiko Android

sounds like bud and budlight 😂

Haha. Yeah. :D

Posted using Partiko Android

so @blocktrades and @gtg were onto the point!

Is the blocktrade proposal still relevant in this roadmap from ned?

Posted using Partiko Android

i think so or at least parts of it

To make everything come in shorter bursts and as the community has requested

Hey @ned , I am glad you are listening to the community.


Wow only 4gb of ram and standard ssd’s is all that is needed! Awesome

Posted using Partiko iOS

Has "Pinned Post" always been a thing?

Pinned - I like it @ned - about time we get to see important information from Steemit Inc on from the CEO when he has something to say! Good Move!- I have missed this since... 2 Years and 9 Months ago. Can we get our own personal pin on our blogs too? :) :)

Hi @ned

Good to see that the storm is almost over and we are about to move into a recovery phase. Great work on the development of new features and the updated SMT roadmap.

I'm writing this message in the spirit of the #justask movement.
I'm part of a group of independent researchers and writers united by our love and believe in cryptocurrency and the potential disruptive applications it has in different industries. Right now many of us lend our efforts to Oracle-D in its quest to draw investment to steemit.

I would like to request an SP delegation to better curate and encourage newcomers.
I invite you to read my blog for assessment and if you deem it fit, delegate to empower the vision. The delegation amount is at your discretion. You know what's best.

Look forward to your reply whether positive or negative. Cheers.

Nice dick sucking


Define dick sucking?

Pretty lame, man. When can we pin our own posts? Such haphazard attempts and reestablishing order here.

I like it, something like "Pin your own business!"

By the way, allows for three pinned posts., Facebook, Twitter, & other websites, allow for just one pin, generally. Adding customization, options, brings us closer to the golden age of MySpace of 2007 which is when I was totally customizing my home page on MySpace with some code to allow redesigning the layout to turn your home MySpace page into like a website home page. I would love to be able to redesign a custom Steemit home page like it was or or some other kind of web builder like but with even more options.

Good to know that we have a game plan in place! 💪🏼🙌🏼💯

Posted using Partiko iOS

I am just happy @ned is upfront about the situation and keeping things transparent

Yep. Same here. Communication is important.

Posted using Partiko Android

Thanks, Ned!

Thank you very much for the update my dear friend

Posted using Partiko Android

Well atleast the new leaner team looks meaner and we're seeing some progress, thanks ned! Also how does one go about getting their post pinned? I'd like to pin my post! You can't just show up with new features like that and not tell us!

Ehhh what progress do you see?

Also how does one go about getting their post pinned?

Probably running your own Condenser Node would do it. 😎

All the communication is appreciated.

Posted using Partiko Android

This sounds exciting, some solid plans for a change.

lol... please let the guy be happy. :)

Shitty comments have been getin 10-15$ upvotes.

Good, Glad to see SMT's are moving forward. This is absolutely essential functionality for paths foward for a variety of applications on STEEM and should, upon release, help provide the economic framework for growth in value of STEEM.

Good chances are blowing up.
What are you doing?

I think the biggest obstacle to the development of the company is you. Am I a bad person?

Mostly just a smart person, who doesn't seem to be under the influence of the cult of nonpersonality.

Just a friendly reminder to anyone paying attention.

When the success of an organization promising rewards for investment that depends on future investment to cover the costs of earlier investors... that's not lowering costs, that's not expanding the platform, it's bordering on a ponzi scheme.

The solution needs to be in looking at revenue streams, not generating new or better cryptocurrencies... part of why SBD value has dropped so much harder than cryptos in general is because SBD is only good for SP or $, and essentially nothing else (of any real scale).

Quote : "It's bordering on a ponzi scheme."

Just a friendly reminder to you paying no attention. It is a ponzi scheme!

Ultimately our target is to have Economic nodes (Full Nodes by a Bitcoin standard) running on commodity hardware with 4GB of RAM and a standard SSD.

Now that's fuckin' exciting. I get that there's been a lot of SMT hype and people are kind of obsessed with them, but if you can pull off the above that's a sea change in Steem usability for devs.

Indeed! I think that even if we had to split the chain across multiple servers, allowing this would be a great development. A 64GB server is still pretty premium. A 16GB server would be perfectly acceptable. To get all the way down to 4GB? That means that people can run it on hardware taking on dust balls in the closet. Such would mean that quite a few more people can run servers.

Please, could you stop the cumming? I liked the critical parts of your being more.

If you read Minnesotan you'll notice there's a pretty big slam on SMT obsession in there.

But part of being critical is celebrating when something is a really good idea, and nodes that can run on commodity hardware are a really good idea.

By the way, what the hell is this?


It is pinned like the name says ^^
an important information which should be provided to all steemians.

I don't think we can ever pin posts - but the dev team and @ned are able to do so.

Nice username btw :P

Thx for the update!! Very excited ;)

Thanks for update useful information.

That is quite the reduction, and will be quite the reduction in potential costs for server hardware. How confident are you that the timeline will be able to be upheld? I know frequently estimates for development can be held up, even by tiny issues that were unforeseen, but there are conservative estimates and quite optimistic ones. If that estimate is able to be upheld, then it will only mean possibly one or two additional months before witnesses can reduce their costs considerably. That is quite good news.

As to SMT's...I'm not as optimistic that they're the golden unicorn people are claiming them to be. I'd prefer more community and curation related features be implemented. So, it's nice to see an SMT-lite is going to be implemented in the near future, but I think a lot more needs to be done.

How do I get this “pinned post” junk off of my newsfeed?

Disappointed Korean Steemians lost faith in the Steem Inc...

U got to strike when the iron is hot. We lost the plot midway...It became all about $$$....It's still a great project. But we need focus and have to implement stuff in a timely manner. We have been talking about SMTs for more than a year I suppose. This is where we lost the plot.

Communities and "hivemind" were promised loooonnnng before SMTs and never delivered. The plot was lost long before you realize I guess.

Pinned post? Can I pin my post?

No. You’re just a peon payjng the guys at the top who haphazardly implement stupid shit to help them keep you on the platform.

You’ve really started spitting that vitriol, haven’t you? Christ... Everybody asks for communication and transparency, and then when they pin an update to the top of people’s feed that they feel needs to be seen, people fly off the handle. You moving to Whaleshares or something? I like you @kafkanarchy84, I really do. Just seems like as something of a community leader, your outlook is less than helpful, at least in this forum...

Sorry for having an opinion you don’t like. I’m nobody’s leader. Why is frustration and an actual opinion that’s not a cult-y happy cheerleading song such a taboo for so many here?

I don’t really think we have to blindly praise Ned and co. or be entirely dismissive of their efforts. Feels like there’s still a thin line of wary hope in the middle there somewhere. Not saying I’m entirely pleased with the situation, hf20 fucking newcomers and the big layoffs, but the people cried for transparency from stinc and that’s what they got. Not really cheerleading for them.

You mean the transparency of not releasing books?

The ire of delayed SMT. You know that it was supposed to be released early this year?

How about how unaffordable RPC nodes, etc. are for average users? They've known that for a while.

It's more than just HF20 and layoffs, you know.

It’s more than just HF20 and layoffs, you know.

That’s all well and good, but you know these issues are the ones being talked about as of late. The only SMT roadmap I saw talked about their release in the first quarter of 2019. I joined in April of this year, so I am unaware of earlier plans. I would be pleasantly surprised if STINC released their books, but I highly doubt that will happen and don’t even really see it as necessary. They fucked up relying on the market to pay people, and failed even with the cushion of the early ninja mine. That’s about as much as I really need to know. I’m gonna venture to say the “average user” cares very little about running their own nodes, and cares more about being able to use this platform without a worry of the whole infrastructure crumbling and everyone leaving.

My faith is less with Steemit Inc., and more with SteemPeak, SteemMonsters, DTube, SteepShot, etc. The things I really use. If STINC dies and they fail to keep the protocol, and therefore these awesome DApps running, then so be it. I’ll be here until it burns to the ground

I’m sorry, you can’t! It’s only for Ned

Posted using Partiko iOS

Just a really really boring idea...nothing glamorous, just basic sales and incentives...

I hope that the ETA of development will not be delayed any longer.

Seriously...if it goes any longer, I'll cry

me too ; )

lol...we'll cry together then

곰돌이가 @ayogom님의 소중한 댓글에 시세변동을 감안하여 $0.001을 보팅해서 $0.022을 지켜드리고 가요. 곰돌이가 지금까지 총 1844번 $24.306을 보팅해서 $22.789을 구했습니다. @gomdory 곰도뤼~

@ned - it is great to adjust the goals in new reality. Something is better than nothing as long as we are progressing towards the main goal.

However, pls also keep exploring the option of taking support from the community. You have great community with diverse talent and reach and that is willing to be supportive too.

It would be great to see few words from U about @blocktrade proposal whether taken up or not. That would help other community members to fine tune their proposals in future.

@blocktrades commented on his own thread that the proposal was rejected.

Enough said, this is a farce, a mind fuck to buy some time to execute his exit plan.

Imagine he would accept the blocktrades proposal. That would mean blocktrades would deliver as promised. And that's not part of the plan.

I am very skeptical and doubtful about what is happening here. How many times are these empty promises needed before more people wake up?

I hope more people use reasoning and not wishful thinking when it comes to Ned Scott and Steemit Inc.

I hope the steem price goes lower much more, so that steemit inc needs to sell more stake to pay for this liquidation fase. Some new kids on the block is the way forward.

Exit plan is certainly an interesting theory. Along those lines @ned has been powering down his entire 2M STEEM account. Why?