Introducing MIRA

in steem •  2 months ago

MIRA.jpg

Hello Steemians,

On Monday we shared a short update on our progress and now we want to update you on a brand new technology we are developing for Steem called MIRA, which stands for Multi Index RocksDB Adapter. The purpose of MIRA is to allow Steem blockchain nodes to store almost all necessary data on disk in a modern database instead of in RAM. One of the things we are focusing on right now is getting RocksDB working with data consistency. We have the first implementation of MIRA done and building with steemd, and are currently working on debugging and testing.

One advantage of this is that nodes will be more performant on average hardware. The other benefit is that nodes will cost much less to run. In both cases, this helps Steem be more easily adopted by application developers and exchanges. Consequently, Steem should become even more decentralized as the barrier to entry for running nodes is dramatically lowered.

The Steemit Team

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Awesome! Decentralisation and lowering of the costs of the full nodes are extremely important.

Is it possible to say by how much this first implementation of Mira will lower the RAM costs?

·

Now can we get someone to just setup a microsoft visual studio stub project with all the npm dependcies setup...

It would open up the world to working with corporate types, and small businesses using Viual Studio, and Sharepoint.

Not to mention connection to any database in the worldthrought dao and odbc.....

it's a no brainer!!!!

It should take a couple of hours if we got 2 people together:
a Programmer familar Microfoft Studio and npm,
a Programmer or Admin with knowlegable of the depencies to build steemdb, steem and steem/js library and tutorials

A simple step by step with screen snaps tutorial post or a video could be done!

I know at least 10 programmers who would start programming on Steem if that was available....

I assume that the end result would be more nodes coming on line!

cyrus_tagline.jpg

·
·

I love this idea.

thumbsupcolor.jpg

I just got unanimous agreement from all the
Members of the Steeminati
(that responded to my text)

This is a worthy addition to the Steem Community....

We will be putting up a post up with a BOUNTY reward for a couple of fellow techies to do it!!!!!
happyuser1.jpg
It will come out of the Steeminati group pool!

Once verified, The files will will be put on Github and shared with the community.....

IntroducingRichAtVNSwithsign.jpg

·
·
·

Rich,

if you get it from someone, can we discuss having my license connect to whatever selling sources are on steem?

I think gamebrain could use Steem as a venue.....

·
·

yes yes yes, I could use it in my class, I'm learning VBA right now....
I could use the reports in accress or excel, and bring threads into powerpoint!!!!

That's a great idea!!!!
kingcal.jpg

I just called rich on this comment ...I bet he chimes in with in 5 mins...lol

·
·
·

What is the main difference between VBA and VSCode? I remember the Microsoft CEO in the CNBC interview about Microsoft purchasing Github,

he states that VSCode is the open source version of VBA, so I assume you must pay for VBA, and I also assume, you might not have access to some of the dependency extensions available to the Open SOurce users. These comments are recent, but there are some pretty legit Mark Down extensions that come very very close to Steemit's style, but I do agree wtith @cyruss33.

THIS WOULD MAKE A WORLD OF DIFFERENCE







anchor.fm/alphawave

·
·
·
·

Decentralized is the first step.
Distributed is the best way to keep it secure and running without having some big guys playing their game.

Thanks for the visualisation.

·
·

i don't like black boxes.

microsoft is a black box, willing to buy your individual text messages from facebook.

·
·

But is Microsoft decentralized enough?

·

could be the first part of a big game changer in securing the network.

·
·

I was thinking the same thing, security is the only thing I take more serious than comedy lol! "Truce"

·
·
·

100% with you there. I do have some questions about how secure this is against Russian hackers though. It would also allow me to put that server I have sitting in my spare bathroom to use and maybe open up a witness node.

·
·
·
·

Do it! I would love to see your witness campaign - perhaps it would be entertaining as a run up to 2020?

·
·
·
·
·

well, from what i hear from the news that trickles down to the hinterland here that might not even be such a far-fetched notion atm ... at the risk of #offtopic on post , i think they're just letting him run into his own demise. The best thing for them do , lol, way better than impeachment since that would mean they're the opposing party and get his quite persistent following on their necks ...

as for the rocksdb thing ... i'm the type o' guy to see it when it gets there ... @goldmanmorgan / @tyrnannoght doesnt really need an SMT as its "virtual" or simulated or whatever you wanna call it (i think technically that could mean i could port it to any chain that allows freebie points and coin to be gathered in the end (by 2050 or so lol) since its only dependent on incoming vote values which could be calculated from any give set (hear me talk lol, like a boss)

however

i always like

cooltext263014691606796.png

... looking forward to it but hoping it doesnt just lead to another zound of currencies dispersing the total capital from those already in existence, so

it's up to the weirdo's and the freaks to make

creative use of resources then, can we rank-amateurs hack it in a way none of the scholars thought of (as in "hack it" ... slang for do it , please dont mis-understand :))

sincerely ,

cathublogo.png

the site is down btw, its a toy bedroom server like ms. Clinton talks about and i had some serious poking i saw in my logfiles so its offline (the front anyway, the autovoters still run)
yay for steemchain ! good to see no one is throwing in the towel

·
·
·
·
·
·

As long as we are around poking, all creative ideas count bonus points. Like you said, Steem is mostly an environment for deployment!

·
·
·
·
·
·
·

that's a nice way of saying it ... it doesn't present itself like that , though :) but it IS , i agree, a fabulous new thing to play with hahah. Compared to any kind of hybrid before i havent gotten this creative in years.
The problem there, as it is presented too, is that a lot of people see "creativity" in very narrow, small kinds of ways , you know ?

Like my old man would say only Bach and Mozart is music (so to speak, because he like the stones and the beatles too and stuff like that and more, but you know what i'm getting at, i suppose)
You write some back-end that's pages long for a front-end that doesn't even fill out half a page and "its not writing" ... therefor it is not

its that kind of ancient greek logic that often gets on my system :p

despite all the kindergarten politics making it look like real elected parliament on top is real elected parliament on top and the content fasciscm, making it look like the government doesn't mind hiring legalized mafia as long as they're in place getting the real money.

Despite all that, it's STILL the demoscene of blockchain. I wouldn't even know where to do this with ETH or TRON or EOS, to name but three of the ones who thought themselves big before the great price war against the vampires and the old gold king, because that are all places you have to buy in ... at least here they pretend you can come in without money lol.

And afaik flixxo doesnt HAVE any kind of api as they don't even have their own blockchain and they're completely hybrid, so that's just a platform in essence where you can't get creative in anything but the thing that's already static by the time you upload (whereas i consider my little baby @tyrnannoght to be slowly growing into something truly dynamic, for instance, as it ever-changes by itself ...)
The upcoming (read : promise by Basic Attention Token (BAT - brave browser) is for now still just that : a promise, but that will be nothing more than getting tokens to give tokens out, which is GOOD , because at least you GET some lol, where i think , like flixxo it should also be a bit more dynamic like : if all you do is watch ads and never buy, the amount you get should be less. But they're REALLY slow in development, for instance i got like ... last time i checked 165 videos seeding, must be close to 200 now but the reward-by-seed number has been stuck at the same for a long time now and i can't even scroll down in they UI/client to see the bottom of the list ... for example hahah, on the other hand the speed at which they're moderating is increasing, it doesn't take a month or more anymore to get a video accepted.

other than that ... despite the governments cracking down on ICO's to protect the gullible from themselves (after they asked for it, after they got too greedy and their gullibility got them ripped off) most of it is still marketeers trying to sell hot air or stuff they don't have yet for a wad of cash. All those "crypto-anarchists" either don't make much noise or are long bored since the crypto-capitalists took over.

Other than steemit , bat and flixx i can't really think of anything i checked out that i would even sign up for as its almost all "pay us first so we can next year "maybe" get you a fabulous new system that's an old system with a new name"

and then some ... i probably used up the internet again with the reply but as you see, i still do reply even if its a month ago, maybe i'll even catch up as i'm not talking much lately

@ecoinstant

·
·
·
·
·
·
·
·
·

Nobody can kill you, if you are distributed ;-)

·
·

Not only a big game changer, but would be one which will make new path to even other such currencies.
And the best part would be that the user like me and you will get benefits.

·
·
·

wasup Truce my big black succesful nigga for real

·
·

hope so, respect your opinion

·

Thanks @exyle!

While a lot of work on this has already been completed, it will be much easier to definitively answer that question after we get a little bit closer to completion.

·

The question is why wasn't this done last year, but better late than never. Steem really needs this.

·

I support 100% with you there. My questions is about how secure this is against Russian hackers though. It would also allow me to put that server I have sitting in my spare bathroom to use and maybe open up a witness node.

·

Agreed, it is very Oatmeal Awesome.

·

Here, have a small upvote from me too. Thank you for showing in such a brilliant yet simple way, what a big shithole this place has turned to be thanks to the idiots running it.

·
·

Well, if you are gonna get screwed you should at least have fun with it. I suggest.........................

FB_IMG_1545414062378.jpg

Disclaimer: May not taste like real pickle.

·

This post has received a 7.72 % upvote from @boomerang.

Ho! Ho! Ho! Merry Christmas!!! I've given you an upvote and left you this amazing automated comment!!!!

How about giving ol' @berniesanders a follow, eh?

·

Hey Bernie can I sit on good ol Santa Bernie's knee and tell him what I want for Christmas :P

lol

berniefunny.jpg

@berniesanders I know what I want for Christmas :D

·

Must be a really good thing if you made ol bernie happy! 😄

·
·

Part of the Flag War Peace Dividend. Bernie is a changed man. 😎

·
·
·
·
·
·
·

Actually steem has many forks from golos to... the upcoming SMTs.... there will be many steem's hopefully bitch

·

Hey man youre picture is triggering my small hole phobia aka Trypophobia

Im shuttering right now , shiveriong, even though IU know the makeup VFX pics are all fake, like oif the tiny holes in peoples hands and feet, but its still triggeringyuck

It's so important to have these updates. They build moral and confidence.

Please keep them coming!

Why is this trash pinned to my feed? I feel raped.

·

You don't own your feed. You are browsing a centralized website controlled by Steem Inc. They have the right to show you everything that is legal no matter how distasteful it is to you.

·
·

And I have every right to complain

·
·
·

hahaha good one @trumpman ! :D

·

You forgot the regular: "Trending is brokan!"
😂

imagine all this downsizing ends up with steem getting caught up in a bull run

I think direction is critical. I am curious about a few things.

  1. Can you through more Ram than required to speed up a node. Say this takes ram requirements down to 16gb. Would having a 64Gb node allow for caching or will that only be Jussi job.

  2. How close to the 4gb ram target do you think the first iteration would take us?

  3. On a node with significant ram (256gb-512gb) would this be slower than the current system?

·

These are all fair questions - while a lot of this project is already completed, it will be much easier to answer these questions once we get a little bit closer to completion. I believe the answer to #1 is 'probably not in the first iteration'. This is also being implemented as a plugin, so it can still be done the 'old' way.

·
·

Good to hear of this progress Justin.

Is development still on the schedule Ned outlined a couple of weeks back? ie to have it ready to roll out by end of January-ish ?

Tomorrow, Thursday 8pm - 11pm on MSP Waves we have the second State of Steem Forum - this week focussing on Technology including nodes, RocksDB etc.

Would you or anyone else from Steemit Inc be able to pop along for 20 minutes to give an update and maybe answer a few questions?

Thank you



·
·

Correct me if I’m wrong, but wouldn’t the extra RAM be used by the OS as buffer cache for the disk-backed RocksDB files, like every other disk-backed database?

·
·
·

Yes, RocksDB itself does technically do this (or not if allow_os_buffer is set to false) - certain things could also be supported through runtime/compile time options. I believe the OP may have been asking about something more like using different storage methods for portions of the DB and not necessarily OS level caching.

·

Some really good points there @themarkymark

imagine all this downsizing ends up with steem getting caught up in a bull run

Outstanding effort, to lower the costs and promote decentralization. Can we get it down to the point we can get regular users to be full, or more likely, partial nodes?

It's great to hear that we are making some steps forward!

All for the future of Steemit!

Exciting times lay ahead.

Best,

@charitycurator

@steemitblog,
Yeah that's a good idea, cost lowering and improve the performance at the same time!

Cheers~

@steemitblog can you add one more field to the data, for feeds and blogs can you add the time it was added to a person's feed/blog? This is so it is easier for me to create offline dbs in my app and reduce the load on nodes further.

i Like it.

Ok.

I wish I could Pin a post in my blog tab. Like a introduce my self.

The solution to the issue of nodes is gradually coming to a point and I must acknowledge the work done so far. Well done Steemit Inc.

But is there a formal agreement between blocktrades developers and StInc as regards the proposal of RocksDB? Or StInc developers are already working on the upgrade alongside the SMT's protocol and co?

·

as far as i know steemit inc did not accepted the offer

·
·

My feeling is that we have been blessed with such a great team who is willing to jump in when needed.

We're so lucky for our witnesses' talent. Ease of running seed and witness nodes should only increase that pool of talent and see ever more development on the blockchain.

This explains everything :O

Looking forward to what this will mean for the nodes and how the applications might leverage it.

Out of curiosity, does this also affect RC cost calculations? Does it make bandwidth cheaper for users?

·

No. They dont work on the RC plugin. When they refer to costs they mean $$$

·
·

Yes i know but I was wondering if lightening the load would change the calculations too. Good to know.

·
·
·

Hm not really imo. The reason why especially seed nodes are so expensive is that you need > 300GB RAM.

Good to know we are moving forward and also nice to see that you guys are updating us often (lately)
Just a question that might sound silly but since I am not a developer I need to ask. Do these changes require a hard fork in order to be officially implemented?

·

Valid question, thanks for asking it. These changes would not require a hardfork. It is essentially changing the way that steem nodes can store data and state, it will remain compatible with the current implementation.

·
·

So we have the option of running super expensive ram insane nodes if we want, but wont' have to? Man, so many choices to make...

·
·
·

Indeed, with MIRA the world is your oyster @aggroed

In all seriousness, this is great news because node operators and exchanges can choose to upgrade at their convenience.

·
·

Good to know.
Thanks for the response

·
·

dude, why don't u ever post? new follow

·

Oh I did have a serious question too. Is this just for seed nodes, or witness nodes too?

·
·

This is applicable to all nodes. Nice Mira' you have there.

Also, thanks for the github issue you submitted in the condenser repo yesterday - much appreciated.

·

Mira, mira on the wall. Who's the fairest hair of them all?

Thanks for the site-wide announcement, but I have one question:

What does it mean?

Most Steemians are developers and programmers and network admins, but for the 3% of us who are merely content-creators... is there a version in English? Thanks.

·

By my underatanding things are moving ahead nicely with a rocksDB module implimentation. Using rocksDB (the DB standing for DataBase) will allow the system to store vast amounts of data on the hard drive, which up to now had been floating around RAM. This large amount of RAM required and (growing with the blockchain's growth) had been increasing the cost of running a seed and/or witness nodes. Running things with the rocksDB Module (MIRA) will drastically reduce server costs and increase the decentralization of the network with more nodes coming online.

Because of the module nature of MIRA it does not require a Hard Fork and running with or without it will not make a node uncompatable with the rest of the network, so can decide to make the upgrade at their own choosing.

·
·

Thanks fam! It sounds like more people are going to be able to get closely involved with the blockchain. I don't think this affects me directly but others will be happy!

·
·
·

It will probably ensure the network's survival. That affects us all. 🖒

·
·
·
·

WOW! Thank goodness somebody did that, and saved us all. Maybe it should have been done sooner if it was so imperative!

·

Its going to be somewhat easier for people to "run" the blockchain on their computer.

·
·
·
·
·

I need to hire you to be my interpreter. lol
thanks
joy

·

It means greater efficiency for the network itself. It means nothing for content producers and consumers because you wont notice anything if it goes well.

·

There are only about 13-20 full nodes of Steem in the world (At least public nodes) because it is very expensive to run a full node. To rent a server that can run a full node, you need roughly 750 GBs to 1 TBs of RAM. We are talking about some 900 USD per month just for the proper hardware. The RocksDB implementation is a storage layer that takes the data out of the RAM and puts it onto a SSD instead. This means that the cost to run a full node can be significantly reduced. This also means more people can run the steem blockchain which further decentralizes the chain.

Does this have anything to do with the disparity between follow count on Steemit.com earlier on the https://api-int.steemit.com node(s)?

See screen cap.

image.png

Bug report submitted

Posted using Partiko Android

·

It has been doing it for a week or so on and off

·
·

It only started to affect me yesterday.

·
·
·

Yeah today was the first time for me noticing it too.

·
·
·
·

Noticed something to, went from 1272 to 1444 Followers and back again. Weird.

·

Probably just pumping everyone's numbers to help sell steemit to the advertisers now that we can support ad space.

·

That is unrelated and we're looking into it. Thanks for your bug report and your comments on the issue that was filed in the condenser github repo.

·

Does this have anything to do with the disparity between follow count on Steemit.com earlier on the https://api-int.steemit.com node(s)?

No - as the work is not yet live anywhere and neither witnesses nor full nodes or API has no code from the MIRA implementation.

Yay! This'll make it a lot easier to set up my app once released :)

Posted using Partiko Android

·

Any whitepaper-ish post(s) about the plan, @petertag, or are things still under wraps?

·
·

Hmm, I guess I should release a whitepaper-ish post pre release, but it's all under wraps for now. Hopefully can at least get a closed alpha and release by January 1 :)

Posted using Partiko Android

·
·
·

Will look forward to it! 🖒

Thank you for sharing this!

Thanks for the update steem! 🙌🏼🙌🏼🙌🏼

Posted using Partiko iOS

Fb for the old and steemit for the wise

This would be great

Thanks for the update. Lowering the cost of running a full node is of utmost importance right now. SMTs can and have been put on the back burner for the moment. This is the big cost saver and obviously comes first.

Great news, my support. Keep on!

This is a nice development and badly needed.

Awesome, been working on similar myself :)

Keep Steemin !!

Something that should have been done years ago. Glad its happening, though the fact that it took such a large drop in the market for you to get motivated to do it is not encouraging.

I hope the cost to run a full node drops significantly so that we can properly proliferate the network outwards to more servers.

This is an immensely important and beneficial move on behalf of all users of the Steem Blockchain ! Great work @steemitblog and thanks really for informing us all in this still up and coming and amazing community of SteemBlockchain lovers and affiliates !

Very interesting.
When is the release to the public expected?

Posted using Steeve, an AI-powered Steem interface

This is great project which will help steemit and steem developmen. Good luck!

Thanks for keeping us in the loop.
Most steemians will either care to know that steem will be here tomorrow.
GOOD to see you guys at work.
Keep on postin

Posted using Partiko iOS

Great news. Making nodes cheaper will definitely help adoption. I can't wait to run my own!

So you're basically just focusing on this, right? Whether it's the right move or not, I'll leave up to the experts. I can only say that I appreciate the updates! Keep them coming...

·

I sense, at this time, it is the only move. Steemit cannot afford to run the way they were and need to cut the node cost. From what the "experts" tell me, having much smaller nodes is a cost reduction.

It seems that Steemit's focus (reducing costs) aligns with the community's (decentralization) since less expensive nodes will allow others to set them up.

·
·

I've also heard some arguments against it though. That it could possibly have some performance issues?

·
·
·

That is the great part of the module aporoach, from the sound of it. An answer to a question from themarkymark in these comments made it sound like committed RAM may, perhaps, be adjustable in the finished version. At least it was not flately denied.

P.S. Right now you can run with or without the mod.

No team steem on binance.info have just updated this ;)

Screenshot at 2018-12-20 07-26-31.png

I am looking for any excuse to stop powering down but unfortunately this kind of news does not get me excited about the future of Steem. Too technical, too esoteric, too focused on damage control. Yeah I know RocksDB makes Steem more efficient but it doesn't do anything to make Steem more fun for posters or bloggers.

·

This RocksDB update will reduce the hardware costs of running a decentralized app (dApp). This would in turn help bring more and better dApps. Posters and bloggers need better dApps (better UI, better features, etc.) in order to have more fun and a better experience. So this update is about laying the groundwork.

Since Steem is a network with many owners (rather than a single company owning it), I imagine you could request a plain English translation from the community and the community will deliver. Communication is up to all of us. Many people are willing to help - it doesn't have to come from Steemit, Inc.

·
·

Laying the groundwork 3 years later.... But better late than never.

·
·
·

It's not Steemits responsibility to bring users to the dapps, that's the responsibility of the Dapps. Steemit is just the most popular dapp out there.

·
·

Apps need users. What good are all these apps if no one is using them or there is no marketing? Yeah ok. Apps and devs will be the only users.

·

On the upside, when users run out of RC's, and can't do anything for 24 hours, , they'll have lots of long techy posts to read..

·
·

;-) good take @@@lucylin

@dana-edwards - IMHO, this will result in cost reduction for dApps too as many of them run full RPC nodes. The cost saved can be used to developer better user experience for the bloggers.

·
·
·

Steemit is not the only way to use the blockchain. Othr apps have been created that have implimented many of the features users have been asking for like https://steempeak.com for example.

·
·

How many users do those apps have? What does the user retention data say? I'm not currently impressed. Show me the money.

·
·
·

All of these apps still crosspost to steemit.com, they just have more features. SteemPeak is new but has been growing pretty fast. All these apps are is another way to access the steem blockchain so it is still posting to steemit. https://steemapps.com

Wow, I am impressed. This would entice adopters to use Steem.

To listen to the audio version of this article click on the play image.

Brought to you by @tts. If you find it useful please consider upvoting this reply.

Thanks for this update, I hope it will helpful for steemians.

Nice want the steem team is putting together! Bravo ! Can't wait

Hey this is quite helpful thank you for sharing. btw i saw this trading competition from an exchange and I just wanted to ask if you can also make a review on the project that is having a competition right now. https://www.kucoin.com/#/rank/AERGO this is the one that i'm talking about.

Its actually a good move. As long as it serves its desired/ intended purpose increasing the hardware performance

Pretty fucking awesome.

Yes profanity is necessary.

Seems then less big boy toys and more legos for all of us :)

·

legos for all is better than spaceships for the elite!!

love the gif for this one..

·
·

Agreed.
!tip

Posted using Partiko Android

Great post .Thanks for sharing this post.

Really interesting. Who is the genius that came up with the idea?

Posted using Partiko iOS

Sounds like an excellent plan. Hope it all works out. And — thanks for the update.

@steemitblog
Everything that is good for steemit is good for me.
Well done guys!

/FF

As the speed of SSD approaches that of RAM, then RAM and HD will become one as good as the other I guess, and change DB architecture a bit, won't it, cutting out the need for RAM/HD swapping, saving a lot of CPU work, making node operations even more efficient, or am I pushing the boundaries beyond what is possible with present software formats...??

Excellent! :) This has my vote and upvote!

Great initiative!

Whoa!!! No more steems deducted by fees....

MirA is awesome try to check it

This is a great update for application developers. Considering the various headaching barriers we encounter from DB platforms, MIRA implementation will be a welcome development to developers. Thanks to the steem team and Merry Christmas guys.

Ah thanks for explaining the low value of steem and what you plan to do to make it higher.................

Why is this in my feed! Microaggression!!

This sounds great!

Finally real work being done from the steemit team?