Exploring Steem Scalability

in steem •  2 years ago  (edited)

In this post, we will address some of the concerns that have been raised regarding the increasing RAM usage of steemd nodes, as well as our future scaling plans. While the challenges associated with scaling are not something we will ever take lightly, we also think that many of the concerns have been raised due to some misunderstandings about how to properly/optimally operate steemd nodes. We will provide some guidance on this in the sections below, and we will also talk about several changes that we have in the pipeline for addressing our future projected growth.

What is Scalability?

The Steem community is rapidly growing, and with it, so is the Steem blockchain. Growth is great, but it brings with it scaling challenges. Other projects (such as Bitcoin and Ethereum) have been stuck at a standstill with their scaling problems for years - unable to adopt any significant changes to meet the growing demands that increased usage has placed on their blockchains. Steem on the other hand has continued to rapidly evolve and is meeting these challenges head on, thereby enabling it to process more transactions than every other blockchain combined. In other words, the majority of blockchain transactions occurring globally are being done on Steem.

We’ve been able to do this because our team is made up of an ever-growing roster of the most talented and innovative blockchain engineers on the planet. This doesn’t make us cocky; it makes us acutely aware of the scaling challenges in front of us, and we want to assure you that we are adequately prepared to deal with them. While we are confident in our strategy, we are also eager to hear your thoughts, objections, and insights in the comments.

A Brief History of Scaling

The most critical decision with respect to scaling is where you start. The more scalable the foundation, the more scalable the stack. A stack’s ability to scale tends to have, at best, an exponential relationship to the starting point. It is incredibly rare for an architecture to go from being able to support 3,000 people to 3,000,000 people overnight. Instead, it goes from 3 to 6 to 12, etc. Starting from an architecture that was already far ahead of the pack in terms of scalability (Graphene) was a critical component of the scaling strategy. Those that failed to make similar decisions now find themselves in the difficult position of having to rebuild their foundation without damaging the entire ecosystem that was built on top of it.

ChainBase and AppBase

The first major scalability-related upgrade was the replacement of Graphene with ChainBase. Thanks to its faster load and exit times, and increased robustness against crashes, ChainBase was critical to enabling Steem to process its current volume of transactions.

The next major improvement that is nearing completion (thanks to the hard work of @vandeberg and the blockchain team) is AppBase, which further improves Steem’s overall scalability through modularization. AppBase will allow many components of the Steem blockchain to run independently, which will permit steemd to take better advantage of the multithreaded nature of computers, and even enable different components of the blockchain to be run on different servers - reducing the need to run the Steem blockchain on individual “high powered, high cost” servers.

Optimizing Steemd Nodes: Block Log + State File

With respect to operating a steemd node currently, it is critical to understand that Steem requires two data stores: the block log and the state file. The block log is the blockchain itself, written to disk. It is accessed infrequently, but is critical to verifying the integrity of new blocks and reindexing the state file if needed.

The state file contains the current state of Steem objects, such as account balances, posts, and votes. It is backed by disk, but accessed via a technique called memory-mapped files. This technique was introduced in December 2016 with the release of ChainBase.

Everything RAM?

Many node operators are suggesting that servers should have enough RAM to hold the entire Steem state file, due to the fact that Steem's performance drops when the operating system begins “paging” Steem's memory, which is a common memory management technique. We want to be very clear that it is not required to run a steemd node in this way. This is certainly a valid technique for increasing the performance of reindexing the node and servicing API calls, but is only useful in a limited number of cases. In the majority of cases (including with witness, seed, and exchange nodes), it is sufficient to store the shared memory file on a fast SSD or NVMe drive, instead of in RAM.

Witness and Seed Node RAM Requirements

When running a steemd node with only the witness plugin enabled (the common configuration for witness and seed nodes), Steemit recommends 16 GB of RAM, although 8 GB is likely sufficient if your node does not need to reindex often. If the shared memory file is stored in /dev/shm/, then additional RAM would be needed to hold the entire state file, but this is not a recommended configuration. To avoid the need for extra RAM, the shared memory file can be stored directly on a fast SSD or NVMe drive.

A server with 8-16 GB of RAM will be slow with reindex, but it will function properly as a seed/witness node once it is up to date with the latest block. Running on a 32 GB server is ideal for optimal replay times, but it is not a requirement for a witness/seed node to properly operate.

Shared Memory File Size

The default configuration for a steemd node stores the shared memory file in the data/blockchain directory. As long as this location is on a fast enough (SSD or NVMe) drive with sufficient space, then the default setting should work.

The current recommendation is to have at least 150 GB of fast SSD storage, which includes the block_log (currently around 90 GB) and shared_memory.bin (currently around 33 GB). These amounts will increase over time.

Whenever the size of the shared memory file has increased beyond the size that is configured in the config.ini file, it has been necessary to update the configuration to a larger size and restart the node. There will be a change included in the next release (Steem 19.4) that will automatically increase this limit as needed, without the need to restart the node. This will be able to be configured and turned off entirely if you want to keep your state file in /dev/shm.

“Full Node” Requirements

Nodes that are running additional API plugins (especially account history) will require more RAM to support a larger state file. A “full node” (one that is running all of the plugins) can technically run on a 64 GB server, but it will be extremely slow to reindex, and it will be slow at serving API calls because the operating system paging algorithm does not handle memory-mapped files very well. A node with 64-256 GB RAM and a fast SSD/NVMe drive may be adequate for many use cases, depending on the load.

Increasing Performance on High Use Nodes

For more heavily used nodes, the best way (currently) to increase the performance is to have enough RAM to hold the entire database. This skips the need for paging altogether, which technically defeats the purpose of having a memory mapped file. For a node running all of the plugins except account history, this currently requires 256 GB RAM on a pre-AppBase node.

A technique that we have been using to lower the memory requirements on a “full node” (one with everything including account history), is to split the API node into two servers. One server runs only “account history,” and the other server runs everything else. This allows both servers to use less than 256 GB RAM, instead of running everything on a 512 GB RAM server. We strongly recommend running account history on a dedicated node if you want a complete history for all accounts, since it eliminates the need to have a single 512 GB RAM server.

Optimizing the use case of a “full node” is a top priority of ours, and one that we will talk about more in the next section. If you only need history for certain accounts though, or only care about certain operations, the hardware requirements may already be significantly reduced.

Future Scaling Plans

We are currently working on several projects that will reduce the memory requirements of “full nodes” by moving much of the API logic into non-consensus plugins such as HiveMind and SBDS. This will allow a lot of the functionality to be run off of SSD storage, rather than in RAM - which will lower the operating costs. By offloading data to hivemind/sbds and/or RocksDB (below), we should be able to reduce the requirements for a full node down to the same requirements for a consensus/seed node, which is an important goal of ours.

RocksDB

In addition to the non-consensus plugins, we have begun research on using alternative data stores and moving away from Chainbase. One such data store that has shown promise is RocksDB.

RocksDB is a fast-on-disk data store with an advanced caching layer, which could further minimize latency when reading/writing to and from the disk as it is optimized for fast, low-latency storage. Used in production systems at multiple web-scale entreprises (Facebook, Yahoo, LinkedIn), RocksDB is based on LevelDB but with increased performance thanks to its ability to exploit multiple CPU cores and SSD storage for input/output bound workloads. Its use in MyRocks, for example, lead to less SSD storage use, longer SSD endurance, and more available IO capacity for handling queries.

Further Modularization

We are also working to modularize the blockchain beyond even what was originally planned for the initial AppBase implementation, for example, by having separate services that can be run on different servers. This will allow processes to be further spread across many small servers, increasing flexibility and decreasing cost.

Conclusion

As blockchain projects continue to become more mainstream, scalability is going to become more and more of a concern. Being a scalable blockchain is not just about being able to make a one-time fix to meet the current resource challenges. It is about being prepared to meet the future challenges as well.

Steem has already proven itself as the fastest and most heavily transacted public blockchain in existence, and scalability continues to remain a top focus of ours. We know that scaling challenges will never completely go away, which is why we plan to continue innovating to ensure that whatever growth comes our way - we'll be ready.

Team Steemit

P.S. Don't forget to share your thoughts, objections, and insights in the comments!

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Can you guys please STOP CREATING NEW ACCOUNTS until you can properly mitigate the exploiters that are creating huge botnets and siphoning rewards from the pool.

You want to talk about scaling and resources? Most of the resource problems we have around here are directly related to exploits/spammers...problems that lie directly at the feet of STINC, via recent the last few hard forks and account creation.

Correct the bad decisions you’ve made and are making that have caused/are causing most of the current problems.

yo_dawg_inertia.jpg

This problem was already brought up to light recently, I believe that blocking new accounts creation is a poor decision, it would just throw out the baby with the bathwater.
Those who are in charge of accounts approval are making a terrible job allowing creating entire botnets and not letting real users in. They should be relieved from that task. Maybe there should be a new mechanism of approval.
Maybe it's better to make a new user create a post (like, obligatory introduction post, for instance) instead of making them wait months for approval. Then something like committee from known and established users would (cu)rate it, and make the decision if the user should be approved. It's a win-win for both parties.
One of the STEEM foundations is Proof of Brain, right? That's how it's done, no spammer can automate the process and those who can't put two words together will also be kept out. It's not like I'm hating all those Indonesian guys, writing about how happy are they to be on steemit with poor grammar and everything else, but they are not bringing any value anyway. There will probably be need for multiple groups for handling multiple languages, but that's another topic altogether.

“I believe that blocking new accounts creation is a poor decision...”

I’m not saying all account creation needs to be stopped. I’m only asking that those created via STINC’s current process be halted. There is an obvious, major malfunction with their system that needs to be addressed.

This is akin to what governments typically do. They undertake certain tasks/functions, they perform these tasks horribly, resulting in many undesired but predictable consequences, then they propose “solutions” for the very problems that they have in fact created. And all the while, everyone else (in this case users, potential users, and investors) ends up paying the price for their inefficiency and complete ineptitude.

Accounts can still be created. But it’s plainly obvious that whatever methods STINC is using are inadequate and are contributing to the creation of large bot-nets that strain the current resources and siphon rewards from the shared pool. The solution is to shut down the failing system in place until it can be corrected so that further damage is not done.

Compounding the errors/damage when it can be easily avoided is irresponsible. It’s time that STINC acknowledges their role in the mess that they have created...and then they ought to FIX IT.

Agree. Centralized account creation is a bloody hell right now. As all centralized solutions.

Yea, agreed too

Yep, as someone that has been downvoted to negatives by on abusive account holder: @haejin

I would say there is a massive loophole that needs to be fixed before @steemitblog can be taken seriously

Yeah, I can see how competing merchants or businesses would get into silly downvoting wars and whoever has the more goons wins.

And then there are the trolls...

until you can properly mitigate the exploiters

This is like asking a fat person to mitigate other fat peoples food intake ... or something.
¯|(ツ)

via recent hard forks

Not so recent, the last one was about a year ago, but I know what you mean.

Yeah, that’s correct. I edited it.

And it should also be noted that they claimed to have HF20 almost ready to go when HF19 was implemented in early June last year. Ten months later...

For reference:

https://steemit.com/steemit/@steemitblog/proposing-hardfork-0-20-0-velocity

nah, first will be 19.4 ...appbase?
Or will they do both, HF20+appbase?
Because, well, why not introduce 2 error factors

Only two would be an improvement. We’re used to a minimum of five at once around here.

Maybe it needs to be an even number? :)

I thought STINC was using a Fibonacci sequence.

Thanks for the feedback. We understand this is an important topic for many users and will be addressing non-scalability related issues in future posts.

Great. Looking forward to it.

But for the record: This is a scalability-related issue.

It's like when you have a design for a bucket.

Don't make a damn if the bucket is 5 gallon or 10 gallon when it is riddled with holes.

Better to worry about patching the holes before increasing the capacity of the bucket.

We are losing users and rewards to scammy bot masters.

Hope they get their priorities straight. I really want this platform to succeed.

So it is more important to make the system more streamline for the bots? Isn't that like removing part of your brain to make room for the tumor? I really don't understand this platform works.

Thanks for sharing! ;)

bad/greedy guys will be bad/greedy and you asking wont stop them lol its kinda what they do.

criminals break laws it dose not mean we avoid arresting them, something should be done about the abuse of the upvote system

Nice comment

I don't personally know much about any of these issues, yet, but the comment here by @ats-david certainly seems like it's worthy of a reply at least. Yes?

Can we please understand each other we have different views and reasons.

I am new here, but I seen a report somewhere that like 90% of the users are Bots, so he has more than a point if that is true, it seems they have co-opted this platform to enrich themselves.

I am also new. I thought an upvote drained power from your computer? So, if that is correct, then just don't upvote a bot, but assuming your report is true, then I'm probably wrong and it's some other technical mumbo jumbo reason. Or maybe astroturfing from competitors? lol, like facebook or something, or even the CIA who rely on corporations to collect private data.

BOOGA-BOOGA-BOOGA!!

I guess it could be a million things besides greed. what report? Link would be nice. I assume they are allowed here. I wont believe you in case you are a bot ;P

wtf-am-i-rrading.jpg

couldn't agree more sir

It's called load testing ;)

  ·  2 years ago Reveal Comment

Hey team, the way you handle so many daily transactions leaves me to think you definitely know what you are doing with scaling. Keep up the good work. All we need is communities or some kind of organizing on this site and it can be a top 100 trafficked site no problem.

There are many things beyond this post that should of been covered, sure steem handles an impressive amount of transactions, does that make it the best blockchain?

we need accountability, on this platform much so with the rampant amount of bots running loose.

Absolutely Agreed !

Agreed.

Congrats, you made the @dtube #steemitminute for today!

Click the Image Below


Where do I get one of those fancy Dtube shirts?

FWIW, /dev/shm does not imply physical RAM. It is backed by virtual memory which can include swap. A properly configured system with the state file in /dev/shm will require less physical RAM and/or perform better with the same amount of RAM than one using a disk file (at least on Linux; I can't comment on other OSs). The trade-off is lack of persistence of the file across reboots (unless it is explicitly copied).

Steem full node is essentially a data-mart with a task to low-latency response to a fixed set of queries. I believe the industry-best solution to this problem for now is a RDBMS cluster with a Redis cache.
Why don't you use it?

Looking forward to seeing how this turns out. As someone running three witness node and five full nodes, these changes are long overdue and hope they provide some relief.

I hope to see more frequent detailed communication from Steemit Inc in the future like this.

Thank you for this detailed explanation of everything, as a non-techy I found I was able to grasp it much better and I'm as determined as ever that Steem has an amazing team behind it focusing on the important aspects which is the scalability of the blockchain. We are already so far ahead of many other blockchains in terms of real world use and promising apps being built on top of it.

Resteemed.

Glad you think so! Thanks!

Great work, Andrew! I'm so excited to see this regular, professional communication. I know how much work goes into creating something like this, and I really, really appreciate it. It's a huge investment of valuable time, but this communication is so important (IMO).

Thanks Luke, we’re getting better at it. Just another scaling challenge ;)

I do not understand technology. So I follow your opinion. My friends in Indonesia say you are a caring Steemian.

I have a renewed sense of confidence. For a while I've been very concerned about scalability and had some doubts about the future of Steem. Thanks for the node and RAM suggestions, I hope the new AppBase/RockDB implementations will slingshot Steem to the next level to accommodate the ever increasing user base and transactions.

Thanks to all the dev team for their relentless hard work.

I like the beards master.

the-master-and-the-student.jpg

hahaha

Thank you for sharing your concerns with us!

  ·  2 years ago (edited)

This is a great explanation, especially for those a bit newer to Steem/Steemit. I have a couple of questions that the devs may be able to help with.

What is the current size of the state file of a full node minus the account history? and what about for a node running only the account history plugin?

In the current Steemit backend, is the Jussi (reverse proxy server) accepting requests, and redirecting them to RPC nodes running different plugins if they're not in the cache, or is it using the SBDS database for some things already?

Hi @andybets - thanks for your question. Currently a node with only account history uses 204GB for the state file while a node with everything except for account history uses 190GB. Today, Jussi (api.steemit.com) is being used to split requests to the different clusters of nodes if they are not already cached. In the future, many of these requests will be split off to SBDS (for account history) and Hivemind (for tags/follows). RocksDB for account history (and later probably tags/follows) will be an alternate option. Either configuration can be forwarded through Jussi without the frontend (Condenser) needing to be aware of the change.

Thanks very much - that's really useful!

You rock, Justin. I hope you're not working too hard and are positioning yourself to get that vacation we talked about in Lisbon. Thanks for all the work you and the team are putting in and the results we're all enjoying, being part of the most performant blockchain on the planet.

Thanks Luke!

Bitcoin has been here for 9 years, the core could't figure it out how to scale well. Operating cost for Bitcoin blockchain is insane, energy costs, forks, clampdowns, you would think some other chain would take a lead by now.

Steem is two years young, it is working great, we are the busiest blockchain alive, more lives have been changed for the better, anyone who has seen the potential here from the early days is quite happy by now. This is just a beginning. I'm glad STINC is thinking ahead, this just reinforces my hopes for the future. Steem on!

Thanks so much for your comment - do you have to call us 'STINC' though? Just sayin' 🙂

Oh, I thought that was just a short version for SteemitINC, not saying that you stink or something :)

STeemit INC...STINC

Seems like an appropriate abbreviation to me. It beats spelling out “Steemit Inc” every time.

  ·  2 years ago (edited)

How about SINC ? (shorter to type lol) 🙄

Sounds and looks better.

Yes STEEM is a great platform. It is a paradigm shift. Nothing happens overnight. And the good thing is that if we don't like something about it we can fix it. What if we all focused on fixing the issues along the way?

Steem is one of the best blockchain. However, scalability is a real issue regarding of growing community.

I would rather have a post on exploring steem development...

Steem team is amazing!

Arca.jpg

Scalability level GOD, Best regards friends of @steemitblog!!

how do we fix the bugs from spammers , creating hundreds of account, posting thousands of the same words of the article.

Very accessible explanation and update for the community. Thanks!

I have been running a witness node since September 2017. I have never put any shared_memory on RAM, all of it on a regular SATA SSD, and my RAM usage hovers under the 1 GB mark. Never had any problems at all, with disk usage around the 5% mark. Replays probably would take a while, but I have actually never replayed - not even once - since September 2017. It has just been rock solid. I've been playing around with a NVMe RAID0 server, and experimenting with /dev/shm, but can't think of a single reason to go the RAM route. Replay finished from scratch in three and a half hours, and as demonstrated above, replays are pretty rare. So, I'll stick with NVMe SSD for now, Optane next. Hopefully, with all of the scalability improvements, I won't be using RAM for years to come.

PS: I have noticed shared_memory is very compressible. I'd be looking forward to some compression tech built into Steemd. Of course, we can use existing workarounds now.

Thanks @liberosist - I really appreciate you communicating your real-world experience with running a witness node on this post.

You are also correct about the shared_memory file being fairly compressible. We actually run a service that compresses state files regularly and are pulled in and uncompressed on startup. This makes it possible for us to autoscale steemd instances on-demand.

Ah, so I would like to see that tech built into Steemd itself so it always stays compressed - even when it's running. I know this will add CPU overhead, but since AMD's EPYC and Intel's Skylake-X response, we are headed into a world with more CPU cores than we know what to do with. I know some have experimented on zram on Linux, and it works fine. Anecdote - back when Steemd ran on Windows, the in-built RAM compression in Windows 10 kicked in. Back then, it was all on RAM (no shared_memory) and a full node typically used 15 GB. In Windows 10, that was down to only 3 GB, the CPU overhead was minimal and it ran flawlessly. Of course, very different times now, but I'd be looking forward to seeing compression tech built into Steemd.

RocksDB has built in compression. It will store all data on disk compressed and in a machine independent format.

Great to hear! Look forward to the transition from Chainbase to RocksDB.

@Vandeberg

on the STEEM master, I did

git log | grep Vand | wc -l = 1801

Appreciate your hard work. I keep seeing your commits !

I wanted check few details:

  1. So RockDB will be will keep the historical data and provide fast access on-demand ?

  2. The new transactions only can be in the memory / blockchain and thus we can get something similar to Nano blockchain ?!!

  3. If we achived, point2, then we may not need multi-threading for the near future ?

  4. Now, if two and three are true, STEEM can be a general purpse, blockchain on steroids and can easily compete with other specific chains like Hyperledger ?!!

  5. On Hivemind, I observed that the PostgreSQL + the communities logic is confined to single boxes. So, right now horizondal scaling is not planned ? ie, in the event of PostgreSQL hitting its limits + CPU requirement for the logic needs much more than the CPU/RAM capabilities of single nodes ? (In a nutshell the current implementation can scale vertically till some point and later for the facebook-scale, we would need to add support for horizondal scale)

@Vandeberg

Sorry for re-bumping this old comment thread, but didn't know how else to get in touch with you! I was referred to you by Julián González.

I'm José Macedo, senior analyst at AmaZix (https://www.amazix.com/ // https://www.linkedin.com/in/ze-macedo-15b1b175/). In case you haven't heard of us, we're one the largest community management, consulting and advisory companies in the crypto space, partnered with Bancor, HDAC, Bankex and many others. We've worked with over 100 projects and now have over 100 employees.

We're now working on a forum project with a token to incentivise quality content/curation. We really like the STEEM inflation algorithm for this and are very interested in building on top of STEEM. Our ideal scenario would be to use the SMT protocol, but we realise launch is scheduled for January and we simply cannot wait that long to launch.

We’re curious if we’d be able to chat to someone from the SMT team to discuss our options in the meanwhile. Currently, we’re leaning towards issuing an ERC-20 and then switching to SMT’s once they launch, but we’d love to find a way to build on STEEM from the get-go.

Let us know if you have some time to talk and discuss our use case.

Thanks,

José

@justinw - read your other comments. Keep up the good work.

I have questions / doubts:

This makes it possible for us to autoscale steemd instances on-demand.

ie you are compressing the block_log files and decompressing for auto-scalng ?

  1. Are just using xz/zip/bzip/lzma or something else ?

  2. Cursious about the auto-scaling as well - so you use something like ELB from AWS or a hardware (F5) ?

Thanks in advance.

How did you update steemd?
There has been a patch at least at the end of December/in January?
For that, recompiling and restart of steemd process, you should have needed a replay.

The update did not require a replay. Just a rebuild and restart.

thanks for the great and the simple explanation this post encouraged me to join the witness team ;) soon

great info

The exponential argument is convincing, even for those of us not-blockchain experts. It's intuitive because it agrees with the reality we are familiar with. It is easier to take a building from being able to support ten people to bring able to support twenty than two hundred. To do that, you would have needed to start with a building that could support one hundred (although the analogy isn't perfect because that you're of progression is usually at best linear).

I have been noticing some glitches since I have been on steemit. Hopefully it does turn out to be majorly scalable. Keep up the hard work. It will be interesting to see what the future holds.

Can't wait to see RocksDB in action ;-) Thank you for sharing all tech details. MOAR! deep dives please ;-)

This is fascinating stuff. I'm interested to see how Steem/Steemit can scale to many millions of users, as it must do to compete. You can't just keep adding memory forever. How much of a bottleneck is CPU?

CPU has not been the bottleneck in steemd - but with the release of appbase it can be. Making CPU the bottleneck is actually a good thing because then it becomes horizontally scalable based on usage. It gives you a metric that you can actually scale instances on - previously it was based on the number of requests, which is difficult to guess at because certain requests can take longer than others. To answer the question directly, appbase steemd can handle an extremely large number of requests before becoming bottlenecked by CPU.

Thank you for addressing the scaling challenges and how steemit working on solutions to stay ahead, excellent news!

Please continue to provide regular updates on planned developments / improvements since this gives people, including me confidence about steems future.

I have a question....this blockchain was created with 3 second transaction time...is that something that could slow down if the scaling you mention in here is not done? Would we see a backlog of transactions, potentially, like BTC and ETH experience?

Or is the scaling issues more to do with the time it takes the applications created to interact with the blockchain?

Those are great questions!

The block interval does not impact scaling in any way other than the fact that it is part of defining the maximum growth of the blockchain itself (not state size). max_block_size / block_interval is the maximum growth rate for the blockchain in bytes. We are not experiencing any problems in the live performance of the blockchain. We would likely only consider changing the block interval if there was a compelling reason to for the benefit of live performance.

Some of the changes we have been working on do specifically target response times for our APIs. Appbase parallelizes all API requests. Previously, requests were handled serially. This doesn't scale well and is a waste of resources as almost all computers now have multiple CPU cores. With Appbase, we are now utilizing the other 90% of our servers that weren't being touched previously.

Why not adopt MPI based code... in the steem source code... I mean... if you can have 100 cell phones.. doing a steem node... because you can't buy a BIG server! Would not that be awesome?

I mean... even if you look further... this will give STEEM a new market... which is full of performance... called HPC (High Performance Computing)... which is also inside the cloud, and reaching out for AI, deep computing and deep learning.

I am not joking, and anyone that can spin up a VM for a normal wallet, can also spin up a wallet for a an HPC cluster that can outperform many of the BIG VMs inside the cloud offering. Just check it out!

HPC is the way! No more... careless infrastructure!

I have bumped my comment (voting on it).

Good to know steemit.inc is scaling up quite nicely.I want to say a big thank you to all the witnesses investing their time and especially money on hardwares to power up the platform to unprecedented heights .You guys are the real MVPs.

Steem Scalability !!!!!

Whales are getting much more than expected, new users are finding it difficult to even sustain on this platform.

Reward pool is decreasing on every 250k blocks as well as it has fixed amount of STEEM in its reward pool. Users are flooding like hell, so What Will They Get From Here ??, Just Pennies !!

What if it onboards the same numbers of users that facebook have. How It Rewards Such A Big Numbers Of It's Users ??

Sorry to say but I think that it doesn't have the potential to compete with facebook, it doesn't have the potential to reward it's users.

I have given the idea that "Instead of using Steem Power as the method of determining the reward amount, We can bind one vote to fix amount. Steem Power will be just used to allocate bandwidth, The more Steem Power a user have the more he can post on this network.

You are missing the main point here. The more users onboard, the more investors onboard and the more 1 Steem will be worth.
With 200 million users you might only get 0.1 Steem per post, but if 1 Steem is worth 1000 dollars, you'll still get a decent reward.

1000 dollars seems a bit to far fetched, more does not mean better.

I just took a huge value for the Steem value and a low value for the reward just to make a point.
The values are not supposed to be realistic.

I figured as much but yea, I am not dying steem is going to be amazing in the near future.

The work and reward some one gets on here is better than any coin out there.

Thank You for explaining👍
Steemit is growing by the minute, everyday.

You seem to be missing the point as well. More users does not mean more investors/value.

That's not entirely true, the more users there are the more interesting Steem becomes for investors. For sure, it also depends on target groups etc. But the more users there are, the more potential target groups there are as well.

Indeed. So it doesn't mean more value. It means there might be more value. That's not the same. It's not like selling cars where selling more means more profit. I didn't mean to drag this open. Just wanted to point out there is no one to one relation.

Of course not, and I'm pretty sure that people who post nothing of value will get way less than they get now, because their share will be smaller, but that people who post things of value will probably get more since the userbase is bigger which brings some sort of bigger investments with it, even if not 1:1 related.

SMT is a fix for this I assume, if Steemit Inc pulls it through, than community tokens and proof of brain will succeed.

That's a really good idea for making it easier for new people to get traction on here when no whales (or even dolphins) are following them to give them bit upvotes. And while in some ways this is a side conversation, since this post is really about scaling the tech, not how to increase the size of the user base, the two are definitely related. So let's play.

So yes, would be good to give new people a chance to more powerfully upvote each other the same way old-timers can powerfully upvote each other. However, there is also something to be said for providing someone who has invested their own money in the platform with some additional stake in things like how rewards get allocated. The same goes for those who have invested many months of their time in posting and growing quality content on the platform. I don't think bandwidth, the ability to post more, is adequate compensation for that investment.

And speaking of investment, we also want STEEM to be an attractive one, even for people who don't intend to post. It may seem like SP wouldn't matter to them at all, but not so, because now they can delegate their SP at an interest rate of about 25-30%. That's a darned good return! In fact, I'm currently developing an advisory relationship with a socially conscious investing firm, and guess which coin I'm going to be recommending at their #1 play! But this only works if other people have a reason to pay those returns to them to buy their delegated SP. And the only reason someone buys it is so that they have bigger upvotes.

The problem is for people who have no money to invest at all and also haven't been here that long, so haven't invested sufficient time either. I don't know that there is a way to equalize for that, or if there even should be. Might the initial SP delegation be bigger? Yes, I think that would be good. I think now the investment of time needed to get any traction is just too long. I also think that delegation should remain longer after the person starts buying small amounts of STEEM they can afford. Right now if you buy only $20 of STEEM you basically just replace some of your delegated amount with that. You get no additional upvote ability. Ouch! I keep trying to explain to people they need to invest at least $100 to get any kind of decent upvote, but not everyone has $100. Do we not want them on STEEM dapps?

So there is definitely room for improvement in how we scale the user base successfully without churn. But I don't think it's as radical as decoupling SP from upvote power.

Hmm🤔, true.

Exactly what I thought @raycoms.

Even if we still get pennies, that would mean that the project is a (sound) success.

Steemit is a big success indeed.
✔️

I disagree about Facebook.
Facebook cannot compete with Steemit.
Facebook is just a waste of time, while Steemit is a community of Supportive people who look out for each other.

I've made more resourceful friends here on Steemit than on Facebook.

Facebook is a lie.❌
Steemit is the real deal.✔️👍

True. Steemit >>>> Facebook.

Definitely.

Working on improving the front end is more important than scaling issues right now because it is impossible to find good content on Steemit. Furthermore, Steemit doesn't offer advanced features like notifications compared to busy.org.

so far I study the workings of steem and I have yet to get a fatal error, for now the system works has been very good, I have yet to ever find greatness in any platform other than this here, the team is very wise in making decisions, all users will be very thankful with this notice, even if they do not know about scaling, Hopefully more good again with spaces larger, blockchain, We don't want to lose steemit, keep working to make great changes, If you agree then I will turn this post into indonesian, and I will re-post with give the original source of your postings, to open an insight on my State about the greatness of scalability, AppBase, Steemd, RAM, Full Node, and the Node performance improvements.

Sounds good to me, go for it!

Andrew Levine
Content Director, Steemit

Hi... I've finished making the posting into my language, I am very grateful to @andrarchy who has given this great opportunity, Hopefully my friend here can add science with this post.
Here's the post I have translate.
Beberapa system penskalaan (Exploring Steem Scalability) yang harus dipelajari sebelum menjadi Witness

Do I really get permission to edit this post into indonesia?
Wow Thanks a lot...

This is great information and timely. I was just looking for an explanation, best one so far. As a few of the others have said well written for those looking to see what it takes to run the different nodes. Much appreciated.

Wow, interesting

Posted using Partiko Android

the majority of blockchain transactions occurring globally are being done on Steem.

Love it!

Thank you for the update!

Wishing you techie wizards the very best with the Steem scalability solutions, because if mass adoption of Steem is to come and over-spill into every day world transactions, and be truly disruptive, as so many hope for, we will need this fundamental development to carry us forward and bear the weight!

So... it's more important to see this focus, than a lot of the vapid opinion about the nature and design of steemit itself, which has surely already proved itself as the effective platform for free speech we need.

As for decentralisation versus centralisation, I personally am less worried about this aspect. I think it's good to have some level of expert, dedicated control over the protocol, for now.

Best of luck and please keep us updated!

You guys have evolved so much, it is encouraging to see how your team has developed so much in just two years and how also the project, Steem and Steemit has come to be a great amount of applications on top of a powerful blockchain.

It is also great that you detailed the problems that Steem has right now with the RAM and how it should be addressed by witnesses and node owners, as this started to be a problem pointed out by many of them.

Overall you are doing great and really improving your communication with the people here who missed it so much. Keep going!

Thanks for the words of encouragement. We're trying!

Kenapa bisa di coret itu

Steem has already proven itself as the fastest and most heavily transacted public blockchain in existence, and scalability continues to remain a top focus of ours. We know that scaling challenges will never completely go away, which is why we plan to continue innovating to ensure that whatever growth comes our way - we'll be ready

I love the fact that @steemitblog and it's team are working daily to improve the blockchain and Steemit experience for its Millions of users.

I DON'T NEED A SCALE BECAUSE I HAVE ONE IN MY BATHROOM ALREADY!
YOU'RE NOT MY MOM.

Really i appreciate you.Thanks for your content written.

Thanks for article...

Such an interesting and educative post,thanks for educating us.

"In other words, the majority of blockchain transactions occurring globally are being done on Steem."

This is one of the best things about STEEM. If only we would have a wider public adoption. :(

Sure but lets fix steemit before it goes even more public, its a shit storm right now.

This has been the argument for two years. We’re not any closer to making this place not a shit storm (and it’s actually worse), nor does it seem that we’re any closer to reaching a more mainstream audience.

Keep up the good work !

Hey @steemitblog and @ned , Thank you all for your excellent work. I have super confidence in you, I think the future is super positive!


Steem has already proven itself as the fastest and most heavily transacted public blockchain in existence, and scalability continues to remain a top focus of ours. We know that scaling challenges will never completely go away, which is why we plan to continue innovating to ensure that whatever growth comes our way - we'll be ready.


I send you my strength!

@lartist-zen

To listen to the audio version of this article click on the play image.

Brought to you by @tts. If you find it useful please consider upvote this reply.

  ·  last year (edited)

If Steemit does take off in the future many of us will be in great shape. It doesn't do anyone any good to fill the platform with bots and the BS fighting we see here sometimes. All of us just keep trying to put out decent content. I always keep trying to come up with something interesting to post., but I know I have posted some less than great stuff sometimes too.

Terimakasih @good-karma