The Bandwidth Limitation - Problems and Solutions

in witness-category •  5 months ago

The bandwidth limitation hasn't been resolved yet. It remains a daily bottleneck to many minnows with low SP. The issue has been discussed for a while now among the witnesses, but no concrete action has been implemented. The problem is multidimensional. However, there are some measures that everyone can undertake to help mitigate the issue to reach an acceptable solution. But first, I'll explain the basics of the problem.

Global bandwidth parameters

The Maximum Virtual Bandwidth represents the amount of the bytes available for all users to share in their transactions. A transaction can be a vote, fund transfer, comment, post... every little thing that's done on the blockchain is a transaction that is signed by the user's private key and validated by the system using the user's public key.

The Maximum Virtual Bandwidth is calculated with this equation:
max_virtual_bandwidth = number_of_blocks_per_week * maximum_block_size * current_reserve_ratio

  • number_of_blocks_per_week, constant value calculated by: 20 blocks/min * 60 min/hour * 24 hour/day * 7 days/week = 201600
  • maximum_block_size, parameter set by witnesses and is currently 65536 bytes.

So in theory, with only those two parameters, we can have a maximum of:
201600 blocks * 65536 bytes = 13212057600 bytes (or 12.3 GB) per week

However, the third parameter current_reserve_ratio acts like an anti-spam mechanism to prevent an overflow of the maximum block size. For example, if a block contains more than 65536 bytes, the exceeding transaction(s) would transfer to the next block unless they expire. So to prevent this overflow, the current_reserve_ratio automatically adjusts when the block size hits 25% of its maximum (25% * 65536 = 16384 bytes), thereby decreasing the max_virtual_bandwidth for everyone.

Notice that when querying the blockchain, e.g. with getState():
current_reserve_ratio has a maximum of 200,000,000 with a precision of 10^5 (to avoid decimals), so it's actually 20,000.
max_virtual_bandwidth has a maximum of 264241152000000000000 with a precision of 10^6 so its value is 264,241,152,000,000 bytes (see below).

So back to our calculations, the total weekly data allowed by the system is:
201600 blocks * 65536 bytes * 20000 = 264241152000000 bytes or (brace yourselves) 240 Terabytes per week

It's a ridiculously astronomical figure that would make Dr Evil very happy.

But, since the reserve ratio is kicking in at 25%, we're down to 60 TB per week (still a big number). Fortunately, those are potential numbers allowed by the system. To transfer 60 TB of data per week would require a sustained amount of transactions saturating the system, which is not the case for now (unless Steem grows to hundreds of millions of users). Furthermore, I don't know why the developers decided to set that 20000 value for the reserve ratio, but something much smaller (e.g. 2000) would have been more realistic.

Individual bandwidth parameters

So far I've covered the global bandwidth parameters handled by the system. The other important consideration is the bandwidth allocation per user. Briefly, the calculation relies on a few parameters:
vesting_shares, user's own vests
delegated_vesting_shares, vests delegated by the user (outgoing)
received_vesting_shares, vests received (incoming)
total_vesting_shares, global vesting shares

allocated bandwidth = (vesting_shares + received_vesting_shares - delegated_vesting_shares) / total_vesting_shares * max_virtual_bandwidth / 1000000
(remember that 10^6 I mentioned earlier)

There's another parameter average_bandwidth (weekly average bandwidth), that is used to determine a user's remaining bandwidth. For a detailed calculation check https://steemit.com/utopian-io/@bloodviolet/how-to-calculate-your-remaining-bandwidth-using-steem-python

Hence, the allocated bandwidth depends on the user's total Steem Power and average weekly bandwidth. By increasing the Steem Power and lowering the weekly usage, a user can increase their allocated bandwidth.

Bandwidth limitation solutions

By understanding how the bandwidth works, we can think of a few solutions for all the blockchain actors to remedy the limitation problem.

Witnesses

Witnesses can increase the maximum block size which is currently at 65536 bytes. I'm personally in favor of this measure, however two concerns have been brought up:

  • Spammers may spam more.
  • The hardware requirements will increase, especially the RAM, which is a huge setback for scalability at the moment. This not a problem for a witness or exchange node, but for a full RPC node it's more serious.

AppBase should solve some of the scalability issues:

Also, Hardfork 20 promised multi-threading for the steemd daemon, but to my knowledge there hasn't been any mention about RAM optimizations neither in AppBase nor in HF20.

Because the algorithms and the daily usage are complex, we can't predict with certainty the extent of increasing the block size. Nonetheless, we can increment the block size by 10% for example and monitor the behavior of the reserve ratio to see if there's any improvement, then adjust the parameter accordingly. If modifying the block size doesn't help, at least we gain this information and move on to other solutions.

Users

Users can check their bandwidth status at https://steemd.com/@accountname (replace "accountname" with your own, of course). As explained above, users can easily address their own issues by:

  • Reducing the weekly transactions.
  • Powering up more STEEM.
  • Avoiding the rush hours.

Bots

Many bots usually write a comment when they do their action (e.g. upvote a post, welcome message, etc). Some of those comments are long and often span multiple paragraphs. Eliminating those automated comments or reducing their size can be beneficial. Although not malicious, repetitive comments is spam after all, so do less of it.

Steemit Inc

One factor that has caused more bandwidth issues for minnows is the fact that Steemit used to create accounts with 57,000 VESTS (~29.5 SP) delegation, but has cut that nearly in half to 29,700 VESTS (~14.5 SP). Also, Steemit has been systematically reducing the old delegations to ~14.5 SP. The reason for that is of course to accommodate the increased sign-ups, but by doing so they unintentionally paved the way to the current bandwidth limitation affecting the minnows.

Fortunately, HF20 promises to address the on-boarding by removing the need to delegate to new accounts by burning the account creation fee instead. So until HF20 is released, I see a few possible options that Steemit can do:

  • Reduce or halt the sign-ups.
  • Reinstate the 29.5 SP delegation.
  • Remove the delegations from accounts that have accumulated a minimally acceptable SP to transact (e.g. 30 SP).
  • Accelerate the release of HF20 (obviously).

Developers

  • The developers could tweak the algorithms to be more permissive by increasing the 25% reserve ratio threshold to something like 30-35%, especially during the rush hours.
  • Add more verbose information about the bandwidth in the profile page, instead of constantly having to check that on https://steemd.com.

Final Thoughts

I hope the bandwidth limitation problem improves soon, because it has been frustrating many users for weeks already. It's important for Steemit to provide a comfortable experience. It's one thing to encourage adoption and on-boarding, but it's another thing to see users slapped in the face when they can't transact and being left in the dark about why this is happening.

How about you? Do you have any solutions you'd like to share? Please comment.

I have mentioned rush hours. I will cover that topic in an upcoming post.


Follow

Available & Reliable. I am your Witness. I want to represent You.

🗳 If you like what I do, consider voting for me 🗳

Vote

If you never voted before, I wrote a detailed guide about Voting for Witnesses.

Go to https://steemit.com/~witnesses. My name is listed in the Top 50. Click once.

Alternatively you can vote via SteemConnect

https://v2.steemconnect.com/sign/account-witness-vote?witness=drakos&approve=1

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

I have to keep an eye on my bandwidth thru the steemd site you mentioned, as I have been strangled a number of times from a sudden drop in allotted bandwidth, and didn't even know what was happening when it first occurred. I am relatively new to the platform, but seeing people talking about issues like abusing the reward pool, whale abuse, bots, the difficulty of getting people to see my posts, and this issue, etc, is hard to all take in. I just wanted to post information and get rewarded based on quality content as seen by the community. I really like this platform, but it seems to me that it needs to stay focused on creating an environment in which original quality content rises to the top and newbies aren't strangled or turned away by all these hurdles, esp with competition like EOS, sapien, and even Facebook.

·

I always suggest to newbies to read the FAQ first. Steem is a complicated platform and requires a learning period to understand and use it effectively. It's still a work in progress, so many things are lacking.

·
·

Yes, that's why I'm sticking around, because I definitely see the potential and can tolerate the inconveniences if there an overall strategy to make sure the problems eventually get ironed out.

Thanks for this awesome post, that made me nearly understand the bandwidth issue, that I have only heard of but fortunately never experienced myself. 😀

Proud to have you voted as a witness. Personally I think 30SP or maybe 50SP delegation to new accounts until they reach 50SP on their own should pretty much most of the problems without doing anything. I mean steemit controls majority of the SP and they are not voting with them either. I never faced bandwidth issues while I was new. It'd be great to know how many people are actually facing these issues.

STEEM is like F2P games. You get the account to try the product. If you also get the product for free, that mess up the economics to make us look like Fakebook. There is no free lunch. If people are using STEEM so much, then spending few bucks to power up or buy delegation shouldn't be a problem unless they are in a very crypto unfriendly jurisdiction.

Yeah. This bandwidth problem is quite problematic.

I first encountered the bandwidth problem when I was registering my account. Got an email saying my account was approved, clicked the link, filled a simple form, clicked the "submit" button, then there it was "Bandwidth limit exceeded" or sth like that. No support page, no email to reply, etc. When I googled it, it told me to go to steem chat or "just wait, and try again later" from a github page. I even registered to github because of this :') Well, luckily after an hour, I was accepted, then here I am.

After being on steemit for quite some time, I believe it's worthwhile to power up using own money when being hit with bandwidth limitation from time to time. However, that should not be the case for new comers. Imagine being advertised "Come to steemit, do things like you are on facebook, and get paid" and suddenly "bandwidth limitation" and people are telling you to pay (=power up) to use the platform, they could just go back to facebook. :D

·

Yeah, the trust issues with that last sentence would prove problematic. I have a lot of people I'm inviting to join me here on Steemit who really trust me, but I don't think even I could sell that line. LOL

Voting for you as a witness @drakos! Thanks for your service. Referred to you by @jrswab's post.

·

Thanks to both of you, I appreciate the support.

·
·

Very welcome @drakos!

Your allocated bandwidth formula is missing :
- delegated vesting shares

Your bandwidth decreases, when delegating SP

·

Thanks for spotting the error. Fixed.

Thanks for this info. I hope the issue of bandwidth be resolved soon because it been a concern to me. I will read your post on voting for witness as well.

Last time I experienced a limitation was in July 2017. Since then I've never seen that red stressful text

Brutally good explanation with possible solutions short and long term.

Thank you.

I hope this bandwidth issue can be settled once and for all, it can be frustrating sometimes. I really don't understand much of what you said but I do hope the witnesses take a look so I will be resteeming this post.

Wow! This is detailed. I didn't understand some of them but i do pick up few things and yes, the spamming need to reduce. I experienced the bandwidth limit also today and it was annoying. I noticed it kicks in for me when i don't have any pending payout in view for couple of hours. Someone also told me about voting for more witness, didn't know how true that is. Power up still remain a realistic option for now.

This is a real problem. Just a few weeks ago when I was totally new and barely had any SP, almost every day I would hit negative bandwidth at peak times, even when I had not commented, upvoted, or done anything for nearly 24 hours.

The learning curve here is insane and new users who show up and face the bandwidth limit are unlikely to even understand what that means and will quickly become discouraged.

This frustrates me as a newbie as im excited to be here and contributing and exploring new content but I often cant get on and submit my new blogs (I guess I can use that time to research and create even better content tho!)

Man, we've got too many bytes...

·
·
·

Yeah, and they're all evil bytes!

Thanks a lot for talk about this. Now a days It's a major problem. i appreciate your thinking.

very informative and well written article. Thank you. I recommend using the #feedback or #steem-feedback tags.

This bandwith issue has been a menace but having explained it I understand it is a necessary evil to checkmate some certain things like spamming and all. I quite agree with the solutions you proferred especially of the bots. They do indeed leave long unnecessary comments.

Is this bandwith issue the reasons why there are no signups lately on steemit? People have been complaining of their inability to sign up.

very interesting your post! I think that for users with low SP the only solution is to immediately ration and moderate what they do inside the page.

awesome is the word

Antrian panjang

I think they've already implemented one of your suggestions. When I joined almost 2 weeks ago I was delegated about 14 SP. Then a few days later I invested $100 of STEEM and powered it up to SP. My delegated SP disappeared within 24 hours! It sucks actually, because that one day my upvotes actually gave people more than a penny!

Interesante, las recomendaciones son muy buenas, de todas formas si hacemos power up igual salimos beneficiados.

Thank you @drakos. That i`ts a good information. Nowday thereis a lot new people in steemit and They have a lot debut.

Excellent post friend, very good. You have my vote. regards

thank you for this explanation, these are growing pains!

Por fin un análisis matemático completo. Gracias, he llegado un poco tarde, me hubiese gustado poder compartir este post, te lo debo para otra publicación.