Gridcoin superblock, we forgive you
Greetings, I'm denravonska (or ravon), one of the volunteer Gridcoin developers. In light of the recent superblock issues where no superblock was created for a few days, I thought it could be a good idea to explain what has happened; why it happened and how we could prevent it from happening in the future. Note that the Neural Network isn't my area of expertise so I might be getting some of the details wrong. However, the big picture still applies.
I am not going to go into details, so it is expected that you know the basics of Gridcoin such as staking, beacons and the Neural Net.
Why do Superblocks Matter?
Superblocks contain a report of every known CPID's magnitude so the nodes know how much each CPID is expected to be rewarded. Sometimes things happen, superblocks are delayed and people start getting anxious.
![Don't_Panic.svg.png](https://steemitimages.com/640x0/https://steemitimages.com/DQmPe4GNGh5KPAcMbKf7DNYhS1j9XtTS7AY2yenciDPpTLV/Don't_Panic.svg.png)
It is not super critical if a superblock is delayed a bit. What will happen is that new CPIDs won't get picked up while current mags are "locked" in place.
Giving Birth to a Superblock
- Collect known beacons
- Download statistics from BOINC servers
- Discard beacons with no recent credits
- Calculate each remaining CPID's magnitude into a contract
- Broadcast contract hash and try to reach a consensus
Roughly speaking, if you're one of the nodes currently eligble for super block participation, and you hold the agreed hash, and you stake a block you can create a superblock. If the nodes have problem reaching a consensus it will delay the superblock. For example, this is the current status:
$ gridcoinresearchd execute currentneuralreport
[
{
"Command" : "currentneuralreport"
},
[
{
"Neural Hash" : "Popularity,Percent %",
"2668d3b02c892081a05983ac5e4092c8" : "2; 1.53%",
"2d4e4c50ba6ff5207ac0aa116a14d103" : "3; 1.85%",
"48e25e844613915eac7a544530d8e7e5" : "31; 20.20%",
"74007022ad3665961bcd7074d8b26aff" : "13; 8.76%",
"7e15dab8447b3b3d632c01298095c2ab" : "33; 21.89%",
"a52e735e1e1b5f8a4f26f1ef4cab9ab6" : "29; 18.76%",
"c9a585cb4115ca69802e4d6354397e85" : "36; 23.88%",
"e240be17fcde4b071f38e70dc49a094c" : "5; 3.13%",
"Superblock Age" : 35314
}
]
]
The nodes are having a heated debate on who's right, forming small gangs of hatred. This is a bit simplified and it's a little more to it but that doesn't matter for now.
Causes
Figuring out the root cause has been a joint effort by the entire community as usual. Lots of feedback, ideas, testing, brainstorming and debugging. The long delay between the two superblocks are believed to be a combination of small flaws in 1, 2 and 3.
1. Collect known beacons - Beacons are continously added and removed to the chain. When loading the wallet, all beacons older than 6 months are skipped but any subsequent beacon received are kept in memory. This means that wallets which have been running for a long time will know about more beacons than wallets just loaded.
2. Download statistics from BOINC servers - Since Gridcoin has seen an uptick in the user base we have also started to put a lot more strain on the BOINC servers. Some of these servers are run on very modest connections such as a home server. As a result some of the download request timed out for some of the nodes, causing them to accept different CPIDs and calculate different mags.
3. Discard beacons with no recent credits - After downloading the BOINC stats the nodes start weeding out CPIDs which are no longer active. They do this by checking the expavg_time
XML field for each CPID. Any CPID with a credit time older than 32 days are discarded. This age check is done using the node's local time which means that different nodes will filter different CPIDs depending on their current timezone. While not massive, there can be up to a 25 hour difference in beacon acceptance, furthering diverging the final CPID list. Credits to bullshark.
These are not new problems but the more people joining the more likely it is to get different final hashes.
Upcoming Fixes
Within a few days Soon :) a leisure will be released aimed at adressing these three known issues.
Nodes will agree on a fixed UTC time window or block height on which to base their filtering. This eliminates beacon collection size differences.
A BOINC statistics proxy server will be setup to mitigate some of the server pressure when needed. Additionally, statistics files will be checked for modifications before download (also bullshark's idea).
Statistics CPID filtering will be done on UTC time or by checking the
expavg_credit
field. See PR #399 and PR #396.
The new server temporarily increases centralization. This is something we just have to live with to get things rolling reliably again in the short term. The long term solution is to distribute the data within the network itself, minimizing the strain on the BOINC servers.
Great post.
Although there are a few community members who disagree that the influx of new users is happening, I can personally vouch for the fact that the number of new users has grown considerably recently.
(Edit ) The administrator of grcpool.com has just mentioned that today ( 29/06/17) 100+ new users joined the pool.
I know this because I have helped many to set up their wallets on #gridcoin-help and I have even set up a new skype name (Mercosity) in order to help a few that were having, what seemed to them, insurmountable issues.
A large number of those joining have started with grcpool.com as they don't have the computational power to go solo. Some of those new pool users have, in a short time span, increased their power and have started to go solo.
We have a friendly and amazingly dedicated community and many unsung heroes who help out, not only on the development side but in #gridcoin-help. They and I know who they are.
I thank you all on behalf of all the new users and the community as a whole, for without you Team Gridcoin would not exist and new users would be frightened away.
Need help? We are here to help you on :
http://webchat.freenode.net/?channels=gridcoin-help&uio=d4
Thanks for doing what you do!
That explains a few things, I only sort of knew what people were talking about but wasn't grasping it. Thank you.
I would again like to ask the same question posed in https://steemit.com/gridcoin/@dutch/analysis-of-the-late-gridcoin-superblock-and-suggested-solutions#@frank0051/re-dutch-analysis-of-the-late-gridcoin-superblock-and-suggested-solutions-20170626t214427390z
We are again seeing the suggestion that scalability is the challenge - particularly in item 2 with the uptick in user base. Again, any idea as to the scale of new users we're talking about here? 10s, 100s, 1000s?
According to boincstats we grow at an average of 9.4 users per day. It's not a huge deal but it should start to takes its toll on the servers.
9.4 is probably an average over a long period of time, maybe even since the team creation. I usually follow the influx of new members here:
https://boincstats.com/en/stats/-1/team/detail/118094994/memberMovement
It's around 20 users per day recently. Of course, those are just solo-miners, users joining the pools are not visible on that list.
Yes, and people running in investor mode won't be visible either.:)
Ah Nate, so good to see you on Steemit!
Gridcoin team is getting too awesome here :)
I was trying to figure out how to create an account, then I realized that I already created one last year and just forgot to actually use it.:P Thanks for the welcome!
I've just started looking into Gridcoin so very useful to get a bit of the 'behind the scenes' intel. Thank you.
thank you for your hard work in resolving (and explaining) the issue :)
Had no idea of superblocks...Now I know...Great post 👍👍
Very detailed report/article and also think the analogies used helps non tech-savvy people understand the principle much clearer. Good Stuff Ravon :)
thanks - helpful in understanding what's going on
This post is pretty entertaining for dev stuff.
Congratulations @ravonn! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOP