Peer Review of Cardano's Ouroboros

in #cardamon6 years ago

I recently had an opportunity to review Cardano’s Ouroboros consensus algorithm as presented in a youtube presentation. The original paper can be found here. The marketing behind Cardano and Ouroboros is that it is the first “peer reviewed”, “provably secure” proof of stake consensus algorithm. Upon reading their paper it becomes clear to those familiar with BitShares 1.0, Graphene, Steem, and EOS that Ouroboros is a copy of Delegated Proof of Stake (DPoS) with a few counter-productive modifications. In fact their paper refers to the term “πDPoS” 17 times without mentioning or recognizing any of my prior work.

Before getting into the more technical review of Ouroboros, let’s look at the performance of the final product.

Block Interval

Block interval determines the latency until a transaction is included in the first block. This is the lower-bound on the responsiveness of decentralized applications built on the protocol. Applications like Steem and BitShares are not really viable unless there is low latency and high certainty of finality.

EOS: 0.5 seconds
Steem/BitShares: 3 seconds
Ouroboros: 20 seconds

Irreversibility

This is how long someone must wait to be certain that a transaction will not be undone by a new/longer fork being released that excludes the transaction. Irreversibility is very important for any multi-step transactions. You don’t want to ship goods until payment is confirmed. You cannot make one trade until the prior trade is locked in. A decentralized exchange is not viable on a platform that has significant latency until irreversibility.

EOS: <= 2 seconds
Steem/BitShares: <= 45 seconds
Ouroboros: > 5 hours

Ouroboros is Unfit for Decentralized Applications

If we assume Ouroboros is actually “more provably secure” by some definition of secure, it is of little practical value because as specified the security completely compromises the practically. It would be like claiming a bullet proof vest is “provably safe” but it weighs 400 pounds. At some point other factors of system design take priority.

Unfortunately we cannot simply assume it has been proven secure. I will demonstrate that despite claims to the contrary, Ouroboros is far less secure due to faulty assumptions in its design. In other words, Ouroboros is a 400 pound bullet proof vest that doesn’t actually stop the real bullets.

Components of Delegated Proof of Stake (aka Ouroboros)

The Delegated Proof of Stake algorithm is divided into two parts:

  • Selecting a set of block producers
  • Scheduling the producers into time slots

The process of selecting block producers is typically derived from proof of stake which may be delegated. The set of block producers is periodically updated. In BitShares this happens every maintenance interval (1 hour), in Steem and EOS it happens every round (N blocks where N is defined as the number of producers in the set). In Ouroboros it happens every 5 days according to Charles Hoskinson in an interview where he describes the protocol.

Once the set of producers are selected they are assigned to time slots where a producer can either produce a block or not. The chain with the most consensus will have the fewest missed blocks and therefore be the longest chain. In ouroboros they call this the “density” of the chain, but the concept is the same.

Selecting Block Producers

Under Ouroboros anyone who wants to be a producer can be selected for a slot proportional to the amount of stake they own or is delegated to them. This is similar to how Steem selects 1 producer in every set (aka every minute). Whereas Steem only schedules 1 out of every 21 time slots in this manner, Ouroboros schedules all time slots with this mathematical distribution. In the case of Ouroboros only those with at least 1% of stake delegated to them are eligible to be in the set of producers, but Steem places no lower limit.

Scheduling Block Producers into Slots

The primary difference between Steem and Ouroboros’ system is that Steem uses deterministic scheduling with pseudorandom shuffling and Ouroboros uses sampling from a source of provable randomness created by a committee of randomly selected stakeholders. Oroboros places a very strong focus on the need for a trustless source of randomness in order to ensure the producer schedule isn’t manipulated by block producers manipulating the block contents so as to control the schedule.

Security Concerns addressed by Randomness

The heart of blockchain security is knowing that during every confirmation window that a diverse set of unlikely-to-collude entities produce blocks. Time to irreversibility depends upon how long it takes to get input from a 2/3+ majority of potential producers.

If the order of producer scheduling can be controlled by the producers there exists a circular dependency. This is the origin of the name Ouroboros (a snake eating its own tail). The focus of the Ouroboros design is to ensure that 2/3+ honest producers can be scheduled without interference. This is why they lock in the set of producers for days and schedule them with a set of provable randomness that no party can manipulate.

Steem / BitShares / EOS

Existing DPOS chains select a set of unlikely to collude entities by approval voting and then schedule them in a pseudorandom order. This shuffling is not really needed because once each of them participates a single block a 2/3+ consensus can be determined. This is why EOS will be removing the random shuffle all together.

With Ouroboros the length of time until 2/3+ of the stake is “randomly selected” is not known. It is entirely possible that in some windows all block producer slots will be randomly assigned to the same producer. While this is statistically unlikely, it is not unreasonable to presume that a long sequence of blocks could be assigned to collusive peers.

You can think of the process of confirmation on Ouroboros to be like an installation progress bar that jumps by random increments. Sometimes it moves forward quickly, other times it taunts you by not making any real progress despite new blocks being produced.

This gives Ouroboros unpredictable latency like Bitcoin. In the best case 2/3+ of the stake might get scheduled in the a half dozen blocks, but in the average case it will take much longer, particularly if there are many producers with 1% weight or a producer with 50% weight is scheduled many times in a row by random chance.

Distribution Security Issues

I have previously made the case that BitShares, Steem, and EOS are the most decentralized because it has the most unique confirmations per confirmation window. In a 6 block confirmation window for Bitcoin only 5 unique individuals confirm a block on average (usually at least one mining pool goes twice in a random 6 block window). In Steem 14 people confirm a block each round.

Because stake and votes are distributed by pareto principle, we know that Ouroboros will assign block producers like mining assigns mining pools. It is exceedingly unlikely that 100 producers will each have 1% of the delegated stake. It is far more likely that there will be fewer than 20 individuals with more than 1% stake required to be approved. Furthermore, there are voting paradoxes in play where voting for someone who doesn’t already have 1% or more of the stake is a wasted vote.

In past articles on proof of stake I have also shown that even if Ouroboros removed the 1% requirement to participate, it would be economically unviable to cover the cost of operating a node with income from less than 1% of the block rewards. I have also argued in the past that because stake is distributed by pareto principle, and voter selection of candidates is also selected by pareto principle, the resulting distribution of stake among producers is pareto2. In other words, stake-weighted voting creates a very high centralization that can only be countered with approval voting followed by giving the top N equal weight (like BitShares, Steem, and EOS do).

Even if the stakeholders divide their stakes and votes to attempt to even out the producer weights, the system is still ultimately under pareto control 1% of the individuals control 51% of the stake. Corruption takes place at the individual level, not the stake level. Furthermore, it is wrong to assume that large stake holders will behave like a group of smaller stakeholders of similar size. Regardless of their stake, they only have one opinion and one interest and therefore the information (good or bad) they contribute to the system is independent of the size of their stake.

Conclusion

Cardano’s Ouroboros algorithm is not mathematically secure due to bad assumptions regarding the relationship between stake and individual-judgment being distributed by the pareto principle. Furthemore, their algorithm is not “new” but a less secure slower variation of the DPOS algorithm I originally introduced in April 2014. The authors of the paper failed to cite relevant prior art or to justify why their deviations from existing art are an improvement.

A blockchain consensus algorithm claiming to value peer review needs to consider who they consider their peers and all such reviews should be public. In the blockchain space, our peers are other blockchain technology companies. From this we can see that DPOS (and variations thereof) is one of the fastest growing consensus algorithms in terms of the number of unique projects choosing it.

I am going to go a step further and claim that much of the academic research and proofs performed by Cardano’s team only bolsters the support and justification of many core DPOS concepts, even if their approach is suboptimal compared to designs of EOS, BitShares, and Steem.

Sort:  

A much needed post! Thank you.

I would also be interested in a post describing the falling out with Hoskinson. Not for its gossip value (although feel free to give details and quotes with some name calling ^^) but for context.
Perhaps it can shed some light on the nature of these two new projects (EOS and Cardano) and how they differ on a more fundamental level.

Loading...

Enjoy the payout ;)

wish my comment would worth 1k (I would even take $1)

dollar on its way

wow a new record for me!

Really it is worth about $4,000 USD with the SBD around $8 and $500 of the comment going to SBD.

I'd argue that the founder appearing in a comment to explain things in such depth is worthy. If hillary clinton can get paid millions for a single speech, you're paying for the appearance just as much as the content, the difference being this is voluntary donation and public

DAT monetization...

Downvote it like I did.

I only up-voted this comment because I want to grab a slice of this big pizza.

That's not how it works. You could have only got a slice if you had voted before Ned. You just gave Ned a crumb of pizza and you got squat

I know. But somehow my up-goat was bigger on this by $0.02 than up-goating my own comments, as far as experience can tell. Given that, only if @ned will tell us in advance who he will be up-goating to next. Lol

Wtf? 1k for your comment? #mindblown

welcome to steemit.

I think that is unnecessary. Charles Hoskinson's behaviour speaks louder than any subjective analysis of the past. Investors have all the information they need to make solid decisions right now.

Right, EOS and cardano new two project.

Nothing personal @cob, but I'm downvoting your comment, $890 is way too excessive for a few lines of text.

competition betwern cardano and eos is a good thing. may the best platform win. in the end its the users that really win. im leaning towards eos ;)

please visit my post you ll find good content upvote me

Bitshares stands out among the second generation of crypto money with the possibilities offered by users, in fact it is a platform beyond being a crypto money. The most striking feature of this platform is the "crypto-entity". The crypto-entity is the name given to the crypto paral- lel that each user can produce at any time he wants. So anyone entering the system on this side can create their own crypto
@cob @ned

Charles brought in the money and then Dan kicked him out. I got that straight from Charles on Skype the week after it happened. But there’s more to the story that reflects even more poorly on Dan that I feel I shouldn’t blabber about. Notice @dan downvoted your comment, probably because he doesn’t ever want that information to come out.

Charles was basically driven out of ETH also. I think in both cases he wanted a more transparent and higher quality compliance with law. Neither Dan nor Vitalik wanted oversight of a board or requirements to produce certain quality of whitepapers and such. Charles likes formalism. Dan wants to operate by the seat of his pants with informalism that amounts to roughly about the academic level of a high school science project.

Yet I am not too overly impressed with IOHK thus far though. Lots of formalism, but very little in the way of significant foundational breakthroughs as far as I can find so far.

Note I used to discuss with both Dan (bytemaster) and Charles (charleshoskinson) on bitcointalk.org in 2013. I thought it was a strange partnership at the time, and I didn’t expect it to last long. They just seem so different. Charles is an academic. Dan is a very strange person. He is sort of mix of an engineer, snake oil salesman, and a progressive, leftist wolf wrapped in libertarian sheepskin. Very complex to try to understand his psyche.

Thanks for your post!
Please Follow, Upvote & Resteem my post to help us to travel & explore more
https://steemit.com/travel/@jonbee/travel-with-us-ep-01-kushtia-sugar-mills-kushtia-bangladesh-bd-steemian

Thank you, Dan. I've had a number of people ask me about this and though I've watched a video or two from Charles Hoskinson about Cardano, I felt a pause when I looked into some history.

Is it true you and him had a big falling out in the early days of BitShares? Is there somewhere people can read about this history?

Related to that, do you have somewhere you send people who are concerned about the BitShares "Great Consolidation" history? I know many investors who avoid BitShares, STEEM, and EOS because they are convinced investors will be screwed. I've been trying to find out more about this, but it's difficult to find the whole story. Currently, it seems to me, a bad decision was made to consolidate BitShares related projects (forgive me if this was during the protoshares/angelshares era, I don't know the timeline well) instead of letting them compete and run free which hurt a lot of investors. It could also be argued that was a necessary action to save BitShares during a time when all cryptocurrency markets were in massive decline. Do you have thoughts you could share or a history you could point to for people with these concerns?

Thanks again for doing so much for the cryptocurrency space to build awesome stuff that actually works.

@stan has a series of posts called "origins of bitshares." I haven't read most of them (yet), but I think what you're asking for might be covered there.

I've read this history but it only has a very small section on The Great Consolidation. To me, if that was a mistake, it's something to be highlighted and explained in detail. This is how we grow and learn. Instead of hiding our mistakes, we can highlight them and demonstrate how we learned from them and rebuilt trust through the process. If this event keeps investors away, why not explain it in detail as to what worked and what didn't and how investors and speculators were handled and how they profited or had losses. Showing how STEEM and EOS learned from that could also be very enlightening. Also, does it relate at all to the disagreement between @ned and @dan related to Smart Media Tokens? Could similar "mistakes" (if they were mistakes) be made with EOS which might harm investors? These are things I'd love to see openly discussed.

I meant posts on steem. There are way more posts here. Here's one of latest ones with list of earlier posts at the bottom. I don't know if it has any more info on The Great Consolidation though.

@sim31 thanks again for posting that. I especially like this BitSharesTalk post @stan links to there. There's a lot of good detail in there explaining the history involved.

Thanks. I've read through many of those, but will go through the latest ones as well.

Thank you, @lukestokes.

I would love to hear these things discussed in more depth as well.

It's frustrating as some people recognize how amazing DPOS technology is but think Dan (or people connected to him) took advantage of investors and so they don't want to invest now. If those people had a complete history, I wonder if they would choose to invest and by doing so, bring more resources to the DPOS community.

i love to here all this things

It would be rather ironic if dan opts not to share after his recent post about transparency.

That aisde, though. I agree with this philosophy wholeheartedly, Of course, it's not always easy to admit our mistakes, because many of us like to appear infallible. I have been guilty of this myself more times than I care to remember. But, if we can overcome this sense of competition that drives the current economy, then that wouldn't matter so much, and perhaps we could step into a world like you just described, with a new, fairer, better way of doing things.

I did learn. I learned to reserve inflation for growth and to not force the community to adopt my radical changes in thinking. I attempt to avoid repeating mistakes.

Thank you, Dan.

Would you consider doing a full, deep-dive post about this history? I would personally love to have a post I could send to potential investors with a detailed perspective on the mistakes made, the lessons learned, and the harsh realities which were unavoidable. I don't mean to sound accusatory, I just want more truth out there and some people are avoiding investment in DPOS technology purely because of their subjective (and possibly incorrect) understanding of the past. I've had conversations with people who admit DPOS is by far the best blockchain technology in existence today, but they still don't want to see it succeed because of their personal experiences. If there was a post I could send them with a detailed history of what happened, why, and what the results/lessons learned were, that would be quite helpful.

Thanks again for the amazing things you've already done and are continuing to do. I truly believe these are historic times, and you are doing historically significant work.

It shows maturity to just avoid the drama that would be caused by filling people in on the details of what happened between two people of different perspectives. They parted. The end.

We're talking billions of dollars at stake here between EOS, Cardona, STEEM, ETH, and BitShares. To not understand the perspectives of the individuals with most of the power within those systems is to be an uniformed investor, trader, or speculator.

Those who can stomach the drama and distill it to facts which give hints about future actions are the ones who do well. Those who blindly invest along with the crowd often get rekt in the long-run.

True maturity involves having difficult conversations about difficult topics without creating drama.

Really Luke, don't you think Dan's time is better spent on solving the issues of the present and future? If those investor friends want to cut their noses off to spite their faces, isn't that up to them. They have all the information they need to make reasonable decisions just like you, don't they?

They have all the information they need...

This is what I'm trying to obtain, because the information isn't (IMO) readily available. Some people were part of this from the beginning and experienced this all first-hand. What baffles me is I know people on both sides of this situation that were part of BitShares from the beginning who have opposite perspectives. Some trust Dan completely and see no wrong doing (other than some forgivable mistakes) while others are convinced he and those associated with him are money-grabbing scammers. Clearly there's some information asymmetry going on here. They both can't be right.

I want to bridge that gap by seeing more facts.

As to Dan's time, he's smarter than I am, so I'll leave it to him to determine how to best spend his time. I'm simply making a request as an investor and someone who talks to other investors. An hour or two to write down a history with a lessons learned from his perspective might bring in more support for the things he wants to build which could multiply his efforts.

Plenty here would love to see a public story from your point of view. No matter how well/poorly handled anything ended up being, it's obvious that it ended with a solid product being created. One that's still getting better, at that.

Are you talking about how changes are made to Steem?

great questions, Luke. I hope @dan can clear some of this up for us.

You might enjoy this post which @sim31 linked to in a comment above. That post links to this post which has a lot of interesting details. Also, see Ned's comment above. Interesting times. :)

I really don't understand the argument for investors getting screwed. Bitshares has performed admirably and is only getting stronger. Back in the earlier days, in the Bitshares forum, many of us suggested that success could only be judged through the lens of a little more time. That being said, Dan has acknowledged mistakes and attempted to learn from them. Isn't his word enough? Dan's tech is being peer reviewed in the public domain against user adoption and it's competition.....the most rigorous kind of peer review there is. I can't see what good can come from dissecting the past any further than that. Where is the value unless it is to stir up acrimony?

Exactly. Every investor is risking their own money on their own initiative out of SPECULATION for the future. When you hear an announcement that something is going to happen that you didn't predict and the price crashes, that's your own fault for investing speculatively and not expecting anything to change. You have to be willing to lose any money you put in and not be bitter about the project going a direction you weren't expecting. Do your due diligence people.

Thanks for commenting. I've had similar feelings that there may have been some differences in timescale (maybe speculator verses investor thinking?).

That said, where can I find a full post about mistakes and lessons learned? I don't necessarily think that would stir up acrimony (though it might), but I do think it might clarify historical reality and hopefully bring peace and harmony to those who are still upset (maybe unjustifiably so?) about the past. I've been told "just read bitsharestalk" when I ask for clarification on what people were so frustrated about. Maybe some specific links might help, but to me, a well thought out post would be best.

Use this wisely, Gresham's law, seems insightful here, (worth googling) , remember the market is highly irrational , especially now. It is consistent with this irrationality that marketing trumps reality and ADA is priced higher than EOS.

I'm digging the Steemit more and more, finding threads like this @lukestokes. I think I need 1 year vacation to spend 8 hours a day on steemit!

Oh what shocking news, after hundreds of people put their money into Cardano it turns out to be a "product" without a chance to survive against others :D One can´t mention it often enough: Due Your own research before you invest!

It's just like living through the dot-com era all over again! Hundreds and hundreds of startups with stock pricing going straight to the moon, only to crash down to earth again. Remember petsupplies.com? Have your pet supplied delivered straight to your door. Well, it did happen, only it's call Amazon.com.

During the dot com era people really had no clue what this internet thing was and how it would develop. Cryptos are also uncharted waters but in my opinion it's easier to predict the future for them because they are based on this infrastructure called the internet. The example you made with petsupplies and amazon.com is very interesting because it shows how this crypto space will likely develop in the future, my guess is that only protocol coins will survive and all the petsupplies.com of this world will be built on top of these blockchain.

Glad to see this response. I lost respect for Charles and the Cardano project after seeing him, multiple times, essentially shilling Cardano in the Bitshares Telegram group as if he's doing the community there a favor.

Then, he posts a link to a YouTube live "AMA" (interview with youtuber) and he claims 1) he can't comment on EOS because he doesn't follow it nor read up on it (I smell stinky bullshit)... and 2) he claims bitUSD as a "wonky failed experiment" or something like that. Ummm.... bitUSD (and its sibling bitassets / smartcoins) are about to take Bitshares to the moon.

(shout out to the mods there, who went on to ban him from that group :)

All this academic talk being tossed around by Charles is funny. Academia is well known for doing research, for the sake of research, and not commercializing or transferring that IP into the real world.

The fact that their market cap is ~4x EOS right now after the insane pump recently only justifies a much higher price point for EOS.

CH cannot bring himself to acknowledge the work of others. He touts standards for peer review, research and acknowledgment yet blatantly falls short of observing those standards himself.

The marketing behind Cardano and Ouroboros is that it is the first “peer reviewed”, “provably secure” proof of stake consensus algorithm. Upon reading their paper it becomes clear to those familiar with BitShares 1.0, Graphene, Steem, and EOS that Ouroboros is a copy of Delegated Proof of Stake (DPoS) with a few counter-productive modifications.

Yeah, that really stuck in my craw while reading the whitepaper, and I immediately called bullshit!

Thanks for writing this @dan.

in the real world, an influencer is paid by his/her ability to generate ad revenue or change opinions which are ultimately paid by the end ad buyer or influence buyer. steemit right now is self jerking its esteem power without any end buyers through steem tokens chased by crypto gamblers not much different tulip gamblers. crypto inventors lack understanding of economics.

@dan,

Holy sh*t!

Enjoyed your post immensely.

Salutations. I am JaiChai. Delighted to make your acquaintance.

RE: Your Post

Never even thought of "subliminal", camouflaged pareto principle-induced outcomes.

Wish I didn't miss the 18 Jan 2018 ICO deadline for EOS.

Anyway, may you and yours be well and loving life today.

Namaste (I recognize the divine in you),

JaiChai

A belated follow. My bad.

Cardano is completely overvalued. They haven't even implemented anything yet.

They are willing to work hard to implement something that is worse than dPoS. nice.

They've implemented plenty. Their native token is being used on their blockchain, with their native wallet. It has withstood a $20 billion bug bounty via the hostile open Internet so far. They have a long way to go, but make regular progress reports. They started from scratch, unlike many projects which were a fork from pre-existing code, so they have done a lot to get this far.

Cardano's market success was driven primarily by brilliant marketing strategy, I suspect. I have both coin/token though.

Existing DPOS chains select a set of unlikely to collude entities by approval voting and then schedule them in a pseudorandom order. This shuffling is not really needed because once each of them participates a single block a 2/3+ consensus can be determined. This is why EOS will be removing the random shuffle all together.

How will EOS determine the order of the block producers? If the order is not pseudorandom can this be exploited in some way?

As I am interested in the issue of determining the order of the block producers in EOS, today I read from their whitepaper( https://github.com/EOSIO/Documentation/blob/master/TechnicalWhitePaper.md )

"The top 20 by total approval are automatically chosen every round and the last producer is chosen proportional to their number of votes relative to other producers. The selected producers are shuffled using a pseudorandom number derived from the block time. This shuffling is done to ensure that all producers maintain balanced connectivity to all other producers."

According to my understanding of the quote, the producers will be shuffled. The whitepaper was updated 4 months ago, though. @dan (or anybody of good will), could you please clarify the issue?

See the "Removing block producer schedule shuffling" section in the most recent eos.io development update

Coin Marketplace

STEEM 0.29
TRX 0.11
JST 0.033
BTC 63458.69
ETH 3084.37
USDT 1.00
SBD 3.99