The Heist: How Big Blue stole Blockchain!

in #blockchain3 years ago (edited)

I should have written this article a long time ago. I am an "IBM-certified Hyperledger Fabric Developer" since February 2018, following a two-days course on Hyperledger Fabric 1.0 technology. The course sparked in my mind a revelation.

A revelation so big, I only dared allude to it obliquely in my post "Blockchain revolution: the CIOs' dilemma" but otherwise shared it just privately until now. I begin to come to grips with it. My affinity with systems theory should have prepared me better, and yet ...

It turns out, blockchain will not be welcomed in the world as the savior, to usher in an era of trust and prosperity. You see, the promised journey toward happiness and abundance, aboard the "trust machine" called blockchain might not be hallowed by everyone and will not be all smooth sailing!

Quite the contrary, the existing IT ecosystem is intent on stopping the challenge coming from the revolutionary nature of the blockchain "trust machine".

Goddess of Chaos and Discord Eris will prevent Sinbad from reaching the Book of Peace in Dreamworks' "Sinbad" animation movie. (c) Dreamworks 2003

To get back the "Book of Peace", we'll need to fight "Eris and her minions", among which ... brilliant Big Blue!


Hyperledger Fabric 1.x is a cunning ploy by the incumbents to fend off the challenge posed by blockchain systems to their business models

Yes, I know that is provocative, but read further and make up your own mind.

Rewind to June 2017

Back then, I got my hands on a thick documentation for IBM's Hyperledger Fabric 0.6 architecture. I read it avidly, it looked like a beautifully designed blockchain platform, if somewhat complex. That good impression about Fabric, I have found in many people since, including @timcliff. I'm almost certain it was based on the 0.6 architecture.

However there were already a few points about Fabric back then, which were disturbing.

Blockchain was never, to my eyes, about simply "a better" (in certain cases) IT architecture. It was about a "revolutionary" IT system, a system that models and incorporates elements of human psychology to become a "social extension".

If we agree that classical "social media" software (FB and such) has altered our behavior and our societies, often in "not very good" directions, blockchain holds the promise of enabling or effecting positive, potentially compensating changes.

For that to be true, I sensed some elements were essential:
- sharing and consensus
- trust-breeding transparency
- incentives

While to an IT architect the structure of Fabric 0.6 was beautiful and there was consensus (PBFT) and "sharing by default", some pieces were missing from the picture:

  • the possibility to build "channels" with differentiated access introduced the possibility to hide, conceal, obscure, much like what we see in the current closed IT systems.

If company B deals with a company A and agrees to lend it money, it does so based on assessing the level of financial resilience of company A. For that, it needs to have a good view of company A's books, its ledger. The latter is made more trustworthy by blockchaining ... but at the same time the trust is undermined by knowing that company A might have opened a separate, hidden "channel" in its Fabric system, where it might have "swept under the rug" part or all the liabilities and "dirty laundry". A channel destined to deceive company B into getting a rosier picture of company A's situation.

Has it ever happened before? You bet it did, of course in the years leading up to the 2007-2008 financial crisis! Big banks were busy building "Special Purpose Vehicles" (SPV) which were used to dump toxic liabilities, delinquent mortgages, off their balance sheet.

A Hyperledger Fabric-based blockchain system would have lent a very helpful hand here indeed because of the additional (fraudulent) impression of "trustworthiness" that the main channel (main ledger) would get from claiming to be using blockchain. While on a second Fabric "channel", whose existence the auditors might not even be aware of, the bank would conceal its transactions with a SPV where nonperforming loans could be parked.

Plouvier, Joost, PhD Thesis: "Financial innovation and the financial crisis of 2007 and 2008:
A Coincidence?"

  • no built-in incentives

IBM was well aware of the rather shady reputation bitcoin, the most successful implementation of the blockchain pattern, had following the Silk Road bust and Mt. Gox scandal. It wanted to clearly separate Hyperledger Fabric from anything similar, thus it decided to build it without an internal token. At first I thought "why not?" but as my understanding progressed, I realised the token, as means to align the incentives of the various actors, was an essential part of a successful blockchain. "A blockchain without a token is just a database" I read somewhere around that time and it resonated stronger and stronger as time passed by.

February 2018

It is nonetheless full of enthusiasm that I sat the first day of the Hyperledger Fabric 1.0 official training given by IBM's Matthew Golby-Kirk.

"End of the world" scene from Sinbad, a 2003 animation by Dreamworks

What I've learnt in that course, very well and clearly explained, was sobering. I'll summarize the important points here and then discuss them:

  1. Fabric 1.x has abandoned the PBFT consensus of 0.6 because ... it doesn't scale over 20 nodes! PBFT has not been replaced with another BFT consensus. Without a BFT consensus mechanism, I don't think Fabric 1.x can be called a "blockchain" anymore, but became an impostor.
  2. From a business prospective, Fabric 1.x has been modified with the explicit goal of allowing companies to ... do business as before ... In other words, "Everything must change so that everything can stay the same”
  3. Fabric 1.x is not as much a blockchain platform as something a lot more "flexible" (read "some assembly required"), i.e. a toolkit, a bundle or development framework where one can (try to) develop "blockchain-like" applications (implicitly taking the risk of introducing bugs).
  4. Fabric results in building complex thus brittle information systems which are expensive to design and implement and expensive to run, as coordination issues increase the costs at all levels

In the following, I'll try to go through these points briefly and might come back with a more in-depth analysis in a later paper. Images come from IBM training material and are copyrighted - credit goes to IBM

No Byzantine Fault Tolerant consensus mechanism

Consensus mechanisms in distributed computing have different levels of resilience:

  • crash fault tolerance (CFT): the system achieves consensus even in the presence of crashed nodes
  • byzantine fault tolerance (BFT): the distributed system achieves consensus even in the presence of up to 1/3 of nodes behaving erratically or being malicious
  • "Nakamoto consensus": the distributed system ultimately achieves consensus over a variable number of pseudonymous nodes that need not be trusted

The "Nakamoto consensus" is of course more resilient but imposes a trade-off in terms of speed. When designing Fabric, IBM has rightly decided that, for a blockchain it aimed at corporations, speed and throughput were more important than resilience to malicious behavior. In a corporate setting, nodes are not pseudonymous but on the contrary identified and would thus need to behave responsibly. In exchange for this assumption, Fabric 0.6 went for the weaker BFT consensus which made possible an increased transaction speed.

However while progressing toward 1.0, they realized PBFT (their chosen BFT flavor) did not scale above 20 nodes. Therefore they went even further and made a design decision that, contrary to the first one, begs askance: they lowered the bar down to "crash fault tolerant" (CFT) consensus. That means consensus can be achieved if some nodes crash. But there is no guarantee of consensus if some nodes start behaving erratically (due for instance to hardware malfunction) or worse, if they are compromised by a malicious attacker

(c) 2017 IBM corporation, the full presentation is available on-line here

Business as usual

This is for me a serious betrayal of the blockchain promise. Bitcoin's creation was strongly in reaction to the way in which "business as usual" had led to predatory and abusive behaviors in the financial world and had let down the majority of people. I believe the IBM engineers who set out to design and implement Fabric were a little bit at least influenced by bitcoin's ideals.

Yet the world is sadly run by salesmen, not by engineers. When the IBM salesmen went to peddle Fabric 0.6 to businesses, this is what they heard (I quote IBM's Matthew Golby-Kirk from memory):

They told us: " - If we implement your software we would need to completely change the way we are doing business. That is too risky and it's not what we want." Therefore IBM heard them and we modified 1.0 to allow businesses to continue working as before.

(c) 2017 IBM corporation, the full presentation is available on-line here. Note the "evaluate the incentives ... to work out a viable business model". As a supplier, IBM has its business model all worked out. Only the customers need to worry

Fabric is a toolkit rather than a blockchain platform

When you buy something that is sold to you as a "blockchain system", what are your expectations? I certainly do not look explicitly for "a chain of blocks" - that is just a means to an end. But what is that end then?

Arguably, the most important feature defining a blockchain system is preventing double spending.

That is, given a data representation of some assets, no transaction in a blockchain system can consume assets which were already consumed by a previous transactions, or assets which aren't there in the first place.

You'd think that a blockchain platform does that right out of the box, correct? I know I would!

Well, Hyperledger Fabric does not ... you need to assign that task to a programmer (each time you start a new Fabric-based project). If you look for one, call IBM, they can help you for the right price. The programmer needs to write specific code in order to prevent double spending.. Of course, then you need to test the code to avoid bugs (if you need testers, do not hesitate to call IBM, they can definitely help you with testing too, for the right price).

(c) 2017 IBM corporation, the full presentation is available on-line here

To understand what I mean, take the "car auction scenario" which you get to implement during the "hands-on" training session. In it "Charlie" has a car he wants to sell and "Alice" and "Bob" have money that they could use to bid for Charlie's car.

The "Fabric blockchain-based application" allows you to play the role of Charlie first and enter a reserve price. If both Alice and Bob bids are below that price, the car is not sold to either of them.

Then you can play either Alice or Bob and enter various bids. The "chaincode" (Fabric lingo for smart contracts) automatically changes the ownership of the car to the highest bidder.

Did that and noticed something quite important: not having a native, "blockchained" token meant that representations of value are always second-layer with no "anchoring" available. They are thus brittle, as prone to bugs as Ethereum Solidity data structures in the time of The DAO.

In our "car auction" scenario I looked at Alice's wallet and noticed she had $3000. I had Bob bid $4000 for Charlie's car and then went on and had Alice bid ... $5000. Sure enough, Alice won the auction and left Bob empty handed! But ... that is not right! Alice did not have $5000 in her wallet!

Hyperledger Fabric, out of the box, lets actors spend "money" they don't have! When I asked, Matthew told me that is the normal behavior and that you need to forbid negative balances specifically in the business rules.

While that is probably fair game for a generic development framework, I feel that a software bundle which claims to be a "blockchain platform" should have done more - either to explicitly lower expectations or to raise the platform up to those the name "blockchain" invites people to assume.

Complex systems fatten the sellers' balance sheet (not the buyers')

Complex systems can handle more complex realities, richer models of the real world, whose complexity is increasing continuously. I am a believer in the necessity of complexity. The ability to master complexity is our best weapon against entropy. I am not going to criticize Fabric because it's complex, at least not while writing on Steem, itself a very complex blockchain system.

The drawback of complex, beautiful, architectures is that they are costly to build, maintain and operate. Blockchains such as steem address this issue through the use of aligned incentives: the "total token balance" is unique and the value of the token motivates all token holders to coordinate as efficiently as possible in order to advance the common platform: it's in their best self-interest.

In a sense, in a blockchain with a token, the relationship is transformed - it is not about "supplier" and "customer" any longer, as it is about "advancing the community", like in a sort of modern cooperative.

Fabric doesn't have a token, hence no intrinsic means to align the incentives of participants. On the contrary, Fabric is a platform designed as a classic software product to be transacted between a customer and a supplier with opposite financial interests.

If you invest in Steem Power for instance, every development anyone pays for, to develop an application or to improve the underlying platform is indirectly beneficial to everybody in the whole ecosystem, to all the stakeholders, including you. System "quasi-uniqueness" (bar forks) means there is a brake ensuring there won't be more complexity added than is reasonable, because everybody collaborates to balance risk and benefit.

With Fabric, every cent that goes out of your pocket and into the supplier's (whether IBM or another "integrator") is a cent fattening the supplier balance sheet with no downside for him. Therefore ever-increasing complexity that make the whole system brittle and expensive to maintain and operate puts more money in IBM's account at no risk for them - all the risk is for you. If your system crashes or suffers a catastrophic hack you stand to lose a lot ... whereas IBM (or another integrator) will be called to the rescue and stands to gain a lot (in consulting fees)!

Not only incentives in the Hyperledger Fabric system are not aligned, they are completely opposed!


How did we get here?

If by now you start understanding that Hyperledger Fabric is a wolf in sheep's clothing, you might wonder how we got here?

Certainly the companies which are expected to buy a blockchain system would be able to do this analysis and a product such as Hyperledger Fabric would not have advanced to where it is today, powering hundreds of blockchain "PoCs" and pilots all over the world!?

Our understanding of how organizations think and act doesn’t always make this distinction between the interests of the enterprise and its managers. For example, The Wall Street Journal might typically report something like the following: “Exxon Corporation has announced that it is moving forward with development of oil fields in Kazakhstan in order to meet the world’s demand for oil.” Fair enough: Exxon does want to find and sell more oil. But the statement also seems to imply that Exxon is a “person” that makes such decisions purely rationally in its interests. News flash: There is no Exxon-level decision maker, there are only the people running Exxon and making the decisions based on their professional interests. Corporations don’t make decisions, people make decisions.
Paul V. Weinstein in HBR, Jan-2015

As I described in my previous post on this topic, Blockchain revolution: the CIOs' dilemma, it started with the blockchain craze of late last year and the surprising effect on share prices the word "blockchain" was having. Remember "Long Island Iced Tea Company" renaming to "Long Island Blockchain Company" and making an instant fortune to its shareholders?

Well, turns out listed companies in the US hold regular press conferences where the shareholders get to ask questions to the CEO. Imagine how many of these press conferences might have gone after that event: "... And we, what are we doing with blockchain?" To which the CEO, barely aware that there is a ... thing ... called "blockchain", would turn to his CIO and repeat the question: "And we, what are we doing with blockchain?"

To which the CIO would get cold sweat and start frantically flipping through his Rolodex ... let's call ... Satoshi Nakamoto! Oh, shucks! Nobody knows who he is, let alone his phone number ... Then let's call ... Vitalik Buterin ! Oh, double shucks! His phone number is not in the Rolodex ... But guess whose number IS in all the Rolodex-es of all the CIOs? Why, IBM's, of course!

"Hello, my friendly IBM account manager, can you help me out here, I need to show my boss that we are doing something with blockchain!" After all, as corporate lore has it, "Nobody ever got fired for buying IBM"!


This is how Big Blue gladly accepted the role of seller of "blockchain indulgences". In exchange for a nice sum of money, they would come in, wielding Hyperledger Fabric as the Holy Cross, and would certify that the CIO is doing something with blockchain! ... while thanks to the way Fabric 1.x has been designed, everything in the IT organisation and in the rest of the company could keep working exactly as before.

So the "happy ending" of the story?

  • The PoCs and pilot projects based on Hyperledger Fabric implementations are all successful (everybody needs to look good to their bosses). Why wouldn't they? As long as they don't go into production to be confronted with reality, "being successful" is only about perception.
  • They are of course a bit expensive (budget is stretched to the limit that the buyer can pay, why would IBM sell its blockchain indulgences for less); as they are perceived as "one off" (pilot projects), it's not a big risk
  • They align the incentives of the involved actors, except for shareholders. The CEO can justify spending more than usual because "We are actually doing a great blockchain project with IBM!". The CIO can beam with pride at his side (he's the one running the show). IBM gets to pocket the money. The way the business works continues as before, nothing needs to really change...

If you know what witnesses are and agree that people commited to keeping this blockchain ticking play an important role ...

(by simply clicking on the picture - thanks to SteemConnect)

Other posts you might enjoy

You might also want to check out my “lighthearted” account, @sorin.lite, perhaps you’ll like what you’ll see.


Highly rEsteemed!

Blockchain was never, to my eyes, about simply "a better" (in certain cases) IT architecture. It was about a "revolutionary" IT system, a system that models and incorporates elements of human psychology to become a "social extension".

Gonna have to read a few times...

slow clap bacon gif test 3  both brows.gif

thank you!

Long read but good one. I develop software for +25yr now, on the level you are writing, blockchain is just what we call a buzzword. Soon there will be the official IBM or other logo stamp and everybody needs to go through some procedures, pay a lot of money, get the stamp, show it in the advertising.

Off course business goes on as usual but we know their ignorance will be fatal in the long term, they simply don't see what this is about. This will be not about another buzzword stamp, don't let this bother you

I write accounting software. If i use the word 'immutable database' and i connect that "old" technology with blockchain, i already get CEO's on their knees. Believe me, they just want that stamp 'blockchain' and if they get the smallest grip on the meaning of it, they are in heaven.

Can't help thinking of that bible verse saying something like "forgive... as they do not know"

You nailed it in your post, the most powerful essential ingredients are simply missing, i am not afraid of what they are doing, time will proof their systems are not reliable...

Posted using Partiko Android

Blockchain is really complex and complexity is costly.

What Bitcoin and other true blockchain systems managed to do is to hit a "sweet spot" where the incentives are powerful enough to mobilize the necessary collaboration level needed to maintain such a complex system.

My prediction is that, absent incentives, Fabric-based systems will prove so costly to run that at some point companies will realize that they are not worth maintaining, as to do the business they are doing, the way they are doing it, a much-cheaper-to-run classical system does the job.

After having wasted (for the shareholders) a lot of money (but made a bunch of money for IBM), they will sound the retreat.

I'm only halfway through and I am mind blown, I was under the impression that IBMs Hyperledger was "a breakthrough" for businesses to find there way into the Blockchain era. But as your heading above states... "business as ususal." What a shame.

Did you see the patent awarded to Alibaba I think for a blokchain with an admin? I heard it was not the first time such a proposal was made. This is such a contradiction but you can not blame. For the incumbents, blockchain has a problem: control.

So looking at what people are proposing in private blockchains space, you only get consortiums trying to get more and more members, business model where they are the trusted third party like before.

That is not necessarily a problem - the goal of a project is to solve a real-world problem, not to be "ideologically pure". Steem is itself a consortium of the top witnesses who are the trusted third party. Ethereum is even worse - Vitalik concentrates more power over his "baby blockchain" than Xi Jinping over China, or Putin over Russia.

"Decentralization" is not my focus. The litmus test, for me is: do you improve the lot of everyone by making possible things which were not economic before? Do you create value?

Wow this looks like quite an epic chapter in the Blockchain revolution, I will have to come back and read more soon...


whoever think of this grand scheme must put a lot of though on it.

Hi @sorin.cristescu!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your UA account score is currently 4.913 which ranks you at #1129 across all Steem accounts.
Your rank has dropped 14 places in the last three days (old rank 1115).

In our last Algorithmic Curation Round, consisting of 388 contributions, your post is ranked at #74.

Evaluation of your UA score:
  • Some people are already following you, keep going!
  • The readers appreciate your great work!
  • You have already shown user engagement, try to improve it further.

Feel free to join our @steem-ua Discord server

Wow nice post, thanks for sharing.

Coins mentioned in post:

CoinPrice (USD)📈 24h📈 7d