Bitcoin transaction queue explained - the bus analogy

We're currently having a "fee market" in bitcoin - one should pay a transaction fee according to the market price to be sure one gets the transaction included in the block chain. Usually the wallet software will do a good estimate and fix that for you, but not all software is up-to-date, and sometimes a bitcoin transaction may take 12 hours or more to process. This is an attempt on explaining why it's like that. Please use the comments if you can think of better analogies, or can spot flaws in this one.

© Tony Zuvela, Jantoo.com.

The bus company

Think of a busy road - and people getting from A to B. Maybe they drive their own cars, maybe they move by bicycles - I don't know, but not all the participants are happy with the situation.

One day a bus company is founded. The company are using luxury buses with twenty seats. There is no time table, the buses drives pretty much non-stop, by average they come every tenth minute - but sometimes it can take more than an hour between two buses, other times there are like five buses coming in a row. This happens randomly. (In real life, such buses actually tend to get clustered, but that's a digression).

The service is subsidized and delivered for free, but the subsidizes does not last forever (perhaps some investor is funding it) - there is the optimistic assumption that when people learn about the bus service and starts using it, there will eventually be sufficient revenue potential from selling tickets. At the bus stop there is a collection box, and people are frequently giving away some money for the ride. The bus driver can see how much everyone paid, can eventually deny people from entering the bus, and can decide how many passengers he want to have in the bus - as long as it's below 20. Many bus drivers wouldn't take more than five passengers. Anyway, in the beginning nobody knew about the buses and very few would use the offer, the buses would often be empty or just with a single passenger.

We've opened business - now we just need customers?

The bitcoin blockchain

We're currently using banking, credit cards and cash for monetary transactions. Not all are happy with the current situation (my eyes got opened up when Wikileaks got boycotted by the credit card duopoly).

One day bitcoin was invented, a new way to transfer and store value. Bitcoins transactions should be recorded in blocks. Those blocks currently have a hard coded 1 MB-limit and hence there is a limit for how many transactions can fit into the blocks. The block generation process (aka "mining") is random, by average the blocks comes every tenth minute, but sometimes it takes more than an hour between two blocks and sometimes several blocks can arrive within some few minutes.

The mining process provides security but costs electricity, the cost is subsidized (through inflation in the bitcoin supply), but the subsidies are tapered out and in the future there will be no revenue except the transaction fees. In the beginning, the transaction fee was optional - one could be pretty sure the transaction would be included. Most software added a static fee of 0.0001 BTC. Miners decides what transactions to include in the block and how many. In the beginning the blocks was seldom more than some few kilobytes big, and most miners had configured up a soft limit, some few hundred kilobytes.

The buses gains popularity

Come take the bus! We still have some free seats!

After some time they start attracting customers, they manage to fill up some of the buses, and they start collecting ticket income. That's all jolly good. Apparently the word-of-mouth is spreading, both the revenue for each trip and the number of trips is increasing. Sometimes during the peak hours several buses in a row will be full. Some bus drivers would still only accept 5 passengers, but most of the bus drivers now allows the bus to be filled up. It's a good thing, the market forces people that really needs to travel in the peak hours or with the next bus to pay up, but at the same time those that bought the economy ticket will be sure that they will get their bus ride if they only wait enough.

Bitcoin gains popularity

As the the default soft limits was much lower than 1 MB, the blocks actually got "full" several times before it got normal to touch the 1 MB limit. The default setting for the soft limit was raised multiple times. Throughout 2015 it was still generally possible to send bitcoins without a fee, but one would have to expect some delays. This is just fine, that's how it should work, one should expect delays if being such a miser that one don't want to spend transaction fees. Still, reliability is important.

The buses gets really filled up

Come queue up! 5 mBTC for position #4, 0.01 mBTC for position #300!

People aren't really buying tickets for the ride, they are buying tickets for getting a place in the queue. Say, you buy a priority ticket and enter the queue in front of several hundreds of other passengers and behind some tens of VIPs; you expect to wait for some ten minutes - except, this time you were unlucky and one hour passes before the next bus shows up. In the meanwhile the queue in front of you have grown with some hundreds of people. Not only that those hundreds of passengers will be served before you, but there are people coming in all the time buying more expensive priority tickets and getting before you in the line - so those ten minutes expected waiting time has now grown to 12 hours!

The manager of the bus company is happy that the average money collected from each passenger is increasing and that the ticket income starts building up - but probably he should consider investing into more buses or exchanging the current buses with bigger buses. After all there are tens of millions of people travelling on that road every day, while only some few thousands are carried by the buses - people don't like queuing, people who got a bad impression of the bus company is unlikely to return soon (maybe they tried the bus offer, got unhappy and went right to the car dealership and bought a new car instead), and there are limits for how much people are willing to pay for the ticket, the manager of the bus company would always hope people are willing to pay more and more and more for riding with the buses if just the capacity is held down ... but, no ... that's not how it works. If you want to be a significant player in the market, you need to grow the capacity, not only the prices.

The bitcoin capacity limit is reached

This started happening with bitcoin during 2016 - and many experienced stuck transactions, some few times even transactions timing out and never going through. This will becoming the norm unless we manage to get out meaningful capacity increases. In quite many situations the problems have been due to bad or old software, still not calculating the market fee but just using the old hard-coded fees - but still, there is only so much room on each bus, if everyone would be using good fee estimates we may end up with the situation above - that the 10 minute estimate for a "priority"-transaction suddenly balloons up to 12 hours.

One of the problems here is that you decide the fee - efficiently, you place a bid - based on how the queue looks when you enter the queue, but when the next block comes the situation can be completely different. By now there exists two "official" ways to try to speed up a stuck transaction, it's Replace by Fee (RBF) and Child Pays for Parent (CPFP) - but both approaches may be non-trivial, not all wallets supports it.

Future capacity increases

For the bus company, there are generally two things to consider - to increase the number of buses or to replace the buses with bigger buses - or both.

Is bigger blocks the solution?

All since 2010 there was discussions that the block size limit should be increased, and there is still discussions about it today. Some people regard the blocksize limit as a temporary thing that was thrown in and supposed to be removed - it was never supposed to be a significant thing, defining how the bitcoin network should work. Some people thinks that it may be dangerous to lift the limit. Some people thinks that it's a good thing the bitcoin protocol cannot be changed easily. And some people think that it's important to keep the capacity down in order to bring up the fees to a level where the miners get paid without the need of the subsidies. In any case, it doesn't seem like the block size will be increased anytime soon.

Then there is "segregated witness" - it was introduced partly to solve other problems and partly to increase the capacity. It's a bit like allowing passengers to put their luggage on the roof or on a trailer ... and then it's room for more passengers in the bus. This has been released in the latest version of Bitcoin Core, and it will be activated as soon as 95% of the miners flag that they support it. Due to disagreements and bad cooperation climate it may be that the 95% threshold will never be reached.

Increasing capacity through segregated witness

Another thing worth mentioning is the Schnorr-algorithm, which can compress big transactions quite a lot - where the definition of a "big" transaction being a transaction with many signatures. For ordinary transactions it won't matter much, but i.e. Localbitcoins is batching up outbound transactions and sending one big transaction instead of many small ones, such transactions has a lot to win on Schnorr.

Then there is the lightning network. Some thinks it's the silver bullet that will solve all capacity problems once and for all, others are concerned it will be impractical, that it will cause centralization, and/or that it won't be much useful for other purposes than micropayments. I can't find a good analogy to the bus situation - most of the traffic will be handled outside the bus network, but in the end we still need one person to ride with the bus carrying a big suitcase.

!()

Thoughts

In 2015 I was thinking ... "the fee market can't be such a bad thing". I still think it's healthy that one has to pay some fees to be reasonably sure the transaction will go through in one of the first blocks, but having seen transactions with a "normal" fee attached being stalled for more than 12 hours and being uncertain they would go through at all, I realize we do need a blockchain with idle capacity.

I think the bus analogy is a good one, and it clearly shows that at least one argument is silly: keeping the bus capacity low in order to maximize ticket income. In the short term, yes, throw in bigger buses and the ticket income may go down - but if there are millions of potential bus passengers out there ... then certainly one should increase the capacity in order to maximize the profits. Keep the ticket prices artificially high by keeping the capacity artificially low, and we'll end up that only those that really loves taking the bus or those that really don't have any other alternatives will use the bus. With sufficiently high fees Bitcoin will almost exclusively be profitable to use for criminals - just like the bus service will only be used by those who really don't have any alternatives (I think I've seen that - a nearly-empty bus running with only children, people with disabilites and very old people - everyone else was out driving their own cars).

Related posts

• The big Reddit Bitcoin schism - rantings on how bad the environment has become for Bitcoin-discussions on Reddit
• Bitcoin transactions - hitchhiking, statistics and CoinTape - my previous take on this subject