Arweave - Solving the Age-Old Data Problem
Data
A major benchmark in regards to human progress can be summarized as the storing and sharing of information. Looking back, a few high-water marks are likely to spring to mind. If you’re an ancient history buff, you’re probably going to point to cave paintings and the invention of papyrus as two seminal moments. The Gutenberg printing press will also likely get a nod as one of humanity’s first important forays into mass communication.
Sooner or later though, the conversation is going to turn to a more recent pivotal point in history (1989 to be precise). It was then that a British Scientist figured out a way for universities around the world to share data electronically, and the Internet was born. Within a few years, the way we thought about and interacted with information indelibly changed. Economics, academia, society, the basic fabric of how we humans get along, have all been transformed by this technology. We’ve come to rely upon the Internet to make almost everything work.
Which brings us to a big problem. Nothing about this huge repository of knowledge we rely on is remotely worthy of the trust we place in it. The Internet is durable as a network but every piece of data on it can disappear in an instant if the servers storing it are switched off or destroyed. The Internet is also fundamentally corruptible. The news is filled with stories of how data manipulation can alter the course of lives and entire nations.
Every piece of data we rely on to make important decisions, be they political, financial, societal or technological can be controlled or warped by a human with sufficient motivation to do so. While the Internet has made a globalized, interconnected world a reality, we face two urgent needs that today’s Internet architecture cannot provide: permanence and trust.
Then There Was Blockchain
In the first decade of this century, a new idea for sharing data emerged. Blockchain looked as though it might solve the data dilemma. Blockchain is a strange concept to explain. While its basic premise is simple, how it actually works can be infuriatingly abstract to grasp.
You can read a simplified but accurate description of blockchain here. Fortunately, to understand blockchain’s significance in resolving our confidence crisis with digital data, there’s only one piece of essential information to know. Blockchain is a system which guarantees data can be trusted.
It achieves this by sharing an identical copy of a database (known as a distributed ledger) to every participant (known as a node) in a network and then collectively establishing proof that the data is identical and therefore reliable. Essentially, the information is reliable because it permeates every part of the network which uses it. It can’t be redacted. It can’t be removed. It can’t be manipulated. The most obvious application for such a confidence-assuring mechanism is cryptocurrency (Bitcoin being the first and most touted example). However, its significance extends well beyond exchanging units of value.
Data stored on the blockchain occupies an odd halfway point between the relative permanence of paper records and the every-changing and easily modified nature of digital data stored and distributed online. The end result is that blockchain can (theoretically at least) store data which its users can consider completely trustworthy and much closer to permanent than any previous electronic records. In blockchain lingo, the data becomes immutable.
In the age of fake news and information warfare, the economic, social and political potential of blockchain began to dawn on developers. Between 2014 and 2018, a flurry of ideas emerged, all aimed at harnessing blockchain to solve problems of data permanence and trust.
But a critical flaw soon became apparent. The idea does not scale.
If every node must maintain a copy of the whole, the task of verifying the reliability of large bodies of complex data gains the fractal complexity of a butterfly’s wing. The complexity of handling large volumes of data rapidly expands beyond our technology’s capacity to keep up. Costs to store and access the data also increase its users in response to the increased resource-requirement to interact with such a complex system.
Blockchain data storage rapidly becomes bogged down by a slowly deteriorating orbit of ever-increasing strain on the system and cost to its users. While blockchain showed promise as a vital part of bringing permanence and trustworthiness to online data, it’s clearly not the solution.
It was a case of 'so close, yet so far'.
A Better Solution: Arweave
Arweave was designed to solve the problem of scale which standard blockchain can’t touch. It builds on current blockchain technology to create a new structure for handling data that is fast, cost-effective and scalable.
This new structure, known as ’Blockweave’, offers the first practical solution for sharing digital data that is both permanent and unmodifiable.
For a more complete understanding of Arweave’s solution, you can check out their overview here. The key to understanding its significance though is simply that Arweave undertakes to resolve blockchain’s fatal scalability flaw. It does this by resolving two interrelated challenges.
Challenge 1: A faster way to get consensus
The problem with consensus mechanisms on the blockchain is that they require vast amounts of energy to function. Every block of data must be shared in its entirety in order to guarantee the safety and integrity of its data. Therefore, the more information it stores, the greater its unwieldiness. In terms of data storage, it’s a system that must eventually fail.
Arweave solves this challenge by allowing each point in the blockweave to verify the integrity of stored data without actually needing to store and process the data itself. It achieves this by having each node share an encrypted code identifying the location of the data (known as a hash) in place of the actual data.
In this way, the data which must be shared to ensure data reliability is vastly reduced and a feasible speed of data transfer can be maintained.
Challenge 2: Costless access to data
Whenever a user wishes to access data stored on a conventional blockchain they must pay to do so. The more information is stored, the greater the cost. This provides a strong disincentive for end-users to access stored data. For a system of sharing immutable, permanent data to be commercially viable, accessing its data needs to involve no usage cost.
Arweave solves this problem by implementing a ranking system for each node, based on how quickly a participant opts to respond to requests to access and share data. Effectively, it’s a system which actively incentivizes participants to be “good data citizens.” The more a node shares data, the better positioned it will be to efficiently mine and the better served it will be by the network. The end result is a system which can allow those receiving data to do so freely.
To understand Arweave simply as a faster and cheaper way of making data completely trustworthy is missing the point. The importance of Arweave is that for the first time we may have a solution to the data problem. It may finally be possible to have freely exchangeable data which is also permanent and immutable. Arweave’s vision is to offer an incorruptible sister-version of the Internet, through which users can access data they know is one hundred percent reliable.
A Taste of Arweave’s Potential
Journalism and political commentary
Tech giants and media outlets worldwide are facing a crisis of trust. In the age of the Cambridge Analytica scandal, the information globalized corporations shape and disseminate to the public are considered to be fundamentally untrustworthy. Arweave may be the part of the solution in restoring modern society’s ability to discern truth from politically motivated falsehoods.
Arweave can deliver truly tamper-proof and censorship-resistant content. Those consuming its content are able to know that any data served by Arweave is exactly identical to the data released by its source. In a time when fake news and a general manipulation of data for political ends abounds, this newfound reliance will be a game-changer for how reputable journalists gather information and disseminate commentary.
Legal data
Administering contemporary law is inextricably tied to data. The volume of data law firms and legal departments must work with is growing exponentially and legal institutions are straining under the load. The fallout is a less responsive, less efficient and less just judicial process.
An agile and responsive legal system is dependent on a capacity to exchange authentic data. In providing a completely trustworthy repository of digital content, Arweave can support legal processes and make a meaningful impact on the stability of civil society.
Globalization and data permanence
Data stored on the Internet lacks permanence. Its removal is as straightforward as turning off the server it’s stored on. In today’s increasingly globalized economy and society, this lack of permanence is rapidly becoming a critical threat to stability. The entire complex system is dependent on persistent records of financial exchange. By solving the problem of scale and cost, Arweave presents a system which can deliver that stability and reliability of data.
What Comes Next
Finding a way to deliver low cost, immutable data storage isn’t just a computing challenge. It’s a crisis of confidence affecting everything from how culture remains cohesive, to our political and judicial systems, to the vast ledger of global transactions our globalized economy demands. One need only turn on the news in the morning to find new and alarming examples of how our unstable data foundations make the whole system vulnerable.
Since its arrival in the first decade of the twenty-first century, blockchain has held tantalizing promise as a way of solving the data problem, but scalability was its downfall.
Arweave is the first idea we’ve seen which can build on blockchain to create a definitive solution. Arweave can scale with our burgeoning data flow, incentivize participation in ethical data sharing and, most crucially of all, provide a platform of trust our globalized society so urgently needs.
It’ll be fascinating to see where we Arweave can take us.
Links:
Arweave Website
Disclaimer: As with all investments, you should do your own research. The information provided in this article is focused primarily on bringing the facts out of the white paper to supplement your due diligence. Please do your own research and make up your own mind.