The year is 2015 and Ethereum is well on its way to realizing a grand vision that will change the world - creating the world computer. A serverless Web, free of large corporations and economically self-sustaining through blockchain technology.
The founders of Ethereum already understood then that for the world computer to live up to its name it needed three crucial components. So they extended this analogy with the concept of the “holy trinity”. Ethereum as the CPU, Whisper as the signaling and messaging protocol, and Swarm as the world computer’s hard drive.
Thus, Swarm was born to store the data of the world’s second-largest blockchain network. Now, nine years after its inception, Ethereum has finally made the much-awaited transition to the proof-of-stake consensus algorithm, dramatically lowering its environmental footprint. The network is now on its way to increasing its transactional capacity by orders of magnitude.
In the shadow of this momentous transition, another one is taking place. A more subtle, more quiet one, but almost as equally important. The Swarm network is introducing storage incentives.
Ethereum brought us disintermediated and trustless services. Unfortunately, their data is still stored within large centralized data silos. Swarm is the missing piece that will provide the Ethereum network with compatible, fully decentralized, and trustless storage, returning trust into the data economy. Storage incentives play a crucial role in this.
In this blog, based on his interview, Viktor Trón, team lead of Ethereum Swam, talks in-depth about storage incentives - what they are, how they work, and most crucially, why they are so important. Not just for Swarm, but for blockchain and crypto as well.
“There were about 10 times when I already thought to myself that we have found the solution [to storage incentives]. But it took us almost 20, 30 iterations, maybe even more, to get it right,” Viktor starts the conversation.
What proved to be a nagging and persistent problem to this day is the separateness of storage and bandwidth incentives. Bandwidth incentives are fairly straightforward to solve. They ensure a smooth operation of the network and motivate nodes to pass messages to each other by adequately compensating them for this work. Nodes keep a direct peer connection to a set of other nodes forming a particular topology called Kademlia.
This topology guarantees to route between any two nodes by relaying messages through direct connections. Wherever requests are ultimately routed, they initially get forwarded to this same set of peers allowing an accounting arrangement based on repeated dealings. Nodes get credit for serving a request, while they get debited for initiating one. If peers provide and demand service to the same rate, the accounting might be balanced in the long run.
“In Swarm, we want to account for where and when a resource has been consumed in the most direct way possible,” he emphasizes. With bandwidth incentives this is possible. When a satisfactory result is achieved you can create a direct accounting entry.
How can you prove your data is stored?
With storage incentives, things are very different. Here, you need to compensate a node for providing disk space for chunks of data. “It turns out that this was a much more sophisticated problem to solve than we initially thought. This is because storing content is a service that is promissory in nature, in contrast to instant gratification for providing bandwidth,” he explains.
The promissory nature means that you can only verify whether your data was actually stored sometime in the future. It’s a problem analogous to sending a package and paying in advance. You need to have a mechanism that will ensure a refund in case you don’t receive the package. The reverse is also true. There needs to be some sort of an escrow account that guarantees you won’t just take the goods. This problem alone makes storage incentives much more complicated than the incentives, Viktor warns.
Postage stamps to the rescue
Another hard problem to solve was presence - proving that you stored the things somebody asked you to store. There is a two-part mechanism that allows you to do that.
The first part is proof of retention. It can prove that you have retained the data in all its integrity. The second part is called proof of relevance, where storers have to provide proof that they are storing relevant content, not just some random files.
“What this means is that proof of relevance is the same construct as the one that proves to the storers that the content has been paid for,” he adds. In Swarm, this construct is called postage stamps.
Postage stamps exist in batches on the blockchain, a sort of “books of stamps” which uploaders use to pay for storage rent. “With them, you signal that a chunk needs to be stored and you also signal the importance of its delivery through the amount of money you pay,” he clarifies their use.
“It is also sort of like staking. You tie up some BZZ liquidity with the stamps, which acts as deflationary pressure on the token’s supply. This aligns the interests of the token holders with nodes, which are incentivized to respect the rules and store chunks since the tied-up liquidity creates upward pressure on the token price,” he describes another aspect of postage stamps.
“Call us slow, but it was a hard one to crack”
While the team was growing the Swarm network and started rolling out different versions of it, they made several discoveries related to choosing the forwarding Kademlia model as the basis for the network.
“Solutions to seemingly unrelated problems fell out of it as if by magic. This was additional proof that we chose the right model and the same goes for storage incentives. The brutal truth here is that they took us seven years. We just figured it out. And you can call us slow, you can call us unimaginative, but that’s how long it took us. It was a hard one to crack,” Viktor unapologetically describes the trodden path.
And as if solving the problem of the promissory nature and proving data presence weren’t enough, there’s also the problem of identity to unravel. Just imagine the grueling task of proving that one person is not posing as maybe two, five, or even ten.
The next step in the evolution of blockchain
If by any chance you’re still not sure what makes storage incentives the “holy grail”, it is the fact that there are so many concurrent and novel challenges to tackle:
- The incentives’ promissory nature,
- how to protect the network against “fake” nodes,
- how to set and keep a long-term price of storage, while preventing nodes from arbitrarily raising them,
- the previous point also raises the question of how to deal with the absence of uploaders,
- how to remove a contractual relationship between uploaders and storers and solve this on a system-wide level,
- how to set up payments for data to be stored,
- how to enable node operators to withdraw their reward from the pot,
- managing additional complexity introduced by redundancy (i.e. when the same information is stored in several nodes).
“If all these are solved, then storage incentives can realize the original idea of a world computer. One that encompasses data storage, processing, computation, and messaging between nodes. The promise of a decentralized eBay, AirBnB, social media… would be possible, not just half-possible. They are the underpinning infrastructure for Web3 as it was envisioned,” Viktor elucidates why the metaphor of a holy grail is so apt.
However, it is not just the storage incentives that are hidden beneath the eye-catching metaphor. It is also the way Swarm solves the problem of anonymity by obfuscating the source of the uploaded data, while only the recipient can decipher and contextualize the messages sent to them.
“It is no small feat. If [the importance of] this is recognized, in a sense it makes Swarm the next big thing. It will turn Swarm into the component that extends the blockchain to the world computer and create the next step in the evolution of blockchain. I know that this is an audacious claim, but that’s what I think,” he drives the point home.
Now is the time for decentralized networks to take the next step in their evolution.