A High-Throughput, Low-Latency Public Blockchain Platform for Decentralized Economies
As blockchain technology marches forward in its quest for mass adoption, the biggest hurdle it faces is arguably scalability. More and more merchants the world over have warmed up to the possibility of accepting crypto, particularly Bitcoin and Ether, as means of payment. But if crypto is to be considered a serious contender for a borderless payment solution, then it has to go head-to-head with the prevalent electronic payment services like Visa and Mastercard. And as things stand today, crypto is lagging far behind. Bitcoin can handle less than 10 TPS, while Ethereum can manage up to about 40 TPS on a good day. Compare this to Visa’s daily 2,000 TPS (which can theoretically scale up to ~50,000 TPS), and you have an idea of the gap that has to be bridged.
Various crypto projects have tried out different techniques to solve the scalability bottleneck. Ethereum has been trying to implement a scaling solution of its own for what seems like an eternity now. Increasing network throughput typically involves making the network more centralized or sacrificing on security. This is known as the classical Blockchain Trilemma — when it comes to Decentralization vs Scalability vs Security, you have to choose 2 out of 3. One scalability technique that doesn’t compromise on either security or decentralization is called ‘sharding’. Sharding involves fragmenting the available dataset horizontally into smaller datasets, called ‘shards’. Centralized systems have already been using this technique for a long time now, but it has caught the attention of the decentralized world fairly recently.
Zilliqa is possibly the poster boy of sharding when it comes to blockchain projects. A ‘high-throughput public blockchain’, Zilliqa has achieved 2828 TPS in their testnet. It should be noted here that in sharding solutions, the TPS increases with the number of nodes. Zilliqa has achieved this with 3600 nodes. Over the last few months, a handful of projects have come up working on the same sharding principle. Harmony is one of the most exciting projects working in this space now. A next generation public, scalabale blockchain infrastructure for the decentralized economy, Harmony promises to connect 10 billion people seamlessly by applying “10x innovations in networking, systems and innovations”. But what is Team Harmony doing differently to make sure that they live up to their promise? And what sets Harmony apart from Zilliqa and other predecessors? Let’s take a closer look here.
Website: click here
Telegram: join here
Whitepaper: read here
State Sharding Technique
Though both Zilliqa and Harmony employ sharding, Zilliqa implements what is known as network sharding and transaction sharding, while Harmony goes one step further and implements state sharding. To put it simply, Zilliqa splits its network into multiple shards, each with hundreds of nodes (network sharding). This allows different transactions are processed by different shards concurrently (transaction sharding). However, in order to be able to process transactions, each node in the shard has to store the entire blockchain state information. This prevents machines which have limited resources from being part of the network — not an encouraging sign for decentralization!
Harmony, on the other hand, shards the blockchain state as well (state sharding). They have christened this unique sharding technique as Deep Sharding, wherein they shard not only the transactional layer but also the layer where consensus happens. Nodes can securely locate other nodes that are relevant to the specific transaction, and transact with only those specific nodes in the consensus process. This increases the level of parallel processing, leading to extremely high network throughput. Moreover, since all nodes do not need to store the entire blockchain state, simple personal computers can join the network as nodes too, leading to a greater amount of decentralization.
PBFT Consensus Mechanism
Zilliqa uses a consensus mechanism known as PBFT (Practical Byzantine Fault Tolerance) protocol. In this algorithm, one node is elected as ‘leader’ and the other nodes become ‘validators’. Each consensus round has 2 phases — prepare and commit. In each phase, the leader node broadcasts a proposal to all validator nodes, and the validator nodes on their part broadcast their votes to every other node. The votes of all validators need to be counted by every other validator. This increases the communication complexity of PBFT to O(n*n), where n is the total number of nodes. Needless to say, this algorithm becomes unrealistic for networks with hundreds of nodes.
Harmony uses a modified consensus algorithm called FBFT, or Fast Byzantine Fault Tolerance. In this method, all validator nodes do not broadcast their votes. Instead, the leader node collects the validators’ nodes through a multi-signature signing process, and then broadcasts it. The multi-signature size is O(1), which reduces the communication complexity from O(n*n) to O(n) — a humongous leap from Zilliqa’s communication complexity. The validators are chosen based on a PoS (Proof of Stake) mechanism, where nodes who wish to become validators have to stake a certain minimum number of tokens. PoS is much less energy extensive than PoW (Proof of Work), which traditional sharding-based blockchains like Zilliqa use. Larger the amount of tokens being staked, greater is the chance to be selected as a leader node. Moreover, validators with larger stakes have a greater incentive of following the protocol and not engaging in any malicious behaviour, since any such activity will result in loss of the entire cache of staked tokens.
Distributed Randomness Generation
Sharding systems generally employ a randomness-based approach when it comes to assigning nodes to shards, in order to reduce the risk of any attack. A random number generation scheme is used in these cases to assign nodes randomly to shards. This random number should be completely unpredictable and unbiased, so that a malicious node has no way of figuring out which shard it will be assigned to. Moreover, the random number generated should also be verifiable at any given time, and the generation algorithm itself should be scalable.
Let us consider some of the sharding-based public blockchain solutions which Harmony has used as case studies. OmniLedger uses the RandHound protocol for distributed randomness generation (DRG), which involves dividing the nodes into groups of size c each. The problem with this approach is that the complexity is O(n*c*c), which is too slow to be feasible for large decebtralized networks. RapidChain, on the other hand, uses a simpler and faster approach, where each node can perform Verifiable Secret Sharing (VSS). Unfortunately, this protocol is fast but not completely secure. Algorand, on its part, uses a Verifiable Random Function (VRF) based cryptographic procedure. And Ethereum’s PoS-based scaling solution (Ethereum 2.0) is proposing a Verifiable Delay Function (VDF), where the reveal of the random number would be delayed so as to lessen the chances of any attack.
Harmony utilizes the strengths of both VRF and VDF. The validator nodes each create a VRF-based random number, and send it to the leader node. The leader collects this set of random numbers, and generates the final random number from this data set. VDF is used to delay the reveal of this random number, which makes this approach more secure. Furthermore, the algorithm complexity is only O(n) in this case, which makes it extremely scalable as well.
Kademlia Cross-shard Communication
For any sharding-based system, the protocol for cross-shard communication is extremely important. Zilliqa depends on the main chain itself for initiating transactions across different shards. In OmniLedger, messages between shards are exchanged through clients. And RapidChain relies on the nodes in the shards themselves to initiate communication with nodes in other shards. Harmony has adopted RapidChain’s shard-driven approach. Typically, the communication complexity for this shard-driven approach is O(n), but Harmony uses the Kademlia routing protocol to reduce this complexity to O(log(n)).
In Kademlia routing protocol, each node in the network maintains a routing table that contains the distance of all other shards. This distance is a function of the shard ID. When nodes in a certain shard want to send a message to another shard, instead of broadcasting the message to all neighbouring shards, the nodes will send the message only to the shard with the closes ID. By this method, messages only travel over O(log(n)) shards before reaching their intended destination.
Beacon Chain
Harmony’s sharding architecture consists of a beacon chain alongwith the shard chains. Each shard chains processes a subset of the total transactions. The beacon chain is a special shard chain with 2 extra responsibilities — generating the random number for DRG, and accepting the stakes from validators. The beacon chain also increases the security of the whole ecosystem. After every new block is added to a shard chain, the block header is sent to the beacon chain (via Kademlia routing). The beacon chain checks if the block header is valid, and broadcasts it to the entire network. Each shard maintains a list of valid block headers, which is used to check if transactions for other shards are valid.
With the beacon chain being present, attackers have to corrupt both the shard and the beacon chain if they want to insert an invalid block in the chain. Moreover, since the beacon chain broadcasts all valid block headers itself, the communication complexity is just O(n) once again. If each shard had to separately broadcast the list of valid block headers to all other shards, then the complexity would have been O(n*n). As it turns out, the beacon chain is an extremely important entity in Harmony’s ecosystem both in terms of security and scalability.
Parting Thoughts
Harmony dreams big, but big dreams are what you need if you want to achieve something monumental. In their testnet, they have achieved 118,000 TPS with around 44,000 nodes. To be honest, bringing forward the product from the testnet to the real world is the actual challenge; and Team Harmony definitely possesses the talent to implement their vision. Moreover, they are a team which listens to the community. You can always find their developers available on Telegram and Discord, ready to answer your queries and listen to your feedback. Finally, this article has been written based on their succint technical whitepaper, and I encourage all readers to give the WP a read too.