Today, let’s look at bloXroute, a network abstraction layer for blockchains (plural). Their focus is to create an agnostic infrastructure that can be leveraged by all blockchains. No code available, so unfortunately we can’t do a code review, but we can dive into some of the internals for a regular tech review.
A primary bottleneck for blockchains is network throughput. Once you have all the transactions locally, you can process them very quickly and you can mine a block as fast as your system allows. The problem is collecting the transactions to put in the transaction pool, and sharing the mined blocks with the rest of the network.
Let’s quickly discuss high level architecture.
Alice creates a transaction via MyEtherWallet (website), MyEthereWallet sends this transaction to their infura node (blockchain server), (this infura node does not mine, so it sends the transaction along to all nodes it is aware of (via the bootnode)). This sentence is already to complex, let’s break it down.
A node is a piece of software you run on your computer. The network is composed of of lots of nodes. When you start up a node it needs awareness of the other nodes on the network, but it can’t just discover this, so instead it begins by talking to a boot node. Boot nodes are hard configured always online nodes with a few failovers, this way we always have a starting point. We ask this boot node to please give us a list of all active nodes, the boot node then gives us that list.
Ok, so the node receives a transaction from another node, it puts this transaction into its transaction pool (collection of transactions). If it is a miner it will take these transactions and apply them (check if they are valid and if their input/outputs would make sense) and put them in a block; it then attempts to seal this block (by mining), if it successfully seals it, it transmits it to another node. When the other node receives it, they check the block solution, if they agree with it, they replay the transactions in the block, applying it to their own storage and then they repeat the cycle.
So the main bottlenecks for scalability are CPU (can be solved with multithreading, asynchronous behavior, sharding, state channels, plasma, and who knows what we will think of next), disk (sharding, parallel processing), and network (??)
bloXroute is attacking this last one, and in quite an interesting way. They implement system wide caching, gigabyte size blocks, and they skip the p2p routing by instead implementing gateways.
So how do they do this, first off a memory store (something akin to Redis) where all transactions are kept and stored for specific intervals (most likely first in first out), second a gateway (point of contact for servers), all transaction broadcasts propagate to here (like it was another node) and it then caches this data and start creating a transfer block, compression, and when requested it passes along the entire block. This increases throughput because of the mem caching aspect, the availability of knowing where to find the data, and the size of transmission. Let’s see how they did it.
A similar solution could be achieved by changing boot nodes to instead be the central point of contact for all transactions, they collect and create and then they propagate, is this a centralized model. The bloXroute network itself will however be decentralized, but nodes will specifically talk to it, so it’s a bit of an amalgamation, it’s not centralized in a control perspective, but centralized in a requirement perspective.
So let’s quickly talk about CDN’s, (Content Delivery Networks), websites nowadays are optimized by CDN’s, we know specific content is re-used time and time again, so we put them on a CDN (which distributes them across the world in specific locations) so that we can optimize getting data from there. They increase delivery by four mechanisms, 1, keep data in memory (memory is much faster than disk), 2, they keep strong network infrastructure (gigabyte infrastructure), 3, they have locations all around the world to facilitate faster lookup, and 4, they have fast routing tables to quickly find the fastest and closest location and provide the data.
This is what bloXroute is doing, they are providing a CDN (or BDN) for blockchains.
Why does a CDN work? Because the data is static. This means it doesn’t change often and when people access it, it feeds the same data. This changes considerably when the data is no longer static, like live streaming transactions. But the argument could be made that this data is still shared across many locations.
After I wrote the above, I had a very fruitful conversation with the bloXroute guys, and we have a bit more data now. So first off, you install a piece of software called a magic gateway on your server. This magic gateway is configured to talk to the bloXroute routing infrastructure. This gateway has the following jobs.
Communicate p2p with other magic gateways
Communicate directly with the bloXroute infrastructure
Keep in memory a list of all known transactions that are not yet archived with a 4 bit lookup index
Share lookup index contained in a block with other magic gateways.
So let’s run through the example practically.
Alice sends a transaction. This transaction arrives at the node A, the node talks locally to the magic gateway and gives it the transaction, the magic gateway creates a deterministic 4 bit unique key for the transaction. At this point normal p2p transaction propagation occurs from the node.
Node B has been busy creating a block. When Node B finishes creating it’s block it sends the block to it’s magic gateway, the magic gateway confirms the transactions in the block, and sends a block skeleton with the 4 bit known transaction keys to bloXroute A.
bloXroute A is located in US. bloXroute has set up infrastructure in all the main data centers in the world to allow for fast routing, so bloXroute A quickly uses AWS us-east-1 to send to AWS eu-west-1 and then out to bloXroute B, so very quickly, all the bloXroute gateways become aware of this new block (in the mean time, the block is still slowly being propagated p2p via the normal p2p node mechanisms.
Think of the p2p spreading like a virus slowly taking over cells until all cells have the same data. Think of bloXroute as linearly connected A to B points.
So from here, all the magic gateways become aware of the data in the bloXroute network, and they each grab their mini block (compressed, streamed, and with 4 bit keys instead of full transactions), they compare this to their known transactions and if they agree they can unpack the block into the full block, and give it to their node locally.
So from the above, which one do you think is going to reach network saturation first? The big block being spread via p2p, or the streamed mini block with direct gateway routing to all magic gateways?
bloXroute Initial Verdict
This is truly an infrastructure project by the very definition of the word, these guys will create the first interconnected blockchain mesh and CDN (or a BDN, blockchain delivery network).
I won’t go into the team and business aspect for now as we may have more to say later, but you should go look for yourself on the bloXroute site. These guys have also already proved themselves with Falcon.
Looks like a great project and I think it can add a lot of value for blockchain scaling, I’ll be running my own systems on them as well.
Comments