Summary
Solana is a high-performance blockchain platform that uses a unique technical architecture to achieve high throughput and low latency. Its core technologies include the Proof of History (POH) algorithm to ensure transaction order and global clock, Leader Rotation Schedule and Tower BFT consensus mechanism to increase the block rate. The Turbine mechanism optimizes large block propagation through Reed-solomon encoding. Solana Virtual Machine (SVM) and Sealevel parallel execution engine speed up transaction execution. These are the architectural designs that Solana uses to achieve high performance, but they also bring some problems, such as network downtime, transaction failures, MEV problems, rapid state growth and centralization problems. We also focus on the problems caused by this mechanism in this article.
Solana's ecosystem is developing rapidly, and various data indicators have grown rapidly in the first half of the year, especially in DeFi, infrastructure, GameFi/NFT, DePin/AI, and consumer applications. Solana's high TPS and consumer application-oriented strategy and the ecological environment with weak brand effect provide entrepreneurs and developers with abundant entrepreneurial opportunities. In terms of consumer applications, Solana has demonstrated its vision for promoting the application of blockchain technology in a wider range of fields. By supporting projects such as Solana Mobile and building SDKs specifically for consumer applications, Solana is committed to integrating blockchain technology into everyday applications, thereby increasing user acceptance and convenience. For example, applications such as Stepn provide users with novel fitness and social experiences by combining blockchain and mobile technologies. Although many consumer applications are still exploring the best business model and market positioning, the technical platform and ecosystem support provided by Solana undoubtedly provide strong backing for these innovative attempts. With the further development of technology and the maturity of the market, Solana is expected to achieve more breakthroughs and success cases in the field of consumer applications.
Although Solana has gained significant market share in the blockchain industry with its high throughput and low transaction costs, it also faces fierce competition from other emerging public chains. As a potential rival in the EVM ecosystem, Base's number of active addresses on the chain is growing rapidly. At the same time, although Solana's total locked volume (TVL) in the DeFi field has set a record high, competitors such as Base are also rapidly occupying market share. The financing amount of the Base ecosystem also surpassed Solana for the first time in Q2.
Although Solana has made certain achievements in technology and market acceptance, it needs to continue to innovate and improve to meet the challenges from competitors such as Base. In particular, in terms of improving network stability, reducing transaction failure rate, solving MEV problems, and slowing down the growth rate of state, Solana needs to continue to optimize its technical architecture and network protocols to maintain its leading position in the blockchain industry.
Technical Architecture
Solana is known for its high TPS and fast Finality brought by its POH algorithm, Tower BFT consensus mechanism, Trubines data transmission network and SVM virtual machine. We will briefly introduce how each component works, how to achieve its high performance goal for architectural design, and the drawbacks and derived problems brought about by this architectural design.
POH Algorithm
POH (Proof of History) is a technology for determining global time. It is not a consensus mechanism, but an algorithm for determining the order of transactions. POH technology is derived from the most basic cryptography SHA 256 technology. SHA 256 is usually used to calculate the integrity of data. Given an input X, there is only one output Y, so any change to X will result in a completely different Y.
POH sequence, source: Solana white paper
In Solana's POH sequence, the integrity of the entire sequence can be ensured by applying the sha 256 algorithm, which also determines the integrity of the transactions therein. For example, if we package the transactions into a block and generate the corresponding sha 256 hash value, then the transactions in this block are determined, and any changes will cause the hash value to change. After that, the hash of this block is used as part of X, which will be the next sha 256 function, and the hash of the next block is added. Then the previous block and the next block are determined, and any changes will result in a different new Y.
This is the core meaning of its Proof of History technology. The hash of the previous block will be part of the next sha 256 function, similar to a chain. The latest Y always contains proof of history.
Transaction Flow Architecture Diagram, Source: Solana White Paper
In Solana's transaction flow architecture diagram, the transaction process under the POH mechanism is described, in a system called Leader Rotation Under the rotation mechanism of Schedule, a Leader node will be generated among all the validators on the chain. The Leader node collects transactions and sorts them for execution, generates a POH sequence, and then generates a block to propagate to other nodes.
Leader election mechanism, source: Helius
In order to avoid single point failure at the Leader node, a time limit is introduced. In Solana, the time unit is divided by epoch. Each epoch contains 432,000 slots (time slots). Each slot lasts for 400 ms. In each slot In the rotation system, a Leader node is assigned to each slot. The Leader node must publish blocks within the given slot time (400 ms). Otherwise, the slot will be skipped and the Leader node of the next slot will be re-elected.
In general, the Leader node uses the POH mechanism to make all historical transactions final. Solana's basic time unit is Slot, and the Leader node needs to broadcast blocks within a slot. The user sends the transaction to the Leader through the RPC node. The Leader node packages the transactions, sorts them, and then executes them to generate blocks. The blocks are propagated to other validators. The validators need to reach a consensus through a mechanism to reach a consensus on the transactions and order within the block. The consensus uses the Tower BFT consensus mechanism.
Tower BFT consensus mechanism
Tower BFT protocol, source: Helius
The Tower BFT consensus protocol comes from the BFT consensus algorithm and is a specific engineering implementation of it. The algorithm is still related to the POH algorithm. When voting on a block, if the validator's vote itself is a transaction, then the block hash formed by the user transaction and the validator exchange can also be used as a historical proof, and the transaction details of each user and the voting details of the validator can be uniquely confirmed.
Voting icon
In the Tower BFT algorithm, if all validators vote on the block and more than 2/3 of the validators vote for approve, then the block can be confirmed. The benefit of this mechanism is that it saves a lot of memory, because only the hash sequence needs to be voted on to confirm the block. However, in the traditional consensus mechanism, block flooding is generally adopted, that is, a validator receives a block and then sends it to the surrounding validators, which will cause a lot of redundancy in the network because a validator receives the same block more than once.
In Solana, due to the large number of validator voting transactions, the efficiency brought by the centralization of Leader nodes and the 400ms Slot time, the overall block size and block frequency are particularly high. When large blocks are propagated, they will also cause great pressure on the network. Solana adopts the Turbine mechanism to solve the problem of large block propagation.
Turbine
Turbine block propagation mechanism, source: Helius
Leader nodes split blocks into shredded sub-blocks through a process called Sharding, and their size is measured in MTU (maximum transmission unit, the maximum amount of data that can be sent from one node to the next without splitting it into smaller units). Then, the integrity and availability of the data are guaranteed by using the Reed-solomon erasure code scheme.
Reed-solomon encoding scheme, source: Helius
By dividing the block into four Data Shreds, and then in order to prevent packet loss and damage during data transmission, Reed-solomon encoding is used to encode four packets into eight packets. This solution can tolerate a packet loss rate of up to 50%. In actual tests, Solana's packet loss rate is about 15%, so this solution is very compatible with the current Solana architecture.
In the underlying data transmission, the UDP/TCP protocol is generally considered. Since Solana has a high tolerance for packet loss, the UDP protocol is used for transmission. Its disadvantage is that it will not retransmit when packets are lost, but its advantage is a faster transmission rate. On the contrary, the TCP protocol will retransmit multiple times when packets are lost, which will greatly reduce the transmission rate and throughput. With Reed-solomon, this solution can significantly increase Solana's throughput. In a real environment, the throughput can be increased by 9 times.
Schematic diagram of layered propagation, source: Helius
After Turbine shards the data, it uses a multi-layer propagation mechanism to propagate it. The Leader node will hand over the block to any block validator before the end of each Slot. The validator will then shard the block into Shreds and generate an erasure code. The validator will then start Turbine propagation. First, it must be propagated to the root node, and then the root node will determine which validators are located on which layer. The process is as follows:
1. Create a node list: The root node aggregates all active validators into a list, and then sorts them according to each validator's equity in the network (that is, the amount of staked SOL). The ones with higher weights are in the first layer, and so on.
2. Node grouping: Each validator in the first layer will then create its own node list to build its own first layer.
3. Layer formation: Divide the nodes into layers from the top of the list. By determining the depth and breadth values, the approximate shape of the entire tree can be determined. This parameter affects the propagation rate of shreds.
Nodes with a higher equity ratio can obtain complete shreds in advance when they are in the upper layer during the hierarchical division, and then they can restore the complete block. However, due to the loss of transmission, the probability of obtaining complete shreds for nodes in the later layers will be reduced. If these shreds are not enough to build a complete fragment, the leader will be required to retransmit directly. At this time, the data transmission will be carried out inside the tree, and the nodes in the first layer have already built a complete block confirmation. The longer the time for the validators in the later layers to vote after completing the block construction.
The idea of this mechanism is similar to the single-node mechanism of the leader node. There are also some priority nodes in the process of block propagation. These nodes first obtain shreds fragments to form a complete block to reach a voting consensus. Pushing redundancy to a deeper level can significantly speed up the process of Finality and maximize throughput and efficiency. Because in fact the first few layers may represent 2/3 of the nodes, then the votes of subsequent nodes are irrelevant.
SVM
Solana can process thousands of transactions per second, mainly due to its POH mechanism, Tower BFT consensus and Turbine data propagation mechanism. However, as a virtual machine for state transition, if the Leader node processes transactions slowly, the throughput of the entire system will be reduced. Therefore, Solana proposed the Sealevel parallel execution engine to speed up transaction execution for SVM.
Sealevel parallel execution diagram, source: Xangle
In SVM, instructions consist of 4 parts, including program ID, program instructions, and a list of accounts that read/write data. By determining whether the current account is in a read or write state and whether the operation to be changed in state is conflicting, the parallelization of the account's transaction instructions without conflicting states can be allowed, and each instruction is represented by a Program ID. This is one of the reasons why Solana’s validator requirements are very high, because the validator’s GPU/CPU is required to support SIMD (single instruction multiple data) and AVX advanced vector extension capabilities.
Ecological Development
Solana Ecological Landscape, source: Gate Ventures
In the current Solana ecological development process, it is increasingly inclined towards practical utility, such as Blinks and Actions and even Solana Mobile, and the development direction of officially supported applications is also more inclined towards consumer applications rather than unlimited involution of infrastructure. With Solana's current performance being sufficient, there are more diverse applications. As for Ethereum, due to its low TPS, the Ethereum ecosystem is still based on infrastructure and expansion technology. If the infrastructure cannot support applications, it will not be able to build consumer applications, which has led to an unbalanced state where too much money is invested in infrastructure but too little in applications.
DeFi
DeFi Landscape, source: Gate Ventures
In the DeFi protocol on Solana, there are a large number of projects that have not issued tokens, including Kamino (the first Lending), Marginfi (Lending + Restaking), SoLayer (Restaking), Meteora, etc. Due to Solana’s united ecological atmosphere, usually when a project is issuing coins, other projects will try to avoid it as much as possible to attract enough market attention.
DEX market share, source: Dune
Currently, the entire DEX is highly competitive, and its leader has also undergone multiple migrations, from Raydium, Orca to the current Jupiter as the dominant force.
Initiator of DEX transactions, source: Dune
It is worth noting that about 50% of DEX transactions are initiated by MEV bots, mainly because of their low fees and active Meme transactions, which have bred the profitability of MEV. This is also one of the main reasons for the frequent failures and downtime of user peak transactions.
Solana TVL, source: Deflama
The DeFi protocol on Solana has experienced an explosive rise along with the rise in the price of SOL, and its USD nominal TVL has also experienced an explosive rise. The rising trend of its TVL has not stopped, and a new wave of rising trend has formed.
In short, although the competition in the Solana track is fierce, there are still changes. Unlike Uniswap on Ethereum, which occupies the brand mind of users, DEX, which should have high stickiness and network effects, will also face the risk of being replaced. The transactions of the Solana main chain are filled with MEV bots, which has caused great user experience problems for users and still needs to be solved. In the overall direction, Solana's TVL is still growing very rapidly, and its subsequent DeFi ecological development is still very worth looking forward to. In addition, the brand mind of these applications is not strong in occupying users, which is a potential driving force for entrepreneurs to choose chains.
Infrastructure
Infrastructure Landscape, source: Gate Ventures
In the construction of infrastructure, the main leaders are oracles Pyth and cross-chain bridges Wormhole, as well as some targeted solutions that the public may know less about, such as:
1. Jito Labs: Focused on building MEV solutions on Solana, its Jito Labs client built Bundle and pseudo-emempool to give researchers MEV. Its market share currently exceeds 50%. In addition, the staked SOL of its LSD protocol Jito is also close to 12 million, and is still growing rapidly.
2. Helius: As an active contribution community on Solana, Helius has the most comprehensive research on Solana and contributes code through research.
3. GenesysGo: Its product ShdwDrive is a data storage project in Solana. It is committed to supporting projects that can be commercialized, including social data, website hosting and other businesses. It is still in the testnet stage. At the same time, its parent company GenesysGo is also building various public goods and researching for the Solana community.
In addition, Solana still has a large number of projects worth exploring waiting for the Chinese community to explore. We do find that these infrastructures have a huge influence on Solana's protocol-level construction, ecological development, and community, and there may be opportunities to further explore their potential, whether through investment or cooperation.
Gaming / NFT
Gaming / NFT Landscape, source: Gate Ventures
Solana also has a relatively rich GameFi and NFT ecosystem, among which Mad Labs occupies a relatively important position in the entire Solana ecosystem, and many project airdrops will give priority to Mad Labs holders, and in the past, this leading position was DeGods. The NFT market has also undergone changes. In the past, the most people used Magic Eden, but now it has changed to Tensor.
DePin / AI
DePin Landscape, source: Gate Ventures
Currently in Solana's DePin market, Render is the well-deserved leader with actual business. With Solana's development strategy centered on practical applications, it has also caught the narrative of Depin in this round of recovery. In the first half of the year, a large number of new Depin projects were built on Solana, including io.net, Nosana, Shadow, etc.
Consumer
Consumer Landscape, source: Gate Ventures
Whether it is Solana Mobile or the column built by the Solana Ecosystem official website specifically for consumer applications, the invention of Actions and Blinks, all illustrate Solana's vision for the commercialization and practicality of blockchain. The release of its Mobile collection is also to put the dapp of the Web end on the mobile end, which is very consistent with human nature and the development trend of the Internet. Therefore, applications are easy to explode on this soil, and the most typical one is Stepn.
Overlooking the consumer applications currently in operation, in fact, most of them still have not found a good breakthrough, so it is still impossible to realize a real application landing in the business world. This includes a variety of factors such as single product innovation, single business model, weak Web2 marketing, demand for Gas Fee, and entry threshold of tokens.
However, consumer applications are the final landing scene of blockchain technology, and also determine the ceiling of the public chain. Therefore, Solana's exploration of mobile consumer applications is very necessary, and our long-term exploration in this direction is also necessary. Especially in the current Ethereum ecosystem, infrastructure is far greater than applications. Ultimately, infrastructure is for applications.
Payments
Payment Landscape, source: Gate Ventures
Wallets on Solana include Phantom, Backpack, TipLink, etc. Like DEX, the brand effect here is not strong, so entrepreneurs have more opportunities. In the past, the leading wallet was Phantom, but now it has turned to Backpack, which was built by Mad Labs. It is worth mentioning that Mad Labs is now also the NFT leader on Solana.
Solana's stable currency issuance status, source: Deflama
It is currently cooperating with Paypal, Visa, etc. to conduct stable currency payment transfers on the chain. This business scenario itself is very conducive to the fast Finality and low Gas Fee Solana chain. At present, its on-chain stable currency is in a state of slow growth.
Stablecoin transfer stack chart YTD, source: Artemis
Solana had a remarkable stablecoin transfer market share in the first half of the year. However, after June, its market share has declined significantly. In the first half of the year, Solana's performance was absolutely outstanding, but its initial transfer data in the second half of the year showed a clear downward trend.
Competitor data
Number of active addresses on the chain, source: Artemis
Among the public chains, Base is regarded as a potential competitor of Solana in the EVM ecosystem. The number of active addresses on Base's chain is increasing rapidly, and Solana is still in a stage of rapid growth despite its first-mover advantage. NEAR maintained its high position, but Aptos and Sui fell behind in the competition among public chains.
TVL comparison, source: Artemis
Solana's TVL in the Defi field has also made significant progress. Its TVL has set a historical high and is still a long way from other public chains, but it is worth noting that Base is also in a stage of rapid growth.
Public chain stablecoin reserves, source: Allium
Currently, Solana's market share of stablecoin supply has been sluggish. Ethereum's market share has naturally shrunk due to the emergence of multiple chains, and Base's market share is quietly increasing.
Financing data, source: Messari
In terms of capital market financing, the financing frequency of the Base ecosystem has increased significantly in the most recent quarter and has exceeded that of the Solana ecosystem. Therefore, it can be seen from the market share and capital financing of various on-chain data that the competition between Base and Solana in the market is valid, and as Base matures, Solana will face greater competitive pressure. Moreover, Base and Solana have similar visions, both hoping to achieve the vision of Mass Adoption's Cryptio Native Consumer App with high TPS.
Technical Challenges
Downtime
Solana has experienced multiple downtimes in history. We have sorted out the specific events and reasons for the downtime:
The network performance declined, resulting in a large number of transactions being unable to be completed
The network was unstable and performance declined, lasting about 1 hour
Grape Protocol's IDO activity on the Raydium platform was hot. Many users sent a large number of transactions through written machine scripts. These transactions caused "memory overflow", causing the verification node to crash. Ultimately, the entire network was unable to produce blocks, and the interruption time was as long as 17 hours.
Due to large market fluctuations, the network was flooded with transactions submitted by a large number of arbitrage robots, causing the network to trigger severe load and interruption time of up to 30 hours.
Due to the casting of a new NFT project, a large number of robot transactions emerged, causing the main network nodes to lose consensus, and then suspended block production for up to 7 hours.
Due to a vulnerability in the durable nonce function in the transaction, the network was restarted, and the interruption time was about 4.5 hours.
Network downtime due to node configuration error
Solana mainnet performance problems eventually forced the validator nodes to automatically enter the "voting only" safety mode and were unable to process user transactions.
The BPF (Berkley Packet Filter) loader failed and the downtime was 4 hours and 46 minutes
Solana Due to its network architecture such as the Gulfstream Leader election mechanism and the single-node risk of the Leader node, subsequent predictions of the Leader node become feasible. As the number of network transactions increases, the memory pressure on the Leader of a single node will be greatly increased. The Leader node needs to be ready to retransmit blocks to the nodes in the Turbine tree at any time, otherwise the consensus vote cannot be completed. When a large number of DDOS attacks occur, system downtime caused by single node failure becomes extremely frequent.
In short, downtime is mainly due to the problem of not being able to produce blocks. It may be due to a single node failure caused by the Leader mechanism, causing problems in the block formation, or it may be that the consensus layer cannot reach a consensus on the block, resulting in the problem of not being able to produce blocks. Overall, this is closely related to Solana's own architecture and the software testing process.
Transaction failure
The proportion of user failed transactions, source: Dune
Users who have used Solana should know that our transactions often cannot be submitted normally. After a while, it is found that the transaction failed, which results in a very poor user experience. As shown in the figure above, according to statistics, among the transactions submitted by users, about 35% of the transactions are failed, requiring users to submit multiple times, and this proportion will be even greater when there are large fluctuations on the chain.
The main reason is the network layer technology QUIC, which is a relatively new technology.
Network protocol layer - 5-layer structure, source: Research Gate
QUIC (Quickl UDP Internet Connections) was proposed by Google and is an improvement to the transport layer of the HTTP 2.0 protocol. This experimental protocol is developed based on the UDP transport layer protocol and is also known as HTTP 3.0.
HTTP/2 and QUIC diagram, source: EMQX
TCP is more reliable than UDP, but UDP has a higher rate than TCP because TCP has a congestion control mechanism when packets are lost and retransmits lost packets. UDP has high speed but low reliability. Google hopes to build a high-reliability and high-speed transport layer protocol, QUIC. The core feature of QUIC is independent logical streams. It allows multiple data streams to be transmitted in parallel on a single connection, and each stream can be processed independently. In contrast, TCP only supports a single data stream and needs to receive and confirm each message in the order it is sent.
Failed transaction diagram, source: bread
The main reason for Solana's downtime is the use of the experimental application layer protocol QUIC. Due to the speed of UDP and multi-channel transmission, and the desire to maintain complete data transmission, it will also design a mechanism to retransmit packet loss multiple times. When the Leader node receives multiple transactions, it opens multiple channels through the QUIC protocol. However, the Leader node is a computer after all, and its transaction capacity is limited. Therefore, when a large number of transactions flow in, the Leader node will cut off some channel connections, which will cause the transaction to be dropped. There is no set standard for how to choose the connection to be cut off (such as cutting off all connections with a fee lower than xxx), and whether all connections will be cut off is random. Therefore, this leads to a certain amount of black box operation space. The Leader node may prefer profitable MEV transactions and abandon users' low-value transactions.
MEV
In Solana's block generation mechanism, since RPC interacts directly with the Leader and adopts the FCFS principle, it does not have the Mmepool like Ethereum. Due to the existence of Mempool and Ethereum's permissionless principle, Ethereum faces a more severe MEV problem in comparison.
MEV architecture, source: Helius
Jito Labs' client currently occupies 50% of the client market share, so Jito Labs built a pseudo mempool by itself. Users enter a pseudo mempool through RPC and stay for about 200 ms. Jito Labs provides an off-chain inclusion guarantee that ensures that all transactions in the bundle are included in the block. Searchers can bid for the opportunity to attack pending transactions. Searchers bid for the bundle that maximizes their profits, and then the Block Engine is responsible for finding the bundle with the highest bid and submitting it to the Leader running the Jito Labs client.
This is the root cause of MEV, but MEV has its positive externalities and demand. If Jito Labs does not make a pseudo mempool, other projects will do it. Therefore, Jito Labs chooses to eat this market to improve the mechanism of MEV and reduce negative externalities. Of course, this demand for MEV bots puts users in the most vulnerable position, because the validator will charge a handling fee, and the mev bot will obtain arbitrage profits, but users will suffer higher slippage and possible failed transactions.
State Growth
Solana's POH mechanism and Turbine consensus result in large blocks, which creates state growth issues. Currently, there is no exact answer to the ledger size, and the ledger is still growing at a rate of one block per 450ms in practice, about 4PB per year (running at maximum performance of 1GBPS). Currently, Solana's history construction occurs after 2 epochs, which is about 4 days (a total of 100-200GB). In the past, data was stored in the Google Bigtable database.
Solana's ledger data is not transparent, and the official has not disclosed much about the extremely high block size and potential impact caused by the pursuit of large blocks and high TPS throughput. The storage of the ledger is also completely dependent on a third party, because the official has also found that centralized databases such as Google have higher performance than Genesys Go and Arweave. At present, these decentralized databases still have problems with commercialization. This extremely growing state and centralized hosting are one of the reasons why Solana has been criticized.
Outlook
Solana also released its future roadmap, including:
1. Improve the protocol for issuing tokens, including transfer encryption, Hooks, and metadata pointers.
2. Client improvements, including the lightweight client Tinydancer, the transitional client Frankendancer, and the final client Firedancer.
3. Supporting development components of the ecosystem: Gmaeshift focuses on game SDK, armada markets focuses on token life cycle improvement, SPE focuses on enterprise-level SVM blockchain, virtual machine improvement, etc.
We can see that Solana's POH algorithm and Turbine consensus mechanism prioritize the performance of the blockchain's trilemma. The benefit is that it has the best performance in the current environment, bringing a wider range of application boundaries to explore. And with Solana's strategic goal of consumer applications, there is a great possibility that some Mass Adoption applications will emerge. At the same time, the brand effect of projects on Solana is weak, so there are more opportunities for entrepreneurs.
Solana's main advantages in ecological development are DePin/AI and Meme, but we can also see that its ecological development has not yet reached the expected development, and Consumer App has not yet been commercialized. In terms of competitors, there are also rising stars such as Base, and Base's financing amount and market share are rapidly increasing.
Solana has also faced some technical problems, including downtime, transaction failures, MEV, rapid state growth, and centralization, but the positive side of Solana is that it does not focus on redundant infrastructure construction, but relies more on the current TPS capacity to build consumer-oriented applications, and its roadmap also revolves around this. With the construction of more and more Layer 2 and the launch of clients, the TPS of the SVM ecosystem will be taken to the next level. Solana is still an oasis, with many ecological projects that have not been fully reached by capital, and there are many opportunities for entrepreneurs to explore.