Detailed description of TON’s technical features and smart contract development paradigm

Author:@Web3Mario

Introduction

With Binance launching Notcoin, the largest game in the TON ecosystem, and the huge wealth effect caused by the fully circulated token economic model, TON has gained great attention in a short period of time. After chatting with a friend, I learned that the technical threshold of TON is relatively high, and the DApp development paradigm is very different from the mainstream public chain protocol. Therefore, I spent some time to study the relevant topics in depth, and I have some experience to share with you. In short, the core design concept of TON is to reconstruct the traditional blockchain protocol in a "bottom-up" way, and to achieve the ultimate pursuit of high concurrency and high scalability at the cost of abandoning interoperability.

The core design concept of TON - high concurrency and high scalability

It can be said that the purpose of all the complex technical selections in TON comes from the pursuit of high concurrency and high scalability. Of course, it is not difficult for us to understand this from the background of its birth. TON, or The Open Network, is a decentralized computing network consisting of a L1 blockchain and multiple components. Originally developed by Telegram founder Nikolai Durov and his team, TON is now supported and maintained by a community of independent contributors around the world. It all began in 2017 when the Telegram team began exploring blockchain solutions for themselves. Since no existing L1 blockchains were able to support Telegram’s nine-digit user base at the time, they decided to design their own blockchain, then called Telegram Open Network. In 2018, in order to obtain the resources needed to realize TON, Telegram launched a sale of Gram tokens (later renamed Toncoin) in the first quarter of 2018. In 2020, the Telegram team withdrew from the TON project due to regulatory issues. Subsequently, a small group of open source developers and Telegram competition winners took over the TON codebase, renamed the project The Open Network, and continue to actively develop the blockchain to this day, following the principles outlined in the original TON white paper.

Since the design goal is to be a decentralized execution environment for Telegram, it naturally faces two problems: high concurrent requests and massive data. As we know, with the development of technology, Solana, which claims to have the highest TPS, has a measured maximum TPS of only 65,000, which is obviously not enough to support the Telegram ecosystem with a million-level TPS requirement. At the same time, with the large-scale application of Telegram, the amount of data it generates has already broken through the sky, and as an extremely redundant distributed system, blockchain requires that each node in the network save a complete copy of the data, which is also unrealistic.

Therefore, in order to solve the above two problems, TON has made two optimizations to the mainstream blockchain protocol:

• By adopting the "Infinite Sharding Paradigm" to design the system, the data redundancy problem is solved, so that it can carry big data and alleviate the performance bottleneck problem;

• By introducing a fully parallel execution environment based on the Actor model, the network TPS is greatly improved;

Be a blockchain chain - through infinite sharding capabilities, each account has an exclusive account chain

Now we know that sharding has become the mainstream solution for most blockchain protocols to improve performance and reduce costs, and TON has taken this to the extreme and proposed an infinite sharding paradigm. The so-called infinite sharding paradigm refers to allowing the blockchain to dynamically increase or decrease the number of shards according to the network load. This paradigm enables TON to handle large-scale transactions and smart contract operations while maintaining high performance. In theory, TON can establish an exclusive account chain for each account and ensure the consistency between these chains through certain rules.

Abstractly, there are four layers of chain structure in TON:

• AccountChain: This layer of chain represents a chain composed of a series of transactions related to a certain account. The reason why transactions can form a chain structure is that for a state machine, as long as the execution rules are consistent, the state machine will get the same result after receiving the same order of instructions. Therefore, all blockchain distributed systems need to sort transactions in chains, and TON is no exception. The account chain is the most basic component unit in the TON network. Usually, the account chain is a virtual concept, and it is unlikely that an independent account chain will actually exist.

• ShardChain: In most contexts, the shard chain is the actual component unit of TON. The so-called shard chain is a collection of account chains.

• WorkChain: It can also be called a set of shard chains with custom rules, such as creating an EVM-based work chain and running Solidity smart contracts on it. In theory, everyone in the community can create their own work chain. In practice, building it is a rather complex task, and before that you have to pay the (expensive) fee to create it and get 2/3 of the votes of the validators to approve the creation of your work chain.

• MasterChain: Finally, there is a special chain in TON called the master chain, which is responsible for bringing finality to all shard chains. Once the hash value of a shard chain's block is merged into the main chain's block, the shard chain block and all its parent blocks are considered final, which means that they can be considered fixed and immutable content and referenced by subsequent blocks of all shard chains.

By adopting such a paradigm, the TON network has the following three characteristics:

• Dynamic Sharding: TON can automatically split and merge shard chains to adapt to changes in load. This means that new blocks are always generated quickly, and transactions do not incur long waiting times.

• Highly Scalable:Through the infinite sharding paradigm, TON is able to support an almost unlimited number of shards, theoretically up to 2 to the power of 60 working chains.

• Adaptability: When the load on a part of the network increases, that part can be subdivided into more shards to handle the increased transaction volume. Conversely, when the load decreases, shards can be merged to improve efficiency.

Then such a multi-chain system first needs to face the problem of cross-chain communication, especially because of the ability of unlimited sharding. When the number of shards in the network reaches a certain level, information routing between chains will become a difficult task. Imagine that there are 4 nodes in the network, each of which is responsible for maintaining an independent work chain. The link relationship indicates that in addition to being responsible for the transaction sorting work in its own work chain, the node also needs to monitor and process the state changes in the target chain. In TON, this is achieved by monitoring the messages in the output queue.

Suppose that account A in work chain 1 wants to send a message to account C in work chain 3. Then it is necessary to design the message routing problem. In this example, there are two routing paths, work chain 1 -> work chain 2 -> work chain 3, and work chain 1 -> work chain 4 -> work chain 3.

When faced with more complex situations, an efficient and low-cost routing algorithm is needed to quickly complete message communication. TON chose the so-called "hypercube routing algorithm" to implement cross-chain message communication routing discovery. The so-called hypercube structure refers to a special network topology. An n-dimensional hypercube is composed of 2^n vertices, each of which can be uniquely identified by an n-bit binary number. In this structure, any two vertices are adjacent if they differ by only one bit in the binary representation. For example, in a 3-dimensional hypercube, vertex 000 and vertex 001 are adjacent because they differ only in the last bit. The above example is a 2-dimensional hypercube.

In the hypercube routing protocol, the routing process of messages from the source work chain to the target work chain is carried out by comparing the binary representation of the source work chain and the target work chain address. The routing algorithm finds the minimum distance between the two addresses (that is, the number of different bits in the binary representation) and forwards the information step by step through the adjacent work chains until it reaches the target work chain. This method ensures that data packets are transmitted along the shortest path, thereby improving the communication efficiency of the network.

Of course, in order to simplify this process, TON also proposed an optimistic technical solution. When the user can provide a valid proof of a certain routing path, which is usually a merkle trie root, the node can directly recognize the credibility of the message submitted by the user, which is also called instant hypercube routing.

Therefore, we can see that the address in TON is significantly different from other blockchain protocols. Most other mainstream blockchain protocols use the hash corresponding to the public key in the public and private keys generated by the elliptic encryption algorithm as the address, because the address is only used for uniqueness distinction, and does not need to carry the function of routing addressing. The address in TON consists of two parts, (workchain_id, account_id), where workchain_id is encoded according to the hypercube routing algorithm address, which will not be elaborated here.

There is another point that is easy to doubt. You may have discovered that the main chain and each work chain are linked. Then all cross-chain information can be relayed through the main chain, just like cosmos. In the design concept of TON, the main chain is only used to handle the most critical tasks, that is, to maintain the finality of many working chains. It is not impossible to route messages through the main chain, but the handling fees generated will be very expensive.

Finally, let's briefly mention its consensus algorithm. TON adopts the BFT+PoS method, that is, any staker has the opportunity to participate in block packaging. TON's election governance contract will randomly select a packaged validator cluster from all Stakers at regular intervals. The nodes selected as validators will package blocks through the BFT algorithm. If the wrong information is packaged or malicious, the tokens of its stake will be confiscated, otherwise they will receive block rewards. This is basically a common choice, so I will not introduce it here.

Smart contracts and fully parallel execution environment based on the Actor model

Another difference between TON and mainstream blockchain protocols is its smart contract execution environment. In order to break through the TPS limitations of mainstream blockchain protocols, TON adopts a bottom-up design approach and reconstructs smart contracts and their execution methods using the Actor model, enabling it to have the ability to execute in full parallel.

We know that most mainstream blockchain protocols use a single-threaded serial execution environment. Taking Ethereum as an example, its execution environment EVM is a state machine that takes transactions as input. When the block-producing node completes the sorting of transactions by packaging blocks, it will execute transactions through EVM in this order. The whole process is completely serial and single-threaded, that is, only one transaction can be executed at a time. The advantage of this is that as long as the transaction order is confirmed, the execution result is consistent in a wide distributed cluster. At the same time, since only one transaction is executed serially at the same time, it means that during the execution process, it is impossible for other transactions to modify a state data to be accessed, thus achieving interoperability between smart contracts. For example, we use USDT to buy ETH through Uniswap. When the transaction is executed, the distribution of LPs in the transaction pair is a certain value, so the corresponding results can be obtained through certain mathematical models. However, if this is not the case, when executing a bonding curve calculation, other LPs add new liquidity, then the calculation result will be an outdated result, which is obviously unacceptable.

However, this architecture also has obvious limitations, that is, the bottleneck of TPS, which seems very old under the current multi-core processor, just like you use the latest PC to play some old computer games, such as Red Alert. When the number of combat units reaches a certain level, you will still find that it is stuck. This is a problem with the software architecture.

You may hear that some protocols have already paid attention to this problem and proposed their own parallel solutions. For example, Solana, which is currently known to have the highest TPS, also has the ability to execute in parallel. However, its design idea is different from TON. In Solana, its core idea is to divide all transactions into several groups according to execution dependencies, and no state data is shared between different groups. That is, there is no same dependency, so that transactions in different groups can be executed in parallel without worrying about conflicts, while for transactions in the same group, the traditional serial method is still used.

In TON, it completely abandons the serial execution architecture and instead adopts a development paradigm designed for parallelism, the Actor model, to reconstruct the execution environment. The so-called Actor model was first proposed by Carl Hewitt in 1973. Its purpose is to solve the complexity of shared state in traditional concurrent programs through message passing. Each Actor has its own private state and behavior, and does not share any state information with other Actors. The Actor model is a computing model for concurrent computing, which implements parallel computing through message passing. In this model, "Actor" is the basic unit of work, which can process received messages, create new Actors, send more messages, and decide how to respond to the next message. The Actor model needs to have the following characteristics:

• Encapsulation and independence: Each Actor is completely independent when processing messages, and can process messages in parallel without interfering with each other.

• Message passing: Actors interact only by sending and receiving messages, and message passing is asynchronous.

• Dynamic structure: Actors can create more Actors at runtime. This dynamism enables the Actor model to expand the system as needed.

TON uses this architecture to design the smart contract model, which means that in TON, each smart contract is an Actor model with completely independent storage space. Because it does not rely on any external data. In addition, calls to the same smart contract are still executed according to the order of messages in the receiving queue, so transactions in TON can be efficiently executed in parallel without worrying about conflicts.

However, such a design also brings some new impacts. For DApp developers, their accustomed development paradigm will be broken, as follows:

1.Asynchronous calls between smart contracts: It is impossible to atomically call external contracts or access external contract data within TON's smart contracts. We know that in Solidity, the function1 of contract A calls the function2 of contract B, or accesses a certain state data through the read-only function3 of contract C. The whole process is atomic and executed in one transaction. This is a very easy thing. However, in TON, this will not be possible. Any interaction with external smart contracts will be executed asynchronously by packaging new transactions. This transaction initiated by smart contracts is also called internal message. And the execution process cannot be blocked to obtain the execution result.

For example, if we develop a DEX, if we adopt the common paradigm in EVM, there will usually be a unified router contract to manage transaction routing, and each Pool will manage the LP data related to a certain transaction pair separately. Then assume that there are currently two pools USDT-DAI and DAI-ETH. When a user wants to buy ETH directly with USDT, he can request these two pools sequentially in one transaction through the router contract to complete the atomic transaction. However, it is not so easy to achieve in TON. We need to think about a new development paradigm. If we still reuse this paradigm, the information flow may be like this. This request will be accompanied by an external message initiated by the user and three internal messages (note that this is used to illustrate the difference. In real development, even the ERC20 paradigm needs to be redesigned).

2.It is necessary to carefully consider the processing flow of execution errors when calling across contracts, and design corresponding bounce functions for each inter-contract call. We know that in the mainstream EVM, when a problem occurs during transaction execution, the entire transaction will be rolled back, that is, reset to the state at the beginning of execution. This is easy to understand in the serial single-threaded model. However, in TON, since the inter-contract calls are executed asynchronously, even if an error occurs in a subsequent link, since the previously successfully executed transaction has been executed and confirmed, this may cause problems. Therefore, a special message type is set in TON, called a bounce message, that is, when an error occurs in the subsequent execution process triggered by an internal message, the triggered contract can trigger the contract to reset certain states by triggering the bounce function reserved by the contract.

3.In some complex cases, the transaction received first may not be executed first, so this timing relationship cannot be preset. In such a system of asynchronous and parallel smart contract calls, it may be difficult to define the order of processing operations. This is why each message in TON has its logical time Lamport time (hereinafter referred to as lt). It is used to understand which event triggers another and what the verifier needs to process first. For a simple model, the transaction received first must be executed first.

In this model, A and B represent two smart contracts respectively, and there is a timing relationship of if msg1_lt < msg2_lt, then tx1_lt < tx2_lt.

However, in more complex cases, this rule will be broken. There is such an example in the official document. Suppose we have three contracts A, B and C. In a transaction, A sends two internal messages msg1 and msg2: one to B and the other to C. Although they are created in the exact order (msg1 first, then msg2), we cannot be sure that msg1 will be processed before msg2. This is because the routes from A to B and from A to C may differ in length and validator set. If these contracts are in different shard chains, one of the messages may take several blocks to reach the target contract. That is, we have two possible transaction paths, as shown in the figure.

4.In TON, the persistent storage of its smart contracts uses a directed acyclic graph with Cell as the unit as the data structure. The data will be compactly compressed into a Cell according to the encoding rules, and at the same time extend downward in the form of a directed acyclic graph.This is different from the structure organization of state data based on hashmap in EVM. Due to the different data request algorithms, TON sets different Gas prices for data processing at different depths. The deeper the Cell data processing, the higher the Gas required. Therefore, there is a DOS attack paradigm in TON, that is, some malicious users occupy all shallow Cells in a smart contract by sending a large number of spam messages, which means that the storage cost of honest users will become higher and higher. In EVM, since the query complexity of hashmap is o(1), there is the same Gas and there will be no similar problems. Therefore, TON Dapp developers should try to avoid unbounded data types in smart contracts. When unbounded data types appear, they should be broken up by sharding.

5. Some features are not so special, such as smart contracts need to pay rent for storage, smart contracts in TON are naturally upgradeable, and native abstract account functions, that is, all wallet addresses in TON are smart contracts, but they are not initialized, etc., which developers need to pay attention to.