a16z disciple Kuzco's practical guide 2: From single-soldier combat to cluster deployment

Author: J1N, Techub News

Introduction: Epoch One to Two

Kuzco is a network dedicated to LLM large language model computing power mining. This year, it was selected for the Crypto Startup Accelerator (CSX) Fall Accelerator Program launched by a16z in New York on September 9. Projects selected by the program will receive at least $500,000 in investment from a16z and will receive guidance and support from the a16z operations team. The accelerator program has now ended.

On November 16, Kuzco announced that the first phase (Epoch One) incentive plan will end on November 18, 2024, all operations will be suspended, data snapshots will be permanently stored, and the final points ranking will be announced on the new leaderboard.

Officially disclosed, Epoch One will be launched on March 6, 2024, with a peak number of more than 8,000 devices. The network runs the 8B-specification Llama-3 AI large language model released by Meta, with a total inference of more than 1 trillion tokens.

It was also announced that the financing information and project development roadmap will be released in the next few weeks, and the second phase (Epoch Two) incentive plan will be launched on December 9. Epoch Two will bring some new features, such as higher throughput and reliability of NVIDIA hardware; encourage users to access top computing devices such as A100 and H100; support more image generation and multimodal language model VLM.

There is still half a month of preparation time before the opening of Epoch Two. This article will discuss:

• Share the practice and results of personal mining, and the transition from single machine to cluster.

• Show the whole process of obtaining financing through research and practice, and building high-specification machines.

• Discuss the matching of hardware configuration and project requirements, and answer investors' common questions.

Epoch One Review: Single Soldier Combat

Configuration

My configuration list includes RTX series graphics cards 2060, 2070S, 3080, 4060, 4060Ti, as well as 4 4070S and 2 Apple M2 and M3 devices. These devices are distributed on several hosts, laptops and a dedicated mining machine.

Cost

It is worth mentioning that these graphics cards were originally purchased by the author every year according to game needs, not specifically for mining. Therefore, the cost of hardware purchase is not included in the cost calculation, only the actual electricity cost of the mining machine is counted. Here we take the mining machine assembled in the first article "a16z "disciple" Kuzco's practical guide: How to efficiently perform AI computing power mining?" as an example.

Configuration of the mining machine:

• Motherboard: z490 (later replaced with industrial board)

• CPU: 10th generation I9

• Graphics card: 2060, 2070s, 3080, 4060ti, 4070s

Hand-held mining machine

The picture below is the mining machine The total electricity consumption in October and November was 564 kWh, and the points (KZO Points) were about 600 million. The total of all machines is about 1.1 billion points. The specific electricity cost needs to be calculated according to the electricity cost in your location, and this is only for reference.

Picture far right, a total of 1 billion points

Preparation for Epoch Two: Cluster deployment

From single-soldier deployment to cluster deployment

Configuration and selection logic of high-specification machines

Combined with the author's practical experience in Epoch One, the motherboard, CPU, graphics card, power supply, platform and network configuration were fully optimized, and a more suitable hardware combination was selected, which not only improved the overall operation stability, security and efficiency, but also paid more attention to the liquidity of the second-hand market in hardware selection. This strategy can effectively reduce the actual investment cost and provide subsequent participants with a more cost-effective option.

Motherboard

The author chose an industrial motherboard instead of the mainstream B85, mainly based on the comprehensive consideration of performance, stability and cost performance.

In terms of performance, running Kuzco's Llama-3 model requires starting multiple Docker processes, and running these processes in parallel will take up a lot of CPU resources, requiring high CPU performance, and the CPU compatible with B85 cannot meet this requirement.

In addition, industrial motherboards have obvious advantages in long-term stable operation, high temperature resistance and manufacturer warranty, and are more liquid in the second-hand market, so they are undoubtedly the best choice.

Graphics card

The author chose to use 4070S as the main graphics card, mainly based on the following points:

Advantages in AI computing performance: Compared with 30 series graphics cards, the performance improvement of 40 series graphics cards in AI computing is far greater than the improvement in game performance. The core reason is that AI computing power mainly depends on the number of CUDA cores of the graphics card, and the CUDA cores of the 40 series graphics card are significantly more than those of the 30 series graphics card.

Energy efficiency advantage: The author conducted detailed tests on multiple GPUs and calculated the average power consumption of each Tokens

• 4060Ti (160W): 0.125 Tokens/W

• 3080 (330W): 0.22 Tokens/W

• 4090 (450W): 0.26 Tokens/W

• 4070S (220W): 0.38 Tokens/W

From the test results, 4070S It performs best in balancing performance and power consumption. Its higher energy efficiency directly reduces electricity costs, making it the most cost-effective choice.

Price and liquidity in the second-hand market: As a mid-to-high-end graphics card, the 4070S has high liquidity and value preservation in the second-hand market, which further reduces the cost of holding the device and provides flexibility for subsequent hardware upgrades.

CPU

As mentioned above, Kuzco's Llama-3 needs to start multiple Dockers when running, which occupies a significant amount of CPU resources, especially when multiple cards are running, the CPU occupancy rate may be as high as 80%-90%. Therefore, multi-core and multi-threaded processing capabilities are particularly important. A high-performance, multi-threaded, and stable CPU can not only effectively support multi-tasking operations, but also ensure the stability and efficiency of the entire mining process.

The 13th generation i5 can reach 70%+ occupancy when running the graphics card at full load

Network environment

The soft router is the square box in the picture

The network environment is also crucial in mining. Even if a high-performance graphics card is configured, if the network is not optimized, the computing power will be seriously affected. According to the author's actual measurement, insufficient network speed may cause the computing power to drop to 30%, and low-quality network nodes may directly lead to the inability to connect to the Kuzco network. Both of these are unacceptable for mining. In order to solve these problems, the author adopts a soft routing solution, which is not only easy to configure, but also can run efficiently without human intervention after the setting is completed. In theory, it can also support the access of unlimited devices. As for the specific operation method, it is recommended that readers consult relevant information according to their needs.

Power supply

Classic Great Wall 2000w nuclear bomb power supply

When choosing a power supply, you need to pay special attention to the peak power consumption. This is why even though the rated power consumption of 7 4070S is only 1540W, the author still chooses to use dual 2000W power supplies, with a total power of 4000W. This is not a waste of resources, but out of consideration for the stability and safety of equipment operation.

Graphics cards will experience peak power consumption during operation, that is, at certain moments, their actual power consumption may reach 1.5 times or even more of the rated power consumption, and then fall back to normal levels. If the power supply is not enough to cope with this peak, it may trigger the forced shutdown mechanism of the power supply, or even cause damage to the graphics card. This is a fatal threat to the normal operation of the mining machine.

4070s operating power consumption performance

Take 4070S as an example. Although its rated power consumption is 220W, its peak power consumption may exceed 400W. The peak power consumption of 7 graphics cards may reach more than 3000W, so dual 2000W power supplies are configured to ensure stable operation of the machine. Users who configure multiple 4090s should pay special attention to the fact that the rated power consumption of a single 4090 is 450W, while the peak power consumption may be as high as 770W. In the case of multiple cards, two power supplies alone may not be able to meet the needs, and three power supplies are usually required to ensure system stability.

4090 running power consumption performance

Supplement

As for BIOS settings, hardware compatibility, and remote management, the author will not elaborate too much here. There are a lot of free tutorials on the Internet for reference. Most problems can be solved by following the tutorials. It is recommended to conduct targeted reference and processing according to your own hardware configuration and needs, which is simple and efficient.

Risks and benefits

Answer the question that everyone is most concerned about: How much money can be mined every day? Frankly speaking, there is no clear answer to this question, because risks and benefits always coexist. I can share a clear point of view: whether it is the cryptocurrency circle or the traditional industry, if any project can accurately calculate the daily income, then you may not make a lot of money. Unless you have some monopoly resources, such as extremely low electricity costs or very cheap mining equipment, you can have an advantage in income. However, not everyone has such resources.

The author chooses equipment with good liquidity precisely to reduce investment risks and cost pressure. Taking Kuzco mining as an example, the cost is mainly concentrated on the depreciation of hardware and electricity costs, so your maximum loss is limited to these fixed costs. If you do not participate under the premise of low cost, then any investment decision loses its meaning. It should be emphasized that the characteristics of mining the first mine determine that there is no clear expectation of income, but this is also the potential of the first mine.

From a subjective judgment, this track has huge market prospects: on the one hand, Kuzco has obtained investment support from a16z; on the other hand, the demand for LLM large language models is rapidly expanding. Think about it, almost no one will not use LLM, right? Platforms like OpenAI's ChatGPT, Meta's Llama, and Musk's XAI have raised rounds of high-value financing, which clearly shows the growth potential of this industry.

For ordinary people, it is not easy to directly participate in the AI ​​industry. On the one hand, the threshold of AI technology is high; on the other hand, the training of AI models requires a huge amount of resources and funds, and most people can hardly afford such costs. By joining the AI ​​computing power network through Kuzco, ordinary people can easily participate in this high-growth field, contribute to AI computing power, and gain benefits at a controllable cost.

In addition, the price of Bitcoin is about to break through $100,000. There is a huge risk of retracement behind the rise from $16,000 in 2022 to the current high point. If you choose to directly purchase the tokens of AI projects, you will also face similar high volatility risks. In contrast, participating in the AI ​​computing network is a more robust option: not only is the cost clearly controllable, but it also allows you to enter the high-growth track of the AI ​​industry with relatively low risk. This is one of the practical ways for ordinary people to enter the AI ​​field in the current environment.