How DeepSeek Cuts AI Costs: From Homegrown Tech to Desert Power

DeepSeek doesn’t rely on NVIDIA chips for inference. They also don’t solely use Western AI development frameworks like PyTorch and TensorFlow. This was a big shock to me as well, as I hadn’t paid too much attention to China’s software industry until recently. I’ve been developing software for several decades now, but the only software from China that I used was Filmora and face beautifying apps.

A Blue Whale Doll in My Hand — This doll was in my house for a long time. I don’t know where it came from.

The Magnificent 7 (Mag7) companies can utilize similar optimization techniques as DeepSeek and save on electricity in remote locations. However, they won’t be able to escape NVIDIA’s influence, since they all use critical AI development frameworks called PyTorch and TensorFlow, which are currently dependent on NVIDIA chips.

Why Do the AI Models of the Mag7 Companies Rely on NVIDIA?

Almost all Large Language Models (LLMs) use PyTorch (developed by Facebook) and TensorFlow (developed by Google) as development platforms or frameworks. These frameworks are specifically designed to handle the intensive vector and matrix calculations required for training and running large-scale neural networks. The developers of PyTorch and TensorFlow have been working with NVIDIA for a long time to customize CUDA, a middleware developed by NVIDIA.

CUDA facilitates orchestrating large-scale calculations across multiple dies, GPUs, and racks. Frameworks like PyTorch and TensorFlow, as well as CUDA, are highly specialized areas, and most AI developers don’t have the resources or time to learn how to develop these programming tools.

This is how the dependency on these programming tools and NVIDIA hardware has been established. It’s similar to how Boeing doesn’t build large jet engines but simply purchases them from either GE or Rolls-Royce. People are working on making these popular frameworks compatible with other AI chip producers, Broadcom being one example. However, this isn’t a task that can be achieved overnight.

Here’s more information on why it’s difficult to escape NVIDIA.

Does China Have Their Own Alternative AI Frameworks?

The answer is yes. People have been paying too much attention to OpenAI and NVIDIA for the past three years, and many of us didn’t know that Chinese companies such as Huawei and Baidu have been developing their own versions of PyTorch and TensorFlow for quite some time.

Meet PaddlePaddle (developed by Baidu) and MindSpore (developed by Huawei) which have become the leading AI frameworks in China. These frameworks are favored for their extensive toolsets, high performance, and strong support for various AI tasks like deep learning, natural language processing, computer vision, and more. [1]

Frameworks Used in China (Image by Author)

PaddlePaddle started in 2012 is not only compatible with Chinese AI chips, it’s compatible with Intel’s CPUs and NVIDIA’s GPUs. [2]

Compatibility of PaddlePaddle AI Framework from China (Image by Author)

Does DeepSeek Use Chinese AI Chips?

According to two online reports and a Korean newspaper article, DeepSeek is likely utilizing Huawei’s Ascend 910 series AI chips, especially for inference (the process of responding to an input, such as a prompt). [3][4][5] The Ascend 910B chip is known to provide 80% of the performance at 30% of the cost compared to NVIDIA’s A100.

For training (the process of learning language patterns or instructions), they initially indicated that their early data centers were equipped with approximately 10,000 NVIDIA A100 GPUs, which is considered a minimum for a top grade LLM training. They had been accumulating NVIDIA GPUs long before US export restrictions were imposed due to their parent company being an AI-based trading firm. DeepSeek stated in their technical report that they used 2,048 H800 GPUs to train the latest version of their DeepSeek AI model at a cost of only $6 million. The H800 is a tuned-down version of the H100, which was permitted for export during a specific period.

Some people in the US have argued that DeepSeek might have accumulated up to 50,000 H100 GPUs. However, these claims are unsubstantiated, especially given the price of each H100 ranging from $25,000 to $40,000. It’s very hard to get hold of high-end NVIDIA GPUs with or without the export ban. Even South Korea has only 1,961 H100 GPUs in the entire country as of 2023. The vast majority of H100 GPUs are in the US. Furthermore, many smaller LLM developers in the West have produced LLMs with less computing power than 2,048 H800 GPUs. Companies like Google have their own AI chips and don’t rely solely on NVIDIA.

Therefore, DeepSeek’s claim could be valid for just training their latest model for the incremental portion. However, a lot of their foundational works could have been processed by at least 10,000 A100 that they had for quite some time.

For training LLMs (learning language patterns or instructions), significantly more computing power is required than for inference (responding to a prompt). Therefore, for training, it’s difficult to find alternatives to high-end NVIDIA GPUs. However, for inference, many alternatives exist, and these can even be designed by startups.

This means that while DeepSeek may not have been able to avoid NVIDIA’s dominance in training their models, they could be using less expensive Huawei Ascend 910-type chips to serve tens of millions of users and process their prompts on their servers.

Is the AI Chip Export Ban Working?

I believe it is. These high-end GPUs are comparable to ammunition in AI battles. However, by having to use their local AI development frameworks and lower-power AI chips, they have apparently developed techniques that can run at 10 to 30 times lower cost. They already possess enough NVIDIA GPUs to train new models. They simply need to wait for Chinese chip designers to develop AI chips as powerful as NVIDIA’s. This is likely to take considerable time — at least five to ten years in my view — since even Samsung, the largest semiconductor manufacturer, cannot currently take orders for producing NVIDIA’s high-end GPUs due to technical challenges. Only TSMC in Taiwan can produce these high-end GPUs at this time.

So, yes, the export ban is definitely hindering China’s ability to catch up with the Mag7 companies and allies of the US. However, the Chinese government is providing generous subsidies for AI chip manufacturing, which further reduces costs. With strong government support, they may find new ways to work around their reliance on NVIDIA hardware in the future.

How Did DeepSeek Make Their AI Model So Lean?

Many LLM optimization techniques exist, such as quantization, distillation, and pruning. Smaller versions of GPT-4o and Gemini, like GPT-4o-mini and Gemini 2.0 Flash, have undergone these types of optimizations. I have written a separate article on LLM optimization here.

Some have argued that DeepSeek might be a distilled product of ChatGPT, implying they used synthetic datasets generated by the ChatGPT API. However, this is a common practice among many LLM developers. There are dozens of companies that specialize in producing synthetic datasets using ChatGPT APIs. With a projected annual growth rate of 31.1% from 2023, the synthetic data market is expected to reach $2.34 billion in value by 2030. [6][7]

What About Electricity?

Sources mention DeepSeek might be getting electricity from Inner Mongolia, which has large solar and wind farms, and Southeastern China, which has numerous hydropower plants. [5] Until 2020, 8% of all Bitcoins were mined in Inner Mongolia. [8] The cost of electricity there is approximately 70% lower than the US average.

Electricity Costs in China and the US (Image by Author)

Conclusion

License Policy of DeepSeek-R1 (source: Hugging Face)

DeepSeek-R1 is licensed under the MIT License. This means Apple, Samsung, Tesla, Microsoft, and even OpenAI can use it as a paid service on their devices and websites. It permits individuals or organizations to use, copy, modify, merge, publish, distribute, and sublicense the software, including for commercial use, provided that the original copyright notice and permission notice are included in all copies and substantial portions of the software.

Even if the motivation might have been to prevent front-runners from profiting too much, releasing such a significant investment to the public domain can be beneficial to many small organizations and economies. Additionally, saving energy is good for the whole world. It may even mitigate the energy crisis we were heading toward, which many so-called AI pioneers were contributing to. We are conditioned to replace our computer operating systems and phones every two years. This is a waste of money and energy. The Earth’s energy resources are finite, and depleting them with the expectation of moving to another planet may not be a sustainable solution.

However, we must remain cautious to ensure that no small group of people gains exclusive control over the greatest discovery since copper.

Like, subscribe, or share if you want to support content like this!

References

[1] https://blog.csdn.net/Ejzq1/article/details/124237225

[2] https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/hardware_support/hardware_info_cn.html

[3] https://www.financialexpress.com/life/technology-deepseeks-secret-weapon-huawei-chip-powers-viral-ai-3730146

[4] https://www.gsmarena.com/huawei_chips_are_powering_deepseek_ai-news-66297.php

[5] https://www.msn.com/ko-kr/money/%EA%B2%BD%EC%A0%9C/%EB%B9%84%EC%9A%A9-%EC%A0%88%EA%B0%90-%EB%B9%84%EA%B2%B0-%EB%94%A5%EC%8B%9C%ED%81%AC%EC%97%90-%EB%AC%BC%EC%96%B4%EB%B3%B4%EB%8B%88-%ED%99%94%EC%9B%A8%EC%9D%B4-%EC%B9%A9-%EC%A0%95%EB%B6%80-%EB%B3%B4%EC%A1%B0%EA%B8%88-%EB%93%B1-%EC%98%81%ED%96%A5/ar-AA1y2A6f

[6] https://www.mercity.ai/blog-post/using-chatgpt-to-build-synthetic-datasets

[7] https://aimresearch.co/ai-startups/top-8-synthetic-data-providing-startups-in-the-us

[8] https://fortune.com/2021/05/02/bitcoin-mining-hashrate-china-inner-mongolia-ban/