OpenAI and Broadcom Developing AI Chips ; Huawei and SMIC Expanding AI HW Capabilities; Explosive Demand for CoWoS Packaging

AI Brief Headlines — 10/31/2024

• OpenAI Partners with Broadcom to Develop Custom AI Chips, Plans to Use AMD Chips for Interim Support
• Huawei and SMIC Bypass U.S. Sanctions, Building China’s AI Hardware Capabilities
• Explosive Demand for CoWoS Packaging Driven by AI Accelerators Spurs Capacity Expansion at TSMC and Partners
• Samsung Focuses on High-Value Memory Products Amid Profit Drop in Semiconductor Division

Photo by BoliviaInteligente on Unsplash

OpenAI Partners with Broadcom to Develop Custom AI Chips, Plans to Use AMD Chips for Interim Support

OpenAI is reportedly collaborating with Broadcom to develop specialized silicon chips capable of handling its large-scale AI inference tasks. In partnership with Taiwan Semiconductor Manufacturing Company (TSMC), OpenAI aims to secure production capacity for these custom chips, though full-scale production might not commence until 2026. The company has reportedly assembled a team of around 20 chip designers, including former Google engineers who worked on Tensor processors. This move aligns OpenAI with tech giants like Microsoft, Meta, Google, and Nvidia, all racing to dominate AI chip design to meet growing demand.

In the interim, OpenAI is incorporating AMD’s MI300 chips into its Microsoft Azure infrastructure. AMD’s advancements in data center technology, particularly with the MI300, have helped it capture a larger market share and intensify competition with Nvidia. While OpenAI had previously explored establishing its own semiconductor foundries, Reuters reports that high costs and time constraints have shelved these plans. As OpenAI navigates the challenges of custom chip production, it may require additional funding to match the progress of industry leaders like Google, Amazon, and Microsoft, which are several generations ahead in AI hardware development.

[embed]OpenAI will start using AMD chips and could make its own AI hardware in 2026 OpenAI will reportedly make a custom AI inference chip with Broadcom and TSMC but without trying to create its own…www.theverge.com

Huawei and SMIC Bypass U.S. Sanctions, Building China’s AI Hardware Capabilities

Despite stringent U.S. restrictions on high-performance AI hardware exports to China, Chinese firms are amassing sufficient resources to build powerful AI clusters. SemiAnalysis reports that Huawei, heavily sanctioned and limited in its access to international tech markets, has bypassed sanctions through proxy channels and domestic production of its Ascend 910 processors. Additionally, Nvidia’s high-performance GPUs, like the H100 and H200, continue to reach China through smuggling from countries like India and Malaysia, with Nvidia also legally supplying cut-down models such as the DGX H20 directly to China. SemiAnalysis projects that Nvidia will officially supply 900,000 HGX H20 processors to China in 2024, with even more sales expected in subsequent years, underscoring China’s growing AI processing power.

In collaboration with SMIC, Huawei has strategically established a domestic semiconductor network capable of producing crucial AI components, circumventing U.S. sanctions with a ‘wafer bridge’ system that links restricted and unrestricted fabs. This allows Huawei to produce chips like the Ascend 910B and Kirin 9000S on advanced nodes without breaching export controls. With plans to invest $7.3 billion in foreign wafer fabrication tools in 2024, Huawei is accelerating China’s self-sufficiency in AI hardware. U.S. analysts and policymakers are reconsidering export control mechanisms to address these loopholes, with proposals to tighten the Foreign Direct Product Rule. However, pushback from U.S. equipment suppliers highlights the economic impact of these restrictions on American businesses, balancing national security concerns against the economic implications for U.S. tech leadership.

[embed]Fab Whack-A-Mole: Chinese Companies are Evading U.S. Sanctions Huawei Fab Network, WFE Vendors Cry Wolf, Framework for Future Controlswww.semianalysis.com

Explosive Demand for CoWoS Packaging Driven by AI Accelerators Spurs Capacity Expansion at TSMC and Partners

The global demand for Chip-on-Wafer-on-Substrate (CoWoS) and similar advanced packaging technologies is projected to soar by 113% annually in 2025, fueled primarily by the growing need for cloud AI accelerators, as reported by DIGITIMES Research. Major packaging suppliers like TSMC, ASE Technology Holding, and Amkor are aggressively expanding their production capabilities to keep up. By late 2025, TSMC’s monthly output capacity is expected to reach over 65,000 12-inch wafers, while Amkor and ASE together will contribute 17,000 wafers per month. The expansion aligns with increased high-end GPU orders, particularly from Nvidia, which relies on TSMC’s CoWoS process to manufacture its powerful Blackwell GPUs. DIGITIMES also notes that other ASIC suppliers like Broadcom and Marvell, serving cloud giants Google and Amazon, are ramping up wafer orders to meet the demands of advanced AI applications.

TSMC’s CoWoS process will transition from the current CoWoS-Short (CoWoS-S) to the more advanced CoWoS-Long (CoWoS-L) process in the fourth quarter of 2025, accommodating the mass production of Nvidia’s Blackwell series GPUs. Nvidia’s demand for CoWoS-L is projected to skyrocket, with estimated orders increasing from 32,000 wafers in 2024 to 380,000 in 2025 — a remarkable 1,018% growth. Consequently, by Q4 2025, CoWoS-L will represent 54.6% of TSMC’s CoWoS output, followed by CoWoS-S at 38.5% and CoWoS-R at 6.9%. Despite concerns about cloud service providers’ (CSPs) return on AI investments, DIGITIMES Research anticipates that CSPs will continue their intensive AI spending, sustaining the strong demand for 2.5D advanced packaging technology worldwide. This ramp-up highlights the pivotal role of advanced packaging in supporting the burgeoning AI hardware ecosystem.

[embed]Global CoWoS and CoWoS-like packaging capacity demand to surge 113% on year in 2025, says DIGITIMES… Global demand for Chip-on-Wafer-on-Substrate (CoWoS) and CoWoS-like packaging capacity will likely grow by 113%…www.digitimes.com

Samsung Focuses on High-Value Memory Products Amid Profit Drop in Semiconductor Division

Samsung Electronics reported a significant quarter-over-quarter decline in operating profit within its Device Solutions (DS) Division, which oversees semiconductors and displays. While DS revenue rose impressively by 78% year-over-year to ₩29.27 trillion (around $21 billion), it showed only a modest 3% increase sequentially. The division’s operating profit saw a sharp dip of 40% from the previous quarter, falling to ₩3.86 trillion ($2.8 billion), impacted by “one-off costs” and currency fluctuations. The challenges were compounded by delays in supplying high-bandwidth memory (HBM) products to key clients like Nvidia, reportedly due to quality concerns, though Samsung denies this. Despite the difficulties, Samsung highlighted that HBM sales had grown by over 70% quarter-on-quarter, with plans to focus on these and other high-value products for the remainder of 2024.

Looking forward, Samsung is concentrating on strengthening its advanced technology capabilities, aiming to increase order volumes in its Foundry Business and focusing on HBM and high-density memory products. The DS Division plans to leverage its 2nm Gate-All-Around (GAA) process for new client acquisitions by 2025, capitalizing on the growing AI-driven demand for high-performance chips. However, signs of cooling demand in the semiconductor sector may dampen these ambitions. Recent data from South Korea’s statistical office indicates a decline in the nation’s semiconductor output for the first time in over a year, suggesting that the AI-driven chip boom may be nearing its peak. Samsung remains committed to its high-value strategy to navigate these challenges, but the industry’s long-term growth is under watch as market conditions evolve.

[embed]Samsung blames 'one-off costs' as Q3 chip profits plummet Unexpected expenses in semiconductor division overshadow revenue gainswww.theregister.com