AI Rack Teardown: Why AI Chips Are Causing Global Semiconductor Shortages & Price Hikes

— 4,500 Chips per AI Rack; Over 45 Million Chips in a 10,000-Rack Data Center

Recently, Deloitte and the Semiconductor Industry Association (SIA) jointly released a comprehensive report. Utilizing a “virtual teardown” methodology, they meticulously deconstructed a top-tier AI rack system from the inside out.

 Key Takeaways from This Article:

    1.  How many chips are installed in a single AI rack, and how is the Capital Expenditure (CapEx) distributed?

    2. Which specific semiconductor components are embedded across the key subsystems torn down in the report?

I

Which Chips Hold the Highest Value vs. Highest Quantity in an AI Rack?

This report systematically deconstructs a top-tier AI rack layer by layer—spanning five distinct structural tiers: Silicon Die, Packaged Chip, Board/Card, Tray, and Rack System—to reveal its underlying semiconductor architecture.

According to the report, a rack consists of five functional tray types: Compute Trays, Power Trays, Networking & IPMI Trays, Accelerator Interconnect Trays, and CDU Trays.

Compute Tray：Core Processing

Power Tray：Power Delivery

Networking & IPMI Tray：O&M Management

Switch/Interconnect Tray：High-Speed Accelerator Interconnect

CDU Tray (Cooling Distribution Unit Tray：Liquid Cooling Distribution

1. Value Distribution

Looking at economic value first: semiconductors account for over 95% of the total system value of a high-end AI rack. This value is heavily concentrated in the Compute Trays, specifically within the AI accelerators. AI accelerators alone represent ~74% of the rack’s total chip value, with individual units priced between $10,000 and $40,000.

 An AI accelerator is a heterogeneously integrated assembly: 1–2 leading-edge logic dies (≤7nm node), paired with 6–12 High Bandwidth Memory (HBM) stacks (stacking up to 16 DRAM dies each), integrated via high-speed interconnect interfaces and interposers using 2.5D/3D advanced packaging.

According to the breakdown, ~70% of an accelerator’s value lies in logic chips and ~20% in memory. HBM alone—calculated at ~$30 to $50 per GB—demands $400,000 to $600,000 worth of memory per rack.

 This implies that the primary value driver combines three critical semiconductor bottlenecks: leading-edge logic, HBM, and advanced packaging—all monopolized by a few key IDMs and foundries.

By unit price tier: chips under $10 account for ~52% of total volume; $10–$100 chips represent ~29%; $100–$1,000 chips make up ~16%; while chips above $1,000 account for only ~3%.

2. Quantity & Unit Price Breakdown

Looking at the specific prices and quantities:

A single cabinet has about 4,500 packaged chips.

When broken down into bare dies, that’s about 20,000 silicon chips.

A 10,000-rack data center has over 45 million chips.

If we break it down by the price per unit:

<$10: about 52% (mainly power supplies, analog, interfaces, sensors, MCUs)  

$10–$100: about 29%  

$100–$1,000: about 16%  

>$1,000: only 3% (mainly AI accelerators/CPUs/switch ASICs)

Conclusion: System value is driven by the top 3% of ultra-high-cost chips, whereas total volume is dominated by sub-$10 peripheral ICs. While low in unit price, a shortage in any single sub-component can halt entire system manufacturing.

You can see that in an AI cabinet, the most valuable chips and the chips in the greatest quantity are not the same ones.

However, these low-cost power, analog, and sensor chips, while not adding much value, are essential for the system to run smoothly. A shortage of any of them could bring the whole assembly to a halt.

II

Which Chips Are Used in Key Subsystems,and How Do They Squeeze Other Industries?

Chip Volume by Tray Type：

Compute Tray：4,000 Chips

Power Tray：600 Chips

Networking & IPMI Tray：100 Chips

Interconnect Tray：70 Chips

CCDU Tray：10 Chips

 These tens of thousands of chips, the vast majority aren’t expensive accelerator-type chips, but various supporting logic, memory, power, analog, and sensor components. They are scattered across every tray and every part.

The report took apart some key components and listed exactly which chips were used inside:

Compute Board : 1–2 AI Accelerators, 1–2 CPUs, 8–32 DIMMs, 50–100+ VRMs, 10+ Control/Support ICs, 1+ Network Interconnect IC.

AI Accelerator (GPU)：1–2 Logic ASICs, 1+ Interconnect PHY, 6–12 HBM Stacks (96–192 dies), 1 Co-packaged Interposer/Bridge Die.

RAM stick (DIMM)： 8–16 DRAM Chips per DIMM, 1 RCD, 1 PMIC, 1 SPD EEPROM, 1 Temp Sensor.

NVMe solid-state drive (SSD)：256 NAND Flash Dies, 1 SSD Controller, 1 DRAM Cache, 1 I/O Interface IC, 1 PMIC.

Data Processing Unit (DPU)： 1 Embedded Logic Die, 1 Packet Engine, 1–2 On-chip DRAMs, 1 PHY Interface, 1 Flash, 6–8 PMICs/VRMs.

High-performance network card (NIC)：1 Network Controller ASIC, 1 Optical Transceiver, 4–8 SerDes/PHYs, 1 DDR Module, 1 PCIe Interface, 2–4 PMIC/VRMs, 1 DSP, 1 Oscillator, 1–2 Clock Buffers, 1 Clock Generator.

Typical power distribution unit in an AI server compute tray:2–6 IBCs, 6–12 PoL Converters, 1–2 MCU/PMBus SoCs, 2–6 PMICs, 5–10 Current/Voltage Sensors, 2–4 Drivers & FETs.

Accelerator interconnect tray:2 High-Speed PHYs, 2 Switch ASICs/tray, 16–72+ SerDes Transceivers/Switch, 1–2 ACC Chips, 4–6 PMICs/VRMs, 1–2 Clock Generators/PLLs, 1 Oscillator, 1–2 Clock Buffers.

Server-grade power supply unit (PSU)：1 Bridge Rectifier, 1 PFC Controller, 2–4 GaN Power MOSFETs, 1 PWM Controller, 2–4 Gate Drivers, 2–4 Synchronous Rectifier FETs, 1 MCU/PMBus SoC, 3–5 Voltage/Temp Sensors.

AI server rack BMC: out-of-band switch: in-band switch: CDU liquid cooling unit:

AI server rack BMC: 1 SoC MCU, 1 Firmware Flash, 1 Ethernet PHY with Magnetics, 1–2 PMICs, 1 Watchdog Timer, 1 Clock Generator.

Out-of-band switch: 1 Switch ASIC, 24 Ethernet PHYs, 1 Management SoC/CPU, 1 Flash, 1 DRAM, 1 PMIC.

In-band switch: 1 Switch ASIC, 100+ SerDes Transceivers, 1 Ethernet/InfiniBand PHY, 72 Optical Transceivers, 1 DSP, 10 PMICs, 1 Fan/Thermal Control MCU, 1 Flash, 1 DRAM.

CDU liquid cooling unit:3 Flow Sensors, 4 Pressure Sensors, 4 Temp Sensors, 1 Controller MCU, 1 Ethernet/PMBus Interface.

 Besides this teardown, the report also points out a few directions of change, and from these changes, we can predict possible future trends.

First is power supply. Rack power is shifting from the current 48V DC to 400V, and even eventually 800V high-voltage DC. The power consumption of a single rack is approaching the megawatt level; currently, the strongest racks are around 100–120kW, but in the future, a single rack might need 1MW. With this shift, compound semiconductors like GaN and SiC are being used more and more in power systems because they can operate at higher voltages and have lower losses and heat generation compared to traditional silicon devices.

Core Tech Evolution Trends

Power delivery is shifting from 48V DC to 400V/800V HVDC as rack power density approaches 1MW. This accelerates adoption of wide-bandgap (GaN/SiC) devices.

Soaring TDP forces a transition to liquid cooling (~30% adoption by 2035), surging demand for dedicated flow/pressure/temp sensors and controller MCUs.

III

Conclusion & Strategic Supply Chain Insights for Trusty IC

Why AI Triggers Global Chip Shortages and Price Hikes?

Foundries and IDMs are aggressively reallocating advanced wafer capacity, CoWoS packaging, and DRAM lines toward high-margin AI ASICs and HBM.

 Traditional mature part numbers (Legacy/Mature Components) are being strategically phased out:

The original manufacturers focusing their resources on AI has squeezed the production capacity for general DRAM, traditional NAND, and standard Power/Analog chips, causing structural supply shortages, longer lead times, and skyrocketing spot prices for commonly used legacy chips in cars, industrial automation, healthcare, and traditional servers.

💡 Why Trusty IC？

Trusty IC helps global OEMs, EMS companies, and suppliers solve three key supply chain challenges:

Shortage Sourcing – Fast sourcing of hard-to-find, shortage, obsolete, and EOL semiconductor components through global sourcing networks and available inventory.

Excess Inventory & Cost Optimization – Support for excess stock, BOM optimization, component cross-reference, alternative sourcing, and PPV reduction.

Quality Assurance – Strict quality control, component traceability, X-Ray inspection, and third-party testing to help ensure authentic and reliable electronic components.

Trusty IC — Helping customers find the right components, reduce inventory costs, and manage semiconductor supply risks.