AI Compute Infrastructure: GPUs, Data Centers, and the Compute Race

Modern artificial intelligence is more than just algorithms. Behind every large language model or computer vision system lies a colossal physical infrastructure: thousands of accelerator chips, data center buildings consuming power equivalent to a small town, along with cooling systems and high-speed networks. Compute infrastructure has become the “strategic resource” of the AI era — whoever controls compute, controls the pace of innovation.

Accelerator Chips: The Heart of AI

At the core of AI infrastructure are specialized accelerators, primarily NVIDIA’s GPUs. Unlike traditional CPUs that process sequentially, GPUs perform thousands of calculations in parallel — which aligns with the nature of neural network training.

The Blackwell architecture generation (launched and widely deployed from 2025) is the current benchmark:

• B200 GPU: 192 GB HBM3e memory, 8 TB/s bandwidth, achieving up to 9,000 TFLOPS of FP4 performance — approximately 4 times the inference throughput compared to the H100 generation. Each B200 GPU consumes about 1,000W.
• GB200 Superchip (1 Grace CPU + 2 B200 GPUs): priced at approximately 60,000–70,000 USD per unit.
• DGX B200 System (8 GPUs): priced at approximately 515,000 USD, requiring about 14 kW of power.

According to NVIDIA CEO Jensen Huang, each B200 GPU has a factory price of 30,000–40,000 USD; effective system-level pricing ranges from 30,000–55,000 USD per GPU. On the cloud, B200 rental prices as of mid-2026 vary widely from approximately 2.99 to over 27 USD per GPU-hour, depending on the provider, contract, and region.

Key takeaway: chips never stand alone. A modern “AI Factory” couples hundreds to thousands of GPUs using ultra-high-speed interconnects (NVLink, InfiniBand) to make them operate as a single supercomputer.

Data Centers and the Power-Cooling Challenge

High-density GPUs create unprecedented physical challenges. A GB200 NVL72 rack can consume tens to over a hundred kW — far exceeding the 5–10 kW per rack of traditional data centers. This entails:

• Liquid cooling instead of air cooling, as air can no longer sufficiently dissipate heat.
• Stable, high-capacity power supply, making data center location highly dependent on available power sources.
• Uptime Tier III/IV operational standards to ensure high availability and redundancy.

These constraints transform AI data centers into national infrastructure projects, not merely information technology projects.

Cloud Computing and the Compute Spending Race

Most businesses don’t buy GPUs themselves but rather rent them via the cloud. Hyperscalers (Amazon, Microsoft, Google, Meta, Oracle) are pouring money into infrastructure at an unprecedented pace:

• Capital expenditure (capex) by the four tech giants Google, Amazon, Microsoft, and Meta is projected to reach approximately 725 billion USD in 2026, a 77% increase over the record high of about 410 billion USD the previous year.
• The top five hyperscalers alone are expected to spend over 600 billion USD on infrastructure in 2026 (a 36% increase over 2025), with approximately 75% (~450 billion USD) allocated to AI infrastructure.
• To fund this, hyperscalers raised approximately 108 billion USD in debt in 2025 alone.

This scale indicates that compute has become a “competitive weapon” — and also raises questions about financial risks if AI demand doesn’t grow fast enough to recoup investments.

AI Infrastructure in Vietnam: The AI Factory Wave

Vietnam is experiencing a strong wave of data center investments, driven by the goal of technological self-reliance:

• FPT AI Factory: strategic partnership with NVIDIA (announced April 2024, with approximately 200 million USD capital). The system is equipped with thousands of H100 and H200 GPUs, with each supercomputer comprising 8 H100 GPUs (80 GB memory, 3.35 TB/s speed), set to begin providing services from January 2025.
• Viettel: Hoa Lac 2 Data Center, achieving Uptime Tier III standard, integrating NVIDIA H200 supercomputers — introduced as the first AI data center by a telecommunications enterprise in the Asia-Pacific region to meet this standard. Viettel is collaborating with NVIDIA to integrate nearly 800 supercomputers and approximately 6,000 GPU cards.
• CMC: investing over 250 million USD to build a large data center called “AI Heart” in Ho Chi Minh City, operating the C-OpenAI system on CMC Cloud infrastructure; simultaneously planning to invest approximately 500 million USD in data centers in Vietnam, Japan, and other locations by 2028.
• VNPT: established VNPT AI at the end of 2025, affirming its role in the AI ecosystem and its goal of ensuring digital sovereignty.
• NVIDIA – Vietnamese Government: in December 2024, signed an agreement to establish an R&D Center and an AI Data Center in Vietnam; NVIDIA also acquired VinBrain (a medical AI company part of Vingroup).

These moves are closely linked to Resolution 57-NQ/TW (December 22, 2024), which sets the goal for Vietnam to be among the top 3 in Southeast Asia for AI research and development by 2030, while also emphasizing the need for technological self-reliance to avoid digital dependence.

Trends

1. Inference surpassing training: As models are widely deployed, the compute cost for running models daily gradually exceeds the initial training cost — leading to a demand for energy-efficient inference chips.
2. Power and sustainability: Power supply is becoming a bottleneck. AI data centers are forced to consider renewable electricity and liquid cooling to control costs and emissions.
3. Compute sovereignty: Nations, including Vietnam, want to own domestic compute infrastructure rather than rely on foreign clouds.
4. Chip diversification: Alongside NVIDIA, custom-designed chips (Google’s TPUs, Amazon’s Trainium) and competitor AMD help reduce vendor lock-in risks.

Conclusion

Compute infrastructure is the physical foundation that determines who can play in the AI arena. With high GPU costs, massive power demands, and a global spending race worth hundreds of billions of USD, compute is both an opportunity and a barrier. Vietnam, through the AI Factory wave by Viettel, FPT, CMC, VNPT, and Resolution 57, is proactively building this foundation to stay in the game.

References

• NVIDIA Blog — Thailand, Vietnam Embrace Sovereign AI
• VietnamPlus — Vietnam riding wave of data centre investments
• FPT — Inside FPT’s first AI Factory in Vietnam
• VietnamPlus — Resolution 57: Vietnam advances domestic AI ecosystem
• IEEE ComSoc Technology Blog — Hyperscaler capex > $600bn in 2026
• Tom’s Hardware — Big Tech AI spending to hit $725 billion in 2026
• Modal — NVIDIA B200 pricing 2025
• Spheron Blog — NVIDIA GB200 NVL72 Guide
• Vietnam Insider — Nvidia Acquires Vingroup’s AI Company