Much has been written about xAI’s Colossus 1. The Memphis build belongs in the history books: the largest AI training cluster, erected from scratch in 122 days. With roughly 200,000 H100/H200s and ~30,000 GB200 NVL72, it remains, today, the largest fully operational, single-coherent cluster (setting apart Google, master of multi-datacenter-training). However, Colossus 1’s ~300 MW […]| SemiAnalysis
Nvidia announced the Rubin CPX, a solution that is specifically designed to be optimized for the prefill phase, with the single-die Rubin CPX heavily emphasizing compute FLOPS over memory bandwidth…| SemiAnalysis
Bridging the gap between the world's most important industry, semiconductors, and business.| SemiAnalysis
6 posts published by Jeremie Eliahou Ontiveros, Dylan Patel, Kimbo Chen, and Tanj Bennett during June 2025| SemiAnalysis
2 posts published by Dylan Patel and Jeremie Eliahou Ontiveros during September 2025| SemiAnalysis
Compute is the lifeblood of AI. He who controls the spice controls the universe the compute will control the production of tokens and reap the benefits of AI. Without compute you do not have a seat at the table. The United States technology community is all in on compute and AI as the next platform […]| SemiAnalysis
Two-and-a-half years ago, we flagged a looming “cloud crisis” at AWS. Today, the evidence has mounted. AWS is the crown jewel of the Amazon empire, generating ~60% of group profits, and dominating …| SemiAnalysis
Frontier model training has pushed GPUs and AI systems to their absolute limits, making cost, efficiency, power, performance per TCO, and reliability central to the discussion on effective training. The Hopper vs Blackwell comparisons are not as simple as Nvidia would have you believe. In this report, we will start by present the results of […]| SemiAnalysis
To many power users (Pro and Plus), GPT5 was a disappointing release. But with closer inspection, the real release is focused on the vast majority of ChatGPT’s users, which is the 700m+ free userbase that is growing rapidly. Power users should be disappointed; this release wasn’t for them. The real consumer opportunity for OpenAI lies […]| SemiAnalysis
The first portion of this report will explain HBM, the manufacturing process, dynamics between vendors, KVCache offload, disaggregated prefill decode, and wide / high-rank EP. The rest of the repor…| SemiAnalysis
Robots have powered manufacturing for decades, yet they stayed single-purpose and thrived only in perfect settings. Previous attempts at intelligent machines overpromised and underdelivered. But th…| SemiAnalysis
Subscribe to get access Read more of this content when you subscribe today.| SemiAnalysis
MGX GB200A NVL36, B102, B20, CoWoS-L, CoWoS-S, GB200A NVL64, ConnectX-8, Liquid Cooling vs Air Cooling, NVLink Backplane, PCB, CCL, Substrate, BMC, Power Delivery Nvidia’s Blackwell family is encou…| SemiAnalysis
Hyperscale customization, NVLink Backplane, NVL36, NVL72, NVL576, PCIe Retimers, Switches, Optics, DSP, PCB, InfiniBand/Ethernet, Substrate, CCL, CDU, Sidecar, PDU, VRM, Busbar, Railkit, BMC Nvidia…| SemiAnalysis
GPT-4 Profitability, Cost, Inference Simulator, Parallelism Explained, Performance TCO Modeling In Large & Small Model Inference and Training Nvidia’s announcement of the B100, B200, and GB200 …| SemiAnalysis
Nvidia H100, Google TPUv5, AMD MI300, Intel Gaudi3/PVC, Cerebras WSE2 AI accelerators are becoming increasingly power-hungry. The Nvidia H100 has thermal design power (TDP) of 700 watts (W) compare…| SemiAnalysis
Specifications, Volumes, GPT-4 performance, Next Generation Timing / Name, Backend Design Partner Microsoft is currently conducting the largest infrastructure buildout that humanity has ever seen. …| SemiAnalysis
Long time readers will recall that SemiAnalysis covers more than just datacenters and AMD. Today we’re back to semiconductors with a tech-focused roundup of the best from this year’s VLSI conference, the premiere design and integration. That includes the latest in chips manufacturing: fab digital twins, the future of advanced logic transistors and interconnects, DRAM […]| SemiAnalysis
Meta’s shocking purchase of 49% of Scale AI at a ~$30B valuation shows that money is of no concern for the $100B annual cashflow ad machine. Despite seemingly unlimited resources, Meta has been falling behind foundation labs in model performance.| SemiAnalysis
Book a Meeting Get VIP Support Book a Meeting Get VIP Support Subscribe to get access Read more of this content when you subscribe today.| SemiAnalysis
From DLRM to LLM, internal workloads win, but how does Google fare in external workloads? The dawn of the AI era is here, and it is crucial to understand that the cost structure of AI-driven softwa…| SemiAnalysis
Oracle’s Cloud Infrastructure business is firing on all cylinders and is greatly outpacing expectations. All eyes are on the high-profile Stargate JV and the massive Abilene, Texas datacenter…| SemiAnalysis
The largest AI labs are racing to build multi-gigawatt-scale datacenters, and stressing our century-old power grid to an unprecedented extent. Not only is the scale massive, but AI training wo…| SemiAnalysis
In our AI Scaling Laws article from late last year, we discussed how multiple stacks of AI scaling laws have continued to drive the AI industry forward, enabling greater than Moore’s Law grow…| SemiAnalysis
Subscribe to get access Read more of this content when you subscribe today.| SemiAnalysis
The test time scaling paradigm is thriving. Reasoning models continue to rapidly improve, and are becoming more effective and affordable. Evaluations measuring real world software engineering tasks…| SemiAnalysis
The SemiAnalysis AI Datacenter Model is used to understand current and forecast datacenter critical IT power capacity for both colocation and hyperscale datacenters with a focus on the demand drive…| SemiAnalysis
Over the last few months, there have been a number of headlines raising concerns about Microsoft’s reduction in datacenter leasing activities including a few datacenter leasing cancellations.…| SemiAnalysis
SemiAnalysis is expanding the AI engineering team! If you have an experience in PyTorch, training, inferencing, system modelling, SLURM/Kubernetes, send us your resume and 5 bullet points demonstra…| SemiAnalysis
Huawei is making waves with its new AI accelerator and rack scale architecture. Meet China’s newest and most powerful Chinese domestic solution, the CloudMatrix 384 built using the Ascend 910C. Thi…| SemiAnalysis
This page was last changed on November 4, 2024, last checked on November 4, 2024 and applies to citizens and legal permanent residents of the United States. 1. Introduction Our website, (here…| SemiAnalysis
The buildout of AI infrastructure in the US has reached a macro-level scale, and ensuring continuous growth will require ample availability of capital. We believe that the economic uncertainty indu…| SemiAnalysis
The ClusterMAX™ Rating System and content within this article were prepared independently by SemiAnalysis. No part of SemiAnalysis’s compensation by our clients was, is, or will be directly or indi…| SemiAnalysis
The Reasoning Token Explosion AI model progress has accelerated tremendously, and in the last six months, models have improved more than in the previous six months. This trend will continue b…| SemiAnalysis
Cluster deployments are an order of magnitude larger in scale with Gigawatt-scale datacenters coming online at full capacity much faster than most believe. As such, there are considerable desi…| SemiAnalysis
IEDM 2022 Round-UpWe recently attended the 68th Annual IEEE International Electron Devices Meeting in San Francisco. IEDM is a premiere conference for state-of-the-art semiconductors device technol…| SemiAnalysis
The DeepSeek Narrative Takes the World by Storm DeepSeek took the world by storm. For the last week, DeepSeek has been the only topic that anyone in the world wants to talk about. As it currently s…| SemiAnalysis
The Open AI Stargate Joint Venture announcement had many folks heads turning, despite us calling out the capital requirements for OpenAI’s immediate plans months ago. The headline $500 billio…| SemiAnalysis
Effective: November 4, 2024 About this Privacy Policy Your privacy and trust are important to us. This Privacy Policy outlines how we collect, use, and share your information (“Information”) throug…| SemiAnalysis
Foundry Cost Wall, Whale Customers, Datacenter Share, The Money Problem Before Pat Gelsinger took over Intel as CEO, the company spent over a decade in a slow descent due to a focus on financial en…| SemiAnalysis
Planar to FinFET to Nanosheet to Complementary FET to 2DThe fundamental component of any chip is the transistor, which recently celebrated its 75th birthday. Today we will discuss the next 25 years…| SemiAnalysis
Merry Christmas has come thanks to Santa Huang. Despite Nvidia’s Blackwell GPU’s having multiple delays, discussed here, and numerous times through the Accelerator Model due to silicon, packaging, …| SemiAnalysis
There has been an increasing amount of fear, uncertainty and doubt (FUD) regarding AI Scaling laws. A cavalcade of part-time AI industry prognosticators have latched on to any bearish narrative the…| SemiAnalysis
Huawei Fab Network, WFE Vendors Cry Wolf, Framework for Future Controls AI competitiveness is a key national security concern. When “expert-level science and engineering” or even AGI are possible o…| SemiAnalysis
Fab Cost, SRAM Scaling, WFE Implications, Backside Power Details, TSMC, Samsung, Intel, Rapidus TSMC won FinFET. All noteworthy leading edge logic designs, even Intel’s, are manufactured on TS…| SemiAnalysis
The US government lobbed the largest salvo in the new technology cold war with its new Framework for Artificial Intelligence Diffusion. These new export restrictions are completely unprecedented in…| SemiAnalysis
Intro SemiAnalysis has been on a five-month long quest to settle the reality of MI300X. In theory, the MI300X should be at a huge advantage over Nvidia’s H100 and H200 in terms of specifications an…| SemiAnalysis
