Jason A. Hoffman, PhD
The industry quotes compute in units of intention. Gigawatts announced, accelerators shipped, capital committed, peak FLOP/s on the spec sheet. Contracts are signed in those units, market share is argued in those units, and national positions are compared in those units. No model has ever been trained on any of them.
The units sort into kinds, and the sorting matters. FLOP/s, accelerator-hours, and joules are resource measures: what a machine can execute or consume, with no word about whether anything useful happened. Tokens, images, simulation steps, and completed tasks are workload observables: what a system emits, countable and billable, which is why the industry has learned to price itself in tokens. But the token is a currency without an exchange rate: it means something only inside a declared model, task, quality bar, and latency budget, and it is not fungible across unlike workloads. Value arrives only at goodput, accepted output per unit time after the quality and deadline gates. Effective compute is the connective term, how much of the resource you bought is actually available to produce observables worth accepting, and it is where the industry’s accounting is weakest.
Here is the puzzle that makes the gap concrete. NVIDIA’s Huang’s Law framing describes roughly a thousandfold gain in single-GPU AI inference over a decade. At GTC 2026, Huang projected the current roadmap version: a one-gigawatt factory’s token-generation rate rising from 2 million to 700 million tokens per second in two years, a factor of 350. Those are different windows and denominators, not one measured series. They are evidence of the same industry thesis: transistor scaling supplies only a fraction of the progress that matters. Take NVIDIA’s own decomposition as the puzzle the industry is posing, a thousandfold in effective output against a factor of two and a half from process technology. The missing factor of four hundred is engineered across the rest of the machine. Knowing where it can come from is the difference between reading this industry and being read to.
I have worked on this post for the last year because deciding where “compute” is going has been part of my work. Data centers are becoming one of the largest and fastest-growing classes of new continuous load in the United States; LBNL’s current scenarios put them at 9.5 to 15.3 percent of national electricity use by 2030.
The economic stake is already disproportionate. A Chicago Fed sectoral decomposition finds that from 1988 through 2023, IT averaged only about 8 percent of private-business value added but supplied 0.43 percentage points of the 0.95 percent average annual TFP growth in the authors’ measure. Remove that measured contribution and 0.52 points remain: the other 92 percent of the economy produced 55 percent of productivity growth. That is not a literal no-IT GDP counterfactual; GDP also grows through added labor and capital, and sector linkages cannot be switched off. It does mean nearly half the growth in output unexplained by additional inputs came from IT, in a sample ending before the current generative-AI buildout could materially register.
That is the long-run productivity account. The current expenditure account is different but equally consequential. J.P. Morgan estimated that AI-related capital expenditure contributed 1.1 percentage points to U.S. GDP growth in the first half of 2025, more than consumer spending. Capex entering GDP is not evidence that the investment will earn a return or raise productivity, and J.P. Morgan notes that imported technology goods offset part of the domestic effect. It does show that the physical buildout was already large enough to move the top-line economy before its downstream productivity effects could be measured at scale.
The buildout is now explicit national economic, energy, and security strategy. Once a system enters politics, the pressure is to compress it into a slogan: win the race, build more power, solve cooling. Slogans can mobilize. They cannot specify a machine.
I have lived through enough transitions to distrust forecasts organized around solving the next bottleneck. Remove one constraint and the machine reorganizes around the rest; cross into a new regime and the constraints themselves change. Those of us who have to engineer over decades need a form of foresight that survives both the slogan and today’s bottleneck. I needed a model that could hold that complexity long enough to make my own thinking explicit.
In On Binary Narratives I argued that the most useful move against a headline is to write the identity underneath it. The identity underneath the datacenter question was about demand: total compute demand equals volume of useful output times compute per unit of output. This post writes the supply-side twin, the identity underneath the word “compute” itself. It runs over eleven engineering axes. A twelfth axis, demand, decides whether the resulting output is wanted rather than how the machine produces it. Each engineering axis has observable production regimes and named alternatives. With demand, they form the twelve-axis taxonomy used here.
An engineering dimension belongs inside the supply identity if changing its design regime changes how much accepted output a configured machine could produce from a declared amount of raw capacity for a declared workload. Constraints that determine how quickly or cheaply a design can be manufactured or deployed belong to feasibility, cost, and time unless they change that design potential, in which case the relevant axis captures them. The taxonomy is open: a new mechanism that meets the rule becomes a new axis rather than being forced into an old one.
The identity reads in both directions, and this post runs it both ways. Forward: how much better could these systems get, and from where, given the transition gains that history and theory offer on each axis. Backward: if the destination is systems some large factor more capable, which axes must move, where and by when, and why those, with the how left to the engineering program the answer defines. Forward is what can be taken advantage of. Backward is what has to be built.
The backward run does not produce a unique allocation. It exposes feasible combinations; cost, timing, confidence, and dependency choose among them.
The Identity
C_eff,design = C_raw × η_numbers × η_distance × η_medium × η_wire × η_control × η_disaggregation × η_thermodynamics × η_algorithm_shape × η_training_topology × η_data_efficiency × η_learning_phase
This first perspective is design potential for a declared workload and acceptance criterion. Effective compute at design state is raw arithmetic capacity times eleven efficiency multipliers. Raw capacity is the term spec sheets and announcements report most readily, and it has to be quoted at a declared number format, because peak FP8 and peak FP32 are different peaks for the same silicon; the declared format keeps the number-representation axis from counting its gain twice. The eleven multipliers are where the rest of the gain lives. Later sections shift to three other perspectives: absolute machine scale, realized operation, and economic demand.
The multiplication sign is an accounting syntax, not a claim that the terms are independent or mechanically stackable. They are coupled and endogenous. A change in the wire axis changes which training topologies are feasible; topology changes the return to control; data efficiency changes the value of memory and raw capacity. Nor does the model require the system to have one bottleneck. It represents the joint state of a complex adaptive system, so a gain attributed to any one term has meaning only with the other terms and the comparison baseline declared.
Think of it as configuring a machine, not climbing eleven independent ladders. Each subsystem has several possible designs. The benefit of choosing one depends on the workload and everything around it. A faster engine is useless without adequate cooling, transmission, tires, and road. Some choices reinforce one another, some substitute for one another, and some combinations are impossible. The complete configuration under a declared workload is the fundamental object.
To display the complete configuration and its interactions directly, let x be the vector of eleven design choices, w the declared workload context including its acceptance criterion, and t the time. The system can be written as a nonseparable joint function:
C_eff,design(w,t) = C_raw(t) × F(x;w,t)
F is the model of the complete configuration; the long product is a reporting view of it. Factorization names the sources of gain. The logarithm shows how proportional gains compose: it turns multiplication into addition, allowing gains associated with individual choices to be distinguished from gains or losses created by their interaction. It does not make the machine linear.
In log form, the structure inside F can be expanded:
ln F = Σ_i g_i(x_i) + Σ_{i<j} g_ij(x_i,x_j) + Σ_{i<j<k} g_ijk(x_i,x_j,x_k) + …
For example, against the same baseline and workload, if wire and topology produce 1.2× and 1.3× separately, independence predicts 1.56× together. A measured 1.8× means additional gain exists only in combination; less than 1.56× means the choices interfere.
The first sum represents single-axis contributions; the later sums represent pairwise and higher-order interactions. If those interaction terms vanish, the simple product behaves as independent multipliers. If they do not, the gain from changing one choice depends on the state of the rest. These terms are finite gains relative to a declared baseline, not infinitesimal derivatives. Read each multiplier in the long form as a conditional or allocated contribution for that comparison, not a portable coefficient with a value of its own. Where an interaction gain is assigned among the multipliers, the attribution can depend on the path through the configurations, so the baseline and path must both be declared.
The identity is deliberately local: one design state, one declared workload, one acceptance criterion. A deployed fleet is a distribution, not a representative machine. Generations coexist, with different number formats, memory, fabrics, software, and failure behavior. Demand is another distribution across model sizes, modalities, latency, quality, locality, privacy, and duration, and that mix changes over time. Fleet-level effective compute is an allocation problem across those two evolving distributions. It is summed over feasible machine-workload pairings, not calculated by applying this product to an average machine. A fleet can gain raw capacity while losing useful capacity if demand moves somewhere its installed systems cannot serve economically. Heterogeneity turns allocation and portability into part of the effective-compute problem.
Each axis has a structure that matters as much as its coefficient: a set of discrete design regimes, not a smooth dial. Many material gains arrive when a machine crosses from one regime to another. The alternatives do not always have one universal order; their value depends on the workload and the complete configuration. Number systems can move from FP32 to FP16 to FP8 to INT4, with a class transition worth roughly two to sixteen times depending on the workload. The wire can move from copper SerDes to mid-board optics to co-packaged optics, three to ten times per fabric generation. Algorithm shape can move from dense quadratic attention toward sparse attention, state-space models, and other sub-quadratic forms. A generation either adopts such a transition or it does not, and each axis addresses a specific, nameable constraint: number representation attacks arithmetic and representation cost, distance attacks the bandwidth ceiling, the wire attacks the communication taxes of parallelism, and data efficiency attacks the training-token requirement.
The eleven axes in the identity group into two engineering families. The seven substrate axes are the hardware you can photograph: numbers; distance, or memory proximity (HBM-fed multipliers to on-chip SRAM to wafer scale, five to fifty times the bandwidth per transition); the compute medium itself (digital toward analog compute-in-memory, ten to one hundred times in theory and still research-class); the wire; the execution control model (dynamic scheduling toward deterministic compile-time dataflow, worth 1.5 to 3 times in scheduling discipline); disaggregation of compute from memory (bundled GPU-plus-HBM toward composable memory fabrics, two to five times in capacity utilization); and, at the far horizon, thermodynamics, from CMOS toward superconducting and reversible logic. Most of the substrate family exists because of one physical fact beneath every abstraction: the arithmetic itself can cost less than a picojoule while moving its operands through the memory hierarchy can cost orders of magnitude more, so the axes that matter most are often the ones that move data less.
The four workload axes in the identity describe what the machine is asked to do rather than what it is made from: algorithm shape, training topology (synchronous all-reduce toward local-update and low-communication methods such as DiLoCo, 1.5 to 3 times in communication reduction), data efficiency (brute-force corpus toward curriculum, self-play, and synthetic data, ten to one hundred times per genuine breakthrough), and the split between training and test-time compute, a reallocation term rather than a machine-efficiency term, where the o1-class shift toward inference-time search moved the industry into a new regime in a single year. Demand is the taxonomy’s fifth workload-side axis and the twelfth overall. It stays outside this supply identity because it determines whether effective output finds a use. It returns in the market check.
Every gain range above characterizes a possible transition within a declared comparison, not a promise or a fixed coefficient. They are order-of-magnitude planning estimates synthesized from published mechanisms, observed systems, and research results. This post does not derive each range, so a backward run should treat them as starting hypotheses and expose the workload, baseline, and source before using them quantitatively. Each axis has observable production regimes and named alternatives, with production systems distinguished from research-class work. The identity is not a forecast. It is an accounting of where improvement can come from, with the state of each source made explicit.
Reading the Thousand Times
Write the identity under NVIDIA’s claim and the claim becomes a question with named inputs. A thousandfold period cannot come from raw silicon alone. It requires several design transitions to work together: lower precision, specialized tensor architecture, sparsity, fabric, model shape, and software control. Measured algorithmic efficiency has improved alongside hardware, which is why hardware-only histories miss part of the curve. That is how recent gains have been assembled, and it is why the scaling story can be simultaneously meaningful and misleading. Meaningful, because a coordinated system can produce a multiplicative gain. Misleading, if a listener hears the number as a property of the silicon or as a sum of independent benchmark claims.
Does another thousand follow the first, and another follow that? Arithmetically, yes. Every realized system state becomes the next baseline; from any fixed starting point, one thousandfold period is 10³, two compound to 10⁶, and three to 10⁹. The identity makes that recursion possible, but it does not make the cadence automatic. The same precision transition cannot be taken again, and every transition changes the returns available on the other axes. The next thousand has to be made from a new joint state: further changes on existing axes, new interactions among them, or an axis that the present taxonomy does not yet contain. The billionfold stress test below therefore asks the identity to allocate three consecutive thousandfold gains; it does not forecast that history will deliver them.
The identity also needs a clock. For a fixed capability threshold and a declared cost basis, define time to 99 percent cost reduction, or T99, as the elapsed time from a baseline training run until a reproducible run reaches the same threshold for one percent of the cost. This measures compression behind the frontier rather than movement of the frontier itself, and it allows any combination of axes to earn the gain. The modded-nanoGPT speedrun is a clean small-scale example: on eight H100s and a fixed FineWeb validation target, its record history falls from forty-five minutes in May 2024 to about seventy-six seconds, roughly a thirty-fivefold reduction in accelerator-time. It has not reached T99, but it makes the clock visible. The companion measure is scale-transfer lag, how long a technique proven in a small fixed-target race takes to appear in frontier training. Disclosure makes that harder to observe, but it is the bridge between experimental efficiency and industrial capability.
That reading has a sharper strategic consequence: the active constraints and their interactions set a generation’s machine geometry. Sometimes one dominates. When memory bandwidth binds, the machine reorganizes around memory proximity; when communication binds, it reorganizes around the fabric. In other periods several constraints co-evolve, and a change on one axis changes the leverage on the rest. An integrated machine with no stable seams between its layers is the substrate that can coordinate several transitions at once, because no interface contract forbids the joint move. The paradigm is permanent. The geometry is fluid. The identity is the compact way to say both things at once.
The recent infrastructure sequence makes this concrete. Liquid cooling relaxed the server thermal envelope and made denser racks possible. Denser racks moved the constraint into power delivery and generation. Once site power reaches multi-gigawatt scale, the constraint moves again, to the ultimate heat sink, where the site can reject the energy after the cooling loop has carried it away. Each solved constraint increases the load on the next boundary. Under the sequence are the two quantities this post separates: the effective compute the system can deliver and the volume and mix of demand that makes the next configuration worth building.
There is a corollary worth stating plainly, because On Binary Narratives and On Arguing Against Jevons both turned on it. Changes on the workload-family axes require no chips, no permits, and no transformers on order. Moonshot’s optimizer result from the K3 story is a transition in data efficiency; the test-time-search shift is a transition in the learning-phase allocation. Changes there multiply the value of every watt already installed and raise the return on compute. Whether they raise total compute demand depends on the twelfth axis: if useful volume grows faster than compute per unit falls, demand rises; if it does not, demand falls. The rebound argument is this identity combined with an elasticity claim, not the identity alone.
A Ten-Gigawatt Coherent-Machine Scenario
Now change perspective from relative design gain to absolute physical scale. The raw-capacity term invites a bounding scenario: how large could one synchronous domain become if it had to fit under one roof? Start with one of the biggest roofs humans have built, roughly 120 acres or half a square kilometer, a building seven hundred meters on a side. Allocate a third of it to accelerator racks and the rest to the machinery that feeds and cools them. Connect the racks with hollow-core fiber, where light travels at nearly its vacuum speed instead of the two-thirds it manages in glass. The far corners are then about three microseconds apart instead of five. If switching, collective operations, and queueing keep the end-to-end fabric inside the workload’s synchronization budget, the building can be modeled as one coherent machine, a chip whose package is a building. Roof area, rack allocation, heat rejection, and allowable collective latency are parameters here, not constants of nature.
Choose a site-average heat-rejection assumption of twenty kilowatts per square meter and the arithmetic lands near ten gigawatts, about twenty times peak sunlight across the footprint. That is an aggressive reference case, not an estimate of typical site density or a demonstrated universal ceiling. At ten kilowatts per square meter the same roof lands near five gigawatts; at forty, if the power and cooling systems can be built, it lands near twenty. Change the roof, rack fraction, or synchronization budget and the answer moves again. A machine in this range sits inside a utility ring: heat-rejection fields, switchyards, water handling, and the land the thermodynamics demands. The scenario depends on innovation that has nothing to do with silicon, in power delivery, cooling, and network reach, utility innovation in the literal sense.
Cooling moves heat; it does not eliminate it. That distinction becomes regional at this scale. A preliminary analysis by Utah State University physicist Rob Davies of the proposed nine-gigawatt Stratos campus and its colocated gas generation estimated roughly sixteen gigawatts of continuous thermal load in Hansel Valley. The comparison to the energy of twenty-three Hiroshima-scale bombs per day made the story travel, but the engineering quantity is continuous heat rejection, not an explosion. The temperature model is not a peer-reviewed environmental assessment and depends on the eventual generation, cooling, layout, and local meteorology. The need to close the heat balance does not. At multi-gigawatt scale, the atmosphere, water system, and surrounding land become parts of the machine’s ultimate heat sink.
Today’s standard large deployment is the hundred-megawatt class. From there to the scenario’s ten-gigawatt coherent machine is a factor of one hundred. That gives raw-capacity scaling two orders of magnitude in the reference case; it does not prove that every training workload shares one immutable roof. Training tends to value coherence, while inference multiplexes statistically across independent work and can scale past one building by adding buildings. Under the assumptions above, the scarce object at the end of the buildout is not capacity in general. It is the coherent domain.
A Billionfold Stress Test
Now put the physical-scale scenario back in the identity and run the identity backward, from a destination to candidate allocations. Suppose the destination, over some decades, is machine intelligence a billion times more capable per dollar than today’s. This is a stress test, not a forecast. Its scale forces the allocation problem across the full system rather than allowing one familiar transition to carry the answer. The CDC 6600, the fastest machine of 1964, delivered about three million floating-point operations per second. It was already the kind of coupled transition this post describes: its design moved from germanium to silicon, air to Freon cooling, and standardized building blocks to dense custom modules, while adding parallel functional units, interleaved memory, and ten peripheral processors. Today’s exascale systems deliver about two quintillion floating-point operations per second, nearly twelve orders of magnitude in sixty years, and the per-dollar measures improved by nine or ten.
Feynman’s 1959 lecture, There’s Plenty of Room at the Bottom, was not a prophecy that one scaling law would continue at a fixed rate. It was an inventory of unrealized physical headroom. He calculated that the Encyclopaedia Britannica could fit on a pinhead, imagined atom-scale information storage and fabrication, and argued that computers would have to shrink because material, power, heat, and propagation delay would otherwise limit them. More important for this identity, he did not say that a large machine could simply be made smaller. Electrical equipment would have to be redesigned, and at sufficiently small scales quantum behavior would create different opportunities and constraints. His argument was not that the next bottleneck had one solution. It was that entering another regime changes the engineering problem. The following sixty years filled part of that physical room by crossing regime after regime. Historical precedent makes another nine orders a useful stress test rather than an arbitrary scale. It does not assign a probability or cadence to reaching it. It shows only that the previous nine came from repeated changes of regime, not the indefinite extension of one scaling law.
The identity says where a further billion would have to come from in this stress test. Under the reference physical-scale assumptions, the expansion from a hundred-megawatt machine to a ten-gigawatt one contributes a hundred. The other seven orders of magnitude must come from everything else in the equation: number systems, memory proximity, the compute medium, the wire, control, disaggregation, and above all the workload terms, algorithm shape, data efficiency, and the training-inference split, the terms that have recently compounded fastest and that require no permits.
What remains is the market check, and it is the question the demand identity from the first post exists to ask: is that much token volume warranted? Past general-purpose inputs whose prices fell by orders of magnitude often saw volume expand, but that history does not set AI’s elasticity. The case for high elasticity is that machine intelligence has unusually deep latent uses; the countercase is that budgets, trust, task quality, and human attention impose satiation. The honest posture is to watch which effect dominates. Elasticity has so far been the entire story of this industry.
The Operating Fraction
The design function says what one configuration could deliver for a declared workload. Operation determines how much of that potential a built system realizes:
C_eff,realized(x,w,t,o) = C_raw(t) × F(x;w,t) × f_operating(x,w,t,o)
Here o denotes the operating conditions, including policy and measurement window. The operating fraction is conditional on the configuration, workload, time, and those conditions. It is not a universal utilization haircut. Density changes thermal and failure behavior, topology changes recovery cost, and workload mix changes scheduling. The separation is analytic, not causal: the design establishes potential; operation measures its realization. The control axis describes the potential of the selected control regime; the operating fraction measures what the deployed system actually delivers.
Model FLOPs utilization at the best documented practice was 55.2 percent on 12,288 GPUs, a figure worth a systems paper. Meta’s Llama 3 team recorded 466 interruptions over 54 days while training on 16,384 H100s and kept effective training time above ninety percent through automated detection and management. Their later cluster reliability analysis models job mean time to failure as falling approximately in inverse proportion to accelerator count, projecting 1.8 hours at 16,384 GPUs and 0.23 hours at 131,072. These figures establish that several execution discounts matter. They do not establish a universal forty-five-percent operating fraction: the papers cover different machines, definitions, and denominators, so their headline percentages cannot simply be multiplied. The operating fraction for a particular machine has to be measured across utilization, interruption and recovery time, checkpoint overhead, validation, and accepted-output yield on that machine.
Keep the two accountings distinct, because they answer different questions and different people own them. The design factors help decide which configuration is worth building. The operating fraction is a conditional execution discount: it determines what the built machine delivers. Normalizing the realized result by the system’s full cost changes perspective once more, from operation to economics. Across a fleet, the operating fraction has to be evaluated over feasible machine-workload pairings rather than inferred from average utilization alone. Between a press release’s petaFLOP/s and a customer’s realized capability per dollar sit eleven interacting design choices, an operating fraction, and a cost base. Almost nobody reports the whole quantity.
What to Watch
The identity converts the vaguest question in the industry, who is ahead, into factual, watchable state. Which regimes are in production on each axis, rather than only in papers? Which constraints bind or co-evolve in this generation, because that is where the geometry will move next? How quickly is a fixed capability’s cost compressing toward T99, and which small-scale techniques survive transfer to frontier training? How are the installed fleet and the demand mix changing, and which machine-workload mismatches are stranding capacity? How far along is the move from the hundred-megawatt class toward the coherent-domain scenario, and which utility innovations are pacing it? Which announced machines coordinate changes across multiple axes, and which are C_raw wearing a new name? When a competitor claims an efficiency breakthrough, which axis changed, what else changed with it, and what does the joint system state now produce? And for anyone building rather than watching, the backward question: which gains does your plan require the identity to deliver, on which axes, by when, and which regime transitions is it silently assuming?
I wrote about the vocabulary problem in GPU failure rates: one phrase covering four different stages, producing contradictory numbers from people who were all technically correct. “Compute” has the same disease at industrial scale, and the cure is the same: name the quantity. Raw FLOP/s is the base beneath eleven interacting design axes; demand is the twelfth taxonomy axis. The announcements and political slogans will keep arriving in units of intention. Engineers inherit the interactions they omit, and the machines will outlive the slogans. Read forward, the identity turns intention into units of work. Run backward, it turns ambition into an engineering program. The how is the work itself.
Leave a Reply