Sam Altman has publicly stated that GPT-4 cost more than $100 million to train — a figure that's been widely reported and is a useful anchor, even though OpenAI hasn't published an exact, itemized number. Figures like that tend to get repeated without context, which leaves most teams with a wildly inflated sense of what "training a deep learning model" costs, and a wildly deflated sense of what it actually takes to do useful, smaller-scale deep learning work.
The honest answer is that training cost scales enormously depending on what you're actually building — and most teams outside a handful of frontier labs are nowhere near that $100M number, nor do they need to be.
What Actually Drives the Cost
Model size and architecture. Parameter count is the headline number, but it's a rough proxy. Training cost scales with the number of parameters multiplied by the number of tokens processed — roughly, compute cost follows what's often summarized as a "scaling law": bigger models trained on more data cost predictably more, in a way researchers at labs like OpenAI and DeepMind have published analysis on (the original "Chinchilla" paper from DeepMind, for instance, specifically argued that many large models of the era were undertrained relative to their size — i.e., you get better results per dollar by training a smaller model on more data than a bigger model on less).
Hardware, and how efficiently you use it. Frontier training runs today are built on large clusters of high-end accelerators — NVIDIA's H100 and newer Blackwell-generation GPUs are the current industry default for large-scale training, typically rented through cloud providers or accessed via dedicated clusters. The sticker price of the hardware is only part of the story: GPU utilization — the percentage of time the accelerators are actually doing useful computation rather than waiting on data loading, network communication between nodes, or checkpoint saves — is often the difference between a training run that's efficient and one that silently costs two or three times more than it should. Getting utilization above 40–50% on a large multi-node cluster is a genuine distributed-systems engineering problem, not just a matter of renting more GPUs.
Data: collection, cleaning, and licensing. Raw web-scraped text is nearly free to acquire but expensive to clean into something a model should actually be trained on — deduplication, quality filtering, and toxicity/safety filtering all require their own compute and engineering time. For domains where clean public data doesn't exist (specialized scientific data, proprietary business data, robotics demonstration data), collection itself becomes a major cost center, and increasingly, data licensing is a real, negotiated line item as publishers and platforms strike deals with AI labs.
The retries you don't see in the demo. Large training runs fail — hardware faults, loss spikes, bad data batches — and production training pipelines budget for restarting from checkpoints, not a single clean run from start to finish. A published final cost estimate almost never includes the failed attempts, hyperparameter search runs, and smaller-scale pilot trainings that happened before the "real" run that produced the model people actually use.
The Part That Matters for Most Teams: You're Probably Not Training From Scratch
The $100M-plus numbers apply to training a frontier foundation model from randomly initialized weights on trillions of tokens. Almost no business needs to do that, and almost none should try. The economically sane options for most teams, in increasing order of cost and control:
- Prompting and in-context learning against an existing hosted model (GPT, Claude, Gemini, or an open model like Llama or Mistral) — effectively zero training cost, and the right starting point for the large majority of use cases.
- Fine-tuning an existing pretrained model on your own data — orders of magnitude cheaper than pretraining, because you're adjusting an already-competent model rather than teaching it language and reasoning from nothing. This is where most "custom model" business value actually gets created.
- Parameter-efficient fine-tuning (LoRA and similar techniques) — adapting a small fraction of a model's parameters, which cuts both compute and memory requirements dramatically compared to full fine-tuning, at some cost to how much the model's behavior can shift.
- Pretraining from scratch — reserved for organizations with a specific reason an existing model architecture or training corpus genuinely can't serve (a fundamentally different modality, a regulatory requirement for full data provenance, or genuine frontier research) and the budget to match.
Most companies that think they need option 4 actually need option 2, and would save themselves months of engineering time and a large, avoidable bill by testing that assumption before committing to a full pretraining run.
Why This Matters Beyond the Bill
It changes the make-vs-buy decision. Understanding that fine-tuning and prompting sit two to three orders of magnitude below pretraining in cost reframes "should we build our own model" from an emotional decision into a straightforward cost-benefit one.
It changes hiring and team design. A team doing prompt engineering and fine-tuning needs ML engineers who understand evaluation and data quality. A team doing frontier pretraining needs distributed-systems engineers who can debug multi-node network bottlenecks and checkpoint recovery — a genuinely different (and more expensive) skill set.
It changes how you evaluate vendor claims. When a vendor says they "built a custom AI model" for a product, the honest technical question is where on this ladder they actually operated — because the answer changes what kind of results (and what kind of ongoing maintenance burden) you should expect.
A Simple Framework for Estimating Your Own Cost
Before committing budget to any training effort beyond prompting, get concrete answers to:
- What's the smallest model that could plausibly solve this task? Bigger is not automatically better for a narrow, well-defined business problem — and smaller models are cheaper to train, fine-tune, and — critically — run in production afterward.
- Do you actually have training data, or do you need to collect it first? Data collection is frequently the larger and less predictable cost, especially for anything involving human demonstrations, labeling, or licensing.
- What's your fallback if the fine-tuned or custom model underperforms a well-prompted off-the-shelf model? If you don't have a clear answer, that's a sign you haven't validated the "buy" option is actually insufficient yet.
Key takeaways:
- Headline frontier-model training costs (reported to be well over $100M for models like GPT-4) are not representative of what most businesses need to spend — they reflect pretraining from scratch, which the overwhelming majority of teams should not be doing.
- The real cost drivers are model size and data volume (following well-documented scaling-law research), hardware utilization efficiency, data cleaning and licensing, and the retries and failed runs that never show up in a published number.
- For nearly all business use cases, prompting or fine-tuning an existing model gets you most of the value at a small fraction of the cost and risk of training from scratch — validate that path before budgeting for anything larger.
If you're trying to figure out where your AI use case actually sits on this cost ladder, get in touch.