Foundation Model
A foundation model is a large model trained on broad data and intended not for one task but as a base for many. It is adapted with prompts, fine-tuning or tools instead of training something new from scratch.
What the approach changed
Previously each task got its own model trained on labelled data: one for review sentiment, another for email classification, a third for translation. Each needed its own dataset and its own team.
A foundation model is trained once on a huge general corpus and then applied to all those tasks without retraining. The cost of entry for an applied problem fell from months of work to a few lines of prompt.
The flip side is concentration. Training such models is within reach of a handful of companies, and everyone else builds on someone else's foundation, inheriting its limits and biases.
How they are adapted
- PromptingThe cheapest route: the task is described in words and examples inside the request. Nothing in the model changes.
- RAGAdds knowledge the model lacked: fresh or private data is placed into the context.
- Fine-tuningChanges behaviour: style, format, a narrow specialisation. Costlier than prompting and more consistent.
- ToolsAccess to search, code and external systems extends what it can do without touching the model.