Large Language Models
Models trained on enormous text corpora that generate and understand language: how they work, what they can and cannot do.
A large language model (LLM) is a neural network — today almost always a decoder-only transformer — trained on vast amounts of text to predict the next token. From that single objective, models with hundreds of billions of parameters trained on trillions of tokens learn grammar, facts, reasoning patterns, and stylistic conventions; at inference time they generate text autoregressively, one token at a time, sampling from the probability distribution the model assigns to each possible next token.
The training pipeline has three standard stages. Pretraining is self-supervised next-token prediction on a large corpus of web text, books, and code. Instruction tuning then fine-tunes the model on curated pairs of instructions and desired responses. Alignment, most notably reinforcement learning from human feedback (RLHF), trains a reward model from human preference judgments and optimizes the policy against it — the step that made modern assistants helpful and safe enough to deploy. At inference, generation is controlled by sampling parameters: temperature shapes randomness, and top-k or top-p truncation restrict the candidate tokens.
The capabilities are real and measured: translation, summarization, question answering, code generation, in-context learning (performing a new task from a few examples in the prompt), tool use, and steadily improving performance on reasoning benchmarks. They are also bounded. LLMs hallucinate — they emit confident, fluent statements that are false — because they optimize for plausible continuation, not truth; their knowledge is frozen at the training cutoff; they inherit and amplify biases present in their training data; and their context windows, while growing, still limit how much information they can consider at once. Evaluation is itself an open problem: benchmark scores are useful but gameable, and human evaluation is expensive and noisy.
The societal questions are live rather than settled: misinformation and impersonation, copyright and data-licensing disputes over training corpora, the environmental and economic costs of training and serving, privacy when users feed data to API endpoints, and the design of governance and regulation — the European Union's AI Act being the first major legislative framework. The research community continues to study alignment, interpretability, and safety, with no consensus yet on how far the scaling approach can go.
LLMs are tools with known failure modes, and the practical discipline is to treat their output as draft material that must be checked — the same stance the gor.bio wiki itself takes toward its AI-generated content. Their economic and technical significance, however, is not in dispute: they are the first general-purpose language technology, and they are already embedded in search, programming, writing, and countless other workflows.
Tags
deep learning natural language processing transformers