LLM: Complete Guide to Large Language Models

Summary

LLMs (Large Language Models) represent one of the most spectacular advances in contemporary artificial intelligence. These models, trained on colossal volumes of textual data, are capable of generating coherent text, answering complex questions, summarizing documents, translating between languages, and performing many additional language tasks without additional specific training. This complete guide explores the underlying transformer architecture, training mechanisms, RLHF fine-tuning, and provides a concrete Python implementation with Hugging Face so you can experiment for yourself. Whether you are a developer, researcher, or simply curious about AI technologies, this guide will give you a thorough understanding of what an LLM really is.

Mathematical Principle

Decoder-Only Transformer Architecture

Modern LLMs are based on the decoder-only transformer architecture, an evolution of the original transformer proposed by Vaswani et al. in 2017. Unlike the encoder-decoder transformer initially designed for machine translation, new-generation LLMs (GPT-3, GPT-4, LLaMA, Claude, PaLM) use only the decoder part, optimized for sequential text generation.

The core mechanism is causal attention (or masked attention). At each position i in the sequence, the model can only look at previous tokens (positions 1 to i-1). Mathematically, for an input sequence X = (x₁, x₂, …, xₙ), causal attention is computed as follows:

Attention(Q, K, V) = softmax(QKᵀ / √dₖ + M) · V

where M is a lower-triangular masking matrix filled with −∞ above the diagonal, ensuring that the prediction at time t depends only on preceding tokens. This causal constraint is fundamental: it prevents the model from “cheating” by looking at future tokens during training.

Each layer of the transformer contains:

• Multi-Head Self-Attention: multiple attention heads computed in parallel, each head learning different relational patterns between tokens.
• Feed-Forward Network (FFN): a two-layer linear network with non-linear activation (usually GELU or SwiGLU), applied independently to each position.
• Layer Normalization and residual connections: stabilize deep training, enabling models with hundreds of layers.

Pre-Training: Next Token Prediction

The training objective is conceptually simple: predict the next token. Given a sequence of tokens (t₁, t₂, …, tₙ), the model maximizes the likelihood of each token given its predecessors:

L(θ) = −Σᵢ log P(tᵢ | t₁, …, tᵢ₋₁; θ)

This cost function is the cross-entropy between the distribution predicted by the model and the true distribution (one-hot on the target token). Training data comprises billions of tokens — Common Crawl, Wikipedia, books, scientific articles, forums, source code — allowing the model to acquire implicit encyclopedic knowledge.

The vocabulary uses Byte Pair Encoding (BPE), a subword algorithm that decomposes rare words into more frequent units, reducing vocabulary size to approximately 30,000 to 100,000 tokens while covering any text, including in underrepresented languages.

Scaling Laws

The work of Kaplan et al. (2020) and then Hoffmann et al. (2022) revealed predictive scaling laws for LLM performance. The fundamental relationship follows a power law:

Performance ∝ (compute)^(0.047) × (data)^(0.107) × (params)^(0.339)

This means that:

• Doubling the parameters (model size) improves performance by approximately 26% (2^0.339 ≈ 1.26)
• Doubling the training data only brings about 8% improvement
• Doubling the total compute brings about 3% improvement

These results have major implications: it is more effective to increase model size than the quantity of data. The subsequent Chinchilla Laws nuanced this result by showing that there is an optimal ratio between model size and training data quantity, but the fundamental principle remains: size matters, and it matters more than initially thought.

RLHF: Fine-Tuning via Reinforcement Learning from Human Feedback

Pre-training alone produces a model capable of generating text, but not necessarily useful, honest, and harmless text. RLHF adds a crucial post-training step in three phases:

1. Supervised Fine-Tuning (SFT): the model is retrained on a small set of high-quality demonstrations written by human annotators, teaching it to follow instructions rather than simply continuing text.
2. Reward Model (RM) training: annotators rank multiple model responses by preference. A separate reward model is trained to predict which response a human would prefer, using a Bradley-Terry type loss function:

L_RM(φ) = −E[(x,y_w,y_l)] ~ D [log σ(r_φ(x, y_w) − r_φ(x, y_l))]

3. PPO Optimization (Proximal Policy Optimization): the language model is optimized via reinforcement learning to maximize the predicted reward, while maintaining KL regularization toward the original model to prevent degradation:

L_PPO = E[r_φ(x, y) − β · KL(π_θ || π_ref)]

This process transforms a raw model, sometimes toxic or unhelpful, into a competent and polite assistant.

Intuition: The Librarian and the Literary Critic

To understand what an LLM is without getting lost in equations, imagine a librarian who has read absolutely every book in the world. This librarian does not know the truth in the philosophical sense — they have never experienced the world, never felt the wind or tasted a fruit. But they know language patterns like no one else. When you ask them a question, they don’t search for an answer in their memory: they predict what sounds right, drawing on the billions of sentences they read during pre-training.

This is both powerful and limiting. The LLM doesn’t “know” anything in the way we understand knowledge: it produces statistically plausible text, not necessarily true text. Its hallucinations are not deliberate lies — they are confident extrapolations of a system that only continues linguistic patterns.

RLHF, in this analogy, is like adding a demanding literary critic next to the librarian. This critic doesn’t change what the librarian has read, but they teach them good taste, politeness, nuance. They say: “Don’t be vulgar. Be useful. Admit when you don’t know.” The result is a model that seems smarter, but whose computational base remains unchanged — only the generation policy has been refined.

Python Implementation

1. Generation Pipeline with Hugging Face

The simplest way to use an LLM is the text-generation pipeline from the Hugging Face transformers library:

from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
import torch

# Load the model (here a small open-source model)
model_name = "meta-llama/Llama-2-7b-chat-hf"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Generation pipeline
generator = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    device_map="auto"
)

# Simple generation
prompt = "Explain the concept of attention in transformers to me."
result = generator(prompt, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(result[0]["generated_text"])

2. Few-Shot Prompting

Few-shot prompting involves providing a few examples in the prompt to guide the model’s behavior without modifying its weights:

def few_shot_prompt():
    prompt = """Classify the sentiment of each sentence as positive, neutral, or negative.

Sentence: "I love this new software, it's incredible!"
Sentiment: positive

Sentence: "The service was okay, nothing special."
Sentiment: neutral

Sentence: "This product is a total disappointment, I do not recommend it."
Sentiment: negative

Sentence: """
    return prompt

# With the model loaded above
new_text = few_shot_prompt() + '"The interface is beautiful, but the battery doesn\'t last."\nSentiment: '
result = generator(new_text, max_new_tokens=16, temperature=0.1)
print(result[0]["generated_text"])
# Expected result: "negative" or "neutral"

3. Fine-Tuning with PEFT/LoRA

Low-Rank Adaptation (LoRA) allows fine-tuning a large model by modifying only a tiny fraction of the parameters (often less than 1%):

from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM, TrainingArguments, Trainer, default_data_collator
from datasets import load_dataset

# LoRA configuration
lora_config = LoraConfig(
    r=16,                    # Rank of the decomposition
    lora_alpha=32,           # Scaling factor
    target_modules=["q_proj", "v_proj"],  # Layers to adapt
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM
)

# Load the base model
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Apply LoRA
model = get_peft_model(base_model, lora_config)
model.print_trainable_parameters()
# Output: "trainable params: 0.15% of total params"

# Prepare training data
dataset = load_dataset("csv", data_files="my_dataset.csv", split="train")

# Training arguments
training_args = TrainingArguments(
    output_dir="./llm-lora-finetune",
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    num_train_epochs=3,
    fp16=True,
    logging_steps=50,
    save_strategy="epoch",
    optim="adamw_torch",
    lr_scheduler_type="cosine",
    warmup_ratio=0.03,
)

# Fine-tuning
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    data_collator=default_data_collator,
)

trainer.train()
model.save_pretrained("./my-lora-model")

4. Model Evaluation

Evaluating an LLM requires several complementary metrics:

from transformers import pipeline
import json

def evaluate_llm(model_path, test_prompts):
    """Evaluates an LLM on a set of test prompts."""
    pipe = pipeline("text-generation", model=model_path, device_map="auto")
    results = []

    for prompt in test_prompts:
        result = pipe(
            prompt,
            max_new_tokens=128,
            temperature=0.7,
            do_sample=True
        )
        results.append({
            "prompt": prompt,
            "response": result[0]["generated_text"][len(prompt):],
            "length": len(result[0]["generated_text"])
        })

    # Save results
    with open("evaluation_results.json", "w", encoding="utf-8") as f:
        json.dump(results, f, ensure_ascii=False, indent=2)

    return results

# Usage example
test_prompts = [
    "Summarize in three sentences: what is photosynthesis?",
    "Write a poem about autumn in four lines.",
    "What is the difference between an LLM and a classic neural network?",
    "Translate into English: 'Sustainable development is essential.'",
]

results = evaluate_llm("./my-lora-model", test_prompts)
for r in results:
    print(f"\nPrompt: {r['prompt']}")
    print(f"Response ({r['length']} tokens): {r['response']}")

Generation Hyperparameters

Decoding hyperparameters control how the model generates text. Understanding them is essential to obtaining consistent results suited to your use case.

Temperature

Temperature controls the randomness of predictions. Mathematically, it divides the logits before the softmax:

Pᵢ = exp(logitᵢ / T) / Σⱼ exp(logitⱼ / T)

• T → 0: the model becomes deterministic, always choosing the most likely token (greedy decoding). The text is coherent but can be repetitive and dull.
• T = 0.7: good compromise for most generative tasks. The text remains coherent while being varied.
• T → 1.0: the model is more creative, but risks producing incoherent or off-topic text.
• T > 1.0: not recommended for LLMs, as it introduces too much randomness and significantly degrades quality.

Top-p (Nucleus Sampling)

Top-p (or nucleus sampling) limits selection to tokens whose cumulative probability reaches a threshold p. For example, with top_p = 0.9, only the most likely tokens whose probability sum reaches 90% are considered.

The advantage over top-k (which takes the k most likely tokens) is that the number of candidates adapts dynamically: when the model is very confident, few tokens are considered; when it is uncertain, more tokens enter the nucleus.

max_new_tokens

Controls the maximum length of the generated response (excluding the prompt). Note: this is not a length guarantee — the model may stop earlier if it generates an end-of-sequence (EOS) token.

repetition_penalty

Penalizes tokens already generated previously, reducing repetition. A value of 1.0 disables the penalty. Typical values are 1.1 to 1.3 — beyond that, the model may refuse to repeat necessary words, which degrades readability.

num_return_sequences

Generates multiple independent responses for the same prompt, useful for exploration or for selecting the best response via subsequent scoring. Each sequence is sampled independently.

# Example combining multiple hyperparameters
result = generator(
    prompt,
    max_new_tokens=512,
    temperature=0.8,
    top_p=0.92,
    repetition_penalty=1.15,
    num_return_sequences=3,
    do_sample=True
)
for i, seq in enumerate(result):
    print(f"--- Sequence {i+1} ---")
    print(seq["generated_text"])

Advantages and Limitations of LLMs

Advantages

• Remarkable versatility: a single model can answer questions, write code, translate, summarize, analyze, and much more — without task-specific fine-tuning.
• Emergent capabilities: beyond a certain size threshold (typically several tens of billions of parameters), capabilities absent from smaller models appear: chain-of-thought reasoning, code understanding, elementary mathematical reasoning.
• Zero-shot transfer: LLMs can perform tasks they have never explicitly seen during training, simply through the richness of their internal representations.
• Natural interface: interaction is done in natural language, the most intuitive means of communication for humans.
• Mature ecosystem: Hugging Face, LangChain, Ollama, and other tools make LLMs accessible even to non-specialist developers.

Limitations

• Hallucinations: LLMs can invent facts with absolute confidence. They do not distinguish truth from linguistic plausibility.
• Bias: models reflect and amplify biases present in their training data (gender, racial, cultural stereotypes).
• Computational cost: inference of a large LLM requires powerful GPUs, and training from scratch costs millions of dollars.
• Limited context window: even the most recent models have a context limit (32K to 128K tokens for the best), which restricts the amount of information they can process in a single pass.
• Frozen knowledge: the model’s knowledge is limited to the training data, with a cutoff date. Without external tools (ReAct, RAG), the model cannot access recent information.
• Lack of deep reasoning: despite appearances, LLMs do not reason in the logical sense of the term. They excel at pattern recognition but fail on tasks requiring rigorous deduction or long-term planning.

Four Concrete Use Cases

1. Intelligent Writing Assistant

An LLM can assist writers by generating drafts, suggesting rewrites, adapting tone and style. A digital newspaper could use an LLM fine-tuned on its own corpus to produce article drafts that journalists only need to polish. Productivity gains are estimated at 30 to 50% according to recent studies on the impact of generative AI in writing professions.

2. Customer Support Chatbot

Integrating an LLM into a customer support system makes it possible to automatically handle complex requests that exceeded the capabilities of rule-based chatbots. The LLM understands the customer’s intent, consults a knowledge base via RAG (Retrieval-Augmented Generation), and formulates a precise and personalized response, while escalating to a human agent if necessary. This approach drastically reduces wait times and improves customer satisfaction.

3. Legal Document Analysis

In the legal domain, LLMs can analyze contracts, identify risky clauses, compare documents, and extract structured information. A law firm could use a specialized LLM to pre-analyze hundreds of pages of documents, leaving only verification and strategy to the lawyers. Fine-tuning on a specific legal corpus is essential here for terminological precision and understanding of contractual nuances.

4. Code Generation and Explanation

Code-specialized LLMs (such as Codex, StarCoder, or CodeLlama) can generate complete functions from natural language descriptions, explain legacy code, detect bugs, write unit tests, and even refactor entire codebases. This is probably the use case where the cost-benefit ratio is most favorable, as code is a domain where automatic verification is possible (execution, tests, linting). Developers report productivity gains of 20 to 55% according to studies conducted by GitHub and independent firms.

See Also

• Enumerating Submasks of a Bitmask in Python: Complete Guide and Practical Tips
• Mastering Intersections in Python: Techniques and Tips to Optimize Your Algorithms