Large Language Models (LLMs) power chatbots, code assistants, search, and countless other products. Training one sounds mysterious (billions of parameters, GPU clusters, weeks of compute), but the ideas are learnable, and you can run a real training job on your laptop or a single GPU using open-source tools.
This guide starts from zero: what an LLM is, the vocabulary you will hear in every paper and tutorial, how full training differs from fine-tuning, and a complete hands-on example fine-tuning DistilGPT-2 (a small open-source model from Hugging Face) on your own text with LoRA, a modern, efficient training method.
1. What Is a Large Language Model?
An LLM is a neural network trained to predict the next piece of text (usually the next token) given everything that came before. After seeing enormous amounts of text during training, it learns grammar, facts, reasoning patterns, and style, not by memorizing sentences blindly, but by compressing statistical structure into billions of adjustable numbers called parameters (weights).
Most modern LLMs use the Transformer architecture (Vaswani et al., 2017). Transformers rely on self-attention: each token can “look at” other tokens in the sequence to decide what context matters. That design scales well and is why models like GPT-2, Llama, Mistral, and Phi are all transformer-based.
2. Essential Terms (Glossary)
| Term | Plain-English meaning |
|---|---|
| Token | A chunk of text (word, sub-word, or symbol). “Hello” might be one token; “unhappiness” might split into pieces. |
| Tokenizer | Converts raw text ↔ token IDs the model understands. |
| Pre-training | Training from random weights on huge general corpora (web, books). Teaches “language” broadly. Very expensive. |
| Fine-tuning | Continuing training on a smaller, task-specific dataset to adapt behavior (support tone, domain, format). |
| Instruction tuning | Fine-tuning on (instruction, response) pairs so the model follows user prompts. |
| RLHF | Reinforcement Learning from Human Feedback: humans rank outputs; a reward model guides further tuning (used in many chat products). |
| Loss | A number measuring prediction error. Training minimizes loss. |
| Epoch | One full pass over your training dataset. |
| Batch size | How many examples the optimizer updates on at once. |
| Learning rate | Step size for weight updates. Too high = unstable; too low = slow or stuck. |
| LoRA | Low-Rank Adaptation: train tiny adapter matrices instead of all weights; fast and light on memory (Hu et al., 2022). |
| PEFT | Parameter-Efficient Fine-Tuning: family of methods including LoRA (Hugging Face PEFT library). |
| Checkpoint | Saved model weights at a point in training. |
| HF Hub | Hugging Face model & dataset hosting, where we download DistilGPT-2. |
3. Three Ways People Say “Train an LLM”
- Pre-train from scratch: You need terabytes of text, hundreds of GPUs, and months. Organizations like Meta, Mistral AI, and EleutherAI do this. Not practical for most individuals.
- Full fine-tuning: Update every parameter of an existing model on your data. Better quality possible, but heavy VRAM (often 40GB+ for 7B models).
- PEFT / LoRA fine-tuning: Update ~0.1–2% of parameters. Fits consumer GPUs; ideal for learning and many production adapters. Our hands-on section uses this.
4. The LLM Training Pipeline (Big Picture)
Even when you only fine-tune, it helps to see where your work sits in the full lifecycle:
LLM lifecycle (conceptual flow)
What happens inside one training step?
- Load a batch of text sequences.
- Tokenizer converts them to input IDs + attention masks.
- Model predicts next token at each position.
- Compute cross-entropy loss vs. true next tokens.
- Backpropagation computes gradients.
- Optimizer (e.g., AdamW) updates weights (or LoRA adapters only).
5. What You Need Before the Hands-On Lab
- Python 3.10+
- GPU recommended (NVIDIA with CUDA). CPU works for DistilGPT-2 but is slow.
- ~4–8 GB disk for model + libraries
- Accounts: Hugging Face (free); accept model licenses if required
Step 0: Create a project folder and virtual environment
mkdir llm-finetune-lab && cd llm-finetune-lab
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install --upgrade pip
Step 1: Install libraries
pip install torch transformers datasets peft accelerate bitsandbytes evaluate
Note: On Windows without CUDA, install CPU PyTorch from pytorch.org. bitsandbytes is mainly for QLoRA on Linux; our DistilGPT-2 example uses standard LoRA in float32/float16.
6. Hands-On: LoRA Fine-Tune DistilGPT-2 on Custom Text
Model: distilgpt2 (82M parameters, GPT-2 architecture distilled by Hugging Face; Sanh et al., 2019; Radford et al., 2019).
Goal: Teach the model a repetitive style (e.g., short “product taglines”) so completions match your pattern.
Method: Causal language modeling + LoRA via peft and transformers.Trainer.
Step 2: Prepare training data
Create train.txt with one example per line (at least 50–200 lines for a visible effect):
Tagline: CloudSync - your files, everywhere, instantly.
Tagline: BrewMaster - coffee that wakes your ambition.
Tagline: FitTrack - steps today, strength tomorrow.
Tagline: CodeNest - build software in calm focus.
Each line is a self-contained training example. The model learns to continue text in the same format.
Step 3: Training script (train_lora.py)
import torch
from datasets import load_dataset
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
TrainingArguments,
Trainer,
DataCollatorForLanguageModeling,
)
from peft import LoraConfig, get_peft_model, TaskType
MODEL_ID = "distilgpt2"
OUTPUT_DIR = "./distilgpt2-tagline-lora"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
tokenizer.pad_token = tokenizer.eos_token
# Load plain text file (one record per line)
dataset = load_dataset("text", data_files={"train": "train.txt"})
dataset = dataset["train"].train_test_split(test_size=0.1, seed=42)
def tokenize(batch):
return tokenizer(
batch["text"],
truncation=True,
max_length=128,
padding="max_length",
)
tokenized = dataset.map(tokenize, batched=True, remove_columns=["text"])
model = AutoModelForCausalLM.from_pretrained(MODEL_ID)
model.config.use_cache = False # required for gradient checkpointing compatibility
lora_config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
r=8,
lora_alpha=16,
lora_dropout=0.05,
target_modules=["c_attn"], # attention projection in GPT-2 blocks
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Example output: trainable params ~0.3% of total
training_args = TrainingArguments(
output_dir=OUTPUT_DIR,
num_train_epochs=3,
per_device_train_batch_size=4,
per_device_eval_batch_size=4,
learning_rate=2e-4,
weight_decay=0.01,
logging_steps=10,
eval_strategy="epoch",
save_strategy="epoch",
load_best_model_at_end=True,
fp16=torch.cuda.is_available(),
report_to="none",
)
collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized["train"],
eval_dataset=tokenized["test"],
data_collator=collator,
)
trainer.train()
model.save_pretrained(OUTPUT_DIR)
tokenizer.save_pretrained(OUTPUT_DIR)
print("Saved LoRA adapter to", OUTPUT_DIR)
Step 4: Run training
python train_lora.py
Watch the console: loss should generally decrease across epochs. If loss is NaN, lower the learning rate (try 5e-5).
Step 5: Inference (test your fine-tuned model)
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
BASE = "distilgpt2"
ADAPTER = "./distilgpt2-tagline-lora"
tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
base = AutoModelForCausalLM.from_pretrained(BASE)
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()
prompt = "Tagline: SolarGrid -"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
out = model.generate(**inputs, max_new_tokens=24, do_sample=True, top_p=0.9)
print(tokenizer.decode(out[0], skip_special_tokens=True))
You should see completions that resemble your training lines. Quality depends on data size, diversity, and epochs; this is a teaching setup, not production scale.
7. Scaling Up: Same Pattern for Llama / Mistral (7B+)
The steps are identical; only configuration changes:
- Swap
MODEL_IDto e.g.meta-llama/Meta-Llama-3-8B(gated; request access on Hugging Face). - Use
target_modules=["q_proj", "v_proj", "k_proj", "o_proj"]for Llama-style architectures. - Add QLoRA: load model in 4-bit (
BitsAndBytesConfig) to fit 8–16 GB VRAM. - Use chat templates and JSONL with
messagesfor instruction tuning. - Consider
trl.SFTTrainerfor supervised fine-tuning with less boilerplate.
Fine-tuning decision flow
Start → Do you have a strong base model on Hugging Face?
No → use an open model (DistilGPT-2, TinyLlama, Phi-3-mini).
Yes → Is VRAM < 16 GB?
Yes → QLoRA + small batch + gradient accumulation.
No → LoRA or full fine-tune; increase batch size carefully.
Evaluate on held-out prompts → iterate data & hyperparameters.
8. Evaluation and Quality Checks
- Perplexity on a held-out set: lower often means better fit (watch for overfitting).
- Human review: sample 20–50 prompts; score helpfulness, factuality, tone.
- Benchmarks (optional): MMLU, HellaSwag for general models; custom JSON tests for your domain.
- Safety: fine-tuning can amplify biases or jailbreaks; test edge cases before deploy.
9. Common Mistakes (and Fixes)
| Problem | Likely cause | Fix |
|---|---|---|
| CUDA OOM | Batch too large / full precision 7B+ | QLoRA, smaller batch, gradient accumulation, shorter max_length |
| Model repeats training lines | Overfitting, tiny dataset | More data, fewer epochs, dropout, early stopping |
| No visible change after training | LR too low, wrong layers targeted | Verify trainable params; increase LR slightly; check data format |
| Garbled output | Tokenizer mismatch | Load tokenizer from same checkpoint as weights |
10. Responsible Training Notes
Training on scraped data raises copyright and privacy issues. Fine-tuning on customer data requires consent and secure storage. Open models ship with licenses (Apache 2.0, Llama Community License, etc.); read them before commercial use. Document your dataset, hyperparameters, and evaluation for reproducibility (Mitchell et al., 2019).
Training a frontier LLM from scratch is a datacenter-scale project, but training in the practical sense (fine-tuning an open model on your data) is something you can do in an afternoon. You learned the vocabulary, the lifecycle from pre-training to deployment, and you walked through LoRA fine-tuning of DistilGPT-2 with Hugging Face. From here, swap in a larger base model, structured instruction data, and QLoRA when you are ready for real domain assistants.
References
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.
- Hugging Face. (2024). PEFT: Parameter-efficient fine-tuning. Hugging Face Documentation. https://huggingface.co/docs/peft
- Hugging Face. (2024). Transformers documentation. Hugging Face. https://huggingface.co/docs/transformers
- Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low-rank adaptation of large language models. Proceedings of the International Conference on Learning Representations. https://arxiv.org/abs/2106.09685
- Mitchell, M., Wu, S., Zanzani, M., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019). Model cards for model reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency, 220-229. https://doi.org/10.1145/3287560.3287596
- Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730-27744.
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
- Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv. https://arxiv.org/abs/1910.01108 (DistilGPT-2 follows the same distillation approach for GPT-2.)
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998-6008.
- Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., ... & Rush, A. M. (2020). Transformers: State-of-the-art natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 38-45. https://doi.org/10.18653/v1/2020.emnlp-demos.6
