The Practitioner's LLM Curriculum ← Week 3 · LoRA Parameter Calculator
Interactive · Week 3 · Section 3

Why QLoRA changed everything.

LoRA trains 0.1–1% of parameters. QLoRA does it on top of a 4-bit-quantized base model. Together they collapse the memory footprint enough to fine-tune a 70B model on a single H100 — or an 8B model on consumer hardware. Pick a base model, set the rank, choose your target modules, and watch the memory bar shrink as you turn quantization on.

Configuration
LoRA rank 16
Higher rank = more capacity, more parameters. 16 is the canonical default; 4–8 for narrow tasks, 32–64 when LoRA underfits.
Base model precision 4-bit (QLoRA)
4-bit (NF4) is the QLoRA default. LoRA adapters always stay in fp16/bf16 regardless of base.
Target modules
All-attention + all-MLP is the canonical "apply to everything" setup. Attention-only is leaner but underperforms on most tasks.
Batch size 4
Sequence length 2048
Trainable parameters
—
Total VRAM needed
—

Memory breakdown

Hardware fit

What's happening here

    Math uses standard formulas: LoRA per-layer = rank × (d_in + d_out) per target module; gradients track only trainable params; Adam optimizer holds two states (assumed 8-bit when base is quantized, fp32 otherwise); activations estimated as batch × seq × hidden × ~16 bytes with gradient checkpointing on. Real-world overhead from CUDA kernels, framework state, and KV cache during eval will add another ~10–20%. For exact numbers, profile the actual run. Source-of-truth lives as <LoRAParameterCalculator/> in the curriculum codebase.
    The Practitioner's LLM Curriculum · Venkatesh Baglodi · all weeks · widgets · about