This article was automatically translated from the original Turkish version.

Training a multibillion-parameter language model to solve a problem specific to our project is a great idea. But are we aware of the memory cost this imposes on our personal computers?
Consider this: If we attempt to train an 8 billion parameter model using conventional methods (full fine-tuning), the model weights alone require a direct 16 GB of GPU memory (VRAM). When we factor in the optimization data and mandatory computational checkpoints that must be maintained in memory during training, we encounter a massive hardware barrier—128 GB or more—that is impossible to overcome on standard computers.
In this article, we will examine the LoRA and QLoRA techniques developed to overcome this hardware barrier. Let us step by step explore how multibillion-parameter models can be fitted into vastly less memory.
Large language models (LLMs) are trained on vast datasets from the internet and acquire general language capabilities. However, this general ability is not always sufficient to solve problems specific to our projects.
We can compare the fine-tuning process to a recently graduated intelligent engineer: The engineer has a strong foundation in engineering principles but needs company-specific orientation training to understand your organization’s unique software architecture. We call the process of specializing a model in a particular domain by training it on your own data fine-tuning.
Full fine-tuning means updating every single neuron in the model’s “brain”—that is, all billions of parameters—from scratch during this specialization training. Imagine the model as a massive 10,000-page encyclopedia. To update just a few pieces of information according to our project’s needs, we are attempting to reprint the entire encyclopedia from the beginning.
This process demands enormous computational power. During training, gradients (learning directions) are computed, and optimization algorithms (such as AdamW) retain original copies, momentums, and variances of all these billions of parameters in memory to prevent learning degradation. This is precisely where we hit the “128 GB GPU” wall mentioned earlier.
LoRA is defined in its original paper by Microsoft researchers as: “A method that significantly reduces the number of trainable parameters by freezing the pre-trained model weights and injecting trainable low-rank matrices into each layer of the Transformer architecture for downstream tasks.”

Small B and A Matrices Trained with the Frozen W₀ Matrix in LoRA (Generated by Artificial Intelligence.)
Let us make this much clearer using our encyclopedia metaphor: LoRA tells us not to reprint the entire 10,000-page encyclopedia. Instead, lock the original pages as they are (freeze them) and simply add small, transparent sticky notes (adapter matrices) only where updates are needed.
Mathematically, this is like magic. Suppose the original model contains a massive weight matrix of size 4096 x 4096 (approximately 16.7 million parameters). Rather than updating this entire matrix, LoRA inserts two tiny multiplicative matrices of rank r=8 (4096 x 8 and 8 x 4096) in between. This reduces the number of parameters to be trained from 16.7 million to just 65,000. We achieve a 99.6% reduction in parameters with almost no loss in model quality. We are now free from the daunting 128 GB optimization burden.
Moreover, this technique has another excellent inference advantage: After training, we can permanently merge these small “sticky notes” (adapter weights) into the original encyclopedia pages. This means that during real-world use (inference), the system does not need to manage two separate sets of data. The model continues operating at its original speed without any additional latency or slowdown.
Thanks to LoRA, we freed ourselves from the largest portion of that massive 128 GB load: the optimization data (AdamW weights). Yet one final obstacle remains: the frozen 10,000-page encyclopedia (the original model weights) still occupies 16 GB in 16-bit format. For an average personal computer, 16 GB of VRAM is already a demanding limit.

Reduction of 16-bit Weights to 4 Bits via Quantization (Generated by Artificial Intelligence.)
It is precisely at this point that the QLoRA paper, published in 2023 by researchers at the University of Washington (Dettmers and colleagues), comes to our aid. QLoRA compresses this massive frozen original model into a special format called 4-bit NormalFloat (NF4) — a process known as quantization.
Let us think of this using our encyclopedia metaphor: we are rewriting all the text in the 10,000-page encyclopedia using a specialized shorthand language with abbreviations that take up far less space. As a result, our model, which originally occupied 16 GB in 16-bit format, shrinks instantly to just 4 to 5 GB when compressed to 4-bit.
But if we compress the data this much, won't the model's intelligence and precision be lost? The real engineering magic behind QLoRA begins here: the model resides in the GPU's memory in its compressed 4-bit form, occupying only a tiny amount of space. Yet during the milliseconds when the model performs learning computations, the numbers are temporarily converted back to 16-bit (BF16) format for calculation. This is analogous to how our minds instantly reconstruct full, flawless sentences from shorthand notes while reading. In this way, we achieve massive memory savings while preventing computational errors (NaN values) and retaining nearly all of the model's accuracy.

Approximate Memory Requirements in Full Fine-Tuning, LoRA, and QLoRA (Generated by Artificial Intelligence.)
When LoRA and QLoRA techniques are combined, they break the monopoly of massive hardware requirements in the AI ecosystem. Models with billions of parameters that yesterday could only be trained on the colossal servers of tech giants can now fit onto standard gaming computers in our homes, thanks to cleverly designed small adapter notes (LoRA) and data compression algorithms resembling shorthand (QLoRA).
Moreover, you do not need to write complex code from scratch to implement all of this. Today, with Hugging Face's PEFT (Parameter-Efficient Fine-Tuning) and BitsAndBytes libraries, you can achieve this magical optimization with just a few lines of Python code. Training your own personal assistant, coding helper, or a specialized model tailored to your data is no longer a dream — it is now within reach in your own room.
Dettmers, Tim, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. “QLoRA: Efficient Finetuning of Quantized LLMs.” Preprint, arXiv, 2023. Accessed November 8, 2026.https://arxiv.org/abs/2305.14314.
Hu, Edward J., Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. “LoRA: Low-Rank Adaptation of Large Language Models.” Preprint, arXiv, 2021. Accessed November 8, 2026.https://arxiv.org/abs/2106.09685.
Hugging Face. “Quantization.” PEFT Documentation. Accessed November 8, 2026.https://huggingface.co/docs/peft/developer_guides/quantization.
What Is Fine-Tuning and What Is Our Goal?
What Is Full Fine-Tuning and Why Do We Struggle?
What Is LoRA (Low-Rank Adaptation)?
Reducing the Remaining Load with QLoRA (Quantized LoRA)
Conclusion: The Revolution That Brought Artificial Intelligence Into Our Homes