What is the key difference between full fine-tuning and parameter-efficient fine-tuning (PEFT) methods such as LoRA?

The key difference between full fine-tuning and parameter-efficient fine-tuning (PEFT) methods like LoRA lies in the number of model parameters that are updated during the fine-tuning process. Full fine-tuning involves updating all the parameters of the pre-trained model, which can be computationally expensive and require a significant amount of memory, especially for large models. Parameter-efficient fine-tuning methods, on the other hand, only update a small subset of the model's parameters, significantly reducing the computational cost and memory footprint. LoRA (Low-Rank Adaptation) achieves this by introducing low-rank matrices to the existing weights of the model and only training these newly introduced, much smaller matrices. The original pre-trained weights remain frozen. This allows for efficient adaptation of the model to a specific task while minimizing the resources required. For instance, instead of updating billions of parameters in a large language model, LoRA might only update a few million, making fine-tuning feasible on less powerful hardware and with smaller datasets. The core benefit is achieving comparable performance to full fine-tuning with significantly reduced computational and storage overhead.