LoRA

Fine-Tuning Language Models on Domain-Specific Data

Fine-tune language models on your domain with parameter-efficient LoRA and QLoRA: dataset preparation, training strategies and evaluation, on a small budget.