An efficient strategy for fine-tuning large language models.

An observed trend with the LoRA and QLoRA methods is that the ratio of LoRA Rank to LoRA Alpha among the top-performing models is, with two exceptions, either a 2:1 or 4:1 ratio.

also in 258 other papers