🤗 PEFT welcomes new merging methods — LoRA adapter merging improvements

AI Impact Summary

The 🤗 PEFT library has introduced new merging methods for LoRA adapters, primarily focusing on improving the efficiency and flexibility of combining adapter weights. These methods – concatenation (cat), linear, SVD, and TIES – offer different approaches to merging adapters, allowing users to experiment with various techniques to optimize performance. The new methods provide a more seamless experience when swapping in and out different adapters, and support LoRA adapters with different ranks.