Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization

The official implementation of "Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization" now lives in the Hugging Face PEFT library.

For the usage guide, see this page to learn how to use BOFT in PEFT.

Citing our work

  @InProceedings{liu2024boft,
      author = {Liu, Weiyang and Qiu, Zeju and Feng, Yao and Xiu, Yuliang and Xue, Yuxuan and Yu, Longhui and Feng, Haiwen and Liu, Zhen 
        and Heo, Juyeon and Peng, Songyou and Wen, Yandong and Black, Michael J. and Weller, Adrian and Sch{\"o}lkopf, Bernhard},
      title = {Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization},
      booktitle = {ICLR},
      year = {2024}
  }