Light-PEFT

Here is the implementation of our Findings of ACL2024 paper "Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning".

Setup

pip install -r requirements.txt
cd peft
pip install -e .

Run Experiments

Estimation

bash scripts/estimate.sh

More Efficient Fine-Tuning

bash scripts/train.sh

Vanilla Fine-Tuning

bash scripts/vanilla_train.sh

Thanks for the open source code of  EarlyBert, LLM-Adapters, and P-Tuning v2. Some of our codes are based on them.