scPEFT

This is the official repository for Harnessing the Power of Single Cell Large Language Models with Parameter Efficient Fine-Tuning using scPEFT. To reproduce the results from the paper, please visit scPEFT_reproduction.

 

Installation

scPEFT works with Python >= 3.7.13. scPEFT is available on PyPI. To install scPEFT, run the following command:

pip install scpeft

For developing, run the following command:

git clone https://github.com/coffee19850519/scPEFT
cd scPEFT

Note: scPEFT is currently built on top of scGPT, scBERT, and Geneformer. Please follow their installation instructions to ensure all necessary versioned dependencies are installed. We provide a requirements. ymal file for the environment in which scPEFT was developed.

Get Started

1. Download the backbone model, e.g., scGPT model checkpoint and place it at e.g., work_dir/scPEFT/save. We recommend using the whole-human model for most applications by default, which pretrained on 33 million normal human cells.

2. The tutorials of scPEFT for downstream tasks in tutorial_peft. Here are the links to the downstream tasks and tutorials mentioned in our article

   Downstream task	Link
   cell type identification	Tutorial_Identification.ipynb
   batch correction	Tutorial_BatchCorrection.ipynb
   perturbation	Tutorial_Perturbation.ipynb
   cell population discovery	Tutorial_CellPopulationDiscovery.ipynb
   marker gene detection	Tutorial_MarkerGeneDetection.ipynb

To-do-list

• Publish to pypi
• Adapting scPEFT for native-attention
• Adapting scPEFT for flash-attention
• Only retain PEFT-related parameters when saving peft-model weights.

Contributing

We greatly welcome contributions to scPEFT. Please submit a pull request if you have any ideas or bug fixes. We also welcome any issues you encounter while using scPEFT.

Built With

We sincerely thank the authors of following open-source projects:

• scGPT
• Geneformer
• scBERT
• scanpy
• scib
• pytorch

Citing scPEFT

@article {He2025.04.21.649754,
	author = {He, Fei and Fei, Ruixin and Krull, Jordan E. and Zhang, Xinyu and Gao, Mingyue and Su, Li and Chen, Yibo and Yu, Yang and Li, Jinpu and Jin, Baichuan and Chang, Yuzhou and Ma, Anjun and Ma, Qin and Xu, Dong},
	title = {Harnessing the Power of Single-Cell Large Language Models with Parameter Efficient Fine-Tuning using scPEFT},
	year = {2025},
	doi = {10.1101/2025.04.21.649754},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2025/04/23/2025.04.21.649754},
	journal = {bioRxiv}
}