Trainable, memory-efficient, and GPU-friendly PyTorch reproduction of AlphaFold 2

Stick-breaking attention

Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"

Official repository for paper: Exploring the Low-Pass Filtering Behavior in Image Super-Resolution

[NeurIPS 2024] BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models

Development repository for the Triton language and compiler

Implementation for MatMul-free LM.

[ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

Code for the paper "Language Models are Unsupervised Multitask Learners"

Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models.

example showing finetuning of fuyu

End-to-End Autonomous Driving with Spiking Neural Networks

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).

Here we will test various linear attention designs.

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型，实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。

Official repository for paper: Tensor Decomposition Based Attention Module for Spiking Neural Networks

LLM training in simple, raw C/CUDA

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators

RWKV model implementation

[NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Sequence Modeling

Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening

This is the official implementation of the 'SFOD: Spiking Fusion Object Detection'.

Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures