Description
I'm separating this from #1502. While we can get rid of cat in some cases, improving nvFuser's codegen for slice and cat will still benefit the RoPE module and the QKV split around it.
https://github.com/NVIDIA/Fuser/blob/bug1597/qkv_split_rope.py has the fusion definition for the forward pass.
Below is a whiteboard illustration for convenience:
Legend:
- S = slice
- C = concat
- R = reshape/view
- T = permute
Metadata
Metadata
Assignees
Labels
No labels