注意力机制

目录

注意力机制#

Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity#

Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation#

Training-free and Adaptive Sparse Attention for Efficient Long Video Generation#

DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Training#

MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention#

FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion#

VORTA: Efficient Video Diffusion via Routing Sparse Attention#

Training-Free Efficient Video Generation via Dynamic Token Carving#

RainFusion: Adaptive Video Generation Acceleration via Multi-Dimensional Visual Redundancy#

Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation#

VMoBA: Mixture-of-Block Attention for Video Diffusion Models#

SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference#

Fast Video Generation with Sliding Tile Attention#

PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models#

Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light#

Astraea: A GPU-Oriented Token-wise Acceleration Framework for Video Diffusion Transformers#

∇NABLA: Neighborhood Adaptive Block-Level Attention#

Compact Attention: Exploiting Structured Spatio-Temporal Sparsity for Fast Video Generation#

A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention#

Bidirectional Sparse Attention for Faster Video Diffusion Training#

Mixture of Contexts for Long Video Generation#

LoViC: Efficient Long Video Generation with Context Compression#

MagiAttention: A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Mask Training#

DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidance#

XAttention: Block Sparse Attention with Antidiagonal Scoring#

VSA: Faster Video Diffusion with Trainable Sparse Attention#

QuantSparse: Comprehensively Compressing Video Diffusion Transformer with Model Quantization and Attention Sparsification#

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention#