Sliding window attention is an attention pattern where each token only attends to a fixed-size local window of nearby tokens, reducing memory from quadratic to linear in sequence length. By stacking multiple layers with sliding windows, information can propagate across the full sequence while each individual layer remains efficient. This approach is used in architectures like Mistral and Longformer, and a 2026 paper by Microsoft researcher Alexia Jolicoeur-Martineau showed that pairing it with attention sinks beats retrofitting a model to linear attention during post-training.