How to Read: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows [Reading Guide]

This guide will help you effectively read and understand "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" using the three-pass approach.

First pass (5–10 min)

The goal of the first pass is to get a high-level overview of the paper's contributions, the problem it addresses, and its main findings.

• What to skim:

  • Title: "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" – Immediately tells you it's about a new Transformer for vision, focusing on "hierarchical" and "shifted windows."
  • Abstract: Read carefully. It summarizes the problem (Transformer challenges in vision), the proposed solution (hierarchical, shifted windows), key benefits (efficiency, cross-window connection, multi-scale modeling, linear complexity), and impressive results across various vision tasks.
  • Introduction (Section 1): Read the first and last paragraphs. The first paragraph sets the stage by discussing CNNs vs. Transformers in vision. The last paragraph reiterates the paper's goal: a general-purpose Transformer backbone for vision.
  • Conclusion (Section 5): Read to grasp the authors' summary of their work and its potential impact.
  • Figures and Tables: Quickly glance at Figure 1 to understand the high-level difference between Swin Transformer and previous Vision Transformers (ViT) – hierarchical vs. single-resolution feature maps.

• What to notice:

  • Contributions: The Swin Transformer is presented as a general-purpose vision backbone. Its key innovations are a hierarchical architecture and shifted windowing scheme. These lead to linear computational complexity and cross-window connections.
  • Problem: Existing Transformers (like ViT) struggle with large scale variations of visual entities and high image resolutions, leading to quadratic computational complexity.
  • Setting: Computer Vision, specifically image classification (ImageNet-1K), object detection (COCO), and semantic segmentation (ADE20K).
  • Key Results: The abstract highlights state-of-the-art performance on all three tasks, significantly outperforming previous methods.
  • Availability: Code and models are publicly available (mentioned in Abstract and Section 1).

Second pass (up to 1 hour)

Using SpatialRead in the Second Pass

• Use the Screenshot tool to capture figures, tables, diagrams, and formulae from the PDF and use AI for simple explanations.
• Ask: "Explain Figure X at a high level", "Compare Table 2 vs 3", or "What does the y-axis represent here?"
• Download SpatialRead →

This pass aims to understand the core concepts, methods, and main results without getting bogged down in every detail.

• What sections/figures to focus on:

  • Introduction (Section 1): Re-read to solidify the motivation and problem statement. Pay attention to how Swin Transformer addresses the limitations of previous Vision Transformers.
  • Related Work (Section 2): Skim the subsections to understand the context: CNNs, self-attention backbones, and other Transformer-based vision backbones. This helps position Swin Transformer against existing work.
  • 3. Method: This is crucial.
    • 3.1. Overall Architecture: Read carefully. Understand the patch splitting, linear embedding, and how hierarchical representation is built through patch merging layers. Refer to Figure 3(a) for the full architecture overview and Figure 1(a) again to visualize the hierarchical feature maps.
    • 3.2. Shifted Window based Self-Attention: This is the core innovation. Understand why self-attention in non-overlapped windows is efficient (linear complexity, see Equations 1 and 2). Crucially, study shifted window partitioning to see how cross-window connections are achieved. Figure 2 is essential here, illustrating the regular and shifted window partitions. Also, look at Figure 3(b) for the structure of two successive Swin Transformer Blocks (W-MSA and SW-MSA).
    • 3.3. Architecture Variants: Understand the different model sizes (Swin-T, Swin-S, Swin-B, Swin-L) and their hyper-parameters. Refer to Table 1 to see their corresponding FLOPs and throughput.
  • 4. Experiments: Focus on the main results and comparisons.
    • 4.1. Image Classification on ImageNet-1K: Review Table 1(a) and Table 1(b) to see how Swin Transformer variants compare to other Transformer and ConvNet backbones in terms of accuracy, parameters, FLOPs, and throughput.
    • 4.2. Object Detection on COCO: Look at Table 2(a), (b), (c) to understand the performance gains on object detection and instance segmentation.
    • 4.3. Semantic Segmentation on ADE20K: Check Table 3 for segmentation results.
    • 4.4. Ablation Study: Briefly review Table 4 (impact of shifted windows and relative position bias) and Table 5 (efficiency of different self-attention methods) to see which design choices were most impactful.

• Key terms/questions to note:

  • Hierarchical feature maps: How are they created?
  • Shifted windows: Why are they important? How do they enable cross-window connection while maintaining efficiency?
  • Linear computational complexity: How is it achieved compared to ViT's quadratic complexity?
  • Relative position bias: What is its role (Equation 4)?
  • Patch merging: What does it do?
  • W-MSA vs. SW-MSA: What's the difference?
  • How does Swin Transformer address the scale variation and high-resolution challenges in vision?

Third pass (1–5 hours)

Using SpatialRead in the Third Pass

• Highlight complex keywords, dense paragraphs, equations, or proofs and ask for a plain-English explanation that preserves notation.
• Use AI to explain the concepts in a way that is easy to understand.
• Ask for alternative phrasings or analogies to confirm understanding without changing technical meaning.
• Connect concepts with each other using AI to build a comprehensive understanding of the paper.
• Download SpatialRead →

This is a deep dive into the paper, understanding the intricacies of the methods, critically evaluating the results, and considering potential for replication.

• Deep reading guidance:

  • Methods (Section 3):
    • Re-read Section 3.1 and 3.2 with a focus on understanding the exact mechanics. Trace the data flow through the Swin Transformer Block in Figure 3(b).
    • Delve into the efficient batch computation for shifted configuration (Section 3.2, and Figure 4). Understand the cyclic-shifting mechanism and why it's efficient.
    • Examine the relative position bias (Section 3.2, Equation 4). Understand how it's incorporated and why it's beneficial (as shown in Table 4).
    • Review the detailed architecture specifications in Appendix A1 and Table 7 to understand the exact layer numbers and dimensions for each Swin Transformer variant.
  • Results (Section 4):
    • Critically analyze the experimental settings in Sections 4.1, 4.2, 4.3 and Appendix A2. Understand the training procedures, optimizers, learning rate schedules, and data augmentations used for each task.
    • Go through all tables (Table 1, 2, 3, 4, 5, 6, 8, 9, 10) in detail. Compare Swin Transformer's performance against baselines and state-of-the-art methods across various metrics (accuracy, AP, mIoU, FLOPs, throughput). Pay attention to the different model sizes and pre-training strategies.
    • Deeply understand the Ablation Study (Section 4.4). For example, how much does "shifted windows" contribute to performance and efficiency (Table 4 and Table 5)? What is the impact of relative position bias?
  • Limitations/Future Work (Section 5): Re-read the conclusion to understand the authors' perspective on the broader implications and potential future research directions.

• Replication pointers:

  • Code Availability: The paper explicitly states that "The code and models are publicly available at https://github.com/microsoft/Swin-Transformer" (Abstract, Section 1). This is your primary resource for replication.
  • Detailed Settings: Appendix A2 (A2.1 for ImageNet-1K, A2.2 for COCO, A2.3 for ADE20K) provides comprehensive details on training settings, optimizers (AdamW), learning rates, schedules (cosine decay, linear warm-up), batch sizes, and data augmentations (RandAugment, Mixup, Cutmix, random erasing, stochastic depth).
  • Architecture Details: Section 3.1, 3.3 and Appendix A1 (Table 7) specify the channel numbers, layer numbers, window size (M=7), query dimension (d=32), and MLP expansion ratio (α=4) for Swin-T, Swin-S, Swin-B, and Swin-L variants.
  • Pre-training: Note the specifics for ImageNet-22K pre-training and subsequent fine-tuning on ImageNet-1K (Section 4.1, Appendix A2.1).
  • Hardware: Throughput measurements are reported using a V100 GPU (Table 1 caption).