How to Read: Going deeper with convolutions [Reading Guide]

This guide will help you efficiently understand "Going deeper with convolutions," the paper introducing the Inception architecture (GoogLeNet), which achieved state-of-the-art results in the ILSVRC14 challenge.

First pass (5–10 min)

What to skim in this paper:

• Title: "Going deeper with convolutions" – Immediately suggests an advancement in convolutional neural networks, likely involving increased depth.
• Abstract: Provides a concise summary of the Inception architecture, its goal (improved resource utilization), key principles (Hebbian principle, multi-scale processing), and its specific incarnation, GoogLeNet (22 layers deep), for classification and detection in ILSVRC14.
• Section 1: Introduction: Read the first and last paragraphs to grasp the context, the problem of increasing network size, and the paper's main contribution: an efficient deep neural network architecture.
• Section 9: Conclusions: Read to understand the authors' summary of their findings, the main advantages of their method (quality gain, modest computational increase), and future work.

What to notice:

• Contributions: The introduction of the "Inception" architecture (specifically "GoogLeNet") that achieves state-of-the-art performance in ILSVRC14 for classification and detection. It emphasizes improved computational resource utilization by increasing depth and width while keeping the computational budget constant.
• Problem: The challenge of making deep neural networks more efficient and less prone to overfitting, especially when increasing depth and width.
• Setting: The ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14), a major benchmark for computer vision.
• Key terms: Inception, GoogLeNet, ILSVRC14, deep convolutional neural network, multi-scale processing, computational budget.

Second pass (up to 1 hour)

Using SpatialRead in the Second Pass

• Use the Screenshot tool to capture figures, tables, diagrams, and formulae from the PDF and use AI for simple explanations.
• Ask: "Explain Figure X at a high level", "Compare Table 2 vs 3", or "What does the y-axis represent here?"
• Download SpatialRead →

What sections/figures to focus on:

• Section 3: Motivation and High Level Considerations: Read this section to understand the core problem with simply scaling up networks (overfitting, computational cost) and the theoretical inspiration for sparse connections. Note the discussion on the inefficiency of sparse data structures on current hardware.
• Section 4: Architectural Details: This is where the Inception module is introduced.
  • Pay close attention to Figure 2: Inception module. Compare the "naïve version" (a) with the "version with dimension reductions" (b). Understand how 1x1 convolutions are used to reduce computational bottlenecks before expensive 3x3 and 5x5 convolutions.
  • Read the accompanying text to understand the rationale behind combining different filter sizes (1x1, 3x3, 5x5) and pooling operations, and concatenating their outputs.
• Section 5: GoogLeNet: This section describes the specific network used in ILSVRC14.
  • Examine Table 1: GoogLeNet incarnation of the Inception architecture. Skim the "type" column to see the sequence of layers, noting the repeated "inception" modules. Observe the "params" and "ops" columns to get a sense of the computational cost.
  • Read the text below Table 1, which clarifies the meaning of columns like "#3x3 reduce" and "#5x5 reduce" and explains the use of auxiliary classifiers.
• Section 7: ILSVRC 2014 Classification Challenge Setup and Results:
  • Review Table 2: Classification performance to see GoogLeNet's top-5 error (6.67%) compared to other leading entries.
  • Examine Table 3: GoogLeNet classification performance break down to understand the impact of using multiple models and aggressive cropping during testing.
• Section 8: ILSVRC 2014 Detection Challenge Setup and Results:
  • Look at Table 4: Detection performance to see GoogLeNet's 1st place result (43.9% mAP).

Key terms/questions to note:

• How do 1x1 convolutions serve a "dual purpose" in the Inception module? (Page 5)
• What is the overall depth of GoogLeNet (22 layers with parameters)? (Page 6)
• What are "auxiliary classifiers" and why were they introduced? (Page 6)
• How significant is the performance gain of GoogLeNet over previous state-of-the-art models like Krizhevsky et al. [9]?
• What strategies (e.g., ensembling, cropping) were used during testing to achieve the final reported performance?

Third pass (1–5 hours)

Using SpatialRead in the Third Pass

• Highlight complex keywords, dense paragraphs, equations, or proofs and ask for a plain-English explanation that preserves notation.
• Use AI to explain the concepts in a way that is easy to understand.
• Ask for alternative phrasings or analogies to confirm understanding without changing technical meaning.
• Connect concepts with each other using AI to build a comprehensive understanding of the paper.
• Download SpatialRead →

Deep reading guidance:

• Methods (Sections 4, 5, 6):
  • Architectural Details: Re-read Section 4 and 5 carefully. Trace the data flow through Figure 2(b), understanding how different filter sizes and pooling operations are performed in parallel and then concatenated. Use Table 1 and Figure 3 (the full network diagram) to understand the complete GoogLeNet architecture, including the initial layers, the stacking of Inception modules, and the final classification layers. Pay attention to the "reduce" columns in Table 1 to see the effect of 1x1 convolutions on dimensionality.
  • Auxiliary Classifiers: Deeply understand the purpose and implementation of the auxiliary classifiers (Page 6), including their structure (pooling, 1x1 conv, FC, dropout, linear+softmax) and how their loss is weighted during training.
  • Training Methodology (Section 6): Analyze the training setup, including the use of asynchronous stochastic gradient descent, momentum, learning rate schedules, Polyak averaging, and various data augmentation techniques (sampling patch sizes, aspect ratios, photometric distortions, random interpolation).
• Results (Sections 7, 8):
  • Classification: Thoroughly analyze Tables 2 and 3. Understand the metrics (top-1 and top-5 error). Compare GoogLeNet's performance against baselines and other top entries. Quantify the impact of ensembling and aggressive cropping on the final error rate.
  • Detection: Read Section 8 and analyze Tables 4 and 5. Understand the mAP metric. Note how GoogLeNet's detection approach builds on R-CNN and uses the Inception model as the region classifier, along with multi-box predictions.
• Limitations (Sections 3, 4, 9): Revisit these sections to fully grasp the theoretical underpinnings (sparse connections, Hebbian principle) and the practical compromises made due to hardware limitations. Understand the authors' cautious stance on whether the guiding principles were fully validated and the potential for future automated architecture design.

Replication pointers if applicable:

• Architecture: Table 1 and Figure 3 provide the complete blueprint for GoogLeNet. Pay attention to filter sizes, strides, output dimensions, and the number of filters in each stage, especially within the Inception modules.
• Input Preprocessing: The network expects 224x224 RGB input with mean subtraction (Page 6).
• Activation Functions: All convolutions, including those within Inception modules and reduction layers, use rectified linear activation (Page 6).
• Training Details: Section 6 outlines the optimization algorithm (SGD with 0.9 momentum), learning rate schedule (decreasing by 4% every 8 epochs), and the use of Polyak averaging.
• Data Augmentation: Section 6 describes the image sampling methods (various sized patches, random aspect ratios, photometric distortions, random interpolation).
• Auxiliary Classifiers: Their structure and the 0.3 discount weight for their loss during training are detailed on Page 6.
• Testing Strategy: Section 7.2 describes the aggressive cropping (4 scales, 3 regions, 4 corners + center, mirrored versions = 144 crops per image) and averaging softmax probabilities across crops and models.