paper brief

Deep Inside Convolutional Networks (arXiv:1312.6034): gradient-based visualizations and saliency maps

This paper presents two gradient-based techniques for visualizing image classification models learned with deep convolutional networks: class score maximization for class “prototypes,” and per-image, per-class saliency maps that can be used for weakly supervised object segmentation, with a stated connection to deconvolutional networks.

February 25, 2026•Mira Vale•ml foundations

Continue in Rorobot with the source paper open and ready for chat.

Open this paper in Rorobot

What this paper is about

The paper addresses the visualisation of image classification models that are learnt using deep Convolutional Networks (ConvNets). [S1] The paper considers two visualisation techniques that are both based on computing the gradient of the class score with respect to the input image. [S1] The paper describes the first technique as generating an image that maximises the class score, and it presents this as a way to visualise the notion of the class captured by a ConvNet. [S1] The paper cites Erhan et al. [S1] , 2009 in connection with generating an image that maximises the class score. [S1] The paper describes the second technique as computing a class saliency map that is specific to a given image and class. [S1] The paper reports that such saliency maps can be employed for weakly supervised object segmentation using classification ConvNets. [S1] The paper also states that it establishes a connection between gradient-based ConvNet visualisation methods and deconvolutional networks, and it cites Zeiler et al. [S1] , 2013 in that connection. [S1]

Core claims to remember

The paper states that it addresses the visualisation of image classification models learnt using deep ConvNets. [S1] The paper states that it considers two visualisation techniques based on computing the gradient of the class score with respect to the input image. [S1] The paper states that one technique generates an image that maximises the class score as a way to visualise the notion of the class captured by a ConvNet. [S1] The paper cites Erhan et al. [S1] , 2009 when describing the approach that generates an image maximising the class score. [S1] The paper states that the other technique computes a class saliency map that is specific to a given image and class. [S1] The paper reports that these class saliency maps can be employed for weakly supervised object segmentation using classification ConvNets. [S1] The paper states that it establishes a connection between gradient-based ConvNet visualisation methods and deconvolutional networks. [S1] The paper cites Zeiler et al. [S1] , 2013 in its stated connection to deconvolutional networks. [S1]

Limitations and caveats

The paper describes its visualisation approach in terms of two techniques that are both based on computing the gradient of a class score with respect to the input image. [S1] The paper presents these techniques in the context of visualising image classification models learnt using deep ConvNets. [S1] The paper describes one technique as generating an image that maximises the class score to visualise the notion of a class captured by a ConvNet. [S1] The paper describes the other technique as computing a class saliency map that is specific to a given image and class. [S1] The paper reports the use of class saliency maps for weakly supervised object segmentation using classification ConvNets. [S1] The paper states that it establishes a connection between gradient-based ConvNet visualisation methods and deconvolutional networks. [S1]

How to apply this in study or projects

Read the parts of the paper that define the two visualisation techniques as gradient computations of the class score with respect to the input image. [S1] Trace how the paper describes generating an image that maximises the class score in order to visualise the notion of a class captured by a ConvNet. [S1] Follow the paper’s description of computing a class saliency map for a specific image and a specific class. [S1] Study the example use described in the paper where class saliency maps are employed for weakly supervised object segmentation using classification ConvNets. [S1] Compare the paper’s gradient-based visualisation methods to the deconvolutional network connection that the paper states it establishes, including the cited reference to Zeiler et al. [S1] , 2013. [S1]

Sources

[S1]arxiv.org
YOLOv4: Optimal Speed and Accuracy of Object Detection
There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required. Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets. We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100. Source code is at https://github.com/AlexeyAB/darknet
Open source Back to article

FAQ

What are the two visualization techniques described in arXiv:1312.6034?

The paper considers two techniques based on computing the gradient of the class score with respect to the input image. [S1] The first technique generates an image that maximises the class score to visualise the notion of the class captured by a ConvNet. [S1] The second technique computes a class saliency map that is specific to a given image and class. [S1]

What does the paper report saliency maps can be used for?

The paper reports that class saliency maps can be employed for weakly supervised object segmentation using classification ConvNets. [S1] The paper also states that it establishes a connection between gradient-based ConvNet visualisation methods and deconvolutional networks. [S1]