What this paper is about
Deep neural networks are described as highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. [S1] The paper states that this expressiveness is the reason they succeed, and it also causes them to learn uninterpretable solutions that could have counter-intuitive properties. [S1] The paper reports two such properties. [S1]
The first property is about representations in high layers of deep neural networks. [S1] The paper reports that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. [S1] The paper states that this result suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. [S1]
The paper’s discussion of “no distinction” is explicitly tied to “various methods of unit analysis. [S1] ” [S1] The paper’s statement about semantic information is explicitly phrased as being in “the space” rather than “the individual units” in the high layers. [S1] The paper presents this as a finding it reports, using the phrasing “we find” and “it suggests. [S1] ” [S1]
The second property is about the continuity of the learned mapping from inputs to outputs. [S1] The paper reports that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extend. [S1] The paper also reports that the network can be caused to misclassify an image by applying a certain imperceptible perturbation. [S1]
The paper introduces both properties in the same context: deep neural networks are highly expressive, and this expressiveness is described as both enabling strong performance and causing uninterpretable solutions with counter-intuitive properties. [S1] The paper’s abstract-level description links the two reported properties to this broader characterization of expressiveness and uninterpretable solutions. [S1]
Core claims to remember
The paper describes deep neural networks as highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. [S1]
The paper states that their expressiveness is the reason they succeed, and it also causes them to learn uninterpretable solutions that could have counter-intuitive properties. [S1]
The paper reports two counter-intuitive properties of deep neural networks. [S1]
The paper reports that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. [S1]
The paper states that this finding suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. [S1]
The paper reports that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extend. [S1]
The paper reports that the network can be caused to misclassify an image by applying a certain imperceptible perturbation. [S1]
Limitations and caveats
The paper states the first property “according to various methods of unit analysis,” and it uses that qualifier when reporting “no distinction” between individual high level units and random linear combinations of high level units. [S1]
The paper describes the second property with qualitative terms when it says the learned input-output mappings are “fairly discontinuous” and that this holds “to a significant extend. [S1] ” [S1]
The paper describes the misclassification result using the qualifier “a certain imperceptible perturbation. [S1] ” [S1]
The paper phrases the semantic-information statement as “it suggests,” while also stating “we find” in the description of the first property. [S1]
How to apply this in study or projects
Locate the paper’s statement that deep neural networks are “highly expressive models” with “state of the art performance on speech and visual recognition tasks,” and copy the exact phrasing into notes. [S1]
Extract the sentence that links expressiveness to both success and “uninterpretable solutions that could have counter-intuitive properties,” and quote it verbatim before paraphrasing it. [S1]
Find the passage that reports “no distinction between individual high level units and random linear combinations of high level units,” and transcribe the phrase “according to various methods of unit analysis” next to that claim. [S1]
Copy the sentence that states “it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks,” and keep the paper’s wording of “space,” “individual units,” and “high layers. [S1] ” [S1]
Locate the sentence that reports the learned mappings are “fairly discontinuous to a significant extend,” and record that exact qualitative wording as written. [S1]
Extract the statement that “we can cause the network to misclassify an image by applying a certain imperceptible perturbation,” and quote the phrases “misclassify,” “image,” and “certain imperceptible perturbation. [S1] ” [S1]