This paper presents Model-Agnostic Meta-Learning (MAML), a meta-learning algorithm that trains model parameters so that a small number of gradient steps using a small amount of data from a new task yields good generalization on that task.
YOLOv3 reports a set of incremental design changes and a newly trained network that is slightly larger than the prior version, more accurate, and still fast, with concrete speed–accuracy numbers at 320×320 and comparisons to SSD, RetinaNet, and mAP@50 timing on a Titan X.
SqueezeNet proposes a small deep neural network architecture that reaches AlexNet-level ImageNet accuracy while using far fewer parameters, and it reports additional size reductions using model-compression techniques.
This paper formulates precipitation nowcasting as spatiotemporal sequence forecasting and introduces ConvLSTM by adding convolutional structure to LSTM transitions, reporting better capture of spatiotemporal correlations and stronger results than FC-LSTM and the operational ROVER system.
This paper reports that large feedforward neural networks trained on small training sets often perform poorly on held-out test data, and it presents random “dropout,” which omits half of the feature detectors on each training case, as a method that greatly reduces overfitting and improves benchmark results in tasks including speech and object recognition.
This paper presents SimCLR as “a simple framework for contrastive learning of visual representations,” and it reports a systematic study of major framework components that affect contrastive prediction tasks. The paper reports three findings about augmentations, a learnable nonlinear transformation before the contrastive loss, and scaling with batch size and training steps.
PointNet++ extends PointNet by building a hierarchical network over nested partitions of a point set, using metric-space distances to learn local features at increasing contextual scales and handling non-uniform sampling densities with multi-scale feature aggregation.
This paper adapts ideas underlying the success of Deep Q-Learning to the continuous action domain and presents an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action spaces. The paper reports that, with the same learning algorithm, network architecture, and hyper-parameters, the method robustly solves more than 20 simulated physics tasks and can learn some tasks end-to-end from raw pixel inputs.
This paper revisits atrous convolution for semantic image segmentation and presents the DeepLabv3 system with multi-scale modules and an augmented ASPP design that adds image-level features for global context.
This paper introduces an attention-based neural model that learns to generate image descriptions, supports both deterministic and stochastic training, and visualizes where the model focuses while producing words.
This paper presents SHAP (SHapley Additive exPlanations), a unified framework for interpreting predictions from complex machine learning models by assigning each feature an importance value for a particular prediction.
This paper introduces deep convolutional generative adversarial networks (DCGANs), a class of convolutional neural networks with specific architectural constraints, and reports evidence that the generator and discriminator learn hierarchical visual representations that transfer to novel tasks as general image features.
This paper introduces conditional generative adversarial nets (cGANs) by feeding a conditioning variable y to both the generator and discriminator, and reports demonstrations on MNIST class-conditional digit generation plus preliminary examples for multimodal modeling and image tagging.
This paper proposes extending an encoder–decoder neural machine translation model by letting the model soft-search the source sentence for the parts most relevant to predicting each target word, addressing a conjectured bottleneck from encoding the entire source sentence into a single fixed-length vector.
TensorFlow is presented as a machine learning system that operates at large scale and in heterogeneous environments. The paper describes TensorFlow as using dataflow graphs to represent computation, shared state, and the operations that mutate that state.
This paper studies transfer learning for NLP through a single text-to-text framework, comparing pre-training objectives, architectures, data, and transfer approaches across many tasks, and reporting state-of-the-art results on multiple benchmarks using scale and the Colossal Clean Crawled Corpus.
A brief summary of arXiv:1412.3555, which compares recurrent units in RNNs and reports that gated units such as LSTM and GRU outperform traditional tanh units on polyphonic music and speech signal modeling tasks, with GRU comparable to LSTM.
MobileNets introduces an efficient CNN family for mobile and embedded vision that uses depth-wise separable convolutions and two global hyper-parameters to trade off latency and accuracy across tasks such as ImageNet classification and object detection.