Papperoni

2014

Generative Adversarial Nets

Ian J. Goodfellow, J. P. Abadie, M. Mirza, B. Xu, D. W. Farley, S. Ozair, Aaron Courville, Yoshua Bengio

Open PDF Google Scholar

citations

Cite Score

99

AI summary

This paper introduces a new generative model estimation framework using adversarial nets, where a generative model and a discriminative model are trained simultaneously via a minimax two-player game; the models, multilayer perceptrons, are trained with backpropagation, and the framework's potential is demonstrated through experiments.

Main Contributions

A new framework for estimating generative models via an adversarial process is proposed.
The framework simultaneously trains a generative model and a discriminative model.
The training procedure for the generative model is to maximize the probability of the discriminative model making a mistake.
The framework corresponds to a minimax two-player game.
Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.

Abstract

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.

Citation Graph

Loading graph...

References [30]

Sort:

Filter:

[1]ImageNet Classification With Deep Convolutional Neural Networks

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton - 2012

71 papers in library cite

I'm giving this a 5 just because of the impact, but this is VEEERY derivative of earlier work. Kudos for them for putting it all together, but really there's nothing revolutionary here.

[2]Gradient-Based Learning Applied to Document Recognition

Yann Lecun, Leon Bottou, Yoshua Bengio, Patrick Haffner - 1998

62 papers in library cite

I absolutely hated this paper. Has ~50 pages but seems like 200 pages. Takes too long to explain some things that really is just repeating itself. Also doesn't seem to add too much on top of LeNet-5. Also, focuses a lot on GTNs, which really didn't stick.

[3]Learning Multiple Layers of Features From Tiny Images

Alex Krizhevsky - 2009

27 papers in library cite

It's alright. It mainly focuses on RBMs and their features and the actual part that describes the dataset is like 1 page. However, it's maybe the best intuitive description of an RBM I have seen. Other than that, it reads very much like an undergraduate thesis.

[4]A Fast Learning Algorithm for Deep Belief Nets

Geoffrey E. Hinton, S. Osindero, Y. Teh - 2006

43 papers in library cite

The paper does not explain anything. It just throws the idea and a bunch of math, but doesn't really care to explain the concepts.

[5]Deep Neural Networks for Acoustic Modeling in Speech Recognition

Geoffrey Hinton - 2012

21 papers in library cite

The core of the paper itself is a bit boring and doesn't introduce anything new (just RBMs and DBNs again) but I am giving this a 4 because it's probably the best explanation of RBMs and DBNs I've read so far.

[6]Deep Sparse Rectifier neural Networks

Xavier Glorot, Antoine Bordes, Yoshua Bengio - 2011

17 papers in library cite

How can you not love Bengio? This paper is everything that Hinton's is not. Really well explained, surprising results, good comparisons... Amazing impact!

[7]Learning Deep Architectures for AI

Yoshua Bengio - 2009

25 papers in library cite

It's a nice overview. Some sections get very theoretical, but the first half is very good and I feel that it does a waaaay better job of explaining RBMs and DBNs than other papers. This feels like Bengio is taking your hand and saying "if you don't know what's going on, here you go, everything you need to know to jump into the deep nets train"

[8]Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors

Geoffrey E. Hinton, N. Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov - 2012

25 papers in library cite

Dropout, super impactful. The idea that you are training many estimators at once is also very nice.

[9]Extracting and Composing Robust Features With Denoising Autoencoders

Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre Antoine Manzagol - 2008

25 papers in library cite

I am *so* glad we found an alternative to DBNs. Also, introduced the idea of denoising which is nice.

[10]What Is the Best Multi-Stage Architecture for Object Recognition?

K. Jarrett, Koray Kavukcuoglu, Marc'aurelio Ranzato, Yann Lecun - 2009

20 papers in library cite

So boring and convoluted... They try to make things more abstract by saying "filter banks" and "2 stages", but in the end it's CNNs. It seems like they are trying to tie NNs to the rest of ML, but it doesn't work.

[11]Maxout Networks

Yoshua Bengio - 2013

17 papers in library cite

A bit hard to understand, but very nice idea.

[12]Theano: New Features and Speed Improvements

F. Bastien, P. Lamblin, Razvan Pascanu, James Bergstra, I. Goodfellow, A. Bergeron, A. Bouchard, N. Nicolas, Yoshua Bengio - 2012

13 papers in library cite

The paper itself is ok, but it seems like they just wanted to make a paper to say "hey, ours is so much better now after these improvements, we crushed Torch7". A bit tryhard tbh.

[13]The Wake-Sleep Algorithm for Unsupervised Neural Networks

Geoffrey Hinton, Peter Dayan, B. Frey, R. Neal - 1995

9 papers in library cite

It's okay... I get the feeling that this is early autoencoders work, but the term still didn't exist. I don't think it adds nothing new though.

[14]Generalized Denoising Auto-Encoders as Generative Models

Pascal Vincent - 2013

2 papers in library cite

The paper is not bad itself, but it's full of jargon and math that makes it very hard to understand. It seems like it was written for a maths venue. It seems that the most impressive results (creating good images) are just an afterthought.

[15]Pylearn2: A Machine Learning Research Library

Yoshua Bengio - 2013

5 papers in library cite

The paper itself is ok, but I think it had little impact. The idea of having an extensible and easy to use framework is nice (and is something that I also try to do in my projects), but seems like it didn't catch on and was only used by Bengio's lab.

[16]Theano: A CPU and GPU Math Expression Compiler

James Bergstra, O. Breuleux, F. Bastien, P. Lamblin, Razvan Pascanu, G. Desjardins, J. Turian, D. W. Farley, Yoshua Bengio - 2010

22 papers in library cite

Very nice framework. Symbolic programming is very nice. However, I think that this had very little impact and was mostly used by Bengio's lab.

[17]Deep Boltzmann Machines

Ruslan Salakhutdinov, Geoffrey E. Hinton - 2009

9 papers in library cite

Not sure what's the difference vs. DBNs

[18]Information Processing in Dynamical Systems: Foundations of Harmony Theory

P. Smolensky - 1986

11 papers in library cite

88 pages; Introduced RBMs

[19]Noise-Contrastive Estimation: A New Estimation Principle for Unnormalized Statistical Models

M. Gutmann, A. Hyvarinen - 2010

7 papers in library cite

[20]Estimation of Non-Normalized Statistical Models Using Score Matching

A. Hyvarinen - 2005

4 papers in library cite

[21]Training Restricted Boltzmann Machines Using Approximations to the Likelihood Gradient

T. Tieleman - 2008

4 papers in library cite

[22]A Generative Process for Sampling Contractive Auto-Encoders

S. Rifai, Yoshua Bengio, Yann Dauphin, Pascal Vincent - 2012

3 papers in library cite

[23]Auto-Encoding Variational Bayes

D. P. Kingma, M. Welling - 2013

3 papers in library cite

[24]Better Mixing via Deep Representations

Yoshua Bengio, G. Mesnil, Yann Dauphin, S. Rifai - 2013

3 papers in library cite

[25]Deep Generative Stochastic Networks Trainable by backprop

Yoshua Bengio, E. T. Laufer, Jason Yosinski - 2013

3 papers in library cite

[26]On the Convergence of Markovian Stochastic Algorithms With Rapidly Decreasing Ergodicity Rates

L. Younes - 1999

3 papers in library cite

[27]Stochastic Backpropagation and Approximate Inference in Deep Generative Models

D. J. Rezende, S. Mohamed, Daan Wierstra - 2014

3 papers in library cite

[28]Multi-Prediction Deep Boltzmann Machines

Ian J. Goodfellow, M. Mirza, Aaron Courville, Yoshua Bengio - 2013

2 papers in library cite

[29]Quickly Generating Representative Samples From an RBM-derived Process

O. Breuleux, Yoshua Bengio, Pascal Vincent - 2011

2 papers in library cite

[30]The Toronto Face Dataset

J. Susskind, A. Anderson, Geoffrey E. Hinton - 2010

2 papers in library cite

Cited by

2

papers in your library

Cites

18

papers in your library

Read

on July 20, 2025

The idea is surprisingly simple. I was a bit afraid of the math, but it's very simple. There's proof that this converges and finds an optimum. I just wish they explained more, or gave better results. It just seems that it finishes and I'm still hungry.

Tags

Paper Aliases

No aliases