Papperoni

2014

Learning Generative Models With Visual Attention

Ruslan Salakhutdinov

Open PDF Google Scholar

citations

Cite Score

6

AI summary

This paper introduces a deep-learning based generative framework that uses visual attention mechanism and Hamiltonian Monte Carlo, which concentrates its resources on the object of interest and ignores scene background clutter. The framework learns generative models of new faces from a novel dataset of large images where the face locations are not known.

Main Contributions

A deep-learning based generative framework that uses visual attention mechanism.
The attentional mechanism propagates signals from the region of interest in a scene to an aligned canonical representation for generative modeling.
The generative model can concentrate its resources on the object of interest by ignoring scene background clutter.
A convolutional neural net is employed to provide good initializations during posterior inference which uses Hamiltonian Monte Carlo.
The model can learn generative models of new faces from a novel dataset of large images where the face locations are not known.

Abstract

Attention has long been proposed by psychologists to be important for efficiently dealing with the massive amounts of sensory stimulus in the neocortex. Inspired by the attention models in visual neuroscience and the need for object-centered data for generative models, we propose a deep-learning based generative frame work using attention. The attentional mechanism propagates signals from the region of interest in a scene to an aligned canonical representation for generative modeling. By ignoring scene background clutter, the generative model can concentrate its resources on the object of interest. A convolutional neural net is employed to provide good initializations during posterior inference which uses Hamiltonian Monte Carlo. Upon learning images of faces, our model can robustly attend to the face region of novel test subjects. More importantly, our model can learn generative models of new faces from a novel dataset of large images where the face locations are not known.

Citation Graph

Loading graph...

References [33]

Sort:

Filter:

[1]Learning Multiple Layers of Features From Tiny Images

Alex Krizhevsky - 2009

27 papers in library cite

It's alright. It mainly focuses on RBMs and their features and the actual part that describes the dataset is like 1 page. However, it's maybe the best intuitive description of an RBM I have seen. Other than that, it reads very much like an undergraduate thesis.

[2]Reducing the Dimensionality of Data With Neural Networks

Geoffrey Hinton, Ruslan Salakhutdinov - 2006

37 papers in library cite

I didn't like the way this is written, very hard to understand without a ton of background knowledge. But hey, it's the first deep learning model!

[3]A Fast Learning Algorithm for Deep Belief Nets

Geoffrey E. Hinton, S. Osindero, Y. Teh - 2006

43 papers in library cite

The paper does not explain anything. It just throws the idea and a bunch of math, but doesn't really care to explain the concepts.

[4]Generalized Denoising Auto-Encoders as Generative Models

Pascal Vincent - 2013

2 papers in library cite

The paper is not bad itself, but it's full of jargon and math that makes it very hard to understand. It seems like it was written for a maths venue. It seems that the most impressive results (creating good images) are just an afterthought.

[5]Deep Boltzmann Machines

Ruslan Salakhutdinov, Geoffrey E. Hinton - 2009

9 papers in library cite

Not sure what's the difference vs. DBNs

[6]On Learning Where to Look

Marc'aurelio Ranzato - 2014

3 papers in library cite

Learning where to look with glimpses

[7]Acoustic Modeling Using Deep Belief Networks

A. Mohamed, G. Dahl, Geoffrey Hinton - 2012

12 papers in library cite

[8]Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations

Honglak Lee, R. Grosse, R. Ranganath, Andrew Y. Ng - 2009

12 papers in library cite

[9]Learning to Combine Foveal Glimpses With a Third-Order Boltzmann Machine

Hugo Larochelle, Geoffrey E. Hinton - 2010

4 papers in library cite

[10]Learning Where to Attend With Deep Architectures for Image Tracking

M. Denil, L. Bazzani, Hugo Larochelle, N. D. Freitas - 2012

4 papers in library cite

[11]On the Quantitative Analysis of Deep Belief Networks

Ruslan Salakhutdinov, I. Murray - 2008

4 papers in library cite

[12]Convolutional Learning of Spatio-Temporal Features

Graham W. Taylor, Rob Fergus, Yann Lecun, C. Bregler - 2010

3 papers in library cite

[13]From Learning Models of Natural Image Patches to Whole Image Restoration

D. Zoran, Y. Weiss - 2011

2 papers in library cite

[14]On Deep Generative Models With Applications to Recognition

Marc'aurelio Ranzato, J. Susskind, V. Mnih, Geoffrey Hinton - 2011

2 papers in library cite

[15]Searching for Objects Driven by Context

B. Alexe, N. Heess, Yee Whye Teh, V. Ferrari - 2012

2 papers in library cite

[16]Using Fast Weights to Improve Persistent Contrastive Divergence

T. Tieleman, Geoffrey E. Hinton - 2009

2 papers in library cite

[17]A Backward Progression of Attentional Effects in the Ventral Stream

E. A. Buffalo, P. Fries, R. Landman, H. Liang, R. Desimone - 2010

1 paper in library cites

[18]A Hierarchical Generative Model of Recurrent Object-Based Attention in the Visual Cortex

D. P. Reichert, P. Series, A. J. Storkey - 2011

1 paper in library cites

[19]A Neurobiological Model of Visual Attention and Invariant Pattern Recognition Based on Dynamic Routing of Information

Bruno A. Olshausen, C. H. Anderson, D. C. V. Essen - 1993

1 paper in library cites

[20]Attention and Primary Visual Cortex

M. I. Posner, C. D. Gilbert - 1999

1 paper in library cites

[21]Computer Vision - Algorithms and Applications

R. Szeliski - 2011

1 paper in library cites

[22]Deep Mixtures of Factor Analysers

Y. Tang, Ruslan Salakhutdinov, Geoffrey E. Hinton - 2012

1 paper in library cites

[23]Fast Normalized Cross-Correlation

J. P. Lewis - 1995

1 paper in library cites

[24]How Does Our Visual System Achieve Shift and Size Invariance?

L. Wiskott - 2004

1 paper in library cites

[25]Hybrid Monte Carlo

S. Duane, A. D. Kennedy, B. J. Pendleton, D. Roweth - 1987

1 paper in library cites

[26]Lucas-kanade 20 Years On: A Unifying Framework

S. Baker, I. Matthews - 2002

1 paper in library cites

[27]McMc Using Hamiltonian Dynamics

Radford M. Neal - 2010

1 paper in library cites

[28]Modeling the Manifolds of Images of Handwritten Digits

Geoffrey E. Hinton, Peter Dayan, M. Revow - 1997

1 paper in library cites

[29]Modeling Visual-Attention via Selective Tuning

J. K. Tsotsos, S. M. Culhane, W. Y. K. Wai, Y. H. Lai, N. Davis, F. Nuflo - 1995

1 paper in library cites

R. Gross, I. Matthews, J. F. Cohn, Takeo Kanade, S. Baker - 2010

1 paper in library cites

[31]Shifter Circuits: A Computational Strategy for Dynamic Aspects of Visual Processing

C. H. Anderson, D. C. V. Essen - 1987

1 paper in library cites

[32]Visual Attention: Insights From Brain Imaging

N. Kanwisher, E. Wojciulik - 2000

1 paper in library cites

[33]What and Where: A Bayesian Inference Theory of Attention

S. Chikkerur, T. Serre, C. Tan, T. Poggio - 2010

1 paper in library cites

Cited by

2

papers in your library

Cites

8

papers in your library

Read

on October 9, 2025

Waaaay to mathy and TBH I think there aren't a lot of good results. But also I don't have a ton of knowledge on generative models, so I may be biased.

Tags

Paper Aliases

No aliases