Papperoni

2011

Theano: Deep Learning on GPUs With Python

Yoshua Bengio

citations

Cite Score

AI summary

This paper introduces Theano, a Python framework that facilitates deep learning research by providing automatic differentiation, GPU support, and faster expression evaluation. Theano achieves state-of-the-art results, making deep learning algorithms compact, elegant, and fast.

Main Contributions

Introduces Theano, a framework in Python for defining, optimizing, and evaluating tensor expressions.
Offers automatic symbolic differentiation and GPU support.
Facilitates research in deep learning.
Demonstrates how Theano makes deep learning algorithms compact, elegant, and fast.
Provides tutorials showcasing recent advances in deep learning.

Abstract

In this paper, we present Theano, a framework in the Python programming language for defining, optimizing and evaluating expressions involving high-level operations on tensors. Theano offers most of NumPy's functionality, but adds automatic symbolic differentiation, GPU support, and faster expression evaluation. Theano is a general mathematical tool, but it was developed with the goal of facilitating research in deep learning. The Deep Learning Tutorials introduce recent advances in deep learning, and showcase how Theano makes such algorithms compact, elegant, and fast.

Citation Graph

Loading graph...

References [6]

Sort:

Filter:

[1]Gradient-Based Learning Applied to Document Recognition

Yann Lecun, Leon Bottou, Yoshua Bengio, Patrick Haffner - 1998

62 papers in library cite

Google Scholar

I absolutely hated this paper. Has ~50 pages but seems like 200 pages. Takes too long to explain some things that really is just repeating itself. Also doesn't seem to add too much on top of LeNet-5. Also, focuses a lot on GTNs, which really didn't stick.

[2]Learning Representations by Back-Propagating Errors

D. E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams - 1986

34 papers in library cite

Google Scholar

Introduced backprop. Short and simple.

[3]A Fast Learning Algorithm for Deep Belief Nets

Geoffrey E. Hinton, S. Osindero, Y. Teh - 2006

43 papers in library cite

Google Scholar

The paper does not explain anything. It just throws the idea and a bunch of math, but doesn't really care to explain the concepts.

[4]Learning Deep Architectures for AI

Yoshua Bengio - 2009

25 papers in library cite

Google Scholar

It's a nice overview. Some sections get very theoretical, but the first half is very good and I feel that it does a waaaay better job of explaining RBMs and DBNs than other papers. This feels like Bengio is taking your hand and saying "if you don't know what's going on, here you go, everything you need to know to jump into the deep nets train"

[5]Extracting and Composing Robust Features With Denoising Autoencoders

Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre Antoine Manzagol - 2008

25 papers in library cite

Google Scholar

I am *so* glad we found an alternative to DBNs. Also, introduced the idea of denoising which is nice.

[6]Theano: A CPU and GPU Math Expression Compiler

James Bergstra, O. Breuleux, F. Bastien, P. Lamblin, Razvan Pascanu, G. Desjardins, J. Turian, D. W. Farley, Yoshua Bengio - 2010

22 papers in library cite

Google Scholar

Very nice framework. Symbolic programming is very nice. However, I think that this had very little impact and was mostly used by Bengio's lab.

Cited by

papers in your library

Cites

papers in your library

Read

on July 31, 2025

I am convinced this was only written to get some citations off their work. The other papers are infinitely better.