Papperoni

2012

Theano: New Features and Speed Improvements

F. Bastien, P. Lamblin, Razvan Pascanu, James Bergstra, I. Goodfellow, A. Bergeron, A. Bouchard, N. Nicolas, Yoshua Bengio

Open PDF Google Scholar

citations

Cite Score

AI summary

This paper presents new features and efficiency improvements to Theano, a linear algebra compiler, and introduces benchmarks demonstrating Theano's performance relative to Torch7 and RNNLM.

Main Contributions

Introduces new features and efficiency improvements to Theano.
Demonstrates Theano's performance relative to Torch7.
Demonstrates Theano's performance relative to RNNLM.
Introduces a symbolic loop in Theano.
Introduces a C implementation of the VM.

Abstract

Theano is a linear algebra compiler that optimizes a user's symbolically-specified mathematical computations to produce efficient low-level implementations. In this paper, we present new features and efficiency improvements to Theano, and benchmarks demonstrating Theano's performance relative to Torch7, a recently introduced machine learning library, and to RNNLM, a C++ library targeted at recurrent neural networks.

Citation Graph

Loading graph...

References [11]

Sort:

Filter:

[1]Learning Internal Representations by Error Propagation

D. E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams - 1986

46 papers in library cite

Google Scholar

I expected very little of this, but was so good in explaining concepts! Very good read. It gets a bit boring when it starts explaining things by the end of the chapter, but good nonetheless.

[2]Torch7: A Matlab-Like Environment for Machine Learning

Clement Farabet - 2011

5 papers in library cite

Google Scholar

Very nice paper! I am impressed that they used Lua, and the API sounds very simple. Plus, it sounds like they didn't even try that hard to get good performance (vs. Theano which was very tryhard)

[3]Learning Recurrent Neural Networks With Hessian-Free Optimization

James Martens, Ilya Sutskever - 2011

13 papers in library cite

Google Scholar

Meh, very very mathy and seems like a minor improvement on LSTMs. They give very impressive results but I don't think it's too much of a fair comparison (vs. a 1999 architecture of LSTMs). I think they did well in showing that exploding/vanishing gradients can be overcome, but their method is not "it". As said by Bengio, they are partially responsible for the revival of RNNs (alongside Mikolov)

[4]Empirical Evaluation and Combination of Advanced Language Modeling Techniques

Tomas Mikolov, A. Deoras, S. Kombrink, Lukas Burget, Jan Cernocky - 2011

13 papers in library cite

Google Scholar

Early work proving that NNs can be good. But very uninteresting overall.

[5]Theano: Deep Learning on GPUs With Python

Yoshua Bengio - 2011

1 paper in library cites

Google Scholar

I am convinced this was only written to get some citations off their work. The other papers are infinitely better.

[6]Theano: A CPU and GPU Math Expression Compiler

James Bergstra, O. Breuleux, F. Bastien, P. Lamblin, Razvan Pascanu, G. Desjardins, J. Turian, D. W. Farley, Yoshua Bengio - 2010

22 papers in library cite

Google Scholar

Very nice framework. Symbolic programming is very nice. However, I think that this had very little impact and was mostly used by Bengio's lab.

[7]Fast Exact Multiplication by the Hessian

B. Pearlmutter - 1994

4 papers in library cite

Google Scholar

[8]Python for Scientific Computing

T. E. Oliphant - 2007

3 papers in library cite

Google Scholar

[9]SciPy: Open Source Scientific Tools for Python

E. Jones, T. Oliphant, P. Peterson - 2001

2 papers in library cite

Google Scholar

[10]IPython: A System for Interactive Scientific Computing

F. Perez, B. Granger - 2007

1 paper in library cites

Google Scholar

[11]Matplotlib: A 2d Graphics Environment

J. Hunter - 2007

1 paper in library cites

Google Scholar

Cited by

papers in your library

Cites

papers in your library

Read

on July 25, 2025

The paper itself is ok, but it seems like they just wanted to make a paper to say "hey, ours is so much better now after these improvements, we crushed Torch7". A bit tryhard tbh.