Papperoni

2016

Dialog-Based Language Learning

J. E. Weston

Open PDF Google Scholar

citations

Cite Score

8

AI summary

This paper introduces dialog-based language learning using the bAbI dataset and large-scale question answering from the MovieQA dataset. It evaluates baseline strategies and introduces a novel model incorporating predictive lookahead, achieving correct question answering without reward-based supervision.

Main Contributions

Introduces a set of tasks that model natural feedback from a teacher
Evaluates baseline models on dialog-based language learning
Introduces a novel forward prediction model for the learner to predict the teacher's replies

Abstract

A long-term goal of machine learning research is to build an intelligent dialog agent. Most research in natural language understanding has focused on learning from fixed training sets of labeled data, with supervision either at the word level (tagging, parsing tasks) or sentence level (question answering, machine translation). This kind of supervision is not realistic of how humans learn, where language is both learned by, and used for, communication. In this work, we study dialog-based language learning, where supervision is given naturally and implicitly in the response of the dialog partner during the conversation. We study this setup in two domains: the bAbI dataset of [30] and large-scale question answering from [4]. We evaluate a set of baseline learning strategies on these tasks, and show that a novel model incorporating predictive lookahead is a promising approach for learning from a teacher's response. In particular, a surprising result is that it can learn to answer questions correctly without any reward-based supervision at all.

Citation Graph

Loading graph...

References [32]

Sort:

Filter:

[1]Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

R. Williams - 1992

11 papers in library cite

It's alright for formalizing the concept, but it's a bit boring and doesn't add a lot from the middle on. Focuses too much in reviewing existing techniques and in stochastic units.

[2]Show, Attend and Tell: Neural Image Caption Generation With Visual Attention

K. Xu, Jimmy Lei Ba, R. Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, R. Zemel, Yoshua Bengio - 2015

12 papers in library cite

It's a nice paper. I liked the soft attention way more than the hard one, and I am a bit mad that it wasn't the best lol And also it's the first paper I read about multimodality, but it seems that this was bustling at the time. Also results are kinda bad.

[3]End-to-End Memory Networks

S. Sukhbaatar, A. Szlam, Jason Weston, Rob Fergus - 2015

18 papers in library cite

This was so surprising! This is very similar to transformers and RAG. Who knew?!

[4]Memory Networks

Jason Weston, S. Chopra, Antoine Bordes - 2015

18 papers in library cite

The first half of the paper (when they discuss the concept in a very abstract way) is amazing. However, the actual methodology was very convoluted - I did not like it. I thought that Neural Turing Machines were inspired in this, but actually they are contemporary... So anyway, the concept is nice, execution is not.

[5]Towards AI-complete Question Answering: A Set of Prerequisite Toy Tasks

Jason Weston, Antoine Bordes, S. Chopra, Tomas Mikolov - 2015

11 papers in library cite

It's a good idea and a nice read but the bad part is that most of the tasks are already easy.

[6]The Goldilocks Principle: Reading Children's Books With Explicit Memory Representations

F. Hill, Antoine Bordes, S. Chopra, Jason Weston - 2015

14 papers in library cite

Cool use of memory networks.

[7]Sequence Level Training With Recurrent Neural Networks

Marc'aurelio Ranzato, S. Chopra, Michael Auli, Wojciech Zaremba - 2015

6 papers in library cite

Exposure bias

[8]A Neural Network Approach to Context-Sensitive Generation of Conversational Responses

A. Sordoni, M. Galley, Michael Auli, Chris Brockett, Yangfeng Ji, M. Mitchell, J. Y. Nie, Jianfeng Gao, B. Dolan - 2015

4 papers in library cite

Generating conversational responses

[9]Large-Scale Simple Question Answering With Memory Networks

Antoine Bordes, Nicolas Usunier, S. Chopra, Jason Weston - 2015

5 papers in library cite

Mem networks for QA - sounds interesting

[10]Language Understanding for Text-Based Games Using Deep Reinforcement Learning

K. Narasimhan, T. Kulkarni, R. Barzilay - 2015

2 papers in library cite

Application on games!

[11]Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems

J. Dodge, A. Gane, X. Zhang, Antoine Bordes, S. Chopra, A. Miller, A. Szlam, Jason Weston - 2015

4 papers in library cite

I like these "prerequisite" papers

[12]Guiding a Reinforcement Learner With Natural Language Advice: Initial Results in Robocup Soccer

G. Kuhlmann, P. Stone, R. Mooney, J. Shavlik - 2004

2 papers in library cite

[13]A Roadmap Towards Machine Intelligence

Tomas Mikolov, Armand Joulin, M. Baroni - 2015

1 paper in library cites

[14]A Survey of Statistical User Simulation Techniques for Reinforcement-Learning of Dialogue Management Strategies

J. Schatzmann, K. Weilhammer, M. Stuttle, S. Young - 2006

1 paper in library cites

[15]Action-Conditional Video Prediction Using Deep Networks in Atari Games

J. Oh, X. Guo, Honglak Lee, R. L. Lewis, Shivalika Singh - 2015

1 paper in library cites

[16]Deepmpc: Learning Deep Latent Features for Model Predictive Control

I. Lenz, R. Knepper, A. Saxena - 2015

1 paper in library cites

[17]Driving Semantic Parsing From the World's Response

J. Clarke, D. Goldwasser, M. W. Chang, Dan Roth - 2010

1 paper in library cites

[18]Early Language Acquisition: Cracking the Speech Code

P. K. Kuhl - 2004

1 paper in library cites

[19]Hierarchical Control Using Networks Trained With Higher-Level Forward Models

G. Wayne, L. Abbott - 2014

1 paper in library cites

[20]Incentivizing Exploration in Reinforcement Learning With Deep Predictive Models

B. C. Stadie, Sergey Levine, P. Abbeel - 2015

1 paper in library cites

[21]Instructive Feedback: Review of Parameters and Effects

M. G. Werts, M. Wolery, A. Holcombe, D. L. Gast - 1995

1 paper in library cites

[22]Interactional Feedback and the Impact of Attitude and Motivation on Noticing 12 Form

M. A. Bassiri - 2011

1 paper in library cites

[23]Learning From Natural Instructions

D. Goldwasser, Dan Roth - 2014

1 paper in library cites

[24]Learning Knowledge Graphs for Question Answering Through Conversational Dialog

B. Hixon, Peter Clark, Hananneh Hajishirzi - 2015

1 paper in library cites

[25]Learning Through Feedback

A. S. Latham - 1997

1 paper in library cites

[26]Learning to Generate Artificial Fovea Trajectories for Target Detection

Jürgen Schmidhuber, R. Huber - 1991

1 paper in library cites

[27]Mazebase: A Sandbox for Learning From Games

S. Sukhbaatar, A. Szlam, Gabriel Synnaeve, S. Chintala, Rob Fergus - 2015

1 paper in library cites

[28]Negative Evidence in Language Acquisition

G. F. Marcus - 1993

1 paper in library cites

[29]Predicting Tasks in Goal-Oriented Spoken Dialog Systems Using Semantic Knowledge Bases

A. Pappu, Alex Rudnicky - 2013

1 paper in library cites

[30]Reinforcement Learning for Adaptive Dialogue Systems: A Data-Driven Methodology for Dialogue Management and Natural Language Generation

V. Rieser, O. Lemon - 2011

1 paper in library cites

[31]Reward Shaping With Recurrent Neural Networks for Speeding Up on-Line Policy Learning in Spoken Dialogue Systems

P. H. Su, D. Vandyke, M. Gasic, N. Mrksic, T. H. Wen, S. Young - 2015

1 paper in library cites

[32]The Conscientious Consumer: Reconsidering the Role of Assessment Feedback in Student Learning

R. Higgins, P. Hartley, A. Skelton - 2002

1 paper in library cites

Cited by

1

papers in your library

Cites

11

papers in your library

Read

on October 31, 2025

It's a nice concept but very abstract and results are very very underwhelming

Tags

Paper Aliases

No aliases