Papperoni

2010

A Survey on Transfer Learning

Sinno Jialin Pan, Qiang Yang

Open PDF Google Scholar

citations

Cite Score

95

AI summary

This paper surveys existing transfer learning methods for classification, regression, and clustering, discussing their relationship with other machine learning techniques and categorizing them into inductive, transductive, and unsupervised settings, and outlining future research directions.

Main Contributions

Provides a comprehensive survey and categorization of current progress in transfer learning.
Classifies transfer learning into three settings: inductive, transductive, and unsupervised transfer learning.
Discusses various approaches for knowledge transfer, including instance-transfer, feature-representation-transfer, parameter-transfer, and relational-knowledge-transfer.
Explores the relationship between transfer learning and related machine learning techniques such as domain adaptation, multi-task learning, and sample selection bias.
Highlights critical issues like negative transfer and heterogeneous transfer learning, and suggests future research directions.

Abstract

A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multi-task learning and sample selection bias, as well as co-variate shift. We also explore some potential future issues in transfer learning research.

Citation Graph

Loading graph...

References [87]

Sort:

Filter:

[1]Multitask Learning

Rich Caruana - 1997

13 papers in library cite

I expected waaaaaay more from this paper. The idea is sooooo simple and the results are underwhelming. Also, 30 pages for something that could be said in 10. The writing style is a bit boring. TBH it seems like it's just a re-writing of Caruana's PhD thesis.

[2]Biographies, Bollywood, Boom-Boxes and Blenders: Domain Adaptation for Sentiment Classification

John Blitzer, Mark Dredze, Fernando Pereira - 2007

4 papers in library cite

It's alright. I think they focus a lot on methodology of prediction but most citations are actually because of the dataset.

[3]Self-Taught Learning: Transfer Learning From Unlabeled Data

Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, A. Ng - 2007

7 papers in library cite

It's nice to make the distinction of self-taught learning vs. semi-supervised learning. However, it doesn't talk about NNs. Overall, LeCun's CNN unsupervised learning was better.

[4]Learning to Learn

Sebastian Thrun, L. Pratt - 1998

3 papers in library cite

Book

[5]Statistical Learning Theory

V. N. Vapnik - 1998

10 papers in library cite

Book, but there's a 12 page overview

[6]Dataset Shift in Machine Learning

J. Q. Candela, M. Sugiyama, A. Schwaighofer, N. D. Lawrence - 2009

4 papers in library cite

Book, 250 pages

[7]Efficient Sparse Coding Algorithms

Honglak Lee, Alexis Battle, Rajat Raina, A. Ng - 2007

6 papers in library cite

[8]Transductive Inference for Text Classification Using Support Vector Machines

T. Joachims - 1999

5 papers in library cite

[9]A High-Performance Semi-Supervised Learning Method for Text Chunking

Rie Kubota Ando, Tong Zhang - 2005

4 papers in library cite

[10]Domain Adaptation With Structural Correspondence Learning

John Blitzer, R. Mcdonald, Fernando Pereira - 2006

4 papers in library cite

[11]Text Classification From Labeled and Unlabeled Documents Using Em

K. Nigam, A. K. Mccallum, Sebastian Thrun, T. Mitchell - 2000

4 papers in library cite

[12]Analysis of Representations for Domain Adaptation

S. B. David, John Blitzer, K. Crammer, Fernando Pereira - 2006

3 papers in library cite

[13]Combining Labeled and Unlabeled Data With Co-Training

A. Blum, T. Mitchell - 1998

3 papers in library cite

[14]Multi-Task Feature Learning

A. Argyriou, T. Evgeniou, M. Pontil - 2006

3 papers in library cite

[15]Regularized Multi-Task Learning

T. Evgeniou, M. Pontil - 2004

3 papers in library cite

[16]Domain Adaptation for Statistical Classifiers

H. Daumeiii, D. Marcu - 2006

2 papers in library cite

[17]Exploiting Task Relatedness for Multiple Task Learning

S. B. David, R. Schuller - 2003

2 papers in library cite

[18]Frustratingly Easy Domain Adaptation

H. D. Iii - 2007

2 papers in library cite

[19]Improving Predictive Inference Under Covariate Shift by Weighting the Log-Likelihood Function

H. Shimodaira - 2000

2 papers in library cite

[20]Instance Weighting for Domain Adaptation in NLP

J. J. Jiang, C. Zhai - 2007

2 papers in library cite

[21]Semi-Supervised Learning Literature Survey

X. Zhu - 2005

2 papers in library cite

[22]Transferring Naive Bayes Classifiers for Text Classification

W. Dai, G. Xue, Qiang Yang, Y. Yu - 2007

2 papers in library cite

[23]10 Challenging Problems in Data Mining Research

Qiang Yang, Xiaobao Wu - 2006

1 paper in library cites

[24]A Comparative Study of Methods for Transductive Transfer Learning

A. Arnold, R. Nallapati, W. W. Cohen - 2007

1 paper in library cites

[25]A Lazy Approach to Associative Classification

E. Baralis, S. Chiusano, P. Garza - 2008

1 paper in library cites

[26]A New Text Categorization Technique Using Distributional Clustering and Learning Logic

H. A. Mubaid, S. A. Umair - 2006

1 paper in library cites

[27]A Spectral Regularization Framework for Multi-Task Structure Learning

A. Argyriou, C. A. Micchelli, M. Pontil, Y. Ying - 2008

1 paper in library cites

[28]Actively Transfer Domain Knowledge

X. Shi, W. Fan, J. Ren - 2008

1 paper in library cites

[29]Adaptive Localization in a Dynamic WiFi Environment Through Multi-View Learning

Sinno Jialin Pan, J. T. Kwok, Qiang Yang, J. J. Pan - 2007

1 paper in library cites

[30]Adaptive Temporal Radio Maps for Indoor Location Estimation

J. Yin, Qiang Yang, L. M. Ni - 2005

1 paper in library cites

[31]An Algorithm for Transfer Learning in a Heterogeneous Environment

A. Argyriou, A. Maurer, M. Pontil - 2008

1 paper in library cites

[32]An Improved Categorization of classifier's Sensitivity on Sample Selection Bias

W. Fan, I. Davidson, B. Zadrozny, P. S. Yu - 2005

1 paper in library cites

[33]Bayesian Multiple Instance Learning: Automatic Feature Selection and Inductive Transfer

V. C. Raykar, B. Krishnapuram, J. Bi, M. Dundar, R. B. Rao - 2008

1 paper in library cites

[34]Boosting for Transfer Learning

W. Dai, Qiang Yang, G. Xue, Y. Yu - 2007

1 paper in library cites

[35]Bridged Refinement for Transfer Learning

D. Xing, W. Dai, G. R. Xue, Y. Yu - 2007

1 paper in library cites

[36]Can Chinese Web Pages Be Classified With English Data Source

X. Ling, G. R. Xue, W. Dai, Y. Jiang, Qiang Yang, Y. Yu - 2008

1 paper in library cites

[37]Can Movies and Books Collaborate? - Cross-Domain Collaborative Filtering for Sparsity Reduction

Boxuan Li, Qiang Yang, X. Xue - 2009

1 paper in library cites

[38]Class Noise Handling for Effective Cost-Sensitive Learning by Cost-Guided Iterative Classification Filtering

X. Zhu, Xiaobao Wu - 2006

1 paper in library cites

[39]Classifier Ensembles With a Random Linear Oracle

L. I. Kuncheva, J. J. Rodrlguez - 2007

1 paper in library cites

[40]Co-Clustering Based Classification for Out-of-Domain Documents

W. Dai, G. Xue, Qiang Yang, Y. Yu - 2007

1 paper in library cites

[41]Combining Subclassifiers in Text Categorization: A DST-Based Solution and a Case Study

K. Sarinnapakorn, M. Kubat - 2007

1 paper in library cites

[42]Constructing Informative Priors Using Transfer Learning

Rajat Raina, Andrew Y. Ng, D. Koller - 2006

1 paper in library cites

[43]Correcting Sample Selection Bias by Unlabeled Data

J. Huang, A. Smola, A. Gretton, K. M. Borgwardt, B. Scholkopf - 2007

1 paper in library cites

[44]Cross-Domain Transfer for Reinforcement Learning

M. E. Taylor, P. Stone - 2007

1 paper in library cites

[45]Deep Transfer via Second-Order Markov Logic

J. Davis, P. Domingos - 2008

1 paper in library cites

[46]Direct Importance Estimation With Model Selection and Its Application to Covariate Shift Adaptation

M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, M. Kawanabe - 2008

1 paper in library cites

[47]Discriminative Learning for Differing Training and Test Distributions

S. Bickel, M. Bruckner, T. Scheffer - 2007

1 paper in library cites

[48]Domain Adaptation via Transfer Component Analysis

Sinno Jialin Pan, I. W. Tsang, J. T. Kwok, Qiang Yang - 2009

1 paper in library cites

[49]Efficient Classification Across Multiple Database Relations: A Crossmine Approach

X. Yin, J. Han, Jihan Yang, P. S. Yu - 2006

1 paper in library cites

[50]Estimating Location Using Wi-Fi

Qiang Yang, Sinno Jialin Pan, V. W. Zheng - 2008

1 paper in library cites

[51]Graph-Based Domain Mapping for Transfer Learning in General Games

G. Kuhlmann, P. Stone - 2007

1 paper in library cites

[52]Improving svm Accuracy by Training on Auxiliary Data Sources

P. Wu, T. G. Dietterich - 2004

1 paper in library cites

[53]Kernel-Based Inductive Transfer

U. Ruckert, S. Kramer - 2008

1 paper in library cites

[54]Knowledge Transfer via Multiple Model Local Structure Mapping

Jianfeng Gao, W. Fan, J. J. Jiang, J. Han - 2008

1 paper in library cites

[55]Learning a Meta-Level Prior for Feature Relevance From Multiple Related Tasks

S. I. Lee, V. Chatalbashev, D. Vickrey, D. Koller - 2007

1 paper in library cites

[56]Learning and Evaluating Classifiers Under Sample Selection Bias

B. Zadrozny - 2004

1 paper in library cites

[57]Learning Bounds for Domain Adaptation

John Blitzer, K. Crammer, A. Kulesza, Fernando Pereira, J. Wortman - 2008

1 paper in library cites

[58]Learning Gaussian Process Kernels via Hierarchical Bayes

A. Schwaighofer, V. Tresp, K. Yu - 2005

1 paper in library cites

[59]Learning to Learn With the Informative Vector Machine

N. D. Lawrence, John C. Platt - 2004

1 paper in library cites

[60]Logistic Regression With an Auxiliary Data Source

X. Liao, Y. Xue, L. Carin - 2005

1 paper in library cites

[61]Manifold Alignment Using Procrustes Analysis

Caitlin Wang, S. Mahadevan - 2008

1 paper in library cites

[62]Mapping and Revising Markov Logic Networks for Transfer Learning

L. Mihalkova, T. Huynh, R. J. Mooney - 2007

1 paper in library cites

[63]Markov logic Networks

M. Richardson, P. Domingos - 2006

1 paper in library cites

[64]Modeling Transfer Relationships Between Learning Tasks for Improved Inductive Transfer

E. Eaton, M. Desjardins, T. Lane - 2008

1 paper in library cites

[65]Multi-Task Feature and Kernel Selection for SVMS

T. Jebara - 2004

1 paper in library cites

[66]Multi-Task Gaussian Process Prediction

E. Bonilla, K. M. Chai, C. Williams - 2008

1 paper in library cites

[67]Self-Taught Clustering

W. Dai, Qiang Yang, G. Xue, Y. Yu - 2008

1 paper in library cites

[68]Spectral Domain-Transfer Learning

X. Ling, W. Dai, G. R. Xue, Qiang Yang, Y. Yu - 2008

1 paper in library cites

[69]Task Clustering and Gating for bayesian multitask Learning

B. Bakker, T. Heskes - 2003

1 paper in library cites

[70]Test-Cost Sensitive Classification on Data With Missing Values

Qiang Yang, C. Ling, X. Chai, R. Pan - 2006

1 paper in library cites

[71]Text Classification Without Negative Examples Revisit

G. P. C. Fung, J. X. Yu, H. Lu, P. S. Yu - 2006

1 paper in library cites

[72]Theory Refinement of Bayesian Networks With Hidden Variables

S. Ramachandran, R. J. Mooney - 1998

1 paper in library cites

[73]To Transfer or Not to Transfer

M. T. Rosenstein, Z. Marx, L. P. Kaelbling - 2005

1 paper in library cites

[74]Top 10 Algorithms in Data Mining

Xiaobao Wu, V. Kumar, J. R. Quinlan, J. Ghosh, Qiang Yang, H. Motoda, G. J. Mclachlan, A. F. M. Ng, Bing Liu, P. S. Yu, Z. H. Zhou, M. Steinbach, D. J. Hand, D. Steinberg - 2008

1 paper in library cites

[75]Topic-Bridged Plsa for Cross-Domain Text Classification

G. R. Xue, W. Dai, Qiang Yang, Y. Yu - 2008

1 paper in library cites

[76]Transfer Learning by Mapping With Minimal Target Data

L. Mihalkova, R. J. Mooney - 2008

1 paper in library cites

[77]Transfer Learning for Collaborative Filtering via a Rating-Matrix Generative Model

Boxuan Li, Qiang Yang, X. Xue - 2009

1 paper in library cites

[78]Transfer Learning for Wifi-Based Indoor Localization

Sinno Jialin Pan, V. W. Zheng, Qiang Yang, D. H. Hu - 2008

1 paper in library cites

[79]Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling

J. Ramon, K. Driessens, T. Croonenborghs - 2007

1 paper in library cites

[80]Transfer Learning Using Kolmogorov Complexity: Basic Theory and Empirical Evaluations

M. M. H. Mahmud, S. R. Ray - 2008

1 paper in library cites

[81]Transfer Learning via Dimensionality Reduction

Sinno Jialin Pan, J. T. Kwok, Qiang Yang - 2008

1 paper in library cites

[82]Transferred Dimensionality Reduction

Zhengtao Wang, Yueqi Song, Chiyuan Zhang - 2008

1 paper in library cites

[83]Transferring Knowledge From Another Domain for Learning Action Models

H. Zhuo, Qiang Yang, D. H. Hu, Lei Li - 2008

1 paper in library cites

[84]Transferring Localization Models Across Space

Sinno Jialin Pan, D. Shen, Qiang Yang, J. T. Kwok - 2008

1 paper in library cites

[85]Transferring Localization Models Over Time

V. W. Zheng, Qiang Yang, W. Xiang, D. Shen - 2008

1 paper in library cites

[86]Transferring Multi-Device Localization Models Using Latent Multi-Task Learning

V. W. Zheng, Sinno Jialin Pan, Qiang Yang, J. J. Pan - 2008

1 paper in library cites

[87]Translated Learning

W. Dai, Yanru Chen, G. R. Xue, Qiang Yang, Y. Yu - 2008

1 paper in library cites

Cited by

1

papers in your library

Cites

6

papers in your library

Read

on February 11, 2026

I love that it formalized what is transfer learning and the main concepts. I think the citations are because of that as well. The part where they survey the methods is a bit boring and TBH most methods are not relevant. Also, transfer learning occurs quite naturally in neural networks, not so much in traditional methods.

Tags

Paper Aliases

No aliases