Concept

Transfer learning

Transfer learning (TL) is a technique in machine learning (ML) in which knowledge learned from a task is re-used in order to boost performance on a related task. For example, for , knowledge gained while learning to recognize cars could be applied when trying to recognize trucks. This topic is related to the psychological literature on transfer of learning, although practical ties between the two fields are limited. Reusing/transferring information from previously learned tasks to new tasks has the potential to significantly improve learning efficiency. In 1976, Bozinovski and Fulgosi published a paper addressing transfer learning in neural network training. The paper gives a mathematical and geometrical model of the topic. In 1981, a report considered the application of transfer learning to a dataset of images representing letters of computer terminals, experimentally demonstrating positive and negative transfer learning. In 1993, Pratt formulated the discriminability-based transfer (DBT) algorithm. In 1997, Pratt and Thrun guest-edited a special issue of Machine Learning devoted to transfer learning, and by 1998, the field had advanced to include multi-task learning, along with more formal theoretical foundations. Learning to Learn, edited by Thrun and Pratt, is a 1998 review of the subject. Transfer learning has been applied in cognitive science. Pratt guest-edited an issue of Connection Science on reuse of neural networks through transfer in 1996. Ng said in his NIPS 2016 tutorial that TL would become the next driver of machine learning commercial success after supervised learning. In the 2020 paper "Rethinking Pre-Training and self-training", Zoph et al. reported that pre-training can hurt accuracy, and advocate self-training instead. Algorithms are available for transfer learning in Markov logic networks and Bayesian networks. Transfer learning has been applied to cancer subtype discovery, building utilization, general game playing, text classification, digit recognition, medical imaging and spam filtering.

Official source

https://en.wikipedia.org/wiki/Transfer_learning

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related courses (8)

MICRO-455: Machine learning I

Real-world engineering applications must cope with a large dataset of dynamic variables, which cannot be well approximated by classical or deterministic models. This course gives an overview of methods from Machine Learning for the analysis of non-linear, highly noisy and multi dimensional data

ChE-605: AI in chemistry and beyond:Highlights in the field

Should have expertise in chemistry, physics or lite and material sciences. Although a very good knowledge in Al-based algorithms is required to fully understand the technical details, a basic knowledge is sufficient to understand the potential of these met ...

CS-503: Visual intelligence

This course covers both classic concepts and recent advances in computer vision and machine learning for processing visual data -- with a primary focus on embodied intelligence and multimodal learning ...

Related units (30)

Computer Vision Laboratory

IGM - Administration

Learning algorithms and systems Laboratory

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Official source

https://en.wikipedia.org/wiki/Transfer_learning

About this result

Related courses (8)

MICRO-455: Machine learning I

ChE-605: AI in chemistry and beyond:Highlights in the field

CS-503: Visual intelligence

Related lectures (32)

Neural Networks: Perceptron Model and Backpropagation Algorithm

Covers the perceptron model and backpropagation algorithm in neural networks.

Visual Intelligence and Learning: Insights and Applications

Explores visual intelligence, robotics perception, and multi-task learning techniques in computer vision.

Convolutional Networks: Applications beyond Object Recognition

Delves into the applications of convolutional networks beyond object recognition, emphasizing their impact on neuroscience, brain sciences, and art.

Related publications (26)

Explainable zero-shot transfer learning for cross-domain In-Situ acoustic monitoring in laser powder bed fusion process using learnable wavelet scattering

• Learnable Wavelet Scattering network decodes acoustic fingerprints of melt-pool dynamics from data, bridging physics and learning. • Adaptive Morlet wavelets converge toward stable sub-40 kHz bands that capture the temporal dynamics of melt-pool regime transitions. • The causal framework spectrally localizes keyhole-mode activity, confining its acoustic signature to low-frequency bands below 30 kHz. • The framework achieves ∼97–99% accuracy and retains robust performance across drift domain embeddings wit

Elsevier BV2026

Efficient species segmentation throughout the growing season of oilseed rape-service plant intercropping using transfer learning L’utilisation des méthodes de transfer learning permet un découpage efficace des espèces sur des images d’association colza-plantes de services à différents stades

Semantic segmentation methods have become increasingly popular in the field of agronomy for their ability to accurately and efficiently analyse images of crops. These methods use machine learning algorithms to assign semantic labels to each pixel in an image and require extensive amount of labelled data. Currently, most of the available models focus on crop and weed identification and target a single growth stage. In this paper, we propose a robust semantic segmentation method for estimating the dynamics of

2026

Deep learning-based surrogate model for generation of wood microstructures

A machine learning-based surrogate model for efficient wood microstructure generation compatible with a physics-based model is developed. The model is based on the U-Net neural network, a variant of convolutional neural network which, due to its architecture, is suitable for image-to-image transformation, and focuses on the crucial step of the microstructure generation, which is distortion of the wood slices according to the prescribed distortion map. For training the U-Net, a dataset consisting of a variet

Elsevier BV2026

Related people (32)

Pierre Dillenbourg

A former teacher in elementary school, Pierre Dillenbourg graduated in educational science (University of Mons, Belgium). He started his research on learning technologies in 1984. In 1986, he has been on of the first in the world to apply machine learning to develop a self-improving teaching system. He obtained a PhD in computer science from the University of Lancaster (UK), in the domain of artificial intelligence applications for education. He has been assistant professor at the University of Geneva. He joined EPFL in 2002. He has been the director of Center for Research and Support on Learning and its Technologies, then academic director of Center for Digital Education, which implements the MOOC strategy of EPFL (over 2 million registrations). He is full professor in learning technologies in the School of Computer & Communication Sciences, where he is the head of the CHILI Lab: "Computer-Human Interaction for Learning & Instruction ». He is the director of the leading house DUAL-T, which develops technologies for dual vocational education systems (carpenters, florists,...). With EPFL colleagues, he launched in 2017 the Swiss EdTech Collider, an incubator with 80 start-ups in learning technologies. He (co-)-founded 4 start-ups, does consulting missions in the corporate world and joined the board of several companies or institutions. In 2018, he co-founded LEARN, the EPFL Center of Learning Sciences that brings together the local initiatives in educational innovation. He is a fellow of the International Society for Learning Sciences. He currently is the Associate Vice-President for Education at EPFL.

Maria Brbic

Maria Brbic is an assistant professor in computer science at EPFL. Prior to joining EPFL, Maria was a postdoctoral researcher in Computer Science at Stanford University working with Jure Leskovec. She received her PhD degree from University of Zagreb in 2019, while also researching at Stanford University and University of Tokyo. Her research was awarded with the Fulbright Scholarship, L’Oreal UNESCO for Women in Science Scholarship, Branimir Jernej award for outstanding publication in biology and biomedicine, and Josip Loncar Silver Plaque award for the best doctoral dissertation. She has been named a Rising Star in EECS by MIT in 2021. Her research is focused on developing new machine learning methods and applying her methods to advance biomedical research.

Alexandre Massoud Alahi

Alexandre Alahi is currently an Assistant Professor at EPFL. He spent five years at Stanford University as a Post-doc and Research Scientist after obtaining his Ph.D. from EPFL. His research enables machines to perceive the world and make decisions in the context of transportation problems and smart environments. He has worked on the theoretical challenges and practical applications of socially-aware Artificial Intelligence, i.e., systems equipped with perception and social intelligence. He was awarded the Swiss NSF early and advanced researcher grants for his work on predicting human social behavior. He won the CVPR Open Source Award (2012) for his work on Retina-inspired image descriptors, and the ICDSC Challenge Prize (2009) for his sparsity-driven algorithm that has tracked more than 100 million pedestrians to date. His research has been covered internationally by BBC, abc, PBS, Euronews, Wall street journal, and other national news outlets around the world. Alexandre has also co-founded multiple startups such as Visiosafe, and won several startup competitions. He was elected as one of the Top 20 Swiss Venture leaders in 2010.

Sokratis Anagnostopoulos

Lenka Zdeborová

Lenka Zdeborová is a Professor of Physics and of Computer Science in École Polytechnique Fédérale de Lausanne where she leads the Statistical Physics of Computation Laboratory. She received a PhD in physics from University Paris-Sud and from Charles University in Prague in 2008. She spent two years in the Los Alamos National Laboratory as the Director's Postdoctoral Fellow. Between 2010 and 2020 she was a researcher at CNRS working in the Institute of Theoretical Physics in CEA Saclay, France. In 2014, she was awarded the CNRS bronze medal, in 2016 Philippe Meyer prize in theoretical physics and an ERC Starting Grant, in 2018 the Irène Joliot-Curie prize, in 2021 the Gibbs lectureship of AMS. She is an editorial board member for Journal of Physics A, Physical Review E, Physical Review X, SIMODS, Machine Learning: Science and Technology, and Information and Inference. Lenka's expertise is in applications of concepts from statistical physics, such as advanced mean field methods, replica method and related message-passing algorithms, to problems in machine learning, signal processing, inference and optimization. She enjoys erasing the boundaries between theoretical physics, mathematics and computer science.

William Curtin

Olivier Verscheure

SWISS DATA SCIENCE CENTER Housed at both EPFL and ETH Zurich, the newly created Swiss Data Science Center (SDSC) will address the fragmentation within today's data and analytics landscapes. The SDSC will be composed of a distributed multi-disciplinary team of data scientists and experts in domains including personalized health and personalized medicine, transportation, earth and environmental science, social science and digital humanities, and economics. The center aims to federate data providers, data and

Rita Drissi Daoudi

Pascal Fua

Pascal Fua received an engineering degree from Ecole Polytechnique, Paris, in 1984 and the Ph.D. degree in Computer Science from the University of Orsay in 1989. He then worked at SRI International and INRIA Sophia-Antipolis as a Computer Scientist. He joined EPFL in 1996 where he is now a Professor in the School of Computer and Communication Science and heads the Computer Vision Laboratory. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and Augmented Reality. His research interests include shape modeling and motion recovery from images, analysis of microscopy images, and machine learning. He has (co)authored over 300 publications in refereed journals and conferences. He is an IEEE Fellow and has been an Associate Editor of IEEE journal Transactions for Pattern Analysis and Machine Intelligence. He often serves as program committee member, area chair, and program chair of major vision conferences and has cofounded three spinoff companies (Pix4D, PlayfulVision, and NeuralConcept).

Mathieu Salzmann

I am a Senior Researcher at EPFL-CVLab, and, since May 2020, an Artificial Intelligence Engineer at ClearSpace (50%). Previously, I was a Senior Researcher and Research Leader in NICTA's computer vision research group. Prior to this, from Sept. 2010 to Jan 2012, I was a Research Assistant Professor at TTI-Chicago, and, from Feb. 2009 to Aug. 2010, a postdoctoral fellow at ICSI and EECS at UC Berkeley under the supervision of Prof. Trevor Darrell. I obtained my PhD in Jan. 2009 from EPFL under the supervision of Prof. Pascal Fua.

Related units (30)

Computer Vision Laboratory

IGM - Administration

Learning algorithms and systems Laboratory

Related concepts (3)

Deep learning

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.

General game playing

General game playing (GGP) is the design of artificial intelligence programs to be able to play more than one game successfully. For many games like chess, computers are programmed to play these games using a specially designed algorithm, which cannot be transferred to another context. For instance, a chess-playing computer program cannot play checkers. General game playing is considered as a necessary milestone on the way to artificial general intelligence.