Publications related to Bootstrapping Language-Audio Pre-training for Music Captioning

Related publications (13)

CHAMELEON: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Training data mixtures greatly impact the generalization performance of large language models. Existing domain reweighting methods often rely on costly weight computations and require retraining when new data is introduced. To this end, we introduce a flex ...

2025

From Embeddings to Explainability: A Tutorial on Large-Language-Model-Based Text Analysis for Behavioral Scientists

Rudolf Georg Debelak

Large language models (LLMs) are transforming research in psychology and the behavioral sciences by enabling advanced text analysis at scale. Their applications range from the analysis of social media posts to infer psychological traits to the automated sc ...

SAGE Publications2025

Exploiting Representation Similarities in Self-Supervised Learning for Vision Tasks

Thomas Grégoire Stegmüller

Recent progress in computer vision has been driven by a simple yet transformative observation: the potential of 30-year-old neural networks can be unlocked by massively scaling their parameters and the amount of human-labeled training data. This breakthrou ...

EPFL2025

Overcoming Distribution Shifts and Imbalance Challenges in Representation Learning for Deep Regression Models

Ismail Nejjar

Deep learning has shown remarkable potential for industrial applications, particularly in predictive maintenance and condition monitoring. A key challenge in this domain is prognostics, specifically predicting the Remaining Useful Life (RUL) of assets - a ...

EPFL2025

On the Utility of Speech and Audio Foundation Models for Marmoset Call Analysis

Mathew Magimai Doss, Eklavya Sarkar

Marmoset monkeys encode vital information in their calls and serve as a surrogate model for neuro-biologists to understand the evolutionary origins of human vocal communication. Traditionally analyzed with signal processing-based features, recent approache ...

2024

Leveraging large language models for predictive chemistry

Berend Smit, Philippe Schwaller, Kevin Maik Jablonka, Andres Adolfo Ortega Guerrero

Machine learning has transformed many fields and has recently found applications in chemistry and materials science. The small datasets commonly found in chemistry sparked the development of sophisticated machine learning approaches that incorporate chemic ...

Nature Portfolio2024

Unveiling the Art of Heading Design: A Harmonious Blend of Summarization, Neology, and Algorithm

Boi Faltings, Shaobo Cui, Yiyang Feng

Crafting an appealing heading is crucial for attracting readers and marketing work or products. A popular way is to summarize the main idea with a refined description and a memorable acronym. However, there lacks a systematic study and a formal benchmark i ...

Association for Computational Linguistics2024

MCMC: Bridging Rendering, Optimization and Generative AI

Wenzel Alban Jakob

Generative artificial intelligence (AI) has made unprecedented advances in vision language models over the past two years. These advances are largely due to diffusion-based generative models, which are very stable and simple to train. These diffusion model ...

Association for Computing Machinery, Inc2024

Anniversary AI reflections

Francesco Stella

For our fifth anniversary, we reconnected with authors of recent Comments and Perspectives in Nature Machine Intelligence and asked them how the topic they wrote about developed. We also wanted to know what other topics in AI they found exciting, surprisin ...

Nature Portfolio2024

ChatGPT takes on the European Exam in Core Cardiology: an artificial intelligence success story?

Emmanuel Abbé, Thabodhan Mahendiran

Chat Generative Pre-trained Transformer (ChatGPT) is currently a trending topic worldwide triggering extensive debate about its predictive power, its potential uses, and its wider implications. Recent publications have demonstrated that ChatGPT can correct ...

Oxford Univ Press2023

Efficacy of temporal and spatial abstraction for training accurate machine learning models: A case study in smart thermostats

Rachid Guerraoui, Alexandre David Olivier Maurer

Smart thermostats are increasingly popular in homes and buildings as they improve occupant comfort, lower energy use in heating and cooling systems, and reduce utility bills by automatically adjusting room temperature according to measurements of their bui ...

ELSEVIER SCIENCE SA2023

DrapeNet: Garment Generation and Self-Supervised Draping

Pascal Fua, Mathieu Salzmann, Benoît Alain René Guillard, Ren Li

Recent approaches to drape garments quickly over arbitrary human bodies leverage self-supervision to eliminate the need for large training sets. However, they are designed to train one network per clothing item, which severely limits their generalization a ...

IEEE2023

MultiMAE: Multi-modal Multi-task Masked Autoencoders

Amir Roshan Zamir, Roman Christian Bachmann, Andrei Atanov, David Mizrahi

We propose a pre-training strategy called Multi-modal Multi-task Masked Autoencoders (MultiMAE). It differs from standard Masked Autoencoding in two key aspects: I) it can optionally accept additional modalities of information in the input besides the RGB ...

SPRINGER INTERNATIONAL PUBLISHING AG2022