Publications related to Classification and Regression of Cognitive and Motivational Style from Text

Related publications (31)

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

EPFL2024

Social-Transmotion: Promptable Human Trajectory Prediction

Alexandre Massoud Alahi, Saeed Saadatnejad, Yang Gao, Kaouther Messaoud Ben Amor

Accurate human trajectory prediction is crucial for applications such as autonomous vehicles, robotics, and surveillance systems. Yet, existing models often fail to fully leverage the non-verbal social cues human subconsciously communicate when navigating ...

2024

Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling

Robert West, Maxime Jean Julien Peyrard, Marija Sakota

Generative language models (LMs) have become omnipresent across data science. For a wide variety of tasks, inputs can be phrased as natural language prompts for an LM, from whose output the solution can then be extracted. LM performance has consistently be ...

Assoc Computing Machinery2024

Driving and suppressing the human language network using large language models

Martin Schrimpf

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict t ...

Nature Portfolio2024

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Karl Aberer, Rémi Philippe Lebret, Mohammadreza Banaei

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

Assoc Computational Linguistics-Acl2023

XTab: Cross-table Pretraining for Tabular Transformers

Mahsa Shoaran, Bingzhao Zhu

The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data. However, most existing tabular self-supervised learning models fail to leverage information across multiple data t ...

JMLR.org2023

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Modeling Structured Data in Attention-based Models

Alireza Mohammadshahi

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

EPFL2023

Framing the News: From Human Perception to Large Language Model Inferences

Daniel Gatica-Perez

Identifying the frames of news is important to understand the articles' vision, intention, message to be conveyed, and which aspects of the news are emphasized. Framing is a widely studied concept in journalism, and has emerged as a new topic in computing, ...

Assoc Computing Machinery2023

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023

Backpropagation-free training of deep physical neural networks

Romain Christophe Rémy Fleury, Babak Rahmani, Ali Momeni, Matthieu Francis Malléjac

Recent successes in deep learning for vision and natural language processing are attributed to larger models but come with energy consumption and scalability issues. Current training of digital deep-learning models primarily relies on backpropagation that ...

American Association for the Advancement of Science2023

Multi-task prompt-RSVQA to explicitly count objects on aerial images

Nicola Antonio Santacroce, Devis Tuia, Christel Marie Tartini-Chappuis, Javiera Francisca Castillo Navarro, Charlotte Louise Sertic

Introduced to enable a wider use of Earth Observation images using natural language, Remote Sensing Visual Question Answering (RSVQA) remains a challenging task, in particular for questions related to counting. To address this specific challenge, we propos ...

2023

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Leonardo Petrini

Artificial intelligence, particularly the subfield of machine learning, has seen a paradigm shift towards data-driven models that learn from and adapt to data. This has resulted in unprecedented advancements in various domains such as natural language proc ...

EPFL2023

Interpretable Representation Learning and Evaluation for Abstractive Summarization

Andreas Thomas Marfurt

Abstractive summarization has seen big improvements in recent years, mostly due to advances in neural language modeling, language model pretraining, and scaling models and datasets. While large language models generate summaries that are fluent, coherent, ...

EPFL2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

Equivariant Neural Architectures for Representing and Generating Graphs

Clément Arthur Yvon Vignac

Graph machine learning offers a powerful framework with natural applications in scientific fields such as chemistry, biology and material sciences. By representing data as a graph, we encode the prior knowledge that the data is composed of a set of entiti ...

EPFL2023

Empathetic Conversational Agents for Distress Support

Kalpani Anuradha Welivita

Due to the increasing demands of today's fast-paced world, mental health concerns are on the rise, which necessitates innovative approaches to provide support to those in need. Open-domain conversational agents known as chatbots, offer a unique opportunit ...

EPFL2023

Deep Generative Models for Autonomous Driving: from Motion Forecasting to Realistic Image Synthesis

Saeed Saadatnejad

Forecasting is a capability inherent in humans when navigating. Humans routinely plan their paths, considering the potential future movements of those around them. Similarly, to achieve comparable sophistication and safety, autonomous systems must embrace ...

EPFL2023

TiC-SAT: Tightly-coupled Systolic Accelerator for Transformers

David Atienza Alonso, Giovanni Ansaloni, Alireza Amirshahi, Joshua Alexander Harrison Klein

Transformer models have achieved impressive results in various AI scenarios, ranging from vision to natural language processing. However, their computational complexity and their vast number of parameters hinder their implementations on resource-constraine ...

2023

Monotonicity Reasoning in the Age of Neural Foundation Models

Zeming Chen

The recent advance of large language models (LLMs) demonstrates that these large-scale foundation models achieve remarkable capabilities across a wide range of language tasks and domains. The success of the statistical learning approach challenges our unde ...

Springer2023