Recherche

Afficher tous les résultats pour

Accueil

Publication

SKILL: Structured Knowledge Infusion for Large Language Models

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (42)

Page 2 sur 3

Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers

Xiuying Wei, Caglar Gulcehre, Skander Moalla

State-of-the-art results in large language models (LLMs) often rely on scale, which becomes computationally expensive. This has sparked a research agenda to reduce these models' parameter counts and computational costs without significantly impacting their ...

2024

Accelerator-driven Data Arrangement to Minimize Transformers Run-time on Multi-core Architectures

David Atienza Alonso, Giovanni Ansaloni, Alireza Amirshahi

The increasing complexity of transformer models in artificial intelligence expands their computational costs, memory usage, and energy consumption. Hardware acceleration tackles the ensuing challenges by designing processors and accelerators tailored for t ...

2024

Random matrix methods for high-dimensional machine learning models

Antoine Philippe Michel Bodin

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024

Quartiers rhodaniens en transition

Sara Sonia Formery Regazzoni

Over the course of history, the relationship between cities and their waters has shown different gradients of interweaving, marked by cycles of bonding and distancing. Following a period of complete neglect of urban watercourses, the versatile, multifacete ...

EPFL2024

Beyond Essentials: Nuanced and Diverse Text-to-video Retrieval

Yuchen Yang

The field of text-to-video retrieval has advanced significantly with the evolution of language models and large-scale pre-training on generated caption-video pairs. Current methods predominantly focus on visual and event-based details, making retrieval lar ...

Institute of Electrical and Electronics Engineers Inc.2024

Fusing Pre-existing Knowledge and Machine Learning for Enhanced Building Thermal Modeling and Control

Loris Di Natale

Buildings play a pivotal role in the ongoing worldwide energy transition, accounting for 30% of the global energy consumption. With traditional engineering solutions reaching their limits to tackle such large-scale problems, data-driven methods and Machine ...

EPFL2024

Machine Learning for Modeling Stock Returns

Teng Andrea Xu

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

EPFL2024

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023

Activation Scaling for Steering and Interpreting Language Models

Robert West

Given the prompt “Rome is in”, can we steer a language model to flip its prediction of an incorrect token “France” to a correct token “Italy” by only multiplying a few relevant activation vectors with scalars? We argue that successfully intervening on a mo ...

Association for Computational Linguistics (ACL)2024

Modeling Structured Data in Attention-based Models

Alireza Mohammadshahi

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

EPFL2023

A comprehensive review on geomembrane systems application in hydropower

Giovanni De Cesare, Samuel Luke Vorlet

Geomembrane systems have been used at hydropower plants as rehabilitation and mitigation technology for several decades and are now used worldwide. In the context of planning an energy transition that promotes the sustainable use of water resources for ene ...

2024

Unlocking a multimodal archive of Southern Chinese martial arts through embodied cues

Sarah Irene Brutton Kenderdine, Yumeng Hou

Purpose: Despite being an authentic carrier of various cultural practices, the human body is often underutilised to access the knowledge of human body. Digital inventions today have created new avenues to open up cultural data resources, yet mainly as appa ...

2023

Enhancing Procedural Writing Through Personalized Example Retrieval: A Case Study on Cooking Recipes

Tanja Christina Käser Jacober, Antoine Bosselut, Thiemo Wambsganss, Seyed Parsa Neshaei, Jibril Albachir Frej, Paola Mejia Domenzain, Luca Mouchel, Tatjana Nazaretsky

Writing high-quality procedural texts is a challenging task for many learners. While example-based learning has shown promise as a feedback approach, a limitation arises when all learners receive the same content without considering their individual input ...

2024

Building a Knowledge Graph of Chinese Kung Fu Masters From Heterogeneous Bilingual Data

Lin Yuan, Yumeng Hou, Lin Yuan

Various endeavours into semantic web technologies and ontology engineering have been made within the organisation of cultural data, facilitating public access to digital assets. Although models for conceptualising objects have reached a certain level of ma ...

2023

The multimodality cell segmentation challenge: toward universal solutions

Maxime Emmanuel Scheder, Jun Ma, Sahand Jamal Rahi, Xin Yang, Vojislav Gligorovski, Marco Labagnara

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimen ...

Nature Publishing Group2024

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Karl Aberer, Rémi Philippe Lebret, Mohammadreza Banaei

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

Assoc Computational Linguistics-Acl2023

Global existence for perturbations of the 2D stochastic Navier-Stokes equations with space-time white noise

Martin Hairer

We prove global in time well-posedness for perturbations of the 2D stochastic Navier-Stokes equations partial derivative( t)u + u center dot del u = Delta u - del p + sigma + xi, u(0, center dot ) = u(0),div (u) = 0, driven by additive space-time white noi ...

Springernature2024

Appropriations and Uses of Travel Time

Ludy Juliana González Villamizar

Time has always been a central factor in understanding the challenges of daily mobility. For a long time, and still today, methods of economic evaluation of transport projects have monetized time savings so that they can be included in the cost–benefit ana ...

ISTE Wiley2023

Multi-task learning for medical foundation models

Jiancheng Yang

To address the challenge of pretraining foundational models with large datasets, a multi-task approach is proposed, thus helping to overcome the data scarcity problem in biomedical imaging. ...

SPRINGERNATURE2024

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023