SKILL: Structured Knowledge Infusion for Large Language Models

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

State-of-the-art results in large language models (LLMs) often rely on scale, which becomes computationally expensive. This has sparked a research agenda to reduce these models' parameter counts and computational costs without significantly impacting their ...

The increasing complexity of transformer models in artificial intelligence expands their computational costs, memory usage, and energy consumption. Hardware acceleration tackles the ensuing challenges by designing processors and accelerators tailored for t ...

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

Over the course of history, the relationship between cities and their waters has shown different gradients of interweaving, marked by cycles of bonding and distancing. Following a period of complete neglect of urban watercourses, the versatile, multifacete ...

The field of text-to-video retrieval has advanced significantly with the evolution of language models and large-scale pre-training on generated caption-video pairs. Current methods predominantly focus on visual and event-based details, making retrieval lar ...

Buildings play a pivotal role in the ongoing worldwide energy transition, accounting for 30% of the global energy consumption. With traditional engineering solutions reaching their limits to tackle such large-scale problems, data-driven methods and Machine ...

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

Given the prompt “Rome is in”, can we steer a language model to flip its prediction of an incorrect token “France” to a correct token “Italy” by only multiplying a few relevant activation vectors with scalars? We argue that successfully intervening on a mo ...

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

Geomembrane systems have been used at hydropower plants as rehabilitation and mitigation technology for several decades and are now used worldwide. In the context of planning an energy transition that promotes the sustainable use of water resources for ene ...

Purpose: Despite being an authentic carrier of various cultural practices, the human body is often underutilised to access the knowledge of human body. Digital inventions today have created new avenues to open up cultural data resources, yet mainly as appa ...

Writing high-quality procedural texts is a challenging task for many learners. While example-based learning has shown promise as a feedback approach, a limitation arises when all learners receive the same content without considering their individual input ...

Various endeavours into semantic web technologies and ontology engineering have been made within the organisation of cultural data, facilitating public access to digital assets. Although models for conceptualising objects have reached a certain level of ma ...

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimen ...

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

We prove global in time well-posedness for perturbations of the 2D stochastic Navier-Stokes equations partial derivative( t)u + u center dot del u = Delta u - del p + sigma + xi, u(0, center dot ) = u(0),div (u) = 0, driven by additive space-time white noi ...

Time has always been a central factor in understanding the challenges of daily mobility. For a long time, and still today, methods of economic evaluation of transport projects have monetized time savings so that they can be included in the cost–benefit ana ...

To address the challenge of pretraining foundational models with large datasets, a multi-task approach is proposed, thus helping to overcome the data scarcity problem in biomedical imaging. ...

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

SKILL: Structured Knowledge Infusion for Large Language Models

Chat with Graph Search

Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers

Accelerator-driven Data Arrangement to Minimize Transformers Run-time on Multi-core Architectures

Random matrix methods for high-dimensional machine learning models

Quartiers rhodaniens en transition

Beyond Essentials: Nuanced and Diverse Text-to-video Retrieval

Fusing Pre-existing Knowledge and Machine Learning for Enhanced Building Thermal Modeling and Control

Machine Learning for Modeling Stock Returns

Improving Generalization of Pretrained Language Models

Activation Scaling for Steering and Interpreting Language Models

Modeling Structured Data in Attention-based Models

A comprehensive review on geomembrane systems application in hydropower

Unlocking a multimodal archive of Southern Chinese martial arts through embodied cues

Enhancing Procedural Writing Through Personalized Example Retrieval: A Case Study on Cooking Recipes

Building a Knowledge Graph of Chinese Kung Fu Masters From Heterogeneous Bilingual Data

The multimodality cell segmentation challenge: toward universal solutions

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Global existence for perturbations of the 2D stochastic Navier-Stokes equations with space-time white noise

Appropriations and Uses of Travel Time

Multi-task learning for medical foundation models

Transformer Models for Vision