Publications related to Fashioning the Future: Unlocking the Creative Potential of Deep Generative Models for Design Space Exploration

Fashioning Creative Expertise with Generative AI: Graphical Interfaces for GAN-Based Design Space Exploration Better Support Ideation Than Text Prompts for Diffusion Models

Pierre Dillenbourg, Kevin Gonyop Kim, Wei Jiang, Tanja Christina Käser Jacober, Richard Lee Davis, Thiemo Wambsganss

This paper investigates the potential impact of deep generative models on the work of creative professionals. We argue that current generative modeling tools lack critical features that would make them useful creativity support tools, and introduce our own ...

2024

Text-to-Microstructure Generation Using Generative Deep Learning

Jamie Paik

Designing novel materials is greatly dependent on understanding the design principles, physical mechanisms, and modeling methods of material microstructures, requiring experienced designers with expertise and several rounds of trial and error. Although rec ...

Wiley-V C H Verlag Gmbh2024

Machine learning-aided generative molecular design

Philippe Schwaller, Jeff Guo

Machine learning has provided a means to accelerate early-stage drug discovery by combining molecule generation and filtering steps in a single architecture that leverages the experience and design preferences of medicinal chemists. However, designing mach ...

Nature Portfolio2024

MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning

Alexandre Massoud Alahi, Mariam Ahmed Mahmoud Hegazy Hassan, Mohamed Ossama Ahmed Abdelfattah

Current transformer-based skeletal action recognition models tend to focus on a limited set of joints and low-level motion patterns to predict action classes. This results in significant performance degradation under small skeleton perturbations or changin ...

2024

Deep Generative Models for Autonomous Driving: from Motion Forecasting to Realistic Image Synthesis

Saeed Saadatnejad

Forecasting is a capability inherent in humans when navigating. Humans routinely plan their paths, considering the potential future movements of those around them. Similarly, to achieve comparable sophistication and safety, autonomous systems must embrace ...

EPFL2023

4M: Massively Multimodal Masked Modeling

Amir Roshan Zamir, Roman Christian Bachmann, Shuqing Teresa Yeo, David Mizrahi, Oguzhan Fatih Kar

Current machine learning models for vision are often highly specialized and limited to a single modality and task. In contrast, recent large language models exhibit a wide range of capabilities, hinting at a possibility for similarly versatile models in co ...

Neural Information Processing Systems (Nips)2023

Deep Learning for Localized-Haptic Feedback in Tactile Surfaces

Camilo Hernandez Mejia

Touchscreens are nowadays the preferred choice for user interfaces in consumer electronics. Significant technological advances have been made in terms of touch sensing and visual quality. However, the haptic feedback offered by commercial products is still ...

EPFL2023

Interpretable Representation Learning and Evaluation for Abstractive Summarization

Andreas Thomas Marfurt

Abstractive summarization has seen big improvements in recent years, mostly due to advances in neural language modeling, language model pretraining, and scaling models and datasets. While large language models generate summaries that are fluent, coherent, ...

EPFL2023

Controlling DNA nanodevices with light-switchable buffers

Cristian Pezzato, Valentin Jean Périllat, Cesare Berton

Control over synthetic DNA-based nanodevices can be achieved with a variety of physical and chemical stimuli. Actuation with light, however, is as advantageous as difficult to implement without modifying DNA strands with photo-switchable groups. Herein, we ...

Royal Society of Chemistry2023

Dense Image-based Predictions for Comics Analysis

Deblina Bhattacharjee

Dense image-based prediction methods have advanced tremendously in recent years. Their remarkable development has been possible due to the ample availability of real-world imagery. While these methods work well on photographs, their abilities do not genera ...

EPFL2023

Explanation of Optical Force and Torque on Nanomotors as the Interplay of Huygens Sources

Olivier Martin, Karim Achouri, Mintae Chung

Being able to understand how optical forces emerge from the interaction of light with matter is paramount for controlling the motion of nanoparticles as well as powering nanomotors. The purpose of this work is to uncover the physical mechanisms at the orig ...

IEEE2022

GANs for All: Supporting Fun and Intuitive Exploration of GAN Latent Spaces

Pierre Dillenbourg, Kevin Gonyop Kim, Wei Jiang, Richard Lee Davis

We have developed a new tool that makes it possible for people with zero programming experience to intentionally and meaningfully explore the latent space of a GAN. We combine a number of methods from the literature into a single system that includes multi ...

PMLR2022

BIGPrior: Towards Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration

Sabine Süsstrunk, Majed El Helou

Classic image-restoration algorithms use a variety of priors, either implicitly or explicitly. Their priors are hand-designed and their corresponding weights are heuristically assigned. Hence, deep learning methods often produce superior image restoration ...

2022

Bayesian Inversion for Nonlinear Imaging Models Using Deep Generative Priors

Michaël Unser, Thanh-An Michel Pham, Pakshal Narendra Bohra, Jonathan Yuelin Dong

Most modern imaging systems incorporate a computational pipeline to infer the image of interest from acquired measurements. The Bayesian approach to solve such ill-posed inverse problems involves the characterization of the posterior distribution of the im ...

2022

Bridging the gap between model-driven and data-driven methods in the era of Big Data

Gael Lederrey

Data-driven and model-driven methodologies can be regarded as competitive fields since they tackle similar problems such as prediction. However, these two fields can learn from each other to improve themselves. Indeed, data-driven methodologies have been d ...

EPFL2022

DeepNC: Deep Generative Network Completion

Andreas Oliver Spitz

Most network data are collected from partially observable networks with both missing nodes and missing edges, for example, due to limited resources and privacy settings specified by users on social media. Thus, it stands to reason that inferring the missin ...

IEEE COMPUTER SOC2022

RG-Flow: a hierarchical and explainable flow model based on renormalization group and sparse prior

Dian Wu

Flow-based generative models have become an important class of unsupervised learning approaches. In this work, we incorporate the key ideas of renormalization group (RG) and sparse prior distribution to design a hierarchical flow-based generative model, RG ...

IOP Publishing Ltd2022

Context-Aware Learning for Generative Models

Ricardo Andres Chavarriaga Lozano, José del Rocio Millán Ruiz, Serafeim Perdikis, Robert Leeb

This work studies the class of algorithms for learning with side-information that emerges by extending generative models with embedded context-related variables. Using finite mixture models (FMMs) as the prototypical Bayesian network, we show that maximum- ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Conditional LSTM-GAN for Melody Generation from Lyrics

Simon Canales, Yi Yu

Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables us to learn and discover latent relationships between interesting lyrics and accompanying melodies. Unfortunately, the limi ...

2021

Unsupervised Domain Adaptation for Cross-Subject Few-Shot Neurological Symptom Detection

Mahsa Shoaran, Bingzhao Zhu

Modern machine learning tools have shown promise in detecting symptoms of neurological disorders. However, current approaches typically train a unique classifier for each subject. This subject-specific training scheme requires long labeled recordings from ...

IEEE2021