Publication

Predicting emotional response to visual stimuli, a machine learning approach

2022
Student Projects

Abstract

We created an emotion predicting model capable of predicting emotions in images using OpenAI CLIP as a backbone. Using the ArtEmis dataset which contains 80K paintings annotated on the base of perceived emotions (amusement, fear, etc..). We show that this method of predicting emotion is effective, outperforming previous methods in predicting dominant emotion (70% vs 60%) or positive/negative images (80% vs 77.7%). We leverage our method on text emotion prediction which allows to quickly identify the reason why an image has a certain affect. With this method we unveil CLIP’s political preferences, and discovered a lean towards the democrats. We also created an affect based query tool that allows the users to search a set of pictures with a prompt as well as an emotion.

Official source

https://infoscience.epfl.ch/entities/publication/6e802fe3-cfde-4d32-bc6c-d8b515df4666

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Graph Chat

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

2022
Student Projects

Abstract

Official source

https://infoscience.epfl.ch/entities/publication/6e802fe3-cfde-4d32-bc6c-d8b515df4666

About this result

Related concepts (32)

Emotion

Emotions are mental states brought on by neurophysiological changes, variously associated with thoughts, feelings, behavioral responses, and a degree of pleasure or displeasure. There is no scientific consensus on a definition. Emotions are often intertwined with mood, temperament, personality, disposition, or creativity. Research on emotion has increased over the past two decades, with many fields contributing, including psychology, medicine, history, sociology of emotions, and computer science.

Emotion in animals

Emotion is defined as any mental experience with high intensity and high hedonic content. The existence and nature of emotions in non-human animals are believed to be correlated with those of humans and to have evolved from the same mechanisms. Charles Darwin was one of the first scientists to write about the subject, and his observational (and sometimes anecdotal) approach has since developed into a more robust, hypothesis-driven, scientific approach.

Emotion recognition

Emotion recognition is the process of identifying human emotion. People vary widely in their accuracy at recognizing the emotions of others. Use of technology to help people with emotion recognition is a relatively nascent research area. Generally, the technology works best if it uses multiple modalities in context. To date, the most work has been conducted on automating the recognition of facial expressions from video, spoken expressions from audio, written expressions from text, and physiology as measured by wearables.

Related publications (2)

Predicting emotional response to visual stimuli, a machine learning approach

Chat with Graph Search

Facial Expressions Evaluation Survey

Recognition of Asymmetric Facial Action Unit Activities and Intensities