Fourth Workshop on Applied
Multimodal Affect Recognition

Workshop at the 18th IEEE International Conference on Automatic Face and Gesture Recognition

Call for Papers

About The Workshop

In recent years, the landscape of affective computing has witnessed remarkable advancements, finding applications in diverse domains such as healthcare and 5th generation mobile networks. Notably, the integration of multiple data sources, including audio, video, brain activity, facial expressions, thermal readings, physiological signals, environmental cues, positional data, and text, has significantly enhanced emotion classification performance. The flexibility in choosing modalities and measurement tools, such as surveillance or mobile device cameras, opens up avenues for revolutionary changes across various industries. Multimodal affect recognition, with its proven effectiveness in minimizing misclassification errors, emerges as a key player in applications ranging from mental health monitoring to automobile driver state recognition. This workshop serves as a pivotal platform for researchers to collaborate and exchange ideas on the future directions of affective computing. Topics of discussion include novel fusion methods, database innovations, interdisciplinary research breakthroughs, and emerging emotion-sensing devices. Recognizing the ethical implications of these advancements, the workshop delves into crucial issues like privacy, user manipulation, and public concerns surrounding affective computing. The anticipated growth of the affective computing market, projected to reach $140 billion by 2025, underscores the potential for groundbreaking applications. This workshop aims to explore the intersection between theoretical frameworks and ethical applications of affective computing, with a specific emphasis on multimodal data for affect recognition, including facial expressions and physiological signals. Building on the success of previous workshops, such as the Ubiquitous Emotion Recognition with Multimodal Mobile Interfaces at UbiComp 2018 and the first three editions of this workshop (AMAR) at FG 2020, ACII 2021, and ICPR 2022, the 4th AMAR workshop continues its focus on ethical applications of affect recognition using pattern recognition, computer vision, and image processing. Join us in shaping the future of affective computing, where innovation meets responsibility, and theory converges with ethical practice.

Keynote Speaker

Dr. Jeffrey Cohn
Professor of Psychology, Psychiatry, and Intelligent Systems, University of Pittsburgh
Adjunct Professor of Computer Science, Robotics Institute, Carnegie Mellon University
Chief Scientist and Co-Founder, Deliberate.ai


Dr. Jeffrey Cohn leads interdisciplinary and inter-institutional efforts to develop advanced methods of automatic analysis and synthesis of facial expression and prosody and applies those tools to research in human emotion, social development, nonverbal communication, psychopathology, and biomedicine. His research has been supported by grants from the U.S. National Institutes of Health, National Science Foundation, Autism Foundation, Office of Naval Research, and Defense Advanced Research Projects Agency among other sponsors. He is a scientific advisor for RealEyes and for Embodied, past Chair of the Steering Committee of the IEEE International Conference on Automatic Face and Gesture Recognition (FG) from 2011-2021, a Fellow of the Association for the Advancement of Affective Computing. He has co-edited IEEE Transactions in Affective Computing (TAC) and special issues on affective computing for IEEE Transactions in Pattern Analysis and Machine Intelligence, Journal of Image and Vision Computing, Pattern Recognition Letters, Computer Vision and Image Understanding, and ACM Transactions on Interactive Intelligent Systems.

Workshop Schedule (Tentative)

Time Activity
9:00AM - 9:10AM Welcome and Opening Remarks from Organizers
9:10AM - 10:10AM Keynote
10:10AM - 11:10AM Workshop Paper Presentation (Session 1)
11:10AM - 11:20AM Coffee Break
11:20AM - 12:20PM Workshop Paper Presentations (Session 2)
12:20PM - 12:30PM Closing Remarks

Call for Papers

This workshop will explore ethical considerations in applied affect recognition, fostering conversations on topics utilizing multimodal data, including but not limited to 2D, 3D, thermal, brain, physiological, and mobile sensor signals. The goal is to highlight current use cases for affective computing and identify emerging applications, stimulating future research. The workshop is designed for researchers across various fields, including Brain-Computer Interfaces (BCI), affective computing, biometrics, computer vision, human-computer interaction, behavioral sciences, social sciences, and policymakers.

The workshop will focus on questions such as:

  • What inter-correlations exist between facial affect (e.g., expression) and other modalities (e.g., EEG, fNIRS)?
  • How can multimodal data be used in real-world applications of affect recognition, such as predicting stress, real-time ubiquitous emotion recognition, and understanding the impact of mood on ubiquitous subject identification?
  • How can we facilitate the collection of multimodal data for applied affect recognition?
  • What are the ethical implications of working on such questions?
  • How can we mitigate the ethical concerns that such work produces?
  • Can we positively address public fears and misconceptions regarding applied affective computing?
Topics of interest include, but are not limited to, ethical applications of the following:
  • Health applications with a focus on multimodal affect
  • Multimodal affective computing for cybersecurity applications (e.g., biometrics and IoT security)
  • Inter-correlations and fusion of ubiquitous multimodal data as it relates to applied emotion recognition (e.g. face and EEG data)
  • Leveraging ubiquitous devices to create reliable multimodal applications for emotion recognition
  • Applications of in-the-wild data vs. lab controlled
  • Facilitation and collection of multimodal data (e.g. ubiquitous data) for applied emotion recognition
  • Engineering applications of multimodal affect (e.g., robotics, social engineering, domain inspired hardware / sensing technologies, etc.)
  • Privacy and security
  • Institutionalized bias
  • Trustworthy applications of affective computing
  • Equal access to ethical applications of affective computing (e.g. medical applications inaccessible due to wealth inequality)
NOTE: Topics that do not demonstrate an existing or potential application of affective computing / emotion recognition are not topics of interest for this workshop.

Authors are invited to submit long and short papers in the IEEE format. Submissions to AMAR2024 should have no substantial overlap with any other paper submitted to FG 2024 or any paper already published. Papers presented at AMAR2024 will appear in the IEEE Xplore digital library. Papers must follow the FG paper format (double-blind).

How to Submit:
Paper should be submitted to EasyChair for review.

Important dates:
Paper submission: March 18, 2024
Decision to Authors: April 8, 2024
Camera-ready papers due: April 22, 2024
Workshop: TBA


Accepted Papers (TBA)

Workshop Organizers

Dr. Shaun Canavan
Shaun Canavan
University of South Florida
Dr. Tempestt Neal
Tempestt Neal
University of South Florida
Dr. Marvin Andujar
Marvin Andujar
University of South Florida
Dr. Saurabh Hinduja
Saurabh Hinduja
University of Pittsburgh
Dr. Lijun Yin
Lijun Yin
Binghamton University

Program Committee (TBA)