About The Workshop

Novel applications of affective computing have emerged in recent years in domains ranging from health care to the 5th generation mobile network. Many of these have found improved emotion classification performance when fusing multiple sources of data (e.g., audio, video, brain, face, thermal, physiological, environmental, positional, text, etc.). Multimodal affect recognition has the potential to revolutionize the way various industries and sectors utilize information gained from recognition of a person's emotional state, particularly considering the flexibility in the choice of modalities and measurement tools (e.g., surveillance versus mobile device cameras). Further, multimodal classification methods have been proven highly effective at minimizing misclassification error in practice and in dynamic conditions, and tend to be more stable over time compared to relying on a single modality, increasing their reliability in sensitive applications such as mental health monitoring and automobile driver state recognition. To continue the trend of lab to practice within the field and encourage new applications of affective computing, this workshop provides a unique forum for researchers to exchange ideas on future directions, including novel fusion methods and databases, innovations through interdisciplinary research, and emerging emotion sensing devices. Importantly, this workshop addresses the ethical use of novel applications of affective computing in real world scenarios, welcoming discussions on privacy, manipulation of users, and public fears and misconceptions regarding affective computing. This workshop seeks to explore the intersection between theory and ethical applications of affective computing, with a specific focus on multimodal data for affect recognition (e.g., expression and physiological signals) using pattern recognition, computer vision, and image processing.

The Effect of Model Compression on Fairness in Facial Expression Recognition

Samuil Stoychev and Hatice Gunes
Deep neural networks are computationally expensive which has motivated the development of model compression techniques to reduce the resource consumption. Nevertheless, recent studies have suggested that model compression can have an adverse effect on algorithmic fairness, amplifying existing biases in machine learning models. With this work we aim to extend those studies to the context of facial expression recognition. To do that, we set up a neural network classifier to perform facial expression recognition and implement several model compression techniques on top of it. We then run experiments on two facial expression datasets, namely the Extended Cohn-Kanade Dataset (CK+DB) and the Real-World Affective Faces Database (RAF-DB), to examine the individual and combined effect that compression techniques have on the model size, accuracy and fairness. Our experimental results show that: (i) Compression and quantisation achieve significant reduction in model size with minimal impact on overall accuracy for both CK+ DB and RAFDB; (ii) in terms of model accuracy, the classifier trained and tested on RAF-DB is more robust to compression compared to the CK+ DB; and (iii) for RAF-DB, the different compression strategies do not increase the gap in predictive performance across the sensitive attributes of gender, race and age which is in contrast with the results on the CK+ DB where compression seems to amplify existing biases for gender.

Multimodal stress state detection from facial videos using physiological signals and facial features

Yassine Ouzar, Lynda Lagha, Frédéric Bousefsaf and Choubeila Maaoui
Stress is a complex phenomenon that affects the body and mind on multiple levels, encompassing both psychological and physiological aspects. Recent studies have used multiple modalities to comprehensively describe stress by exploiting the complementarity of multimodal signals. In this paper, we study the feasibility of fusing facial features with physiological cues on human stress state estimation. We adopt a multiple modalities fusion using a camera as a single input source and based on remote photoplethysmography method for non-contact physiological signals measurement. The frameworks rely on modern AI techniques and the experiments were conducted using the new UBFC-Phys dataset dedicated to multimodal psychophysiological studies of social stress. The experimental results revealed high performance when fusing facial features with remote pulse rate variability with an accuracy of 91.07%.

An Ethical Discussion on BCI-Based Authentication

Tyree Lewis, Rupal Agarwal and Marvin Andujar
In recent times, Brain Computer Interface (BCI) applications have progressed as biometric authentication systems. Existing systems each follow the fundamental concepts of ensuring a user can be successfully authenticated into it. This comes with the complexity of developing systems that prevent a user’s data from being at risk. Furthermore, with each new integration to make these systems more secure, they may lead to ethical concerns that should be discussed to better understand if they will be beneficial to individuals in their daily life. In this paper, we present an ethical discussion of a BCI-based authentication system, that has the potential to combat the risks that occur in other forms of authentication systems. We additionally provide insight as to the ethical issues that can occur, so that it can be effective for everyone to utilize.

Expression Recognition using a Flow-based Latent-space Representation

Saandeep Aathreya and Shaun Canavan
Facial expression Recognition is a growing and important field that has applications in fields such as medicine, security, education, and entertainment. While there have been encouraging approaches that have shown accurate results on a wide variety of datasets, in many cases it is still a difficult problem to explain the results. To enable deployment of expression recognition applications in-the-wild, being able to explain why an particular expression is classified is an important task. Considering this, we propose to model flow-based latent representations of facial expressions, which allows us to further analyze the features and grants us more granular control over which features are produced for recognition. Our work is focused on posed facial expressions with a tractable density of the latent space. We investigate the behaviour of these tractable latent space features in the case of subject dependent and independent expression recognition. We employ a flow-based generative approach with minimal supervision introduced during training and observe that traditional metrics give encouraging results. When subject independent expressions are evaluated, a shift towards a stochastic nature, in the probability space, is observed. We evaluate our flow-based representation on the BU-EEG dataset showing our approach provides good separation of classes, resulting in more explainable results.