An End-to-End Platform for Multi-Modal Machine Learning Affective Computing Services

Name: Emad Kasaeyan Naeini

Chair: Nikil Dutt

Date: August 17, 2022

Time: 10:30 AM

Location: DBH 3011

Committee: Amir Rahmani, Fadi Kurdahi

Title:  An End-to-End Platform for Multi-Modal Machine Learning Affective Computing Services

Abstract:

Smart affective computing applications deliberately influence pain, emotion and other affective phenomena, and are fundamental to human experience, health and well-being. Affective states such as pain, stress, and emotion are intrinsically subjective in nature, posing challenges for objective assessment and quantification of such affective phenomena. Prior efforts in affective computing provide a foundation for the automated analysis of affective states, but still face challenges in real-life, everyday settings. We use pain as an exemplar for affective computing services. Pain assessment is critical for optimal treatment, and is particularly important during periods of acute pain since inadequately treated acute pain increases the risk of chronic pain. Historically, patients have served as the main assessment tool as they are able to self-report their pain presence and severity on standard, but somewhat subjective pain scales. However, it remains a challenge to assess pain from patients who cannot self-report. Automatic pain recognition systems could be crucial to facilitating accurate, objective, and real-time pain measurement, that can in turn improve pain management and ultimately lead to improved patient outcomes. Using pain as an exemplar of smart affective computing services, this thesis proposes the use of multimodal sensing of physiological and behavioral input data, transmits the data to edge and/or cloud nodes, and processes data with compute-intensive machine learning (ML) algorithms. The efficiency of ML-driven applications for affective computing (e.g., pain assessment) is greatly affected by run-time variations resulting from continuous stream of noisy input data, unreliable network connections, and the variations in computational requirements of ML algorithms. Towards that end, this thesis evaluates and automates objective and real-time multimodal pain assessment algorithms, and performs design space exploration of accuracy-performance-energy trade-offs and sense-compute co-optimization for multimodal machine learning (MMML) methods. The approach developed in this thesis could be used for the design and development of an end-to-end platform for multimodal machine learning affective computing services, together with a thorough analysis of their roles in the prediction, performance and energy in future studies.

Short bio:

Emad Kasaeyan Naeini is a Ph.D. candidate in Computer Engineering at the University of California, Irvine. He received his M.Sc. degree in Electrical and Computer Engineering from the Department of Electrical Engineering and Computer Science at the University of California Irvine in 2020 and his B.Sc. in Electrical Engineering from Sharif University of Technology, Tehran, Iran, in 2017. His research interests primarily include deep learning and machine learning, signal processing, healthcare Internet of Things (IoT), and healthcare data analysis.