International Journal of Innovative Research in Computer and Communication Engineering
ISSN Approved Journal | Impact factor: 8.771 | ESTD: 2013 | Follows UGC CARE Journal Norms and Guidelines
| Monthly, Peer-Reviewed, Refereed, Scholarly, Multidisciplinary and Open Access Journal | High Impact Factor 8.771 (Calculated by Google Scholar and Semantic Scholar | AI-Powered Research Tool | Indexing in all Major Database & Metadata, Citation Generator | Digital Object Identifier (DOI) |
| TITLE | A Survey on Multimodal Mental State Detection using Artificial Intelligence |
|---|---|
| ABSTRACT | Stress and mental health disorders have become increasingly prevalent, demanding reliable and continuous assessment methods beyond traditional clinical evaluation. Recent advances in artificial intelligence have enabled automated stress detection using behavioral and physiological signals; however, unimodal approaches often suffer from limited robustness and subjectivity. This paper presents a structured survey of AI-based stress and mental state detection systems with an emphasis on multimodal frameworks. Various data modalities, including facial expressions, speech, survey responses, galvanic skin response, heart rate, and motion signals, are reviewed along with commonly used datasets and fusion strategies. Furthermore, a stress-relevant class mapping and variance-weighted fusion space compression strategy is discussed to address the exponential growth in multimodal decision complexity. The proposed approach reduces the outcome space from 35,000 combinations to 45 clinically meaningful stress states while preserving interpretability. The survey highlights current challenges, applications, and future research directions toward scalable, real-time, and ethical multimodal stress monitoring systems. |
| AUTHOR | PROF. VAISHNAVI SONAWANE, MASOOD MADKI Department of Artificial Intelligence & Machine Learning, AISSMS Polytechnic, Pune, India Diploma Student, Department of Artificial Intelligence & Machine Learning, AISSMS Polytechnic, Pune, India |
| VOLUME | 181 |
| DOI | DOI: 10.15680/IJIRCCE.2026.1402016 |
| pdf/16_A Survey on Multimodal Mental State Detection using Artificial Intelligence.pdf | |
| KEYWORDS | |
| References | [1] S. R. Livingstone and F. A. Russo, “The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions,” PLoS ONE, vol. 13, no. 5, e0196391, 2018. [2] I. J. Goodfellow et al., “Challenges in representation learning: A report on three machine learning contests,” Neural Networks, vol. 64, pp. 59–63, 2015. (FER2013 Dataset) [3] S. Koelstra et al., “DEAP: A database for emotion analysis using physiological signals,” IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18–31, 2012. [4] P. Schmidt et al., “Introducing WESAD, a multimodal dataset for wearable stress and affect detection,” Proceedings of the ACM International Conference on Multimodal Interaction (ICMI), pp. 400–408, 2018. [5] G. Giannakakis et al., “Review on psychological stress detection using biosignals,” IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 440–460, 2022. |
