International Workshop on Affective Interaction between Humans and Machines in Multicultural Society (IWAM2022)

The AI-Avatar system is a system that helps psychologists assess a participant's emotions. The system's method of predicting emotions takes into account a number of factors, including action units, facial keypoints, body movement, eye movement, and voice, each of which employs machine learning to extract key features. The action unit is used as part of the feature because the movements of the facial muscles correspond to the emotions expressed. We have applied a machine learning model to detect action units, which will allow us to determine the emotional expressions of the participants. Facial expression analysis is one of the few techniques available to assess emotions in real time. Movement of the upper body in a forward or backward inclination was another feature that corresponded to the emotions expressed by the participants. Movement of the upper body in a forward or backward inclination was a feature that corresponded to the emotions expressed by the participants. To determine upper body movement, we use pose estimation models in a different method. Rolling eyes is another feature that indicates the emotions expressed by the participants, so we detected eye rolling to help assess the participants' moods. Rolling eyes is another feature that corresponds to the emotions expressed by the participants, so we detect eye rolling to help assess the participants' moods. The AI-Avatar system also applied an acoustic model to help predict the participants' moods. All of the aforementioned feature inputs are then used to predict the participants' moods for the most accurate results.