Towards a cyber space where you can speak with peace of mind

Laboratory on Speech, Hearing, and Interaction
Senior lecturer：MAWALIM, Candy Olivia

E-mail：
［Research areas］
Voice Privacy, Spoken Language Processing, Acoustic Phonetics, Machine Learning
［Keywords］
Voice conversion, Speaker anonymization, Cognitive hearing, Hearing Aid Technology, Low-resource language processing, Human-computer interaction

Skills and background we are looking for in prospective students

Our laboratory welcomes students who are passionate about improving communication through speech and hearing technology, who can utilize the fundamentals of machine learning and signal processing, who have technical English skills, and who are motivated to engage in research with an international perspective.

What you can expect to learn in this laboratory

Through activities in this laboratory, students will acquire two core competencies: advanced data-scientific thinking and human-centered application skills. Specifically, students will acquire practical AI development skills, designing and implementing voice and acoustic signal processing technologies. In addition, through research on voice privacy, students will develop deep knowledge of data security measures necessary for protecting personal information and the ability to assess technical risks. Ultimately, students will refine their engineering design skills, applying cognitive science and HCI methods to solve abstract challenges like creating natural dialogue and improving speech intelligibility. This provides a robust foundation for developing socially impactful applications.

【Job category of graduates】 Information and communication technology, systems development, research positions

Research outline

Our laboratory focuses on the fundamental human communication skills of listening and speaking, aiming to achieve better interaction between humans and digital technology. Specifically, we integrate three fields: speech science, cognitive science, which elucidates human hearing and cognitive functions; and human-computer interaction, to address the following research themes.

1. Research on Human Speech and Hearing

Going beyond basic speech recognition and synthesis, we research how people convey emotions and intentions in their voices and how those are perceived. We develop speech technologies that consider human listening and speaking with the goal of solving issues in diverse communication settings. We also investigate acoustic phonetics across various languages, including those with low resources, and focuses on distinguishing between human-produced and machine-generated speech.

2. Privacy-Preserving Speech Technology

Our laboratory's primary focus is building speech technology that users can trust. Because voice data contains a significant amount of personal information, its handling requires consideration in ethics and technology. We are developing voice anonymization technology to prevent identification of individuals from privacy-preserving AI that learns while protecting confidential information. This allows users to safely interact with AI systems.

3. Application to Solving Social Issues

The technology developed in our research has practical applications in enhancing speech-related technologies, such as hearing aids and systems that assist people with communication disabilities. Our privacy-preserving technology will also enable the use of AI in highly confidential fields like healthcare, leading to practical applications that benefit society.

Figure 1. A privacy-preserving and multilingual speech AI system

Key publications

1. C. O. Mawalim, K. Galajit, J. Karnjana, S. Kidani, M. Unoki, “Speaker Anonymization by Modifying Fundamental Frequency and X-Vectors Singular Value”, Computer Speech and Language, Elsevier, vol. 73, 101326, 2022.
2. C. O. Mawalim, B. A. Titalim, S. Okada, and M. Unoki, “Non-Intrusive Speech Intelligibility Prediction Using an Auditory Periphery Model with Hearing Loss”, Applied Acoustics, vol. 214, 2023.
3. C. O. Mawalim, C. W. Leong, G. Sivan, H-H. Huang, and S. Okada, “Beyond accuracy: Multimodal modeling of structured speaking skill indices in young adolescents,” Computers and Education: Artificial Intelligence, Vol. 8, June 2025.

Equipment

Studio condenser microphones, advanced digital MEMS arrays, video cameras, podcast studio microphones, and Arduino prototyping kits