Augmenting Humans and AI with User Interfaces
Associate Professor：XIE Haoran
Computer Graphics, User Interface, Human Augmentation
Deep Learning, Sketch, Human-AI Interaction, Human-Centered AI, Physical Augmentation, Wearable Device
Skills and background we are looking for in prospective students
Strong desire to conduct the challenging topics in novel interdisciplinary research fields and realize their own "dreams". It is desirable but not required, willing to master the graphics and machine learning technologies.
What you can expect to learn in this laboratory
Students can develop future value-creating capabilities with professional expertise and wide knowledge in the coming super-smart society. They can gain the ability to think cross-disciplinarily, utilize feasible approaches, and create innovations. We encourage students to accumulate knowledge from different fields of study and foster interdisciplinary thinking. The development of augmented interfaces acquires practical skills for actual social issues.
【Job category of graduates】
ICT, R&D, Manufacturing, Creative Industry, etc.
Fig.1 Research topics in augmented interfaces
We aim to conduct the advanced topics in Augmented(X) Interfaces for realizing Society 5.0※ (Fig. 1). We are proposing user interfaces that enable even novice users to democratize the advanced artificial intelligence and robotics technologies which are only available only by domain experts. We aim to realize a future society in which anyone can pursue own dreams by enabling users to augment own creativity, physical capabilities, and cognitive abilities. Our research vision is to realize the super human and super information technology through the interface technology in the creative society, aiming at the social implementation that can harmonize the human and machine.
※super-smart society, the concept of the future society proposed in Japan 5th Science and Technology Basic Plan.
(1) Sketch Interfaces with Creative AI
The rapid development of deep learning related technologies has led to social reforms nowadays. However, creative artificial intelligence (AI) technology is still a challenging and important research topic attracting worldwide attention. This research topic aims to support the media content production by using data-driven methods such as deep learning based generative models. Specific examples include design interface that supports the interactive design of 3D airplane geometry and generative interface that enables to edit the flow motion from user sketches.
(2) Augmented AI with Human Impact
This research topic aims to conduct the user interfaces to consider human impact in the loop of machine learning process including interfaces for human-AI interaction, human-in-the-loop machine learning and human computation. Specific examples include annotation interface for data collection using crowdsourcing, and 3D model construction from point clouds.
(3) Augmented Human with AI Technologies
Recent developments of AI, XR and 5G technologies are augmenting every aspect of our lives. We are in the "Augmented Age" which offers us the greatest advantages and possibilities in human history. This research topic aims to develop technologies to recover or augment human's capabilities using information technology and robotics. Specific examples include wearable device that mimics the animal tails to augment human body functions and emotional expressiveness, and the wearable supernumerary robotic limbs to support humans during everyday activities.
- Z. Huang, Y. Peng, T. Hibino, C. Zhao, H. Xie, T. Fukusato, K. Miyata. dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching. Journal of Computational Visual Media, Vol 8, 63-77, 2022.
- Z. Hu, H. Xie, T. Fukusato, T. Sato and T. Igarashi. Sketch2VF: Sketch-Based Flow Design with Conditional Generative Adversarial Network. Computer Animation and Virtual Worlds, 30(3-4),1-11, 2019.
- H. Xie, T. Igarashi and K. Miyata. Precomputed Panel Solver for Aerodynamics Simulation. ACM Transactions on Graphics. 37(2), 1-12, 2018. (ACM SIGGRAPH 2018)
EquipmentDeep Learning Server, 3D Printer and Scanner
Sensors, Projectors, Depth Cameras, JAIST FabLab
Our goal is to help students to be ready to work immediately in research and development through collaborative and individual projects, emphasizing own initiative in our research vision of the augmented interface. To achieve these goals, students are encouraged to develop own research styles and communication skills through research projects. They are required to participate in a variety of research projects to clarify their own potential and research vision, and to explore novel solutions in collaboration with faculty member. In addition, we will support students to achieve top-level research outcomes by providing opportunities for joint research and industry-academic collaboration with domestic and international research institutions.