Researchers developed a soft, vision-based sensor that enables both touch and proximity sensing in flexible robotic arms

Soft robots need advanced sensing to safely interact with humans, but combining multiple sensors in soft materials is difficult. Researchers have developed a new vision-based sensing system called ProTac that allows a robot to switch between touch and proximity sensing using a soft skin that changes transparency. Integrated into robot arms, ProTac enables safer and more precise control. This innovation offers a scalable solution for improving human-robot interaction in complex, real-world environments.

Robots are becoming increasingly integrated into everyday environments--- from homes and hospitals to factories and farms. However, safely operating around humans requires more than strength or speed. Robots must also sense their surroundings, detect physical contact, and respond quickly. Conventional sensors, especially those embedded in soft materials, often fall short when it comes to real-time, large-area tactile and proximity sensing.

To address this challenge, a research team led by Professor Van Anh Ho and Dr. Quan Khanh Luu from Japan Advanced Institute of Science and Technology (JAIST), Japan, developed ProTac, a novel vision-based soft sensing skin for robots. The study was published in the journal IEEE Transactions on Robotics on July 28, 2025, and presents a scalable approach to multimodal robotic perception.

At the core of ProTac is a polymer-dispersed liquid crystal (PDLC) layer that can switch between transparent and opaque states when voltage is applied. When transparent, embedded cameras can "see" through the skin to detect nearby objects. When opaque, those same cameras track how the skin deforms, allowing the robot to sense touch, pressure, and the location of contact. This dual-mode sensing enables real-time environmental perception using a single soft layer, eliminating the need for complex embedded electronics.

"This innovation with sensor design simplicity allows robots to perceive both contact and nearby obstacles across a large-area skin in real time, which is difficult to achieve with conventional electronic skins," Prof. Ho explained.

To validate the design, the team developed a prototype called the ProTac link, a cylindrical robot arm segment wrapped in a soft sensing skin and equipped with stereo cameras at both ends. This prototype can detect approaching objects from multiple angles, estimate distance, and recognise multiple touch points with high accuracy. It also supports adaptive behaviors such as proximity-based speed adjustment and reflexive contact avoidance.

"ProTac can be applied to dexterous robotic manipulation in various domains where safety and delicate physical interaction are critical," says Dr. Luu.

The researchers envision broad applications across collaborative robotics, assistive devices, and soft robotic systems in agriculture or domestic care. In factories, ProTac-equipped robots could slow or stop when a human worker approaches. In elder care, service robots could provide physical assistance while maintaining safe, responsive contact. In the long term, ProTac could contribute to the development of full-body sensing systems for humanoid robots.

"In the long term, this research lays the groundwork for intelligent general-purpose humanoid robots equipped with full-body multimodal soft sensing skins, especially those with a sense of touch," said Dr. Ho.

Alongside the skin, the team developed learning algorithms and control strategies that convert sensor data into robotic actions, enabling responsive and autonomous behavior in dynamic environments. Designed with simplicity, modularity, and scalability in mind, the ProTac system can be integrated into both new and existing robotic platforms.

To accelerate progress in the field, the researchers have made their design files, models, and software open source. By combining mechanical simplicity with multimodal perception, ProTac brings robots a step closer to understanding--- and safely interacting with--- the complex, tactile world around them.

Image title: ProTac's multimodal sensing mechanism and real-world applications
Image caption: Conceptual overview of the vision-based ProTac sensing technology. ProTac can actively switch between proximity and tactile sensing modes, relying on input images captured by inner cameras and a soft functional skin with controllable transparency.
Image credit: Van Anh Ho from JAIST
License type: Original Content
License statement: Cannot be reproduced without permission.

Image title: Representative applications of ProTac in human-robot interaction
Image caption: Left: A collaborative robotic arm equipped with ProTac detects proximity and contact regions during interaction with a human co-worker. Right: A service robot with ProTac-enhanced arms operates safely in dynamic environments, using proximity sensing to avoid collisions and tactile sensing for delicate object manipulation.
Image credit: Van Anh Ho from JAIST
License type: Original Content
License statement: Cannot be reproduced without permission.

Reference

Funding information

This work was supported in part by the JSPS Grant-in-Aid for Scientific Research (KAKENHI) under Grant 24KJ1203 and in part by the JST Precursory Research for Embryonic Science and Technology PRESTO under Grant JPMJPR2038.

August 21, 2025