Preference-Driven Haptic Feedback for Realistic Virtual Surfaces

A team at the University of Southern California developed a system to create highly realistic and personalized ‘virtual surfaces’ that can inform more advanced haptic feedback systems. Haptic feedback uses specific vibrations to simulate how it feels to touch a surface. This new approach allows people to customize their own virtual surfaces using a preference-based approach. The technology has many applications, including more advanced prosthetic devices that provide a highly realistic sense of touch and surgical training simulations that provide the trainee surgeon with a realistic impression of the virtual tissues they are interacting with.

Haptic feedback, in the form of specific vibrations, can be used to simulate our sense of touch, and has enormous potential in medical training, as well as in advanced prosthetic systems. By tweaking the vibrations slightly, the feedback can simulate what it feels like to touch various surfaces, from rough to smooth, hard to squishy, and everything in between. However, getting it right is tricky, as our sense of touch is very sensitive, and we are highly attuned to the subtleties of various surfaces.

Simply allowing a computer to ‘read’ a surface, and play it back as a haptic sensation can lead to poor accuracy in mimicking how a surface actually feels. This new approach aims to allow people to craft their own haptic sensations that mimic what they perceive a surface to feel like. “You’re measuring parameters of exactly how they feel it, rather than just mimicking what we can record,” said Matthew Fontaine, a researcher involved in the study. “There’s going to be some error in how you recorded that texture, to how you play it back.”

This new system is based on a preference-driven framework. “We ask users to compare their feeling between the real texture and the virtual texture,” said Shihan Lu, another researcher involved in the study. “The model then iteratively updates a virtual texture so that the virtual texture can match the real one in the end.”

Put simply, a user touches a real surface, and then the computer provides them with three haptic feedback options, each of which is intended to mimic the real texture. The user picks the one that they feel is most like the real texture. Then the computer generates two new virtual textures that are similar to the chosen texture. Again, the user chooses the most realistic texture.

Through several iterative rounds, the computer begins to narrow in on the features of the virtual texture that make it most realistic for the user. The user can also change the amount of friction included in the virtual texture using a simple slider – friction is a key component in how we perceive surfaces and textures. The end result is a highly realistic virtual surface.

See a video about the technique below: