During our meetup on April 6th, 2022, we explored the role of haptics in XR. Guido Gioioso presented recent research on the topic and a wearable haptic device equipped with novel features developed by WEART.
About Guido Gioioso
Guido Gioioso is a PhD graduate in robotics and haptics at the University of Siena and a co-founder of WEART. With a special interest in haptics, his expertise is in robotics and automation, particularly the control of underactuated robotic hands. In 2019, his research team at the University of Siena evolved into WEART — a company that develops wearable devices designed for the digitization of the sense of touch.
Wearable Haptic Devices
WEART develops wearable haptic devices that enable a naturalistic and comfortable interaction with virtual objects, surfaces, and materials, allowing people to grasp, touch, feel, and manipulate content. By implementing rich haptic feedback — a combination of forces, vibrations, and thermal cues — the experience is brought to a high level of realism, which is deemed crucial to engagement and effective learning.
From Grounded to Wearable
The starting point of the research that led to the development of WEART is based on understanding the sense of touch as an important aspect of perception and the human hand as the most powerful interface for interacting with the world. The most common and commercially available haptic devices operated with hands are grounded (seen in the video below).
Because grounded haptic devices are operated in constrained environments, one of the goals is to make haptic devices more accessible by developing a “wearable, light-weight, and comfortable” device. As Gioioso explained, such a device must be able to render the forces involved in those actions onto the body so that people are given the perception of “having the object in their hands without having it in reality.” Another goal is to bring haptic devices beyond the typical use cases within hospitals, medical institutions, and laboratories into people’s everyday lives for work, learning, entertainment, and other activities.
Influential Research on Cutaneous Feedback
Building a device that achieves the above-stated goals involves both engineering and neuroscience because it requires an understanding of how the human body works. Based on research, a wearable haptic device must operate on cutaneous feedback — information acquired through sensors in the skin. Gioioso provided a video from a 2005 study in which a participant is seen attempting to grasp objects on a table without tactile perception. As demonstrated, without cutaneous feedback, the task cannot be performed successfully.
The result of the study, explained Gioioso, is the “foundation of a new generation of [haptic devices] that only act on a cutaneous level, to render, onto the human body, tactile sensations.”
The Impact of Haptics on Learning
Generally, effective XR experiences rely on immersion and engagement, which is facilitated by the digitization of the sense of touch. Specifically, within education and training, haptics has shown to facilitate the learning experience. Gioioso provided two main findings from a 2018 study. First, haptics increases memorability of the experience, specifically, 69% more memorability during video play when compared to non-haptic conditions. Secondly, haptics increases engagement (appeal and interest) by 19%. Moreover, concerning recognition and perception, content enhanced with haptics resulted in an 11% increase in brain recognition; an 18% increase in perception of content quality; an 8% higher motivation and engagement responses; and a 40% increase in recalling visuals.
From Prototype to Product
In 2013, Gioioso and the research team at the University of Siena prototyped a simple device that only applied force (i.e., compression) on the skin, specifically the fingertip. Then in 2015, the prototype became more complex with the integration of a vibration effect, giving the perception of the texture of the object, surface, or material. A thermal property was integrated in 2017, another important component in the perception of objects, surfaces, and materials. By 2019, with the idea of bringing the research results to the market, WEART was founded, and the transformation of the prototypes into a product began. The product, called TouchDIVER, incorporates three pillars of touch: forces, temperature, and texture.
TouchDIVER allows people to feel the weight, roughness, stiffness, and thermal properties of virtual objects, surfaces, and materials. In the image below are the features, including software and hardware compatibility. It also comes with a software development kit (SDK) and a tactile library with forty default textures that are all customizable and to which other textures can be added.
Features of TouchDIVER
Use cases of TouchDIVER range from XR training and marketing experiences to collaborative design and entertainment. For industrial and medical training, a high level of realism is particularly important in making the experience compelling and effective. This is the case in surgical training, for example, because a surgeon “relies on the sense of touch in the surgical room.” For marketing experiences, crucial aspects are engagement and emotional responses. Through digitizing the sense of touch, novel purchasing experiences are made possible, whether in a store or at home. For designers, TouchDIVER enables remote interaction and collaboration. While working on the same object at different locations, designers can manipulate and feel the object in a virtual space to make design decisions based on material properties.
To sustain growth plans, WEART is working with companies ranging from a medical manufacturer of XR tools for surgical training and a furniture retailer to an Italian aerospace conglomerate and XR content developers within different fields. WEART also exhibited at Laval Virtual Europe. Watch the video below to hear what the AR/VR community had to say about their experience of TouchDIVER.
