Teaser
The haptic processing pipeline consists of 5 main steps: sensors to acquire the data, processing and modeling to store acquired data, communication to transmit information associated with the collected signals, haptic devices to render the haptic feedback to the user, and perception that studies the factor that influence the haptic experience of the user.
Our survey focuses on the processing and modeling step.
Abstract
Haptic feedback has been integrated into Virtual and Augmented Reality, complementing acoustic and visual information and contributing to an all-round immersive experience in multiple fields, spanning from the medical domain to entertainment and gaming. Haptic technologies involve complex cross-disciplinary research that encompasses sensing, data representation, interactive rendering, perception, and quality of experience. The standard processing pipeline, consists of (I) sensing physical features in the real world using a transducer, (II) modeling and storing the collected information in some digital format, (III) communicating the information, and finally, (IV) rendering the haptic information through appropriate devices, thus producing a user experience (V) perceptually close to the original physical world. Among these areas, sensing, rendering and perception have been deeply investigated and are the subject of different comprehensive surveys available in the literature. Differently, research dealing with haptic surface modeling and data representation still lacks a comprehensive dissection. In this work, we aim at providing an overview on modeling and representation of haptic surfaces from a signal processing perspective, covering the aspects that lie in between haptic information acquisition on one side and rendering and perception on the other side. We analyze, categorize, and compare research papers that address the haptic surface modeling and data representation, pointing out existing gaps and possible research directions.
Real world objects representation in the physical and the perceptual spaces
The classic signal processing pipeline (top row) aims at mapping a physical phenomenon to a physical feature space such that a similar experience of the phenomenon can be delivered to the user in the perception feature space (bottom row). While the pipeline is well-established in both the visual and acoustic domains, it still lacks a common standard in the haptic domain. In the example in bottom row illustrating the section of a tree, the red line is the roughness, the green line is the bumpiness, the blue line represents the stiffness, and the dark gray line is the friction.
A preview on Haptic Surface Modeling
Haptic surface modeling aims at mapping a physical object surface into the corresponding physical feature space of choice (i.e., roughness, bumpiness, stiffness and friction). Along the modeling, physical features may require some processing, filtering, and adaptation to be transmitted to an haptic device and eventually rendered in the perception space.
A preview on datasets and simulators
Various datasets and simulators have been developed to capture and model the complex sensory experiences associated with touch. These resources provide comprehensive data that combine visual, tactile, auditory, and sometimes linguistic information, facilitating the development and evaluation of algorithms for tasks such as texture classification, material recognition, and robotic manipulation.
A preview on research tasks
We provide a thorough classification of the available works for each task. While some of them, like classification and compression, are akin to other domains and straightforward to understand, other tasks are very specific to haptics, like contact localization and grasp prediction.
BibTeX
@article{stefani2024signal,
title={Signal Processing for Haptic Surface Modeling: a Review},
author={Stefani, Antonio Luigi and Bisagno, Niccol{\`o} and Rosani, Andrea and Conci, Nicola and De Natale, Francesco},
journal={arXiv preprint arXiv:2409.20142},
year={2024}
}
Funding
We acknowledge the support of the MUR PNRR project iNEST-Interconnected Nord-Est Innovation Ecosystem (ECS00000043) funded by the European Union under NextGenerationEU. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or The European Research Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.