Visuo-haptic mixed reality simulation using unbound handheld tools

Creative Commons License

Aygün M. M., Ögüt Y. Ç., Baysal H., Taşcıoğlu Y.

Applied Sciences (Switzerland), vol.10, no.15, 2020 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 10 Issue: 15
  • Publication Date: 2020
  • Doi Number: 10.3390/app10155344
  • Journal Name: Applied Sciences (Switzerland)
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Agricultural & Environmental Science Database, Applied Science & Technology Source, Communication Abstracts, INSPEC, Metadex, Directory of Open Access Journals, Civil Engineering Abstracts
  • Keywords: visuo-haptic, mixed reality, encountered-type haptic display, haptic interfaces, AUGMENTED REALITY, VIRTUAL-REALITY, HIGH-PRECISION, PERCEPTION, INTERFACE, SURFACE
  • TED University Affiliated: Yes


© 2020 by the authors.Visuo-haptic mixed reality (VHMR) adds virtual objects to a real scene and enables users to see and also touch them via a see-through display and a haptic device. Most studies with kinesthetic feedback use general-purpose haptic devices, which require the user to continuously hold an attached stylus. This approach constrains users to the mechanical limits of the device even when it is not needed. In this paper, we propose a novel VHMR concept with an encountered-type haptic display (ETHD), which consists of a precision hexapod positioner and a six-axis force/torque transducer. The main contribution is that the users work with unbound real-life tools with tracking markers. ETHD's end-effector remains inside the virtual object and follows the tooltip to engage only during an interaction. We have developed a simulation setup and experimentally evaluated the relative accuracy and synchronization of the three major processes, namely tool tracking, haptic rendering, and visual rendering. The experiments successfully build-up to a simple simulation scenario where a tennis ball with a fixed center is deformed by the user.