This post has been republished via RSS; it originally appeared at: Microsoft Research.
Editor’s Note: Bimanual controllers are frequently used to enhance the realism and immersion of virtual reality experiences such as games and simulations. Researchers have typically relied on mechanical linkages between the controllers to recreate the sensation of holding different objects with both hands. However, those linkages cannot quickly adapt to simulate dynamic objects. They also make for bulky controllers that can’t be disconnected to support free, independent movements. This is the problem that researchers seek to solve in the recent paper titled “GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip Deformation”.
GamesBond is the outcome of a recent collaboration between Michel Pahud and Mike Sinclair from Microsoft Research and Andrea Bianchi, associate professor at KAIST and director of the MAKinteract lab, with two of his students, Neung Ryu, the original author of the paper, and Hye-Young Jo. The paper was accepted at ACM CHI Conference on Human Factors in Computing Systems (CHI 2021), where it received an honorable mention award.
In this project, we explored a pair of novel 4-DoF controllers, without actual physical linkage between them, that can bend, twist, and stretch together in concert to create the illusion of being connected as a single device with a physical link. Each controller can bend from 0 to 30 degrees in any direction, twist from -70 degrees to 70 degrees and stretch from -2.5mm to 9.0mm (the paper provides all the details of the mechanism).
Figure 1: The controller can move in multiple directions, including bending, twisting and stretching 
Figure 2: The bonds can be (a) rigid to create an experience such as holding a solid bar with two hands, (b) soft to create deformable and pliable objects, or (c) dynamic and capable of rendering inertia and oscillations to perceive objects like elastic ropes. 
Figure 3: The controllers open doors for many user experiences, such as jumping rope held by (a) one or (b) two people 
Figure 4: The controllers can create a force illusion that feels like breaking a bar 
Figure 5: The controllers can create the illusion of moving weight, such as when holding a fishing net 
Figure 6: Applications can render different grip shapes, such as: (a) a pair of separate electric screwdrivers with independent handles, (b) a hammer drill with parallel handles, or (c) a jackhammer with aligned handles… 
Figure 7: A fishing rod application can give the illusion of a) pulling a line and b) winding a winch.
These figures are just a few examples explored, but the sky is the limit.
We conducted two complementary technical evaluations to characterize the rendering capabilities of the GamesBond controllers – a pilot study with four participants and a user study with 12 participants. Both quantitative and qualitative metrics support the hypothesis that GamesBond increases the realism, immersion and enjoyment of a VR experience. It also shows that the device can best render soft and dynamic bodies (e.g., a jumping rope), indicating possible avenues of exploration for future applications.
How did this collaboration with KAIST begin
This collaboration began by chance when Pahud and Bianchi met at the ISS2019 conference in Daejeon, Korea, and had side discussions about collaboration on novel user experiences. Bianchi showed Pahud his lab and its recent projects. “I was really impressed about the environment of the lab, the amount of innovative human computer interaction (HCI) hardware projects going on there, and the quality of the students. I thought we definitely have to find a way to collaborate together,” said Pahud.
Microsoft research webinars
Lectures from Microsoft researchers with live Q&A and on-demand viewing.
After staying in touch even despite the COVID-19 pandemic, Bianchi’s students at the MAKinteract lab showed Pahud an early prototype of a controller that can bend, twist and stretch. It was clear that we would need two of these controllers to create a bimanual interaction and the feel of a virtual link between them. We quickly decided to invite Mike Sinclair to join and share his expertise in novel hardware and advise the students how to improve the controllers.
Both Pahud and Sinclair, along with former colleague Hrvoje Benko, had previously worked together on innovative hardware and experiences such as the TouchMover haptic feedback display, a 3D touchscreen which won a best paper award at ISS2014.
“This collaboration with KAIST has been amazing,” Pahud said, “Andrea and his students are so talented and able to very quickly build compelling and novel hardware. We met regularly and every time I suggested new experiences or modifications, they implemented it by the next meeting! It’s impressive to see how quickly they make progress, which allowed us to submit to the CHI conference in time. We keep meeting regularly now and I am truly looking forward to future research projects with Andrea’s team.” Pahud was also pleased to join the PhD committee for one of Bianchi’s PhD students.
To learn more about GamesBond, please read the paper. You can also attend the CHI session: GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip Deformation on Thursday, May 13 at 10:00 PST.
The post Microsoft Research collaborates with KAIST in Korea to explore bimanual interactions with haptic feedback in virtual reality appeared first on Microsoft Research.