resume panda@payodpanda.com

Morphaces

In the physical world, we can touch and feel objects. The tangibility of the object tells the artist that they are touching the surface and are painting on it. How can we bring this sensation to VR? This project explores morphable surfaces for tangible sketching in VR.


Panda, P., Ho, C., & Ham, D. (2021). Morphaces: Exploring Morphable Surfaces for Tangible Sketching in VR. C&C ’21: Creativity and Cognition.
🌐https://doi.org/10.1145/3450741.3465387
📄download paper

Introduction

This paper documents our inquiry into the design and utility of morphable surfaces to provide tangible feedback while sketching in VR. We explored materials and various structures that could enable a surface to morph. We designed and implemented the Morphaces ecosystem that includes 3D printed accessories that enable handheld and desk-mounted pen-and-surface interaction for Oculus Quest. We present this preliminary exploration to show the utility of morphable surfaces and the benefits of low cost design solutions for high-fidelity media like VR.

The following design rationale guided our investigation:

  • One-to-many correlation between physical to virtual object. The same physical object should be able to represent a multitude of virtual objects.
  • Low-cost, accessible designs. Should not use exotic materials.
  • Should work with existing VR devices. Do not create a new VR device or controllers--use passive haptics and proxies.

Material explorations for surface

We started our material and structural explorations with paper and a thin 24 AWG wire. However, this proved too flimsy and we quickly switched to foam. We explored various structures like a cross-grid (X), standard grid (+), a "tent" structure, and disjointed grid. We tried wire gauges from 24-12 AWG.

We then switched our material from foam to 3D printed PETG filament. This decision was driven by the relative softness of the foam material. The springiness of the foam made it difficult to write on. Our first 3D printed structure used a 5x5 grid of dome-like structures, that had holes to enable the creation of a wire mesh. We also explored other structures, including a jigsaw-like fitting double layer of PETG, as well as several mounting explorations.

Morphaces Ecosystem

Using what we learnt from our material and structure explorations, we designed an ecosystem of tangible accessories that work with the Oculus touch controllers to provide morphable surfaces for an artist to sketch and texture on. Our system consists of four parts:

  1. The Morphace morphable surface, including a mounting mechanism for the surface to attach to the controller.
  2. A table-mounted arm, with a mounting mechanism to hold the controller with the Morphace.
  3. A 3D printed pen accessory for the Oculus touch controller.
  4. A software interface (NapkinSketchVR) that enables the user to use the morphable surface with the Oculus Quest or Rift VR devices.

Usage

Morphing the surface

The morphable nature of the Morphace lends itself to being manipulated and taking on the shape of the surface of a variety of different objects. The same physical prop can serve as a proxy for many virtual objects. This enables the artist to switch between drawing / texturing different 3D objects, or different parts of the same 3D object. For instance, part of a 3D model might be flat (like the tail wing of a space ship), and another part might be curved (like the main body of the spaceship).

The morphable nature of the Morphace lends itself to being manipulated.

Desk-mounted operation

The Morphace ecosystem allows the surface to be mounted to a desk-mounted mechanically actuated arm. For our prototype, we used a commercially available desktop monitor mount for this purpose, and designed and 3D printed a mounting mechanism for a Quest touch controller to mount to the VESA interface offered on the monitor mount.

Desk-mounted operation allows the surface to remain relatively stable and grounded in the world frame of reference. The user is free to move around the arm, and the surface stays put. This is particularly useful when sketching a complex shape.

The Morphace ecosystem allows the surface to be mounted to a desk-mounted mechanically actuated arm.

Handheld operation

In addition to the desk-mounted operation, the Quest controller can also be detached from the monitor mount in order to enable handheld operation. This usage is similar to writing or sketching on a notepad.

Handheld usage allows the user more flexibility in the positioning of the surface. However, since the surface is held in the user’s hand, this tends to be less stable. As a countermeasure to this, the user may hold the virtual object that he or she is drawing on, so that the object moves with the user’s hand.

In our tests, we were able to paint fairly intricate designs on the virtual model of the spaceship. This can be compared to the photo of a user painting a 3D printed model of the same size.
The user may hold the virtual object that he or she is drawing on, so that the object moves with the user’s hand.