Shoki Play

by Felix Scholder, MA Information Experience Design student (slides and art work) with commentary by Dylan Yamada-Rice, Senior Tutor IED, Royal College of Art

Shōki Play is a mixed reality installation that combines VR technology with the concept of ancient shadow play to enrich museum experiences. Step behind the paper doors onto the tatami mats and become the vanisher of ghosts.

3D printed replica of a netsuke

The installation pushes the boundaries of an individual VR experience to include others through physical props and set design. In a full-body narrative experience a Japanese Netsuke can be explored in detail, while its fantastical story is told through an immersive game. Therefore the active player becomes the shadow play performer for the audience outside.

Details of how the work came into being are described next:

The work built on Felix’s MA dissertation: ‘Transcended Embodied Experience Perception in Virtual Reality.’ After which, he began by exploring the history of immersive technologies. Gunther Kress (2010) writes that in order to understand the future of communication practices we have to look both historically at what has gone before, as well as to the consider the future of technological and social changes, because these are inseparable from communication.

Research: Immersive Technologies

Felix saw that the current wave 3 of XR technology is providing new ways of storytelling that were not picked up in the first two waves.

VR Experiences

Analysis of other VR experiences is an important step in designing a new one.



Analysis of ‘Ocean of Air’

The conclusions of such analysis indicated that there is a huge potential for current VR experiences in museums to be positioned as follows:

VR in Exhibitions

Felix questioned what the point was of traveling to a museum or gallery space merely to put on a headset. He questioned how the space of the museum could be better brought into the VR experience.

Following this Felix began to explore how a location-based VR experience could make use of the virtual space inside the headset, as well as, the physical space in which it is set up. Further, that the affordances of the two different spaces could tell different but connected parts of the story. In doing so, he extended on the interest people initially have in watching others use VR and combined this with an ancient art of telling stories through shadow play.

Research: Light and Surfaces

In order to turn the VR user into a performer in the shadow play, for an audience outside of the VR experience, Felix experimented with how lighting and materials cast different types of shadows:

Shadow experiments

Tactility in VR

Felix was interested in how the ability to see your hands, with some VR systems, directly affected the performative nature of the user. Also how tracking a physical object in VR could provide a direct link between virtual and physical worlds.

Thus in his Work in Progress show at the Royal College of Art, Felix set about exploring how people responded to seeing their own hands in VR. He also placed a mirror inside the VR experience to see if this further altered user’s interactions with the virtual environment.

Additionally, he asked visitors to the WIP show to place a sticker next to the artefact from the British Museum collection that they would most like to learn more about with the intention of basing his work on the most popular:

This data was used to inform his selection of a Japanese netsuke, which was the artefact people were most interested in from the list provided.

Shoki capturing a demon with a hat

Netsuke are small carved ornaments that were traditionally attached to Japanese kimono. The netsuke below is the one Felix chose to work with in his final piece and is a kind of talisman used to ward off bad spirits by capturing them.

Fieldwork: British Museum

Following this Felix undertook observations at the British Museum to understand the kinds of objects that drew people’s attention or not. He found that some artefacts such as the Egyptian mummies are so well known and attract so much attention that perhaps it would be more interesting to focus on smaller objects, like netsuke, that are harder to see at a distance and cannot be held.


Observation of British Museum visitors

Next came exploration of a range of references to the Japanese aesthetics of room design, shadows and images of Japanese spirits:

User testing was undertaken as part of the UK-Japan Location-Based VR networking week to see how a variety of the members responded to the VR experiences and physical set up:

Installation details
Exploration of cutouts to be used in connection to the VR user’s shadow

Designing the Spectator Experience

The following diagrams are based on the one in ‘Designing the Spectator Experience’ by Reeves et al and show how Felix positioned the audience and user in his installation:

Kenya Hara’s diagram (on the right) was used to to explore how the experience could use multiple senses and this connect to the mind in multiple ways:

The final outcome

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s