(Photo Source: Unilad.CO.UK)
During the past months, I explored several storytelling projects produced in virtual reality (VR). I started taking notes about my VR experiences because they were so different from what I felt watching a movie or playing a video game. My first notes were about the design of the VR headset and my circular walks tethered by a cable. I was intrigued to see my VR body, two virtual hands floating inside a virtual scene, but I was not sure where my legs and feet were. I wrote about my complex interactions with the virtual space of the story. There was no comparison with the passive experience of watching images on a screen. Virtual reality added new possibilities to the visual language of storytelling that we know from movies, video games or animations.
Some scenes from the movies of Michael Powell and Stanley Kubrick were often present in my mind during the exploration of stories in VR. Michael Powell created two films that pioneered storytelling in a virtual reality scene, Stairway to Heaven, (1946) and The Red Shoes, (1948). The ballerina scene of The Red Shoes, is shot from the point of view of a dancer spinning on stage. The scene is contained inside a 360 cylindrical black volume structured by streaks of white fluorescent lights. At this point of the story, the ballerina is trying to remain in control of her motion in spite of the hypnotic effect of the volume of lights. The spatial experiments created for The Red Shoes are developed on a different scale by Stanley Kubrick in 2001, Space Odyssey. A flight sequence through spatial structures made of lights illustrates the compression of time and the continuity of the story. The following sequence is a science fiction astronaut walking through a continuum of virtual reality scenes -- a direct homage to Michael Powell’s Stairway to Heaven. Kubrick said “It is not a message I ever intended to convey in words. 2001 is a non-verbal experience… I tried to create a visual experience, one that bypasses verbalised pigeonholing and directly penetrates the subconscious with an emotional and philosophic content.”
In Space Odyssey (2001), the flight sequence through spatial structures made of light is at 02:02:45. The science fiction astronaut walking through a continuum of virtual reality scenes is at 02:13.
A VR experience is very different from what may be experienced with other media. Coco VR, Pixar Animation Studios’ first VR production, was released just after the original Coco, a 3D computer-animated fantasy film. Coco VR invites the viewer to play with different levels of narration: the non-verbal, the social and a visual poetry combining powerful images from memories and from the imaginary. There is not the necessity to see the movie in order to have a fulfilling experience with Coco VR; though elements of the VR world were clearly utilizing assets from the film in an immersive storytelling environment – bringing Coco to a new level. You can get the free Coco VR experience HERE
In the case of storytelling in VR, the virtual space becomes part of the story. The virtual reality environment provides a mixed reality perception involving seeing, feeling and memory. Social virtual reality offers the possibility to create social places where people meet and spend time together in virtual reality. Secrets of the First Pyramids by Ubisoft reuses assets developed for the game Assassins Creed, to create a VR experience. The scenes are reorganized around lessons about the history of Ancient Egypt. The viewer can be an avatar and take a virtual tour of the library of Alexandria, which was designed to house all human knowledge of the time. The visit may turn into exploring many levels of detail of virtual objects with a great level of freedom. I was surprised to end my visit with a virtual lesson of geometry by the ancient mathematician Euclid.
Glen Keane, a legendary Disney animator and 2018 Academy Award Winner, says that his goal in virtual reality is “a full immersion into the drawing of a character”, “not to draw a drawing but that I could step and be in that world”. Although a visitor to a VR world may find themselves in a virtual body that is imperfect, it will likely be relative to the scale of a virtual reality scene. You can find Glen Keane drawing in VR HERE. The video was produced for the upcoming Future of Storytelling Summit.
The current VR headsets have extensions that can also track hands, feet, clothing (wearable sensors), eye gaze and even brain waves. These sensors act as the first elements to map a viewer’s virtual body in VR. For example, in Coco VR, you can dress up your virtual body - a skeleton - by selecting clothing in front of a virtual mirror. You can also watch a miniature version of yourself dancing as a small puppet. It has become evident that the virtual self replaces the real self in the virtual scene.
In a movie theatre, you can look away from the screen or watch emotions on the faces of people in the audience. In VR, you are actively part of the story – generally, there is not the possibility to be an outsider in the story that you experience. I have found that virtual spaces can be experienced without the limitations of seeing the world inside of a frame – since it is often a space without boundaries. The visual experience of the visitor in virtual reality is a continuum of spaces that can be seen as 360 degree panoramic images combining multiple cameras in front, back, up, down and sides of a virtual camera. If done well, all of this happens seamlessly. The connection between a viewer and the 360 panoramic views of a VR scene seems to be immediate. Virtual worlds presented inside a 360 VR applications can mimic the way images are projected on the spherical surface of the back of our eyes. In my own production work, I notice that creating a spatial continuity in VR creates the feeling of personal attachment to a space. Some examples of 360 VR applications from 2017:
Elbphilarmonie Hamburg in 360 a cultural landmark where all musics meet (Hamburg, Germany) the VR world in a spaceship designed as modern architecture.
In Disney’s Pete’s Dragon 360 in 4K video, flying on the back of a dragon may not be a full story, but offers an experience which could not be had in reality. Through Mowglie’s Eyes is a fantastic example of experiential storytelling. It was created by Cold Lairs and Weta Digital, Peter Jackson’s special effects studio in Wellington, New Zealand.
Coded worlds can bring visitors into another type of experience. Another World by Glenn Marshall’s studio, Wheelworks, presents an exquisite procedural world in VR. The landscapes in the film were procedurally generated from algorithms, using OpenGL and GLSL shaders. The uniqueness of the world generated with code offers the ambiance of a story within on its own. The visitor is creating the world they explore – and the generated landscape is mesmerizing as they unfold their own story.
Glen Marshall awesome virtual worlds can be seen HERE.
Following a virtual camera zooming through space with very small objects has no limitation in virtual reality. A viewer can visit sequences of very small objects nested inside each other without breaking the string of the narration. In Small Wonders by Priam Givord, we follow a virtual camera zooming through gothic miniatures of wood boxes from the 16th century. The camera navigates inside tiny and delicate woodcarvings. This experience would be impossible to share with others outside of virtual reality.
Small Wonders by Priam Givord can be seen HERE.
As technology advances and becomes less and less visible, we can expect to discover new dimensions of storytelling in VR. I think that we will see the development of VR environments that will enable groups of writers, VR designers and audiences to collaborate together and branch out from the VR experiences that we are only getting to know today.
Jean-Marc Gauthier is an Associate Professor at University of Idaho and Managing Director of Tinker Too, an interactive design studio based in Singapore, currently focused on designing prototypes for automotive and mobility products.
Recent Comments