Journal Archive

VR(Virtual Reality) is an interactive computer generated experience taking place within a simulated environment. It incorporates mainly auditory and visual feedback, but may also allow other types of sensory feedback like haptic. This immersive environment can be similar to the real world or it can be fantastical. Augmented reality systems may also be considered a form of VR that layers virtual information over a live camera feed into a headset or through a smartphone or tablet device giving the user the ability to view three-dimensional images.

Current VR technology most commonly uses Virtual Reality headsets ormulti-projected environments, sometimes in combination with physical environments or props, to generate realistic images, sounds and other sensations that simulate a user's physical presence in a virtual or imaginary environment. A person using virtual reality equipment is able to "look around" the artificial world, move around in it, and interact with virtual features or items. The effect is commonly created by VR headsets consisting of a head-mounted display with a small screen in front of the eyes, but can also be created through specially designed rooms with multiple large screens.

VR systems that include transmission of vibrations and other sensations to the user through a game controller or other devices are known as haptic systems. This tactile information is generally known as force feedback in medical, video gaming, and military training applications.

In1994 G.Burdea and P.Coiffet. Published an article about Virtual Reality Technology (Virtual Reality Technology). This paper summarizes the characteristics of virtual reality technology. Virtual reality technology has three characteristics: Immersion, Interaction and Imagination.

Immersion refers to the use of computer generated three-dimensional images to put people in a virtual environment, as in the real objective world, can give people an immersive feeling; Interactivity. In this virtual environment generated by computer, people can use some sensing equipment to interact with each other. It feels like they are in the real and objective world. Imagination, virtual environment can enable users to immerse themselves in it and acquire new knowledge, improve their perceptual and rational understanding, so as to deepen the concept and generate new associations. In other words, virtual reality technology has inspired people's creative thinking.

Fig. 2. 
《Virtual Reality Technology》- G.Burdea /P.Coiffet

2016 is the most prosperous year for the development of the VR animation industry. This year, represented by Google Spotlight Stories and Baobab Studio, there are many VR animations on the international market. As of 2018, Disney also produced the first VR animation, and VR animation is also growing rapidly in China.

From 2016 to the present, the number of VR animations is increasing, the quality is getting better and better, and the time is getting longer. Prior to 2017, represented by Google Spotlight Stories and Baobab Studio, the main markets for VR animation were in Europe and the United States. By 2018, VR animation has also been vigorously developed in Asia. The VR animations such as Fresh Out made in China have also been well received internationally.

The current VR animation time is relatively short, most of the time is about 5 to 10 minutes. It can be seen that the development of VR animation is not mature. Although more and more animation enthusiasts have begun to pay attention to VR animation, VR technology has not been fully matured into the field of animation creation. At present, the use of VR technology can only describe a short story, which is not even a complete animation, showing that the new technical form and animation art are not well integrated.

Image is the most basic element of audio-visual language and one of the most important senses of human beings. There are several important elements in the audio-visual language of images: shape is the first layer of information received visually, followed by color. On the basis of shape and color, objects move to form the image we see. How to guide the audience's visual interest points is an important problem to be solved in VR animation creation. Because excessive guidance will affect the audience's freedom of watching the film, but only pay attention to the freedom of watching, the audience will get lost in the panoramic perspective and miss the key points of the story.

Below is a list of papers on visual guidance [Table. 1]. Through these 15 researchs, I analyzed how the producers guide the visual interest of the audience in the existing VR animation from the perspectives of lighting, character, background and sound.

In current VR animation, the most common way to guide the audience's perspective is through the changes of object movement, light and sound. In The VR animation, "The dream collector," there is a snippet: a snowflake appears in a dark environment, and as The snowflake moves, it can travel to another place, where The surrounding area becomes bright. The audience will see an old man cleaning up the dirty football beside the table. In the animation Henry, when the audience is watching a certain place, a bug will be arranged nearby to attract the audience's attention[Figure. 3].Which on the one hand encourages the audience to participate and explore, and on the other hand reminds the audience to change their perspective to have new experience and interaction.

Fig. 3. 
Object movement in VR animation <Henry>

In addition to moving objects, lighting is also an important tool for guiding the viewer's gaze. In this regard, VR animation is somewhat similar to stage play. In stage plays, spotlights are often used to illuminate the positions that need to be seen by the audience. When watching the stage, the audience will instinctively look at the positions of lights. Where there is light focus, there is usually more information, so light is a very good guiding tool in VR animation. In <The dream collector>, when The lights are on in The corner of The image, The viewer's attention will naturally return to The subject[Figure. 4].

Fig. 4. 
The light changes in VR animation <The dream collector>.

Sound is another important element in VR animation. Sound can not only enhance the emotional appeal, but also guide the audience's vision. Hearing is one of the most important senses when we watch movies.

In VR animation, sound can be superimposed on each other in the same scene to ensure the integrity of the story. Or through some mimicry, to change the perspective of the audience. VR animation 《Henry》 has made many attempts in this regard.Besides sound and light, background composition in VR animation is also very important. We are watching a film, Angle of view would be about 270 °and range, set the main space and secondary space, can let the audience more attention to the main storyline. Secondly, the gaze of the characters can also lead the audience into the story.For example, in the animation <Henry>, the producer directly leads the audience into another scene through the interaction of the protagonist Henry and the audience's gaze.

Fig. 5. 
Character gaze in VR animation <Henry>.

In traditional animation, the content of the story we are looking at is usually linear. The authors present the content that needs to be presented to the audience directly to the audience. Montages are used to combine animation content from different locations, distances and angles, and lens arrangements taken in different ways. But in VR animation, the focus of narrative has changed from story content to interactive experience. So under this new carrier, the narrative approach is more focused on finding a balance between content and experience.

Here are some of the research on Storytelling in VR animation[Table. 2]. From the four perspectives of character, camera, script and scene conversion, this paper analyzes how VR animation maintains the integrity of narrative from a free perspective.

Traditional animation uses a lens to tell a story. Link the story content in a straight line. In VR animation, since the audience is free to experience and not directly follow the lens to see the content, the narrative rhythm must consider the time it takes for the audience to view the content. This requires increasing the length of the performance rhythm and giving the audience enough length. Exploration time. That is, the speed and rhythm of traditional linear storytelling need to match the time of the audience's exploration. The narrative rhythm of the VR animation <The dream collector> is significantly slower than the average animation. This is because if the audience has not completed the exploration and experience, the next story begins, not only unnatural, but also makes the audience miss the details of the story.

Scene scheduling and editing are the basic methods of traditional image language. In the VR animation process, the traditional audio-visual language is limited: the picture has no lens, no scene, the audience has its own specific perspective, and can rotate the angle of view. The changes in all objects and scenes are based on the perspective changes of the audience. The rights to camera scheduling were delivered to each viewer. The motion of the picture will become the motion in the stereo environment, and the scene connection created by the clip can be imagined as a mosaic of a three-dimensional space. Therefore, in most VR animations, there is no need to have a lens switch or a scene overlay, and there is no longer a broken visual presentation. The traditional editing function has been eliminated, and the picture is rendered as a natural long shot. For example, the VR animation <The dream collector> uses a lot of transition methods: fade in、fade out、 etc. [Figure. 6]. Therefore, in order to ensure the smoothness of the narrative, VR animation uses the World-To-World editing method more than the traditional animation Frame-To-Frame editing method.

Fig. 6. 
Fade in in VR animation <The dream collector>

In order to balance the story content and technical experience, the camera position in VR animation is also very important. In the VR animation <Pearl>, the creator tells the story by fixing the position of the camera [Figure. 7], and the whole story uses long shots, which reduces the jumping of the viewing angle and makes the story content smoother.

Fig. 7. 
Camera position in VR animation <Pearl>

Most of the current VR animations still maintain a straight story script. In fact, this conflicts with the multi-linear narrative of VR animation. The story content of future VR animations should be composed of multi-linear scripts. This allows the viewer to enter the story from different angles, which not only satisfies the technical experience of the audience, but also ensures the integrity of the content of the animated story.

For example, the VR animation <Pearl> is a multi-linear script structure. There are several plot trigger points in the middle of the animation. And the animation innovatively uses a large number of skipping lenses to convert space. Faced with such a narrative approach, viewers may see different story endings, ranging from 5 minutes to 7 minutes. In the process of watching, the audience may have read the stories brought by multiple clues, and may also feel the general in the interaction. This is because of the different experience results caused by different ways of interacting with the audience. This makes the digital animation in the virtual reality context different from the traditional animation: the route the viewer walks, the viewpoints encountered, the length of the viewing, or the resources collected and used, etc., vary with the interaction. Therefore, the audience may preset a variety of story endings according to the experience results, and even directly influence the trend of the story content in the performance and selection of the story. It can be seen that in VR animation, the multi-linear script allows the viewer's eyes to interact directly with the story content.