Korea Digital Contents Society
[ Article ]
Journal of Digital Contents Society - Vol. 20, No. 3, pp.459-467
ISSN: 1598-2009 (Print) 2287-738X (Online)
Print publication date 31 Mar 2019
Received 03 Jan 2019 Revised 28 Feb 2019 Accepted 20 Mar 2019
DOI: https://doi.org/10.9728/dcs.2019.20.3.459

The change of animation narrative structure caused by virtual reality technology

Yi-meng Li ; Jong-woo Joo*
Department of New media art Master’s course, Chung-Ang University, 84 Heukeok-ro, Seoul, Korea
가상현실 기술에 의한 애니메이션 서사구조 변화
이이맹 ; 주종우*
중앙대학교 뉴미디어아트 대학원

Correspondence to: *Jongwoo Joo Tel: +82-31-670-3112 E-mail: jw4150@cau.ac,kr

Copyright ⓒ 2019 The Digital Contents Society
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-CommercialLicense(http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

VR(Virtual Reality) technology is a computer simulation technology that can create and experience a Virtual world, using a computer to generate interactive 3D images in real time.These physical behavior simulations can help you immerse yourself in a virtual environment.At present, VR technology is used in many fields, such as games, healthcare, education and entertainment.And now, the animation industry has developed into a paperless 3d image, which is closely related to technological innovation. But all the animations have a common disadvantage, that is "the single transmission of information."In this research, four VR animations of Google company were selected and analyzed from multiple directions based on visual guidance and animation stories.In addition, by analyzing the existing visual guidance methods of VR animation, the method of developing multi-linear story content is found, and on this basis, the technology and art in VR animation are balanced.Through this research, we can find the development direction of VR animation in the future.

초록

가상현실(VR; Virtual Reality) 기술은 가상 세계를 만들고 경험할 수 있는 컴퓨터 시뮬레이션 기술이며, 컴퓨터를 사용하여 대화식 실시간 3D 영상을 생성한다. 이와 같은 물리적 행동 시뮬레이션 시스템은 사용자가 가상의 환경에 몰입할 수 있게 도와준다. 오늘날 VR 기술은 게임, 건강관리, 교육, 엔터테인먼트 등 다양한 분야에서 사용되고 있다. 현재와 같은 정보화 시대에서 애니메이션은 종이가 존재하지 않는 디지털 및 3 차원 영상을 제작하고 있으며, 이는 기술 혁신과 불가분의 관계에 있다. 전통적인 애니메이션 산업에서 VR 기술은 변화를 일으킬 것이며, VR 애니메이션은 완벽한 몰입감, 신뢰성 및 상호 작용으로 인해 애니메이션의 예술개발이 영향을 받고 있다. 그러나 모든 애니메이션은 정보전송의 단일성이라는 공통된 과제가 있다. 본 연구에서는 구글사의 대표 VR애니메이션 중 4개의 작품을 선별하였으며, 시각적 방법과 애니메이션의 스토리텔링을 기초로 다양한 방향으로 분석하였다. 또한, 기존 VR애니메이션이 시청자를 시각적으로 안내하는 방법을 분석하여 스토리의 다중선형성을 향상시키면서 애니메이션의 내용도 균형 있게 조절할 수 있었다. 본 연구를 통해 VR애니메이션의 나아갈 방향을 예측해본다.

Keywords:

VR, Animation, Visual guidance, Sound, Storytelling, Characterntroduction

키워드:

가상현실, 애니메이션, 시각적구조화, 소리, 스토리텔링, 캐릭터

I. Introduction

Animation is a kind of comprehensive art. It is an artistic expression that combines painting, comics, film, digital media, photography, music, literature and many other art categories. The animation originated in the UK in the first half of the 19th century and flourished in the United States. It is the only art that has a certain date of birth - on October 28, 1892. Emile Reynolds first presented an optical film to the audience at the famous Greifan Wax Museum in Paris, marking the official birth of the animation, while Emile Renault is also known as the "father of animation." There are many expressions in English for animation, such as animation, cartoon, animated cartoon, and cameracature. The more formal word "Animation" is derived from the Latin anima, meaning "soul", and the verb animate is the meaning of "giving life", which can be extended to make something live. So animation can be defined as the use of painting techniques to create the art of life movement.

The original form of animation art was to use the "visual persistence principle"of the human eye, and the three-dimensional animation began to rise in the 1990s.

With the development of technology, VR (Virtual Reality) technology has gradually become an important form of technology. VR animation combines traditional animation art with advanced virtual reality technology. The animation expression method relies on virtual reality technology and has the characteristics of Immersion, Interactivity and Imagination. VR animation is different from the linear narrative of traditional animation, which transforms the previous "frame vision" into a "panoramic view." From 360° perspective, viewers can change from recipients of information to experiencers, not only to enter the real animation world, but also to choose the development of animated stories according to their own interests. VR animation fully integrates computer multimedia technology with various elements such as images, sounds, and images, making animation creation more rich and animation expression more intuitive.

Unlike traditional animation, the free perspective of VR animation changes the way animations are told. By 2018, although many VR animations have appeared, these VR animations are relatively short. At present, the development of VR animation faces two biggest problems. One is that the audience's viewing angle needs to be guided; the other is that the linear narrative method needs to be broken. The balance between these two points is also the balance between technical experience and content feeling in VR animation. I will find an effective way to solve these two problems by analyzing existing VR animations and related research, laying the foundation for future animation creation.

Fig. 1.

Research plan


II. Principal Argument

2-1 Theoretical background

1) Concept of VR technology

VR(Virtual Reality) is an interactive computer generated experience taking place within a simulated environment. It incorporates mainly auditory and visual feedback, but may also allow other types of sensory feedback like haptic. This immersive environment can be similar to the real world or it can be fantastical. Augmented reality systems may also be considered a form of VR that layers virtual information over a live camera feed into a headset or through a smartphone or tablet device giving the user the ability to view three-dimensional images.

Current VR technology most commonly uses Virtual Reality headsets ormulti-projected environments, sometimes in combination with physical environments or props, to generate realistic images, sounds and other sensations that simulate a user's physical presence in a virtual or imaginary environment. A person using virtual reality equipment is able to "look around" the artificial world, move around in it, and interact with virtual features or items. The effect is commonly created by VR headsets consisting of a head-mounted display with a small screen in front of the eyes, but can also be created through specially designed rooms with multiple large screens.

VR systems that include transmission of vibrations and other sensations to the user through a game controller or other devices are known as haptic systems. This tactile information is generally known as force feedback in medical, video gaming, and military training applications.

2) Features of VR technology

In1994 G.Burdea and P.Coiffet. Published an article about Virtual Reality Technology (Virtual Reality Technology). This paper summarizes the characteristics of virtual reality technology. Virtual reality technology has three characteristics: Immersion, Interaction and Imagination.

Immersion refers to the use of computer generated three-dimensional images to put people in a virtual environment, as in the real objective world, can give people an immersive feeling; Interactivity. In this virtual environment generated by computer, people can use some sensing equipment to interact with each other. It feels like they are in the real and objective world. Imagination, virtual environment can enable users to immerse themselves in it and acquire new knowledge, improve their perceptual and rational understanding, so as to deepen the concept and generate new associations. In other words, virtual reality technology has inspired people's creative thinking.

Fig. 2.

《Virtual Reality Technology》- G.Burdea /P.Coiffet

3) The status quo of VR animation

2016 is the most prosperous year for the development of the VR animation industry. This year, represented by Google Spotlight Stories and Baobab Studio, there are many VR animations on the international market. As of 2018, Disney also produced the first VR animation, and VR animation is also growing rapidly in China.

From 2016 to the present, the number of VR animations is increasing, the quality is getting better and better, and the time is getting longer. Prior to 2017, represented by Google Spotlight Stories and Baobab Studio, the main markets for VR animation were in Europe and the United States. By 2018, VR animation has also been vigorously developed in Asia. The VR animations such as Fresh Out made in China have also been well received internationally.

The current VR animation time is relatively short, most of the time is about 5 to 10 minutes. It can be seen that the development of VR animation is not mature. Although more and more animation enthusiasts have begun to pay attention to VR animation, VR technology has not been fully matured into the field of animation creation. At present, the use of VR technology can only describe a short story, which is not even a complete animation, showing that the new technical form and animation art are not well integrated.

2-2 Method of research

2016 is called "VR Animation Year". Represented by Google, there are many VR animations on the market. These VR animations are the beginning of VR technology used in the field of animation art creation. Among these VR animations, there are well-known "Lost" and "Henrry" by Oculus Story Studio and "Pearl" and "Rain or Shine" by Google Spotlight Storie. From 2016 to 2018, there are also many animation enthusiasts and technology enthusiasts who have studied VR animation. Based on the two directions of visual guidance and animation storytelling, I have collected 27 related research.

In this research, I will analyze how VR animation visually guides viewers through the existing VR animation content, combining domestic and foreign related papers, from the perspective of animation elements such as sound, lighting, characters, and background. And in the 360° perspective, the narrative mode of VR animation has changed through the new features of each animation element. Through the research of visual guidance and animation storytelling, we can explore the new direction of future VR animation creation.

2-3 Research result

1) Visual guidance in VR animation.

Image is the most basic element of audio-visual language and one of the most important senses of human beings. There are several important elements in the audio-visual language of images: shape is the first layer of information received visually, followed by color. On the basis of shape and color, objects move to form the image we see. How to guide the audience's visual interest points is an important problem to be solved in VR animation creation. Because excessive guidance will affect the audience's freedom of watching the film, but only pay attention to the freedom of watching, the audience will get lost in the panoramic perspective and miss the key points of the story.

Below is a list of papers on visual guidance [Table. 1]. Through these 15 researchs, I analyzed how the producers guide the visual interest of the audience in the existing VR animation from the perspectives of lighting, character, background and sound.

Research on visual guidance.

In current VR animation, the most common way to guide the audience's perspective is through the changes of object movement, light and sound. In The VR animation, "The dream collector," there is a snippet: a snowflake appears in a dark environment, and as The snowflake moves, it can travel to another place, where The surrounding area becomes bright. The audience will see an old man cleaning up the dirty football beside the table. In the animation Henry, when the audience is watching a certain place, a bug will be arranged nearby to attract the audience's attention[Figure. 3].Which on the one hand encourages the audience to participate and explore, and on the other hand reminds the audience to change their perspective to have new experience and interaction.

Fig. 3.

Object movement in VR animation <Henry>

In addition to moving objects, lighting is also an important tool for guiding the viewer's gaze. In this regard, VR animation is somewhat similar to stage play. In stage plays, spotlights are often used to illuminate the positions that need to be seen by the audience. When watching the stage, the audience will instinctively look at the positions of lights. Where there is light focus, there is usually more information, so light is a very good guiding tool in VR animation. In <The dream collector>, when The lights are on in The corner of The image, The viewer's attention will naturally return to The subject[Figure. 4].

Fig. 4.

The light changes in VR animation <The dream collector>.

Sound is another important element in VR animation. Sound can not only enhance the emotional appeal, but also guide the audience's vision. Hearing is one of the most important senses when we watch movies.

In VR animation, sound can be superimposed on each other in the same scene to ensure the integrity of the story. Or through some mimicry, to change the perspective of the audience. VR animation 《Henry》 has made many attempts in this regard.Besides sound and light, background composition in VR animation is also very important. We are watching a film, Angle of view would be about 270 °and range, set the main space and secondary space, can let the audience more attention to the main storyline. Secondly, the gaze of the characters can also lead the audience into the story.For example, in the animation <Henry>, the producer directly leads the audience into another scene through the interaction of the protagonist Henry and the audience's gaze.

Fig. 5.

Character gaze in VR animation <Henry>.

2) Storytelling in VR animation

In traditional animation, the content of the story we are looking at is usually linear. The authors present the content that needs to be presented to the audience directly to the audience. Montages are used to combine animation content from different locations, distances and angles, and lens arrangements taken in different ways. But in VR animation, the focus of narrative has changed from story content to interactive experience. So under this new carrier, the narrative approach is more focused on finding a balance between content and experience.

Here are some of the research on Storytelling in VR animation[Table. 2]. From the four perspectives of character, camera, script and scene conversion, this paper analyzes how VR animation maintains the integrity of narrative from a free perspective.

Research on Storytelling in VR animation

Traditional animation uses a lens to tell a story. Link the story content in a straight line. In VR animation, since the audience is free to experience and not directly follow the lens to see the content, the narrative rhythm must consider the time it takes for the audience to view the content. This requires increasing the length of the performance rhythm and giving the audience enough length. Exploration time. That is, the speed and rhythm of traditional linear storytelling need to match the time of the audience's exploration. The narrative rhythm of the VR animation <The dream collector> is significantly slower than the average animation. This is because if the audience has not completed the exploration and experience, the next story begins, not only unnatural, but also makes the audience miss the details of the story.

Scene scheduling and editing are the basic methods of traditional image language. In the VR animation process, the traditional audio-visual language is limited: the picture has no lens, no scene, the audience has its own specific perspective, and can rotate the angle of view. The changes in all objects and scenes are based on the perspective changes of the audience. The rights to camera scheduling were delivered to each viewer. The motion of the picture will become the motion in the stereo environment, and the scene connection created by the clip can be imagined as a mosaic of a three-dimensional space. Therefore, in most VR animations, there is no need to have a lens switch or a scene overlay, and there is no longer a broken visual presentation. The traditional editing function has been eliminated, and the picture is rendered as a natural long shot. For example, the VR animation <The dream collector> uses a lot of transition methods: fade in、fade out、 etc. [Figure. 6]. Therefore, in order to ensure the smoothness of the narrative, VR animation uses the World-To-World editing method more than the traditional animation Frame-To-Frame editing method.

Fig. 6.

Fade in in VR animation <The dream collector>

In order to balance the story content and technical experience, the camera position in VR animation is also very important. In the VR animation <Pearl>, the creator tells the story by fixing the position of the camera [Figure. 7], and the whole story uses long shots, which reduces the jumping of the viewing angle and makes the story content smoother.

Fig. 7.

Camera position in VR animation <Pearl>

Most of the current VR animations still maintain a straight story script. In fact, this conflicts with the multi-linear narrative of VR animation. The story content of future VR animations should be composed of multi-linear scripts. This allows the viewer to enter the story from different angles, which not only satisfies the technical experience of the audience, but also ensures the integrity of the content of the animated story.

For example, the VR animation <Pearl> is a multi-linear script structure. There are several plot trigger points in the middle of the animation. And the animation innovatively uses a large number of skipping lenses to convert space. Faced with such a narrative approach, viewers may see different story endings, ranging from 5 minutes to 7 minutes. In the process of watching, the audience may have read the stories brought by multiple clues, and may also feel the general in the interaction. This is because of the different experience results caused by different ways of interacting with the audience. This makes the digital animation in the virtual reality context different from the traditional animation: the route the viewer walks, the viewpoints encountered, the length of the viewing, or the resources collected and used, etc., vary with the interaction. Therefore, the audience may preset a variety of story endings according to the experience results, and even directly influence the trend of the story content in the performance and selection of the story. It can be seen that in VR animation, the multi-linear script allows the viewer's eyes to interact directly with the story content.


Ⅲ.Conclusion

The VR animation turns the viewer from the recipient of the message into the explorer at 360° free viewing angle. The traditional framework in VR animation disappears, and the linearity of the narrative method has been disintegrated, but the current VR animation still retains the linear story content. There is a certain conflict between linear story content and non-linear narrative, so the VR animations on the market are short and the content is not perfect. Through the research of existing VR animation content and related papers, we can find two main problems in current VR animation: First, in the free perspective, the visual interest points of the audience need to be guided. Second, the narrative style of VR animation needs innovation.

In VR animation, over-guiding will affect the viewer's freedom of viewing, but without guidance, the audience will be lost in the panoramic view, even missing the key points of the story, affecting the narrative steps. In the current VR animations, the most common is to use lighting and sound elements to attract the viewer's vision. The movements and changes of lights and sounds can guide the movement and change of the points of interest of the audience. Secondly, the interaction with the characters in the VR animation also has the effect of guiding the visual changes of the audience. The interaction of the characters can directly lead the audience into important episodes, and can create suspense in the narrative. Compared with traditional animation, the composition of the background in VR animation is more important. The background setting should fully consider the location of the audience. Setting the main space and the secondary space can leave the audience in important episodes, avoiding the omission of the audience. To a certain extent, to ensure the integrity of the plot.

Compared with traditional animation, the lack of montage in VR animation greatly affects the narrative length and fluency of animation. Therefore, in addition to the visual guidance of the audience, the narrative mode of VR animation needs multi-angle innovation. The multi-linear script allows the viewer to enter important episodes from different perspectives, thus ensuring that the narrative is complete and that the audience has absolute freedom of viewing. Secondly, the interaction between the audience and the character can supplement the animation plot, extend the narrative rhythm of the animation, give the audience enough reaction time, and also enhance the fun of the VR animation content.

Through research, we can find that each element in VR animation has new features and plays a more important role in animation. In VR animation, through the animation elements to guide the viewer's vision, and enhance the multi-linearity of the narrative content, the technical experience and story content of the VR animation can be balanced to some extent, so that the audience can completely explore the storyline completely, or Guarantee the integrity of the animation content.

Reference

  • Wikipedia, The Free Encyclopedia. [Internet], Available: https://en.wikipedia.org/wiki/Virtual_reality.
  • Lang-Goo Lee, Jean-Hun Chung, "A Study on Visual Mise-en-Scene of VR Animation <Pearl>", Journal of Digital Convergence, 15(9), (2017), p407-413.
  • Doo-hui Lee, A Study on Continuous Transition Technique in VR Animation – focused on author's artwork, M.A. dissertation, Chung-Ang Univ., (2018).
  • Hong-joo Shin, A Study on the Storytelling of VR Animation, Ph.D. dissertation, Seojong Univ., (2018).
  • Jiyoung Kang, Sanl-Il Choi, "A Study on Pipeline for Effective Production of VR Animation", Journal of Digital Contents Society, 19(5), (2018, May), p971-979. [https://doi.org/10.9728/dcs.2018.19.5.971]
  • Shin, Hongju, "An Examination of Point of View in VR Contents", Korean Animation Society Conference, (2016, Jun), p183-188.
  • YongWoo Park, Study on VR animation production :focused on VR animation Minimon, M.A. dissertation, Chung–Ang Univ., (2017).
  • Hyo-Jin Chang, Sun-Hee Chang, Ihn-Jou Kim, "A Study on VR Contents Line of Sight Design : Focusing on 〈INVASION!〉", The Korean Journal of animation, 14(1), (2018, Mar), p148-162.
  • Fangyuan Song, 「Story world and virtual space -- exploration and thinking on narrative mode of VR animation」, (2017).
  • Qingke Liu, 「On the influence of virtual reality on animation art creation」, (2012).
  • Calahan, Sharon, 「Storytelling through Lighting, a Computer Graphics Perspective」, (2016).
  • Lu jun, 「360-degree One Take.Analysis of VR Animation Movie Help」, (2017).
  • Marc Cavazza, 「Interacting with virtual characters in interactive storytelling」, (2002). [https://doi.org/10.1145/544741.544819]
저자소개
이이맹(Yi-Meng Li)

2018년 : 세한대학교(B.A)

2014~2018: 세한대학교 만화애니메이션 학과

2018~현재: 중앙대학교 뉴미디어아트 학과

※ 관심분야:뉴미디어아트(New media art), 애니메이션(Animation), 가상 현실(Virtual Reality technology)

주종우(Jong-Woo Joo)

2001년 : 중앙대학교 (B.A)

2004년 : Brooks Institute of Photography (M.S)

2008년 : 중앙대학교 (M.A)

2014년 : 중앙대학교 (ph.D)

2013년~현 재: 국가기술표준원 사진분과 전문위원

2014년~현 재: 기술표준원 문화예술서비스 심의의원

2015년~현 재: 사진진흥법 추진위원회 부회장

2015년~현 재: 현대사진영상학회 편집위원

2017년~현 재: 중앙대학교 공연영상창작학부 조교수 및 일반대학원 뉴미디어아트전공 학과장

※ 관심분야: 디지털 이미지 (Digital Image), 가상 현실 (Virtual Reality Technology) 드론 (Drone)

Fig. 1.

Fig. 1.
Research plan

Fig. 2.

Fig. 2.
《Virtual Reality Technology》- G.Burdea /P.Coiffet

Fig. 3.

Fig. 3.
Object movement in VR animation <Henry>

Fig. 4.

Fig. 4.
The light changes in VR animation <The dream collector>.

Fig. 5.

Fig. 5.
Character gaze in VR animation <Henry>.

Fig. 6.

Fig. 6.
Fade in in VR animation <The dream collector>

Fig. 7.

Fig. 7.
Camera position in VR animation <Pearl>

Table 1.

Research on visual guidance.

Big-classification Small –classification Research Research result
Visual guidance a. Lighting -VR Animation ProductionResearch..2017 -If the lighting does not induce the viewer's attention, the flow of animation may be missed.
-The Role of Eye..2014 -The concentration of gaze through lighting can be used as a dramatic factor that triggers expectations with dramatic effects connected with the story.
b.Character -VRAnimation 〈pearl〉..2017 -The familiar face and the relatively natural motion and smoke can attract the viewer's attention and immerse themselves in the work.
-VR Veiw point of content..2017 -In the case of a method of inducing the viewer's gaze through the movement of an interesting person or object, there is a risk of disturbing the concentration on the narrative.
c. Background -VRAnimation 〈pearl〉..2017 -The diversity of the background concept and the consistency that appeared in the place are characteristic of persisting the viewer's gaze and continuing the immersion.
-VR Animation Production. 2012 -In the VR content, the background design should be designed in consideration of the shape, style, size, etc. in accordance with the location of the audience and the composition of the presentation of the scenario.
-VR Animation 〈Henry〉. 2018 -The effect of the gaze collection through proper size and position, and the elements of interest should be appropriately harmonized.
d. Sound -VR Animation story. 2018 -The deceased sound of the interaction region is implemented segmentally, and both the dejecis sound and the vidisys sound are implemented unilaterally.
-VR content..2017 -Hearing allows you to recognize the approximate position of the sound source according to the arrival time difference of both sound waves and to prepare for the change such as the sound source approaching or departing according to the change of sound wave and frequency I make it.
-VR Animation 〈Henry〉. .2018 -In free-of-sight situation, sound plays a role of attention.

Table 2.

Research on Storytelling in VR animation

Big-classification Small-classification  Research result Research result
Storytelling a. Camera -360-degree One Take..2017 -One-take lens makes narration of VR animation story more smooth and reduces dizziness.
-View point of VR content...2017 -In VR images, the position of the viewer, that is, the position of the 360 degree camera becomes more important than the field of view or angle of the existing movie camera.
-Understanding virtualreality..2013 -In VR, the presentation of space according to the position of the camera becomes more important.
-VR Animation Production Research..2017 -In the case of VR animation, since the position of the viewer is the position of the camera and the viewer's eyes are the frame of the camera, when designing the background, it is necessary to design the level considering the position of the viewer.
-Application Research..2014 -We were able to reduce the nausea by fixing the camera and using other passengers that could move the camera.
b.Scenario -AI-based Animation..2001 -Under interactive mode, multilevel playwriting is needed, and technology and content must be balanced.
-View point of VR content.. 2017 -The free viewpoint of VR requires the paradigm shift of video grammar.
-VR content..2017 -Nonlinear characteristics in VR images can play an important role as catalysts for expanding the richness of the enjoyment through the arrangement of micro contents between the main constituent points of the story.
c. Scene conversion -Continuous in VR animation..2017 -In the survey results, it was confirmed that the scene conversion method of VR contents presented in this study is natural and induces immersion.