Research on Graphical User Interface Design for Virtual Reality Content : Based on Racing Drone Content
Copyright ⓒ 2019 The Digital Contents Society
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-CommercialLicense(http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Unlike general contents that are experienced through monitors, HMD-based virtual reality contents are characterized by users entering the virtual environment and experiencing them. These features provide users with more immersion and presence. However, the use of GUI designs of general content in VR content can instead obstruct immersion or cause inconvenience to users. In particular, it is difficult to use GUI, which is common for contents that reproduce fast speed, such as racing drones. Therefore, this research implemented a GUI design that is suitable for VR content focused on racing drone content. Through this, this study proposes solutions to issues of GUI design for VR content by presenting minimizing unnecessary elements, centralized design, curved spacial design. This can be expected to provide a theoretical basis for VR content with the speed and application of GUI suitable for the VR environment.
초록
모니터를 통해 체험하는 일반적인 콘텐츠와는 달리 HMD 기반의 가상현실 콘텐츠는 가상환경 속에 사용자가 직접 들어가서 체험한다는 특징이 있다. 이러한 특징은 사용자에게 더 큰 몰입감과 현존감을 제공한다. 그러나 가상현실 콘텐츠에 일반적인 콘텐츠의 GUI 디자인을 사용하면 오히려 몰입을 저해하거나 사용자에게 불편을 줄 수 있다. 특히 레이싱 드론과 같이 빠른 속도를 재현하는 콘텐츠에 일반적인 GUI는 사용하기 어렵다. 따라서 본 연구는 레이싱 드론 콘텐츠를 중심으로 가상현실 콘텐츠에 적합한 GUI 디자인을 구현하였다. 이를 통해 불필요한 요소의 최소화, 화면 중앙으로 집중시키는 디자인, 곡선 형태의 공간형 디자인을 제시하여 가상현실 콘텐츠의 GUI 적용 시 발생하는 문제점에 대한 해결방법을 제안하고자 한다. 이는 레이싱 드론 콘텐츠 이외에도 빠른 속도감을 재현하는 가상현실 콘텐츠와 가상현실 환경에 적합한 GUI 적용에 대한 이론적 기반을 제공할 것으로 기대할 수 있다.
Keywords:
Virtual reality, Graphical user interface, Design, Game, Drone키워드:
가상현실, 그래픽 사용자 인터페이스, 디자인, 게임, 드론Ⅰ. Introduction
1-1 Background and Purpose of Research
The demand for drones has also been on the rise in recent years as the level of development technology of drones has improved and the general public's accessibility to drones has increased [1]. As the entry barrier for drones has been lowered, more and more people are entering leisure drones such as toy drones and racing drones (Fig. 1) [2]. Unlike relatively easy-to-handle toy drones, however, racing drones are not only very difficult to control, but also very fast, which is dangerous, and there are various real problems that high maintenance costs can occur in drone crashes [3]. To overcome the realistic limitations of these racing drones, Virtual Reality (VR) content is being produced that allows users to enjoy drone racing in a virtual reality environment rather than in real space. VR content for racing drones is Head Mounted Display (HMD) based content. HMD has an incomparable sense of immersion with other virtual reality devices. It uses stereo-scope technology to give the user a three-dimensional feel, and has a wide viewing angle, which greatly increases the sense of field [4]. Thus, HMD-based VR racing drone content can not only overcome spatial and monetary limitations, but also allow more people to pay attention to racing drones.
North America consumer drone market size by application, 2015-2024 (USD Million), As of 2018.
The process of creating VR content for racing drones also involved the User Interface (UI) design process, which found several problems with the normal Graphical User Interface (GUI) method, as the characteristics of VR content that must be played with HMD on must be taken into account. First, there were difficulties in operating the GUI while wearing HMD due to the number of interaction buttons placed on the main screen. Second, the problem was that the GUI design method of non-VR games was applied to reduce immersion and remote extensibility, which are the hallmarks of VR games. Finally, it was difficult for users to use the GUI during game play because they did not take into account the characteristics of racing drones flying at speeds of over 150 kilometers per hour.
This study aims to present a suitable GUI design for VR racing drones content with the aim of solving the above problems that arise when applying the general game GUI to VR content. Research is needed to complement UI design in order to ensure that users do not have difficulty playing VR games due to GUI, and to preserve immersion and remote extension.
1-2 Research Process
First, to design the UI applied to VR content for racing drones, an analysis was performed on the difference between GUI design used in general games and GUI used in VR environments. In order to apply GUI design that is suitable for VR environment rather than general GUI design to contents of VR racing drones that are manufactured, it is planning to set GUI flow and interface stage. In addition, the GUI design was carried out by taking into account the connectivity of VR environments for UI design methods with low interference in virtual reality environments and the characteristics of racing drone contents that play at high speed. The completed GUI has been applied to the Unreal Engine 4 to verify the operation of the interaction.
The GUI design for VR content in this study is limited to gaming content, especially to VR content for racing drones.
Ⅱ. Theoretical Investigation
2-1 Importance of Game GUI
In general, UI refers to a physical and virtual medium intended for temporary or permanent access to communicate between a person and an object or system, especially a machine, a computer program, etc [5]. However, GUI in games is different from other GUI because it contains additional elements [6]. Game GUI is all the more important because it should respond instantly to users' behavior, especially in games set in virtual space, as users are often confused about what to see or get lost.
Game GUI is one of the most undervalued but most important factors in game development. GUI is overlaid at the top of the screen to display health, scores, etc., as is commonly referred to as Heads Up Display (HUD) [7]. It doesn't exist in the world, but it means something to the player in the context of watching the game. GUI is a way for players to interact with games and receive feedback on interactions. For example, simply drawing a simple blinking dot on a map can have a big impact on how a player interacts with a game.
2-2 Comparison of VR GUI to General Game GUI
The GUI and Virtual Reality GUI of a typical game have something in common that provide information to users about content, making it easier for users to use the content. However, different game platforms need UI designs that take into account the differences [8]. Therefore, there are differences of GUI between VR and general game. First, in a common game, the GUI is located in the outer part of the screen, but it is not the right GUI for virtual reality. In a normal game, a GUI is placed on the outside of the screen, especially at the bottom, so that users can get a lot of information at a glance through the monitor. But in a virtual reality environment, users don't look at the graphics through the monitors, they go into the graphics world. Therefore, if a GUI is placed in the outer part of the screen, the user must look and turn around to see much of the information.
Virtual reality is immersive, which allows the media itself to disappear and makes the objects that are represented in front of you seem natural. However, the disappearance of media is virtually difficult because of the devices required for virtual reality [9]. Virtual reality has a great advantage of telepresence, and to create it, it must exclude something that might not be in reality such as the GUI. Since the general GUI is hyper-mediated, it is difficult to apply the general GUI to non-dispatch-oriented virtual reality content. Because general GUI is hyper-mediated, it is different from GUI of virtual reality content that is non-mediated. Therefore, it is difficult to apply the GUI of general content to virtual reality content.
Ⅲ. GUI Design for VR Contents
3-1 GUI Design Flow
Prior to GUI design, it is easy to design GUI flow and proceed with work. GUI flow allows users to predict in what order or flow they use when using any content or service, and helps users use content or services more naturally. GUI flowchart was first designed for GUI flow design. A flowchart is a type of diagram that represents a workflow or process. A flowchart can also be defined as a diagrammatic representation of an algorithm, a step-by-step approach to solving a task. The flowchart shows the steps as boxes of various kinds, and their order by connecting the boxes with arrows [10]. Flowcharts used for GUI design works efficiently for mapping out the overall flow of a game. This flowchart only shows the flow of the main menu GUI of the content, and the 'drwa.io' was used to design flow charts. The GUI flowchart image is shown in Fig. 2.
Graphical User Interface Flow Chart
3-2 Interface Depth Settings
It was needed to minimize the number of interface screens by taking into account the limited interaction method of VR compared to the interaction method of PC games or console games. To meet these conditions, a method of depth setting is applied to the interface. As shown in Fig. 3, when interactions are made on higher-level GUI screens, the next screen, the lower-level GUI screen, is designed in such a way that it creates depth as it overlaps on the previous GUI screen. This makes it easier for users to see what level of GUI they are currently using without having to place many interaction buttons or screens. The layer classification for depth stage representation is shown in Table 1.
Layer Depth Representation of User Interface
Classification of Graphical User Interface Layers by Level of Depth
3-3 Implementation of Interaction Design
After finishing the design of the GUI, the interaction operation was verified using ‘Unreal Engine 4’ to implement the actual operation of the GUI. UI in virtual reality should be developed using World Space coordinates, unlike UI that uses screen coordinate system in regular games. This is because virtual reality games are not like regular games where users play games through monitors. First step is to extract the designed GUI as a texture file, then produce a 3D mesh and mapped it to it. The 3D mesh is constructed in curved form rather than straight form to create a spatial UI for users to use the UI more comfortably.
Setting the User Interface on Unreal Engine 4
If a screen coordinate system's GUI needs to be displayed on the screen in a virtual reality environment, 'Stereo Layers' must be activated. The stereo layer sends a single screen coordinate system GUI texture to HMD so that the GUI can appear in both lenses of the HMD.
3-4 Connectivity of VR Environment and Adequacy of Racing Drone
Humans have a slightly over 210 degree forward-facing horizontal arc of their visual field, and the vertical range of the visual field in humans is around 150 degrees [11]. However, since this is the maximum viewing angle, it must be located at a much smaller angle to make it easier for users to see. Because what is as important as providing telepresence to users when experiencing virtual reality content is to provide them with comfort. According to Mike Alger, radius in which users can view virtual reality content unnecessarily without moving much of their eyes is 94 degree forward-facing horizontal arc, and the vertical range of it is 32 degree (Fig. 5) [12].
Comfortable Views in Virtual Reality Environment
The interface design of a typical game is placed outside the screen. Many gamers get caught up in information while playing games. If you need an immediate, quick decision as you go through the game, the important thing is to make sure the player knows what's under control immediately. In the iconic e-sports real-time strategy game StarCraft, released in 1998, the colors of the units we command don't have to doubt which players are in the same team fighting each other. It's very important because dozens of small moving units can be found on the screen at the same time [13].
This GUI design would have been designed to allow users to become more engrossed in game graphics for a common game such as PC games. On the other hand, GUI designs in virtual reality environments should be placed within an angle that is less than the user's viewing angle. The reason is that if the UI design of a typical game is applied to virtual reality contents, users will have to move their eyes from place to place while wearing VR HMD. And this inconvenience causes the user to become motionless, preventing them from focusing on the game or playing the game for a long time.
The UI of a normal game that enjoys speed mostly carries a variety of information such as game progress time, record-high lap time, current player speed, number of checkpoints to pass, and hit point (HP). And these UI are located in the outer part of the screen. This contains too much information to apply as it is to virtual reality. Therefore, the task was to exclude unnecessary information and to identify only the information that users needed. Extraction of essential GUI elements was divided into critical and unnecessary elements. And in detail, we divided the elements that we need to know in real time, the elements that we know after the game, and the elements that we don't need to know after the game.
Graphical user interface elements in game play
Detailed Graphic User Interface Element Classification
In addition, the content applied with GUI design in this paper was racing drone content, so the characteristics of racing drones had to be considered. The racing drone is a speed-sensitive drone that produces up to 120 meters per hour of power if controlled by a skilled pilot [14]. And there's also a sense of distance because racing drones don't run on the ground like racing cars do, but they're 3D driving in the air. Therefore, it is recommended that the GUI be placed in the center of the screen instead of the GUI distributed outside the screen as shown in Fig 6. Due to the characteristics of the 3D driving in the air, users may not be able to locate the checkpoint well. So this design applied a checkpoint guidance GUI that shows the location of the checkpoint. In addition, the horizontal scale GUI was applied to enable users to better recognize the sense of space.
Comparison of Image with UI for General Game and Virtual Reality Content
Ⅳ. Conclusion
Applying the UI to the game screen must be a way to make it easier for users to enjoy the game, while at the same time increasing the immersion and fun of game play. However, if you want to describe it as part of a VR game that plays with HMD on, applying the GUI format for a typical game can cause a negative user experience, such as distracting, reducing concentration, or disrupting game play so that users can’t play VR game for a long time or avoid using VR. Nowadays, more and more sectors want to introduce virtual reality technology. Therefore, content designers need to engage in user convenience in developing future virtual reality content and design interfaces.
Three design suggestions are made through the study of GUI design suitable for virtual reality content. First, to maintain immersion and presence in VR environment, spatial GUI design with curved surface is recommended. Second, minimizing and excluding unnecessary GUI elements provides for accessibility to the current situation of the game. Lastly, centralizing GUI to the screen helps users to receive information conveniently and to focus on the game especially for speed-sensitive content as racing drones.
The GUI produced in this study is a proposed design to apply to virtual reality content with features different from the typical game. The design method of the GUI is expected to provide a manual for not only GUI design for the racing drone content, but also for the other virtual reality content that is produced to enjoy speed and distance. Furthermore, it is expected that the study of GUI design suitable for such virtual reality content could provide directions for applying the appropriate UI to the virtual reality environment. Such research is expected to be available in developing contents in various fields as a manual proposal for developing VR game contents in the first place. It is expected that the development of contents that mitigate motion sickness will spread public use of VR contents in the short and long term.
In future studies, the use of VR content using general GUI design will be evaluated by comparing the usability of VR content with GUI that is suitable for virtual reality content. So that to make sure that these GUI design is actually suitable in virtual reality environments and provide people with positive user experience.
Acknowledgments
This research is supported by Ministry of Culture, Sports and Tourism(MCST) and Korea Creative Content Agency(KOCCA) in the Culture Technology(CT) Research & Development Program 2017.
References
-
R. Bharat, G. A. Goutham, M. Romana, The societal impact of commercial drones, Technology in Society, pp. 83-90, May 2016.
[https://doi.org/10.1016/j.techsoc.2016.02.009]
- Consumer drone market size by technology in North America from 2015 to 2024, Statista, Jan 8, 2018.
- Dronehaha. (April 2019) The ultimate drone repair guide. Drone repair and maintenance. [Internet]. Available: https://dronehaha.com/the-ultimate-drone-repair-guide/
- E. J. Song and A. L. Jung, "A Study for Reducing of Cyber Sickness on Virtual Reality", The Journal of Digital Contents Society, Vol. 18, No. 3, pp. 429-434, June 2017.
- SEVOCAB: Software Systems Engineering Vocabulary. Term: User interface. IEEE Computer Society, 2012.
- Y. C. Kwon, S. Y Park, W. H Lee, “A Study on Game Interface Using Multi-Touch”, Journal of The Korean Society for Computer Game, Vol. 19, pp. 14-20, December 2009.
- A. Minetti. The evolution of the UI in Games. What is a UI or HUD? [Internet]. Available: https://medium.com/mobile-lifestyle/the-evolution-of-the-ui-in-games-2be067fcc4ef
- Y. N. Choi and J. Y. Chun, "A Study on User Interface Design of MMORPG Game Based on Affordance: Focused on Comparative Analysis of UI Design according to Platform Differences of <Lineage>", The Journal of Digital Contents Society, Vol. 18, No. 7, pp. 1239-1248, Nov 2017.
- J. D. Bolter, R. Grusin, Remediation: Understanding New Media, MIT Press, pp. 20-52, 1999.
- SEVOCAB: Software Systems Engineering Vocabulary. Term: Flow chart. IEEE Computer Society, 2008.
- T. H. Moss, An Introduction to Clinical Perimetry, Chpt. 1. London: Henry Kimpton, 1938.
- M. Alger (September 2015). Visual Design Methods for Virtual Reality. Ravensbourne [Online]. Available: http://aperturesciencellc.com/vr/VisualDesignMethodsforVR_MikeAlger.pdf
- J. Bartoszewicz (June 2019). 5 Things That UX Design Should Learn from the Gaming Industry [Internet]. Available: https://medium.com/beautiful-code-smart-design-by-10clouds/5-things-that-ux-design-should-learn-from-gaming-industry-3eade3af97ad
- The Drone Racing League. Learn More. DRL Nesletter. What is FPV Drone Racing? [Internet]. Available: https://thedroneracingleague.com/learn-more/
2018년 : 경희대학교 디지털콘텐츠학과 (예술학 학사)
20018년～현 재: 경희대학교 대학원 디지털콘텐츠학과 재학 (예술학 석사과정)
※관심분야： 게임 그래픽, 게임 디자인, VR/AR 콘텐츠
2002년 : University of Dundee (UK), Electronic Imaging. BA (Honours)
2004년 : University of Dundee (UK), Electronic Imaging. MSc (이학석사)
2010년 : University of Dundee (UK), Electronic Imaging. (게임학), PhD (이학박사)
2004년～2007년: University of Dundee, Lecturer
2007년～2010년: KAIST 엔터테인먼트 공학연구소, 선임연구원 (기능성 게임랩)
2010년～2012년: KAIST 문화기술대학원, Digital Art & Entertainment Track 교수 (게임)
2012년～2013년: 서울대학교 융합기술대학원, 게임미디어랩 교수
2012년～2013년: 서울대학교 차세대융합기술연구원, 게임융합미디어연구센터 센터장
2013년～현 재: 경희대학교 디지털콘텐츠학과 교수
※관심분야： 기능성 게임, 게임화, 게임문화, VR/AR 콘텐츠