Korea Digital Contents Society

Journal Archive

Journal of Digital Contents Society - Vol. 21 , No. 8

[ Article ]
Journal of Digital Contents SocietyVol. 21, No. 8, pp.1401-1408
Abbreviation: J. DCS
ISSN: 1598-2009 (Print) 2287-738X (Online)
Print publication date 31 Aug 2020
Received 23 Jul 2020 Revised 10 Aug 2020 Accepted 19 Aug 2020
DOI: https://doi.org/10.9728/dcs.2020.21.8.1401

Use of AR interaction principles for product design research : Using China's Yunnan Pu'er Tea Product Design as an Example
Zhao Jian1 ; Younghwan Pan2, *
1PhD Course, Department of Smart Experience Design, TED, Kookmin University, Seoul 136-702, Korea
2Professor, Department of Smart Experience Design, TED, Kookmin University, Seoul 136-702, Korea

AR의 인터렉션 원리를 이용한 제품 디자인 연구 : 중국 운남 보이차 제품디자인을 예로
조건1 ; 반영환2, *
1국민대학교 테크노디자인전문대학원 스마트경험디자인학과 박사과정
2국민대학교 테크노디자인전문대학원 스마트경험디자인학과 교수
Correspondence to : *Younghwan Pan E-mail: 568196760@qq.com

Copyright ⓒ 2020 The Digital Contents Society
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-CommercialLicense(http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Funding Information ▼


This research aims to prove the effectiveness and rationality of utilizing Augmented Reality (AR) in the product design of Yunnan Pu’er Tea, a Chinese Intangible Cultural Heritage.This research constructs three methods of information exchange utilizing AR technology based on existing Pu' Er Tea product information; 1) AR static information exhibiting method. 2) AR dynamic information exhibiting method. 3) AR tactile interactive presentation exhibiting method. In conclusion, the new method allows better representation of the artistic value of intangible historical cultural heritage. This modernized method of passing down traditional cultural heritages allows better protection and universalization, as well as provide a platform for future AR informational exchange based product design


본 연구는 중국 인류무형문화유산 등재 후보에 오른 중국 운남 보이차 제품 디자인에 인터렉션으로 대표되는 증강 현실(AR) 기법을 도입하여 이러한 신 개념 디자인의 유효성과 합리성에 대해 증명해 보고자 하였다.이를 위해 본 연구에서는 기존의 보이차 제품 디자인을 기반으로 세 가지 디자인 기법 및 프로세스를 제안하였다. 먼저 전통적으로 전해져 내려오는 제품 디자인 정보를 수합하고 그를 기반으로 1) 제품에 대한 정적인(static) 정보를 추가한다. 2) 제품에 대한 동적인(dynamic) 디자인 정보를 추가한다. 3) 증강현실 내에서 콘텐츠를 인터렉션하는 경로 및 유효한 전달 방법을 설계한다. 그 결과 전통적인 디자인 방식에 비해 새로운 방식이 인류무형문화유산의 역사적이고 예술적인 가치를 더욱 잘 드러내고 더욱 현대적이고 미래지향적인 방법으로 전승할 수 있게 한다는 결론에 이르렀다. 이러한 방식으로 전 세계적으로 전통문화유산에 대한 보호와 보편화에 기여할 수 있으며 향후 AR 인터렉션 플랫폼을 이용한 제품 디자인에 응용할 수 있다.

Keywords: AR information, ICH, AR technology, AR interaction, Tea
키워드: AR정보, 무형문화재, AR기술, AR 인터렉션, 찻잎

Ⅰ. Introduction
1-1 Background of AR and Intangible Cultural Heritage tea

As shown on the international intangible cultural heritage big data platform of UNESCO, in the intangible cultural field, the overall cognition of users in intangible cultural heritage stays on the basic concept, the deep cognition of intangible cultural heritage is less and the methods for experience cognition are limited.[1] It is the same in the intangible cultural heritage tea field. When the traditional tea enters the modern society, users do not have enough cognition in the traditional process and historical culture of the tea, and the experience is single.

With the development of modern science and technology information, the Internet as the core and foundation has been continuously applied to various network models, information exchange and communication equipment have gradually expanded to the network interaction of objects, objects and people interaction, and information interaction structure.[2] On this basis, artificial intelligence virtual interaction technology, identification technology, mobile phones with camera, etc. are widely used, and augmented reality applications on mobile terminals are gradually shaped. In 2003, Wagner et al. Took the lead in realizing the augmented reality application of the mobile terminal PDA (Personal Digital Assistant, PDA).[3] In 2007, Sony Computer Entertainment Inc. SCE (Sony Computer Entertainment Inc) released the first augmented reality consumer game "THE EYE OF THE JUDGEMENT"; in 2011, Nintendo of Japan released the nintendo 3Ds portable augmented reality game with game consoles, more and more children can experience augmented reality in a general sense. AR makes the scene more realistic and allows users to experience better.[4] Leverage the power of AR interactive technology to inherit cultural heritage, integrate technology and non-legacy humanities, study the principle and construction of AR information interaction will help designers develop and design non-legacy AR products, and promote the experiences for users to recognize, protect and inherit intangible cultural heritage.

1-2 Principles of AR Information Interaction

AR augmented reality technology is to identify the designed identification information through the mobile device terminal (AR headsets or cameras, gyros, sensors and other accessories on smart mobile devices).[5] Real-time information and data interaction are realized by superimposing the real-world image information and virtual object position matching information and tracking information after recognition through mobile device terminals The interactive information can be various data such as images, text, video, sound, etc. Virtual three-dimensional information is presented from real two-dimensional information collection.[6] The enhanced AR information is digital virtual information, which guides users to recognize the objective world from the blend of virtual and real information. It' s not just about getting information from the outer world and getting to know the world. This new information interaction mode has storage capacity, portability and virtuality.

Fig. 1. 
Diagram of AR information interaction principle structure

1-3 Output Form of AR Information

AR information output is based on the real-time interaction between real-time real environment and virtual scene information, through AR device screening information, and combined diverse information output, so that the device terminal AR information display result can be divided into: static information, dynamic information and tactile interactive information.See Figure2.

Fig. 2. 
Output Form of AR Information structure

Ⅱ. Method
2-1 Preparations for Experiment

To verify the effectiveness of the AR information interaction principle clarified before in the science popularization of intangible cultural heritage-tea to users in practice, the study designs the science popularization experiment based on the packages of Chinese Pu'er Tea of Yun Nan intangible cultural heritage in three forms of AR information output.[7] In the early stage of the experiment, the study conducts information collection and AR-APP design. As shown in the Figure3, Figure 4[8]:

Fig. 3. 
AR-APP Principle Design Process

Fig. 4. 
AR virtual technology design process for output devices (mobile phones)

The experimental control variables include unified learning content and learning time, and no significant differences in the tested users. The independent variables are different cognitive methods. A total of 40 subjects participated in this experiment. Volunteers participating in the experiment were divided into four groups, namely traditional paper reading non-legacy content, AR static information presenting non-legacy content, AR dynamic information presenting non-legacy content, AR tactile interactive presentation of non-legacy content. Each group is independent to each other. In the experiment, the variables that need to be controlled include the same received content, the same processing time, and the significant differences in the general cognitive level of the individuals participating in the experiment. Meanwhile, different groups adopt different methods to obtain the same information of products. The details of each group are as shown in Figure5 below.

Fig. 5. 
Experimental groups and contents

2-2 Analytic Hierarchy Process (AHP)

Analytic Hierarchy Process can subjective evaluation through numerical calculation to build an evaluation system of different goals, it is an effective method of assisting decision-making and has been applied in most fields.[9] In this experiment, the analytic hierarchy process will be used to build an evaluation system for different product information and intangible cultural heritage delivery methods to initially determine whether AR interaction technology has advantages in the transmission of science popularization of intangible cultural heritage-tea. This paper builds a hierarchical structure model based on the subjective evaluation of different methods obtained by the participants after participating the experiment, and performs a pairwise comparison according to Table1 to construct a judgment matrix.

Table 1. 
Comparison of the importance of different indicators
The A indicator
compared to the B indicator

Extremely important Very important Important Equal
7:1 5:1 3:1 1
Not important Very unimportant Extremely unimportant
1:3 1:5 1:7

The criterion matrix judgment matrix constructed based on the evaluation data after the test is shown in Table2.[10] Among them, fun reflects whether the participant interested in the way information is obtained in the experiment, and it is related to whether we can continue and concentrate on obtaining the corresponding information in a period of time; endurance reflects the participant’s information. The endurance shown in the reception process corresponds to impatience. Endurance is usually related to fun, but it is not completely positively related. It may also be related to the speed of audio playback and the sensitivity of tactile reflection in AR interaction; convenience reflects the participant's perception of how convenient it is to obtain information in the experiment, that is, whether this method can only be carried out in specific scenarios and whether it is easy to carry and not easy to lose; the quality of information not only reflects the participant's The evaluation of the value of the information itself also reflects the degree of acceptance of the information by the participants. When the quality of the information obtained is higher, the participants believe that the value of the information is higher, and they will know more about the subject of the information. Going deeper, Based on this, it is beneficial to the science popularization of intangible cultural heritage-tea.

Table 2. 
Criteria layers of judgment matrix( A
Endurance Convenience Fun Quality of the
Endurance 1 1/3 1/5 1/7
Convenience 3 1 1/3 1/4
Fun 5 3 1 1/2
Quality of the information 7 4 2 1

Ⅲ. Results
3-1 The experimental results

On the basis of Calculating the weights using arithmetic average method, and the obtained feature vector is ①, which means that the participants think that they are considering, for example, in the dissemination of intangible cultural heritage related content and product information, information quality is the primary consideration, followed by the degree of novelty in the communication method, that is, whether it is interesting, and again, whether it is patient during the information reception process, then convenience of the medium performs consistency on the obtained weight vector ②.

Table 3. 
Numerical result correspondence
W=0.05860.13350.30490.5030 AW=11/31/51/7311/31/45311/274210.05860.13350.30490.5030=AW1AW2AW3AW4=0.23590.53671.24992.0570

The calculated maximum feature value is ③, the maximum feature root is 4.0588, and the further calculation is ④ and the table lookup is R.I.=0.9, so the final consistency index can be calculated by ⑤, which meets the consistency condition.

According to this idea, the decision-layer judgment matrix can be further constructed.See Figure6. Bi means the object to be investigated, B1 means the way displayed in group 1, namely to read the product manual with product information and accompanying intangible cultural heritage content; B2 means group 2, that is, to recognize the product through AR interaction and understand the relevant content of intangible cultural heritage, but only for the proportion of video; B3 means group 3, that is, animation effects added on the basis of group 2; B4 means increase on the basis of group 2 and group 3 Tactile interaction to understand products and understand the corresponding content of intangible cultural heritage. The constructed judgment matrix is shown below, and the following four judgment matrixes have all passed the consistency test.Table 4.

Fig. 6. 
Hierarchical target framework

Table 4. 
Numerical result correspondence

Based on the weight vectors calculated by the above decision layer and criterion layer, four groups of weight vectors can be obtained: that is, AR with tactile interaction from the four perspectives of fun, patience,convenience and information quality Interaction is the best in the process of information acquisition, which not only ensures the quality of information transmission, but also improves the fun of information transmission, so that users have the patience to receive the knowledge content and product information of intangible cultural heritage. Group 2 is not much different from Group 3, Group 3 is slightly better than Group 2, which means that adding animation effects based on AR interactive imaging can improve the overall effectiveness of the subjects in the information acquisition process.

Fig. 7. 
Comparison of traditional reading and AR interaction effects

3-2 Influence factors of AR information interaction in the science popularization of intangible cultural heritage-tea.

During the process of AR interaction in this part, the study will evaluate what AR factors will impact the science popularization of intangible cultural heritage-tea to users. In this part, the mainly studied factors are as shown in Table5. Among which C1 and C2 use instruments to measure objective data.

Table 5. 
Numerical result correspondence
Variable description Average value Maximum Minimum Standard deviation
C1 Scanning distance (m) 0.52 1 0.1 0.383
C2 AR imaging time after scanning 0.60 1 0 0.498
C3 Your satisfaction with the smoothness of this AR interaction 3.23 5 1 1.104
C4 Your satisfaction with the video playback and voice speed in AR interaction 3.57 5 3 0.568
C5 Your satisfaction of AR interaction effects when you are moving 3.17 4 1 0.834
C6 Your satisfaction of AR interaction effects when the object is moving 2.73 4 1 0.785
C7 Your satisfaction with AR imaging when there is occlusion on the scanned object 2.77 4 1 0.935
C8 Your satisfaction with this AR interactive animation element 3.33 4 2 0.547
C9 Your evaluation of AR tactile interaction sensitivity 3.33 4 2 0.758
C10 Does AR interaction help you better understand intangible cultural heritage related content 3.30 5 2 0.877
C11 Do you think it is necessary to use AR interaction in non-legacy heritage 3.60 5 1 1.037
C12 Your overall evaluation of this AR interactive experience 3.07 5 1 1.143

Note: If the image is formed within 1 second after scanning, the corresponding AR imaging time is set as 1; if the imaging time is more than 1 second, the corresponding AR imaging time is 0.

In order to deduct the correlation between different factors, this paper uses a correlation coefficient matrix to quantify the linkage between different variables. The correlation coefficient reflects the degree of linear correlation between the two variables. The closer the absolute value is to 1, the stronger the correlation is. When the correlation coefficient is positive, there is a positive linear relationship between the two variables. When the correlation coefficient goes negative, there is a negative linear relationship between the corresponding two variables. The correlation coefficient matrix obtained from the above variables is shown in the following table. As shown in the Figure8.

Fig. 8. 
Thermal map of correlation coefficient

1) Overall evaluation of this AR interaction

According to the correlation coefficient matrix, it can be found that there is a strong negative correlation between the scanning distance and the imaging time, tactile interaction sensitivity, and the overall evaluation of this AR interaction. Longer scanning distance usually leads to longer imaging time, which gives the subjects a bad experience in the initial stage of the experiment, which leads to a low overall evaluation of the AR interactive experience. The AR interaction of the object when it is covered, when it is moving, and when the object is moving will also affect the final experience. AR interaction depends on the captured target object, and special effects animation is generated on this basis, but when the object appears to be covered or difficult to capture, the success rate of AR interaction will decrease accordingly. Therefore, if it is difficult to capture objects if AR technology can still better grasp the target and quickly generate interaction, the user's evaluation of AR interaction can be significantly improved. In addition, AR interactive tactile sensitivity will also significantly affect the subject's evaluation of AR interactive technology. In general, to increase users' recognition of AR interaction technology, it is necessary to grasp the success rate of imaging in AR interaction, because imaging is the first step in the subsequent development of interactive functions.

2) Influence on the science popularization of intangible cultural heritage-tea

From the perspective of correlation coefficient alone, relevancy between each factor and the science popularization of intangible cultural heritage-tea is relatively low, with C9 and C10 had the highest correlation coefficient of 0.52, and C8 and C11 had the lowest correlation coefficient of 0.48, which means the subjects' understanding on knowledge of intangible cultural heritage-tea will be influenced by the tactile sensitivity in AR interaction and the quality of animation effects. As there may be strong collinearity between each factors, in order to further consider the influence of multiple factors on AR interaction technology in inheritance of intangible cultural heritage, multiple linear stepwise regression is conducted here with C10 and C11 as the explained variables respectively, and multiple linear regression can investigate the influence of multiple variables on specific explained variables. Regression results of multiple linear regression are as follows.

Table 6. 
Table of regression results with dependent variable C10
Variable name Coefficient Standard error t-statistic Sig.
Constant term 1.300 0.638 2.036 0.051
C9 0.600 0.187 3.210 0.003
R-squared 0.269 F-Statistic 10.307

Table 7. 
Table of regression results with dependent variable C11
Variable name Coefficient Standard error t-statistic Sig.
Constant term 0.523 1.058 0.495 0.625
C8 0.923 0.313 2.947 0.006
R-squared 0.237 F-Statistic 8.682

With C 10 as the independent variable, the regression obtained a model with an R-squared of 0.269, indicating that the chosen explanatory variables explained approximately 26.9% of the changing process of the explained variables, and the model as a whole passed the F test. The results of stepwise regression showing that subjects' answer to whether AR interaction could be applied to the science popularization of intangible cultural heritage-tea will be significantly influenced by the tactile sensitivity in AR interaction.

With C11 as the independent variable, the regression obtained a model with an R-squared of 0.237, indicating that the chosen explanatory variables explained approximately 23.7% of the changing process of the explained variables, and the model as a whole passed the F test. The results of regression showing that whether people believe that AR interaction is indispensable and necessary in the science popularization of intangible cultural heritage-tea will be influenced by the animation effects in AR interaction. Only when the animation effects AR interaction truly reflect the knowledge of tea process and tea culture could it play a vital role in the inheritance of intangible cultural heritage-tea.

From this, it can be said that AR interaction did contribute to people's science popularization of intangible cultural heritage-tea and its relevant tactile sensitivity in application for the reason that tactile interaction allows people to touch the content of science popularization more realistically, and it can change their routine cognitive processes. When considering whether AR interaction plays a role in science popularization of the intangible cultural heritage-tea, animation effects in AR interaction show a significant effect. Thus, it can be seen that both animation effects and tactile sensitivity of AR interaction will become the two key factors which are experimentally applied in the science popularization of intangible cultural heritage-tea. The three information interaction methods of AR are conducive to the science popularization of intangible cultural heritage-tea.

Ⅳ. Summary

The AR process design of tea culture is guided and popularized by arranging the interaction input and output principle of the AR product information. The presentation for AR contents of tea culture is designed by establishing three AR product interactive output forms. After user experience, the influence factors of AR product information interaction in popularizing the intangible cultural heritage-tea by collecting data from examination paper and questionnaires, and analyzing and comparing the experimental data.

It can be seen from the experimental results that in the traditional reading method of science popularization and the AR reading method of science popularization, the results of users receiving information about intangible cultural heritage-tea are different. For the process of science popularization of intangible cultural heritage-tea, there are many deficiencies in the simple paper-based reading method, because users can only receive information through pictures and text descriptions, and it is of low efficiency for users to receive knowledge about tea. The AR information interaction method can give rise to users' interests with an information display mode, which is superior to the traditional paper-based reading method, and it is of high efficiency for users to receive knowledge about tea. With the three information methods which are outputs of the combination of AR real and virtual, users can have an intuitive feeling and cognition, which enhance the visual experience and effect of science popularization. The superiority of the AR information output form for the science popularization of intangible cultural heritage-tea and the influence of AR key factors on user experience were proved by experimental data. The principle of AR information interaction and information output form will help designers and users better design and popularize the field of intangible cultural heritage-tea respectively.


This study was completed with the support of TED studio of kookmin university of South Korea. I would like to thank my supervisor, professor Young Hwan,Pan, for his dedicated guidance on the topic selection, the structure of the paper, and the later adjustment. I would also like to thank my classmates who helped me.

1. DIICH Intangible Cultural Heritage Big Data Platform, 2018 Survey Report on Chinese Internet Users' Non-legacy Cognition and Demand, 2018. Available: https://www.sohu.com/a/249915853_99954114.
2. Market Insights Reports. Global Augmented Reality (AR) Market Size, Status And Forecast 2022. 2017.
3. Jihong Liu, Yang Yang, Guide for Augmented Reality (AR): Foundation. Development. Practice, 1th ed. Publishing by Electronic Industry Press, 2018.2.
4. NewsCenter, 2010.5. Available: http://news.178.com/201005/67199570143.html.
5. Hong-Seok Park,Hung-Won Choi, "Generation virtualization technology, AR Technology", CAD&Graphics, pp.156-175, may 2008.
6. Irshad S, Awang Rambli D R, Multi-layered Mobile Augmented Reality Framework for Positive User Experience, Acm Digital Library, 2016.4.
7. Yang W, Liu L, Wang Y, et al, Design and Development of Alcohol Packaging Anti-counterfeiting System Based on Augmented Reality Technology, 2019.3.
8. Kefa Zhang,Xing Zhao, Youlong Xie, AR/VR Development Step By Step, 2016.11.
9. Belmont, California: Wadsworth,Decision Making for Leaders: The Analytical Hierarchy Process for Decisions in a Complex World, 1982.
10. Yanping Jiang, Zhiping Fan, Decision theory and method based on judgment matrix, 2008.12.

Author Information
조건(Zhao Jian)

2010년 : 후난산업대학교 (문학 학사)

2015년 : 쿤밍이공대학교 (에술 석사)

2019년~현재: 국민 대학교 테크노디자인전문대학원 스마트경험디자인학과 박사과정

2010년~2014년: 후난산업대학교

2015년~현 재: 쿤밍이공대학교

※관심분야 : 인터랙션디자인(Interaction Design), 가상현실(AR)

반영환(Young-hwan Pan)

1999년 : KAIST 인간공학 박사

1993년 : KAIST 인간공학 석사

1991년 : KAIST 전기전자공학 학사

국민대학교 테크노디자인전문대학원 스마트경험디자인학과 주임교수

삼성전자 DTV UX 책임 연구원

삼성전자 전사 사용성 인증 위원

팬택&큐리텔 UI 팀장 및 팬택 계열 해외 UI 팀장

중국 칭화대학교 방문교수

사단법인 대한사용자경험전문가협회 협회장

사단법인 한국서비스디자인학회 학회장

※관심분야 : 인터랙션디자인(Interaction Design), 서비스디자인(Service Design), 도시UX(City UX)