Korea Digital Contents Society
[ Article ]
Journal of Digital Contents Society - Vol. 20, No. 11, pp.2203-2210
ISSN: 1598-2009 (Print) 2287-738X (Online)
Print publication date 30 Nov 2019
Received 23 Sep 2019 Revised 22 Oct 2019 Accepted 05 Nov 2019
DOI: https://doi.org/10.9728/dcs.2019.20.11.2203

A Case Study on Development of Mobile Chatbot for Promoting Creative Ideation

Hyojung Jung1 ; Chae-Yeon Park2, * ; Jin-Ju Lee2 ; Sung Youn Pak3
1College of General Education, Dankook University, Gyeonggi-do, 16890, Korea
2Department of Educational Technology, Hanyang University, Seoul 04763, Korea
3Crevate, Seoul 04768, Korea, Korea
창의적 아이데이션 촉진을 위한 모바일 챗봇 개발에 관한 사례 연구
정효정1 ; 박채연2, * ; 이진주2 ; 박성연3
1단국대학교 교양교육대학
2한양대학교 교육공학과
3크리베이트

Correspondence to: *Chae-Yeon Park E-mail: chaeyeon.p.park@gmail.com

Copyright ⓒ 2019 The Digital Contents Society
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-CommercialLicense(http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The purpose of this study is to develop an educational chatbot for the enhancement of creativity and analyze the direct experience of using the chatbot through contextual inquiry. 23 users were required to utilize the chatbot named Idea Ping Pong and the experience of recognition and actual use of a said chatbot. As a result, most participants indicated that the chatbot could contribute to creative ideas. In particular, learners were satisfied with the chatbot's suggestion of a new perspective and approach. They responded that the chatbots helped with the development of joyful feelings and sustaining focus on their development of creative ideas. However, learners felt bored with the repetitive conversation patterns held with the chatbots and a quick way to communicate with chatbots hindering them to develop mature ideas. This research is valuable in that it provides an opportunity to explore how chatbots can be used to promote higher-order creative thinking.

초록

이 연구의 목적은 창의성 향상을 위한 교육용 챗봇을 개발하고 사용자들의 활용 경험을 분석하는 것이다. 이를 위하여 Idea Ping Pong이라는 챗봇을 개발하였으며, 23명의 사용자를 대상으로 챗봇에 대한 인식 및 활용 경험에 대한 설문을 진행하였다. 연구 참여자들은 챗봇이 창의적인 아이디어를 얻는 데 기여한다고 인식하였으며, 챗봇이 사례를 토대로 새로운 관점 및 접근을 제안해준 것에 만족한 것으로 나타났다. 또한, 챗봇이 즐거운 감정으로 창의적 아이디어를 도출하는 과정에 몰입할 수 있도록 도왔다고 응답하였다. 그러나 사용자들은 챗봇과의 대화 패턴이 반복되면서, 챗봇과 의사소통하는 방식에 대해 지루함을 느끼게 된 것으로 나타났다. 이 연구는 챗봇을 사용하여 고차적 차원의 창의적 사고를 촉진하는 방법을 탐색하는 기회를 제공한다는 점에서 가치를 갖는다.

Keywords:

Chatbot, Creativity, Ideation, Chatbot Mediated Learning (CML)

키워드:

챗봇, 창의성, 아이디어 도출, 챗봇매개학습

Ⅰ. Introduction

Mobile chatbots are a type of system that enables users to exchange information while providing support based on interactive agents on a mobile platform, which is a useful technology for millennials who are familiar with text-based interactions [1]. However, many existing chatbot services have not been developed for educational purposes [2]. Also, creativity is emphasized as a necessary capability for future generations, but creativity education tends to be in the form of one-time education or one-way lectures. The meta-study of creativity training shows that creativity training is most effective when conducted across more than eight weeks with less than 20 individuals. Existing research emphasizes the expansion of the framework of thinking by accessing creative cases, supporting self-directed idea searching, promoting intrinsic motivation, and creating an open atmosphere to enhance creativity [3]. In addition, to effectively promote creativity in training, considering the context and situation where creativity is required is also needed. Therefore, a plan for assisting with daily training should be prepared [4].

This study focuses on the insufficient level of discussion on the educational use of chatbots while exploring the possibility of using them in the context of creativity education. In addition, this study analyzes the user's perceptions and experiences with mobile chatbot services developed to enhance creativity, as well as suggests a future direction for development.

Research Question 1. How are users aware of the mobile chatbot service to increase creativity?

Research Question 2. What experiences did users experience while using mobile chatbots to increase creativity?

The rest of the article is defined as follows. In Section 2, we analyze the role and type of chatbot and usage cases in education. Section 3 describes the chatbot Idea Ping pong application and procedure of a case study. In section 4, we present the results of the perceptions and experiences of 23 participants about the usage of chatbot. Then, we lead to a conclusion including some suggestions for future research and limitations of this study.


Ⅱ. Literature review

2-1 Chatbot

1) The role of chatbots

Chatbots can imitate the conversational patterns of humans and provide a conversational interface in natural language. The chatbot has been used for retrieving information for finding the right answers to questions and practicing the learning of foreign languages through holding conversations with users [5]. As an information retrieval tool, chatbots answer users’ questions in such a way to do interactive FAQs to help customers make decisions. It is also used as a tool for language acquisition by playing the part of a conversational partner.

2) The type of chatbot

There are two types of chatbots; retrieval-based model and the generative model [6]. The retrieval-based model is the degree that the heuristic imitation of human conversation and memory usage to respond, such as answering a question using simple pairing. The generative models are based on comprehension, not predetermined answers. This method encodes user inputs and response outputs [6]. There are three types of chatbots according to the system environment. The first is a web-based chatbot that is online and accessed through a web interface. The second type is a standalone computer program which is accessed on a single computer [7]. The third is a mobile application that is accessed via a mobile interface, such as Android or iOS.

2-2 Chatbot Usage cases in education

Implementing a conversational bot in the educational domain is an emerging trend. The usage of chatbots has been considered beneficial in education for some time. Chatbots can be used as a tool to support the learning process in various system environments such as mobile [8, 9, 10, 11, 12, 13], web-based [14, 15, 16, 17], or a software program installed a single computer [18, 19, 20, 21, 22, 23]. These days many students can learn the subject anytime and anywhere(Table 1).

Research regarding educational chatbots

As mentioned above, chatbots can help teachers and students throughout the learning process. Educational chatbots have been developed for various purposes. A number of bodies of research suggest that chatbots can used as a tool to retrieve information on a specific topic [8, 10, 11, 13, 17, 18, 19, 23], to offer test questions to students [9, 10, 11, 12, 15, 16, 18, 23], to practice a language [14, 21], to strength student motivation [20], or to enhance creativity [22].

The recent research has been on the development of a mobile-based chatbot. Mobile-based systems have advantages in the educational domain. They can support personalization and students’ involvement in their learning. These factors are crucial for efficient learning [8, 11]. A mobile-based chatbot will provide higher availability and accessibility, more so than a web- and program-based solution to students.

Most research aimed to serve information on a specific topic and converse with users. Leveraging chatbots for enhancing learners’ creativity is unusual. Goel et al [22] showed that IBM WATSON was used to enhance human-computer co-creativity in the computer creativity class of the Georgia Tech Spring 2015 project. In computer creativity classes, students use biomimicry design methods to solve problems.

This research showed that this chatbot could support the biomimicry design process and enhance human-computer co-creativity. WATSON ,demonstrated in Fig.1, however, is an intelligent robot and not a chatbot designed to help students spur their creative ideas. There is a need to develop an optimized version of a chatbot that promotes learner’s creativity. In the classroom, it is hard to enhance creativity with real robot solutions such as WATSON. To cover them, we propose a mobile base chatbot designed to help students develop creative ideas.

Fig. 1.

Erasmus Chatbot Use Case (Goel et al., 2015)


Ⅲ. Materials and Methodology

3-1 Overview of Idea Ping Pong

The name of the chatbot developed in this study to promote creative ideas is called 'Idea Ping Pong.' Idea Ping Pong enhances user creativity by suggesting a keyword that requires an idea, selecting a field, and then generating and presenting a specific idea based on innovation cases(Fig. 2).

Fig. 2.

Idea PingPong use case (1) - Enter keyword and present idea

Idea PingPong was produced using our own DB, not API, and the rule-based heuristic system was used to match new ideas to innovation cases. In other words, the innovation case DB is designed to select an appropriate response in a heuristic manner based on the user’ input and context where the conservation takes place.

Fig. 3 shows the system structure of Idea Ping Pong. First, the keyword analysis engine took a method of selecting key keywords by morphing the user input. The inference engine is responsible for suggesting new ideas based on generation rules among various innovation cases. The evaluation engine is an engine that collects the user's evaluation results on the innovation idea proposed by the system, thereby improving the service.

Fig. 3.

Idea PingPong System Structure

Idea Ping Pong provides customized content based on the user's evaluation of the proposed idea. If the user writes his or her ideas, the user may provide emotional feedback for the ideas presented and additionally offer related ideas(Fig. 4).

Fig. 4.

Idea PingPong use case (2) - Enter emerging ideas and providing feedback

3-2 Contextual Inquiry and Participants

In this study, Contextual Inquiry was conducted with graduate students and office workers in order to analyze mobile chatbot experiences. Contextual exploration is the process of understanding the user through the process of observation, analysis, interpretation, and discussion in order to identify the user's needs. 23 participants took part in the experiment, 7 males and 16 females. The participants' basic information in this study is detailed, as follows (Table 2).

Basic information of participants

We gathered information about experiences when using chatbots. 13 participants responded to less frequency of chatbot usage. Participants did not have much experience in terms of using chatbots(Table 3). The most frequent type of chatbot that participants had used is ‘Interactive commerce service and O2O’, followed ‘Personal assistant service’(Table 4). We used SPSS to calculate the means and standard deviation of the responses that the participants made about awareness of chatbot. Participants thought they have the skills needed to use chatbot (M = 3.57, SD = 1.363) and using chatbot can have a positive impression on people (M = 3, SD = .836). After conducting a series of tasks based on mobile chatbots developed in this study, we collected chatbot experiences and related opinions.

Experience using chatbots

Types of chatbots they have used

Awareness of the use of chatbots

Some tasks that were presented to learners for the analysis of chatbot experience are as follows. Participants ran the idea ping-pong for 10 minutes, during which they were able to: 1) identify the innovations provided by the idea ping-pong, 2) evaluate the innovations, 3) check multiple innovations, and 4) Continued five or more times generating their ideas. After conducting the tasks, we presented participants with a questionnaire and held interviews to confirm the opinions regarding the utilization process, the opinions on how to provide innovation information, on how to communicate, on the idea generation process, and other relevant opinions.


Ⅳ. Results

4-1 Research Question 1. How are users aware of the mobile chatbot service for increasing creativity?

Examining the learners' perceptions of the idea of using ping-pong, they deliberated much on the process of using it (M = 3.52, SD = 1.030), were somewhat immersed (M = 3.52, SD = .980), and to some extent meaningful ideas could be generated (M = 3.95, SD = .864). In addition, they did not feel that the application of Idea Ping Pong was too complicated (M = 3.14, SD = 1.388) and did not feel uncomfortable in its use (M = 3.00, SD = 1.342)(Table 6). Through this, we found that the usability of the idea ping-pong and the perceived usefulness to the user were appropriate.

Opinions on the idea ping-pong application process

As a result of gathering opinions on the educational effectiveness of Idea Ping Pong, learners recognized that Idea Ping Pong contributed to the creative performance of learners (M = 3.67, SD = .795), and could be used positively by anyone (M = 3.86, SD = .793). The participants responded moderately positively with respect to the level of utilization in the workplace (M = 3.48, SD = .872) or the effects of continuous use (M = 3.43, SD = 1.121). In addition, as a result of collecting opinions on the paid service for idea ping-pong, it was determined that willingness to pay among the respondents was rather low; see Table 7

Opinions on the educational use of idea ping-pong

4-2 Research Question 2. What experiences did users undergo while using mobile chatbots to increase creativity?

To examine in detail the experience of using the idea ping-pong service, we analyzed the responses to the open questionnaire. The first question was, 'What did you feel positive about using idea ping pong?' As a result of the learners' responses, the positive aspects of using the idea ping-pong were classified into three categories.

1) Viewpoint switch

Learners have the opportunity to generate ideas in a new perspective through the innovation case proposed by Idea Ping Pong. Idea Ping Pong has provided innovation cases in a variety of fields and contexts, which not only shorten the time to collect cases, but also provide insights through opportunities such as connecting the various cases presented.

I was able to see a variety of ideas about one idea.

Unexpected answers open the way for other thoughts.

It is very useful as a search side because it finds and informs on the unfamiliar materials or unfamiliar contents.

There are two keywords that can't be combined within my head. It seems good for divergent thinking training.

2) Interesting feedback

Learners were able to experience fast, entertaining feedback during the conversation with the chatbot, which helped them think that the process of generating ideas was a pleasant and enjoyable process, rather than being a painful one.

It was interesting because I felt like I was talking to someone.

It was like having a daily conversation with friends, or just pouring out any thoughts that came to mind.

3) Focusing on their own ideas

Some participants also responded that they could concentrate on their thoughts by talking to the chatbot.

It was great to recommend ideas that I never thought about personally and to reorient myself.

The second question was, ‘what did you feel negative about using idea ping pong?’ As a result of the learners' responses, the disappointing aspects of using the idea ping-pong were classified into three categories.

4) Simple pattern

An idea Ping-Pong's dialogue is centered around the innovation case view> see other cases> present ideas> feedback loop. After repeating this process several times, learners continued to choose different cases and became bored.

At first, it was interesting because I felt like I was talking to someone, but the idea wasn't developed anymore, and the structure was simple. The algorithm needs to be able to recognize more complex situations to maintain persistence, but at present it seems to be weak.

In addition, the learner's response to the case was collected in order to provide an optimized case, and it was found that the situation where the user had to input his / her own response was generally bothersome to the learner.

Is it useful? Is it new? Repetitive questions were boring.

5) Wacky responses

Learners felt beneficial in helping Idea Ping-Pong access from a new perspective but often felt that responses overly irrelevant or out of context, making it difficult to derive useful ideas.

I entered ‘lipstick’. New idea comes out as a ‘lipstick that checks the freshness of fish’. It can lead to some really clever ideas, but I have often asked how to connect them.

The categorization was too wide and irrelevant in relation to the keywords I entered, so I didn't find any ideas that were decisively useful.

6) Fast-paced conversation

The chatbot's quick responses helped learners have fun and immerse themselves in the conversation, but there were limits for supporting them to review and mature their ideas fully.

In fact, the chatbot's responses were immediately presented, so I quickly reviewed and overcame the information or suggestions.


Ⅴ. Conclusion

This study aimed to develop an educational chatbot for promoting creativity, as well as to explore the possibility of providing assistance with further development and suggest direction through usability evaluations. The analysis of chatbot experiences among 23 learners, including 12 graduate students and 11 office workers, showed that a significant number of respondents agree that chatbots can help promote creativity and contribute to creative ideas. In particular, learners were positively aware of promoting new perspectives and approaches, fostering pleasant emotions in the process of drawing creative ideas, and encouraging them to focus on their ideas. However, the patterning of conversations with the chatbot was boring at times, and the way of linking keywords with excessively strange cases sparked feelings of inefficiency. As well, the quick communication made it difficult to lead to mature ideas.

The result of this study is valuable in that it provides an opportunity to explore how chatbots can be used to enhance higher-order thinking and creativity. However, in order for the chatbot developed in this study to lead to better performance, the following attempts should be made.

Firstly, it is necessary to make the chatbot's suggestions feel more appropriate by further subdividing the categories of ideas to promote creative ideas, in addition to supplementing the relevant case and idea data.

Secondly, we should consider whether the divergence or convergence of ideas should be promoted. If the focus is on promoting the release of ideas, then it is necessary to provide a variety of questions and checklists to promote innovation as well as provide examples of innovation. If we plan to support the convergence and evolution of ideas, it is also necessary to support the storage, sorting, filtering, and evaluation of said ideas.

Thirdly, creative ideas are often required in group contexts, such as team meetings, rather than in individual situations. Thus, a contextual change should be considered ranging from services for individual users to services for group users. Using chatbots together for evolving collaborative ideas would also be an effective way to connect with new possibilities for educational chatbots.

Acknowledgments

This work was supported by National Research Foundation of Korea Grant funded by the Korean Government(KRF-2019-S1A5A8-036708)

References

  • Geetha Swaminathan, Dr. R. Magesh., "Conversational intelligence-power of chatbot", International Journal of Computer Engineering and Applications, Vol. 12, No. 1, pp. 269-274, January 2018.
  • Følstad, A., Brandtzaeg, P. B., Feltwell, T., Law, E. L., Tscheligi, M., & Luger, E. A., "SIG: Chatbots for Social Good", in Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, New york , p. SIG06, 2018.
  • Kazerounian, K., Foley, S., "Barriers to creativity in engineering education: A study of instructors and students perceptions", Journal of Mechanical Design, Vol. 129, No. 7, pp. 761-768, 2007. [https://doi.org/10.1115/1.2739569]
  • Richards, R., Everyday creativity, in The Cambridge handbook of creativity. pp. 189-215, 2010. [https://doi.org/10.1017/CBO9780511763205.013]
  • Shawar, B. A., Atwell, E., " Chatbots: are they really useful?", in LDV forum. pp. 29-49. 2007.
  • Molnár, G., Szüts, Z., "The Role of Chatbots in Formal Education", in 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY), Subotica, pp. 197-202, 2018. [https://doi.org/10.1109/SISY.2018.8524609]
  • Kane, D., The role of chatbots in teaching and learning, in E-Learning and the Academic Library: Essays on Innovative Initiatives. 131, 2016.
  • Alepis, Efthymios, and Maria Virvou., "Automatic generation of emotions in tutoring agents for affective e-learning in medical education", Expert Systems with Applications, Vol. 38, NO. 8, pp 9840-9847, August 2011. [https://doi.org/10.1016/j.eswa.2011.02.021]
  • Dutta, D. "Developing an Intelligent Chat-bot Tool to assist high school students for learning general knowledge subjects. Georgia Institute of Technology, 2017
  • Griol, D., Molina, J. M., Callejas, Z., "Incorporating android conversational agents in m-learning apps", Expert Systems, Vol. 34, No. 4, e12156, June 2016. [https://doi.org/10.1111/exsy.12156]
  • Fonte, F. A. M., Burguillo, J. C., Caeiro Rodriguez, M., Llamas Nistal, M., Rial, J. C. B., Caeiro Rodríguez, M., " NLAST: A natural language assistant for st2016 IEEE Global Engineering Education Conference(EDudents", in UCON), Abu Dhabi, pp. 709–713, 2016. [https://doi.org/10.1109/EDUCON.2016.7474628]
  • Pereira, J., "Leveraging chatbots to improve self-guided learning through conversational quizzes", in Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality, New york, pp. 911-918, 2016. [https://doi.org/10.1145/3012430.3012625]
  • Ranoliya, B. R., Raghuwanshi, N., Singh, S., "Chatbot for university related FAQs", in 2017 International Conference on Advances in Computing, Communications and Informatics(ICACCI 2017), Udupi, pp. 1525-1530, 2017. [https://doi.org/10.1109/ICACCI.2017.8126057]
  • Ayedoun, E., Hayashi, Y., Seta, K., "A Conversational Agent to Encourage Willingness to Communicate in the Context of English as a Foreign Language", Procedia Computer Science, Vol. 60, pp. 1433–1442, January 2015. [https://doi.org/10.1016/j.procs.2015.08.219]
  • Crockett, K., Latham, A., & Whitton, N., "On predicting learning styles in conversational intelligent tutoring systems using fuzzy decision trees", International Journal of Human Computer Studies, Vol. 97, pp. 98-115, January 2017. [https://doi.org/10.1016/j.ijhcs.2016.08.005]
  • Gehl, R. W., "Teaching to the Turing test with Cleverbot", Transformations: The Journal of Inclusive Scholarship and Pedagogy, Vol. 24. No. 1-2, pp. 56-66. January 2014.
  • Rossi, P. G., Carletti, S., "MAPIT: a pedagogical-relational ITS ", Procedia Computer Science, Vol. 3, pp. 820–826, 2011. [https://doi.org/10.1016/j.procs.2010.12.135]
  • Bickmore, T. W., Mitchell, S. E., Jack, B. W., Paasche-Orlow, M. K., Pfeifer, L. M., O’donnell, J., "Response to a relational agent by hospital patients with depressive symptoms", Interacting with Computers, Vol.22, No.4 , pp. 289-298, July 2010. [https://doi.org/10.1016/j.intcom.2009.12.001]
  • Bickmore, T. W., Schulman, D., Sidner, C., "Automated interventions for multiple health behaviors using conversational agents", Patient Education and Counseling, Vol. 92. No. 2, pp. 142-148, August 2013. [https://doi.org/10.1016/j.pec.2013.05.011]
  • Heller, R., Procter, M., "Animated pedagogical agents: The effect of visual information on a historical figure application", International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), Vo. 4, No.1, pp. 54-65, January 2009. [https://doi.org/10.4018/jwltt.2009010104]
  • Jia, J., "CSIEC: A computer assisted English learning chatbot based on textual knowledge and reasoning", Knowledge-Based Systems, Vol. 22, No. 4, pp. 249-255 May 2009. [https://doi.org/10.1016/j.knosys.2008.09.001]
  • Goel, A., Creeden, B., Kumble, M., Salunke, S., Shetty, A., Wiltgen, B., "Using Watson for Enhancing Human-Computer Co-Creativity", in 2015 AAAI Fall Symposium Series, Arlington, 2015.
  • Mikic, F. A., Burguillo, J. C., Llamas, M., Rodríguez, D. A., Rodríguez, E., "Charlie: An aiml-based chatterbot which works as an interface among ines and humans", in 2009 EAEEIE Annual Conference, Valencia, pp. 1-6, 2009. [https://doi.org/10.1109/EAEEIE.2009.5335493]
저자소개
정효정(Hyojung Jung)

2006년 : 한양대학교 대학원 (교육학 석사)

2010년 : 한양대학교 대학원 (일반대학원 교육학 박사- 교수설계 및 이러닝)

2013년~현 재: 단국대학교 교양교육대학 교수

※관심분야:이러닝(E-learning), 창의적문제해결(Creative Problem Solving), 교육혁신(Educational Innovation)

박채연(Chae-yeon Park)

2018년 : 한양대학교 교육공학 학사

2018년~현 재: 한양대학교 대학원 교육공학과 석사과정 재학

※관심분야:Eye Movement Modeling Example, 문제해결(Problem-solving), 멀티미디어학습(multimedia learning)

이진주(Jin-ju Lee)

2018년 : 한양대학교 대학원 (교육학 석사)

2018년~현 재: 한양대학교 대학원 교육공학과 박사과정 재학

※관심분야: 생산적 실패(Productive failure), 문제해결(Problem-solving), ITS(Intelligent Tutoring System)

박성연(Sung Youn Pak)

2018년 : 서울대학교 소비자학과 박사 수료

2007년~현 재: 크리베이트 대표

2007년~2010년: 서울대학교 소비자학과 박사 수료

※관심분야:창의성(Creativity), 창의적문제해결(Creative Problem Solving), 혁신(Innovation)

Fig. 1.

Fig. 1.
Erasmus Chatbot Use Case (Goel et al., 2015)

Fig. 2.

Fig. 2.
Idea PingPong use case (1) - Enter keyword and present idea

Fig. 3.

Fig. 3.
Idea PingPong System Structure

Fig. 4.

Fig. 4.
Idea PingPong use case (2) - Enter emerging ideas and providing feedback

Table 1.

Research regarding educational chatbots

First Author [Source] Year System environment Purpose of chatbot
Mikic, F. A. [23] 2009 program To retrieve information&To offer test question
Heller, R.B. [20] 2009 program To strength student’ motivation
Jia, J. [21] 2009 program To practice a language
Bickmore, T. W. [18] 2010 program To retrieve information&To offer test question
Alepis, E. [8] 2011 mobile To retrieve information&To offer test question
Rossi, P. G. [17] 2011 web To retrieve information
Bickmore, T. W. [19] 2013 program To retrieve information
Gehl, R. W. [16] 2014 web To offer test question
Goel, A. [22] 2015 program To enhance student’s creativity.
Ayedoun, E. [14] 2015 web To practice a language
Fonte, F. A. M. [11] 2016 mobile To retrieve information&To offer test question
Pereira, J. [12] 2016 mobile To offer test question
Griol, D. [10] 2017 mobile To retrieve information&To offer test question
Dutta, D. [9] 2017 mobile To offer test question
Crockett, K. [15] 2017 web To offer test question
Ranoliya, B. R. [13] 2017 mobile To retrieve information

Table 2.

Basic information of participants

Attribute Classification Frequency (People) Percentage (%)
Sex Male 7 30
Female 16 69
Age 20s 10 43
30s 8 34
40s 5 21
Vocation Graduate student 12 52
Office worker 11 47

Table 3.

Experience using chatbots

Content Frequency (People) Percentage (%)
I didn't know what a chatbot was 3 13
No experience 2 8
Less experience (1-3 instances of use) 13 56
Medium (4-10 instances of use) 5 21
Experienced (everyday) 0 0

Table 4.

Types of chatbots they have used

Content Frequency (People) Percentage (%)
Interactive commerce service and O2O shopping / booking / ordering / calling etc. 12 12
Personal assistant service Healthcare / Weather Information / Financial Counseling / Schedule Management / Directions 10 43
Public service Legal Counseling / Tax Payment / Real Estate Information / Jobs 0 0
Entertainment services Advertising / Media / Broadcasting Information / Performance, etc. 1 4
Corporate messenger information search / file sharing / data archiving / team member information sharing / OA / CRM, etc. 5 21

Table 5.

Awareness of the use of chatbots

Content M SD
I've seen a lot of people using chatbots. 2.09 .830
Someone suggested using a chatbot around. 2.94 .889
I think using chatbots can have a positive impression on people. 3.00 .836
I have the skills needed to use a chatbot. 3.57 1.363

Table 6.

Opinions on the idea ping-pong application process

Content M SD
Idea Ping pong was developed very uncomfortably. 3.00 1.342
The process of deriving ideas using idea ping pong was very complicated. 3.14 1.388
During the process of using the idea ping pong I was very immersed. 3.52 .980
During the process of using the idea ping pong, we had to think a lot. 3.52 1.030
Efforts were made to derive meaningful ideas using the idea ping pong. 3.95 .864

Table 7.

Opinions on the educational use of idea ping-pong

Content M SD
Educational use of the idea ping pong would work. 3.48 .872
Idea Ping-Pong will help improve creative performance. 3.67 .795
Anyone will be good at using ideas ping pong. 3.86 .793
Continued use of idea ping pong will boost creativity. 3.43 1.121
Willing to use PingPong's paid service? 2.29 .717
I am willing to recommend Idea Ping Pong's paid service. 2.43 .597