A Case Study on Development of Mobile Chatbot for Promoting Creative Ideation

Ⅰ. Introduction

Mobile chatbots are a type of system that enables users to exchange information while providing support based on interactive agents on a mobile platform, which is a useful technology for millennials who are familiar with text-based interactions [1]. However, many existing chatbot services have not been developed for educational purposes [2]. Also, creativity is emphasized as a necessary capability for future generations, but creativity education tends to be in the form of one-time education or one-way lectures. The meta-study of creativity training shows that creativity training is most effective when conducted across more than eight weeks with less than 20 individuals. Existing research emphasizes the expansion of the framework of thinking by accessing creative cases, supporting self-directed idea searching, promoting intrinsic motivation, and creating an open atmosphere to enhance creativity [3]. In addition, to effectively promote creativity in training, considering the context and situation where creativity is required is also needed. Therefore, a plan for assisting with daily training should be prepared [4].

This study focuses on the insufficient level of discussion on the educational use of chatbots while exploring the possibility of using them in the context of creativity education. In addition, this study analyzes the user's perceptions and experiences with mobile chatbot services developed to enhance creativity, as well as suggests a future direction for development.

Research Question 1. How are users aware of the mobile chatbot service to increase creativity?

Research Question 2. What experiences did users experience while using mobile chatbots to increase creativity?

The rest of the article is defined as follows. In Section 2, we analyze the role and type of chatbot and usage cases in education. Section 3 describes the chatbot Idea Ping pong application and procedure of a case study. In section 4, we present the results of the perceptions and experiences of 23 participants about the usage of chatbot. Then, we lead to a conclusion including some suggestions for future research and limitations of this study.

Ⅱ. Literature review

2-2 Chatbot Usage cases in education

Implementing a conversational bot in the educational domain is an emerging trend. The usage of chatbots has been considered beneficial in education for some time. Chatbots can be used as a tool to support the learning process in various system environments such as mobile [8, 9, 10, 11, 12, 13], web-based [14, 15, 16, 17], or a software program installed a single computer [18, 19, 20, 21, 22, 23]. These days many students can learn the subject anytime and anywhere(Table 1).

Table 1.

Research regarding educational chatbots

As mentioned above, chatbots can help teachers and students throughout the learning process. Educational chatbots have been developed for various purposes. A number of bodies of research suggest that chatbots can used as a tool to retrieve information on a specific topic [8, 10, 11, 13, 17, 18, 19, 23], to offer test questions to students [9, 10, 11, 12, 15, 16, 18, 23], to practice a language [14, 21], to strength student motivation [20], or to enhance creativity [22].

The recent research has been on the development of a mobile-based chatbot. Mobile-based systems have advantages in the educational domain. They can support personalization and students’ involvement in their learning. These factors are crucial for efficient learning [8, 11]. A mobile-based chatbot will provide higher availability and accessibility, more so than a web- and program-based solution to students.

Most research aimed to serve information on a specific topic and converse with users. Leveraging chatbots for enhancing learners’ creativity is unusual. Goel et al [22] showed that IBM WATSON was used to enhance human-computer co-creativity in the computer creativity class of the Georgia Tech Spring 2015 project. In computer creativity classes, students use biomimicry design methods to solve problems.

This research showed that this chatbot could support the biomimicry design process and enhance human-computer co-creativity. WATSON ,demonstrated in Fig.1, however, is an intelligent robot and not a chatbot designed to help students spur their creative ideas. There is a need to develop an optimized version of a chatbot that promotes learner’s creativity. In the classroom, it is hard to enhance creativity with real robot solutions such as WATSON. To cover them, we propose a mobile base chatbot designed to help students develop creative ideas.

Fig. 1.

Erasmus Chatbot Use Case (Goel et al., 2015)

Ⅲ. Materials and Methodology

3-1 Overview of Idea Ping Pong

The name of the chatbot developed in this study to promote creative ideas is called 'Idea Ping Pong.' Idea Ping Pong enhances user creativity by suggesting a keyword that requires an idea, selecting a field, and then generating and presenting a specific idea based on innovation cases(Fig. 2).

Fig. 2.

Idea PingPong use case (1) - Enter keyword and present idea

Idea PingPong was produced using our own DB, not API, and the rule-based heuristic system was used to match new ideas to innovation cases. In other words, the innovation case DB is designed to select an appropriate response in a heuristic manner based on the user’ input and context where the conservation takes place.

Fig. 3 shows the system structure of Idea Ping Pong. First, the keyword analysis engine took a method of selecting key keywords by morphing the user input. The inference engine is responsible for suggesting new ideas based on generation rules among various innovation cases. The evaluation engine is an engine that collects the user's evaluation results on the innovation idea proposed by the system, thereby improving the service.

Fig. 3.

Idea PingPong System Structure

Idea Ping Pong provides customized content based on the user's evaluation of the proposed idea. If the user writes his or her ideas, the user may provide emotional feedback for the ideas presented and additionally offer related ideas(Fig. 4).

Fig. 4.

Idea PingPong use case (2) - Enter emerging ideas and providing feedback

3-2 Contextual Inquiry and Participants

In this study, Contextual Inquiry was conducted with graduate students and office workers in order to analyze mobile chatbot experiences. Contextual exploration is the process of understanding the user through the process of observation, analysis, interpretation, and discussion in order to identify the user's needs. 23 participants took part in the experiment, 7 males and 16 females. The participants' basic information in this study is detailed, as follows (Table 2).

Table 2.

Basic information of participants

We gathered information about experiences when using chatbots. 13 participants responded to less frequency of chatbot usage. Participants did not have much experience in terms of using chatbots(Table 3). The most frequent type of chatbot that participants had used is ‘Interactive commerce service and O2O’, followed ‘Personal assistant service’(Table 4). We used SPSS to calculate the means and standard deviation of the responses that the participants made about awareness of chatbot. Participants thought they have the skills needed to use chatbot (M = 3.57, SD = 1.363) and using chatbot can have a positive impression on people (M = 3, SD = .836). After conducting a series of tasks based on mobile chatbots developed in this study, we collected chatbot experiences and related opinions.

Table 3.

Experience using chatbots

Table 4.

Types of chatbots they have used

Table 5.

Awareness of the use of chatbots

Some tasks that were presented to learners for the analysis of chatbot experience are as follows. Participants ran the idea ping-pong for 10 minutes, during which they were able to: 1) identify the innovations provided by the idea ping-pong, 2) evaluate the innovations, 3) check multiple innovations, and 4) Continued five or more times generating their ideas. After conducting the tasks, we presented participants with a questionnaire and held interviews to confirm the opinions regarding the utilization process, the opinions on how to provide innovation information, on how to communicate, on the idea generation process, and other relevant opinions.

Ⅳ. Results

4-1 Research Question 1. How are users aware of the mobile chatbot service for increasing creativity?

Examining the learners' perceptions of the idea of using ping-pong, they deliberated much on the process of using it (M = 3.52, SD = 1.030), were somewhat immersed (M = 3.52, SD = .980), and to some extent meaningful ideas could be generated (M = 3.95, SD = .864). In addition, they did not feel that the application of Idea Ping Pong was too complicated (M = 3.14, SD = 1.388) and did not feel uncomfortable in its use (M = 3.00, SD = 1.342)(Table 6). Through this, we found that the usability of the idea ping-pong and the perceived usefulness to the user were appropriate.

Table 6.

Opinions on the idea ping-pong application process

As a result of gathering opinions on the educational effectiveness of Idea Ping Pong, learners recognized that Idea Ping Pong contributed to the creative performance of learners (M = 3.67, SD = .795), and could be used positively by anyone (M = 3.86, SD = .793). The participants responded moderately positively with respect to the level of utilization in the workplace (M = 3.48, SD = .872) or the effects of continuous use (M = 3.43, SD = 1.121). In addition, as a result of collecting opinions on the paid service for idea ping-pong, it was determined that willingness to pay among the respondents was rather low; see Table 7

Table 7.

Opinions on the educational use of idea ping-pong

Ⅴ. Conclusion

This study aimed to develop an educational chatbot for promoting creativity, as well as to explore the possibility of providing assistance with further development and suggest direction through usability evaluations. The analysis of chatbot experiences among 23 learners, including 12 graduate students and 11 office workers, showed that a significant number of respondents agree that chatbots can help promote creativity and contribute to creative ideas. In particular, learners were positively aware of promoting new perspectives and approaches, fostering pleasant emotions in the process of drawing creative ideas, and encouraging them to focus on their ideas. However, the patterning of conversations with the chatbot was boring at times, and the way of linking keywords with excessively strange cases sparked feelings of inefficiency. As well, the quick communication made it difficult to lead to mature ideas.

The result of this study is valuable in that it provides an opportunity to explore how chatbots can be used to enhance higher-order thinking and creativity. However, in order for the chatbot developed in this study to lead to better performance, the following attempts should be made.

Firstly, it is necessary to make the chatbot's suggestions feel more appropriate by further subdividing the categories of ideas to promote creative ideas, in addition to supplementing the relevant case and idea data.

Secondly, we should consider whether the divergence or convergence of ideas should be promoted. If the focus is on promoting the release of ideas, then it is necessary to provide a variety of questions and checklists to promote innovation as well as provide examples of innovation. If we plan to support the convergence and evolution of ideas, it is also necessary to support the storage, sorting, filtering, and evaluation of said ideas.

Thirdly, creative ideas are often required in group contexts, such as team meetings, rather than in individual situations. Thus, a contextual change should be considered ranging from services for individual users to services for group users. Using chatbots together for evolving collaborative ideas would also be an effective way to connect with new possibilities for educational chatbots.

Acknowledgments

This work was supported by National Research Foundation of Korea Grant funded by the Korean Government(KRF-2019-S1A5A8-036708)

References

• Geetha Swaminathan, Dr. R. Magesh., "Conversational intelligence-power of chatbot", International Journal of Computer Engineering and Applications, Vol. 12, No. 1, pp. 269-274, January 2018.
• Følstad, A., Brandtzaeg, P. B., Feltwell, T., Law, E. L., Tscheligi, M., & Luger, E. A., "SIG: Chatbots for Social Good", in Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, New york , p. SIG06, 2018.
• Kazerounian, K., Foley, S., "Barriers to creativity in engineering education: A study of instructors and students perceptions", Journal of Mechanical Design, Vol. 129, No. 7, pp. 761-768, 2007. [https://doi.org/10.1115/1.2739569]
• Richards, R., Everyday creativity, in The Cambridge handbook of creativity. pp. 189-215, 2010. [https://doi.org/10.1017/CBO9780511763205.013]
• Shawar, B. A., Atwell, E., " Chatbots: are they really useful?", in LDV forum. pp. 29-49. 2007.
• Molnár, G., Szüts, Z., "The Role of Chatbots in Formal Education", in 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY), Subotica, pp. 197-202, 2018. [https://doi.org/10.1109/SISY.2018.8524609]
• Kane, D., The role of chatbots in teaching and learning, in E-Learning and the Academic Library: Essays on Innovative Initiatives. 131, 2016.
• Alepis, Efthymios, and Maria Virvou., "Automatic generation of emotions in tutoring agents for affective e-learning in medical education", Expert Systems with Applications, Vol. 38, NO. 8, pp 9840-9847, August 2011. [https://doi.org/10.1016/j.eswa.2011.02.021]
• Dutta, D. "Developing an Intelligent Chat-bot Tool to assist high school students for learning general knowledge subjects. Georgia Institute of Technology, 2017
• Griol, D., Molina, J. M., Callejas, Z., "Incorporating android conversational agents in m-learning apps", Expert Systems, Vol. 34, No. 4, e12156, June 2016. [https://doi.org/10.1111/exsy.12156]
• Fonte, F. A. M., Burguillo, J. C., Caeiro Rodriguez, M., Llamas Nistal, M., Rial, J. C. B., Caeiro Rodríguez, M., " NLAST: A natural language assistant for st2016 IEEE Global Engineering Education Conference(EDudents", in UCON), Abu Dhabi, pp. 709–713, 2016. [https://doi.org/10.1109/EDUCON.2016.7474628]
• Pereira, J., "Leveraging chatbots to improve self-guided learning through conversational quizzes", in Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality, New york, pp. 911-918, 2016. [https://doi.org/10.1145/3012430.3012625]
• Ranoliya, B. R., Raghuwanshi, N., Singh, S., "Chatbot for university related FAQs", in 2017 International Conference on Advances in Computing, Communications and Informatics(ICACCI 2017), Udupi, pp. 1525-1530, 2017. [https://doi.org/10.1109/ICACCI.2017.8126057]
• Ayedoun, E., Hayashi, Y., Seta, K., "A Conversational Agent to Encourage Willingness to Communicate in the Context of English as a Foreign Language", Procedia Computer Science, Vol. 60, pp. 1433–1442, January 2015. [https://doi.org/10.1016/j.procs.2015.08.219]
• Crockett, K., Latham, A., & Whitton, N., "On predicting learning styles in conversational intelligent tutoring systems using fuzzy decision trees", International Journal of Human Computer Studies, Vol. 97, pp. 98-115, January 2017. [https://doi.org/10.1016/j.ijhcs.2016.08.005]
• Gehl, R. W., "Teaching to the Turing test with Cleverbot", Transformations: The Journal of Inclusive Scholarship and Pedagogy, Vol. 24. No. 1-2, pp. 56-66. January 2014.
• Rossi, P. G., Carletti, S., "MAPIT: a pedagogical-relational ITS ", Procedia Computer Science, Vol. 3, pp. 820–826, 2011. [https://doi.org/10.1016/j.procs.2010.12.135]
• Bickmore, T. W., Mitchell, S. E., Jack, B. W., Paasche-Orlow, M. K., Pfeifer, L. M., O’donnell, J., "Response to a relational agent by hospital patients with depressive symptoms", Interacting with Computers, Vol.22, No.4 , pp. 289-298, July 2010. [https://doi.org/10.1016/j.intcom.2009.12.001]
• Bickmore, T. W., Schulman, D., Sidner, C., "Automated interventions for multiple health behaviors using conversational agents", Patient Education and Counseling, Vol. 92. No. 2, pp. 142-148, August 2013. [https://doi.org/10.1016/j.pec.2013.05.011]
• Heller, R., Procter, M., "Animated pedagogical agents: The effect of visual information on a historical figure application", International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), Vo. 4, No.1, pp. 54-65, January 2009. [https://doi.org/10.4018/jwltt.2009010104]
• Jia, J., "CSIEC: A computer assisted English learning chatbot based on textual knowledge and reasoning", Knowledge-Based Systems, Vol. 22, No. 4, pp. 249-255 May 2009. [https://doi.org/10.1016/j.knosys.2008.09.001]
• Goel, A., Creeden, B., Kumble, M., Salunke, S., Shetty, A., Wiltgen, B., "Using Watson for Enhancing Human-Computer Co-Creativity", in 2015 AAAI Fall Symposium Series, Arlington, 2015.
• Mikic, F. A., Burguillo, J. C., Llamas, M., Rodríguez, D. A., Rodríguez, E., "Charlie: An aiml-based chatterbot which works as an interface among ines and humans", in 2009 EAEEIE Annual Conference, Valencia, pp. 1-6, 2009. [https://doi.org/10.1109/EAEEIE.2009.5335493]