Analysing the Metaverse in Smart City Management and Promotional Policies

Ⅰ. Introduction

A study by [1] argued that the image of a city can be more significant than its reality and noted that marketing techniques are frequently employed to transform a city into a post-industrial center of tourism, culture, and redevelopment. A study by [2] stated that the dimension of city marketing is divided into key segments including residents, visitors, and investors and tested this framework on policymakers in a region, using them as the unit of analysis. A study by [3] emphasized that city marketing involves promoting and selling cities, towns, or regions to a specific target markets. According to [4], the research domain of city branding has been a topic of ongoing debate across various academic disciplines, each of which has examined the phenomenon using diverse methods, conceptual frameworks, and empirical approaches. A study by [5] addressed that a city attraction hypothesis posits that global intercity competition revolves around a city’s ability to attract the highest possible value from global flows of resources and investments to promote urban development. A study by [5] also highlighted that attraction strategies can take several major forms, which may be used individually or in combinations,such as providing premises, offering incentives, and disseminating information. A study by [1] examined the significance of city marketing in urban governance decisions and investigated its relationship with urban tourism planning. According to [6], the promoting cities to become smart primarily stems from the problems caused by rapid urbanization, which often deteriorates the quality of life in large urban areas. According to [7], smart city marketing has become a tool to understand citizens’ needs and a driving force in creating a new market.

According to [8], industry 4.0, or the Fourth Industrial Revolution, is driven by innovative technologies. A study by [8] stated that this evolution has ensured that marketing activities align with technological advancements and address consumers’ current needs. A study by [9] emphasized the key enabling technologies for the metaverse, highlighting the primary benefits of adopting this technology and the opportunities it presents for smart city applications. Among the applications of the Fourth Industrial Revolution, this study investigated citizens’ perceptions of the application of the metaverse. According to [10], the rapid development of digital technology may accelerate society’s transition into the virtual world, blurring the boundaries between the physical and digital realms and fostering the growth of the metaverse. A study by [11] highlighted the need to focus research on a specific area within the metaverse to enhance both user-recognized aspects and technical developments. A study by [46] emphasized the potential success of the metaverse by applying Novelty, Convenience, and Usefulness (NCU) theory, widely recognized criteria for evaluating new technologies. A study by [47] highlighted the impact of the Unified Theory of Acceptance and Use of Technology (UTAUT), proposed by [48], in examining user satisfaction and intention to use. the metaverse platform. A study by [49] also applied UTAUT framework to explore the relationship between personality characteristics and intention to use the metaverse.

This study explores the perception and application of the metaverse in city management, focusing on its influences on overall attitude, quality of life, and promotional policies. Specifically, this study examines how citizens perceive the use of the metaverse in city management and how this application enhances their quality of life. Proposed factors affecting overall attitude include city value, affective factor, cognitive factor, customized offer, interactive factor, cost-saving factor, citizen relationship factor, and promotional factor. Furthermore, this study explored the impact of overall attitude, satisfaction, and loyalty on quality of life. The impacts of expansion for other cities, the prospects of the metaverse, and city management on better promotional policies were also analyzed. Among the applications of the metaverse in cities, this study measured its effects specifically within Seoul Metropolitan City. It explored Seoul’s innovative use of metaverse technologies in city management and promotional policies, highlighting the transformative potential of advanced technology.

II. Literature Review on Technology Applications for Cities

III. Hypotheses Development

Ⅳ. Methodology

This study conducted an online survey with the assistance of Korea Research International, a reputable research firm with a long-standing history since its establishment in [45]. As a global research company, it is trusted by over 500 businesses as a preferred research partner [45]. The questionnaire includes major questions along with warm-up and demographic questions. The major questions focus on factors that influence attitude, satisfaction, loyalty, and perceptions of the metaverse, as well as policies related to the application of high technology in city management. A 5-point Likert scale was applied for major questionnaire items. Stratified sampling was applied, taking demographics into account.The survey was distributed to citizens of Seoul, South Korea, to gather their perceptions on the application of metaverse platforms. Seoul is the focus of this study as the city is utilizing the metaverse for virtual city planning, enabling city planners to create and manipulate digital twins of urban areas. Data was collected from 396 respondents. The survey was utilized back translation to ensure accuracy. Participation in the survey was anonymous and voluntary, with informed consent obtained from all participants. All data was stored confidentially and used solely for research purposes. Factor analysis, Analysis of Variance (ANOVA), and multiple regression analysis was used to test the main hypotheses of the study. All respondents indicated that they have heard of the metaverse. Among them, 29.7% reported being very familiar with the metaverse, 54.7% stated they are familiar with it, and 15.6% said they are not very familiar with it. 61.4% of respondents indicated that they have experienced the metaverse. Regarding the application of the metaverse offered by the Seoul metropolitan City, 6.0% reported being very familiar with it,18.4% stated they are familiar with it, and 28.2% said they are somewhat familiar with it. This study conducted Cronbach’s alpha to check reliability. The results of as follows: 0.877 for city value, 0.874 for affective factor, 0.886 for cognitive factor, 0.807 for customized offer, 0.848 for interactivity factor, 0.885 for cost-saving factor, 0.870 for citizen relationship factor, and 0.883 for promotional factor. Table 1 summarizes demographics of the respondents.

Table 1.

Demographics of respondents

V. Data Analysis

This study conducted factor analysis to extract scale items based on constructs. Principal component analysis was used for extraction, with maximum iterations for convergence, and factors with eigenvalues greater than 1 are extracted. VARIMAX with Kaiser Normalization was applied as the rotation method, also with maximum iterations for convergence. Table 2 summarizes the component matrix, including factor loadings. The questionnaire items were developed as follows: i) City Value: Items were developed to measure how citizens perceive the application of the metaverse as increasing the value of the city, ii) Affective Factor: Items were developed to measure how citizens perceive the usage of the metaverse as playful and fun for the city iii) Cognitive Factor: Items were developed to measure how citizens perceive the usage of the metaverse as informative for the city, iv) Customized Offer: Items were developed to measure how citizens perceive personalized services provided from the application of the metaverse for the city, v) Interactive Factor: Items were developed to measure how citizens perceive the interactive and immersive technology provided by the application of the metaverse for the city, vi) Cost-Saving Factor: Items were developed to measure how citizens perceive the application of the metaverse as a cost-saving measure for the city, vii) Citizen Relationship Factor: Items were developed to measure how citizens perceive the application of the metaverse in building relationships with the public sector, and viii) Promotional Factor: Items were developed to measure how citizens perceive the application of the metaverse as a tool to enhance communication between the city and its residents and visitors.

Table 2.

Component Matrix for City Value, Affective Factor, Cognitive Factor, Customized Offer, Interactive Factor, Cost-Saving Factor, Citizen Relationship Factor, & Promotional Factor

This study conducted multiple regression analysis to test hypotheses. Factor scores were used in the regression analysis. The independent variables included city value, affective factor, cognitive factor, customized offer, interactive factor, cost-saving factor, citizen relationship factor, and promotional factor, while the dependent variable was overall attitude. The ANOVA results showed that the overall model is significant with F = 169.153 at p<0.01 and an R-square = 0.815 (adjusted R-square = 0.810). As shown in Table 3, the results indicated that the city value, affective factor, interactive factor, cost-saving factor, citizen relationship factor, and promotional factor significantly influenced overall attitudes toward metaverse application in Seoul Metropolitan City. Therefore, H1, H2, H5, H6, H7, and H8 were accepted. Among the significant results, the promotional factor had the greatest effect size on overall attitude, followed by the citizen relationship factor, affective factor, city value, interactive factor, and cost-saving factor. Overall, the effect sizes were strongest for the promotional factor and citizen relationship factor.

Table 3.

Effects of proposed factors on overall attitude

This study also conducted multiple regression analyses to test the effect of overall attitude, satisfaction, and loyalty on the quality of life regarding the application of the metaverse application for city management.The results of ANOVA showed that overall model is significant, with F = 409.875 at p< 0.01 and an R-square = 0.807 (adjusted R-square = 0.805). As shown in Table 4, the effect sizes were as follows: the effect size of citizen satisfaction on quality of life was 0.371, the effect size of loyalty on quality of life was 0.292,and the effect size of overall attitude on quality of life was 0.292. Therefore, hypotheses 9a, 9b, and 9c were accepted.

Table 4.

Effects on quality of life

Furthermore, this study conducted a regression analysis to test the effect of expansion to other cities, the prospect of metaverse application, and city management on the preparation of better promotional policies. The results of ANOVA showed that the overall model is significant,with F = 429.636 at p< 0.01 and an R-square = 0.718 (adjusted R-square = 0.710). As shown in Table 5, the effect sizes were as follows: the effect size of citizen satisfaction on promotional policy was 0.478, the effect size of prospect of metaverse application on promotional policy was 0.306, and the effect size of city management on promotional policy was 0.147. Therefore, hypotheses 10a, 10b, and 10c were accepted.

Table 5.

Effects on promotional policy

Ⅵ. Conclusion

This study explores the application of metaverse technologies in city management, highlighting the innovative approaches and potential benefits. The purpose of this study is to investigate the factors that influence overall attitudes, quality of life, and promotional policies regarding the adoption of the metaverse in city management. Specifically, this study focuses on the application of these technologies in Seoul Metropolitan city, based on perspectives of its residents. Seoul is a pertinent case as the city is actively utilizing the metaverse for virtual city planning, allowing planners to create and manipulate digital twins of urban areas. This study proposed various factors including city value, affective factor, cognitive factor, customized offer, interactive factor,cost-saving factor, citizen relationship factor, and promotional factor to measure their effects on overall attitudes toward the metaverse application in city management. Additionally, this study measures the effects of attitude, citizen satisfaction,and loyalty on quality of life, as well as the impacts of metaverse application expansion to other cities, the prospect of the metaverse, and city management on promotional policies.

The results of this study revealed that the impacts of city value, affective factor, interactive factor, cost-saving factor, citizen relationship factor, and promotional factor on overall attitude were significant. Among these, the promotional factor had the highest effect size on overall attitude, followed by affective factor, city value, interactive factor, and cost-saving factors. Therefore, this study found that citizens perceive the role of the metaverse as a tool to enhance communication between the city and its residents and visitors as highly important. The study also found that citizens view the use of the metaverse as playful and fun for the city, and as a means of increasing the city’s value. The study also found that citizens perceive the interactive and immersive technology provided by the application of the metaverse as beneficial for the city and view it as a cost-saving measure. However, the study found that the roles of the metaverse as an informative tool for the city and in providing personalized services do not have a significant impact on overall attitude. Furthermore, the results of this study found that the effects of citizen satisfaction on quality of life are highly significant, followed by the effects of loyalty and overall attitude. The results of this study found that the effects of city management on promotional policy are highly significant, followed by the effects of prospects toward the metaverse and its expansion to other cities.

The results of this study offer significant managerial and policy implications for the Seoul Metropolitan Government. Seoul’s innovative use of metaverse technologies in city management and promotional policies showcases the transformative potential of high technology. By adopting virtual city planning, smart infrastructure, virtual governance, and immersive promotional strategies, Seoul is setting a global benchmark. The online integration of the metaverse is expected to enhance urban living, drive economic growth, and establish Seoul as a leader in technological advancement and smart city development. The city could enhance its promotions by utilizing public advertising and integrated media services, combining mobile platforms and e-governance to raise awareness of metaverse services among citizens. Development of public promotional messages tailored to different age cohorts could help improve both awareness and outreach. Furthermore, highlighting and developing additional benefits of using the metaverse to enhance quality of life will improve both its usage and usefulness. This study also found that the development of effective promotional policies will enhance the implementation of the metaverse for city marketing.

To optimize the application of the metaverse in city management, it is essential to focus on enhancing citizen engagement through immersive experiences and improving feedback mechanisms. The study also suggests strategies to enhance the roles of the metaverse as an informative tool for the city and in providing personalized services. The city might consider creating virtual information hubs within the metaverse where citizens can access detailed and up-to-date information about city services, policies,and planning initiatives. Additionally, the city could use data analytics to provide personalized recommendations relevant to each citizen. Virtual assistants or chatbots could be implemented on a real-time basis to offer immediate support and information. By implementing these strategies, the metaverse can become a more effective and personalized tool for disseminating information and delivering services, enhancing its overall impact on city management. Additionally, better promotional policies should be developed to enhance citizen awareness of the metaverse application in city management. This study has limitations and provides implications for future research. Future studies might consider increasing the sample size and comparing the application of metaverse in other cities.

Acknowledgments

The author is grateful for the financial support from the KDI School of Public Policy and Management.

References

• C. Liouris and A. Deffner, “City Marketing: A Significant Planning Tool for Urban Development in a Globalised Economy,” in Proceedings of 45th Congress of the European Regional Science Association, Amsterdam, Netherlands, August 2005.
• A. Asnawi, D. Kartini, F. Afiff, and P. Rufaidah, “City Marketing: Scale Development and Measurement Indicators Applicated to Maluku Province – Indonesia,” Cogent Business & Management, Vol. 5, No. 1, 1525827, 2018. [https://doi.org/10.1080/23311975.2018.1525827]
• C. F. Osorio-Andrade, C. P. Murcia-Zorrilla, and E. Arango-Espinal, “City Marketing Research: A Bibliometric Analysis,” Revista Escuelade Administración de Negocios, Vol. 89, pp. 113-130, 2020. [https://doi.org/10.21158/01208160.n89.2020.2838]
• A. Lucarelli and P. O. Berg, “City Branding: A State-of-the-Art Review of the Research Domain,” Journal of Place Management and Development, Vol. 4, No. 1, pp. 9-27, 2011. [https://doi.org/10.1108/17538331111117133]
• A. Anttiroiko, “City Branding as a Response to Global Intercity Competition,” Growth and Change, Vol. 46, No. 2, pp. 233-252, June 2015. [https://doi.org/10.1111/grow.12085]
• A. Maček, R. Ovin, and U. Start-Peceny, “Smart Cities Marketing and Its Conceptual Grounds,” Našegospodarstvo/Our Economy, Vol. 65, No. 4, pp. 110-116, December 2019. [https://doi.org/10.2478/ngoe-2019-0024]
• P. Ekman, J. Röndell, and Y. Yang, “Exploring Smart Cities and Market Transformations from a Service-Dominant Logic Perspective,” Sustainable Cities and Society, Vol. 51, 101731, November 2019.
• A. T. Rosário and J. C. Dias, “Industry 4.0 and Marketing: Towards an Integrated Future Research Agenda,” Journal of Sensor and Actuator Networks, Vol. 11, No. 3, 30, 2022. [https://doi.org/10.3390/jsan11030030]
• I. Yaqoob, K. Salah, R. Jayaraman, and M. Omar, “Metaverse Applications in Smart Cities: Enabling Technologies, Opportunities, Challenges, and Future Directions,” Internet of Things, Vol. 23, 100884, October 2023. [https://doi.org/10.1016/j.iot.2023.100884]
• J. Huang, P. Sun, and W. Zhang, “Analysis of the Future Prospects for the Metaverse,” in Proceedings of the 2022 7th International Conference on Financial Innovation and Economic Development, pp. 1899-1904, March 2022. [https://doi.org/10.2991/aebmr.k.220307.312]
• S. Yoon and S. Choi, “A Study on the Future Prospect of the Metaverse through Human Cognition,” Journal of KIIT, Vol. 21, No. 7, pp. 149-168, July 2023. [https://doi.org/10.14801/jkiit.2023.21.7.149]
• M. Boisen, K. Terlouw, P. Croote, and O. Couwenberg, “Reframing Place Promotion, Place Marketing, and Place Branding – Moving beyond Conceptual Confusion,” Cities, Vol. 80, pp. 4-11, 2018. [https://doi.org/10.1016/j.cities.2017.08.021]
• P. Foroudi, S. Gupta, P. Kitchen, M. M. Foroudi, and B. Nguyen, “A Framework of Place Branding, Place Image, and Place Reputation: Antecedents and Moderators,” Qualitative Market Research, Vol. 19, No. 2, pp. 241-264, 2016. [https://doi.org/10.21158/01208160.n89.1010.2838]
• S. Verma, R. Sharma, S. Deb, and D. Maitra, “Artificial Intelligence in Marketing: Systematic Review and Future Research Direction,” International Journal of Information Management Insights, Vol. 1, pp. 1-9, 2021. [https://doi.org/10.1016/j.jjimei.2020.100002]
• D. L. Hoffman, C. P. Moreau, S. Stremersch, and M. Wedel, “The Rise of New Technologies in Marketing: A Framework and Outlook,” Journal of Marketing, Vol. 86, No. 1, pp. 1-6, 2022. [https://doi.org/10.1177/00222429211061636]
• W. J. Scarborough and R. Crabbe, “Place Brands across U.S. Cities and Growth in Local High-Technology Sectors,” Journal of Business Research, Vol. 130, pp. 70-85, June 2021.
• Quartz. Seoul Will Be the First City Government to Join the Metaverse [Internet]. https://qz.com/2086353/seoul-is-developing-a-metaverse-government-platform
• D. Kulasooriya, M. Khoo, M. Tan, and L. Kianchong, The Metaverse in Asia: Strategies for Accelerating Economic Impact, Deloitte, 2023.
• F. Maier and M. Weinberger, “Metaverse Meets Smart Cities – Applications, Benefits, and Challenges,” Future Internet, Vol. 16, No. 4, 126, 2024.
• M. Tukur, J. Schneider, M. Househ, A. H. Dokoro, U. I. Ismail, M. Dawaki, and M. Agus, “The Metaverse Digital Environments: A Scoping Review of the Techniques, Technologies, and Applications,” Journal of King Saud University – Computer and Information Sciences, Vol. 36, No. 2, 101967, February 2024. [https://doi.org/10.1016/j.jksuci.2024.101967]
• L. Geraghty, T. Lee, J. Glickman, and B. Rainwater, The Future of Cities: Cities and the Metaverse, National League of Cities, 2022.
• A. T. Kusuma and S. H. Supangkat, “Metaverse Fundamental Technologies for Smart City: A Literature Review,” in Proceedings of 2022 International Conference on ICT for Smart Society, Bandung, Indonesia, August 2022.
• Z. Chen, W. Gan, J. Wu, H. Lin, and C. Chen, “Metaverse for Smart Cities: A Survey,” Internet of Things and Cyber-Physical Systems, Vol. 4, pp. 203-216, 2024. [https://doi.org/10.1016/j.iotcps.2023.12.002]
• Seoul Metropolitan Government. Seoul, First Local Government to Start New-Concept Public Service with “Metaverse Platform,” [Internet]. https://english.seoul.go.kr/seoul-first-local-govt-to-start-new-concept-public-service-with-metaveres-platform/
• CNBC. Shanghai Doubles Down on the Metaverse by Including It in a Development Plan [Internet]. https://www.cnbc.com/2021/12/31/shanghai-releases-five-year-plans-for-metaverse-development.html
• Shanghai Administration of Foreign Experts Affairs. Shanghai Emerges as a Leader in Metaverse Development [Internet]. https://stcsm.sh.gov.cn/english/News/20231127/
• Forkast. Shanghai Targets US $6.9 Billion Metaverse Revenue from Culture, Tourism Projects [Internet]. https://forkast.news/shanghai-culture-tourism-metaverse/
• T. O’Donovan, “Virtual Smart Cities Using Metaverse, Digital Twins,” Business Transformation: The Change to Futuristic Business, July 2022.
• E. Thomson, “Urban Transformation: These 3 Cities Already Have Their Own Metaverse,” World Economic Forum, Geneva, Switzerland, November 2023.
• J. Liang, S. Fan, M. Jiang, X. Zhang, and Z. Qi, “How to Promote Consumption in City Metaverse? Research on XR Experience Design and Consumer Behaviors of Commercial Streets,” in Proceedings of International Association of Societies of Design Research Congress 2023, Milano, Italy, October 2023.
• Z. Lv, W. Shang, and M. Guizani, “Impact of Digital Twins and Metaverse on Cities: History, Current Situation, and Application Perspectives,” Applied Sciences, Vol. 12, No. 24, 12820, 2022.
• M. Marzouk and A. Othman, “Planning Utility Infrastructure Requirements for Smart Cities Using the Integration between BIM and GIS,” Sustainable Cities and Society, Vol. 57, 102120, June 2020. [https://doi.org/10.1016/j.scs.2020.102120]
• I. Chatzopoulou, P. Tsoutsa, and P. Fitsilis, “How Metaverse Is Affecting Smart Cities Economy,” in Proceedings of the 27th Pan-Hellenic Conference on Progress in Computing and Informatics, Lamia, Greece, pp. 254-259, November 2023.
• X. Cheng, S. Zhang, S. Fu, W. Liu, C. Guan, J. Mou, Q. Ye, and C. Huang, “Exploring the Metaverse in the Digital Economy: An Overview and Research Framework,” Journal of Electronic Business & Digital Economics, Vol. 1, No. 1/2, pp. 206-224, 2022. [https://doi.org/10.1108/JEBDE-09-2022-0036]
• A. Khalil, U. Saher, and A. Haqdad, “Prospects and Challenges of Educational Metaverse in Higher Education,” Journal of Positive School Psychology, Vol. 7, No. 4, pp. 1648-1663, 2023.
• D. Owens, A. Mitchell, D. Khazanchi, and I. Zigurs, “An Empirical Investigation of Virtual World Projects and Metaverse Technology Capabilities,” The Database for Advances in Information Systems, Vol. 42, No. 1, pp. 74-101, February 2011. [https://doi.org/10.1145/1952712.1952717]
• M.-J. Park and W. J. Chon, “Avatar Customization Experience of Generation Z Metaverse Users,” Journal of Digital Contents Society, Vol. 25, No. 4, pp. 895-907, April 2024. [https://doi.org/10.9728/dcs.2024.25.4.895]
• M. Gleim, H. McCullough, O. C. Ferrell, and C. Gabler, “Metaverse: Shifting the Reality of Services,” Journal of Services Marketing, Vol. 38, No. 1, pp. 13-27, 2024. [https://doi.org/10.1108/JSM-01-2023-0021]
• D. Buhalis, D. Leung, and M. Lin, “Metaverse as a Disruptive Technology Revolutionising Tourism Management and Marketing,” Tourism Management, Vol. 97, 104724, August 2023. [https://doi.org/10.1016/j.tourman.2023.104724]
• S. S. Thakur, S. Bandyopadhyay, and D. Datta, “The Metaverse as a Virtual Form of Smart Cities: Opportunities and Challenges,” International Journal of Computer Applications, Vol. 185, No. 17, pp. 45-53, June 2023. [https://doi.org/10.5120/ijca2023922892]
• J. Ajzen, Attitude Structure and Behavior, In A. R. Pratkanis, S. J. Breckler, and A. G. Greenwald, Eds., Attitude Structure and Function, Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 241-274, 1989.
• S. Zygiaris, Z. Hameed, M. A. Alsubaie, and S. U. Rehman, “Service Quality and Customer Satisfaction in the Post Pandemic World: A Study of Saudi Auto Case Industry,” Frontiers in Psychology, Vol. 13, 842141, March 2022. [https://doi.org/10.3389/fpsyg.2022.842141]
• Y. Choi and Y. S. Kim, “A Study on Satisfaction with Virtual Object Manipulation in Metaverse Based on Mixed Reality,” Journal of Positive School Psychology, Vol. 6, No. 4, pp. 10282-10291, 2022.
• D. Choi, H. K. Lee, and D. Y. Kim, “Mood Management through Metaverse Enhancing Life Satisfaction,” International Journal of Consumer Studies, Vol. 47, No. 4, pp. 1533-1543, July 2023. [https://doi.org/10.1111/ijcs.12934]
• Korea Research International. Homepage [Interent]. https://kr.co.kr/en/outline.html
• D. Lee and W. Kang, “Study on Cutting-Edge Technology and Metaverse Success Factors,” Journal of Digital Contents Society, Vol. 25, No. 6, pp. 1545-1555, June 2024. [https://doi.org/10.9728/dcs.2024.25.6.1545]
• S.-J. Moon, “Effects on the Unified Theory of Acceptance and Use of Technology (UTAUT)-based Metaverse Platform Satisfaction and Intention to Use,” Journal of Digital Contents Society, Vol. 25, No. 7, pp. 1715-1725, July 2024. [https://doi.org/10.9728/dcs.2024.25.7.1715]
• I. Arpaci, K. Karatas, I. Kusci, and M. Al-Emran, “Understanding the Social Sustainability of the Metaverse by Integrating UTAUT2 and Big Five Personality Traits: A Hybrid SEM-ANN Approach,” Technology in Society, Vol. 71, 102120, November 2022. [https://doi.org/10.1016/j.techsoc.2022.102120]
• H.-M. Cho and C.-W. Kim, “Implications for Using the Metaverse by Applying the UTAUT and Personality,” Journal of Digital Contents Society, Vol. 25, No. 3, pp. 637-648, March 2024. [https://doi.org/10.9728/dcs.2024.25.3.637]