Analysis of Generative AI Chatbot in Improving Public Services: A Case Study on Program Indonesia Pintar (PIP) for Primary and Secondary Education
DOI:
https://doi.org/10.51278/aj.v8i2.2469Keywords:
Chatbot, Generative AI, Public Service, Program Indonesia PintarAbstract
The Smart Indonesia Program (PIP), as a supporter of educational access, faces challenges in disseminating information that still relies on conventional methods such as email and telephone, thus requiring the implementation of Generative AI-based chatbot technology as a potential solution to improve the effectiveness of communication and public service delivery. This study aims to analyze the influence of the readiness, design, and implementation of Generative AI chatbots on the effectiveness of information delivery to PIP service users. This study uses a descriptive quantitative approach by distributing questionnaires to 407 respondents. The instrument was tested for validity and reliability. Regression testing and t-tests were conducted to assess the influence between variables. The results show that the design variable (X₂) has a strong effect on service effectiveness, with a t-value of 14.757 (significance level of 0.000). The regression coefficient of 0.683 indicates that each unit increase in chatbot design quality contributes significantly to an increase in perceived service effectiveness. The implementation variable (X₃) also has a significant effect with a t-value of 7.489 and a regression coefficient of 0.217. The design and implementation variables influence service effectiveness with a model contribution of 77.8%. However, the readiness variable (X₁) did not show a significant effect (p = 0.077). The R-squared value of 0.778 indicates that 77.8% of the variation in public service effectiveness can be explained by readiness, design, and implementation variables. The remaining 22.2% is influenced by other factors, such as external policies, user digital literacy, or organizational issues. These findings provide a strong foundation for further development of AI technology in the digital transformation of public services, particularly in the education sector.
References
Abdenebaoui, L., Aljuneidi, S., Horstmannshoff, F., Meyer, J., & Boll, S. (2025). Value-Driven Design for Public Administration: Insights from a Generative Chatbot in a Housing Application Case Study. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, 1554–1564. https://doi.org/10.1145/3715275.3732103
Akpan, I. J., Kobara, Y. M., Owolabi, J., Akpan, A. A., & Offodile, O. F. (2025). Conversational and generative artificial intelligence and human–chatbot interaction in education and research. International Transactions in Operational Research, 32(3), 1251–1281. https://doi.org/10.1111/itor.13522
Androniceanu, A. (2024). Generative artificial intelligence, present and perspectives in public administration.«(RAMP). Dalam Revista» Administratie si Management Public (43 ed.).
Annepaka, Y., & Pakray, P. (2025). Large language models: A survey of their development, capabilities, and applications. Knowledge and Information Systems, 67(3), 2967–3022. https://doi.org/10.1007/s10115-024-02310-4
Berry, S., Trochmann, M. B., & Millesen, J. L. (2024). Putting the Humanity Back Into Public Human Resources Management: A Narrative Inquiry Analysis of Public Service in the Time of COVID-19. Review of Public Personnel Administration, 44(1), 8–31. https://doi.org/10.1177/0734371X211069656
Bonett, D. G., & Wright, T. A. (2015). Cronbach’s alpha reliability: Interval estimation, hypothesis testing, and sample size planning. Journal of Organizational Behavior, 36(1), 3–15. https://doi.org/10.1002/job.1960
Branda, F., Stella, M., Ceccarelli, C., Cabitza, F., Ceccarelli, G., Maruotti, A., Ciccozzi, M., & Scarpa, F. (2025). The Role of AI-Based Chatbots in Public Health Emergencies: A Narrative Review. Future Internet, 17(4), 145. https://doi.org/10.3390/fi17040145
Chandra, Y. W., & Suyanto, S. (2019). Indonesian Chatbot of University Admission Using a Question Answering System Based on Sequence-to-Sequence Model. Procedia Computer Science, 157, 367–374. https://doi.org/10.1016/j.procs.2019.08.179
Chaves, A. P., & Gerosa, M. A. (2019). How should my chatbot interact? A survey on human-chatbot interaction design. https://doi.org/10.48550/ARXIV.1904.02743
Chen, T., ascó-Hernandez, M, & Esteve, M. (2024). The adoption and implementation of artificial intelligence chatbots in public organizations: Evidence from US state governments. The American Review of Public Administration. 54(3). https://doi.org/10.1177/027507402312005
Chopra, D, Prashar, A, & ain, M. (2013). Natural Language Processing (Bag. 42–45). 82(7).
Damar, M., Özen, A., Çakmak, Ü. E., Özoğuz, E., & Erenay, F. S. (2024). Super AI, Generative AI, Narrow AI and Chatbots: An Assessment of Artificial Intelligence Technologies for The Public Sector and Public Administration. Journal of AI, 8(1), 83–106. https://doi.org/10.61969/jai.1512906
Di Giulio, M., & Vecchi, G. (2023). Implementing digitalization in the public sector. Technologies, agency, and governance. Public Policy and Administration, 38(2), 133–158. https://doi.org/10.1177/09520767211023283
Diyoga, I. K., Setyowati, E., Khannanah, S. F., Juniati, L., & Syukron, A. (2026). Evaluation of the Effectiveness of the Smart Indonesia Program in Supporting the Sustainability of Education for Underprivileged Junior High School Students in Malang City. International Journal of Research in Social Science and Humanities, 07(01), 447–453. https://doi.org/10.47505/IJRSS.2026.1.40
Fadilah, S, & Ramdani, I. (2019). Implementasi Kecerdasan Buatan pada Layanan Publik: Studi Kasus di Sektor Pendidikan (Bag. 210-220). 10(3).
Ferrucci, D., Brown, E., Chu‐Carroll, J., Fan, J., Gondek, D., Kalyanpur, A. A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J., Schlaefer, N., & Welty, C. (2010). Building Watson: An Overview of the DeepQA Project. AI Magazine, 31(3), 59–79. https://doi.org/10.1609/aimag.v31i3.2303
Goodfellow, I, Bengio, Y, & Courville, A. (2016). Deep Learning. MIT Press.
Gunawan, A, & Putri, L. (2021). Penggunaan Teknologi Chatbot untuk Meningkatkan Efisiensi Layanan Publik (Bag. 233-242). 15(4).
Hao Qin, & Zhi Li. (2024). A Study on Enhancing Government Efficiency and Public Trust: The Transformative Role of Artificial Intelligence and Large Language Models. https://doi.org/10.5281/ZENODO.12619360
Haroud, S., & Saqri, N. (2025). Generative AI in Higher Education: Teachers’ and Students’ Perspectives on Support, Replacement, and Digital Literacy. Education Sciences, 15(4), 396. https://doi.org/10.3390/educsci15040396
Jarupunphol, P., Ikonnikov, O., Roncevic, I., Kapustina, S., Kataeva, A., Parfjonovs, M., & Tsarev, R. (2025). Applying Cronbach’s Alpha to Ensure Reliable Online Testing in E-Learning Environments. Dalam R. Silhavy & P. Silhavy (Ed.), Artificial Intelligence and System Engineering (Vol. 1490, hlm. 120–139). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-96759-7_8
Kartika, E, & Sembiring, J. (2019). Studi Implementasi Chatbot dalam Meningkatkan Layanan Pemerintah. 8(1), 65–75.
Ma’rup, M., Tobirin, & Ali Rokhman. (2024). Utilization of Artificial Intelligence (AI) Chatbots in Improving Public Services: A Meta-Analysis Study. Open Access Indonesia Journal of Social Sciences, 7(4), 1610–1618. https://doi.org/10.37275/oaijss.v7i4.255
McGrath, C., Farazouli, A., & Cerratto-Pargman, T. (2025). Generative AI chatbots in higher education: A review of an emerging research area. Higher Education, 89(6), 1533–1549. https://doi.org/10.1007/s10734-024-01288-w
Moylan, K., & Doherty, K. (2025). Expert and Interdisciplinary Analysis of AI-Driven Chatbots for Mental Health Support: Mixed Methods Study. Journal of Medical Internet Research, 27, e67114. https://doi.org/10.2196/67114
Ninghardjanti, P., Murtini, W., Hindrayani, A., & Sangka, K. B. (2023). Evaluation of the Smart Indonesia Program as a Policy to Improve Equality in Education. Sustainability, 15(6), 5114. https://doi.org/10.3390/su15065114
Plonsey, R., & Barr, R. C. (Ed.). (2007). Bioelectricity: A Quantitative Approach (Third Edition). Springer Science+Business Media, LLC. https://doi.org/10.1007/978-0-387-48865-3
Rahmawati, H., & Sudrajat, A. (2025). Implementasi Chatbot pada Penerimaan Mahasiswa Baru di Politeknik Tedc Bandung Menggunakan Natural Language Processing. Jurnal Informatika dan Teknik Elektro Terapan, 13(1). https://doi.org/10.23960/jitet.v13i1.5456
Saunders, L. J., Russell, R. A., & Crabb, D. P. (2012). The Coefficient of Determination: What Determines a Useful R2 Statistic? Investigative Opthalmology & Visual Science, 53(11), 6830. https://doi.org/10.1167/iovs.12-10598
Senadheera, S., Yigitcanlar, T., Desouza, K. C., Mossberger, K., Corchado, J., Mehmood, R., Li, R. Y. M., & Cheong, P. H. (2025). Understanding Chatbot Adoption in Local Governments: A Review and Framework. Journal of Urban Technology, 32(3), 35–69. https://doi.org/10.1080/10630732.2023.2297665
Sugiono, S. (2021). Pemanfaatan Chatbot Pada Masa Pandemi Covid-19: Kajian Fenomena Society 5.0 Chatbot Utilization During The Covid-19 Pandemic: Revisiting The Concept Of Society 5.0. 22(2).
Vallejo Blanxart, A., & Nicolas Sans, R. (2025). The role of generative AI chatbots in higher education: A student-centric conceptual analysis of benefits, ethics, and privacy concerns. Journal of Technology and Science Education, 15(3), 810. https://doi.org/10.3926/jotse.3643
Van Velsen, L., Ludden, G., & Grünloh, C. (2022). The Limitations of User-and Human-Centered Design in an eHealth Context and How to Move Beyond Them. Journal of Medical Internet Research, 24(10), e37341. https://doi.org/10.2196/37341
Virvou, M. (2023). Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art. Intelligent Decision Technologies, 17(1), 73–125. https://doi.org/10.3233/IDT-230092
Zhou, T., Liu, Y., Kumar, M., & Wang, S. (2026). Generative AI-enabled chatbots for user-centred design: A state-of-the-art review. Journal of Engineering Design, 1–28. https://doi.org/10.1080/09544828.2026.2633488
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Muhammad Nur, Tanty Oktavia
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
