Feasible but Unproven: A Systematic Review of Generative AI and Agentic Copilots in Administrative Work

Authors

  • Irfan Taufik Universitas Pendidikan Indonesia, Indonesia
  • Rasto Universitas Pendidikan Indonesia, Indonesia
  • Edi Suryadi Universitas Pendidikan Indonesia, Indonesia

DOI:

https://doi.org/10.57235/jetish.v6i1.9421

Keywords:

Agentic AI; Generative AI Copilots; Administrative Work; Human Oversight; Automation Levels; Evidence Synthesis

Abstract

Generative AI copilots and agentic systems are increasingly proposed to reduce administrative work, yet evidence on their effects has not been appraised across sectors. This systematic review asked which administrative tasks and technologies have been studied, which effects on productivity, quality, and errors are reported, which human–AI collaboration mechanisms accompany them, and which risks and governance measures are reported. Following PRISMA 2020 with an unregistered protocol, we searched Scopus on 30 October 2026 for English-language journal articles published in 2022–2026 (314 records), appraised quality with the Mixed Methods Appraisal Tool, and synthesized findings narratively using context–intervention–mechanism–outcome logic. Twenty-nine studies from health care, private organizations, public administration, and higher education were included; none was randomized, and one evaluated an explicitly agentic system. Of 19 rated studies, three were high quality and seven low. The largest reductions in task time (60–90%) were reported only by system-development studies that were not quality-rated, typically single cases or internal tests. Accuracy results were mixed and lowest for medical coding requiring exact matches (ICD-10 concordance 58.1%). The evidence shows feasibility under supervision, not yet organizational benefit; designs that attribute effects and measure work quality, downstream errors, and oversight are needed

Downloads

Download data is not yet available.

References

Acharya, D. B., Kuppan, K., & Divya, B. (2025). Agentic AI: Autonomous intelligence for complex goals—A comprehensive survey. IEEE Access, 13, 18912–18936. https://doi.org/10.1109/ACCESS.2025.3532853

Aiumtrakul, N., Thongprayoon, C., Kookanok, C., Poochanasri, M., Phichedwanichskul, K., & Cheungpasitporn, W. (2026). Quality assessment of large language model–generated prior authorization letters in nephrology. Frontiers in Digital Health, 8, Article 1767648. https://doi.org/10.3389/fdgth.2026.1767648

Alon-Barkat, S., & Busuioc, M. (2023). Human–AI interactions in public sector decision making: "Automation bias" and "selective adherence" to algorithmic advice. Journal of Public Administration Research and Theory, 33(1), 153–169. https://doi.org/10.1093/jopart/muac007

Artsi, Y., Sorin, V., Glicksberg, B. S., Korfiatis, P., Nadkarni, G. N., & Klang, E. (2025). Large language models in real-world clinical workflows: A systematic review of applications and implementation. Frontiers in Digital Health, 7, Article 1659134. https://doi.org/10.3389/fdgth.2025.1659134

Baas, J., Schotten, M., Plume, A. M., Côté, G., & Karimi, R. (2020). Scopus as a curated, high-quality bibliometric data source for academic research in quantitative science studies. Quantitative Science Studies, 1(1), 377–386. https://doi.org/10.1162/qss_a_00019

Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), Article e2422633122. https://doi.org/10.1073/pnas.2422633122

Bedi, S., Liu, Y., Orr-Ewing, L., Dash, D., Koyejo, S., Callahan, A., Fries, J. A., Wornow, M., Swaminathan, A., Lehmann, L. S., Hong, H. J., Kashyap, M., Chaurasia, A. R., Shah, N. R., Singh, K., Tazbaz, T., Milstein, A., Pfeffer, M. A., & Shah, N. H. (2025). Testing and evaluation of health care applications of large language models: A systematic review. JAMA, 333(4), 319–328. https://doi.org/10.1001/jama.2024.21700

Black, K. C., Haberkorn, W. J., Ma, S. P., Kiani, H., Bhasin, A., Chen, J. H., Shah, N. H., & Morse, K. (2026). Uses of generative AI by non-clinician staff at an academic medical center. npj Health Systems, 3(1), Article 13. https://doi.org/10.1038/s44401-025-00063-y

Bramer, W. M., Giustini, D., de Jonge, G. B., Holland, L., & Bekhuis, T. (2016). De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association, 104(3), 240–243. https://doi.org/10.3163/1536-5050.104.3.014

Brynjolfsson, E., Li, D., & Raymond, L. R. (2025). Generative AI at work. The Quarterly Journal of Economics, 140(2), 889–942. https://doi.org/10.1093/qje/qjae044

Busuioc, M. (2021). Accountable artificial intelligence: Holding algorithms to account. Public Administration Review, 81(5), 825–836. https://doi.org/10.1111/puar.13293

Callari, T. C., & Puppione, L. (2025). Can generative artificial intelligence productivity tools support workplace learning? A qualitative study on employee perceptions in a multinational corporation. Journal of Workplace Learning, 37(3), 266–283. https://doi.org/10.1108/JWL-11-2024-0258

Campbell, M., McKenzie, J. E., Sowden, A., Katikireddi, S. V., Brennan, S. E., Ellis, S., Hartmann-Boyce, J., Ryan, R., Shepperd, S., Thomas, J., Welch, V., & Thomson, H. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: Reporting guideline. BMJ, 368, Article l6890. https://doi.org/10.1136/bmj.l6890

Carroll, A., Brown, D., Silvio, D., Maszatics, J., & Hanauer, D. A. (2026). Evaluating large language models for automated REDCap support ticket triage and response. JAMIA Open, 9(3), Article ooag073. https://doi.org/10.1093/jamiaopen/ooag073

Chen, J., Liu, Y., Liu, P., Zhao, Y., Zuo, Y., & Duan, H. (2025). Adoption of large language model AI tools in everyday tasks: Multisite cross-sectional qualitative study of Chinese hospital administrators. Journal of Medical Internet Research, 27, Article e70789. https://doi.org/10.2196/70789

Chen, L., Zaharia, M., & Zou, J. (2024). How is ChatGPT's behavior changing over time? Harvard Data Science Review, 6(2). https://doi.org/10.1162/99608f92.5317da47

Choroszewicz, M. (2026). Positioning public sector practitioners as 'moral crumple zones': Mechanisms in the early use of generative AI work support tools. Government Information Quarterly, 43(2), Article 102128. https://doi.org/10.1016/j.giq.2026.102128

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104

de Almeida, P. G. R., & dos Santos Júnior, C. D. (2025). Artificial intelligence governance: Understanding how public organizations implement it. Government Information Quarterly, 42(1), Article 102003. https://doi.org/10.1016/j.giq.2024.102003

Denyer, D., Tranfield, D., & van Aken, J. E. (2008). Developing design propositions through research synthesis. Organization Studies, 29, 393–413. https://doi.org/10.1177/0170840607088020

Doropoulos, S., Vologiannidis, S., & Magnisalis, I. (2026). DevNous: An LLM-based multi-agent system for grounding IT project management in unstructured conversation. Information and Software Technology, 195, Article 108078. https://doi.org/10.1016/j.infsof.2026.108078

Du, X., Ling, C., Yang, L., & Wei, L. (2026). Factors shaping the adoption of large language models among hospital administrative staff: A cross-sectional survey study. Digital Health, 12. https://doi.org/10.1177/20552076261450815

Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2024). GPTs are GPTs: Labor market impact potential of LLMs. Science, 384(6702), 1306–1308. https://doi.org/10.1126/science.adj0998

Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2022). Cognitive challenges in human–artificial intelligence collaboration: Investigating the path toward productive delegation. Information Systems Research, 33(2), 678–696. https://doi.org/10.1287/isre.2021.1079

Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., Hansen, L., Landman, A., Lehmann, L., McCoy, L., Miller, T., Moreno, A., Munch, N., Restrepo, D., Savova, G., . . . Bitterman, D. (2025). The TRIPOD-LLM reporting guideline for studies using large language models. Nature Medicine, 31(1), 60–69. https://doi.org/10.1038/s41591-024-03425-5

Gerlich, M. (2025). AI tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15(1), Article 6. https://doi.org/10.3390/soc15010006

Goyal, R., Deshmukh, S. G., & Bolia, N. (2026). Navigating barriers to GenAI adoption in public administration: A systematic evaluation and policy roadmap. Socio-Economic Planning Sciences, 105, Article 102461. https://doi.org/10.1016/j.seps.2026.102461

Green, B. (2022). The flaws of policies requiring human oversight of government algorithms. Computer Law & Security Review, 45, Article 105681. https://doi.org/10.1016/j.clsr.2022.105681

Haesevoets, T., Verschuere, B., & Roets, A. (2025). AI adoption in public administration: Perspectives of public sector managers and public sector non-managerial employees. Government Information Quarterly, 42(2), Article 102029. https://doi.org/10.1016/j.giq.2025.102029

Haesevoets, T., Verschuere, B., Van Hiel, A., & Roets, A. (2026). Public servants' attitudes toward the use of artificial intelligence in policymaking: An experimental study. Government Information Quarterly, 43(2), Article 102134. https://doi.org/10.1016/j.giq.2026.102134

Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O'Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P. (2018). The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Education for Information, 34(4), 285–291. https://doi.org/10.3233/EFI-180221

Humlum, A., & Vestergaard, E. (2025). The unequal adoption of ChatGPT exacerbates existing inequalities among workers. Proceedings of the National Academy of Sciences, 122(1), Article e2414972121. https://doi.org/10.1073/pnas.2414972121

Jonathan, G. M. (2026). Public-sector digital transformation in the age of generative AI. Discover Artificial Intelligence, 6(1), Article 968. https://doi.org/10.1007/s44163-026-02077-3

Jo, H. (2024). Subscription intentions for ChatGPT Plus: A look at user satisfaction and self-efficacy. Marketing Intelligence & Planning, 42(6), 1052–1073. https://doi.org/10.1108/MIP-08-2023-0411

Jo, H., & Park, D.-H. (2024). Effects of ChatGPT's AI capabilities and human-like traits on spreading information in work environments. Scientific Reports, 14(1), Article 7806. https://doi.org/10.1038/s41598-024-57977-0

Jo, H., & Park, D.-H. (2025). The fear of being replaced by generative AI: An examination of influential factors among office workers. Technological Forecasting and Social Change, 220, Article 124326. https://doi.org/10.1016/j.techfore.2025.124326

Kalasapudi, V. S., Seelam, A., Ganupa, S., & Tofferi, C. (2026). The overlooked frontier of AI in construction: Conversational, document-native automation for administrative workflows. Frontiers in Built Environment, 11, Article 1713342. https://doi.org/10.3389/fbuil.2025.1713342

Kim, E. (2026). Generative AI in public administration: A quasi-experimental analysis of bureaucratic productivity. Government Information Quarterly, 43(1), Article 102108. https://doi.org/10.1016/j.giq.2026.102108

Kiser, K. J., Waters, M., Reckford, J., Lundeberg, C., & Abraham, C. D. (2024). Large language models to help appeal denied radiotherapy services. JCO Clinical Cancer Informatics, 8, Article e2400129. https://doi.org/10.1200/CCI.24.00129

Klang, E., Sorin, V., Korfiatis, P., Sawant, A. S., Freeman, R., Charney, A. W., Nadkarni, G. N., & Glicksberg, B. S. (2026). Large language models are poor clinical administrators: An evaluation of structured queries in real-world electronic health records. PLOS Digital Health, 5, Article e0001326. https://doi.org/10.1371/journal.pdig.0001326

Kumar, N., Wei, X., & Zhang, H. (2026). Agentic artificial intelligence as a new frontier in information systems: Promise, peril, and research opportunities. Information & Management, 63(3), Article 104317. https://doi.org/10.1016/j.im.2026.104317

Latario, L. D., & Fowler, J. R. (2026). Chatbot demonstrates moderate interrater reliability in billing for hand surgery clinic encounters. Hand, 21(1), 68–72. https://doi.org/10.1177/15589447241295328

Lebovitz, S., Lifshitz-Assaf, H., & Levina, N. (2022). To engage or not to engage with AI for critical judgments: How professionals deal with opacity when using AI for medical diagnosis. Organization Science, 33(1), 126–148. https://doi.org/10.1287/orsc.2021.1549

López-Galisteo, A. J. (2026). Optimizing university administrative services with generative AI: Evidence from email inquiry reduction and assistant performance. Information, 17(6), Article 587. https://doi.org/10.3390/info17060587

Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2025). Hallucination-free? Assessing the reliability of leading AI legal research tools. Journal of Empirical Legal Studies, 22(2), 216–242. https://doi.org/10.1111/jels.12413

Menano de Carvalho, L., Peças, P., & Jorge, D. (2026). AI-enabled process improvement in information-intensive administrative work: Real-case applications of LLMs in a Lean Six Sigma context. Sustainability, 18(10), Article 4787. https://doi.org/10.3390/su18104787

Mitroulias, D., & Sioutas, S. (2026). Empirical evaluation of a domain-specific RAG-based conversational assistant for payroll management: A mixed-methods study of accuracy, satisfaction, and usability. Big Data and Cognitive Computing, 10(6), Article 193. https://doi.org/10.3390/bdcc10060193

Moynihan, D., Herd, P., & Harvey, H. (2015). Administrative burden: Learning, psychological, and compliance costs in citizen-state interactions. Journal of Public Administration Research and Theory, 25(1), 43–69. https://doi.org/10.1093/jopart/muu009

Nai, R., Sulis, E., Audrito, D., Trifiletti, V. M. S., Meo, R., & Genga, L. (2025). Leveraging process mining and event log enrichment in European public procurement analysis: A case study. Computer Law & Security Review, 57, Article 106144. https://doi.org/10.1016/j.clsr.2025.106144

Natali, C., Marconi, L., Dias Duran, L. D., & Cabitza, F. (2025). AI-induced deskilling in medicine: A mixed-method review and research agenda for healthcare and beyond. Artificial Intelligence Review, 58(11), Article 356. https://doi.org/10.1007/s10462-025-11352-1

Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://doi.org/10.1126/science.adh2586

Olson, K. D., Meeker, D., Troup, M., Barker, T. D., Nguyen, V. H., Manders, J. B., Stults, C. D., Jones, V. G., Shah, S. D., Shah, T., & Schwamm, L. H. (2025). Use of ambient AI scribes to reduce administrative burden and professional burnout. JAMA Network Open, 8(10), Article e2534976. https://doi.org/10.1001/jamanetworkopen.2025.34976

Paek, E. S., Fan, T., Finch, J. D., & Choi, J. D. (2024). Enhancing task-oriented dialogue systems through synchronous multi-party interaction and multi-group virtual simulation. Information, 15(9), Article 580. https://doi.org/10.3390/info15090580

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., . . . Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, Article n71. https://doi.org/10.1136/bmj.n71

Parasuraman, R., & Manzey, D. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381–410. https://doi.org/10.1177/0018720810376055

Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354

Porumbescu, G., Young, M., & Yang, K. (2025). Introduction to special issue on generative AI and public administration: Performance, accountability, and democratic governance. Public Performance & Management Review, 48(6), 1263–1267. https://doi.org/10.1080/15309576.2025.2570315

Raisch, S., & Krakowski, S. (2021). Artificial intelligence and management: The automation–augmentation paradox. Academy of Management Review, 46(1), 192–210. https://doi.org/10.5465/amr.2018.0072

Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. B. (2021). PRISMA-S: An extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic Reviews, 10, Article 39. https://doi.org/10.1186/s13643-020-01542-z

Rodriguez Saavedra, M. O., Lopez, J. M. B., Nina, W. Q., Morales Gonzales, A. V., Galindo, I. C., Ascuña, L. M. C., Colana, A. S. S., Almanza Cabe, R. B., Tito, P. G. L., & Teran, S. V. L. (2026). Machine learning and generative AI in administrative processes in Peru: Administrative efficiency in the national public sector. Informatics, 13(3), Article 44. https://doi.org/10.3390/informatics13030044

Ruhwedel, T., Rogasch, J. M. M., Dahlke, P. M., Shnayien, S., Furth, C., Wetz, C., Amthauer, H., Schatka, I., & Beetz, N. L. (2026). Less time coding, more time caring: Performance evaluation of ChatGPT-5 for ICD-10 coding of radiology reports. International Journal of Medical Informatics, 210, Article 106296. https://doi.org/10.1016/j.ijmedinf.2026.106296

Selten, F., & Klievink, B. (2024). Organizing public sector AI adoption: Navigating between separation and integration. Government Information Quarterly, 41(1), Article 101885. https://doi.org/10.1016/j.giq.2023.101885

Soma, R. (2026). End-to-end architecture for LLM-based business process automation: A low-code integration framework with performance benchmarking. IEEE Access, 14, 105921–105940. https://doi.org/10.1109/ACCESS.2026.3696852

Soroush, A., Glicksberg, B. S., Zimlichman, E., Barash, Y., Freeman, R., Charney, A. W., Nadkarni, G. N., & Klang, E. (2024). Large language models are poor medical coders—Benchmarking of medical code querying. NEJM AI, 1(5). https://doi.org/10.1056/AIdbp2300040

Syed, R., Suriadi, S., Adams, M., Bandara, W., Leemans, S. J. J., Ouyang, C., ter Hofstede, A. H. M., van de Weerd, I., Wynn, M. T., & Reijers, H. A. (2020). Robotic process automation: Contemporary themes and challenges. Computers in Industry, 115, Article 103162. https://doi.org/10.1016/j.compind.2019.103162

Tahtali, M. A., Le Blanc, P., Dirne, C., & Snijders, C. (2026). Why public procurement professionals accept (or resist) generative AI: A qualitative study. Journal of Public Procurement, 26(4), 727–761. https://doi.org/10.1108/JOPP-09-2025-0110

Tai-Seale, M., Baxter, S. L., Vaida, F., Walker, A., Sitapati, A. M., Osborne, C., Diaz, J., Desai, N., Webb, S., Polston, G., Helsten, T., Gross, E., Thackaberry, J., Mandvi, A., Lillie, D., Li, S., Gin, G., Achar, S., Hofflich, H., . . . Longhurst, C. A. (2024). AI-generated draft replies integrated into health records and physicians' electronic communication. JAMA Network Open, 7(4), Article e246565. https://doi.org/10.1001/jamanetworkopen.2024.6565

Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8, Article 45. https://doi.org/10.1186/1471-2288-8-45

Tseng, P., Kaplan, R. S., Richman, B. D., Shah, M. A., & Schulman, K. A. (2018). Administrative costs associated with physician billing and insurance-related activities at an academic health care system. JAMA, 319(7), 691–697. https://doi.org/10.1001/jama.2017.19148

Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour, 8(12), 2293–2303. https://doi.org/10.1038/s41562-024-02024-1

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., & Wen, J. (2024). A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6), Article 186345. https://doi.org/10.1007/s11704-024-40231-1

Wang, M. H., Lee, S. M. Y., Hou, G., Wang, Y., Alves, J. C., Xie, R., He, Y., Liu, J., Fang, X., Yang, Y., Cai, X., Zheng, S., Yu, Z., Lin, E. Z., Cheang, C., Ian, K. K., & Qin, S. (2026). Explainable multimodal AI and neuro-symbolic clinical decision support system for chronic eye disease management: A digital health implementation study. Frontiers in Digital Health, 8, Article 1836890. https://doi.org/10.3389/fdgth.2026.1836890

Wilson, L. A., Konsynski, B., & Yisrael, T. (2025). Advancing research administration with AI: A case study from Emory University. Journal of Research Administration, 56(2), 52–74.

Zaki, N., Alneyadi, A., Alamoodi, M., Alshaali, S., Alkaabi, F., & Alhashmi, A. (2026). An AI-enabled framework for reducing administrative bureaucracy in higher education while maintaining governance and compliance. Discover Artificial Intelligence, 6(1), Article 879. https://doi.org/10.1007/s44163-026-01565-w

Zhou, L., Schellaert, W., Martínez-Plumed, F., Moros-Daval, Y., Ferri, C., & Hernández-Orallo, J. (2024). Larger and more instructable language models become less reliable. Nature, 634(8032), 61–68. https://doi.org/10.1038/s41586-024-07930-y

Downloads

Published

2026-11-21

Issue

Section

Articles

Citation Check