An AI Educational Agent for Medical Machine Learning Courses

NCT07449182 | Recruiting

• 2 other identifiers: L2025SYSU-HL-03225XJ0215
• Study Type: interventional
• Target: 56
• Locations: 1 country
• Sites: 1

Brief Summary

The goal of this interventional study is to evaluate the effectiveness of a Large Language Model (LLM)-based educational AI Agent in graduate students (Masters and PhD) specializing in medicine or nursing who are enrolled in the "Machine Learning and Data Mining" course. The main questions it aims to answer are: Does the use of an educational AI Agent improve students' academic performance and practical skills in machine learning compared to traditional methods? Does the AI intervention enhance students' learning confidence, satisfaction, and cognitive engagement? Researchers will compare students currently using the AI Agent (experimental group) to a historical control group (students from the previous cohort who did not use the AI tool) to see if the AI-assisted learning model leads to significantly higher learning achievements and better educational experiences. Participants will: Utilize the Teaching Agent for real-time answers to theoretical questions, personalized study planning, and knowledge reinforcement. Engage with the Research Agent to assist with literature reviews, research design optimization, and academic writing structure. Use the Practice Innovation Agent for guidance on coding, algorithm debugging, and applying machine learning models to medical data analysis projects.

Conditions

Medical Education; Artificial Intelligence in Medicine

Keywords

Educational Agent; Machine Learning; KGRAG; Large Language Models

Outcome Measures

Primary Outcomes (1)

• Composite Academic Performance Score

  Assessed through the final cumulative course grade (range: 0-100), which indicates the student's overall mastery of machine learning concepts and applications. The score is calculated based on three weighted components: In-class Assignments (20%): Evaluations of regular assignments submitted via the course platform. Research Progress Paper (40%): A written paper on a free-exploration topic assessing theoretical understanding and research design skills. Group Final Project Presentation (40%): Assessment of a practical project where students present solutions and results based on given medical cases and datasets. Higher scores indicate better academic performance. The experimental group's scores will be compared with the historical control group

  After the intervention (at the end of the course, approximately week 3)

Secondary Outcomes (5)

• Objective Knowledge Acquisition Rate

  After the intervention (at the end of the course, approximately week 3)

• Perceived Usefulness and Technology Acceptance

  After the intervention (at the end of the course, approximately week 3)

• AI Agent Engagement: Interaction Frequency

  At the end of the course (approximately Week 3)

• AI Agent Engagement: Temporal Patterns

  At the end of the course (approximately Week 3)

• AI Agent Engagement: Query Themes

  At the end of the course (approximately Week 3)

Study Arms (1)

AI Agent Intervention Group

EXPERIMENTAL

Graduate students enrolled in the "Machine Learning and Data Mining" course during the 2025-2026 academic year. Participants in this group will utilize the custom-developed KGRAG-based AI Educational Agent system throughout the semester. The system includes three modules: a Teaching Agent for concept explanation, a Research Agent for academic writing support, and a Practice Innovation Agent for code generation and debugging

Other: KGRAG-based AI Educational Agent System

Interventions

KGRAG-based AI Educational Agent System OTHER

The intervention involves a custom-developed AI educational system powered by Large Language Models (LLMs) and Knowledge Graph-based Retrieval-Augmented Generation (KGRAG) technology. The system comprises three specialized agents to support self-directed learning: 1. Teaching Agent: Provides real-time concept explanations, personalized study plans, and knowledge reinforcement based on the course curriculum. 2. Research Agent: Assists with literature review, research question refinement, and academic writing structure. 3. Practice Innovation Agent: Guides students through code generation, algorithm debugging, and data mining projects using Socratic tutoring methods to foster problem-solving skills. Participants have 24/7 access to this system throughout the semester.

AI Agent Intervention Group

Eligibility Criteria

• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Medical graduate students from universities in the Guangdong-Hong Kong-Macao Greater Bay Area;
• Graduate students who have taken the "Machine Learning and Data Mining" course;
• Have completed the required prerequisite courses: "Medical Statistics" and "Nursing Research";
• Capable of operating the AI Educational Agent system normally and willing to undergo relevant teaching interventions and assessments during the study period.

You may not qualify if:

• Unwilling to use the AI education agent system, or refusing to allow the research team to collect their relevant data;
• Students who cannot commit to the full duration of the course or have known scheduling conflicts that would prevent regular attendance;
• Students who have previously enrolled in or audited this course in prior academic years to avoid learning effect bias

Sponsors & Collaborators

• Sun Yat-sen University: lead

Study Sites (1)

North Campus of Sun Yat-sen University

Guangzhou, Guangdong, 510000, China

RECRUITING

Central Study Contacts

Wei Xia, PhD

CONTACT

8618823359471; xiaw23@mail.sysu.edu.cn

Jiebing Luo

CONTACT

8618885639072; luojiebing2002@163.com

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: NA
• Masking: NONE
• Purpose: OTHER
• Intervention Model: SINGLE GROUP
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Associate Professor

Model Details: This is a non-randomized study utilizing a historical control design. The current cohort of students serves as the single experimental group receiving the AI Agent intervention. Their outcomes will be compared to a historical control group (students from the previous academic year) who received standard instruction. Propensity Score Matching (PSM) will be used to control for baseline differences between the two non-concurrent cohorts.

Study Record Dates

• First Submitted: February 27, 2026
• First Posted: March 4, 2026
• Study Start: May 1, 2025
• Primary Completion: March 31, 2026
• Study Completion: March 31, 2026
• Last Updated: March 5, 2026

Record last verified: 2026-02

Data Sharing

• IPD Sharing: Will share
• Shared Documents: STUDY PROTOCOL, SAP, ANALYTIC CODE
• Time Frame: After the publication of the study
• Access Criteria: The researchers can access the data by contacting the PI at xiaw23@mail.sysu.edu.cn with the research purpose described.

Locations

CN (1)