CClinicalTrials.gg
CompletedNCT05963802AIHSSUpdated Feb 6, 2024

Evaluation of the Efficacy and Usability of Artificial Intelligence (ChatGPT) for Health Sciences Students

An interventional study of Artificial Intelligence (ChatGPT) Group in Educational Activities and AI (Artificial Intelligence), sponsored by Carleton University. Completed at 1 site in Canada. Open to participants aged 18 Years and older, including healthy volunteers. Per ClinicalTrials.gov, last updated 2024-02-06.

Sponsored by Carleton University · Not applicable, Interventional, and Other

Phase
Not applicable
Study type
Interventional
Enrollment
26
Allocation
Randomized
Ages
18 Years and older
Sex
All
01

Study summary

Crossover Randomized Control trial, in which subjects are randomly assigned to one of two groups: one (ChatGPT) receiving the intervention that is being tested, and the control group receiving usual online resources.

Read the detailed description

A two-stage randomization strategy will be adopted. The allocation of participants into event groups A and B will be determined through computer-generated randomization in an allocation ratio: 1:1. To address ethical concerns regarding power dynamics between professors and students; the investigators have opted to involve an individual who is independent of the research team. This decision aims to minimize any potential bias or influence that could arise from direct professor involvement in the intervention. This approach helps ensure that the study is conducted fairly and unbiasedly, fostering a research environment that prioritizes equal opportunities for all participants. Both groups will simultaneously work on an assignment (A and B).

The investigators employed a cross-over design to ensure equity in the learning experience for all students. This involved assigning students either the ChatGPD (A) or the regular online tools (B) for accessing resources and assisting them with their assignments. During the initial phase of the intervention, participants in group A will utilize ChatGPT to complete their assignments, while participants in Group B will serve as the control group. In the subsequent period, the interventions will switch between the two groups. To assess the participants' technology usability, both groups will complete pre-and post-questionnaires. Additionally, participants will be requested to fill out a post-intervention questionnaire to evaluate their perceptions of the assistance methods. After a period of a wash-out of 13 days/week, participants who were previously allocated to Group A will be allocated to Group B and the same with participants previously allocated to Group B will be to Group A. This crossover randomized control trial will include 50 health sciences students who are registered to the course Chronic Health Conditions and Disability ( HLTH 3503), Carleton University. During the first part of the intervention, A group will use ChatGPT to complete their assignment, while B participants will act as a control group. In the second week, the interventions will cross over. Both groups' participants will complete a pre-and post-questionnaire to assess their knowledge. Participants will also complete a post-intervention questionnaire to evaluate their perceptions. The System Usability Scale (SUS) will be administered at the end of the interventions and when participants cross over to the other assistance tool. This allows for the assessment of outcomes and comparisons within and between the two intervention arms. By collecting data at multiple points in the study, the investigators can evaluate the effects of each intervention and any potential carryover effects during the crossover period. Additionally, it provides an opportunity to measure changes in outcomes within individual participants as they switch from one intervention to another. The questionnaire about socioeconomic, literacy questions, Benefits of AI and risks and negative experiences associated with AI usage.

02

Conditions studied

  • Educational Activities
  • AI (Artificial Intelligence)

Keywords

  • Artificial intelligence
  • Education
  • Health Science Students
03

In context

Lead sponsor

Carleton University is the lead sponsor of 3 studies on the registry; 1 is open to participants now.

Counted across the registry records on this site, refreshed daily.

04

Who can participate

Ages eligible
18 Years and older
Sexes eligible
All
Accepts healthy volunteers
Yes

Inclusion criteria

  • Health Science students
  • Must be registered for the course Chronic Health Conditions and Disability (HLTH 3503), Fall 2023.

Exclusion criteria

Exclusion Criteria:

  • Unable to provide informed consent
05

Study design

Phase
Not applicable
Primary purpose
Other
Allocation
Randomized
Intervention model
Crossover assignment
Masking
Single (Investigator)
Enrollment
26 participants (actual)

Study arms

  • No intervention
    Traditional Online tools Group

    Group B will serve as the control group in this study. Participants in this group will receive instructions on how to complete the assignment using traditional online tools available on the internet, without the use of artificial intelligence. They will have a timeframe of six days to complete the assignment using these conventional tools. Similar to participants in Arm 1, they will also be required to fill out a survey on technology usability, providing feedback on their experience with the online tools.

  • Experimental
    Artificial Intelligence (ChatGPT) Group

    Participants in Group A will be assigned to utilize ChatGPT as their tool to complete assignments. They will be given a period of six days to utilize artificial intelligence through ChatGPT for assignment completion. Along with the assignment instructions, participants will receive an ethical guideline and specific guidelines on how to effectively utilize ChatGPT. Once the intervention period is concluded, participants will be given a 24-hour window to complete a survey that assesses the usability of the technology. The survey aims to gather valuable feedback on the participants' experience and perception of using ChatGPT for their assignments. In addition, participants in Arm 1 will also be asked to fill out a survey regarding their perception of using artificial intelligence (AI) as an assistance tool to complete their assignments. This survey aims to gather insights into their thoughts, opinions, and attitudes towards utilizing AI in the learning process

    Other: Artificial Intelligence (ChatGPT) Group

Interventions

  • OtherArtificial Intelligence (ChatGPT) Group

    Participants in Group A will be assigned to utilize ChatGPT as their tool to complete assignments. They will be given a period of six days to utilize artificial intelligence through ChatGPT for assignment completion. Along with the assignment instructions, participants will receive an ethical guideline and specific guidelines on how to effectively utilize ChatGPT. Once the intervention period is concluded, participants will be given a 24-hour window to complete a survey that assesses the usability of the technology. The survey aims to gather valuable feedback on the participants' experience and perception of using ChatGPT for their assignments. In addition, participants in Arm 1 will also be asked to fill out a survey regarding their perception of using artificial intelligence (AI) as an assistance tool to complete their assignments. This survey aims to gather insights into their thoughts, opinions, and attitudes towards utilizing AI in the learning process.

06

What researchers measure

Primary outcomes

  1. System Usability Scale (SUS) scores are 0-100,The average SUS score is 68.

    Technology usability

    Time frame: within 24 hours

Secondary outcomes

  1. Student's perception of artificial intelligence, questionnaire reviwed by experts, No recognised gold standard measurement for outcome.

    Student's perception about the use of artificial intelligence

    Time frame: within 24 hours

07

Study locations

1 site
  • Carleton University
    Ottawa, Canada
08

References and documents

Publications

  • Dwan K, Li T, Altman DG, Elbourne D. CONSORT 2010 statement: extension to randomised crossover trials. BMJ. 2019 Jul 31;366:l4378. doi: 10.1136/bmj.l4378. PubMed 31366597 ↗
  • Luna A, Casertano L, Timmerberg J, O'Neil M, Machowsky J, Leu CS, Lin J, Fang Z, Douglas W, Agrawal S. Artificial intelligence application versus physical therapist for squat evaluation: a randomized controlled trial. Sci Rep. 2021 Sep 13;11(1):18109. doi: 10.1038/s41598-021-97343-y. PubMed 34518568 ↗
  • Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK; SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension. BMJ. 2020 Sep 9;370:m3164. doi: 10.1136/bmj.m3164. PubMed 32909959 ↗
  • Zhou L, Bao J, Setiawan IMA, Saptono A, Parmanto B. The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study. JMIR Mhealth Uhealth. 2019 Apr 11;7(4):e11500. doi: 10.2196/11500. PubMed 30973342 ↗

Individual participant data

Plan to share: No

09

Updates

Tracking since Sep 25, 2026
No changes since tracking began. The registry record was last updated on Feb 6, 2024, before this site started recording changes on Sep 25, 2026. Its history is on ClinicalTrials.gov ↗
10

Registry details

Key details

Study ID
NCT05963802
Lead sponsor
Carleton University
Collaborators
Université de Montréal
Responsible party
Mirella Veras (Principal Investigator, Carleton University) — Principal investigator
First posted
Jul 27, 2023
Start date
Sep 1, 2023
Primary completion
Dec 1, 2023
Completion
Dec 1, 2023
Last update
Feb 6, 2024

Study contacts

Mirella Veras, PhD
principal investigator · Carleton University

Oversight

Data monitoring committee
Yes
FDA-regulated drug
No
FDA-regulated device
No
View the source record on ClinicalTrials.gov ↗

Not currently enrolling

This study is completed, as verified in Feb 2024. You cannot join it, but the record below documents what was studied.

Follow this study

Get an email when the registry record changes — status, dates, results — or when someone posts here.

Sign in to follow

Discussion

Questions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.

Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.

Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.

Start the discussion