Fellow University of Texas at Austin Dell Medical School Austin, Texas, United States
Background: The rapid evolution of Artificial Intelligence (AI) has recently catalyzed breakthrough advancements across various sectors. ChatGPT, a publicly accessible AI model by OpenAI, stands as a prime example. Amassing over 100 million users since its launch in November 2022, it has emerged as the fastest growing consumer software app in history. In the realm of medical writing, ChatGPT’s capabilities hold immense potential- from synthesizing complex research data, to aiding in the drafting of medical documents and ensuring effective patient communication. Objective: This research aims to evaluate the effectiveness of ChatGPT in assisting medical writing, with a focus on its proficiency in crafting titles and abstracts for pediatric emergency medicine articles and comparing its output to that of human authors. Design/Methods: Ten pediatric emergency medicine articles published in 2022 were selected from 5 leading emergency medicine journals. One retrospective and one prospective randomized study were chosen from each journal. The main content of these articles, excluding their titles and abstracts, was input into ChatGPT-4. Using a tailored prompt, ChatGPT was asked to formulate an original title and abstract for each article. Two blinded expert reviewers will score both the original human-generated and ChatGPT-derived titles and abstracts using a standardized rubric. Each abstract section (Introduction, Methods, Results, Conclusion) and the Title, along with the overall writing quality, will be rated on a 0-5 scale. Scores will be analyzed to discern potential differences in the quality of title and abstract generation between humans and Chat-GPT, utilizing a t-test or non-parametric equivalent.
As of now, our expert reviewers are assessing the titles and abstracts. Data collection is expected to conclude by December 2023 with analysis shortly afterwards. . The IRB has categorized this study as “Not Human Research”, and has stated that further review and approval is not required.
