Can Artificial Intelligence (AI) Successfully Identify Novel Ideas for Hand Surgery Research?
Naem A Mufarreh, DO1, Michael R McDermott, DO1 and H. Brent Bamberger, DO, FAOAO2, (1)Kettering Health Dayton, Dayton, OH, (2)Orthopedic Associates of Southwest Ohio, Dayton, OH
Introduction: Recently there has been a focus in the medical community on integration of newly advanced technologies into daily practice. One of the recent advancements is ChatGPT released by Open AI. ChatGPT is an open access, AI model that can have conversations, answer questions, and learn from individual users. The aim of this study is to delineate a usage of ChatGPT for the hand surgeon and assess the feasibility of incorporation into an aspect of medical practice. The study below begins this process by exploring ChatGPTâ€™s ability to isolate research ideas that have not been explored, assessing its accuracy for identifying novel research opportunities in the field of hand surgery.
Methods: ChatGPT was accessed and asked to come up with 50 specific and novel systematic review ideas for the field of hand surgery. Each of the suggestion were then subjected to a through independent literature review to assess novelty. The literature search was conducted through PubMed (National institutes of health, Bethesda, MD) and the search criteria consisted of reviewing the 50 best matches, filtering for systematic reviews and publication date within the last 10 years. A two-part criterion was established in order to deem a suggestion as novel. First of which the suggestion must not have any similar publication in the literature review. Secondly, the suggestion cannot propose a question that could be answered by a matched publication in the search. Only suggestions that matched both criteria were deemed novel. Results of individual suggestions were tracked to identify an overall accuracy of ChatGPT for this task.
Results: In response to the proposed question, ChatGPT produced 46 suggestions that the program deemed as novel ideas for systematic reviews in hand surgery. From these suggestions, our methods identified 31 of the 46 suggestions as novel, yielding an overall accuracy of 67.4% for ChatGPT for identifying specific and novel systematic reviews in the field of hand surgery.
Conclusion: ChatGPT was successful at identifying specific and novel ideas for systematic reviews in the field of hand surgery. While not every suggestion was truly novel, ChatGPT was able to recognize 31 novel ideas, showing the utility ChatGPT as an adjunctive tool for expanding hand surgery research. As this technology continues to advance, it is crucial for hand surgeons to consider the use of ChatGPT to help guide their research.
Back to 2024 Abstracts