Table 3 Objectives and key findings of the included systematic reviews and systematic reviews with meta-analysis (n = 17)

[10]

Patient care and medical research

To explore the capacity of ChatGPT in enhancing patient care and its contribution to medical research and medical writing

ChatGPT can help with handling patient inquiries, documenting notes, making decisions, enrolling in trials, managing data, providing decision support, assisting with research, and educating patients

The provided solutions often fall short and present conflicting information, leading to concerns about their originality, privacy, accuracy, bias, and legality. Content generated by ChatGPT raises issues related to bias and potential plagiarism

Longitudinal studies on the role of ChatGPT in healthcare and medical education and research as well as comparison of ChatGPT with other AI tools

N/A

[27]

Support of ChatGPT in conducting a medical systematic review

To evaluate human researchers' performance versus ChatGPT in systematic review tasks (inter- and intrarater reliability, sensitivity, specificity, accuracy, precision, chance hit rate)

A substantial level of agreement between ChatGPT and human researchers in extracting information for systematic review except for reporting study design, clinical task, and clinical implementation

The limitations of ChatGPT render augmented systematic reviews inefficient for experienced researchers. Ethical implications of using ChatGPT in medical scientific writing

Range of application of ChatGPT and other transformer based models in healthcare should be increased

N/A

[25]

Healthcare education, practice and research

To explore the effectiveness of ChatGPT in healthcare Education, research, and practice, while also emphasizing its potential drawbacks

Research: Review and writing, data analysis, code generation, saving Time for tasks requiring human intelligence

Practice: drug discovery and development, workflow optimization, cost reduction, documentation, personalized medicine, health literacy

Education: personalized learning, critical thinking and problem-based learning

Ethical concerns, copyright issues, transparency challenges, and legal considerations, along with the potential for bias, Plagiarism, lack of originality, inaccurate content leading to hallucination, limited knowledge, incorrect citations, cybersecurity risks, and the threat of infodemics

Potential areas to be explored include the application of AI in reviewing and editing tasks (for journals) and incorporation of Emotional support in patient care involving the use of ChatGPT. Role of ChatGPT in refining communication skills is also an area of importance

N/A

[18]

Treatment decisions (for professionals and public)

To explore the usage and pitfalls of ChatGPT in healthcare

The question-and-answer (QA) design of ChatGPT's interface facilitates integration into the current clinical workflow

Unsuitable for direct clinical deployment as it is not designed for clinical applications

Future development of ‘ChatGPT medical professional version’, specific according to medical specialities

N/A

[36]

Healthcare practice

To investigate the various applications, potential benefits and risks, ethical issues of ChatGPT in healthcare and recommendations for its adoption in medicine and cellular imaging

1. Disseminates critical information about pandemics and infectious diseases

2. virtual assistant for orthognathic surgery consultations

3. addresses inquiries related to dental practice

4. simplifies complex medical concepts

5. differential diagnoses based on patient history

6. analysis of

medical images and cellular imaging

7. expedited research processes through quick literature review

1.Inability to understand specialized medical terminology

2. Can augment but not replace human judgment in clinical settings

3. In case of inaccurate clinical advice, accountability (legal and ethical regulatory frameworks) have not yet been devised

4. Balancing automation with the human touch

Work should be done for dynamic and real time learning of ChatGPT for obtaining updated information in medicine and health care

N/A

[31]

Radiology

Evaluating the current applications and future

directions of ChatGPT in the field of radiology

Demonstrates substantial potential to augment decision-making and optimizing workflow

Concerns regarding radiologic image processing, ethical and legal implications, potential self-diagnosis and self-management by patients

The image processing potential of ChatGPT 4.0 needs to be validated further

N/A

[2]

Medical and dental research

To evaluate the reliability and accuracy of ChatGPT in medical and dental research

Meta-analysis shows that the accuracy of

ChatGPT in providing correct responses was significantly higher compared to the total responses to queries related to medical examination, systematic reviews, clinical reasoning, diagnostic imaging, liver diseases, and COVID-19

vaccination

Transparency, ethical concern, erroneous content, variation in population, data reproducibility

The studies on ChatGPT have small data points tested (small sample size) which poses threat to extrapolation of findings. Therefore, larger datasets (including concepts, queries, prompts etc.) should be tested using ChatGPT to improve the generalizability of the study results

The meta-analysis showed an odds ratio (OR) of 2.25 and a relative risk (RR) of 1.47

with a 95% confidence interval (CI), indicating that the accuracy of ChatGPT in providing correct responses was significantly higher compared to the total responses for queries

[3]

Neurosurgery

To examine the potential benefits and limitations of ChatGPT in neurosurgical practice and education

Personalized treatment plans, supporting surgical planning and navigation, and enhancing large data processing efficiency and accuracy

Question format limitations, validation challenges, and algorithmic bias along with ethical issues related to its usage

Incorporating longitudinal patient data into predictive models can enhance outcome prediction

N/A

[17]

Medical examinations

To assess the performance in medical examinations with

multiple-choice questions

ChatGPT correctly answered the majority of multiple-choice questions in medical examinations with a passing grade

Preparations for medical examinations using ChatGPT should be done with caution

Development of training data set specific for medical education

Exploring the role of Future AI chatbots in medical examination preparation

Overall performance of ChatGPT ranged from 40% in the biomedical admission test to 100% in a diabetes knowledge questionnaire. The mean performance of ChatGPT was 61.1% (95% CI 56.1%–66.0%

[20]

Medicine and healthcare

To explore the

potential of ChatGPT in medicine

Streamline and simplify

complex tasks, improve patient care, enhance clinical decision making,

and facilitate communication among healthcare professionals

Privacy, ethical considerations, tokenization, sensitivity of wording of prompt, lack of capacity to handle image based questions

Involvement of medical practitioners in training ChatGPT

N/A

[15]

Gastroenterology

To assess the applications, benefits and limitations of ChatGPT in the field of gastroenterology

It can provide recommendations, enhance communication between patients and caregivers, and prompt valuable research inquiries

Obstacles in decoding intricate medical questions, yielded

inconsistent responses at times, and exhibited limitations in generating novel content

The model of ChatGPT used in research influences its outcomes, so a comparable version should be used

Research on prompt selection for ChatGPT is lacking which can drastically change the outcome/ response

N/A

[32]

Public health dentistry

To find out applications and drawbacks of ChatGPT in public dental health schooling, writing for academic use, research and clinical practice in public dental health

Helps scholars with the authoring of scientific research and

dental studies

Scientists can focus and allocate more time on experimentation by delegating some tasks to ChatGPT

Prejudice in the training data, undervaluing human skills, possible fraud, legal and reproducibility concerns

Inclusion of more homogenous studies in terms of quality in order to improve the generalizability of systematic review findings

N/A

[22]

Neurosurgery

To explore the potential benefits and limitations of ChatGPT in the field of neurosurgery

Accuracy and efficiency of neurosurgical procedures, as well as diagnosis, treatment, and patient outcomes

1. Need for large datasets

2. Potential for errors in the output

To provide extensive database necessary to train ChatGPT without breaching patient confidentiality

N/A

[4]

Radiology

To conduct a qualitative and quantitative analysis of ChatGPT literature in radiology, assessing its scope and impact

Enhances patient education, protocol selection, and differential diagnosis generation

Improves radiology report structuring and examination preparation

1. Inconsistency in performance and information accuracy

2. Challenges in fully integrating AI into clinical radiology workflows

Research on factually incorrect information/ hallucinations generated from ChatGPT is needed

N/A

[5]

Disease risk detection, personalized healthcare procedures, and enhancing patient care safety and quality

To explore the efficacy of ChatGPT and generative AI tools in medical diagnostics, treatment recommendations, and improving healthcare practices

1. Enhances diagnostic accuracy and surgical planning

2. Reduces administrative tasks, improving physician efficiency

3. Supports evidence-based decision-making and clinical education

1. Potential inaccuracies in clinical letter generation

2. Requires regulation and careful integration into clinical workflows

3. Risk of misinterpreting treatment guidelines affecting patient care

Predictive analytics to observe how well can ChatGPT assess the real-time data streams

N/A

[35]

Ophthalmology

To evaluate the effectiveness and potential of ChatGPT in improving ophthalmological care, specifically in diagnosis, patient interaction, and educational roles

1. Offers rapid, accessible information and support for clinical decisions

2. Enhances patient education through simplified explanations of conditions and treatments

1. Potential inaccuracies in medical advice or diagnostic information

2. Ethical concerns around patient data privacy and the reliability of AI-generated advice

Explore the effectiveness of ChatGPT in a diverse linguistic landscape

N/A

[24]

Medical research

To evaluate ChatGPT's application and effectiveness in medical research, including treatment, diagnosis, medication provision, and more

Offers potential benefits in drug development, medical report improvement, providing treatment and medical information, literature review writing, research conduction, data analysis, and personalized medicine

Concerns about ChatGPT's accuracy, originality, academic integrity, and ethical issues like privacy and security in medical research

Overcome issues pertaining to academic integrity, privacy, and ethics

N/A