-
Journal of Medical Internet Research Jun 2024Due to recent advances in artificial intelligence (AI), language model applications can generate logical text output that is difficult to distinguish from human writing.... (Comparative Study)
Comparative Study
Assessing the Reproducibility of the Structured Abstracts Generated by ChatGPT and Bard Compared to Human-Written Abstracts in the Field of Spine Surgery: Comparative Analysis.
BACKGROUND
Due to recent advances in artificial intelligence (AI), language model applications can generate logical text output that is difficult to distinguish from human writing. ChatGPT (OpenAI) and Bard (subsequently rebranded as "Gemini"; Google AI) were developed using distinct approaches, but little has been studied about the difference in their capability to generate the abstract. The use of AI to write scientific abstracts in the field of spine surgery is the center of much debate and controversy.
OBJECTIVE
The objective of this study is to assess the reproducibility of the structured abstracts generated by ChatGPT and Bard compared to human-written abstracts in the field of spine surgery.
METHODS
In total, 60 abstracts dealing with spine sections were randomly selected from 7 reputable journals and used as ChatGPT and Bard input statements to generate abstracts based on supplied paper titles. A total of 174 abstracts, divided into human-written abstracts, ChatGPT-generated abstracts, and Bard-generated abstracts, were evaluated for compliance with the structured format of journal guidelines and consistency of content. The likelihood of plagiarism and AI output was assessed using the iThenticate and ZeroGPT programs, respectively. A total of 8 reviewers in the spinal field evaluated 30 randomly extracted abstracts to determine whether they were produced by AI or human authors.
RESULTS
The proportion of abstracts that met journal formatting guidelines was greater among ChatGPT abstracts (34/60, 56.6%) compared with those generated by Bard (6/54, 11.1%; P<.001). However, a higher proportion of Bard abstracts (49/54, 90.7%) had word counts that met journal guidelines compared with ChatGPT abstracts (30/60, 50%; P<.001). The similarity index was significantly lower among ChatGPT-generated abstracts (20.7%) compared with Bard-generated abstracts (32.1%; P<.001). The AI-detection program predicted that 21.7% (13/60) of the human group, 63.3% (38/60) of the ChatGPT group, and 87% (47/54) of the Bard group were possibly generated by AI, with an area under the curve value of 0.863 (P<.001). The mean detection rate by human reviewers was 53.8% (SD 11.2%), achieving a sensitivity of 56.3% and a specificity of 48.4%. A total of 56.3% (63/112) of the actual human-written abstracts and 55.9% (62/128) of AI-generated abstracts were recognized as human-written and AI-generated by human reviewers, respectively.
CONCLUSIONS
Both ChatGPT and Bard can be used to help write abstracts, but most AI-generated abstracts are currently considered unethical due to high plagiarism and AI-detection rates. ChatGPT-generated abstracts appear to be superior to Bard-generated abstracts in meeting journal formatting guidelines. Because humans are unable to accurately distinguish abstracts written by humans from those produced by AI programs, it is crucial to exercise special caution and examine the ethical boundaries of using AI programs, including ChatGPT and Bard.
Topics: Humans; Spine; Abstracting and Indexing; Reproducibility of Results; Artificial Intelligence; Writing
PubMed: 38924787
DOI: 10.2196/52001 -
Headache Jun 2024To apply the 2019 joint American Academy of Neurology (AAN) and American Headache Society (AHS) quality measures for headache management to a cohort of neurology...
OBJECTIVE
To apply the 2019 joint American Academy of Neurology (AAN) and American Headache Society (AHS) quality measures for headache management to a cohort of neurology resident physicians and then assess outcomes related to guideline adherence.
BACKGROUND
The optimization of headache management is essential to reduce both the individual and systemic impact of these disorders. In 2014, the AAN developed 10 quality measures for evidence-based management of patients with headache. A workgroup updated and condensed its headache quality measures in 2019, narrowing the set to six measurements, four of which would primarily focus on the management of migraine and two of which would address the management of cluster headache.
METHODS
This quality improvement study was conducted using a pretest-posttest study design. A pre-intervention survey based on retrospective analysis of five clinic notes for adherence to the measures was designed and distributed to all neurology residents (n = 32) at a large, academic tertiary referral center. The intervention included the creation of an electronic medical record template to aid residents in following the measures during clinical encounters, as well as the provision of direct feedback based on pre-intervention results. Finally, a post-intervention survey was distributed for completion based on notes written during the intervention period. Analysis was limited to migraine, given the low percentage of cluster headache seen in clinic.
RESULTS
An increase in adherence was seen in three of the four migraine-related quality measures, with the Use of Abortive Medications for Migraine and Documentation of Counseling on Modifiable Lifestyle and Chronification Factors demonstrating statistically significant improvements (75.8% to 88.0% [p = 0.013] and 83.9% to 94.0% [p = 0.029] adherence, respectively). For secondary outcomes, the increase in the utilization of appropriate diagnostic criteria (82.6% to 93.2%, p = 0.018) was significant, and the self-assessed confidence rating for adherence to guidelines was significant (p < 0.001).
CONCLUSIONS
This study provides evidence that the quality improvement intervention led to increased adherence to the AAN and AHS migraine-related measures. It is anticipated that increased adherence may lead to improved patient outcomes.
PubMed: 38923523
DOI: 10.1111/head.14754 -
The International Journal of Oral &... Jun 2024Data collection is a fundamental process in any scientific investigation. This article outlines best practices for three general elements of the data-collection process:...
Data collection is a fundamental process in any scientific investigation. This article outlines best practices for three general elements of the data-collection process: (1) determining the specific aim, (2) design, and (3) documentation. We discuss these elements in the style of a tutorial, using extended examples specific to dental research. Each section of the tutorial concludes with a bullet-point summary for ease of reference to the readers. The supplemental material for this article includes templates designed to assist dental researchers in initiating the data-collection process in their respective research efforts, and selected references are organized by topic. Although written for an audience of clinical scientists in dentistry, the principles outlined here could be generalized to other health science research contexts.
Topics: Humans; Dental Research; Research Design; Data Collection; Documentation
PubMed: 38905115
DOI: 10.11607/jomi.10953 -
Journal of Advanced Nursing Jun 2024To identify the safety incident reporting systems and processes used within care homes to capture staff reports of safety incidents, and the types and characteristics of... (Review)
Review
AIMS
To identify the safety incident reporting systems and processes used within care homes to capture staff reports of safety incidents, and the types and characteristics of safety incidents captured by safety incident reporting systems.
DESIGN
Systematic review following PRISMA reporting guidelines.
METHODS
Databases were searched January 2023 for studies published after year 2000, written in English, focus on care homes and incident reporting systems. Data were extracted using a bespoke data extraction tool, and quality was assessed. Data were analysed descriptively and using narrative synthesis, with types and characteristics of incidents analysed using the International Classification for Patient Safety.
DATA SOURCES
Databases were CINAHL, MEDLINE, PsycINFO, EMBASE, HMIC, ASSISA, Nursing and Allied Health Database, MedNar and OpenGrey.
RESULTS
We identified 8150 papers with 106 studies eligible for inclusion, all conducted in high-income countries. Numerous incident reporting processes and systems were identified. Using modalities, typical incident reporting systems captured all types of incidents via electronic computerized reporting, with reports made by nursing staff and captured information about patient demographics, the incident and post-incident actions, whilst some reporting systems included medication- and falls-specific information. Reports were most often used to summarize data and identify trends. Incidents categories most often were patient behaviour, clinical process/procedure, documentation, medication/intravenous fluids and falls. Various contributing and mitigating factors and actions to reduce risk were identified. The most reported action to reduce risk was to improve safety culture. Individual outcomes were often reported, but social/economic impact of incidents and organizational outcomes were rarely reported.
CONCLUSIONS
This review has demonstrated a complex picture of incident reporting in care homes with evidence limited to high-income countries, highlighting a significant knowledge gap. The findings emphasize the central role of nursing staff in reporting safety incidents and the lack of standardized reporting systems and processes.
IMPLICATIONS FOR THE PROFESSION AND/OR PATIENT CARE
The findings from this study can inform the development or adaptation of safety incident reporting systems in care home settings, which is of relevance for nurses, care home managers, commissioners and regulators. This can help to improve patient care by identifying common safety issues across various types of care home and inform learning responses, which require further research.
IMPACT
This study addresses a gap in the literature on the systems and processes used to report safety incidents in care homes across many countries, and provides a comprehensive overview of safety issues identified via incident reporting.
REPORTING METHOD
PRISMA.
PATIENT OR PUBLIC CONTRIBUTION
A member of the research team is a patient and public representative, involved from study conception.
PubMed: 38895931
DOI: 10.1111/jan.16264 -
BMC Nursing Jun 2024A pressure injury refers to localized damage to the skin and/or tissue due to prolonged pressure, and it has recently been defined to include pressure injuries related...
BACKGROUND
A pressure injury refers to localized damage to the skin and/or tissue due to prolonged pressure, and it has recently been defined to include pressure injuries related to medical devices. Medical device-related pressure injuries occur in various sites and are difficult to detect. Even if it is detected, medical devices are essential to life for critically ill patients. Thus, it is difficult to remove or change the position of the medical device; therefore, prevention is essential. This study aims to integrate the literature on medical device-related pressure injury prevention protocols among critically ill patients.
METHODS
The literature inclusion criteria were (1) critically ill patients, (2) device-related pressure injury interventions, (3) randomized controlled trials and quasi-experimental designs, and (4) written in Korean or English. The literature search and selection were performed following the Cochrane Handbook for Systematic Reviews of Interventions with the support of the PRISMA Guidelines.
RESULTS
Twelve articles were finally selected. The incidence of medical device-related pressure injury decreased from 8.1-96.7% before intervention to 0.3-53.3% after intervention, respectively. Medical device-related pressure injury prevention was effective in reducing medical device-related pressure injury incidence when applied to patients of all ages, from neonates to adults, in a variety of intensive care units. Medical device-related pressure injury prevention strategies include nurse education, assessment, documentation, and interventions (hygiene, repositioning, emergent therapy such as protective dressing or designed equipment reducing pressure) of pressure injury. Pressure injury dressings primarily included hydrocolloid foam dressings, but transparent hydrocolloid formulations also effectively reduced medical device-related pressure injury incidence rates.
CONCLUSIONS
In the future, it is necessary to increase the level of evidence by applying specialized medical device-related pressure injury prevention methods for different medical devices and areas of pressure injuries, and verifying their effectiveness.
TRIAL REGISTRATION
The review protocol was registered (PROSPERO registration number: CRD42022346450).
PubMed: 38886734
DOI: 10.1186/s12912-024-02080-y -
International Journal of Computer... Jun 2024Surgical documentation has many implications. However, its primary function is to transfer information about surgical procedures to other medical professionals. Thereby,...
INTRODUCTION
Surgical documentation has many implications. However, its primary function is to transfer information about surgical procedures to other medical professionals. Thereby, written reports describing procedures in detail are the current standard, impeding comprehensive understanding of patient-individual life-spanning surgical course, especially if surgeries are performed at a timely distance and in diverse facilities. Therefore, we developed a novel model-based approach for documentation of visceral surgeries, denoted as 'Surgical Documentation Markup-Modeling' (SDM-M).
MATERIAL AND METHODS
For scientific evaluation, we developed a web-based prototype software allowing for creating hierarchical anatomical models that can be modified by individual surgery-related markup information. Thus, a patient's cumulated 'surgical load' can be displayed on a timeline deploying interactive anatomical 3D models. To evaluate the possible impact on daily clinical routine, we performed an evaluation study with 24 surgeons and advanced medical students, elaborating on simulated complex surgical cases, once with classic written reports and once with our prototypical SDM-M software.
RESULTS
Leveraging SDM-M in an experimental environment reduced the time needed for elaborating simulated complex surgical cases from 354 ± 85 s with the classic approach to 277 ± 128 s. (p = 0.00109) The perceived task load measured by the Raw NASA-TLX was reduced significantly (p = 0.00003) with decreased mental (p = 0.00004) and physical (p = 0.01403) demand. Also, time demand (p = 0.00041), performance (p = 0.00161), effort (p = 0.00024), and frustration (p = 0.00031) were improved significantly.
DISCUSSION
Model-based approaches for life-spanning surgical documentation could improve the daily clinical elaboration and understanding of complex cases in visceral surgery. Besides reduced workload and time sparing, even a more structured assessment of individual surgical cases could foster improved planning of further surgeries, information transfer, and even scientific evaluation, considering the cumulative 'surgical load.'
CONCLUSION
Life-spanning model-based documentation of visceral surgical cases could significantly improve surgery and workload.
PubMed: 38884892
DOI: 10.1007/s11548-024-03214-y -
Vascular and Endovascular Surgery Jun 2024Smoking is an important modifiable risk factor in all vascular diseases and verbal advice from providers has been shown to increase rates of tobacco cessation. We sought...
OBJECTIVES
Smoking is an important modifiable risk factor in all vascular diseases and verbal advice from providers has been shown to increase rates of tobacco cessation. We sought to identify factors that will improve tobacco cessation and recall of receiving verbal cessation advice in vascular surgery patients at a single institution.
METHODS
The study is a retrospective cohort study. Patients seen in outpatient vascular surgery clinic who triggered a tobacco Best Practice Advisory (BPA) during their office visits over a 10-month period were contacted post-clinic and administered surveys detailing smoking status, cessation advice recall, and validated scales for nicotine dependence and willingness to quit smoking. This BPA is a "hard stop" that requires providers to document actions taken. Charts were reviewed for tobacco cessation documentation. Nine-digit zip-codes identified the area deprivation index, a measure of socioeconomic status. Univariate analysis was used to identify factors associated with cessation and advice recall.
RESULTS
One hundred out of 318 (31.4%) patients responded to the survey. Epic Slicer Dicer found 97 BPA responses. To dismiss the BPA, 89 providers (91.8%) selected "advised tobacco cessation" and "Unable to Advise" otherwise. Of the 318 patients, 115 (36.1%) had cessation intervention documented in their provider notes and 151 (47.5%) received written tobacco cessation advice. Of survey respondents, 70 recalled receiving verbal advice, 27 recalled receiving written advice, 28 reported receiving offers of medication/therapy for cessation. 55 patients reported having tobacco cessation plans, and among those 17 reported having quit tobacco. Recall of receiving written advice (P < .001) and recall of receiving medication/therapy (P = .008) were associated with recall of receiving verbal cessation advice.
CONCLUSIONS
Providing patients with tobacco cessation medication/therapy and written tobacco cessation education during office visits is associated with increased patients' recall of tobacco cessation advice. Vascular surgeons should continue to provide directed tobacco cessation advice.
PubMed: 38877384
DOI: 10.1177/15385744241259224 -
Scientific Reports Jun 2024Fair allocation of funding in multi-centre clinical studies is challenging. Models commonly used in Germany - the case fees ("fixed-rate model", FRM) and up-front...
Fair allocation of funding in multi-centre clinical studies is challenging. Models commonly used in Germany - the case fees ("fixed-rate model", FRM) and up-front staffing and consumables ("up-front allocation model", UFAM) lack transparency and fail to suitably accommodate variations in centre performance. We developed a performance-based reimbursement model (PBRM) with automated calculation of conducted activities and applied it to the cohorts of the National Pandemic Cohort Network (NAPKON) within the Network of University Medicine (NUM). The study protocol activities, which were derived from data management systems, underwent validation through standardized quality checks by multiple stakeholders. The PBRM output (first funding period) was compared among centres and cohorts, and the cost-efficiency of the models was evaluated. Cases per centre varied from one to 164. The mean case reimbursement differed among the cohorts (1173.21€ [95% CI 645.68-1700.73] to 3863.43€ [95% CI 1468.89-6257.96]) and centres and mostly fell short of the expected amount. Model comparisons revealed higher cost-efficiency of the PBRM compared to FRM and UFAM, especially for low recruitment outliers. In conclusion, we have developed a reimbursement model that is transparent, accurate, and flexible. In multi-centre collaborations where heterogeneity between centres is expected, a PBRM could be used as a model to address performance discrepancies.Trial registration: https://clinicaltrials.gov/ct2/show/NCT04768998 ; https://clinicaltrials.gov/ct2/show/NCT04747366 ; https://clinicaltrials.gov/ct2/show/NCT04679584 .
Topics: Humans; Germany; Cost-Benefit Analysis; Reimbursement Mechanisms; Cohort Studies; COVID-19
PubMed: 38871878
DOI: 10.1038/s41598-024-63945-5 -
Scientific Reports Jun 2024National consensus recommendations have recently been developed to standardize colorectal tumour localization and documentation during colonoscopy. In this qualitative...
National consensus recommendations have recently been developed to standardize colorectal tumour localization and documentation during colonoscopy. In this qualitative semi-structured interview study, we identified and contrast the perceived barriers and facilitators to using these new recommendations according to gastroenterologists and surgeons in a large central Canadian city. Interviews were analyzed according to the Consolidated Framework for Implementation Research (CFIR) through directed content analysis. Solutions were categorized using the Expert Recommendations for Implementing Change (ERIC) framework. Eleven gastroenterologists and ten surgeons participated. Both specialty groups felt that the new recommendations were clearly written, adequately addressed current care practice tensions, and offered a relative advantage versus existing practices. The new recommendations appeared appropriately complex, applicable to most participants, and could be trialed and adapted prior to full implementation. Major barriers included a lack of relevant external or internal organizational incentives, non-existing formal feedback processes, and a lack of individual familiarity with the evidence behind some recommendations. With application of the ERIC framework, common barriers could be addressed through accessing new funding, altering incentive structures, changing record systems, educational interventions, identifying champions, promoting adaptability, and employing audit/feedback processes. Future research is needed to test strategies for feasibility and effectiveness.
Topics: Humans; Colorectal Neoplasms; Surgeons; Colonoscopy; Gastroenterologists; Canada; Male; Female; Attitude of Health Personnel; Practice Guidelines as Topic; Middle Aged
PubMed: 38849393
DOI: 10.1038/s41598-024-63753-x -
Language, Speech, and Hearing Services... Jun 2024There are well-established guidelines for the recording, transcription, and analysis of spontaneous oral language samples by researchers, educators, and speech...
PURPOSE
There are well-established guidelines for the recording, transcription, and analysis of spontaneous oral language samples by researchers, educators, and speech pathologists. In contrast, there is presently no consensus regarding methods for the written documentation of sign language samples. The Handshape Analysis Recording Tool (HART) is an innovative method for documenting and analyzing word level samples of signed languages in real time. Fluent sign language users can document the expressive sign productions of children to gather data on sign use and accuracy.
METHOD
The HART was developed to document children's productions in Australian Sign Language (Auslan) in a bilingual-bicultural educational program for the Deaf in Australia. This written method was piloted with a group of fluent signing Deaf educational staff in 2014-2016, then used in 2022-2023 with a group of fluent signing professionals to examine inter- and intrarater reliability when coding parameters of sign accuracy.
RESULTS
Interrater reliability measured by Gwet's Agreement Coefficient, was "good" to "very good" across the four phonological parameters that are components of every sign: location, movement, handshape, and orientation.
CONCLUSIONS
The findings of this study indicate that the HART can be a reliable tool for coding the accuracy of location, orientation, movement, and handshape parameters of Auslan phonology when used by professionals fluent in Auslan. The HART can be utilized with any sign language to gather word level sign language samples in a written form and document the phonological accuracy of signed productions.
PubMed: 38843410
DOI: 10.1044/2024_LSHSS-23-00189