-
Journal of Medical Internet Research May 2024Clinical natural language processing (NLP) researchers need access to directly comparable evaluation results for applications such as text deidentification across a...
BACKGROUND
Clinical natural language processing (NLP) researchers need access to directly comparable evaluation results for applications such as text deidentification across a range of corpus types and the means to easily test new systems or corpora within the same framework. Current systems, reported metrics, and the personally identifiable information (PII) categories evaluated are not easily comparable.
OBJECTIVE
This study presents an open-source and extensible end-to-end framework for comparing clinical NLP system performance across corpora even when the annotation categories do not align.
METHODS
As a use case for this framework, we use 6 off-the-shelf text deidentification systems (ie, CliniDeID, deid from PhysioNet, MITRE Identity Scrubber Toolkit [MIST], NeuroNER, National Library of Medicine [NLM] Scrubber, and Philter) across 3 standard clinical text corpora for the task (2 of which are publicly available) and 1 private corpus (all in English), with annotation categories that are not directly analogous. The framework is built on shell scripts that can be extended to include new systems, corpora, and performance metrics. We present this open tool, multiple means for aligning PII categories during evaluation, and our initial timing and performance metric findings. Code for running this framework with all settings needed to run all pairs are available via Codeberg and GitHub.
RESULTS
From this case study, we found large differences in processing speed between systems. The fastest system (ie, MIST) processed an average of 24.57 (SD 26.23) notes per second, while the slowest (ie, CliniDeID) processed an average of 1.00 notes per second. No system uniformly outperformed the others at identifying PII across corpora and categories. Instead, a rich tapestry of performance trade-offs emerged for PII categories. CliniDeID and Philter prioritize recall over precision (with an average recall 6.9 and 11.2 points higher, respectively, for partially matching spans of text matching any PII category), while the other 4 systems consistently have higher precision (with MIST's precision scoring 20.2 points higher, NLM Scrubber scoring 4.4 points higher, NeuroNER scoring 7.2 points higher, and deid scoring 17.1 points higher). The macroaverage recall across corpora for identifying names, one of the more sensitive PII categories, included deid (48.8%) and MIST (66.9%) at the low end and NeuroNER (84.1%), NLM Scrubber (88.1%), and CliniDeID (95.9%) at the high end. A variety of metrics across categories and corpora are reported with a wider variety (eg, F-score) available via the tool.
CONCLUSIONS
NLP systems in general and deidentification systems and corpora in our use case tend to be evaluated in stand-alone research articles that only include a limited set of comparators. We hold that a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons. Our open pipeline should reduce barriers to evaluation and system advancement.
Topics: Natural Language Processing
PubMed: 38805692
DOI: 10.2196/55676 -
Frontiers in Public Health 2024Diabetes education is an integral part of the treatment for the metabolic control of patients with diabetes. The use of the Internet as a tool for diabetes education, as...
BACKGROUND
Diabetes education is an integral part of the treatment for the metabolic control of patients with diabetes. The use of the Internet as a tool for diabetes education, as well as its acceptance, is still under study.
AIM
To assess the usability of the educational website "I understand my diabetes" designed for patients with type 2 diabetes attending primary care clinics.
MATERIAL AND METHOD
A cross-sectional study was done in 110 patients with type 2 diabetes from two family medicine clinics, each of whom was assigned a user account on the educational website "Entiendo mi diabetes." The web site assigned a user name and password to each patient. They were able to access the educational website at home. After a 15-day review period, participants were asked to evaluate usability using the Computer System Usability Questionnaire. Additionally, we developed an eight-item questionnaire usability focusing on diabetes care. Sociodemographic data, blood pressure, and anthropometric measurements were recorded. Glucose levels and lipid profiles were also measured.
RESULTS
The patients with diabetes had a mean age of 52.7 years and a median of 5 years since they were diagnosed with diabetes. The website received a good usability rating from 89.1% of participants, with favorable assessments in all three dimensions: 87.3% for information, 85.5% for quality, and 88.2% for interface. Regarding usability specifically for diabetes care, 98.2% rated it as having good usability.
CONCLUSION
The website for education about the disease in patients "I understand my diabetes" had an adequate usability evaluation by patients, so they also considered it very useful for diabetes care. The diabetes care instrument had adequate usability and reliability.
Topics: Humans; Diabetes Mellitus, Type 2; Middle Aged; Mexico; Female; Male; Cross-Sectional Studies; Patient Education as Topic; Internet; Surveys and Questionnaires; Adult; Aged
PubMed: 38799692
DOI: 10.3389/fpubh.2024.1394066 -
Preventive Medicine Reports Jul 2024To identify and support correction of misspelled medication names recorded as free text, we compared the relative effectiveness of two user-friendly methods, used...
OBJECTIVE
To identify and support correction of misspelled medication names recorded as free text, we compared the relative effectiveness of two user-friendly methods, used without reliance on clinical knowledge.
METHODS
Leveraging the SAS® COMPGED function, fuzzy string search programs examined 1.8 million medication records from 183,600 World Trade Center General Responder Cohort monitoring visits conducted in New York and New Jersey between 7/16/2002 and 3/31/2021, producing replicable generalized edit distance scores between the reported and correct spelling. Scores < 120 were selected as optimal and compared to Stedman's 2020 Plus Medical/Pharmaceutical Spell Checker first suggested word, used as the comparative standard because it employs both spelling and phonetic similarities to suggest matching words. We coded each methods' results as identifying or not identifying the medications within each visit.
RESULTS
Most types of medications (94.4 % anxiety, 98.4 % asthma and 94.6 % ulcer/gastroesophageal reflux disease) were correctly spelled. Cross tabulations assessed the agreement (anxiety 99.9 %, asthma 99.6 % and 98.4 % ulcer/ gastroesophageal reflux disease), false positive (respectively 0.02 %, 0.03 % and 2.0 %) and false negative (respectively 1.9 %, 0.5 % and 1.0 %) values. Scores < 120 occasionally correctly identified medications missed by the spell checker. We observed no difference in medication misspellings across socio-economically and culturally diverse patient characteristics.
CONCLUSIONS
Both methods efficiently identified most misspelled medications, greatly minimizing the review and rectification needed. The fuzzy method is more universally applicable for condition-specific medications identification, but requires more programming skills. The spell checker is inexpensive, but benefits from modest programming skills and is only available in some languages.
PubMed: 38798907
DOI: 10.1016/j.pmedr.2024.102765 -
Experimental Gerontology Aug 2024To investigate the cognitive function and nutritional status of elderly patients with gastric cancer during perioperative period, and to analyze their correlation.
OBJECTIVE
To investigate the cognitive function and nutritional status of elderly patients with gastric cancer during perioperative period, and to analyze their correlation.
METHODS
Aged patients undergoing gastric cancer surgery in The Affiliated Cancer Hospital of Shandong First Medical University from March to October 2021 were selected as the subjects of this study. The monitoring data of cognitive function and nutritional status were retrospectively analyzed from 1 to 3 days before surgery, 1 and 3 days after surgery, 7 days after surgery (before discharge) and 30 days after surgery to analyze the correlation between cognitive function and nutritional status in elderly patients with gastric cancer.
RESULTS
the incidence of mild cognitive impairment in elderly patients with gastric cancer was 52.43 %, the visual space of the two groups' (mild cognitive impairment) ability of execution, name, attention, language, abstract thinking, delayed memory and cognitive function scores were lower than 1 set of directional force (cognitive function in normal group), statistically significant difference (P < 0.05). The nutritional status of elderly patients with gastric cancer was lower than that of healthy elderly group at the same period (P < 0.05). The scores of visual spatial executive function, name, attention, delayed memory, orientation and total score of cognitive function in elderly gastric cancer patients were positively correlated with nutritional status (P < 0.05).
CONCLUSIONS
The cognitive function and nutritional status of elderly patients with gastric cancer are both in a low state during treatment and a higher level of cognitive function can help patients maintain a more correct nutritional cognition, and the nutritional status of patients will be relatively better. There is a positive correlation between cognitive function and nutritional status in elderly patients with gastric cancer, which should be paid attention to in the treatment.
Topics: Humans; Stomach Neoplasms; Nutritional Status; Female; Male; Aged; Cognition; Retrospective Studies; Cognitive Dysfunction; Perioperative Period; Aged, 80 and over; Middle Aged
PubMed: 38797287
DOI: 10.1016/j.exger.2024.112467 -
BMC Oral Health May 2024This randomized controlled trial compared the efficacy and tolerability of danzhixiaoyao pills in the accurate treatment of patients with burning mouth syndrome (BMS). (Randomized Controlled Trial)
Randomized Controlled Trial
OBJECTIVES
This randomized controlled trial compared the efficacy and tolerability of danzhixiaoyao pills in the accurate treatment of patients with burning mouth syndrome (BMS).
METHOD
Collect a total of 78 patients (75 female patients and 3 male patients) from the oral mucosa department who were considered eligible fromOctober 2020 to October 2022.The patients were randomized and divided into trial group and control group.The trail group received danzhixiaoyao pills and mecobalamine tablets while the control group was given mecobalamine tablets.The Visual Analogue Scale (VAS), Beck Anxiety Inventory(BAI), Beck Depression Inventory (BDI), Oral Health Impact Profile (OHIP-14), Traditional Chinese medicine(TCM) syndrome integral and adverse reactions were performed at baseline and after 2, 4, and 6 weeks of treatment. Descriptive statistics, including the Wilcoxon rank-sum test and the Chi-square test for median comparisons between different times, were used.
RESULT
1.After treatment, the VAS, BDI,OHIP-14, and TCM syndrome integral in the trial group had a significant decrease than the control group(P< 0.05).However, there was no statistical difference in the BAI scores between the two groups (P> 0.05). 2.According to the efficacy determination criteria , the total effective rate of the test group was 73.68% , the control group was 52.94% and the recurrence rate was 0. There was a significant difference between the two groups (Z=-2.688, P < 0.05). The results showed that the curative effect of test group was better than that of control group.3. No adverse effects occurred in patients in either group.
CONCLUSION
Danzhixiaoyao pills has demonstrated to have a positive effect in relieving BMS symptoms and in improving a patient's overall quality of life with no AEs compared with the control group. The efficacy evaluation systems that can be verified and complementary in this study provide a perfect, effective and referential evaluation system for the use of Chinese patent medicine in the treatment of oral mucosal diseases.
TRIAL REGISTRATION
Registry name: Chinese Clinical trail Registry Registration number: ChiCTR2000038189 Date of Registration: 2020-09-13 Please visit ( https://www.chictr.org.cn/showproj.html?proj=61462 ) to the protocol.
Topics: Humans; Burning Mouth Syndrome; Male; Female; Vitamin B 12; Middle Aged; Drugs, Chinese Herbal; Tablets; Treatment Outcome; Aged; Drug Therapy, Combination; Adult
PubMed: 38789997
DOI: 10.1186/s12903-024-04318-2 -
Healthcare (Basel, Switzerland) May 2024Anticoagulation therapy plays a crucial role in the management of atrial fibrillation (AF) by significantly reducing the risk of stroke. Direct oral anticoagulants...
Role of a National Health Service Electronic Prescriptions Database in the Detection of Prescribing and Dispensing Issues and Adherence Evaluation of Direct Oral Anticoagulants.
BACKGROUND
Anticoagulation therapy plays a crucial role in the management of atrial fibrillation (AF) by significantly reducing the risk of stroke. Direct oral anticoagulants (DOAC) became preferred over warfarin due to their superior safety and efficacy profile. Assessing adherence to anticoagulation therapy is necessary in clinical practice for optimising patient outcomes and treatment efficacy, thus emphasising its significance.
METHODS
A retrospective study utilised the Latvian National Health Service reimbursement prescriptions database, covering prescriptions for AF and flutter from January 2012 to December 2022. The proportion of days covered method was selected for adherence assessment, categorising it into three groups: (1) below 80%, (2) between 80% and 90%, and (3) above 90%.
RESULTS
A total of 1,646,648 prescriptions were analysed. Dabigatran prescriptions started declining after 2020, coinciding with a decrease in warfarin prescriptions since 2018. The total adherence levels to DOAC therapy were 69.4%. Only 44.2% of users achieved an adherence level exceeding 80%. The rate of paper prescriptions decreased from 98.5% in 2017 to 1.3% in 2022. Additionally, the utilisation of international non-proprietary names reached 79.7% in 2022. Specifically, 16.7% of patients selected a single pharmacy, whereas 27.7% visited one or two pharmacies. Meanwhile, other patients obtained medicines from multiple pharmacies.
CONCLUSIONS
The total adherence level to DOAC therapy is evaluated as low and there was no significant difference in age, gender, or "switcher" status among adherence groups. Physicians' prescribing habits have changed over a decade.
PubMed: 38786385
DOI: 10.3390/healthcare12100975 -
BMC Primary Care May 2024Community pharmacies are responsible for dispensing of medicines and related counselling in outpatient care. Dispensing practices have remarkably changed over time, but...
BACKGROUND
Community pharmacies are responsible for dispensing of medicines and related counselling in outpatient care. Dispensing practices have remarkably changed over time, but little is known about how the changes have influenced medication safety. This national study investigated trends in dispensing errors (DEs) related to prescribed medicines, which were reported in Finnish community pharmacies within a 6-year period.
METHODS
This national retrospective register study included all DEs reported to a nationally coordinated voluntary DE reporting system by Finnish community pharmacies during 2015-2020. DE rates, DE types, prescription types, individuals who detected DEs and contributing factors to DEs were quantified as frequencies and percentages. Poisson regression was used to assess the statistical significance of the changes in annual DE rates by type.
RESULTS
During the study period, altogether 19 550 DEs were reported, and the annual number of error reports showed a decreasing trend (n = 3 913 in 2015 vs. n = 2 117 in 2020, RR 0.54, p < 0.001). The greatest decrease in reported DEs occurred in 2019 after the national implementation of the Medicines Verification System (MVS) and the additional safety feature integrated into the MVS process. The most common error type was wrong dispensed strength (50% of all DEs), followed by wrong quantity or pack size (13%). The annual number of almost all DE types decreased, of which wrong strength errors decreased the most (n = 2121 in 2015 vs. n = 926 in 2020). Throughout the study period, DEs were most commonly detected by patients (50% of all DEs) and pharmacy personnel (30%). The most reported contributing factors were factors related to employees (36% of all DEs), similar packaging (26%) and similar names (21%) of medicinal products.
CONCLUSIONS
An overall decreasing trend was identified in the reported DEs and almost all DE types. These changes seem to be associated with digitalisation and new technologies implemented in the dispensing process in Finnish community pharmacies, particularly, the implementation of the MVS and the safety feature integrated into the MVS process. The role of patients and pharmacy personnel in detecting DEs has remained central regardless of changes in dispensing practices.
Topics: Finland; Humans; Medication Errors; Retrospective Studies; Registries; Community Pharmacy Services; Male; Female; Middle Aged; Pharmacies; Adult; Aged; Adolescent
PubMed: 38783197
DOI: 10.1186/s12875-024-02428-y -
BMC Nephrology May 2024The Low Physical Activity Questionnaire (LoPAQ) was specifically developed to measure the low activity level observed in extremely inactive hemodialysis (HD) patients....
BACKGROUND
The Low Physical Activity Questionnaire (LoPAQ) was specifically developed to measure the low activity level observed in extremely inactive hemodialysis (HD) patients. This study aims to evaluate reliability and validity of Persian version of the LoPAQ.
METHODS
This study was a cross sectional study, conducted in three HD centers in Iran. The LoPAQ was translated into Persian. After cultural adaptions, it was filled out by 120 HD patiens. Convergent validity, was evaluated by calculating the correlations among the Persian version of the LoPAQ and Persian version of the Community Healthy Adults Model Program for Seniors (CHAMPS) questionnaire, physical function scale of the SF-36 and physical function (Short Physical Performance Battery (SPPB) test) using Spearman's correlation coefficients. The test-retest reliability was analyzed using the intraclass correlation coefficient (ICC).
RESULTS
In total, 109 patients completed all of the questionnaires, took part in physical performance tests and had valid data. Their mean age was 64 ± 11 years, with a dialysis history of 31 ± 10 months. For total calories, there was a strong correlation between the Persian version of the LoPAQ and CHAMPS-measured physical activity (rho = 0.85, p < 0.001). In addition, the higher physical activity level reported by Persian version of the LoPAQ was also correlated with better self-reported physical function (rho = 0.7, p < 0.001) and better physical performance (rho = 0.67, p < 0.001). The ICC ranged from 0.65 to 0.78, indicating strong reliability.
CONCLUSION
The assessment of the validity and reliability of the Persian version of the questionnaire confirmed its suitability for evaluating the level of physical activity.
TRIAL REGISTRATION
ClinicalTrials.gov Identifier: NCT05930964, Registered on 05/07/2023. Registered trial name: Validity and Reliability of Persian Version of Low Physical Activity Questionnaire (LoPAQ).
Topics: Aged; Female; Humans; Male; Middle Aged; Cross-Sectional Studies; Exercise; Iran; Renal Dialysis; Reproducibility of Results; Sedentary Behavior; Surveys and Questionnaires; Translations
PubMed: 38778292
DOI: 10.1186/s12882-024-03615-w -
Scientific Reports May 2024Clinical application of cardiac magnetic resonance (CMR) is expanding but CMR assessment of LV diastolic function is still being validated. The purpose of this study was... (Clinical Trial)
Clinical Trial
Clinical application of cardiac magnetic resonance (CMR) is expanding but CMR assessment of LV diastolic function is still being validated. The purpose of this study was to validate assessments of left ventricular (LV) diastolic dysfunction (DD) using CMR by comparing with transthoracic echocardiography (TTE) performed on the same day. Patients with suspected or diagnosed cardiomyopathy (n = 63) and healthy volunteers (n = 24) were prospectively recruited and included in the study. CMR diastolic parameters were measured on cine images and velocity-encoded phase contrast cine images and compared with corresponding parameters measured on TTE. A contextual correlation feature tracking method was developed to calculate the mitral annular velocity curve. LV DD was classified by CMR and TTE following 2016 guidelines. Overall DD classification was 78.1% concordant between CMR and TTE (p < 0.0001). The trans-mitral inflow parameters correlated well between the two modalities (E, r = 0.78; A, r = 0.90; E/A, r = 0.82; all p < 0.0001) while the remaining diastolic parameters showed moderate correlation (e', r = 0.64; E/e', r = 0.54; left atrial volume index (LAVi), r = 0.61; all p < 0.0001). Classification of LV diastolic function by CMR showed good concordance with standardized grades established for TTE. CMR-based LV diastolic function may be integrated in routine clinical practice.Name of the registry: Technical Development of Cardiovascular Magnetic Resonance Imaging. Trial registration number: NCT00027170. Date of registration: November 26, 2001. URL of trial registry record: https://clinicaltrials.gov/ct2/show/NCT00027170.
Topics: Adult; Aged; Female; Humans; Male; Middle Aged; Cardiomyopathies; Diastole; Echocardiography; Magnetic Resonance Imaging, Cine; Prospective Studies; Ventricular Dysfunction, Left; Ventricular Function, Left
PubMed: 38778036
DOI: 10.1038/s41598-024-61992-6 -
Proceedings of SPIE--the International... Feb 2024For deep learning-based machine learning, not only are large and sufficiently diverse data crucial but their good qualities are equally important. However, in real-world...
For deep learning-based machine learning, not only are large and sufficiently diverse data crucial but their good qualities are equally important. However, in real-world applications, it is very common that raw source data may contain incorrect, noisy, inconsistent, improperly formatted and sometimes missing elements, particularly, when the datasets are large and sourced from many sites. In this paper, we present our work towards preparing and making image data ready for the development of AI-driven approaches for studying various aspects of the natural history of oral cancer. Specifically, we focus on two aspects: 1) cleaning the image data; and 2) extracting the annotation information. Data cleaning includes removing duplicates, identifying missing data, correcting errors, standardizing data sets, and removing personal sensitive information, toward combining data sourced from different study sites. These steps are often collectively referred to as data harmonization. Annotation information extraction includes identifying crucial or valuable texts that are manually entered by clinical providers related to the image paths/names and standardizing of the texts of labels. Both are important for the successful deep learning algorithm development and data analyses. Specifically, we provide details on the data under consideration, describe the challenges and issues we observed that motivated our work, present specific approaches and methods that we used to clean and standardize the image data and extract labelling information. Further, we discuss the ways to increase efficiency of the process and the lessons learned. Research ideas on automating the process with ML-driven techniques are also presented and discussed. Our intent in reporting and discussing such work in detail is to help provide insights in automating or, minimally, increasing the efficiency of these critical yet often under-reported processes.
PubMed: 38774479
DOI: 10.1117/12.3005875