-
Journal of Biomedical Informatics May 2020This review aims to: 1) evaluate the quality of model reporting, 2) provide an overview of methodology for developing and validating Early Warning Score Systems (EWSs)... (Review)
Review
OBJECTIVES
This review aims to: 1) evaluate the quality of model reporting, 2) provide an overview of methodology for developing and validating Early Warning Score Systems (EWSs) for adult patients in acute care settings, and 3) highlight the strengths and limitations of the methodologies, as well as identify future directions for EWS derivation and validation studies.
METHODOLOGY
A systematic search was conducted in PubMed, Cochrane Library, and CINAHL. Only peer reviewed articles and clinical guidelines regarding developing and validating EWSs for adult patients in acute care settings were included. 615 articles were extracted and reviewed by five of the authors. Selected studies were evaluated based on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist. The studies were analyzed according to their study design, predictor selection, outcome measurement, methodology of modeling, and validation strategy.
RESULTS
A total of 29 articles were included in the final analysis. Twenty-six articles reported on the development and validation of a new EWS, while three reported on validation and model modification. Only eight studies met more than 75% of the items in the TRIPOD checklist. Three major techniques were utilized among the studies to inform their predictive algorithms: 1) clinical-consensus models (n = 6), 2) regression models (n = 15), and 3) tree models (n = 5). The number of predictors included in the EWSs varied from 3 to 72 with a median of seven. Twenty-eight models included vital signs, while 11 included lab data. Pulse oximetry, mental status, and other variables extracted from electronic health records (EHRs) were among other frequently used predictors. In-hospital mortality, unplanned transfer to the intensive care unit (ICU), and cardiac arrest were commonly used clinical outcomes. Twenty-eight studies conducted a form of model validation either within the study or against other widely-used EWSs. Only three studies validated their model using an external database separate from the derived database.
CONCLUSION
This literature review demonstrates that the characteristics of the cohort, predictors, and outcome selection, as well as the metrics for model validation, vary greatly across EWS studies. There is no consensus on the optimal strategy for developing such algorithms since data-driven models with acceptable predictive accuracy are often site-specific. A standardized checklist for clinical prediction model reporting exists, but few studies have included reporting aligned with it in their publications. Data-driven models are subjected to biases in the use of EHR data, thus it is particularly important to provide detailed study protocols and acknowledge, leverage, or reduce potential biases of the data used for EWS development to improve transparency and generalizability.
Topics: Adult; Early Warning Score; Humans; Intensive Care Units; Models, Statistical; Prognosis; Vital Signs
PubMed: 32278089
DOI: 10.1016/j.jbi.2020.103410 -
International Journal of Environmental... Apr 2021Current lifestyles are marked by sedentary behaviour; thus, it is of great importance for policymaking to have valid and reliable tools to measure sedentary behaviour in... (Meta-Analysis)
Meta-Analysis Review
Current lifestyles are marked by sedentary behaviour; thus, it is of great importance for policymaking to have valid and reliable tools to measure sedentary behaviour in order to combat it. Therefore, the aim of this review and meta-analysis is to critically review, assess, and compile the reliability, criterion validity, and construct validity of the single-item sedentary behaviour questions within national language versions of most commonly used international physical activity questionnaires for adults in the European Union: The International Physical Activity Questionnaire-Short Form and the Global Physical Activity Questionnaire. A total of 1749 records were screened, 287 full-text papers were read, and 14 studies were included in the meta-analysis. The results and quality of studies were evaluated by the Quality Assessment of Physical Activity Questionnaires checklist. Meta-analysis indicated moderate to high reliability (r = 0.59) and concurrent validity (r = 0.55) of national language versions of single-item sedentary behaviour questions. Criterion validity was rather low (r = 0.23) but in concordance with previous studies. The risk of bias analysis highlighted the poor reporting of methods and results, with a total bias score of 0.42. Thus, we recommend using multi-item SB questionnaires and smart trackers for providing information on SB rather than single-item sedentary behaviour questions in physical activity questionnaires.
Topics: European Union; Exercise; Reproducibility of Results; Sedentary Behavior; Surveys and Questionnaires
PubMed: 33926123
DOI: 10.3390/ijerph18094602 -
Journal of Medical Systems Jan 2022In clinical practice, assessing digital health literacy is important to identify patients who may encounter difficulties adapting to digital health using digital...
In clinical practice, assessing digital health literacy is important to identify patients who may encounter difficulties adapting to digital health using digital technology and service. We developed the Digital Health Technology Literacy Assessment Questionnaire (DHTL-AQ) to assess the ability to use digital health technology, services, and data. The DHTL-AQ was developed in three phases. In the first phase, the conceptual framework and domains and items were generated from a systematic literature review using relevant theory and surveys. In the second phase, a cross-sectional survey with 590 adults age ≥ 18 years was conducted at an academic hospital in Seoul, Korea in January and February 2020 to test face validity of the items. Then, psychometric validation was conducted to determine the final items and cut-off scores of the DHTL-AQ. The eHealth literacy scale, the Newest Vital Sign, and 10 mobile app task ability assessments were examined to test validity. The final DHTL-AQ includes 34 items in two domains (digital functional and digital critical literacy) and 4 categories (Information and Communications Technology terms, Information and Communications Technology icons, use of an app, evaluating reliability and relevance of health information). The DHTL-AQ had excellent internal consistency (overall Cronbach's α = 0.95; 0.87-0.94 for subtotals) and acceptable model fit (CFI = 0.821, TLI = 0.807, SRMR = 0.065, RMSEA = 0.090). The DHTL-AQ was highly correlated with task ability assessment (r = 0.7591), and moderately correlated with the eHealth literacy scale (r = 0.5265) and the Newest Vital Sign (r = 0.5929). The DHTL-AQ is a reliable and valid instrument to measure digital health technology literacy.
Topics: Adolescent; Adult; Biomedical Technology; Cross-Sectional Studies; Digital Technology; Humans; Psychometrics; Reproducibility of Results; Surveys and Questionnaires
PubMed: 35072816
DOI: 10.1007/s10916-022-01800-8 -
European Urology Focus Nov 2021Early detection of kidney cancer improves survival; however, low prevalence means that population-wide screening may be inefficient. Stratification of the population... (Review)
Review
CONTEXT
Early detection of kidney cancer improves survival; however, low prevalence means that population-wide screening may be inefficient. Stratification of the population into risk categories could allow for the introduction of a screening programme tailored to individuals.
OBJECTIVE
This review will identify and compare published models that predict the risk of developing kidney cancer in the general population.
EVIDENCE ACQUISITION
A search identified primary research reporting or validating models predicting the risk of kidney cancer in Medline and EMBASE. After screening identified studies for inclusion, we extracted data onto a standardised form. The risk models were classified using the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines and evaluated using the PROBAST assessment tool.
EVIDENCE SYNTHESIS
The search identified 15 281 articles. Sixty-two satisfied the inclusion criteria; performance measures were provided for 11 models. Some models predicted the risk of prevalent undiagnosed disease and others future incident disease. Six of the models had been validated, two using external populations. The most commonly included risk factors were age, smoking status, and body mass index. Most of the models had acceptable-to-good discrimination (area under the receiver-operating curve >0.7) in development and validation. Many models also had high specificity; however, several had low sensitivity. The highest performance was seen for the models using only biomarkers to detect kidney cancer; however, these were developed and validated in small case-control studies.
CONCLUSIONS
We identified a small number of risk models that could be used to stratify the population according to the risk of kidney cancer. Most exhibit reasonable discrimination, but a few have been validated externally in population-based studies.
PATIENT SUMMARY
In this review, we looked at mathematical models predicting the likelihood of an individual developing kidney cancer. We found several suitable models, using a range of risk factors (such as age and smoking) to predict the risk for individuals. Most of the models identified require further testing in the general population to confirm their usefulness.
Topics: Biomarkers; Humans; Kidney Neoplasms; Mass Screening; Prognosis; Risk Factors
PubMed: 32680829
DOI: 10.1016/j.euf.2020.06.024 -
Clinical Microbiology and Infection :... Aug 2021Only clinically validated HPV assays can be accepted in cervical cancer screening. (Meta-Analysis)
Meta-Analysis
BACKGROUND
Only clinically validated HPV assays can be accepted in cervical cancer screening.
OBJECTIVES
To update the list of high-risk HPV assays that fulfil the 2009 international validation criteria (Meijer-2009).
DATA SOURCES
PubMed/Medline, Embase, Scopus, references from selected studies; published in January 2014 to August 2020.
STUDY ELIGIBILITY CRITERIA
HPV test validation studies and primary screening studies, involving testing with an index HPV test and a comparator HPV test with reporting of disease outcome (occurrence of histologically confirmed cervical precancer; CIN2+).
PARTICIPANTS
Women participating in cervical cancer screening.
INTERVENTIONS
Testing with an index and a comparator HPV test of clinician-collected cervical specimens and assessment of disease outcome (
validation guidelines, or tests with consistent previous validations. METHODS
Assessment of relative clinical accuracy (including non-inferiority statistics index vs comparator assay) and test reproducibility in individual studies; random effects meta-analyses of the relative clinical sensitivity and specificity of index vs comparator tests.
RESULTS
Seven hrHPV DNA tests consistently fulfilled all validation criteria in multiple studies using predefined test positivity cut-offs (Abbott RealTime High Risk HPV, Anyplex II HPV HR Detection, BD Onclarity HPV Assay, Cobas 4800 HPV Test, HPV-Risk Assay, PapilloCheck HPV-Screening Test and Xpert HPV). Another assay (Alinity m HR HPV Assay) was fully validated in one validation study. The newer Cobas 6800 HPV Test, was validated in two studies against Cobas 4800. Other tests partially fulfilled the international validation criteria (Cervista HPV HR Test, EUROArray HPV, Hybribio's 14 High-Risk HPV, LMNX Genotyping Kit GP HPV, MALDI-TOF, RIATOL qPCR and a number of other in-house developed assays) since the non-inferior accuracy was reached after a posteriori cut-off optimization, inconsistent accuracy findings in different studies, and/or insufficient reproducibility assessment. The APTIMA HPV Assay targeting E6/E7 mRNA of hrHPV was fully validated in one formal validation study and showed slightly lower pooled sensitivity but higher specificity than the standard comparator tests in seven screening studies. However, the current international validation criteria relate to DNA assays. The additional requirement for longitudinal performance data required for non-DNA based HPV assays was not assessed in this review.
CONCLUSIONS
Eleven hrHPV DNA assays fulfil all requirements for use in cervical cancer screening using clinician-collected specimens.
Topics: Alphapapillomavirus; Early Detection of Cancer; Female; Genotyping Techniques; Humans; Papillomaviridae; Papillomavirus Infections; Reproducibility of Results; Sensitivity and Specificity; Uterine Cervical Neoplasms
PubMed: 33975008
DOI: 10.1016/j.cmi.2021.04.031 -
Neuropsychobiology 2023The utility of heart rate variability (HRV) for characterizing psychological stress is primarily impacted by methodological considerations such as study populations,... (Review)
Review
The utility of heart rate variability (HRV) for characterizing psychological stress is primarily impacted by methodological considerations such as study populations, experienced versus induced stress, and method of stress assessment. Here, we review studies on the associations between HRV and psychological stress, examining the nature of stress, ways stress was assessed, and HRV metrics used. The review was performed according to the PRISMA guidelines on select databases. Studies that examined the HRV-stress relationship via repeated measurements and validated psychometric instruments were included (n = 15). Participant numbers and ages ranged between 10 and 403 subjects and 18 and 60 years, respectively. Both experimental (n = 9) and real-life stress (n = 6) have been explored. While RMSSD was the most reported HRV metric (n = 10) significantly associated with stress, other metrics, including LF/HF (n = 7) and HF power (n = 6) were also reported. Various linear and nonlinear HRV metrics have been utilized, with nonlinear metrics used less often. The most frequently used psychometric instrument was the State-Trait Anxiety Inventory (n = 10), though various other instruments have been reported. In conclusion, HRV is a valid measure of the psychological stress response. Standard stress induction and assessment protocols combined with validated HRV measures in different domains will improve the validity of findings.
Topics: Humans; Adult; Heart Rate; Stress, Psychological
PubMed: 37290411
DOI: 10.1159/000530376 -
Sensors (Basel, Switzerland) Feb 2020Rigidity is one of the cardinal symptoms of Parkinson´s disease (PD). Present in up 89% of cases, it is typically assessed with clinical scales. However, these...
Rigidity is one of the cardinal symptoms of Parkinson´s disease (PD). Present in up 89% of cases, it is typically assessed with clinical scales. However, these instruments show limitations due to their subjectivity and poor intra- and inter-rater reliability. To compile all of the objective quantitative methods used to assess rigidity in PD and to study their validity and reliability, a systematic review was conducted using the Web of Science, PubMed, and Scopus databases. Studies from January 1975 to June 2019 were included, all of which were written in English. The Strengthening the Reporting of observational studies in Epidemiology Statement (STROBE) checklist for observational studies was used to assess the methodological rigor of the included studies. Thirty-six studies were included. Rigidity was quantitatively assessed in three ways, using servomotors, inertial sensors, and biomechanical and neurophysiological study of muscles. All methods showed good validity and reliability, good correlation with clinical scales, and were useful for detecting rigidity and studying its evolution. People with PD exhibit higher values in terms of objective muscle stiffness than healthy controls. Rigidity depends on the angular velocity and articular amplitude of the mobilization applied. There are objective, valid, and reliable methods that can be used to quantitatively assess rigidity in people with PD.
Topics: Electromyography; Humans; Joints; Movement; Muscle Rigidity; Muscles; Observational Studies as Topic; Parkinson Disease
PubMed: 32041374
DOI: 10.3390/s20030880 -
International Journal of Environmental... Oct 2019This study aimed to systematically review previous studies on the reliability and concurrent validity of the Global Physical Activity Questionnaire (GPAQ). A systematic...
This study aimed to systematically review previous studies on the reliability and concurrent validity of the Global Physical Activity Questionnaire (GPAQ). A systematic literature search was conducted ( = 26) using the online EBSCOHost databases, PubMed, Web of Science, and Google Scholar up to September 2019. A previously developed coding sheet was used to collect the data. The Modified Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies was employed to assess risk of bias and study quality. It was found that GPAQ was primarily revalidated in adult populations in Asian and European countries. The sample size ranged from 43 to 2657 with a wide age range (i.e., 15-79 years old). Different populations yielded inconsistent results concerning the reliability and validity of the GPAQ. Short term (i.e., one- to two-week interval) and long-term (i.e., two- to three-month apart) test-retest reliability was good to very good. The concurrent validity using accelerometers, pedometers, and physical activity (PA) log was poor to fair. The GPAQ data and accelerometer/pedometer/PA log data were not compared using the same measurements in some validation studies. Studies with more rigorous research designs are needed before any conclusions concerning the concurrent validity of GPAQ can be reached.
Topics: Exercise; Humans; Reproducibility of Results; Surveys and Questionnaires
PubMed: 31717742
DOI: 10.3390/ijerph16214128 -
BMJ (Clinical Research Ed.) Sep 2021To identify and assess the quality and accuracy of prognostic models for nephropathy and to validate these models in external cohorts of people with type 2 diabetes.
OBJECTIVES
To identify and assess the quality and accuracy of prognostic models for nephropathy and to validate these models in external cohorts of people with type 2 diabetes.
DESIGN
Systematic review and external validation.
DATA SOURCES
PubMed and Embase.
ELIGIBILITY CRITERIA
Studies describing the development of a model to predict the risk of nephropathy, applicable to people with type 2 diabetes.
METHODS
Screening, data extraction, and risk of bias assessment were done in duplicate. Eligible models were externally validated in the Hoorn Diabetes Care System (DCS) cohort (n=11 450) for the same outcomes for which they were developed. Risks of nephropathy were calculated and compared with observed risk over 2, 5, and 10 years of follow-up. Model performance was assessed based on intercept adjusted calibration and discrimination (Harrell's C statistic).
RESULTS
41 studies included in the systematic review reported 64 models, 46 of which were developed in a population with diabetes and 18 in the general population including diabetes as a predictor. The predicted outcomes included albuminuria, diabetic kidney disease, chronic kidney disease (general population), and end stage renal disease. The reported apparent discrimination of the 46 models varied considerably across the different predicted outcomes, from 0.60 (95% confidence interval 0.56 to 0.64) to 0.99 (not available) for the models developed in a diabetes population and from 0.59 (not available) to 0.96 (0.95 to 0.97) for the models developed in the general population. Calibration was reported in 31 of the 41 studies, and the models were generally well calibrated. 21 of the 64 retrieved models were externally validated in the Hoorn DCS cohort for predicting risk of albuminuria, diabetic kidney disease, and chronic kidney disease, with considerable variation in performance across prediction horizons and models. For all three outcomes, however, at least two models had C statistics >0.8, indicating excellent discrimination. In a secondary external validation in GoDARTS (Genetics of Diabetes Audit and Research in Tayside Scotland), models developed for diabetic kidney disease outperformed those for chronic kidney disease. Models were generally well calibrated across all three prediction horizons.
CONCLUSIONS
This study identified multiple prediction models to predict albuminuria, diabetic kidney disease, chronic kidney disease, and end stage renal disease. In the external validation, discrimination and calibration for albuminuria, diabetic kidney disease, and chronic kidney disease varied considerably across prediction horizons and models. For each outcome, however, specific models showed good discrimination and calibration across the three prediction horizons, with clinically accessible predictors, making them applicable in a clinical setting.
SYSTEMATIC REVIEW REGISTRATION
PROSPERO CRD42020192831.
Topics: Aged; Albuminuria; Calibration; Clinical Decision Rules; Diabetes Mellitus, Type 2; Diabetic Nephropathies; Female; Humans; Kidney Failure, Chronic; Male; Middle Aged; Predictive Value of Tests; Prognosis; Renal Insufficiency, Chronic; Reproducibility of Results; Risk Assessment; Risk Factors
PubMed: 34583929
DOI: 10.1136/bmj.n2134 -
British Journal of Clinical Pharmacology Apr 2021Numerous algorithms have been developed to guide warfarin dosing and improve clinical outcomes. We reviewed the algorithms available for various populations and the... (Review)
Review
AIMS
Numerous algorithms have been developed to guide warfarin dosing and improve clinical outcomes. We reviewed the algorithms available for various populations and the covariates, performances and risk of bias of these algorithms.
METHODS
We systematically searched MEDLINE up to 20 May 2020 and selected studies describing the development, external validation or clinical utility of a multivariable warfarin dosing algorithm. Two investigators conducted data extraction and quality assessment.
RESULTS
Of 10 035 screened records, 266 articles were included in the review, describing the development of 433 dosing algorithms, 481 external validations and 52 clinical utility assessments. Most developed algorithms were for dose initiation (86%), developed by multiple linear regression (65%) and mostly applicable to Asians (49%) or Whites (43%). The most common demographic/clinical/environmental covariates were age (included in 401 algorithms), concomitant medications (270 algorithms) and weight (229 algorithms) while CYP2C9 (329 algorithms), VKORC1 (319 algorithms) and CYP4F2 (92 algorithms) variants were the most common genetic covariates. Only 26% and 7% algorithms were externally validated and evaluated for clinical utility, respectively, with <2% of algorithm developments and external validations being rated as having a low risk of bias.
CONCLUSION
Most warfarin dosing algorithms have been developed in Asians and Whites and may not be applicable to under-served populations. Few algorithms have been externally validated, assessed for clinical utility, and/or have a low risk of bias which makes them unreliable for clinical use. Algorithm development and assessment should follow current methodological recommendations to improve reliability and applicability, and under-represented populations should be prioritized.
Topics: Algorithms; Anticoagulants; Cytochrome P-450 CYP2C9; Dose-Response Relationship, Drug; Genotype; Humans; Pharmacogenetics; Reproducibility of Results; Vitamin K Epoxide Reductases; Warfarin
PubMed: 33080066
DOI: 10.1111/bcp.14608