-
Psychiatry Research Sep 2020Mental illness is one of the largest contributors to the global disease burden. The importance of valid and reliable mental health measures is crucial in order to...
Mental illness is one of the largest contributors to the global disease burden. The importance of valid and reliable mental health measures is crucial in order to accurately measure said burden, to capture symptom improvement, and to ensure that symptoms are appropriately identified and quantified. This is of particular importance in low and middle-income countries (LMICs), where the burden of mental illness is relatively high, and there is heterogeneity in linguistic, racial, and ethnic groups. Using the PHQ-9 as an illustrative example, this systematic review aims to provide an overview of existing work and highlight common validation and reporting practices. A systematic review of published literature validating the use of the PHQ-9 in LMICs as indexed in the PubMed and PsychInfo databases was conducted. The review included n = 49 articles (reduced from n = 2,349). This manuscript summarizes these results in terms of the frequency of reporting on important procedures and in regards to the types of reliability and validity measured. Then, building off of the existing literature, we provide key recommendations for measure validation in LMICs, which can be generalized for any type of measure used in a setting in which it was not initially developed.
Topics: Health Resources; Humans; Mass Screening; Mental Disorders; Patient Health Questionnaire; Poverty; Psychological Techniques; Reproducibility of Results
PubMed: 32593853
DOI: 10.1016/j.psychres.2020.113236 -
Health Technology Assessment... Feb 2016Unprovoked first venous thromboembolism (VTE) is defined as VTE in the absence of a temporary provoking factor such as surgery, immobility and other temporary factors.... (Review)
Review
Prediction of risk of recurrence of venous thromboembolism following treatment for a first unprovoked venous thromboembolism: systematic review, prognostic model and clinical decision rule, and economic evaluation.
BACKGROUND
Unprovoked first venous thromboembolism (VTE) is defined as VTE in the absence of a temporary provoking factor such as surgery, immobility and other temporary factors. Recurrent VTE in unprovoked patients is highly prevalent, but easily preventable with oral anticoagulant (OAC) therapy. The unprovoked population is highly heterogeneous in terms of risk of recurrent VTE.
OBJECTIVES
The first aim of the project is to review existing prognostic models which stratify individuals by their recurrence risk, therefore potentially allowing tailored treatment strategies. The second aim is to enhance the existing research in this field, by developing and externally validating a new prognostic model for individual risk prediction, using a pooled database containing individual patient data (IPD) from several studies. The final aim is to assess the economic cost-effectiveness of the proposed prognostic model if it is used as a decision rule for resuming OAC therapy, compared with current standard treatment strategies.
METHODS
Standard systematic review methodology was used to identify relevant prognostic model development, validation and cost-effectiveness studies. Bibliographic databases (including MEDLINE, EMBASE and The Cochrane Library) were searched using terms relating to the clinical area and prognosis. Reviewing was undertaken by two reviewers independently using pre-defined criteria. Included full-text articles were data extracted and quality assessed. Critical appraisal of included full texts was undertaken and comparisons made of model performance. A prognostic model was developed using IPD from the pooled database of seven trials. A novel internal-external cross-validation (IECV) approach was used to develop and validate a prognostic model, with external validation undertaken in each of the trials iteratively. Given good performance in the IECV approach, a final model was developed using all trials data. A Markov patient-level simulation was used to consider the economic cost-effectiveness of using a decision rule (based on the prognostic model) to decide on resumption of OAC therapy (or not).
RESULTS
Three full-text articles were identified by the systematic review. Critical appraisal identified methodological and applicability issues; in particular, all three existing models did not have external validation. To address this, new prognostic models were sought with external validation. Two potential models were considered: one for use at cessation of therapy (pre D-dimer), and one for use after cessation of therapy (post D-dimer). Model performance measured in the external validation trials showed strong calibration performance for both models. The post D-dimer model performed substantially better in terms of discrimination (c = 0.69), better separating high- and low-risk patients. The economic evaluation identified that a decision rule based on the final post D-dimer model may be cost-effective for patients with predicted risk of recurrence of over 8% annually; this suggests continued therapy for patients with predicted risks ≥ 8% and cessation of therapy otherwise.
CONCLUSIONS
The post D-dimer model performed strongly and could be useful to predict individuals' risk of recurrence at any time up to 2-3 years, thereby aiding patient counselling and treatment decisions. A decision rule using this model may be cost-effective for informing clinical judgement and patient opinion in treatment decisions. Further research may investigate new predictors to enhance model performance and aim to further externally validate to confirm performance in new, non-trial populations. Finally, it is essential that further research is conducted to develop a model predicting bleeding risk on therapy, to manage the balance between the risks of recurrence and bleeding.
STUDY REGISTRATION
This study is registered as PROSPERO CRD42013003494.
FUNDING
The National Institute for Health Research Health Technology Assessment programme.
Topics: Anticoagulants; Cost-Benefit Analysis; Decision Support Systems, Clinical; Humans; Prognosis; Risk Factors; Secondary Prevention; Technology Assessment, Biomedical; Venous Thromboembolism
PubMed: 26879848
DOI: 10.3310/hta20120 -
Breast Cancer (Tokyo, Japan) 2003Little is known about the availability and psychometric properties of instruments to measure quality of life (QOL) in clinical research on Japanese patients with breast... (Review)
Review
BACKGROUND
Little is known about the availability and psychometric properties of instruments to measure quality of life (QOL) in clinical research on Japanese patients with breast cancer. The purpose of this systematic review is to find reliable and valid instruments available in Japan, and to summarize their characteristics.
METHODS
Instruments available in Japan were found through a systematic search of the literature. Each instrument identified was evaluated for item development, reliability, validity, interpretability and utility.
RESULTS
Six questionnaires to measure health-related QOL (the QOL-ACD, the EORTC QLQ-C30, the EORTC QLQ-BR23, the FACT-B, the SF-36, the WHO/QOL-26) and five scales to quantify the psychological burden (the STAI, the POMS, the SDS, the HADS, the GHQ), for which reliability and validity have been documented, are available in Japanese. All instruments were developed in foreign countries except for the QOL-ACD. Two of the QOL questionnaires were specific to breast cancer (the EORTC QLQ-BR23, the FACT-B). Though the measurements can be interpreted in some manner, the meaning of change scores over time has been documented for only three instruments (the EORTC QLQ-C30, the FACT-B, and the GHQ).
CONCLUSIONS
The review provides grounds for designing and implementing quantitative research on QOL of breast cancer patients in Japan. Methodological challenges, however, continue, particularly for validating instruments with regard to various study populations of Japanese people and demonstrating the clinical importance of change scores.
Topics: Breast Neoplasms; Female; Humans; Japan; Quality of Life; Reproducibility of Results; Surveys and Questionnaires
PubMed: 12955032
DOI: 10.1007/BF02966719 -
BMC Medical Research Methodology Jan 2017Data on health-related quality of life (HRQoL) is paramount for shared and evidence based decision-making. Since an overview of cervical cancer HRQoL tools and their... (Review)
Review
BACKGROUND
Data on health-related quality of life (HRQoL) is paramount for shared and evidence based decision-making. Since an overview of cervical cancer HRQoL tools and their validity appears to be lacking, we performed a systematic review on usage of disease specific HRQoL instruments in cervical cancer patients and their psychometric properties to identify the most suitable cervical cancer specific HRQoL tool.
METHODS
We searched Pubmed, EMBASE and PsycINFO from inception up to 18 October 2016 for studies on quality of life in cervical cancer patients. Data extraction and HRQoL identification was performed by two independent reviewers. Validation studies of the identified cervical cancer specific HRQoL tools were retrieved and assessed on psychometric properties using the COSMIN checklist. All used cervical cancer specific HRQoL instruments were scored and ranked according to their psychometric properties.
RESULTS
We included 156 studies (20,690 patients) and identified 31 HRQoL tools. The EORTC QLQ-CX24 (35 studies; 5,556 patients) and FACT-Cx (22 studies; 4,224 patients) were the only cervical cancer specific tools. The EORTC QLQ-CX24 had 4 out of 9 positive rated psychometric properties; internal consistency, content and construct validity, and agreement. Criterion validity, reliability, and interpretability scored doubtful. Responsiveness and floor- and ceiling effects were not reported. The FACT-Cx had 2 out of 9 positive rated psychometric properties; internal consistency and agreement. Content validity, reliability, and interpretability scored doubtful while criterion and construct validity scored negative. Responsiveness and floor- and ceiling effects were not reported.
CONCLUSION
The validity of the often used EORTC QLQ-CX24 questionnaire for cervical cancer patients remains uncertain as 5 out of 9 psychometric properties were doubtful or not reported in current literature. Cervical cancer specific HRQoL tools should therefore always be used in conjunction with validated generic cancer HRQoL tools until proper validity has been proven, or a more valid tool has been developed.
Topics: Female; Health Status; Humans; Psychometrics; Quality of Life; Reproducibility of Results; Surveys and Questionnaires; Uterine Cervical Neoplasms
PubMed: 28125962
DOI: 10.1186/s12874-016-0289-x -
The Journal of Orthopaedic and Sports... Aug 2017Study Design Systematic review and validation study. Background Many prognostic models of knee pain outcomes have been developed for use in primary care. Variability... (Review)
Review
Study Design Systematic review and validation study. Background Many prognostic models of knee pain outcomes have been developed for use in primary care. Variability among published studies with regard to patient population, outcome measures, and relevant prognostic factors hampers the generalizability and implementation of these models. Objectives To summarize existing prognostic models in patients with knee pain in a primary care setting and to develop and internally validate new summary prognostic models. Methods After a sensitive search strategy, 2 reviewers independently selected prognostic models for patients with nontraumatic knee pain and assessed the methodological quality of the included studies. All predictors of the included studies were evaluated, summarized, and classified. The predictors assessed in multiple studies of sufficient quality are presented in this review. Using data from the Musculoskeletal System Study (BAS) cohort of patients with a new episode of knee pain, recruited consecutively by Dutch general medical practitioners (n = 372), we used predictors with a strong level of evidence to develop new prognostic models for each outcome measure and internally validated these models. Results Sixteen studies were eligible for inclusion. We considered 11 studies to be of sufficient quality. None of these studies validated their models. Five predictors with strong evidence were related to function and 6 to recovery, and were used to compose 2 prognostic models for patients with knee pain at 1 year. Running these new models in another data set showed explained variances (R) of 0.36 (function) and 0.33 (recovery). The area under the curve of the recovery model was 0.79. After internal validation, the adjusted R values of the models were 0.30 (function) and 0.20 (recovery), and the area under the curve was 0.73. Conclusion We developed 2 valid prognostic models for function and recovery for patients with nontraumatic knee pain, based on predictors with strong evidence. A longer duration of complaints predicted poorer function but did not adequately predict chance of recovery. Level of Evidence Prognosis, levels 1a and 1b. J Orthop Sports Phys Ther 2017;47(8):518-529. Epub 16 Jun 2017. doi:10.2519/jospt.2017.7142.
Topics: Humans; Knee Joint; Models, Statistical; Osteoarthritis, Knee; Pain; Pain Management; Physical Therapy Modalities; Primary Health Care; Prognosis
PubMed: 28622751
DOI: 10.2519/jospt.2017.7142 -
Seminars in Arthritis and Rheumatism Oct 2022In the management of rheumatoid arthritis (RA), there is a clinical need to identify which patients are at high-risk of not responding to methotrexate (MTX), or... (Meta-Analysis)
Meta-Analysis Review
BACKGROUND
In the management of rheumatoid arthritis (RA), there is a clinical need to identify which patients are at high-risk of not responding to methotrexate (MTX), or experiencing adverse events (AEs), to enable earlier alternative treatments. Many clinical prediction models (CPMs) have previously been developed, but a summary of such models and their methodological quality is lacking. This systematic review aimed to (i) identify and summarize previously published CPMs of MTX outcomes in biologic-naïve RA patients, and (ii) critically appraise their methodological properties.
METHODS
Medline and Embase were searched to identify studies developing or validating CPMs of MTX outcomes in RA patients. The risk of bias (ROB) was assessed using PROBAST (prediction model risk of bias assessment tool). A fixed effects meta-analysis summarised discrimination for models with multiple external validations.
RESULTS
The systematic review identified 20 CPMs across 13 studies, and 4 validation studies. Three outcome types were used: a state of disease activity (n = 14, 70%); EULAR response criteria (n = 4, 20%); or discontinuation due to AEs (n = 2, 10%). Only one model accounted for potential competing risks, and nine (45%) were internally validated. Eight (40%) models used multiple imputation for missing data, others were often limited to complete case analysis. There was overall high ROB. The meta-analysis summarised c-statistics for two models with multiple external validations was 0.77 (95% CI: 0.69, 0.84) and 0.68 (0.64, 0.71).
CONCLUSION
This review highlights several methodological shortcomings that should be addressed in future model development to increase potential for implementation into practice.
Topics: Antirheumatic Agents; Arthritis, Rheumatoid; Humans; Methotrexate; Models, Statistical; Prognosis; Treatment Outcome
PubMed: 35921745
DOI: 10.1016/j.semarthrit.2022.152076 -
The Journal of Arthroplasty Dec 2022Total hip arthroplasty (THA) carries a substantial litigative burden. THA may introduce leg length discrepancy (LLD), necessitating a valid and reliable technique for... (Meta-Analysis)
Meta-Analysis
BACKGROUND
Total hip arthroplasty (THA) carries a substantial litigative burden. THA may introduce leg length discrepancy (LLD), necessitating a valid and reliable technique for LLD measurement. This study investigates the reliability and validity of techniques quantitively measuring LLD in both pre- and post-THA.
METHODS
Embase and MEDLINE databases were searched following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for articles assessing either the validity or reliability of LLD measurement techniques. Data was pooled using random effects meta-analysis to derive reliability estimates. Study quality was assessed using the Brink and Louw checklist.
RESULTS
Forty-two articles with 2059 participants were included. Thirty-three investigated reliability and 25 validity. Reliability displayed high heterogeneity. Poor to excellent intra-rater reliability was reported for antero-posterior pelvis radiographs, moderate to excellent for computed tomography scanograms, and good to excellent for clinical methods and teleradiography, and excellent for bi-planar radiography (BPR). Poor to excellent inter-rater reliability was reported for antero-posterior pelvis radiographs and clinical methods, moderate to excellent for teleradiography, good to excellent for computed tomography scanogram and excellent for BPR. The tape measure method is a valid clinical measure of LLD whilst markerless motion analysis and the block method are not. Imaging techniques are appropriately cross-validated with the exception of BPR.
CONCLUSION
The reported intra- and inter-rater reliability for most measurement techniques vary widely. The tape measure method is a valid clinical measurement of LLD. Imaging techniques have been appropriately cross-validated, with the exception of BPR, although they lack validation against a common reference technique.
Topics: Humans; Arthroplasty, Replacement, Hip; Reproducibility of Results; Leg; Leg Length Inequality; Radiography
PubMed: 35605764
DOI: 10.1016/j.arth.2022.05.029 -
JMIR Public Health and Surveillance Mar 2022HIV assays designed to detect recent infection, also known as "recency assays," are often used to estimate HIV incidence in a specific country, region, or subpopulation,... (Review)
Review
BACKGROUND
HIV assays designed to detect recent infection, also known as "recency assays," are often used to estimate HIV incidence in a specific country, region, or subpopulation, alone or as part of recent infection testing algorithms (RITAs). Recently, many countries and organizations have become interested in using recency assays within case surveillance systems and routine HIV testing services to measure other indicators beyond incidence, generally referred to as "non-incidence surveillance use cases."
OBJECTIVE
This review aims to identify published evidence that can be used to validate methodological approaches to recency-based incidence estimation and non-incidence use cases. The evidence identified through this review will be used in the forthcoming technical guidance by the World Health Organization (WHO) and United Nations Programme on HIV/AIDS (UNAIDS) on the use of HIV recency assays for identification of epidemic trends, whether for HIV incidence estimation or non-incidence indicators of recency.
METHODS
To identify the best methodological and field implementation practices for the use of recency assays to estimate HIV incidence and trends in recent infections for specific populations or geographic areas, we conducted a systematic review of the literature to (1) understand the use of recency testing for surveillance in programmatic and laboratory settings, (2) review methodologies for implementing recency testing for both incidence estimation and non-incidence use cases, and (3) assess the field performance characteristics of commercially available recency assays.
RESULTS
Among the 167 documents included in the final review, 91 (54.5%) focused on assay or algorithm performance or methodological descriptions, with high-quality evidence of accurate age- and sex-disaggregated HIV incidence estimation at national or regional levels in general population settings, but not at finer geographic levels for prevention prioritization. The remaining 76 (45.5%) described the field use of incidence assays including field-derived incidence (n=45), non-incidence (n=25), and both incidence and non-incidence use cases (n=6). The field use of incidence assays included integrating RITAs into routine surveillance and assisting with molecular genetic analyses, but evidence was generally weaker or only reported on what was done, without validation data or findings related to effectiveness of using non-incidence indicators calculated through the use of recency assays as a proxy for HIV incidence.
CONCLUSIONS
HIV recency assays have been widely validated for estimating HIV incidence in age- and sex-specific populations at national and subnational regional levels; however, there is a lack of evidence validating the accuracy and effectiveness of using recency assays to identify epidemic trends in non-incidence surveillance use cases. More research is needed to validate the use of recency assays within HIV testing services, to ensure findings can be accurately interpreted to guide prioritization of public health programming.
Topics: Algorithms; Female; HIV Infections; Humans; Incidence; Male
PubMed: 35275085
DOI: 10.2196/34410 -
Child Psychiatry and Human Development Feb 2024Parenting significantly influences youth development, yet there's a dearth of research on measuring parenting among LGBTQIA+ caregivers, or caregivers of... (Review)
Review
Parenting significantly influences youth development, yet there's a dearth of research on measuring parenting among LGBTQIA+ caregivers, or caregivers of LGBTQIA+ children (hereafter LGBTQIA+ families). In this systematic review we identified and evaluated the psychometrics of parenting scales validated for this population. The inclusion criteria encompassed studies with LGBTQIA+ families in major databases and secondary sources, psychometric assessment, and English language. Eight studies validating ten scales measuring parenting practices, parental attitudes, beliefs, and perceptions were identified. Generally, studies reported promising psychometrics, showing evidence of construct validity in all and reliability in seven. However, the review also unveiled crucial gaps: a paucity of scales validated among LGBTQIA+ fathers, and predominantly featured non-Hispanic White participants. Findings underscore the necessity for more inclusive samples that reflect the diversity of LGBTQIA+ families. The validation of parenting scales is crucial for understanding parenting in LGBTQIA+ families and developing parenting interventions to promote their well-being.
PubMed: 38411766
DOI: 10.1007/s10578-024-01672-8 -
Journal of Clinical Epidemiology May 2022To identify and critically appraise risk prediction models for living donor solid organ transplant counselling. (Review)
Review
OBJECTIVE
To identify and critically appraise risk prediction models for living donor solid organ transplant counselling.
STUDY DESIGN AND SETTING
We systematically reviewed articles describing the development or validation of prognostic risk prediction models about living donor solid organ (kidney and liver) transplantation indexed in Medline until April 4, 2021. Models were eligible if intended to predict, at transplant counselling, any outcome occurring after transplantation or donation in recipients or donors. Duplicate study selection, data extraction, assessment for risk of bias and quality of reporting was done using the CHARMS checklist, PRISMA recommendations, PROBAST tool, and TRIPOD Statement.
RESULTS
We screened 4691 titles and included 49 studies describing 68 models (35 kidney, 33 liver transplantation). We identified 49 new risk prediction models and 19 external validations of existing models. Most models predicted recipients outcomes (n = 38, 75%), e.g., kidney graft loss (29%), or mortality of liver transplant recipients (55%). Many new models (n = 46, 94%) and external validations (n = 17, 89%) had a high risk of bias because of methodological weaknesses. The quality of reporting was generally poor.
CONCLUSION
We advise against applying poorly developed, reported, or validated prediction models. Future studies could validate or update the few identified methodologically appropriate models.
Topics: Humans; Kidney Transplantation; Prognosis; Tissue Donors
PubMed: 35124188
DOI: 10.1016/j.jclinepi.2022.01.025