-
Statistics in Medicine May 2005The systematic review community has become increasingly aware of the importance of addressing the issues of heterogeneity and publication bias in meta-analyses. A... (Comparative Study)
Comparative Study Review
The systematic review community has become increasingly aware of the importance of addressing the issues of heterogeneity and publication bias in meta-analyses. A potentially bigger threat to the validity of a meta-analysis appears relatively unnoticed. The within-study selective reporting of outcomes, defined as the selection of a subset of the original variables recorded for inclusion in publication of trials, can theoretically have a substantial impact on the results. A cohort of meta-analyses on the Cochrane Library was reviewed to examine how often this form of within-study publication bias was suspected and explained some of the evident funnel plot asymmetry. In cases where the level of suspicion was high, sensitivity analysis was undertaken to assess the robustness of the conclusion to this bias. Although within-study selection was evident or suspected in several trials, the impact on the conclusions of the meta-analyses was minimal. This paper deals with the identification of, sensitivity analysis for, and impact of within-study selective reporting in meta-analysis.
Topics: Clinical Trials as Topic; Humans; Meta-Analysis as Topic; Outcome Assessment, Health Care; Selection Bias; United Kingdom
PubMed: 15580591
DOI: 10.1002/sim.2025 -
PloS One 2015Bias in randomized controlled trials (RCTs) of complementary therapy interventions seems to be associated with specific factors and to potentially distort the studies'... (Review)
Review
BACKGROUND
Bias in randomized controlled trials (RCTs) of complementary therapy interventions seems to be associated with specific factors and to potentially distort the studies' conclusions. This systematic review assessed associated factors of risk of bias and consequences for the studies' conclusions in RCTs of yoga as one of the most commonly used complementary therapies.
METHODS
Medline/PubMed, Scopus, IndMED and the Cochrane Library were searched through February 2014 for yoga RCTs. Risk of selection bias was assessed using the Cochrane tool and regressed to a) publication year; b) country of origin; c) journal type; and d) impact factor using multiple logistic regression analysis. Likewise, the authors' conclusions were regressed to risk of bias.
RESULTS
A total of 312 RCTs were included. Impact factor ranged from 0.0 to 39.2 (median = 1.3); 60 RCT (19.2%) had a low risk of selection bias, and 252 (80.8%) had a high or unclear risk of selection bias. Only publication year and impact factor significantly predicted low risk of bias; RCTs published after 2001 (adjusted odds ratio (OR) = 12.6; 95% confidence interval (CI) = 1.7, 94.0; p<0.001) and those published in journals with impact factor (adjusted OR = 2.6; 95%CI = 1.4, 4.9; p = 0.004) were more likely to have low risk of bias. The authors' conclusions were not associated with risk of bias.
CONCLUSIONS
Risk of selection bias was generally high in RCTs of yoga; although the situation has improved since the publication of the revised CONSORT statement 2001. Pre-CONSORT RCTs and those published in journals without impact factor should be handled with increased care; although risk of bias is unlikely to distort the RCTs' conclusions.
Topics: Complementary Therapies; Humans; Logistic Models; Meditation; Randomized Controlled Trials as Topic; Risk; Selection Bias; Yoga
PubMed: 26629905
DOI: 10.1371/journal.pone.0144125 -
The Cochrane Database of Systematic... Jun 2019Frequent consumption of excess amounts of sugar-sweetened beverages (SSB) is a risk factor for obesity, type 2 diabetes, cardiovascular disease and dental caries....
BACKGROUND
Frequent consumption of excess amounts of sugar-sweetened beverages (SSB) is a risk factor for obesity, type 2 diabetes, cardiovascular disease and dental caries. Environmental interventions, i.e. interventions that alter the physical or social environment in which individuals make beverage choices, have been advocated as a means to reduce the consumption of SSB.
OBJECTIVES
To assess the effects of environmental interventions (excluding taxation) on the consumption of sugar-sweetened beverages and sugar-sweetened milk, diet-related anthropometric measures and health outcomes, and on any reported unintended consequences or adverse outcomes.
SEARCH METHODS
We searched 11 general, specialist and regional databases from inception to 24 January 2018. We also searched trial registers, reference lists and citations, scanned websites of relevant organisations, and contacted study authors.
SELECTION CRITERIA
We included studies on interventions implemented at an environmental level, reporting effects on direct or indirect measures of SSB intake, diet-related anthropometric measures and health outcomes, or any reported adverse outcome. We included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-after (CBA) and interrupted-time-series (ITS) studies, implemented in real-world settings with a combined length of intervention and follow-up of at least 12 weeks and at least 20 individuals in each of the intervention and control groups. We excluded studies in which participants were administered SSB as part of clinical trials, and multicomponent interventions which did not report SSB-specific outcome data. We excluded studies on the taxation of SSB, as these are the subject of a separate Cochrane Review.
DATA COLLECTION AND ANALYSIS
Two review authors independently screened studies for inclusion, extracted data and assessed the risks of bias of included studies. We classified interventions according to the NOURISHING framework, and synthesised results narratively and conducted meta-analyses for two outcomes relating to two intervention types. We assessed our confidence in the certainty of effect estimates with the GRADE framework as very low, low, moderate or high, and presented 'Summary of findings' tables.
MAIN RESULTS
We identified 14,488 unique records, and assessed 1030 in full text for eligibility. We found 58 studies meeting our inclusion criteria, including 22 RCTs, 3 NRCTs, 14 CBA studies, and 19 ITS studies, with a total of 1,180,096 participants. The median length of follow-up was 10 months. The studies included children, teenagers and adults, and were implemented in a variety of settings, including schools, retailing and food service establishments. We judged most studies to be at high or unclear risk of bias in at least one domain, and most studies used non-randomised designs. The studies examine a broad range of interventions, and we present results for these separately.Labelling interventions (8 studies): We found moderate-certainty evidence that traffic-light labelling is associated with decreasing sales of SSBs, and low-certainty evidence that nutritional rating score labelling is associated with decreasing sales of SSBs. For menu-board calorie labelling reported effects on SSB sales varied.Nutrition standards in public institutions (16 studies): We found low-certainty evidence that reduced availability of SSBs in schools is associated with decreased SSB consumption. We found very low-certainty evidence that improved availability of drinking water in schools and school fruit programmes are associated with decreased SSB consumption. Reported associations between improved availability of drinking water in schools and student body weight varied.Economic tools (7 studies): We found moderate-certainty evidence that price increases on SSBs are associated with decreasing SSB sales. For price discounts on low-calorie beverages reported effects on SSB sales varied.Whole food supply interventions (3 studies): Reported associations between voluntary industry initiatives to improve the whole food supply and SSB sales varied.Retail and food service interventions (7 studies): We found low-certainty evidence that healthier default beverages in children's menus in chain restaurants are associated with decreasing SSB sales, and moderate-certainty evidence that in-store promotion of healthier beverages in supermarkets is associated with decreasing SSB sales. We found very low-certainty evidence that urban planning restrictions on new fast-food restaurants and restrictions on the number of stores selling SSBs in remote communities are associated with decreasing SSB sales. Reported associations between promotion of healthier beverages in vending machines and SSB intake or sales varied.Intersectoral approaches (8 studies): We found moderate-certainty evidence that government food benefit programmes with restrictions on purchasing SSBs are associated with decreased SSB intake. For unrestricted food benefit programmes reported effects varied. We found moderate-certainty evidence that multicomponent community campaigns focused on SSBs are associated with decreasing SSB sales. Reported associations between trade and investment liberalisation and SSB sales varied.Home-based interventions (7 studies): We found moderate-certainty evidence that improved availability of low-calorie beverages in the home environment is associated with decreased SSB intake, and high-certainty evidence that it is associated with decreased body weight among adolescents with overweight or obesity and a high baseline consumption of SSBs.Adverse outcomes reported by studies, which may occur in some circumstances, included negative effects on revenue, compensatory SSB consumption outside school when the availability of SSBs in schools is reduced, reduced milk intake, stakeholder discontent, and increased total energy content of grocery purchases with price discounts on low-calorie beverages, among others. The certainty of evidence on adverse outcomes was low to very low for most outcomes.We analysed interventions targeting sugar-sweetened milk separately, and found low- to moderate-certainty evidence that emoticon labelling and small prizes for the selection of healthier beverages in elementary school cafeterias are associated with decreased consumption of sugar-sweetened milk. We found low-certainty evidence that improved placement of plain milk in school cafeterias is not associated with decreasing sugar-sweetened milk consumption.
AUTHORS' CONCLUSIONS
The evidence included in this review indicates that effective, scalable interventions addressing SSB consumption at a population level exist. Implementation should be accompanied by high-quality evaluations using appropriate study designs, with a particular focus on the long-term effects of approaches suitable for large-scale implementation.
Topics: Adolescent; Adult; Animals; Artificially Sweetened Beverages; Child; Commerce; Controlled Before-After Studies; Drinking Behavior; Drinking Water; Environment; Fast Foods; Food Supply; Fruit; Humans; Interrupted Time Series Analysis; Milk; Nutritive Value; Product Labeling; Randomized Controlled Trials as Topic; Schools; Selection Bias; Social Environment; Sugar-Sweetened Beverages; Young Adult
PubMed: 31194900
DOI: 10.1002/14651858.CD012292.pub2 -
Journal of the National Cancer... Jul 2020Low-dose, penetrating photon radiation exposure is ubiquitous, yet our understanding of cancer risk at low doses and dose rates derives mainly from high-dose studies....
BACKGROUND
Low-dose, penetrating photon radiation exposure is ubiquitous, yet our understanding of cancer risk at low doses and dose rates derives mainly from high-dose studies. Although a large number of low-dose cancer studies have been recently published, concern exists about the potential for confounding to distort findings. The aim of this study was to describe and assess the likely impact of confounding and selection bias within the context of a systematic review.
METHODS
We summarized confounding control methods for 26 studies published from 2006 to 2017 by exposure setting (environmental, medical, or occupational) and identified confounders of potential concern. We used information from these and related studies to assess evidence for confounding and selection bias. For factors in which direct or indirect evidence of confounding was lacking for certain studies, we used a theoretical adjustment to determine whether uncontrolled confounding was likely to have affected the results.
RESULTS
For medical studies of childhood cancers, confounding by indication (CBI) was the main concern. Lifestyle-related factors were of primary concern for environmental and medical studies of adult cancers and for occupational studies. For occupational studies, other workplace exposures and healthy worker survivor bias were additionally of interest. For most of these factors, however, review of the direct and indirect evidence suggested that confounding was minimal. One study showed evidence of selection bias, and three occupational studies did not adjust for lifestyle or healthy worker survivor bias correlates. Theoretical adjustment for three factors (smoking and asbestos in occupational studies and CBI in childhood cancer studies) demonstrated that these were unlikely to explain positive study findings due to the rarity of exposure (eg, CBI) or the relatively weak association with the outcome (eg, smoking or asbestos and all cancers).
CONCLUSION
Confounding and selection bias are unlikely to explain the findings from most low-dose radiation epidemiology studies.
Topics: Asbestos; Bias; Confounding Factors, Epidemiologic; Epidemiologic Studies; Humans; Occupational Exposure; Selection Bias; Smoking
PubMed: 32657349
DOI: 10.1093/jncimonographs/lgaa008 -
Molecular Autism 2019Current global estimates suggest the proportion of the population with autism spectrum disorder (ASD) who have intellectual disability (ID) is approximately 50%. Our... (Meta-Analysis)
Meta-Analysis
BACKGROUND
Current global estimates suggest the proportion of the population with autism spectrum disorder (ASD) who have intellectual disability (ID) is approximately 50%. Our objective was to ascertain the existence of selection bias due to under-inclusion of populations with ID across all fields of autism research. A sub-goal was to evaluate inconsistencies in reporting of findings.
METHODS
This review covers all original research published in 2016 in autism-specific journals with an impact factor greater than 3. Across 301 included studies, 100,245 participants had ASD. A random effects meta-analysis was used to estimate the proportion of participants without ID. Selection bias was defined as where more than 75% of participants did not have ID.
RESULTS
Meta-analysis estimated 94% of all participants identified as being on the autism spectrum in the studies reviewed did not have ID (95% CI 0.91-0.97). Eight out of ten studies demonstrated selection bias against participants with ID. The reporting of participant characteristics was generally poor: information about participants' intellectual ability was absent in 38% of studies ( = 114). Where there was selection bias on ID, only 31% of studies mentioned lack of generalisability as a limitation.
CONCLUSIONS
We found selection bias against ID throughout all fields of autism research. We recommend transparent reporting about ID and strategies for inclusion for this much marginalised group.
Topics: Autism Spectrum Disorder; Biomedical Research; Humans; Intellectual Disability; Patient Selection; Selection Bias
PubMed: 30867896
DOI: 10.1186/s13229-019-0260-x -
The Lancet Regional Health. Western... Jun 2023Previous systematic reviews naïvely combined biased effects of screening radiography or endoscopy observed in studies with various designs. We aimed to synthesize...
BACKGROUND
Previous systematic reviews naïvely combined biased effects of screening radiography or endoscopy observed in studies with various designs. We aimed to synthesize currently available comparative data on gastric cancer mortality in healthy, asymptomatic adults by explicitly classifying the screening effects through study designs and types of intervention effects.
METHODS
We searched multiple databases through October 31, 2022 for this systematic review and meta-analysis. Studies of any design that compared gastric cancer mortality among radiographic or endoscopic screening and no screening in a community-dwelling adult population were included. The method included a duplicate assessment of eligibility, double extraction of summary data, and validity assessment using the Risk Of Bias In Non-randomized Studies of Interventions tool. Bayesian three-level hierarchical random-effects meta-analysis synthesized data corrected for self-selection bias on the relative risk (RR) for per-protocol (PP) and intention-to-screen (ITS) effects. The study registration number at PROSPERO is CRD42021277126.
FINDINGS
We included seven studies in which a screening program was newly introduced (median attendance rate, 31%; at moderate-to-critical risk of bias), and seven cohort and eight case-control studies with ongoing screening programs (median attendance rate, 21%; all at critical risk of bias); thus, data of 1,667,117 subjects were included. For the PP effect, the average risk reduction was significant for endoscopy (RR 0.52; 95% credible interval: 0.39-0.79) but nonsignificant for radiography (0.80; 0.60-1.06). The ITS effect was not significant for both radiography (0.98; 0.86-1.09) and endoscopy (0.94; 0.71-1.28). The magnitude of the effects depended on the assumptions for the self-selection bias correction. Restricting the scope to East Asian studies only did not change the results.
INTERPRETATION
In limited-quality observational evidence from high-prevalence regions, screening reduced gastric cancer mortality; however, the effects diminished at a program level.
FUNDING
National Cancer Center Japan; and Japan Agency for Medical Research and Development.
PubMed: 37424675
DOI: 10.1016/j.lanwpc.2023.100741 -
LGBT Health Jul 2019The majority of research on sexual minority individuals relies on nonprobability community venue samples. These samples are prone to selection bias; however, empirical...
The majority of research on sexual minority individuals relies on nonprobability community venue samples. These samples are prone to selection bias; however, empirical syntheses of evidence of these biases are not available. We conducted, therefore, a systematic review of published sexual minority health research to summarize methods used to identify characteristics and health outcomes found to differ in nonprobability samples. We searched five health and social science databases to identify observational studies that included a nonprobability sexual minority community sample and applied an empirical method to infer selection bias. We extracted data regarding sociodemographic characteristics, behaviors, and health outcomes and examined whether the nonprobability sample was found to differ disproportionately (over- or underrepresenting the characteristic) based on appropriate statistical tests ( < 0.05 was considered statistically significant). A total of 21 studies were included. Nineteen studies used a cross-sample comparison, one used time/location sampling weights, and one used frequency of venue attendance adjustment to assess differences. The evidence was mostly consistent, that is, associations were in the same direction in >50% of studies examined, for nine variables. Nonprobability community venue samples tended to overrepresent sexual minority individuals with the following characteristics: higher income, current employment, lesbian/gay-identified, greater number of sex partners, past-year sexually transmitted infection diagnosis, suicidal ideation, alcohol use, and substance use; nonprobability community venue samples tended to underrepresent married/partnered sexual minority individuals. This review provides a nuanced empirical picture of aggregate differences in sample characteristics presumed to threaten the validity of nonprobability sexual minority community venue studies, and highlights feasible methods that can be applied to future studies to add specificity to researchers' description of selection biases.
Topics: Data Interpretation, Statistical; Humans; Observational Studies as Topic; Selection Bias; Sexual and Gender Minorities; Sexuality
PubMed: 31135260
DOI: 10.1089/lgbt.2018.0241 -
Tobacco Induced Diseases 2023Cigarette smoking is a well-known cancer-causing behavior and a leading cause of death from cancer. However, according to previously published research and... (Review)
Review
INTRODUCTION
Cigarette smoking is a well-known cancer-causing behavior and a leading cause of death from cancer. However, according to previously published research and meta-analyses, cigarette smoking has a significant inverse association with prostate cancer incidence. Therefore, this study aims to investigate this association based on updated evidence by conducting a systematic review and meta-analysis.
METHODS
A search for relevant articles was performed in PubMed and Scopus databases to obtain the pooled relative risk (RR) and the corresponding 95% confidence intervals (CIs) for the risk of prostate cancer incidence among smokers compared to non-smokers. Our search was limited to prospective cohort studies.
RESULTS
A total of 17 cohort studies were included in the systematic review. Fifteen studies were included in the meta-analysis and showed that cigarette smoking has an inverse association with prostate cancer incidence with a relative risk of 0.84 (95% CI: 0.78-0.91). From all cohorts included in this systematic review, five studies examined the association between current smokers and the risk of death from prostate cancer. Therefore, a meta-analysis of these cohort studies was performed and showed that current smokers had a 42% higher risk of death from prostate cancer when compared to non-smokers with a relative risk of 1.42 (95% CI: 1.20-1.68).
CONCLUSIONS
Data from observational studies suggest that cigarette smoking has an inverse association with prostate cancer incidence. However, smokers have an increased risk of death from prostate cancer. Important to realize that this lower risk for smokers might be attributed to low prostate cancer screening uptake among smokers, misclassification bias, or selection bias from the included original studies. In summary, our results indicate that the incidence of prostate cancer is lower among smokers. Nevertheless, smokers who develop the disease have a significantly worse prognosis.
PubMed: 36762260
DOI: 10.18332/tid/157231 -
Preventive Medicine Jun 2016Overweight and obesity are among the most important modifiable risk factors for chronic diseases and premature death. The aim of this review was to systematically assess... (Meta-Analysis)
Meta-Analysis Review
INTRODUCTION
Overweight and obesity are among the most important modifiable risk factors for chronic diseases and premature death. The aim of this review was to systematically assess and analyze the effects of yoga on weight-related outcomes.
METHODS
Medline/PubMed, Scopus, and the Cochrane Library were screened through March 2015 for randomized controlled trials on yoga for weight-related outcomes in the general population or overweight/obese individuals. Risk of bias was assessed using the Cochrane risk of bias tool on the following domains: selection bias, performance bias, detection bias, attrition bias, reporting bias, and other bias.
RESULTS
Out of 445 records identified during literature search, 30 trials with a total of 2173 participants were included. No effects on weight, body mass index, body fat percentage or waist circumference were found. In studies with healthy adult participants an effect of yoga compared to usual care was found regarding waist/hip ratio (SMD=--1.00; 95% CI=--1.44, -0.55; p<0.001). In studies with overweight/obese participants only, effects relative to usual care were found for body mass index (SMD=-0.99; 95% CI=-1.67, -0.31; p=0.004). Effects however were not robust against selection bias; and publication bias could not be ruled out. No intervention-related adverse events were reported.
CONCLUSIONS
Despite methodological drawbacks, yoga can be preliminarily considered a safe and effective intervention to reduce body mass index in overweight or obese individuals.
Topics: Body Weight; Health Status; Humans; Obesity; Recreation; Risk Factors; Weight Loss; Yoga
PubMed: 27058944
DOI: 10.1016/j.ypmed.2016.03.013 -
Obstetrics and Gynecology Jul 2022To assess whether antenatal corticosteroid treatment is associated with improved neonatal outcomes in twins. (Meta-Analysis)
Meta-Analysis
OBJECTIVE
To assess whether antenatal corticosteroid treatment is associated with improved neonatal outcomes in twins.
DATA SOURCES
We searched MEDLINE, PubMed, EMBASE, and the Cochrane Library, from inception through August 12, 2021. We did not search ClinicalTrials.gov because our inclusion criteria were restricted to nonrandomized studies.
METHODS OF STUDY SELECTION
Records (n=7,802) were screened in Rayyan by two independent reviewers. We included all nonrandomized studies that compared antenatal corticosteroid treatment with no treatment in twins. Our outcomes of interest were neonatal mortality, respiratory distress syndrome (RDS), intraventricular hemorrhage, bronchopulmonary dysplasia, necrotizing enterocolitis, periventricular leukomalacia, and retinopathy of prematurity.
TABULATION, INTEGRATION, AND RESULTS
We used the ROBINS-I tool (Risk Of Bias In Non-randomised Studies - of Interventions) to assess risk of bias. We performed random-effects meta-analyses of estimates from studies without critical risk of bias due to confounding, and reported summary adjusted odds ratios (aORs) and 95% CIs. Eighteen cohort studies (that reported on 33,152 neonates) met inclusion criteria. Sixteen studies restricted to preterm gestational ages, and 11 defined exposed neonates based on an optimal corticosteroid administration-to-birth interval. Limitations due to confounding and selection bias were common concerns for the risk-of-bias assessments (n=14 at critical or higher), and 11 studies did not account for clustering within twin pairs in their analyses. All included studies had at least moderate risk of bias. Meta-analysis showed that antenatal corticosteroid administration was associated with lower odds of neonatal mortality (aOR 0.59, 95% CI 0.43-0.80, I2 69%, five studies, 20,312 neonates) and RDS (aOR 0.70, 95% CI 0.57-0.86, I2 67%, seven studies, 20,628 neonates) in twins. Results were inconclusive for the other outcomes.
CONCLUSION
Evidence from nonrandomized studies suggests antenatal corticosteroids are associated with lower incidence of neonatal mortality and RDS in twins.
SYSTEMATIC REVIEW REGISTRATION
PROSPERO, CRD42020205302.
Topics: Adrenal Cortex Hormones; Bronchopulmonary Dysplasia; Child; Female; Humans; Infant, Newborn; Infant, Newborn, Diseases; Pregnancy; Prenatal Care; Respiratory Distress Syndrome, Newborn; Twins
PubMed: 35849452
DOI: 10.1097/AOG.0000000000004835