-
PloS One Aug 2008The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a... (Review)
Review
BACKGROUND
The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention.
METHODOLOGY/PRINCIPAL FINDINGS
We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.
CONCLUSIONS
Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Topics: Clinical Trials as Topic; Cohort Studies; Humans; Patient Selection; Publication Bias; Publishing; Randomized Controlled Trials as Topic; Treatment Outcome
PubMed: 18769481
DOI: 10.1371/journal.pone.0003081 -
Annals of the Royal College of Surgeons... Jul 2018Introduction Surgeon-specific outcome data, or consultant outcome publication, refers to public access to named surgeon procedural outcomes. Consultant outcome... (Review)
Review
Introduction Surgeon-specific outcome data, or consultant outcome publication, refers to public access to named surgeon procedural outcomes. Consultant outcome publication originates from cardiothoracic surgery, having been introduced to US and UK surgery in 1991 and 2005, respectively. It has been associated with an improvement in patient outcomes. However, there is concern that it may also have led to changes in surgeon behaviour. This review assesses the literature for evidence of risk-averse behaviour, upgrading of patient risk factors and cessation of low-volume or poorly performing surgeons. Materials and methods A systematic literature review of Embase and Medline databases was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines. Original studies including data on consultant outcome publication and its potential effect on surgeon behaviour were included. Results Twenty-five studies were identified from the literature search. Studies suggesting the presence of risk-averse behaviour and upgrading of risk factors tended to be survey based, with studies contrary to these findings using recognised regional and national databases. Discussion and conclusion Our review includes instances of consultant outcome publication leading to risk-averse behaviour, upgrading of risk factors and cessation of low-volume or poorly performing surgeons. As UK data on consultant outcome publication matures, further research is essential to ensure that high-risk patients are not inappropriately turned down for surgery.
Topics: Humans; Outcome Assessment, Health Care; Patient Selection; Practice Patterns, Physicians'; Publishing; Quality Improvement; Risk Assessment; Risk-Taking; Surgeons; United Kingdom; United States
PubMed: 29962298
DOI: 10.1308/rcsann.2018.0052 -
PloS One 2013The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a... (Meta-Analysis)
Meta-Analysis Review
BACKGROUND
The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making.
METHODOLOGY/PRINCIPAL FINDINGS
In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.
CONCLUSIONS
This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Topics: Cohort Studies; Humans; Outcome Assessment, Health Care; Publication Bias; Publications; Randomized Controlled Trials as Topic
PubMed: 23861749
DOI: 10.1371/journal.pone.0066844 -
Eating Behaviors Aug 2023Dimensions of perfectionism are associated with the onset and maintenance of eating disorder pathology in both clinical and non-clinical samples. The aim of this study... (Meta-Analysis)
Meta-Analysis Review
OBJECTIVE
Dimensions of perfectionism are associated with the onset and maintenance of eating disorder pathology in both clinical and non-clinical samples. The aim of this study was to conduct a systematic review and meta-analysis of the association between perfectionism and eating disorders in adults.
METHOD
A literature search was conducted using the PsycINFO, Medline, Scopus, Embase, Web of Science and ProQuest databases. Ninety-five studies met the inclusion criteria and included a total sample of 32,840 participants (clinical eating disorder diagnosis N = 2414, non-clinical N = 30,428). Correlation coefficients (r) for the association between eating disorders and perfectionism were pooled. A meta-analysis to determine the association between two dimensions of perfectionism and eating disorder symptoms was conducted. Subgroup analyses were conducted with studies using clinical samples, and studies using the Eating Disorder Examination Questionnaire.
RESULTS
The pooled effect size for the association between perfectionistic concerns and eating disorder symptoms was r = 0.33 [0.30, 0.37]; and r = 0.20 [0.14, 0.25] for the association between perfectionistic strivings and eating disorder symptoms. In the clinical subgroup analyses the effect sizes were r = 0.40 [0.22, 0.58]; and r = 0.35 [0.26, 0.44] respectively. Medium to high heterogeneity was identified across all subgroup analyses and a significant level of publication bias was also identified.
DISCUSSION
The findings indicate both perfectionistic strivings and perfectionistic concerns have significant associations with eating disorders, and further support the argument that both dimensions of perfectionism are important factors in the prevention and treatment of eating disorders.
Topics: Adult; Female; Humans; Male; Feeding and Eating Disorders; Perfectionism; Publication Bias
PubMed: 37327637
DOI: 10.1016/j.eatbeh.2023.101769 -
Medicine Mar 2018The utilization and impact of the studies published using the National Cancer Database (NCDB) is currently unclear. In this study, we aim to characterize the published... (Meta-Analysis)
Meta-Analysis Review
BACKGROUND
The utilization and impact of the studies published using the National Cancer Database (NCDB) is currently unclear. In this study, we aim to characterize the published studies, and identify relatively unexplored areas for future investigations.
METHODS
A literature search was performed using PubMed in January 2017 to identify all papers published using NCDB data. Characteristics of the publications were extracted. Citation frequencies were obtained through the Web of Science.
RESULTS
Three hundred 2 articles written by 230 first authors met the inclusion criteria. The number of publications grew exponentially since 2013, with 108 articles published in 2016. Articles were published in 86 journals. The majority of the published papers focused on digestive system cancer, while bone and joints, eye and orbit, myeloma, mesothelioma, and Kaposi Sarcoma were never studied. Thirteen institutions in the United States were associated with more than 5 publications. The papers have been cited for a total of 9858 times since the publication of the first paper in 1992. Frequently appearing keywords congregated into 3 clusters: "demographics," "treatments and survival," and "statistical analysis method." Even though the main focuses of the articles captured a extremely wide range, they can be classified into 2 main categories: survival analysis and characterization. Other focuses include database(s) analysis and/or comparison, and hospital reporting.
CONCLUSION
The surging interest in the use of NCDB is accompanied by unequal utilization of resources by individuals and institutions. Certain areas were relatively understudied and should be further explored.
Topics: Databases, Factual; Health Resources; Humans; Journal Impact Factor; Neoplasms; Publications; United States
PubMed: 29489679
DOI: 10.1097/MD.0000000000009823 -
Annual Review of Public Health 2006The systematic review "movement" that has transformed medical journal reports of clinical trials and reviews of clinical trials has taken hold in public health, with the... (Review)
Review
The systematic review "movement" that has transformed medical journal reports of clinical trials and reviews of clinical trials has taken hold in public health, with the most recent milestone, the publication of the first edition of The Guide to Community Health Services in 2005. In this paper we define and distinguish current terms, point out important resources for systematic reviews, describe the impact of systematic review on the quality of primary studies and summaries of the evidence, and provide perspectives on the promise of systematic reviews for shaping the agenda for public health research. Several pitfalls are discussed, including a false sense of rigor implied by the terms "systematic review" and "meta-analysis" and substantial variation in the validity of claims that a particular intervention is "evidence based," and the difficulty of translating conclusions from systematic reviews into public health advocacy and practice.
Topics: Evidence-Based Medicine; Health Knowledge, Attitudes, Practice; Health Policy; Humans; Journalism, Medical; Meta-Analysis as Topic; Public Health; Public Health Practice; Review Literature as Topic; United States
PubMed: 16533110
DOI: 10.1146/annurev.publhealth.27.021405.102239 -
Nursing Outlook 2015Systematic reviews (SRs) and meta-analyses (MAs) of nursing interventions have become increasingly popular in China. This review provides the first examination of... (Review)
Review
OBJECTIVES
Systematic reviews (SRs) and meta-analyses (MAs) of nursing interventions have become increasingly popular in China. This review provides the first examination of epidemiological characteristics of these SRs as well as compliance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses and Assessment of Multiple Systematic Reviews guidelines. The purpose of this study was to examine epidemiologic and reporting characteristics as well as the methodologic quality of SRs and MAs of nursing interventions published in Chinese journals.
METHODS
Four Chinese databases were searched (the Chinese Biomedicine Literature Database, Chinese Scientific Journal Full-text Database, Chinese Journal Full-text Database, and Wanfang Database) for SRs and MAs of nursing intervention from inception through June 2013. Data were extracted into Excel (Microsoft, Redmond, WA). The Assessment of Multiple Systematic Reviews and Preferred Reporting Items for Systematic Reviews and Meta-analyses checklists were used to assess methodologic quality and reporting characteristics, respectively.
RESULTS
A total of 144 SRs were identified, most (97.2%) of which used "systematic review" or "meta-analyses" in the titles. None of the reviews had been updated. Nearly half (41%) were written by nurses, and more than half (61%) were reported in specialist journals. The most common conditions studied were endocrine, nutritional and metabolic diseases, and neoplasms. Most (70.8%) reported information about quality assessment, whereas less than half (25%) reported assessing for publication bias. None of the reviews reported a conflict of interest.
CONCLUSIONS
Although many SRs of nursing interventions have been published in Chinese journals, the quality of these reviews is of concern. As a potential key source of information for nurses and nursing administrators, not only were many of these reviews incomplete in the information they provided, but also some results were misleading. Improving the quality of SRs of nursing interventions conducted and published by nurses in China is urgently needed in order to increase the value of these studies.
Topics: China; Humans; Meta-Analysis as Topic; Nursing; Periodicals as Topic; Publishing; Quality Control; Review Literature as Topic
PubMed: 26187084
DOI: 10.1016/j.outlook.2014.11.020 -
Health Technology Assessment... Feb 2010To identify and appraise empirical studies on publication and related biases published since 1998; to assess methods to deal with publication and related biases; and to... (Review)
Review
OBJECTIVES
To identify and appraise empirical studies on publication and related biases published since 1998; to assess methods to deal with publication and related biases; and to examine, in a random sample of published systematic reviews, measures taken to prevent, reduce and detect dissemination bias.
DATA SOURCES
The main literature search, in August 2008, covered the Cochrane Methodology Register Database, MEDLINE, EMBASE, AMED and CINAHL. In May 2009, PubMed, PsycINFO and OpenSIGLE were also searched. Reference lists of retrieved studies were also examined.
REVIEW METHODS
In Part I, studies were classified as evidence or method studies and data were extracted according to types of dissemination bias or methods for dealing with it. Evidence from empirical studies was summarised narratively. In Part II, 300 systematic reviews were randomly selected from MEDLINE and the methods used to deal with publication and related biases were assessed.
RESULTS
Studies with significant or positive results were more likely to be published than those with non-significant or negative results, thereby confirming findings from a previous HTA report. There was convincing evidence that outcome reporting bias exists and has an impact on the pooled summary in systematic reviews. Studies with significant results tended to be published earlier than studies with non-significant results, and empirical evidence suggests that published studies tended to report a greater treatment effect than those from the grey literature. Exclusion of non-English-language studies appeared to result in a high risk of bias in some areas of research such as complementary and alternative medicine. In a few cases, publication and related biases had a potentially detrimental impact on patients or resource use. Publication bias can be prevented before a literature review (e.g. by prospective registration of trials), or detected during a literature review (e.g. by locating unpublished studies, funnel plot and related tests, sensitivity analysis modelling), or its impact can be minimised after a literature review (e.g. by confirmatory large-scale trials, updating the systematic review). The interpretation of funnel plot and related statistical tests, often used to assess publication bias, was often too simplistic and likely misleading. More sophisticated modelling methods have not been widely used. Compared with systematic reviews published in 1996, recent reviews of health-care interventions were more likely to locate and include non-English-language studies and grey literature or unpublished studies, and to test for publication bias.
CONCLUSIONS
Dissemination of research findings is likely to be a biased process, although the actual impact of such bias depends on specific circumstances. The prospective registration of clinical trials and the endorsement of reporting guidelines may reduce research dissemination bias in clinical research. In systematic reviews, measures can be taken to minimise the impact of dissemination bias by systematically searching for and including relevant studies that are difficult to access. Statistical methods can be useful for sensitivity analyses. Further research is needed to develop methods for qualitatively assessing the risk of publication bias in systematic reviews, and to evaluate the effect of prospective registration of studies, open access policy and improved publication guidelines.
Topics: Bias; Biomedical Research; Evidence-Based Medicine; Humans; Information Dissemination; Publication Bias; Review Literature as Topic
PubMed: 20181324
DOI: 10.3310/hta14080 -
The Cochrane Database of Systematic... Jan 2021Smoking is a leading cause of disease and death worldwide. In people who smoke, quitting smoking can reverse much of the damage. Many people use behavioural... (Meta-Analysis)
Meta-Analysis
BACKGROUND
Smoking is a leading cause of disease and death worldwide. In people who smoke, quitting smoking can reverse much of the damage. Many people use behavioural interventions to help them quit smoking; these interventions can vary substantially in their content and effectiveness.
OBJECTIVES
To summarise the evidence from Cochrane Reviews that assessed the effect of behavioural interventions designed to support smoking cessation attempts and to conduct a network meta-analysis to determine how modes of delivery; person delivering the intervention; and the nature, focus, and intensity of behavioural interventions for smoking cessation influence the likelihood of achieving abstinence six months after attempting to stop smoking; and whether the effects of behavioural interventions depend upon other characteristics, including population, setting, and the provision of pharmacotherapy. To summarise the availability and principal findings of economic evaluations of behavioural interventions for smoking cessation, in terms of comparative costs and cost-effectiveness, in the form of a brief economic commentary.
METHODS
This work comprises two main elements. 1. We conducted a Cochrane Overview of reviews following standard Cochrane methods. We identified Cochrane Reviews of behavioural interventions (including all non-pharmacological interventions, e.g. counselling, exercise, hypnotherapy, self-help materials) for smoking cessation by searching the Cochrane Library in July 2020. We evaluated the methodological quality of reviews using AMSTAR 2 and synthesised data from the reviews narratively. 2. We used the included reviews to identify randomised controlled trials of behavioural interventions for smoking cessation compared with other behavioural interventions or no intervention for smoking cessation. To be included, studies had to include adult smokers and measure smoking abstinence at six months or longer. Screening, data extraction, and risk of bias assessment followed standard Cochrane methods. We synthesised data using Bayesian component network meta-analysis (CNMA), examining the effects of 38 different components compared to minimal intervention. Components included behavioural and motivational elements, intervention providers, delivery modes, nature, focus, and intensity of the behavioural intervention. We used component network meta-regression (CNMR) to evaluate the influence of population characteristics, provision of pharmacotherapy, and intervention intensity on the component effects. We evaluated certainty of the evidence using GRADE domains. We assumed an additive effect for individual components.
MAIN RESULTS
We included 33 Cochrane Reviews, from which 312 randomised controlled trials, representing 250,563 participants and 845 distinct study arms, met the criteria for inclusion in our component network meta-analysis. This represented 437 different combinations of components. Of the 33 reviews, confidence in review findings was high in four reviews and moderate in nine reviews, as measured by the AMSTAR 2 critical appraisal tool. The remaining 20 reviews were low or critically low due to one or more critical weaknesses, most commonly inadequate investigation or discussion (or both) of the impact of publication bias. Of note, the critical weaknesses identified did not affect the searching, screening, or data extraction elements of the review process, which have direct bearing on our CNMA. Of the included studies, 125/312 were at low risk of bias overall, 50 were at high risk of bias, and the remainder were at unclear risk. Analyses from the contributing reviews and from our CNMA showed behavioural interventions for smoking cessation can increase quit rates, but effectiveness varies on characteristics of the support provided. There was high-certainty evidence of benefit for the provision of counselling (odds ratio (OR) 1.44, 95% credibility interval (CrI) 1.22 to 1.70, 194 studies, n = 72,273) and guaranteed financial incentives (OR 1.46, 95% CrI 1.15 to 1.85, 19 studies, n = 8877). Evidence of benefit remained when removing studies at high risk of bias. These findings were consistent with pair-wise meta-analyses from contributing reviews. There was moderate-certainty evidence of benefit for interventions delivered via text message (downgraded due to unexplained statistical heterogeneity in pair-wise comparison), and for the following components where point estimates suggested benefit but CrIs incorporated no clinically significant difference: individual tailoring; intervention content including motivational components; intervention content focused on how to quit. The remaining intervention components had low-to very low-certainty evidence, with the main issues being imprecision and risk of bias. There was no evidence to suggest an increase in harms in groups receiving behavioural support for smoking cessation. Intervention effects were not changed by adjusting for population characteristics, but data were limited. Increasing intensity of behavioural support, as measured through the number of contacts, duration of each contact, and programme length, had point estimates associated with modestly increased chances of quitting, but CrIs included no difference. The effect of behavioural support for smoking cessation appeared slightly less pronounced when people were already receiving smoking cessation pharmacotherapies.
AUTHORS' CONCLUSIONS
Behavioural support for smoking cessation can increase quit rates at six months or longer, with no evidence that support increases harms. This is the case whether or not smoking cessation pharmacotherapy is also provided, but the effect is slightly more pronounced in the absence of pharmacotherapy. Evidence of benefit is strongest for the provision of any form of counselling, and guaranteed financial incentives. Evidence suggested possible benefit but the need of further studies to evaluate: individual tailoring; delivery via text message, email, and audio recording; delivery by lay health advisor; and intervention content with motivational components and a focus on how to quit. We identified 23 economic evaluations; evidence did not consistently suggest one type of behavioural intervention for smoking cessation was more cost-effective than another. Future reviews should fully consider publication bias. Tools to investigate publication bias and to evaluate certainty in CNMA are needed.
Topics: Adult; Bayes Theorem; Behavior Therapy; Bias; Counseling; Exercise; Female; Humans; Hypnosis; Male; Middle Aged; Network Meta-Analysis; Publication Bias; Randomized Controlled Trials as Topic; Self Care; Smoking Cessation; Systematic Reviews as Topic; Time Factors; Young Adult
PubMed: 33411338
DOI: 10.1002/14651858.CD013229.pub2 -
Annals of Vascular Surgery Sep 2022Artificial intelligence (AI) and machine learning (ML) have seen increasingly intimate integration with medicine and healthcare in the last 2 decades. The objective of... (Review)
Review
BACKGROUND
Artificial intelligence (AI) and machine learning (ML) have seen increasingly intimate integration with medicine and healthcare in the last 2 decades. The objective of this study was to summarize all current applications of AI and ML in the vascular surgery literature and to conduct a bibliometric analysis of published studies.
METHODS
A comprehensive literature search was conducted through Embase, MEDLINE, and Ovid HealthStar from inception until February 19, 2021. Reporting of this study was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Title and abstract screening, full-text screening, and data extraction were conducted in duplicate. Data extracted included study metadata, the clinical area of study within vascular surgery, type of AI/ML method used, dataset, and the application of AI/ML. Publishing journals were classified as having either a clinical scope or technical scope. The author academic background was classified as clinical, nonclinical (e.g., engineering), or both, depending on author affiliation.
RESULTS
The initial search identified 7,434 studies, of which 249 were included for a final analysis. The rate of publications is exponentially increasing, with 158 (63%) studies being published in the last 5 years alone. Studies were most commonly related to carotid artery disease (118, 47%), abdominal aortic aneurysms (51, 20%), and peripheral arterial disease (26, 10%). Study authors employed an average of 1.50 (range: 1-6) distinct AI methods in their studies. The application of AI/ML methods broadly related to predictive models (54, 22%), image segmentation (49, 19.4%), diagnostic methods (46, 18%), or multiple combined applications (91, 37%). The most commonly used AI/ML methods were artificial neural networks (155/378 use cases, 41%), support vector machines (64, 17%), k-nearest neighbors algorithm (26, 7%), and random forests (23, 6%). Datasets to which these AI/ML methods were applied frequently involved ultrasound images (87, 35%), computed tomography (CT) images (42, 17%), clinical data (34, 14%), or multiple datasets (36, 14%). Overall, 22 (9%) studies were published in journals specific to vascular surgery, with the majority (147/249, 59%) being published in journals with a scope related to computer science or engineering. Among 1,576 publishing authors, 46% had exclusively a clinical background, 48% a nonclinical background, and 5% had both a clinical and nonclinical background.
CONCLUSIONS
There is an exponentially growing body of literature describing the use of AI and ML in vascular surgery. There is a focus on carotid artery disease and abdominal aortic disease, with many other areas of vascular surgery under-represented. Neural networks and support vector machines composed most AI methods in the literature. As AI/ML continue to see expanded applications in the field, it is important that vascular surgeons appreciate its potential and limitations. In addition, as it sees increasing use, there is a need for clinicians with expertise in AI/ML methods who can optimize its transition into daily practice.
Topics: Artificial Intelligence; Bibliometrics; Carotid Artery Diseases; Humans; Machine Learning; Treatment Outcome; Vascular Surgical Procedures
PubMed: 35339595
DOI: 10.1016/j.avsg.2022.03.019