-
Journal of Clinical Epidemiology Jun 2024Given the key role that pilot and feasibility (PAF) trials play in addressing the challenges of surgical trials, adequate reporting completeness is essential. Our aim... (Meta-Analysis)
Meta-Analysis Review
OBJECTIVES
Given the key role that pilot and feasibility (PAF) trials play in addressing the challenges of surgical trials, adequate reporting completeness is essential. Our aim was to assess completeness of reporting and evaluate if the items of the Consolidated Standards of Reporting Trials (CONSORT) extension for PAF trials have been reported in surgical PAF trials.
STUDY DESIGN
This is a metaresearch study reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies were included if they were pilot or feasibility randomized trials evaluating a surgical intervention. The primary outcome was overall adherence to the CONSORT statement extension to randomized PAF trials checklist. A Poisson regression was performed to explore the association between research factors and reporting completeness.
SETTING
MEDLINE, Embase, and Cochrane Central Register of Controlled Trials were searched from January 1-December 31, 2011 and 2021.
RESULTS
After screening 1991 citations, 38 studies from 2011 to 34 studies from 2021 were included. The mean CONSORT reporting score across all included studies was 21.5 (standard deviation 6.3). After excluding items that were not applicable to all studies, a mean of 20.1 (standard deviation 6.1) of 34 items (0.59) were reported. Studies published in 2021 (vs 2011) did not have a greater number of CONSORT items reported (incidence rate ratio [IRR] 1.01, 95% confidence interval [CI] 0.89-1.15). Studies registered in a clinical trial registry (IRR 1.29, 95% CI 1.12-1.48) and randomizing more than 50 patients (IRR 1.16, 95% CI 1.04-1.30) were associated with more CONSORT items reported.
CONCLUSION
The reporting completeness of surgical PAF trials is poor and has not improved after the publication of the CONSORT extension.
Topics: Humans; Feasibility Studies; Checklist; Pilot Projects; Randomized Controlled Trials as Topic; Surgical Procedures, Operative; Research Design; Guideline Adherence
PubMed: 38548230
DOI: 10.1016/j.jclinepi.2024.111335 -
Journal of Clinical Epidemiology Jan 2023The aim of this study is to describe and pilot a novel method for continuously identifying newly published trials relevant to a systematic review, enabled by combining...
OBJECTIVES
The aim of this study is to describe and pilot a novel method for continuously identifying newly published trials relevant to a systematic review, enabled by combining artificial intelligence (AI) with human expertise.
STUDY DESIGN AND SETTING
We used RobotReviewer LIVE to keep a review of COVID-19 vaccination trials updated from February to August 2021. We compared the papers identified by the system with those found by the conventional manual process by the review team.
RESULTS
The manual update searches (last search date July 2021) retrieved 135 abstracts, of which 31 were included after screening (23% precision, 100% recall). By the same date, the automated system retrieved 56 abstracts, of which 31 were included after manual screening (55% precision, 100% recall). Key limitations of the system include that it is limited to searches of PubMed/MEDLINE, and considers only randomized controlled trial reports. We aim to address these limitations in future. The system is available as open-source software for further piloting and evaluation.
CONCLUSION
Our system identified all relevant studies, reduced manual screening work, and enabled rolling updates on publication of new primary research.
Topics: Humans; Artificial Intelligence; Pilot Projects; COVID-19 Vaccines; COVID-19; PubMed
PubMed: 36150548
DOI: 10.1016/j.jclinepi.2022.08.013 -
The International Journal of Behavioral... Feb 2020Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a... (Meta-Analysis)
Meta-Analysis
BACKGROUND
Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity.
METHODS
We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes.
RESULTS
A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively.
CONCLUSIONS
Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.
Topics: Bias; Diet; Exercise; Humans; Pediatric Obesity; Pilot Projects; Research Design; Sedentary Behavior; Sleep; Treatment Outcome; Weight Reduction Programs
PubMed: 32046735
DOI: 10.1186/s12966-020-0918-y -
International Journal of Tryptophan... 2023In this systematic review and meta-analysis, a normative dataset is generated from the published literature on the kynurenine pathway in control participants extracted...
In this systematic review and meta-analysis, a normative dataset is generated from the published literature on the kynurenine pathway in control participants extracted from case-control and methodological validation studies. Study characteristics were mapped, and studies were evaluated in terms of analytical rigour and methodological validation. Meta-analyses of variance between types of instruments, sample matrices and metabolites were conducted. Regression analyses were applied to determine the relationship between metabolite, sample matrix, biological sex, participant age and study age. The grand mean concentrations of tryptophan in the serum and plasma were 60.52 ± 15.38 μM and 51.45 ± 10.47 μM, respectively. The grand mean concentrations of kynurenine in the serum and plasma were 1.96 ± 0.51 μM and 1.82 ± 0.54 μM, respectively. Regional differences in metabolite concentrations were observed across America, Asia, Australia, Europe and the Middle East. Of the total variance within the data, mode of detection (MOD) accounted for up to 2.96%, sample matrix up to 3.23%, and their interaction explained up to 1.53%; the latter of which was determined to be negligible. This review was intended to inform future empirical research and method development studies and successfully synthesised pilot data. The pilot data reported in this study will inform future precision medicine initiatives aimed at targeting the kynurenine pathway by improving the availability and quality of normative data.
PubMed: 38034059
DOI: 10.1177/11786469231211184 -
Addiction (Abingdon, England) Apr 2022Up to 95% of pregnant women seeking treatment for alcohol and other drug (AOD) use smoke tobacco. Previous reviews indicate few effective smoking cessation treatments... (Review)
Review
BACKGROUND AND AIMS
Up to 95% of pregnant women seeking treatment for alcohol and other drug (AOD) use smoke tobacco. Previous reviews indicate few effective smoking cessation treatments for this group. This updated review aimed to identify and measure the efficacy of smoking cessation interventions trialled among pregnant women in AOD treatment settings who smoke tobacco.
METHODS
A narrative synthesis was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. Studies involving psychological, behavioural or pharmacological interventions used to treat tobacco use, including electronic nicotine delivery systems, for pregnant women of any age, who smoked tobacco and were seeking/receiving treatment, or in post-treatment recovery for AOD concerns, were reviewed. MEDLINE, PsycINFO, CINAHL, EMBASE and ProQuest databases, grey literature and reference lists were searched, and field experts were contacted for unpublished study data. The Effective Public Health Practice Project tool assessed study quality. The review was pre-registered with PROSPERO no. CRD42018108777.
RESULTS
Seven interventions (two randomised controlled trials, two single-arm pilot studies, two program evaluations and one causal comparative study) treating 875 women were identified. All were United States (US)-based and targeted women with drug dependence, but not alcohol dependence. Three interventions used contingency management, five provided behavioural counselling, and one offered nicotine replacement therapy. All reported reductions in cigarette consumption; one contingency management-based study demonstrated higher abstinence rates compared with controls at treatment-end that were not maintained at follow-up. Four of six studies were rated as methodologically weak and one unpublished study was not rated.
CONCLUSIONS
Conclusions about the efficacy of smoking interventions for pregnant women with alcohol and other drug concerns who also smoke tobacco are hindered by the paucity of available data and poor methodological quality of included studies.
Topics: Alcoholism; Behavior Therapy; Female; Humans; Male; Pregnancy; Pregnant Women; Randomized Controlled Trials as Topic; Smoking Cessation; Substance-Related Disorders; Tobacco Use Cessation Devices; United States
PubMed: 34374145
DOI: 10.1111/add.15663 -
Acta Paediatrica (Oslo, Norway : 1992) Jan 2021This study described the development, and pilot evaluation, of the Implementing Pediatric Advance Care Planning Toolkit (IMPACT).
AIM
This study described the development, and pilot evaluation, of the Implementing Pediatric Advance Care Planning Toolkit (IMPACT).
METHODS
Key elements of paediatric advance care planning (ACP) were defined using a systematic review, a survey of 168 paediatricians and qualitative studies of 13 children with life-limiting conditions, 20 parents and 18 paediatricians. Participants were purposively recruited from six Dutch university hospitals during September 2016 and November 2018. Key elements were translated into intervention components guided by theory. The acceptability of the content was evaluated by a qualitative pilot study during February and September 2019. This focused on 27 children with life-limiting conditions from hospitals, a hospice and home care, together with 41 parents, 11 physicians and seven nurses who cared for them.
RESULTS
IMPACT provided a holistic, caring approach to ACP, gave children a voice and cared for their parents. It provided information on ACP for families and clinicians, manuals to structure ACP conversations and training for clinicians in communication skills and supportive attitudes. The 53 pilot study participants felt that IMPACT was appropriate for paediatric ACP.
CONCLUSION
IMPACT was an appropriate intervention that supported a holistic approach towards paediatric ACP, focused on the child's perspective and provided care for their parents.
Topics: Advance Care Planning; Child; Communication; Humans; Parents; Pilot Projects; Qualitative Research
PubMed: 32434275
DOI: 10.1111/apa.15370 -
Campbell Systematic Reviews Jun 2021Agroforestry, the intentional integration of trees or other woody perennials with crops or livestock in production systems, is being widely promoted as a conservation... (Review)
Review
BACKGROUND
Agroforestry, the intentional integration of trees or other woody perennials with crops or livestock in production systems, is being widely promoted as a conservation and development tool to help meet the 2030 UN Sustainable Development Goals. Donors, governments, and nongovernmental organizations have invested significant time and resources into developing and promoting agroforestry policies and programs in low- and middle-income countries (LMICs) worldwide. While a large body of literature on the impacts of agroforestry in LMICs is available, the social-ecological impacts of agroforestry is less well-studied. This knowledge gap on the effectiveness of agroforestry interventions constrains possibilities for evidence-based policy and investment decisions to advance sustainable development objectives.
OBJECTIVES
The primary objective of this Campbell systematic review was to synthesize the available evidence on the impacts of agroforestry interventions in LMICs on agricultural productivity, ecosystem services, and human well-being. The secondary objectives were to identify key pathways through which agroforestry interventions lead to various outcomes and how the interventions affect different sub-groups of the population.
SEARCH METHODS
This review is based on a previously created evidence and gap map (EGM) of studies evaluating the impacts of agroforestry practices and interventions on agricultural productivity, ecosystem services, and human well-being. We included published and unpublished literature in the English language covering the period between 2000 and October 20, 2017. We searched six academic databases and 19 organization websites to identify potentially relevant studies. The search was conducted for our EGM in mid-2017, and we did not conduct an additional search for this systematic review.
SELECTION CRITERIA
We included randomized control trials (RCTs) and quasi-experimental studies assessing the effect of an agroforestry intervention on at least one outcome measure of agricultural productivity, ecosystem services, or human well-being for farmers and their farmland in LMICs. Agroforestry interventions include any program or policy designed to promote and support the adoption or maintenance of agroforestry practices, which include trees on farms, silvopasture, shade-grown crops, and homegardens with trees, among others. Moreover, the studies needed to include a nonagroforestry comparator, such as conventional agriculture or forestry systems or a before-after comparison.
DATA COLLECTION AND ANALYSIS
We used a standardized data extraction spreadsheet to extract details about each included study. We also used a standardized form to assess risk of bias for each of the included studies in this SR. Meta-analysis techniques were used to combine and synthesize effect size estimates for the outcomes measures that had sufficient data. We used a random effects models for the meta-analyses and use Hedge's (difference in means divided by the pooled standard deviation) to report effect size estimates. The outcomes without enough evidence for meta-analysis were discussed narratively.
MAIN RESULTS
We identified 11 studies across nine countries, all of which used quasi-experimental methods. Overall, the quality of the evidence base was assessed as being low. Studies were rated as having high or critical risk of bias if they failed to convincingly address more than one of the main potential sources of bias, namely selection bias, group equivalence, and spillover effects. Given the low number of studies and the high risk of bias of the evidence base, the results of this SR are limited and should be considered a baseline for future work. The results of the meta-analysis for impacts on yields indicated that agroforestry interventions overall may lead to a large, positive impact on yield (Hedge's = 1.16 [-0.35, 2.67] ( = .13)), though there was high heterogeneity in the results ( = 98.99%, = 2.94, Q(df = 4) = 370.7). There were positive yield impacts for soil fertility replenishment practices, including incorporating trees in agricultural fields and improved fallow practices in fields where there are severe soil fertility issues. In other cases, incorporating trees into the production system reduced productivity and took land out of production for conservation benefits. These systems generally used an incentive provision scheme to economically offset the reductions in yields. The result of the meta-analysis on income suggests that agroforestry interventions overall may lead to a small, positive impact on income (Hedge's = 0.12 [-0.06, 0.30] ( = .20)), with moderately high heterogeneity in the results ( = 75.29%, = 0.04, Q(df = 6) = 19.16). In cases where improvement yields were reported, there were generally attendant improvements in income. In the cases where payments were provided to offset the potential loss in yields, incomes also generally improved, though there were mixed results for the certification programs and the tenure security permitting scheme. One program, which study authors suggested may have been poorly targeted, had negative yield impacts. There was not enough comparable evidence to quantitatively synthesize the impacts of agroforestry interventions on nutrition and food security outcomes, though the results indicted positive or neutral impacts on dietary diversity and food intake were likely. Surprisingly, there was little evidence on the impacts of agroforestry interventions on environmental outcomes, and there was no consistency of environmental indicator variables used. However, what has been studied indicates that the environmental benefits are being achieved to at least some extent, consistent with the broader literature on agroforestry practices. The evidence base was insufficient to evaluate the interaction between environmental and social impacts. Several studies explicitly considered variable impacts across different population sub-groups, including differential impacts on small-holders versus large-holders, on woman-headed households versus male-headed households, and on richer groups versus poorer groups. Small-holder farmers typically experienced the most positive effect sizes due to the agroforestry interventions. Women and poorer groups had mixed outcomes relative to men and richer households, highlighting the importance of considering these groups in intervention design.
AUTHORS' CONCLUSIONS
There is limited evidence of the impacts of agroforestry interventions, restricting our ability to draw conclusions on the effect sizes of different intervention types. The existing evidence forms a baseline for future research and highlights the importance of considering equity and socio-economic factors in determining suitable intervention design. Some key implications for practice and policy include investing in programs that include pilot programs, funding for project evaluation, and that address key equity issues, such as targeting to smallholders, women, poor, and marginalized groups. Funding should also be given to implementing RCTs and more rigorous quasi-experimental impact evaluations of agroforestry interventions over longer time-periods to collect robust evidence of the effectiveness of various schemes promoting agroforestry practices.
PubMed: 37131923
DOI: 10.1002/cl2.1167 -
Evidence-based Mental Health Dec 2022Behavioural and cognitive interventions remain credible approaches in addressing loneliness and depression. There was a need to rapidly generate and assimilate... (Meta-Analysis)
Meta-Analysis Randomized Controlled Trial
Can we mitigate the psychological impacts of social isolation using behavioural activation? Long-term results of the UK BASIL urgent public health COVID-19 pilot randomised controlled trial and living systematic review.
BACKGROUND
Behavioural and cognitive interventions remain credible approaches in addressing loneliness and depression. There was a need to rapidly generate and assimilate trial-based data during COVID-19.
OBJECTIVES
We undertook a parallel pilot RCT of behavioural activation (a brief behavioural intervention) for depression and loneliness (Behavioural Activation in Social Isolation, the BASIL-C19 trial ISRCTN94091479). We also assimilate these data in a living systematic review (PROSPERO CRD42021298788) of cognitive and/or behavioural interventions.
METHODS
Participants (≥65 years) with long-term conditions were computer randomised to behavioural activation (n=47) versus care as usual (n=49). Primary outcome was PHQ-9. Secondary outcomes included loneliness (De Jong Scale). Data from the BASIL-C19 trial were included in a metanalysis of depression and loneliness.
FINDINGS
The 12 months adjusted mean difference for PHQ-9 was -0.70 (95% CI -2.61 to 1.20) and for loneliness was -0.39 (95% CI -1.43 to 0.65).The BASIL-C19 living systematic review (12 trials) found short-term reductions in depression (standardised mean difference (SMD)=-0.31, 95% CI -0.51 to -0.11) and loneliness (SMD=-0.48, 95% CI -0.70 to -0.27). There were few long-term trials, but there was evidence of some benefit (loneliness SMD=-0.20, 95% CI -0.40 to -0.01; depression SMD=-0.20, 95% CI -0.47 to 0.07).
DISCUSSION
We delivered a pilot trial of a behavioural intervention targeting loneliness and depression; achieving long-term follow-up. Living meta-analysis provides strong evidence of short-term benefit for loneliness and depression for cognitive and/or behavioural approaches. A fully powered BASIL trial is underway.
CLINICAL IMPLICATIONS
Scalable behavioural and cognitive approaches should be considered as population-level strategies for depression and loneliness on the basis of a living systematic review.
Topics: Humans; Behavior Therapy; COVID-19; Depression; Pilot Projects; Public Health; Social Isolation; United Kingdom
PubMed: 36223980
DOI: 10.1136/ebmental-2022-300530 -
PloS One 2019The effectiveness of breast cancer screening is still under debate. Our objective was to systematically review studies assessing personalized breast cancer screening...
BACKGROUND
The effectiveness of breast cancer screening is still under debate. Our objective was to systematically review studies assessing personalized breast cancer screening strategies based on women's individual risk and to conduct a risk of bias assessment.
METHODS
We followed the standard methods of The Cochrane Collaboration and PRISMA declaration and searched the MEDLINE, EMBASE and Clinical Trials databases for studies published in English. The quality of the studies was assessed using the ISPOR-AMCP-NPC Questionnaire and The Cochrane Risk of Bias Tool. Two independent reviewers screened full texts and evaluated the risk of bias.
RESULTS
Out of the 1533 initially retrieved citations, we included 13 studies. Three studies were randomized controlled trials, while nine were mathematical modeling studies, and one was an observational pilot study. The trials are in the recruitment phase and have not yet reported their results. All three trials used breast density and age to define risk groups, and two of them included family history, previous biopsies, and genetic information. Among the mathematical modeling studies, the main risk factors used to define risk groups were breast density, age, family history, and previous biopsies. Six studies used genetic information to define risk groups. The most common outcome measures were the gain in quality-adjusted life years (QALY), absolute costs, and incremental cost-effectiveness ratio (ICER), while the main outcome in the observational study was the detection rate. In all models, personalized screening strategies were shown to be effective. The randomized trials were of good quality. The modeling studies showed moderate risk of bias but there was wide variability across studies. The observational study showed a low risk of bias but its utility was moderate due to its pilot design and its relatively small scale.
CONCLUSIONS
There is some evidence of the effectiveness of screening personalization in terms of QUALYs and ICER from the modeling studies and the observational study. However, evidence is lacking on feasibility and acceptance by the target population.
REVIEW REGISTRATION
PROSPERO: CRD42018110483.
Topics: Breast Neoplasms; Cost-Benefit Analysis; Early Detection of Cancer; Female; Humans; Models, Theoretical; Observational Studies as Topic; Pilot Projects; Precision Medicine; Quality-Adjusted Life Years; Randomized Controlled Trials as Topic; Risk Assessment
PubMed: 31841563
DOI: 10.1371/journal.pone.0226352 -
JAMA Network Open Jul 2021Genomic newborn screening (gNBS) may optimize the health and well-being of children and families. Screening programs are required to be evidence based, acceptable, and...
IMPORTANCE
Genomic newborn screening (gNBS) may optimize the health and well-being of children and families. Screening programs are required to be evidence based, acceptable, and beneficial.
OBJECTIVES
To identify what has been discovered following the reporting of the first gNBS pilot projects and to provide a summary of key points for the design of gNBS.
EVIDENCE REVIEW
A systematic literature review was performed on April 14, 2021, identifying 36 articles that addressed the following questions: (1) what is the interest in and what would be the uptake of gNBS? (2) what diseases and genes should be included? (3) what is the validity and utility of gNBS? and (4) what are the ethical, legal, and social implications? Articles were only included if they generated new evidence; all opinion pieces were excluded.
FINDINGS
In the 36 articles included, there was high concordance, except for gene disease inclusion, which was highly variable. Key findings were the need for equitable access, appropriate educational materials, and informed and flexible consent. The process for selecting genes for testing should be transparent and reflect that parents value the certainty of prediction over actionability. Data should be analyzed in a way that minimizes uncertainty and incidental findings. The expansion of traditional newborn screening (tNBS) to identify more life-threatening and treatable diseases needs to be balanced against the complexity of consenting parents of newborns for genomic testing as well as the risk that overall uptake of tNBS may decline. The literature reflected that the right of a child to self-determination should be valued more than the possibility of the whole family benefiting from a newborn genomic test.
CONCLUSIONS AND RELEVANCE
The findings of this systematic review suggest that implementing gNBS will require a nuanced approach. There are gaps in our knowledge, such as the views of diverse populations, the capabilities of health systems, and health economic implications. It will be essential to rigorously evaluate outcomes and ensure programs can evolve to maximize benefit.
Topics: Genetic Testing; Humans; Infant, Newborn; Neonatal Screening; Program Development
PubMed: 34283230
DOI: 10.1001/jamanetworkopen.2021.14336