-
Trials Jun 2023Many randomized trials that aim to assess new or commonly used medical or surgical interventions have been so small that the validity of conclusions becomes questionable.
BACKGROUND
Many randomized trials that aim to assess new or commonly used medical or surgical interventions have been so small that the validity of conclusions becomes questionable.
METHODS
We illustrate the small trial problem using the power calculation of five Cochrane-reviewed studies that compared vertebroplasty versus placebo interventions. We discuss some of the reasons why the statistical admonition not to dichotomize continuous variables may not apply to the calculation of the number of patients required for trials to be meaningful.
RESULTS
Placebo-controlled vertebroplasty trials planned to recruit between 23 and 71 patients per group. Four of five studies used the standardized mean difference of a continuous pain variable (centimeters on the visual analog scale (VAS)) to plan implausibly small trials. What is needed is not a mean effect at the population level but a measure of efficacy at the patient level. Clinical practice concerns the care of individual patients that vary in many more respects than the variation around the mean of a single selected variable. The inference from trial to practice concerns the frequency of success of the experimental intervention performed one patient at a time. A comparison of the proportions of patients reaching a certain threshold is a more meaningful method that appropriately requires larger trials.
CONCLUSION
Most placebo-controlled vertebroplasty trials used comparisons of means of a continuous variable and were consequently very small. Randomized trials should instead be large enough to account for the diversity of future patients and practices. They should offer an evaluation of a clinically meaningful number of interventions performed in various contexts. Implications of this principle are not specific to placebo-controlled surgical trials. Trials designed to inform practice require a per-patient comparison of outcomes and the size of the trial should be planned accordingly.
Topics: Humans; Pain; Randomized Controlled Trials as Topic; Vertebroplasty
PubMed: 37349843
DOI: 10.1186/s13063-023-07348-3 -
Advances in Therapy Mar 2020Traditional efficacy double-blind randomised controlled trials (DBRCTs) measure the benefit a treatment produces under near-ideal test conditions in highly selected... (Review)
Review
Traditional efficacy double-blind randomised controlled trials (DBRCTs) measure the benefit a treatment produces under near-ideal test conditions in highly selected patient populations; however, the behaviour of patients and investigators in such trials is highly controlled, highly compliant and adherent, and non-representative of routine clinical practice. Pragmatic effectiveness trials measure the benefit a treatment produces in patients in everyday "real-world" practice. Ideally, effectiveness trials should recruit patients as similar as possible to those who will ultimately be prescribed the medicine, and create freedom within the study design to allow normal behaviours of patients and healthcare professionals (HCPs) to be expressed. The Salford Lung Study (SLS) was a world-first, prospective, phase III, pragmatic randomised controlled trial (RCT) programme in patients with chronic obstructive pulmonary disease and asthma to evaluate the effectiveness of a pre-licensed medication (fluticasone furoate/vilanterol) in real-world practice using electronic health records and through collaboratively engaging general practitioners and community pharmacists in clinical research. The real-world aspect of SLS was unique, requiring careful planning and attention to the goals of maximising the external validity of the trials while maintaining scientific rigour and securing suitable electronic processes for proper interpretation of safety data. Key learnings from SLS that may inform the design of future pragmatic effectiveness RCTs include: (1) ensuring the trial setting and operational infrastructure are aligned with routine clinical care; (2) recruiting a broad patient population with characteristics as close as possible to patients in routine clinical practice, to maximise the generalisability and applicability of trial results; (3) ensuring that patients and HCPs are suitably engaged in the trial, to maximise the chances of successful trial delivery; and (4) careful study design, incorporating outcomes of value to patients, HCPs, policymakers and payers, and using pre-planned analyses to address scientifically valid research hypotheses to ensure robustness of the trial data.
Topics: Androstadienes; Benzyl Alcohols; Bronchodilator Agents; Chlorobenzenes; Clinical Trials, Phase III as Topic; Double-Blind Method; Drug Therapy, Combination; Electronic Health Records; Health Behavior; Humans; Patient Selection; Product Surveillance, Postmarketing; Prospective Studies; Pulmonary Disease, Chronic Obstructive; Randomized Controlled Trials as Topic; Reproducibility of Results
PubMed: 31927698
DOI: 10.1007/s12325-019-01192-1 -
Malawi Medical Journal : the Journal of... Sep 2022Pragmatic clinical trials generally rely on real world data and have the potential to generate real world evidence. This approach arose from concerns that many trial...
BACKGROUND
Pragmatic clinical trials generally rely on real world data and have the potential to generate real world evidence. This approach arose from concerns that many trial results did not adequately inform real world practice. However, maintaining the real world setting during the conduct of a trial and ensuring adequate protection for research participants can be challenging. Best practices in research oversight for pragmatic clinical trials are nascent and underdeveloped, especially in developing countries.
METHODS
We use the PRECIS-2 tool to present a case study from Lilongwe in Malawi to describe ethical and regulatory challenges encountered during the conduct of a pragmatic trial and suggest possible solutions.
RESULTS
In this article, we highlight the following six issues: (1) one public facility hosting several pragmatic trials within the same period; (2) research participants refusing financial incentives; (3) inadequate infrastructure and high workload to conduct research; (4) silos among partner organisations involved in delivery of health care; (5) individuals influencing the implementation of revised national guidelines; (6) difficulties with access to electronic medical records.
CONCLUSION
Multiple stakeholder engagement is critical to the conduct of pragmatic trials, and even with careful stakeholder engagement, continuous monitoring by gatekeepers is essential. In the Malawian context, active engagement of the district research committees can complement the work of the research ethics committees (RECs).
Topics: Humans; Delivery of Health Care; Malawi; Pragmatic Clinical Trials as Topic; Organizational Case Studies
PubMed: 36406092
DOI: 10.4314/mmj.v34i3.12 -
BMC Medicine Oct 2022Pragmatic trials aim to generate evidence to directly inform patient, caregiver and health-system manager policies and decisions. Heterogeneity in patient...
BACKGROUND
Pragmatic trials aim to generate evidence to directly inform patient, caregiver and health-system manager policies and decisions. Heterogeneity in patient characteristics contributes to heterogeneity in their response to the intervention. However, there are many other sources of heterogeneity in outcomes. Based on the expertise and judgements of the authors, we identify different sources of clinical and methodological heterogeneity, which translate into heterogeneity in patient responses-some we consider as desirable and some as undesirable. For each of them, we discuss and, using real-world trial examples, illustrate how heterogeneity should be managed over the whole course of the trial.
MAIN TEXT
Heterogeneity in centres and patients should be welcomed rather than limited. Interventions can be flexible or tailored and control interventions are expected to reflect usual care, avoiding use of a placebo. Co-interventions should be allowed; adherence should not be enforced. All these elements introduce heterogeneity in interventions (experimental or control), which has to be welcomed because it mimics reality. Outcomes should be objective and possibly routinely collected; standardised assessment, blinding and adjudication should be avoided as much as possible because this is not how assessment would be done outside a trial setting. The statistical analysis strategy must be guided by the objective to inform decision-making, thus favouring the intention-to-treat principle. Pragmatic trials should consider including process analyses to inform an understanding of the trial results. Needed data to conduct these analyses should be collected unobtrusively. Finally, ethical principles must be respected, even though this may seem to conflict with goals of pragmatism; consent procedures could be incorporated in the flow of care.
Topics: Humans; Pragmatic Clinical Trials as Topic; Randomized Controlled Trials as Topic; Research Design
PubMed: 36303153
DOI: 10.1186/s12916-022-02569-w -
Trials Jun 2020Traditional randomised controlled trials remain the gold standard for improving clinical care but they do have their limitations, including their associated high costs,... (Review)
Review
BACKGROUND
Traditional randomised controlled trials remain the gold standard for improving clinical care but they do have their limitations, including their associated high costs, high failure rate and limited external validity. An alternative methodology is the newly defined, prospective, registry-based randomised controlled trial (RRCT), where treatment and outcome data is collected in an existing registry. This scoping review explores the current literature regarding RRCTs to help identify the key design elements of RRCTs and the characteristics of clinical registries on which they are reliant on.
METHODS
A scoping review methodology conducted in accordance with the Joanna Briggs Institute guidelines was performed. Four databases were searched for articles published from inception to June 2018: Medline; Embase; the Cumulative Index to Nursing and Allied Health Literature and; Scopus. The search strategy included MeSH and text words related to RRCT.
RESULTS
We identified 2369 articles of which 75 were selected for full-text screening. Of these, only 17 articles satisfied our inclusion criteria. All studies were published between 1996 and 2017 and all were investigator-initiated. Study designs were mainly multi-site comparative/effectiveness studies incorporating the use of disease registries (n = 8), procedure registries (n = 8) and a health services registry (n = 1). The low cost, reduced administrative burden and enhanced external validity of RRCTs make them an attractive research methodology which can be used to address questions of public health importance. We identified that that there are variable definitions of what constituted a RRCT and that issues related to ethical conduct and data integrity, completeness, timeliness, validation and endpoint adjudication need to be carefully addressed.
CONCLUSION
RRCTs potentially have an important role to play in informing best clinical practice and health policy. There are a number of issues that need to be addressed to optimise the utility of this approach, including establishing universally accepted criteria for the definition of a RRCT.
Topics: Humans; Multicenter Studies as Topic; Pragmatic Clinical Trials as Topic; Prospective Studies; Registries; Research Design
PubMed: 32571382
DOI: 10.1186/s13063-020-04459-z -
BMJ Open Sep 2023Incorrect penicillin allergy records are recognised as an important barrier to the safe treatment of infection and affect an estimated 2.7 million people in England....
Penicillin allergy status and its effect on antibiotic prescribing, patient outcomes and antimicrobial resistance (ALABAMA): protocol for a multicentre, parallel-arm, open-label, randomised pragmatic trial.
INTRODUCTION
Incorrect penicillin allergy records are recognised as an important barrier to the safe treatment of infection and affect an estimated 2.7 million people in England. Penicillin allergy records are associated with worse health outcome and antimicrobial resistance. The ALlergy AntiBiotics And Microbial resistAnce (ALABAMA) trial aims to determine if an intervention package, centred around a penicillin allergy assessment pathway (PAAP) initiated in primary care, is safe and effective in improving patient health outcomes and antibiotic prescribing.
METHODS AND ANALYSIS
The ALABAMA trial is a multicentre, parallel-arm, open-label, randomised pragmatic trial with a nested pilot study. Adults (≥18 years) with a penicillin allergy record and who have received antibiotics in the previous 24 months will be eligible for participation. Between 1592 and 2090 participants will be recruited from participating National Health Service general practices in England. Participants will be randomised to either usual care or intervention to undergo a pre-emptive PAAP using a 1:1 allocation ratio. The primary outcome measure is the percentage of treatment response failures within 28 days of an index prescription. 2090 and 1592 participants are estimated to provide 90% and 80% power, respectively, to detect a clinically important absolute difference of 7.9% in primary outcome at 1 year between groups. The trial includes a mixed-methods process evaluation and cost-effectiveness evaluation.
ETHICS AND DISSEMINATION
This trial has been approved by London Bridge Research Ethics Committee (ref: 19/LO/0176). It will be conducted in compliance with Good Clinical Practice guidelines according to the Declaration of Helsinki. Informed consent will be obtained from all subjects involved in the study. The primary trial results will be submitted for publication to an international, peer-reviewed journal.
TRIAL REGISTRATION
ISRCTN20579216.
Topics: Adult; Humans; Alabama; Anti-Bacterial Agents; Drug Hypersensitivity; Drug Resistance, Bacterial; Hypersensitivity; Multicenter Studies as Topic; Penicillins; Pilot Projects; Randomized Controlled Trials as Topic; State Medicine; Pragmatic Clinical Trials as Topic
PubMed: 37666558
DOI: 10.1136/bmjopen-2023-072253 -
Geriatric Nursing (New York, N.Y.) 2022Randomized controlled trials are considered the most rigorous research design in efficacy and effectiveness research; however, such trials present numerous challenges...
Randomized controlled trials are considered the most rigorous research design in efficacy and effectiveness research; however, such trials present numerous challenges that limit their applicability in real-world settings. As a consequence, pragmatic trials are increasingly viewed as a research design that overcomes some of these barriers with the potential to produce data that are more reproducible. Although pragmatic methodology in long-term care is receiving increasing attention as an approach to improve successful dissemination and implementation, pragmatic trials present complexities of their own. To address these complexities and related issues, experts with experience conducting pragmatic trials, developing nursing home policy, participating in advocacy efforts, and providing clinical care in long-term care settings participated in a virtual consensus conference funded by the National Institute on Aging in Spring 2021. Participants recommended 4 cross-cutting principles key to dissemination and implementation of pragmatic trial interventions: (1) engage stakeholders, (2) ensure diversity and inclusion, (3) assess organizational strain and readiness, and (4) learn from adaptations. Specifically related to implementation, participants provided 2 recommendations: (1) integrate interventions into existing workflows and (2) maintain agility and responsiveness. Finally, participants had 3 recommendations specific to dissemination: (1) package the message for the audience, (2) engage diverse audiences, and (3) apply dissemination and diffusion tools. Participants emphasized that implementation processes must be grounded in the perspectives of the people who will ultimately be responsible for implementing the intervention once it is proven to be effective. In addition, messaging must speak to long-term care staff and all others who have a stake in its outcomes. Although our understanding of dissemination and implementation strategies remains underdeveloped, this article is designed to guide long-term care researchers and community providers who are increasingly aware of the need for pragmatism in disseminating and implementing evidence-based care interventions.
Topics: Humans; Long-Term Care; Nursing Homes; Pragmatic Clinical Trials as Topic
PubMed: 35219533
DOI: 10.1016/j.gerinurse.2022.02.006 -
Contemporary Clinical Trials Feb 2023The GetReal Trial Tool is a decision support tool to assess the impact of design choices on generalizability of clinical trials to routine clinical practice, while...
BACKGROUND
The GetReal Trial Tool is a decision support tool to assess the impact of design choices on generalizability of clinical trials to routine clinical practice, while taking into account the risk of bias, precision, acceptability and operational feasibility. This study describes the validation of the GetReal Trial Tool.
METHODS
Twelve experts took part in the GetReal Trial tool validation using the protocols of 6 trials conducted with pragmatic elements. The tool entails 7 domains with a total of 43 questions. A pooled Kappa statistic (95% CI) using random effects model was estimated using Open Meta (analyst) software. The possible operational challenges were collated and discussed with the trialists that conducted the trials.
RESULTS
Agreement in the design choices made for the trial protocols was >50% for all the trials and all teams reached consensus during discussion. The pooled Kappa statistic (95% CI) was 0.236 (0.154-0.318). The GetReal Trial tool highlighted several operational challenges, of which almost half had been experienced previously by the trialists. Out of 25 additional operational challenges mentioned by the trialists, 76% were already highlighted by the tool. The tool was considered helpful to optimize trials right from the design stage.
CONCLUSION
The GetReal Trial Tool helps to scrutinize the choice of study design in the light of Real World Evidence generation. The tool identifies most of the operational challenges experienced by trialists to date. The tool serves the intended purpose of facilitating discussion and understanding more pragmatic design choices and their implications.
Topics: Humans; Research Design; Clinical Trials as Topic; Decision Support Techniques
PubMed: 36529438
DOI: 10.1016/j.cct.2022.107054 -
Trials Dec 2019All clinical trial investigators have ethical and regulatory obligations to monitor participant safety and trial integrity. Specific procedures for meeting these...
BACKGROUND
All clinical trial investigators have ethical and regulatory obligations to monitor participant safety and trial integrity. Specific procedures for meeting these obligations, however, may differ substantially between pragmatic trials and traditional explanatory clinical trials.
METHODS/RESULTS
Appropriate monitoring of clinical trials typically includes assessing rate of recruitment or enrollment; monitoring safe and effective delivery of study treatments; assuring that study staff act to minimize risks; monitoring quality and timeliness of study data; and considering interim analyses for early detection of benefit, harm, or futility. Each of these responsibilities applies to pragmatic clinical trials. Just as design of pragmatic trials typically involves specific and necessary departures from methods of explanatory clinical trials, appropriate monitoring of pragmatic trials typically requires specific departures from monitoring procedures used in explanatory clinical trials. We discuss how specific aspects of pragmatic trial design and operations influence selection of monitoring procedures and illustrate those choices using examples from three ongoing pragmatic trials conducted by the Mental Health Research Network.
CONCLUSIONS
Pragmatic trial investigators should not routinely adopt monitoring procedures used in explanatory clinical trials. Instead, investigators should consider core principles of trial monitoring and design monitoring procedures appropriate for each pragmatic trial.
Topics: Adolescent; Adult; Aged; Data Accuracy; Humans; Middle Aged; Pragmatic Clinical Trials as Topic; Research Design; Young Adult
PubMed: 31815644
DOI: 10.1186/s13063-019-3869-3 -
PloS One 2022Child and family social workers in the UK work closely with other agencies including schools and the police, and typically they are based in local authority offices....
BACKGROUND
Child and family social workers in the UK work closely with other agencies including schools and the police, and typically they are based in local authority offices. This study will evaluate the effectiveness of placing social workers in schools (SWIS) on the need for social care interventions. SWIS was piloted in three local authorities in 2018-2020, and findings from a feasibility study of the pilots suggests SWIS may operate through three key pathways: (1) by enhancing schools' response to safeguarding issues, (2) through increased collaboration between social workers, school staff, and parents, and (3) by improving relationships between social workers and young people.
METHODS
The study is a two-arm pragmatic cluster randomised controlled trial building on three feasibility studies which found SWIS to be promising. Social workers will work within secondary schools across local authorities in England. 280 mainstream secondary schools will be randomly allocated with a 1:1 ratio to SWIS or a comparison arm, which will be schools that continue as normal, without a social worker. The primary outcome will be the rate of Child Protection (Section 47) enquiries. Secondary outcomes will comprise rate of referrals to children's social care, rate of Child in Need (Section 17) assessments, days spent in care, and educational attendance and attainment. The study also includes an economic evaluation, and an implementation and process evaluation. Social care outcomes will be measured in July 2022, and educational outcomes will be measured in July 2023. Days in care will be measured at both time points.
DISCUSSION
Findings will explore the effectiveness and cost-effectiveness of SWIS on the need for social care interventions. A final report will be published in January 2024.
TRIAL REGISTRATION
The study was registered retrospectively with the International Standard Randomised Controlled Trial Number registry on 13.11.2020 (ISRCTN90922032).
Topics: Adolescent; Child; Cost-Benefit Analysis; Feasibility Studies; Humans; Parents; Pragmatic Clinical Trials as Topic; Randomized Controlled Trials as Topic; Retrospective Studies; Schools; Social Work
PubMed: 35679281
DOI: 10.1371/journal.pone.0265354