-
Journal of Rehabilitation Medicine Oct 2022The European Academy of Rehabilitation Medicine (EARM) held a debate in Hannover, Germany, on 1st of September 2016 on the pros and cons of randomized controlled...
Pros and Cons of Randomized Controlled Trials and Benchmarking Controlled Trials in Rehabilitation: An Academic Debate within the European Academy of Rehabilitation Medicine.
The European Academy of Rehabilitation Medicine (EARM) held a debate in Hannover, Germany, on 1st of September 2016 on the pros and cons of randomized controlled trials (RCTs) and observational effectiveness studies (benchmarking controlled trials; BCTs). The debate involved a chairperson, a person presenting the substance of the debate, an opponent, and a rapporteur. The academicians participated in the discussion. Eight propositions and proposed statements formed the substance of the debate. There was agreement that a study question should be the starting point of an effectiveness study, and not the study method, i.e. RCT or BCT. The term "benchmarking" was questioned: does it mean market-oriented medicine? It was clarified that benchmarking refers to the methodological features of this study design: there must always be a comparison between peers. It was agreed that BCTs might be better than RCTs for use in rehabilitation studies, in which one often needs multi-centred studies, such as in the assessment of the effectiveness of pathways when there is complexity of processes, health systems, organizational issues, structures and facilities; or where interactions between therapists, doctors and patients differ between centres; and when assessing the implementation of rehabilitation. In addition, BCTs may deal with ethical issues, e.g. the acceptability of interventions, more easily than RCTs. Recommendations regarding the different approaches (RCTs or BCTs) should be provided by the scientific rehabilitation societies. Concern over the validity of BCTs was considered justified, as the validity criteria of BCTs cover all those related to RCTs and include the risk of selection bias between treatment arms. Appropriate description of the essentials of the study object, including adequate description of how the interventions were actualized in comparison to the study plan, are essential features for a valid and generalizable study for both RCTs and BCTs. BCTs are necessary to widen the evidence-base of effectiveness in rehabilitation. It was suggested that the rehabilitation field should support the concept of BCTs. It was proposed that education regarding BCTs is indicated, and stakeholders need to be convinced that BCTs are a valid alternative to RCTs. EARM and other physical and rehabilitation medicine (PRM) bodies could advance the use of BCTs for clinical and health policy decision-making.
Topics: Benchmarking; Germany; Humans; Physical and Rehabilitation Medicine; Randomized Controlled Trials as Topic
PubMed: 35797064
DOI: 10.2340/jrm.v54.2511 -
The International Journal of... Aug 2021Mathematical modelling is increasingly used to inform budgeting and strategic decision-making by national TB programmes. Despite the importance of these decisions, there...
Mathematical modelling is increasingly used to inform budgeting and strategic decision-making by national TB programmes. Despite the importance of these decisions, there is currently no mechanism to review and confirm the appropriateness of modelling analyses. We have developed a benchmarking, reporting, and review (BRR) approach and accompanying tools to allow constructive review of country-level TB modelling applications. This approach has been piloted in five modelling applications and the results of this study have been used to revise and finalise the approach. The BRR approach consists of 1) quantitative benchmarks against which model assumptions and results can be compared, 2) standardised reporting templates and review criteria, and 3) a multi-stage review process providing feedback to modellers during the application, as well as a summary evaluation after completion. During the pilot, use of the tools prompted important changes in the approaches taken to modelling. The pilot also identified issues beyond the scope of a review mechanism, such as a lack of empirical evidence and capacity constraints. This approach provides independent evaluation of the appropriateness of modelling decisions during the course of an application, allowing meaningful changes to be made before results are used to inform decision-making. The use of these tools can improve the quality and transparency of country-level TB modelling applications.
Topics: Humans; Benchmarking; Models, Theoretical; Tuberculosis
PubMed: 34330345
DOI: 10.5588/ijtld.21.0127 -
IEEE Transactions on Bio-medical... Mar 2022Machine learning techniques used in computer-aided medical image analysis usually suffer from the domain shift problem caused by different distributions between... (Review)
Review
Machine learning techniques used in computer-aided medical image analysis usually suffer from the domain shift problem caused by different distributions between source/reference data and target data. As a promising solution, domain adaptation has attracted considerable attention in recent years. The aim of this paper is to survey the recent advances of domain adaptation methods in medical image analysis. We first present the motivation of introducing domain adaptation techniques to tackle domain heterogeneity issues for medical image analysis. Then we provide a review of recent domain adaptation models in various medical image analysis tasks. We categorize the existing methods into shallow and deep models, and each of them is further divided into supervised, semi-supervised and unsupervised methods. We also provide a brief summary of the benchmark medical image datasets that support current domain adaptation research. This survey will enable researchers to gain a better understanding of the current status, challenges and future directions of this energetic research field.
Topics: Benchmarking; Image Processing, Computer-Assisted; Machine Learning
PubMed: 34606445
DOI: 10.1109/TBME.2021.3117407 -
PloS One 2022This paper offers a comprehensive analysis model for android malware. The model presents the essential factors affecting the analysis results of android malware that are...
This paper offers a comprehensive analysis model for android malware. The model presents the essential factors affecting the analysis results of android malware that are vision-based. Current android malware analysis and solutions might consider one or some of these factors while building their malware predictive systems. However, this paper comprehensively highlights these factors and their impacts through a deep empirical study. The study comprises 22 CNN (Convolutional Neural Network) algorithms, 21 of them are well-known, and one proposed algorithm. Additionally, several types of files are considered before converting them to images, and two benchmark android malware datasets are utilized. Finally, comprehensive evaluation metrics are measured to assess the produced predictive models from the security and complexity perspectives. Consequently, guiding researchers and developers to plan and build efficient malware analysis systems that meet their requirements and resources. The results reveal that some factors might significantly impact the performance of the malware analysis solution. For example, from a security perspective, the accuracy, F1-score, precision, and recall are improved by 131.29%, 236.44%, 192%, and 131.29%, respectively, when changing one factor and fixing all other factors under study. Similar results are observed in the case of complexity assessment, including testing time, CPU usage, storage size, and pre-processing speed, proving the importance of the proposed android malware analysis model.
Topics: Algorithms; Benchmarking; Neural Networks, Computer
PubMed: 35788205
DOI: 10.1371/journal.pone.0270647 -
Health Informatics Journal 2021The Movember funded TrueNTH Global Registry (TNGR) aims to improve care by collecting and analysing a consistent dataset to identify variation in disease management,...
BACKGROUND
The Movember funded TrueNTH Global Registry (TNGR) aims to improve care by collecting and analysing a consistent dataset to identify variation in disease management, benchmark care delivery in accordance with best practice guidelines and provide this information to those in a position to enact change. We discuss considerations of designing and implementing a quality of care report for TNGR.
METHODS
Eleven working group sessions were held prior to and as reports were being built with representation from clinicians, data managers and investigators contributing to TNGR. The aim of the meetings was to understand current data display approaches, share literature review findings and ideas for innovative approaches. Preferred displays were evaluated with two surveys (survey 1: 5 clinicians and 5 non-clinicians, 83% response rate; survey 2: 17 clinicians and 18 non-clinicians, 93% response rate).
RESULTS
Consensus on dashboard design and three data-display preferences were achieved. The dashboard comprised two performance summary charts; one summarising site's relative quality indicator (QI) performance and another to summarise data quality. Binary outcome QIs were presented as funnel plots. Patient-reported outcome measures of function score and the extent to which men were bothered by their symptoms were presented in bubble plots. Time series graphs were seen as providing important information to supplement funnel and bubble plots. R Markdown was selected as the software program principally because of its excellent analytic and graph display capacity, open source licensing model and the large global community sharing program code enhancements.
CONCLUSIONS
International collaboration in creating and maintaining clinical quality registries has allowed benchmarking of process and outcome measures on a large scale. A registry report system was developed with stakeholder engagement to produce dynamic reports that provide user-specific feedback to 132 participating sites across 13 countries.
Topics: Benchmarking; Delivery of Health Care; Humans; Male; Quality Indicators, Health Care; Registries; Surveys and Questionnaires
PubMed: 34082597
DOI: 10.1177/14604582211015704 -
Genome Biology Dec 2021Single-cell RNA-sequencing (scRNA-seq) technologies and associated analysis methods have rapidly developed in recent years. This includes preprocessing methods, which...
BACKGROUND
Single-cell RNA-sequencing (scRNA-seq) technologies and associated analysis methods have rapidly developed in recent years. This includes preprocessing methods, which assign sequencing reads to genes to create count matrices for downstream analysis. While several packaged preprocessing workflows have been developed to provide users with convenient tools for handling this process, how they compare to one another and how they influence downstream analysis have not been well studied.
RESULTS
Here, we systematically benchmark the performance of 10 end-to-end preprocessing workflows (Cell Ranger, Optimus, salmon alevin, alevin-fry, kallisto bustools, dropSeqPipe, scPipe, zUMIs, celseq2, and scruff) using datasets yielding different biological complexity levels generated by CEL-Seq2 and 10x Chromium platforms. We compare these workflows in terms of their quantification properties directly and their impact on normalization and clustering by evaluating the performance of different method combinations. While the scRNA-seq preprocessing workflows compared vary in their detection and quantification of genes across datasets, after downstream analysis with performant normalization and clustering methods, almost all combinations produce clustering results that agree well with the known cell type labels that provided the ground truth in our analysis.
CONCLUSIONS
In summary, the choice of preprocessing method was found to be less important than other steps in the scRNA-seq analysis process. Our study comprehensively compares common scRNA-seq preprocessing workflows and summarizes their characteristics to guide workflow users.
Topics: Benchmarking; Cluster Analysis; Gene Expression Profiling; RNA-Seq; Sequence Analysis, RNA; Single-Cell Analysis; Software; Transcriptome; Workflow
PubMed: 34906205
DOI: 10.1186/s13059-021-02552-3 -
Neural Networks : the Official Journal... Jan 2022We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning...
We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark datasets. Our theoretical analysis shows that ICT corresponds to a certain type of data-adaptive regularization with unlabeled points which reduces overfitting to labeled points under high confidence values.
Topics: Algorithms; Benchmarking; Neural Networks, Computer; Supervised Machine Learning
PubMed: 34735894
DOI: 10.1016/j.neunet.2021.10.008 -
Molecular & Cellular Proteomics : MCP Apr 2023Immunopeptidomes are the peptide repertoires bound by the molecules encoded by the major histocompatibility complex [human leukocyte antigen (HLA) in humans]. These...
Immunopeptidomes are the peptide repertoires bound by the molecules encoded by the major histocompatibility complex [human leukocyte antigen (HLA) in humans]. These HLA-peptide complexes are presented on the cell surface for immune T-cell recognition. Immunopeptidomics denotes the utilization of tandem mass spectrometry to identify and quantify peptides bound to HLA molecules. Data-independent acquisition (DIA) has emerged as a powerful strategy for quantitative proteomics and deep proteome-wide identification; however, DIA application to immunopeptidomics analyses has so far seen limited use. Further, of the many DIA data processing tools currently available, there is no consensus in the immunopeptidomics community on the most appropriate pipeline(s) for in-depth and accurate HLA peptide identification. Herein, we benchmarked four commonly used spectral library-based DIA pipelines developed for proteomics applications (Skyline, Spectronaut, DIA-NN, and PEAKS) for their ability to perform immunopeptidome quantification. We validated and assessed the capability of each tool to identify and quantify HLA-bound peptides. Generally, DIA-NN and PEAKS provided higher immunopeptidome coverage with more reproducible results. Skyline and Spectronaut conferred more accurate peptide identification with lower experimental false-positive rates. All tools demonstrated reasonable correlations in quantifying precursors of HLA-bound peptides. Our benchmarking study suggests a combined strategy of applying at least two complementary DIA software tools to achieve the greatest degree of confidence and in-depth coverage of immunopeptidome data.
Topics: Humans; Benchmarking; Peptides; Histocompatibility Antigens Class I; Proteomics; Tandem Mass Spectrometry; Histocompatibility Antigens Class II
PubMed: 36796644
DOI: 10.1016/j.mcpro.2023.100515 -
Health Services Research Aug 2021
Topics: Accountable Care Organizations; Benchmarking; Efficiency, Organizational; Health Expenditures; Humans; Reimbursement, Incentive; United States
PubMed: 34105147
DOI: 10.1111/1475-6773.13689 -
Computers in Biology and Medicine Dec 2022U-Net includes encoder, decoder and skip connection structures. It has become the benchmark network in medical image segmentation. However, the direct fusion of...
BACKGROUND
U-Net includes encoder, decoder and skip connection structures. It has become the benchmark network in medical image segmentation. However, the direct fusion of low-level and high-level convolution features with semantic gaps by traditional skip connections may lead to problems such as fuzzy generated feature maps and target region segmentation errors.
OBJECTIVE
We use spatial enhancement filtering technology to compensate for the semantic gap and propose an enhanced dense U-Net (E-DU), aiming to apply it to multimodal medical image segmentation to improve the segmentation performance and efficiency.
METHODS
Before combining encoder and decoder features, we replace the traditional skip connection with a multiscale denoise enhancement (MDE) module. The encoder features need to be deeply convolved by the spatial enhancement filter and then combined with the decoder features. We propose a simple and efficient deep full convolution network structure E-DU, which can not only fuse semantically various features but also denoise and enhance the feature map.
RESULTS
We performed experiments on medical image segmentation datasets with seven image modalities and combined MDE with various baseline networks to perform ablation studies. E-DU achieved the best segmentation results on evaluation indicators such as DSC on the U-Net family, with DSC values of 97.78, 97.64, 95.31, 94.42, 94.93, 98.85, and 98.38 (%), respectively. The addition of the MDE module to the attention mechanism network improves segmentation performance and efficiency, reflecting its generalization performance. In comparison to advanced methods, our method is also competitive.
CONCLUSION
Our proposed MDE module has a good segmentation effect and operating efficiency, and it can be easily extended to multiple modal medical segmentation datasets. Our idea and method can achieve clinical multimodal medical image segmentation and make full use of image information to provide clinical decision support. It has great application value and promotion prospects.
Topics: Semantics; Neural Networks, Computer; Benchmarking
PubMed: 36395592
DOI: 10.1016/j.compbiomed.2022.106206