-
Medical Physics Nov 2017The purpose of this study was to apply statistical metrics to identify outliers and to investigate the impact of outliers on knowledge-based planning in radiation...
PURPOSE
The purpose of this study was to apply statistical metrics to identify outliers and to investigate the impact of outliers on knowledge-based planning in radiation therapy of pelvic cases. We also aimed to develop a systematic workflow for identifying and analyzing geometric and dosimetric outliers.
METHODS
Four groups (G1-G4) of pelvic plans were sampled in this study. These include the following three groups of clinical IMRT cases: G1 (37 prostate cases), G2 (37 prostate plus lymph node cases) and G3 (37 prostate bed cases). Cases in G4 were planned in accordance with dynamic-arc radiation therapy procedure and include 10 prostate cases in addition to those from G1. The workflow was separated into two parts: 1. identifying geometric outliers, assessing outlier impact, and outlier cleaning; 2. identifying dosimetric outliers, assessing outlier impact, and outlier cleaning. G2 and G3 were used to analyze the effects of geometric outliers (first experiment outlined below) while G1 and G4 were used to analyze the effects of dosimetric outliers (second experiment outlined below). A baseline model was trained by regarding all G2 cases as inliers. G3 cases were then individually added to the baseline model as geometric outliers. The impact on the model was assessed by comparing leverages of inliers (G2) and outliers (G3). A receiver-operating-characteristic (ROC) analysis was performed to determine the optimal threshold. The experiment was repeated by training the baseline model with all G3 cases as inliers and perturbing the model with G2 cases as outliers. A separate baseline model was trained with 32 G1 cases. Each G4 case (dosimetric outlier) was subsequently added to perturb the model. Predictions of dose-volume histograms (DVHs) were made using these perturbed models for the remaining 5 G1 cases. A Weighted Sum of Absolute Residuals (WSAR) was used to evaluate the impact of the dosimetric outliers.
RESULTS
The leverage of inliers and outliers was significantly different. The Area-Under-Curve (AUC) for differentiating G2 (outliers) from G3 (inliers) was 0.98 (threshold: 0.27) for the bladder and 0.81 (threshold: 0.11) for the rectum. For differentiating G3 (outlier) from G2 (inlier), the AUC (threshold) was 0.86 (0.11) for the bladder and 0.71 (0.11) for the rectum. Significant increase in WSAR was observed in the model with 3 dosimetric outliers for the bladder (P < 0.005 with Bonferroni correction), and in the model with only 1 dosimetric outlier for the rectum (P < 0.005).
CONCLUSIONS
We established a systematic workflow for identifying and analyzing geometric and dosimetric outliers, and investigated statistical metrics for outlier detection. Results validated the necessity for outlier detection and clean-up to enhance model quality in clinical practice.
Topics: Algorithms; Humans; Male; Organs at Risk; Pelvis; Prostatic Neoplasms; Radiometry; Radiotherapy Dosage; Radiotherapy Planning, Computer-Assisted
PubMed: 28869649
DOI: 10.1002/mp.12556 -
Heliyon Jan 2024This review aimed to harmoniously summarize and compare outlier rates for various cardiac troponin (cTn) assays, including high-sensitivity-cTn (hs-cTn) assays and... (Review)
Review
OBJECTIVES
This review aimed to harmoniously summarize and compare outlier rates for various cardiac troponin (cTn) assays, including high-sensitivity-cTn (hs-cTn) assays and contemporary cTn (generation of assays prior to hs-cTn ones) assays, from the published studies.
METHODS
The PRISMA guidelines were utilized to perform this systematic review. Five databases, including PubMed, Scopus, Embase, Cochrane Library, and Web of Science, were searched using specific keywords up to June 30th, 2023. Studies reporting specifically calculated outlier rates for cTn assays when conducting in-vitro diagnosis in human samples were included. Selected studies were then further assessed using the GRADE tool.
RESULTS
Thirteen studies were included. The data from the studies were summarized statistically in this review. The results showed substantial evidence of improved analytical robustness or reduced respective mean rates of outliers, critical outliers, and analytical outliers for hs-cTn assays (0.14 %, 0.18 %, and 0.18 %) compared to contemporary cTn assays (0.63 %, 0.71 %, and 0.50 %).
CONCLUSION
The findings offer promisingly provide a comprehensive reference for laboratory scientists and clinical staff in choosing the most suitable cTn assay for patient care regrading outlier rates. Besides, this review reveals the advancements of hs-cTn assays with lower outlier rates than contemporary cTn assays. The emerging challenges for continuously improving analytical robustness of cTn assays are also elaborated.
PubMed: 38205298
DOI: 10.1016/j.heliyon.2023.e23788 -
BMC Medical Informatics and Decision... Oct 2022Outliers and class imbalance in medical data could affect the accuracy of machine learning models. For physicians who want to apply predictive models, how to use the...
BACKGROUND
Outliers and class imbalance in medical data could affect the accuracy of machine learning models. For physicians who want to apply predictive models, how to use the data at hand to build a model and what model to choose are very thorny problems. Therefore, it is necessary to consider outliers, imbalanced data, model selection, and parameter tuning when modeling.
METHODS
This study used a joint modeling strategy consisting of: outlier detection and removal, data balancing, model fitting and prediction, performance evaluation. We collected medical record data for all ICH patients with admissions in 2017-2019 from Sichuan Province. Clinical and radiological variables were used to construct models to predict mortality outcomes 90 days after discharge. We used stacking ensemble learning to combine logistic regression (LR), random forest (RF), artificial neural network (ANN), support vector machine (SVM), and k-nearest neighbors (KNN) models. Accuracy, sensitivity, specificity, AUC, precision, and F1 score were used to evaluate model performance. Finally, we compared all 84 combinations of the joint modeling strategy, including training set with and without cross-validated committees filter (CVCF), five resampling techniques (random under-sampling (RUS), random over-sampling (ROS), adaptive synthetic sampling (ADASYN), Borderline synthetic minority oversampling technique (Borderline SMOTE), synthetic minority oversampling technique and edited nearest neighbor (SMOTEENN)) and no resampling, seven models (LR, RF, ANN, SVM, KNN, Stacking, AdaBoost).
RESULTS
Among 4207 patients with ICH, 2909 (69.15%) survived 90 days after discharge, and 1298 (30.85%) died within 90 days after discharge. The performance of all models improved with removing outliers by CVCF except sensitivity. For data balancing processing, the performance of training set without resampling was better than that of training set with resampling in terms of accuracy, specificity, and precision. And the AUC of ROS was the best. For seven models, the average accuracy, specificity, AUC, and precision of RF were the highest. Stacking performed best in F1 score. Among all 84 combinations of joint modeling strategy, eight combinations performed best in terms of accuracy (0.816). For sensitivity, the best performance was SMOTEENN + Stacking (0.662). For specificity, the best performance was CVCF + KNN (0.987). Stacking and AdaBoost had the best performances in AUC (0.756) and F1 score (0.602), respectively. For precision, the best performance was CVCF + SVM (0.938).
CONCLUSION
This study proposed a joint modeling strategy including outlier detection and removal, data balancing, model fitting and prediction, performance evaluation, in order to provide a reference for physicians and researchers who want to build their own models. This study illustrated the importance of outlier detection and removal for machine learning and showed that ensemble learning might be a good modeling strategy. Due to the low imbalanced ratio (IR, the ratio of majority class and minority class) in this study, we did not find any improvement in models with resampling in terms of accuracy, specificity, and precision, while ROS performed best on AUC.
Topics: Humans; Electronic Health Records; Reactive Oxygen Species; Machine Learning; Support Vector Machine; Cerebral Hemorrhage
PubMed: 36284327
DOI: 10.1186/s12911-022-02018-x -
Radiation Oncology (London, England) Oct 2022To save time and have more consistent contours, fully automatic segmentation of targets and organs at risk (OAR) is a valuable asset in radiotherapy. Though current deep...
AIMS
To save time and have more consistent contours, fully automatic segmentation of targets and organs at risk (OAR) is a valuable asset in radiotherapy. Though current deep learning (DL) based models are on par with manual contouring, they are not perfect and typical errors, as false positives, occur frequently and unpredictably. While it is possible to solve this for OARs, it is far from straightforward for target structures. In order to tackle this problem, in this study, we analyzed the occurrence and the possible dose effects of automated delineation outliers.
METHODS
First, a set of controlled experiments on synthetically generated outliers on the CT of a glioblastoma (GBM) patient was performed. We analyzed the dosimetric impact on outliers with different location, shape, absolute size and relative size to the main target, resulting in 61 simulated scenarios. Second, multiple segmentation models where trained on a U-Net network based on 80 training sets consisting of GBM cases with annotated gross tumor volume (GTV) and edema structures. On 20 test cases, 5 different trained models and a majority voting method were used to predict the GTV and edema. The amount of outliers on the predictions were determined, as well as their size and distance from the actual target.
RESULTS
We found that plans containing outliers result in an increased dose to healthy brain tissue. The extent of the dose effect is dependent on the relative size, location and the distance to the main targets and involved OARs. Generally, the larger the absolute outlier volume and the distance to the target the higher the potential dose effect. For 120 predicted GTV and edema structures, we found 1887 outliers. After construction of the planning treatment volume (PTV), 137 outliers remained with a mean distance to the target of 38.5 ± 5.0 mm and a mean size of 1010.8 ± 95.6 mm. We also found that majority voting of DL results is capable to reduce outliers.
CONCLUSIONS
This study shows that there is a severe risk of false positive outliers in current DL predictions of target structures. Additionally, these errors will have an evident detrimental impact on the dose and therefore could affect treatment outcome.
Topics: Humans; Radiotherapy Planning, Computer-Assisted; Glioblastoma; Organs at Risk; Radiotherapy Dosage; Radiotherapy, Intensity-Modulated
PubMed: 36273161
DOI: 10.1186/s13014-022-02137-9 -
... IEEE International Conference on... Dec 2022Outlier detection is a fundamental data analytics technique often used for many security applications. Numerous outlier detection techniques exist, and in most cases are...
Outlier detection is a fundamental data analytics technique often used for many security applications. Numerous outlier detection techniques exist, and in most cases are used to directly identify outliers without any interaction. Typically the underlying data used is often high dimensional and complex. Even though outliers may be identified, since humans can easily grasp low dimensional spaces, it is difficult for a security expert to understand/visualize why a particular event or record has been identified as an outlier. In this paper we study the extent to which outlier detection techniques work in smaller dimensions and how well dimensional reduction techniques still enable accurate detection of outliers. This can help us to understand the extent to which data can be visualized while still retaining the intrinsic outlyingness of the outliers.
PubMed: 38094985
DOI: 10.1109/tps-isa56441.2022.00028 -
Turkish Journal of Orthodontics Dec 2022To determine whether multiple siblings resemble one another in their craniofacial characteristics as measured on cephalometric radiographs.
OBJECTIVE
To determine whether multiple siblings resemble one another in their craniofacial characteristics as measured on cephalometric radiographs.
METHODS
This study was conducted retrospectively using the Forsyth Moorrees twin sample. A total of 32 families were included, each with ≥4 postpubertal siblings, totaling 142 subjects. Only 1 monozygotic twin was included per family. Headfilms were digitized, skeletal landmarks were located, and 6 parameters that indicated sagittal jaw relationships and vertical status were measured. Diverse statistical approaches were used. Dixon's Q-test detected outliers in a family for a given parameter. Manhattan Distance quantified similarity among siblings per parameter. Scatter plots visually displayed subject's measure relative to the mean and standard deviation of each parameter to assess the clinical relevance of the differences.
RESULTS
A total of 11 families (34.4%) had no outliers on any parameter, 13 families (40.6%) had outliers on 1 parameter, and 8 families (25%) had outliers on ≥2 parameters. We identified 29 individuals with at least 1 outlying measure (20.4%). Among these, only 2 individuals (1.4%) were significantly different from their siblings for more than 1 measurement. Although the majority of the families did not demonstrate any statistical outlier, the ranges of the measurements were clinically relevant as they might suggest different treatment. For example, the mean range of SNB (Sella-Nasion-B point) angles was 7.23°, and the mean range of MPA was 9.42°.
CONCLUSION
Although families are generally not dissimilar in their craniofacial characteristics, measurements from siblings cannot be used to predict the measurements of another sibling in a clinically meaningful way.
PubMed: 36594544
DOI: 10.5152/TurkJOrthod.2022.21237 -
Annals of the New York Academy of... Sep 2020Convergent evolution, where independent lineages evolve similar phenotypes in response to similar challenges, can provide valuable insight into how selection operates... (Review)
Review
Convergent evolution, where independent lineages evolve similar phenotypes in response to similar challenges, can provide valuable insight into how selection operates and the limitations it encounters. However, it has only recently become possible to explore how convergent evolution is reflected at the genomic level. The overlapping outlier approach (OOA), where genome scans of multiple independent lineages are used to find outliers that overlap and therefore identify convergently evolving loci, is becoming popular. Here, we present a quantitative analysis of 34 studies that used this approach across many sampling designs, taxa, and sampling intensities. We found that OOA studies with increased biological sampling power within replicates have increased likelihood of finding overlapping, "convergent" signals of adaptation between them. When identifying convergent loci as overlapping outliers, it is tempting to assume that any false-positive outliers derived from individual scans will fail to overlap across replicates, but this cannot be guaranteed. We highlight how population demographics and genomic context can contribute toward both true convergence and false positives in OOA studies. We finish with an exploration of emerging methods that couple genome scans with phenotype and environmental measures, leveraging added information from genome data to more directly test hypotheses of the likelihood of convergent evolution.
Topics: Acclimatization; Adaptation, Physiological; Animals; Animals, Wild; Biological Evolution; Evolution, Molecular; Genome; Genomics
PubMed: 31241191
DOI: 10.1111/nyas.14177 -
PloS One 2024The detection of water quality indicators such as Temperature, pH, Turbidity, Conductivity, and TDS involves five national standard methods. Chemically based measurement...
The detection of water quality indicators such as Temperature, pH, Turbidity, Conductivity, and TDS involves five national standard methods. Chemically based measurement techniques may generate liquid residue, causing secondary pollution. The water quality monitoring and data analysis system can effectively address the issues that conventional methods require multiple pieces of equipment and repeated measurements. This paper analyzes the distribution characteristics of the historical data from five sensors at a specific time, displays them graphically in real time, and provides an early warning of exceeding the standard; It selects four water samples from different sections of the Li River, based on the national standard method, the average measurement errors of Temperature, PH, TDS, Conductivity and Turbidity are 0.98%, 2.23%, 2.92%, 3.05% and 3.98%.;It further uses the quartile method to analyze the outlier data over 100,000 records and five historical periods are selected. Experiment results show the system is relatively stable in measuring Temperature, PH and TDS, and the proportion of outlier is 0.42%, 0.84% and 1.24%. When Turbidity and Conductivity are measured, the proportion is 3.11% and 2.92%. In the experiment of using 7 methods to fill outlier, K nearest neighbor algorithm is better than others. The analysis of data trends, outliers, means, and extreme values assists in making decisions, such as updating and maintaining equipment, addressing extreme water quality situations, and enhancing regional water quality oversight.
Topics: Water Quality; Rivers; Environmental Monitoring; Fresh Water; Cluster Analysis
PubMed: 38498583
DOI: 10.1371/journal.pone.0299435 -
Finding the Genomic Basis of Local Adaptation: Pitfalls, Practical Solutions, and Future Directions.The American Naturalist Oct 2016Uncovering the genetic and evolutionary basis of local adaptation is a major focus of evolutionary biology. The recent development of cost-effective methods for... (Review)
Review
Uncovering the genetic and evolutionary basis of local adaptation is a major focus of evolutionary biology. The recent development of cost-effective methods for obtaining high-quality genome-scale data makes it possible to identify some of the loci responsible for adaptive differences among populations. Two basic approaches for identifying putatively locally adaptive loci have been developed and are broadly used: one that identifies loci with unusually high genetic differentiation among populations (differentiation outlier methods) and one that searches for correlations between local population allele frequencies and local environments (genetic-environment association methods). Here, we review the promises and challenges of these genome scan methods, including correcting for the confounding influence of a species' demographic history, biases caused by missing aspects of the genome, matching scales of environmental data with population structure, and other statistical considerations. In each case, we make suggestions for best practices for maximizing the accuracy and efficiency of genome scans to detect the underlying genetic basis of local adaptation. With attention to their current limitations, genome scan methods can be an important tool in finding the genetic basis of adaptive evolutionary change.
Topics: Adaptation, Physiological; Animals; Gene Frequency; Genetics, Population; Genome; Genomics; Selection, Genetic
PubMed: 27622873
DOI: 10.1086/688018 -
NeuroImage Feb 2017Even after thorough preprocessing and a careful time series analysis of functional magnetic resonance imaging (fMRI) data, artifact and other issues can lead to... (Review)
Review
Even after thorough preprocessing and a careful time series analysis of functional magnetic resonance imaging (fMRI) data, artifact and other issues can lead to violations of the assumption that the variance is constant across subjects in the group level model. This is especially concerning when modeling a continuous covariate at the group level, as the slope is easily biased by outliers. Various models have been proposed to deal with outliers including models that use the first level variance or that use the group level residual magnitude to differentially weight subjects. The most typically used robust regression, implementing a robust estimator of the regression slope, has been previously studied in the context of fMRI studies and was found to perform well in some scenarios, but a loss of Type I error control can occur for some outlier settings. A second type of robust regression using a heteroscedastic autocorrelation consistent (HAC) estimator, which produces robust slope and variance estimates has been shown to perform well, with better Type I error control, but with large sample sizes (500-1000 subjects). The Type I error control with smaller sample sizes has not been studied in this model and has not been compared to other modeling approaches that handle outliers such as FSL's Flame 1 and FSL's outlier de-weighting. Focusing on group level inference with a continuous covariate over a range of sample sizes and degree of heteroscedasticity, which can be driven either by the within- or between-subject variability, both styles of robust regression are compared to ordinary least squares (OLS), FSL's Flame 1, Flame 1 with outlier de-weighting algorithm and Kendall's Tau. Additionally, subject omission using the Cook's Distance measure with OLS and nonparametric inference with the OLS statistic are studied. Pros and cons of these models as well as general strategies for detecting outliers in data and taking precaution to avoid inflated Type I error rates are discussed.
Topics: Adolescent; Adult; Data Interpretation, Statistical; Decision Making; Female; Functional Neuroimaging; Humans; Magnetic Resonance Imaging; Male; Models, Statistical; Psychomotor Performance; Young Adult
PubMed: 28030782
DOI: 10.1016/j.neuroimage.2016.12.058