-
ISA Transactions Mar 2023This paper proposes a recursive filter for discrete-time linear dynamic systems subject to output outliers or heavy-tailed noises. First, we introduce a weight matrix in...
This paper proposes a recursive filter for discrete-time linear dynamic systems subject to output outliers or heavy-tailed noises. First, we introduce a weight matrix in the conventional MAP estimation. It is shown that this matrix plays an influential role in the innovation whitening and asymptotic variance of the modified MAP estimation and, consequently, can be used in outlier detection. Then, we propose two different constrained optimization problems to obtain this weight. These constraints, stemming from environmental noise characteristics, help to obtain the weight matrix more precisely, which increases the filtering performance significantly. In the first approach, we introduce a convex optimization problem to minimize the estimation upper bound of the error covariance matrix. The second approach converts the modified MAP estimation to a min-min optimization problem with a concave cost function. Consequently, to reduce the effect of outliers in estimation, a semidefinite program (SDP) is proposed for outlier detection. At last, simulation results show the effectiveness and verify the performance of the proposed filter for dynamic systems in the presence of measurement outliers.
PubMed: 36127183
DOI: 10.1016/j.isatra.2022.08.031 -
Journal of Experimental Psychology.... Nov 2023A methodological problem in most reaction time (RT) tasks is that some measured RTs may be outliers, being either too fast or too slow to reflect the task-related...
A methodological problem in most reaction time (RT) tasks is that some measured RTs may be outliers, being either too fast or too slow to reflect the task-related processing of interest. Numerous ad hoc procedures have been used to identify these outliers for exclusion from further analyses, but the accuracies of these methods have not been systematically compared. The present study compared the performance of 58 different outlier exclusion procedures (OEPs) using four huge datasets of real RTs. The results suggest that these OEPs are likely to do more harm than good, because they incorrectly identify outliers, increase noise, introduce bias, and generally reduce statistical power. The results suggest that RT researchers should not automatically apply any of these OEPs to clean their RT data prior to the main analyses. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
PubMed: 37498697
DOI: 10.1037/xge0001450 -
Critical Care Nursing Clinics of North... Mar 1999Outliers account for a large amount of technology utilization and resources consumed in acute care, despite accounting for only a small portion of the total patient... (Review)
Review
Outliers account for a large amount of technology utilization and resources consumed in acute care, despite accounting for only a small portion of the total patient population. Most current efforts to reduce costs, such as re-engineering and downsizing, are nonspecific methods of controlling costs. Focusing efforts in high-cost areas, such as outlier management, is much more likely to improve patient care and improve the use of technology while achieving real advances in cost control. Most hospitals will have to build an infrastructure to support outlier management, which includes developing clinical staff. These processes will take time and careful planning, but they are essential for the effective management of technology utilization and outliers. The failure to employ focused efforts like outlier management will result in the superficial treatment of high costs in acute care. The benefit to employing these methods leads to the best use of technology and the improved management of a difficult patient population.
Topics: Acute Disease; Critical Care; Health Care Rationing; Health Resources; Hospital Costs; Humans; Outliers, DRG; Technology, High-Cost
PubMed: 10373828
DOI: No ID Found -
Sensors (Basel, Switzerland) Apr 2021Anomaly Detection research is focused on the development and application of methods that allow for the identification of data that are different enough-compared with the...
Anomaly Detection research is focused on the development and application of methods that allow for the identification of data that are different enough-compared with the rest of the data set that is being analyzed-and considered anomalies (or, as they are more commonly called, outliers). These values mainly originate from two sources: they may be errors introduced during the collection or handling of the data, or they can be correct, but very different from the rest of the values. It is essential to correctly identify each type as, in the first case, they must be removed from the data set but, in the second case, they must be carefully analyzed and taken into account. The correct selection and use of the model to be applied to a specific problem is fundamental for the success of the anomaly detection study and, in many cases, the use of only one model cannot provide sufficient results, which can be only reached by using a mixture model resulting from the integration of existing and/or ad hoc-developed models. This is the kind of model that is developed and applied to solve the problem presented in this paper. This study deals with the definition and application of an anomaly detection model that combines statistical models and a new method defined by the authors, the Local Transilience Outlier Identification Method, in order to improve the identification of outliers in the sensor-obtained values of variables that affect the operations of wind tunnels. The correct detection of outliers for the variables involved in wind tunnel operations is very important for the industrial ventilation systems industry, especially for vertical wind tunnels, which are used as training facilities for indoor skydiving, as the incorrect performance of such devices may put human lives at risk. In consequence, the use of the presented model for outlier detection may have a high impact in this industrial sector. In this research work, a proof-of-concept is carried out using data from a real installation, in order to test the proposed anomaly analysis method and its application to control the correct performance of wind tunnels.
PubMed: 33916611
DOI: 10.3390/s21072532 -
IEEE Journal of Biomedical and Health... May 2023Since brain network organization is essentially governed by the harmonic waves derived from the Eigen-system of the underlying Laplacian matrix, discovering the...
Since brain network organization is essentially governed by the harmonic waves derived from the Eigen-system of the underlying Laplacian matrix, discovering the harmonic-based alterations provides a new window to understand the pathogenic mechanism of Alzheimer's disease (AD) in a unified reference space. However, current reference (common harmonic waves) estimation studies over the individual harmonic waves are often sensitive to outliers, which are obtained by averaging the heterogenous individual brain networks. To address this challenge, we propose a novel manifold learning approach to identify a set of outlier-immunized common harmonic waves. The backbone of our framework is calculating the geometric median of all individual harmonic waves on the Stiefel manifold, instead of Fréchet mean, thus improving the robustness of learned common harmonic waves to the outliers. A manifold optimization scheme with theoretically guaranteed convergence is tailored to solve our method. The experimental results on synthetic data and real data demonstrate that the common harmonic waves learned by our approach are not only more robust to the outliers than the state-of-the-art methods, but also provide a putative imaging biomarker to predict the early stage of AD.
Topics: Humans; Brain; Alzheimer Disease
PubMed: 37028067
DOI: 10.1109/JBHI.2023.3250711 -
Applied Psychological Measurement Jan 2023In equating practice, the existence of outliers in the anchor items may deteriorate the equating accuracy and threaten the validity of test scores. Therefore, stability...
In equating practice, the existence of outliers in the anchor items may deteriorate the equating accuracy and threaten the validity of test scores. Therefore, stability of the anchor item performance should be evaluated before conducting equating. This study used simulation to investigate the performance of the -test method in detecting outliers and compared its performance with other outlier detection methods, including the logit difference method with 0.5 and 0.3 as the cutoff values and the robust statistic with 2.7 as the cutoff value. The investigated factors included sample size, proportion of outliers, item difficulty drift direction, and group difference. Across all simulated conditions, the -test method outperformed the other methods in terms of sensitivity of flagging true outliers, bias of the estimated translation constant, and the root mean square error of examinee ability estimates.
PubMed: 36425288
DOI: 10.1177/01466216221124045 -
IEEE Letters of the Computer Society 2020The increasing societal demand for data privacy has led researchers to develop methods to preserve privacy in data analysis. However, outlier analysis, a fundamental...
The increasing societal demand for data privacy has led researchers to develop methods to preserve privacy in data analysis. However, outlier analysis, a fundamental data analytics task with critical applications in medicine, finance, and national security, has only been analyzed for a few specialized cases of data privacy. This work is the first to provide a general framework for private outlier analysis, which is a two-step process. First, we show how to identify the relevant problem-specifications and then provide a practical solution that formally meets these specifications.
PubMed: 32803135
DOI: 10.1109/LOCS.2020.2994342 -
Journal of Classification Apr 2015In model-based clustering based on normal-mixture models, a few outlying observations can influence the cluster structure and number. This paper develops a method to...
In model-based clustering based on normal-mixture models, a few outlying observations can influence the cluster structure and number. This paper develops a method to identify these, however it does not attempt to identify clusters amidst a large field of noisy observations. We identify outliers as those observations in a cluster with minimal membership proportion or for which the cluster-specific variance with and without the observation is very different. Results from a simulation study demonstrate the ability of our method to detect true outliers without falsely identifying many non-outliers and improved performance over other approaches, under most scenarios. We use the contributed R package MCLUST for model-based clustering, but propose a modified prior for the cluster-specific variance which avoids degeneracies in estimation procedures. We also compare results from our outlier method to published results on National Hockey League data.
PubMed: 26806993
DOI: 10.1007/s00357-015-9171-5 -
Frontiers in Bioinformatics 2023Conventional dimensionality reduction methods like Multidimensional Scaling (MDS) are sensitive to the presence of orthogonal outliers, leading to significant defects in...
Conventional dimensionality reduction methods like Multidimensional Scaling (MDS) are sensitive to the presence of orthogonal outliers, leading to significant defects in the embedding. We introduce a robust MDS method, called (Detection and Correction of Orthogonal outliers using MDS), based on the geometry and statistics of simplices formed by data points, that allows to detect orthogonal outliers and subsequently reduce dimensionality. We validate our methods using synthetic datasets, and further show how it can be applied to a variety of large real biological datasets, including cancer image cell data, human microbiome project data and single cell RNA sequencing data, to address the task of data cleaning and visualization.
PubMed: 37637212
DOI: 10.3389/fbinf.2023.1211819 -
Sensors (Basel, Switzerland) Aug 2022Accurate identification of the degradation stage is key to the prediction of the remaining useful life (RUL) of bearings. The 3σ method is commonly used to identify the...
Accurate identification of the degradation stage is key to the prediction of the remaining useful life (RUL) of bearings. The 3σ method is commonly used to identify the degradation point. However, the recognition accuracy is seriously disturbed by the random outliers in the normal stage. Therefore, this paper proposes an adaptive recognition method for the degradation stage based on outlier cleaning. Firstly, an improved multi-scale kernel regression outlier detection method is adopted to roughly search the abnormal signal segments. Then, a method for the accurate locating of the start and end points of abnormal impulses is established. After that, indexes are constructed for screening abnormal segments and an iterative strategy is proposed to achieve an accurate and efficient removal of abnormal impulses. After outlier cleaning, the 3σ approach is used to set the degradation warning threshold adaptively to realize the degradation stage recognition of the bearings. The PHM 2012 rotating machinery dataset is used to verify the effectiveness of the proposed method. Experimental results show that the proposed method can accurately locate and remove the outliers adaptively. After the cleaning of the outliers, the identification of the degradation stage is no longer disturbed by the selection of the reference signal of the normal stage and the robustness and the accuracy of the degradation stage identification have been improved significantly.
PubMed: 36080939
DOI: 10.3390/s22176480