A Comparison Between New Modification of ANWK and Classical ANWK Methods in Nonparametric Regression

Nonparametric kernel estimators are mostly used in a variety of statistical research fields. Nadaraya-Watson kernel estimator (NWK) is one of the most important nonparametric kernel estimator that is often used in regression models with a fixed bandwidth. In this article, we consider the four new Proposed Adaptive Nadaraya-Watson Kernel Regression Estimators (Interquartile Range, Standard Deviation, Mean Absolute Devotion, and Median Absolute Deviation) rather than (Fixed Bandwidth, Adaptive Geometric, Adaptive Mean, Adaptive Range, and Adaptive Median). The outcomes in both simulation and actual data in Leukemia Cancer show that the four new ANW Kernel Estimators (Interquartile Range, Standard Deviation, Mean Absolute devotion, and Median Absolute Deviation) is more effective than the kernel estimations with fixed bandwidth in previous studies using Mean Square Error (MSE) Criterion.

eHMCOKE: an enhanced overlapping clustering algorithm for data analysis

Improved multi-cluster overlapping k-means extension (IMCOKE) uses median absolute deviation (MAD) in detecting outliers in datasets makes the algorithm more effective with regards to overlapping clustering. Nevertheless, analysis of the applied MAD positioning was not considered. In this paper, the incorporation of MAD used to detect outliers in the datasets was analyzed to determine the appropriate position in identifying the outlier before applying it in the clustering application. And the assumption of the study was the size of the cluster and cluster that are close to each other can led to a higher runtime performance in terms of overlapping clusters. Therefore, additional parameters such as radius of clusters and distance between clusters are added measurements in the algorithm procedures. Evaluation was done through experimentations using synthetic and real datasets. The performance of the eHMCOKE was evaluated via F1-measure criterion, speed and percentage of improvement. Evaluation results revealed that the eHMCOKE takes less time to discover overlap clusters with an improvement rate of 22% and achieved the best performance of 91.5% accuracy rate via F1-measure in identifying overlapping clusters over the IMCOKE algorithm. These results proved that the eHMCOKE significantly outruns the IMCOKE algorithm on mosts of the test conducted.

Approximating median absolute deviation with bounded error

The median absolute deviation (MAD) is a statistic measuring the variability of a set of quantitative elements. It is known to be more robust to outliers than the standard deviation (SD), and thereby widely used in outlier detection. Computing the exact MAD however is costly, e.g., by calling an algorithm of finding median twice, with space cost O ( n ) over n elements in a set. In this paper, we propose the first fully mergeable approximate MAD algorithm, OP-MAD, with one-pass scan of the data. Remarkably, by calling the proposed algorithm at most twice, namely TP-MAD, it guarantees to return an (ϵ, 1)-accurate MAD, i.e., the error relative to the exact MAD is bounded by the desired ϵ or 1. The space complexity is reduced to O ( m ) while the time complexity is O ( n + m log m ), where m is the size of the sketch used to compress data, related to the desired error bound ϵ. To get a more accurate MAD, i.e., with smaller ϵ, the sketch size m will be larger, a trade-off between effectiveness and efficiency. In practice, we often have the sketch size m ≪ n , leading to constant space cost O (1) and linear time cost O ( n ). The extensive experiments over various datasets demonstrate the superiority of our solution, e.g., 160000× less memory and 18x faster than the aforesaid exact method in datasets pareto and norm . Finally, we further implement and evaluate the parallelizable TP-MAD in Apache Spark, and the fully mergeable OP-MAD in Structured Streaming.

A method for accurately and dynamically optimising pacemaker atrio-ventricular delay timing using implantable physiological biomarkers

Funding Acknowledgements Type of funding sources: Other. Main funding source(s): BRAVO trial: BHF SP/10/002/28189, FS/10/038, FS/11/92/29122, FS/13/44/30291) National Institute for Health Research Imperial Biomedical Research Centre. HOPE-HF trial: British Heart Foundation (CS/15/3/31405, FS/13/44/30291, FS/15/53/31615, FS/14/27/30752, FS/10/038). Introduction The optimal atrioventricular (AV) delay for implantable cardiac devices can be derived by echocardiography or beat-by-beat blood pressure measurements. However, both of these approaches are labour intensive and neither could be incorporated into an implantable cardiac device for frequent repeated optimisations. Laser Doppler perfusion monitoring (LDPM) measures blood flow through tissue. LDPM has been miniaturised ready to be incorporated into future implantable cardiac devices. Purpose We studied if LDPM is a clinically reliable alternative method to blood-pressure measurements to determine optimal AV delay. Methods Data from 58 patients undergoing 94 clinical AVD optimisations using LDPM and simultaneous non-invasive beat-by-beat blood pressure was obtained. The optimal AV delay for each method and for each optimisation was determined using a curve of haemodynamic response to switching from AAI (reference state) to DDD (test state) at a series of AV delays (40, 80, 120, 160, 200, 240 ms). We then compared the derived optimal AV delays between the two measurement approaches. We also assessed the impact of the paced heart-rate on agreement between laser Doppler and Blood-Pressure derived optimal AV delays. Results The AV delay derived using LDPM was not clinically significant different from that derived by blood pressure changes. The median difference was -9ms (IQR -26 to 7, p = 0.05). Variability between the two methods was low (median absolute deviation 17ms). Optimisations performed at higher heart-rates resulted in a non-significant smaller difference between the LDPM and blood-pressure derived AV delays (median absolute deviation 12 vs 22 ms, p = 0.11). Conclusions Optimal AVDs derived from non-invasive blood-pressure or laser Doppler perfusion methods are clinically equivalent. The addition of laser Doppler to future implantable cardiac devices may enable devices to dynamically and reliably optimise AV delays. Abstract Figure 1

High-Dimensional Satellite Image Compositing and Statistics for Enhanced Irrigated Crop Mapping

Accurate irrigated area maps remain difficult to generate, as smallholder irrigation schemes often escape detection. Efforts to map smallholder irrigation have often relied on complex classification models fitted to temporal image stacks. The use of high-dimensional geometric median composites (geomedians) and high-dimensional statistics of time-series may simplify classification models and enhance accuracy. High-dimensional statistics for temporal variation, such as the spectral median absolute deviation, indicate spectral variability within a period contributing to a geomedian. The Ord River Irrigation Area was used to validate Digital Earth Australia’s annual geomedian and temporal variation products. Geomedian composites and the spectral median absolute deviation were then calculated on Sentinel-2 images for three smallholder irrigation schemes in Matabeleland, Zimbabwe, none of which were classified as areas equipped for irrigation in AQUASTAT’s Global Map of Irrigated Areas. Supervised random forest classification was applied to all sites. For the three Matabeleland sites, the average Kappa coefficient was 0.87 and overall accuracy was 95.9% on validation data. This compared with 0.12 and 77.2%, respectively, for the Food and Agriculture Organisation’s Water Productivity through Open access of Remotely sensed derived data (WaPOR) land use classification map. The spectral median absolute deviation was ranked among the most important variables across all models based on mean decrease in accuracy. Change detection capacity also means the spectral median absolute deviation has some advantages for cropland mapping over indices such as the Normalized Difference Vegetation Index. The method demonstrated shows potential to be deployed across countries and regions where smallholder irrigation schemes account for large proportions of irrigated area.

Attack Context Embedded Data Driven Trust Diagnostics in Smart Metering Infrastructure

Spurious power consumption data reported from compromised meters controlled by organized adversaries in the Advanced Metering Infrastructure (AMI) may have drastic consequences on a smart grid’s operations. While existing research on data falsification in smart grids mostly defends against isolated electricity theft, we introduce a taxonomy of various data falsification attack types, when smart meters are compromised by organized or strategic rivals. To counter these attacks, we first propose a coarse-grained and a fine-grained anomaly-based security event detection technique that uses indicators such as deviation and directional change in the time series of the proposed anomaly detection metrics to indicate: (i) occurrence, (ii) type of attack, and (iii) attack strategy used, collectively known as attack context . Leveraging the attack context information, we propose three attack response metrics to the inferred attack context: (a) an unbiased mean indicating a robust location parameter; (b) a median absolute deviation indicating a robust scale parameter; and (c) an attack probability time ratio metric indicating the active time horizon of attacks. Subsequently, we propose a trust scoring model based on Kullback-Leibler (KL) divergence, that embeds the appropriate unbiased mean, the median absolute deviation, and the attack probability ratio metric at runtime to produce trust scores for each smart meter. These trust scores help classify compromised smart meters from the non-compromised ones. The embedding of the attack context, into the trust scoring model, facilitates accurate and rapid classification of compromised meters, even under large fractions of compromised meters, generalize across various attack strategies and margins of false data. Using real datasets collected from two different AMIs, experimental results show that our proposed framework has a high true positive detection rate, while the average false alarm and missed detection rates are much lesser than 10% for most attack combinations for two different real AMI micro-grid datasets. Finally, we also establish fundamental theoretical limits of the proposed method, which will help assess the applicability of our method to other domains.

Motor-Imagery Classification Using Riemannian Geometry with Median Absolute Deviation

Motor imagery (MI) from human brain signals can diagnose or aid specific physical activities for rehabilitation, recreation, device control, and technology assistance. It is a dynamic state in learning and practicing movement tracking when a person mentally imitates physical activity. Recently, it has been determined that a brain–computer interface (BCI) can support this kind of neurological rehabilitation or mental practice of action. In this context, MI data have been captured via non-invasive electroencephalogram (EEGs), and EEG-based BCIs are expected to become clinically and recreationally ground-breaking technology. However, determining a set of efficient and relevant features for the classification step was a challenge. In this paper, we specifically focus on feature extraction, feature selection, and classification strategies based on MI-EEG data. In an MI-based BCI domain, covariance metrics can play important roles in extracting discriminatory features from EEG datasets. To explore efficient and discriminatory features for the enhancement of MI classification, we introduced a median absolute deviation (MAD) strategy that calculates the average sample covariance matrices (SCMs) to select optimal accurate reference metrics in a tangent space mapping (TSM)-based MI-EEG. Furthermore, all data from SCM were projected using TSM according to the reference matrix that represents the featured vector. To increase performance, we reduced the dimensions and selected an optimum number of features using principal component analysis (PCA) along with an analysis of variance (ANOVA) that could classify MI tasks. Then, the selected features were used to develop linear discriminant analysis (LDA) training for classification. The benchmark datasets were considered for the evaluation and the results show that it provides better accuracy than more sophisticated methods.

MAD saccade: statistically robust saccade threshold estimation via the median absolute deviation

Saccade detection is a critical step in the analysis of gaze data. A common method for saccade detection is to use a simple threshold for velocity or acceleration values, which can be estimated from the data using the mean and standard deviation. However, this method has the downside of being influenced by the very signal it is trying to detect, the outlying velocities or accelerations that occur during saccades. We propose instead to use the median absolute deviation (MAD), a robust estimator of dispersion that is not influenced by outliers. We modify an algorithm proposed by Nyström and colleagues, and quantify saccade detection performance in both simulated and human data. Our modified algorithm shows a significant and marked improvement in saccade detection - showing both more true positives and less false negatives – especially under higher noise levels. We conclude that robust estimators can be widely adopted in other common, automatic gaze classification algorithms due to their ease of implementation.