False-Positive Probability and Compression Optimization for Tree-Structured Bloom Filters

2016 ◽  
Vol 1 (4) ◽  
pp. 1-39
Author(s):  
Yongquan Fu ◽  
Ernst Biersack
Keyword(s):  
False Positive ◽  
Bloom Filters ◽  
Positive Probability ◽  
False Positive Probability

scholarly journals Accurate Counting Bloom Filters for Large-Scale Data Processing

2013 ◽  
Vol 2013 ◽  
pp. 1-11 ◽  
Author(s):  
Wei Li ◽  
Kun Huang ◽  
Dafang Zhang ◽  
Zheng Qin
Keyword(s):  
Data Processing ◽  
False Positive ◽  
Large Scale ◽  
Bloom Filters ◽  
Positive Probability ◽  
Large Scale Data ◽  
Large Scale Data Processing ◽  
False Positive Probability ◽  
Scale Data ◽  
Counting Bloom Filters

Bloom filters are space-efficient randomized data structures for fast membership queries, allowing false positives. Counting Bloom Filters (CBFs) perform the same operations on dynamic sets that can be updated via insertions and deletions. CBFs have been extensively used in MapReduce to accelerate large-scale data processing on large clusters by reducing the volume of datasets. The false positive probability of CBF should be made as low as possible for filtering out more redundant datasets. In this paper, we propose a multilevel optimization approach to building an Accurate Counting Bloom Filter (ACBF) for reducing the false positive probability. ACBF is constructed by partitioning the counter vector into multiple levels. We propose an optimized ACBF by maximizing the first level size, in order to minimize the false positive probability while maintaining the same functionality as CBF. Simulation results show that the optimized ACBF reduces the false positive probability by up to 98.4% at the same memory consumption compared to CBF. We also implement ACBFs in MapReduce to speed up the reduce-side join. Experiments on realistic datasets show that ACBF reduces the false positive probability by 72.3% as well as the map outputs by 33.9% and improves the join execution times by 20% compared to CBF.

scholarly journals Midas: Microcluster-Based Detector of Anomalies in Edge Streams

2020 ◽  
Vol 34 (04) ◽  
pp. 3242-3249 ◽  
Author(s):  
Siddharth Bhatia ◽  
Bryan Hooi ◽  
Minji Yoon ◽  
Kijung Shin ◽  
Christos Faloutsos
Keyword(s):  
Network Traffic ◽  
False Positive ◽  
State Of The Art ◽  
Denial Of Service ◽  
Constant Time ◽  
Traffic Data ◽  
Positive Probability ◽  
Dynamic Graph ◽  
Unusual Behavior ◽  
False Positive Probability

Given a stream of graph edges from a dynamic graph, how can we assign anomaly scores to edges in an online manner, for the purpose of detecting unusual behavior, using constant time and memory? Existing approaches aim to detect individually surprising edges. In this work, we propose Midas, which focuses on detecting microcluster anomalies, or suddenly arriving groups of suspiciously similar edges, such as lockstep behavior, including denial of service attacks in network traffic data. Midas has the following properties: (a) it detects microcluster anomalies while providing theoretical guarantees about its false positive probability; (b) it is online, thus processing each edge in constant time and constant memory, and also processes the data 108–505 times faster than state-of-the-art approaches; (c) it provides 46%-52% higher accuracy (in terms of AUC) than state-of-the-art approaches.

scholarly journals Planetary candidates transiting cool dwarf stars from campaigns 12 to 15 of K2

2020 ◽  
Vol 499 (4) ◽  
pp. 5416-5441
Author(s):  
A Castro González ◽  
E Díez Alonso ◽  
J Menéndez Blanco ◽  
John H Livingston ◽  
Jerome P de Leon ◽  
...  
Keyword(s):  
False Positive ◽  
Habitable Zone ◽  
Positive Probability ◽  
Low Mass Stars ◽  
Dwarf Stars ◽  
Cool Stars ◽  
Nearby Stars ◽  
Low Mass ◽  
False Positive Probability ◽  
Gaia Dr2

ABSTRACT We analysed the photometry of 20 038 cool stars from campaigns 12, 13, 14, and 15 of the K2 mission in order to detect, characterize, and validate new planetary candidates transiting low-mass stars. We present a catalogue of 25 new periodic transit-like signals in 22 stars, of which we computed the parameters of the stellar host for 19 stars and the planetary parameters for 21 signals. We acquired speckle and AO images, and also inspected archival Pan-STARRS1 images and Gaia DR2 to discard the presence of close stellar companions and to check possible transit dilutions due to nearby stars. False positive probability (FPP) was computed for 22 signals, obtaining FPP < $1{{\ \rm per\ cent}}$ for 17. We consider 12 of them as statistically validated planets. One signal is a false positive and the remaining 12 signals are considered as planet candidates. 20 signals have an orbital period of P$_{\rm orb} \lt 10\,\mathrm{ d}$, 2 have $10\, \mathrm{ d} \lt $  P$_{\rm orb} \lt 20\, \mathrm{ d}$, and 3 have P$_{\rm orb} \gt 20\, \mathrm{ d}$. Regarding radii, 11 candidates and validated planets have computed radius R < 2R⊕, 9 have 2R⊕ < R < 4R⊕, and 1 has R > 4R⊕. Two validated planets and two candidates are located in moderately bright stars ($\rm \mathit{ m}_{kep}\lt 13$) and two validated planets and three candidates have derived orbital radius within the habitable zone according to optimistic models. Of special interest is the validated warm super-Earth K2-323 b (EPIC 248616368 b) with T$_{\rm eq} = 318^{+24}_{-43} \, \mathrm{ K}$, S$_{\rm p} = 1.7\pm 0.2 \, \mathrm{ S}_{\oplus }$, and R$_{\rm p} = 2.1\pm 0.1 \, \mathrm{ R}_{\oplus }$, located in an m$\rm _{kep}$ = 14.13 star.

Sign in / Sign up

Export Citation Format

Share Document