Exact set reconciliation based on bloom filters

2011 ◽  
Author(s):  
Xiaomei Tian ◽  
Dafang Zhang ◽  
Kun Xie ◽  
Can Hu ◽  
Mengfan Wang ◽  
...  
Keyword(s):  
Bloom Filters ◽  
Set Reconciliation

scholarly journals Comparing set reconciliation methods based on bloom filters and their variants

2016 ◽  
Vol 21 (2) ◽  
pp. 157-167
Author(s):  
Zhiyao Hu ◽  
Xiaoqiang Teng ◽  
Deke Guo ◽  
Bangbang Ren ◽  
Pin Lv ◽  
...  
Keyword(s):  
Bloom Filters ◽  
Set Reconciliation

scholarly journals Space- and computationally-efficient set reconciliation via parity bitmap sketch (PBS)

2020 ◽  
Vol 14 (4) ◽  
pp. 458-470
Author(s):  
Long Gong ◽  
Ziheng Liu ◽  
Liang Liu ◽  
Jun Xu ◽  
Mitsunori Ogihara ◽  
...  
Keyword(s):  
Computational Complexity ◽  
Performance Metrics ◽  
Optimal Parameter ◽  
Parameter Tuning ◽  
Bloom Filters ◽  
Communication Overhead ◽  
Algorithmic Problem ◽  
Low Computational Complexity ◽  
Theoretical Minimum ◽  
Set Reconciliation

Set reconciliation is a fundamental algorithmic problem that arises in many networking, system, and database applications. In this problem, two large sets A and B of objects (bitcoins, files, records, etc.) are stored respectively at two different network-connected hosts, which we name Alice and Bob respectively. Alice and Bob communicate with each other to learn A Δ B , the difference between A and B , and as a result the reconciled set A ∪ B. Current set reconciliation schemes are based on either invertible Bloom filters (IBF) or error-correction codes (ECC). The former has a low computational complexity of O(d) , where d is the cardinality of A Δ B , but has a high communication overhead that is several times larger than the theoretical minimum. The latter has a low communication overhead close to the theoretical minimum, but has a much higher computational complexity of O(d 2 ). In this work, we propose Parity Bitmap Sketch (PBS), an ECC-based set reconciliation scheme that gets the better of both worlds: PBS has both a low computational complexity of O(d) just like IBF-based solutions and a low communication overhead of roughly twice the theoretical minimum. A separate contribution of this work is a novel rigorous analytical framework that can be used for the precise calculation of various performance metrics and for the near-optimal parameter tuning of PBS.

THE PROBLEM OF PROVIDING CACHE COHERENCE IN MULTIPROCESSOR SYSTEMS WITH MANY PROCESSORS

2018 ◽  
pp. 47-53
Author(s):  
B. Z. Shmeylin ◽  
E. A. Alekseeva
Keyword(s):  
Cache Coherence ◽  
Bloom Filters ◽  
Multiprocessor Systems ◽  
Cache Line ◽  
Maintenance Systems ◽  
Processor Caches ◽  
Conventional Systems ◽  
Additional Hardware

In this paper the tasks of managing the directory in coherence maintenance systems in multiprocessor systems with a large number of processors are solved. In microprocessor systems with a large number of processors (MSLP) the problem of maintaining the coherence of processor caches is significantly complicated. This is due to increased traffic on the memory buses and increased complexity of interprocessor communications. This problem is solved in various ways. In this paper, we propose the use of Bloom filters used to accelerate the determination of an element’s belonging to a certain array. In this article, such filters are used to establish the fact that the processor belongs to some subset of the processors and determine if the processor has a cache line in the set. In the paper, the processes of writing and reading information in the data shared between processors are discussed in detail, as well as the process of data replacement from private caches. The article also shows how the addresses of cache lines and processor numbers are removed from the Bloom filters. The system proposed in this paper allows significantly speeding up the implementation of operations to maintain cache coherence in the MSLP as compared to conventional systems. In terms of performance and additional hardware and software costs, the proposed system is not inferior to the most efficient of similar systems, but on some applications and significantly exceeds them.

Implementation of bloom filters in reconfigurable hardware for tracing network attacks

2006 ◽  
Vol 39 (21) ◽  
pp. 310-315
Author(s):  
Maciej Wołowiec ◽  
Jakub Botwicz ◽  
Piotr Sapiecha
Keyword(s):  
Reconfigurable Hardware ◽  
Bloom Filters ◽  
Network Attacks
Sign in / Sign up

Export Citation Format

Share Document