Approximate Hash-Based Set Reconciliation for Distributed Replica Repair

2015 ◽  
Author(s):  
Nico Kruber ◽  
Maik Lange ◽  
Florian Schintke
Keyword(s):  
Set Reconciliation

scholarly journals Comparing set reconciliation methods based on bloom filters and their variants

2016 ◽  
Vol 21 (2) ◽  
pp. 157-167
Author(s):  
Zhiyao Hu ◽  
Xiaoqiang Teng ◽  
Deke Guo ◽  
Bangbang Ren ◽  
Pin Lv ◽  
...  
Keyword(s):  
Bloom Filters ◽  
Set Reconciliation

scholarly journals Simple multi-party set reconciliation

2017 ◽  
Vol 31 (6) ◽  
pp. 441-453 ◽  
Author(s):  
Michael Mitzenmacher ◽  
Rasmus Pagh
Keyword(s):  
Set Reconciliation

scholarly journals Space- and computationally-efficient set reconciliation via parity bitmap sketch (PBS)

2020 ◽  
Vol 14 (4) ◽  
pp. 458-470
Author(s):  
Long Gong ◽  
Ziheng Liu ◽  
Liang Liu ◽  
Jun Xu ◽  
Mitsunori Ogihara ◽  
...  
Keyword(s):  
Computational Complexity ◽  
Performance Metrics ◽  
Optimal Parameter ◽  
Parameter Tuning ◽  
Bloom Filters ◽  
Communication Overhead ◽  
Algorithmic Problem ◽  
Low Computational Complexity ◽  
Theoretical Minimum ◽  
Set Reconciliation

Set reconciliation is a fundamental algorithmic problem that arises in many networking, system, and database applications. In this problem, two large sets A and B of objects (bitcoins, files, records, etc.) are stored respectively at two different network-connected hosts, which we name Alice and Bob respectively. Alice and Bob communicate with each other to learn A Δ B , the difference between A and B , and as a result the reconciled set A ∪ B. Current set reconciliation schemes are based on either invertible Bloom filters (IBF) or error-correction codes (ECC). The former has a low computational complexity of O(d) , where d is the cardinality of A Δ B , but has a high communication overhead that is several times larger than the theoretical minimum. The latter has a low communication overhead close to the theoretical minimum, but has a much higher computational complexity of O(d 2 ). In this work, we propose Parity Bitmap Sketch (PBS), an ECC-based set reconciliation scheme that gets the better of both worlds: PBS has both a low computational complexity of O(d) just like IBF-based solutions and a low communication overhead of roughly twice the theoretical minimum. A separate contribution of this work is a novel rigorous analytical framework that can be used for the precise calculation of various performance metrics and for the near-optimal parameter tuning of PBS.

Sign in / Sign up

Export Citation Format

Share Document