Efficient weighted probabilistic frequent itemset mining in uncertain databases

2020 ◽  
pp. e12551
Author(s):  
Zhiyang Li ◽  
Fengjuan Chen ◽  
Junfeng Wu ◽  
Zhaobin Liu ◽  
Weijiang Liu
Keyword(s):  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Uncertain Databases ◽  
Itemset Mining

Mining Weighted Frequent Itemsets without Candidate Generation in Uncertain Databases

2017 ◽  
Vol 16 (06) ◽  
pp. 1549-1579 ◽  
Author(s):  
Jerry Chun-Wei Lin ◽  
Wensheng Gan ◽  
Philippe Fournier-Viger ◽  
Tzung-Pei Hong ◽  
Han-Chieh Chao
Keyword(s):  
Real Life ◽  
Search Space ◽  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Uncertain Databases ◽  
Two Phase ◽  
Itemset Mining ◽  
Meaningful Relationships ◽  
Weighted Probability ◽  
Novel Structures

Frequent itemset mining (FIM) is a fundamental set of techniques used to discover useful and meaningful relationships between items in transaction databases. In recent decades, extensions of FIM such as weighted frequent itemset mining (WFIM) and frequent itemset mining in uncertain databases (UFIM) have been proposed. WFIM considers that items may have different weight/importance. It can thus discover itemsets that are more useful and meaningful by ignoring irrelevant itemsets with lower weights. UFIM takes into account that data collected in a real-life environment may often be inaccurate, imprecise, or incomplete. Recently, these two ideas have been combined in the HEWI-Uapriori algorithm. This latter considers both item weights and transaction uncertainty to mine the high expected weighted itemsets (HEWIs) using a two-phase Apriori-based approach. Although the upper-bound proposed in HEWI-Uapriori can reduce the size of the search space, it still generates a large amount of candidates and uses a level-wise search. In this paper, a more efficient algorithm named HEWI-Utree is developed to efficiently mine HEWIs without performing multiple database scans and without generating candidates. This algorithm relies on three novel structures named element (E)-table, weighted-probability (WP)-table and WP-tree to maintain the information required for identifying and pruning unpromising itemsets early. Experimental results show that the proposed algorithm is generally much more efficient than traditional methods for WFIM and UFIM, as well as the state-of-the-art HEWI-Uapriori algorithm, in terms of runtime, memory consumption, and scalability.

Probabilistic maximal frequent itemset mining methods over uncertain databases

2019 ◽  
Vol 23 (6) ◽  
pp. 1219-1241
Author(s):  
Haifeng Li ◽  
Mo Hai ◽  
Ning Zhang ◽  
Jianming Zhu ◽  
Yue Wang ◽  
...  
Keyword(s):  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Uncertain Databases ◽  
Itemset Mining ◽  
Mining Methods

Weighted frequent itemset mining over uncertain databases

2015 ◽  
Vol 44 (1) ◽  
pp. 232-250 ◽  
Author(s):  
Jerry Chun-Wei Lin ◽  
Wensheng Gan ◽  
Philippe Fournier-Viger ◽  
Tzung-Pei Hong ◽  
Vincent S. Tseng
Keyword(s):  
Frequent Itemset ◽  
Frequent Itemset Mining ◽  
Uncertain Databases ◽  
Itemset Mining

scholarly journals A Synopsis Based Approach for Itemset Frequency Estimation over Massive Multi-Transaction Stream

2021 ◽  
Vol 16 (2) ◽  
pp. 1-30
Author(s):  
Guangtao Wang ◽  
Gao Cong ◽  
Ying Zhang ◽  
Zhen Hai ◽  
Jieping Ye
Keyword(s):  
Frequency Estimation ◽  
Frequent Itemsets ◽  
Frequent Itemset ◽  
Experimental Results ◽  
Closure Property ◽  
Frequent Itemset Mining ◽  
Itemset Mining ◽  
Minimum Value ◽  
Downward Closure ◽  
Bounded Size

The streams where multiple transactions are associated with the same key are prevalent in practice, e.g., a customer has multiple shopping records arriving at different time. Itemset frequency estimation on such streams is very challenging since sampling based methods, such as the popularly used reservoir sampling, cannot be used. In this article, we propose a novel k -Minimum Value (KMV) synopsis based method to estimate the frequency of itemsets over multi-transaction streams. First, we extract the KMV synopses for each item from the stream. Then, we propose a novel estimator to estimate the frequency of an itemset over the KMV synopses. Comparing to the existing estimator, our method is not only more accurate and efficient to calculate but also follows the downward-closure property. These properties enable the incorporation of our new estimator with existing frequent itemset mining (FIM) algorithm (e.g., FP-Growth) to mine frequent itemsets over multi-transaction streams. To demonstrate this, we implement a KMV synopsis based FIM algorithm by integrating our estimator into existing FIM algorithms, and we prove it is capable of guaranteeing the accuracy of FIM with a bounded size of KMV synopsis. Experimental results on massive streams show our estimator can significantly improve on the accuracy for both estimating itemset frequency and FIM compared to the existing estimators.

Sign in / Sign up

Export Citation Format

Share Document