An Innovative Framework for Supporting Cognitive-Based Big Data Analytics for Frequent Pattern Mining

2018 ◽  
Author(s):  
Deyu Deng ◽  
Carson K. Leung ◽  
Bryan H. Wodi ◽  
Jialiang Yu ◽  
Hao Zhang ◽  
...  
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Pattern Mining ◽  
Big Data Analytics ◽  
Frequent Pattern Mining ◽  
Frequent Pattern

scholarly journals MISFP-Growth: Hadoop-Based Frequent Pattern Mining with Multiple Item Support

2019 ◽  
Vol 9 (10) ◽  
pp. 2075 ◽  
Author(s):  
Chen-Shu Wang ◽  
Jui-Yen Chang
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Pattern Mining ◽  
High Efficiency ◽  
Big Data Analytics ◽  
Frequent Pattern Mining ◽  
Experimental Results ◽  
Frequent Pattern ◽  
Multiple Item ◽  
Two Phases

In practice, single item support cannot comprehensively address the complexity of items in large datasets. In this study, we propose a big data analytics framework (named Multiple Item Support Frequent Patterns, MISFP-growth algorithm) that uses Hadoop-based parallel computing to achieve high-efficiency mining of itemsets with multiple item supports (MIS). The proposed architecture consists of two phases. First, in the counting support phase, a Hadoop MapReduce architecture is employed to determine the support for each item. Next, in the analytics phase, sub-transaction blocks are generated according to MIS and the MISFP-growth algorithm identifies the frequency of patterns. To facilitate decision makers in setting MIS, we also propose the concept of classification of item (COI), which classifies items of higher homogeneity into the same class, by which the items inherit class support as their item support. Three experiments were implemented to validate the proposed Hadoop-based MISFP-growth algorithm. The experimental results show approximately 38% reduction in the execution time on parallel architectures. The proposed MISFP-growth algorithm can be implemented on the distributed computing framework. Furthermore, according to the experimental results, the enhanced performance of the proposed algorithm indicates that it could have big data analytics applications.

Big Data Analytics and Mining for Knowledge Discovery

2021 ◽  
pp. 1817-1830
Author(s):  
Carson K. Leung
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Pattern Mining ◽  
Real Life ◽  
Big Data Analytics ◽  
Organizational Knowledge ◽  
Frequent Pattern ◽  
Data Sets ◽  
Technology Applications ◽  
Rich Data

Big data analytics and mining aims to discover implicit, previously unknown, and potentially useful information and knowledge from big data sets that contain huge volumes of valuable veracious data collected or generated at a high velocity from a wide variety of rich data sources. Among different big data analytic and mining tasks, this chapter focuses on frequent pattern mining. By relying on the MapReduce programming model, researchers only need to specify the “map” and “reduce” functions to discover (organizational) knowledge from (i) big data sets of precise data in a breadth-first manner or depth-first manner and/or from (ii) big data sets of uncertain data. Such a big data analytics process can be sped up by focusing the mining according to the user-specified constraints that express the user interests. The resulting (constrained or unconstrained) frequent patterns mined from big data sets provide users with new insights and a sound understanding of users' patterns. Such (organizational) knowledge is useful is many real-life information science and technology applications.

scholarly journals Big Data Frequent Pattern Mining

2014 ◽  
pp. 225-259 ◽  
Author(s):  
David C. Anastasiu ◽  
Jeremy Iverson ◽  
Shaden Smith ◽  
George Karypis
Keyword(s):  
Big Data ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Frequent Pattern

scholarly journals Frequent Pattern Mining over Unstructured Data using Semi-Structured Doc-Model and Pattern Ranking

2020 ◽  
pp. 36-42
Author(s):  
Sudhir Tirumalasetty ◽  
A. Divya ◽  
D. Rahitya Lakshmi ◽  
Ch. Durga Bhavani ◽  
D. Anusha
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Unstructured Data ◽  
Frequent Pattern ◽  
Frequent Patterns ◽  
Innovative Methods ◽  
Mining Algorithms ◽  
Doc Model

Frequent pattern mining is an essential data-mining task, with a goal of discovering knowledge in the form of repeated patterns. Many efficient pattern-mining algorithms have been discovered in the last two decades, yet most do not scale to the type of data we are presented with today, the so-called “Big Data”. Scalable parallel algorithms hold the key to solving the problem in this context. This paper reviews recent advances in parallel frequent pattern mining, analysing them through the Big Data lens. Load balancing and work partitioning are the major challenges to be conquered. These challenges always invoke innovative methods to do, as Big Data evolves with no limits. The biggest challenge than before is conquering unstructured data for finding frequent patterns. To accomplish this Semi Structured Doc-Model and ranking of patterns are used.

scholarly journals A MapReduce-Based Parallel Frequent Pattern Growth Algorithm for Spatiotemporal Association Analysis of Mobile Trajectory Big Data

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-16 ◽  
Author(s):  
Dawen Xia ◽  
Xiaonan Lu ◽  
Huaqing Li ◽  
Wendong Wang ◽  
Yantao Li ◽  
...  
Keyword(s):  
Big Data ◽  
Association Analysis ◽  
Intelligent Transportation Systems ◽  
Large Scale ◽  
Pattern Mining ◽  
Frequent Pattern Mining ◽  
Transportation Systems ◽  
Frequent Pattern ◽  
Trajectory Data ◽  
Pattern Growth

Frequent pattern mining is an effective approach for spatiotemporal association analysis of mobile trajectory big data in data-driven intelligent transportation systems. While existing parallel algorithms have been successfully applied to frequent pattern mining of large-scale trajectory data, two major challenges are how to overcome the inherent defects of Hadoop to cope with taxi trajectory big data including massive small files and how to discover the implicitly spatiotemporal frequent patterns with MapReduce. To conquer these challenges, this paper presents a MapReduce-based Parallel Frequent Pattern growth (MR-PFP) algorithm to analyze the spatiotemporal characteristics of taxi operating using large-scale taxi trajectories with massive small file processing strategies on a Hadoop platform. More specifically, we first implement three methods, that is, Hadoop Archives (HAR), CombineFileInputFormat (CFIF), and Sequence Files (SF), to overcome the existing defects of Hadoop and then propose two strategies based on their performance evaluations. Next, we incorporate SF into Frequent Pattern growth (FP-growth) algorithm and then implement the optimized FP-growth algorithm on a MapReduce framework. Finally, we analyze the characteristics of taxi operating in both spatial and temporal dimensions by MR-PFP in parallel. The results demonstrate that MR-PFP is superior to existing Parallel FP-growth (PFP) algorithm in efficiency and scalability.

Sign in / Sign up

Export Citation Format

Share Document