scholarly journals OLAP on multidimensional text databases: Topic network cube and its applications

Filomat ◽  
2018 ◽  
Vol 32 (5) ◽  
pp. 1973-1982
Author(s):  
Zhiyuan Zhang ◽  
Hong Wang ◽  
Xingjie Feng
Keyword(s):  
Gibbs Sampling ◽  
Numerical Data ◽  
Experimental Results ◽  
Related Topic ◽  
Text Data ◽  
Text Databases ◽  
Unstructured Text ◽  
Structured Information ◽  
Analytical Processing ◽  
Olap Analysis

Multidimensional text data contains both structured attributes and unstructured text. Unlike the traditional numerical data, it is not straightforward to apply online analytical processing on multidimensional text data. Although some OLAP methods such as topic cube have been proposed in order to effectively utilize its structured information and valuable text data, these methods cant tell the relations of topicwords. Considering that topics usually consist of several subtopics and each subtopic usually contains some topic words, we here use a topic network manner, in which related topic words are connected, to express the complex relations of topics. This paper introduces a new concept of topic network cube to perform OLAP analysis on multidimensional text databases. Firstly, we propose a method called GL-LDA based on Gibbs sampling outputs of Labeled LDA to measure the relations between topic words. Secondly, we give a storagemodel of topic network cubewhich can efficiently generate topic network using GL-LDA. Thirdly, we show how to perform OLAP analysis on topic network cube. Experimental results show that we can analyze multidimensional text databases in different granularity easily and effectively using just a few simple SQL statements, and the output network provides rich and useful information of topics.

Incorporating Text OLAP in Business Intelligence

2011 ◽  
pp. 77-101 ◽  
Author(s):  
Byung-Kwon Park ◽  
Il-Yeol Song
Keyword(s):  
Information Retrieval ◽  
Text Mining ◽  
Business Intelligence ◽  
Multidimensional Analysis ◽  
Web Pages ◽  
Data Types ◽  
Text Documents ◽  
Text Data ◽  
Platform Architecture ◽  
Unstructured Text

As the amount of data grows very fast inside and outside of an enterprise, it is getting important to seamlessly analyze both data types for total business intelligence. The data can be classified into two categories: structured and unstructured. For getting total business intelligence, it is important to seamlessly analyze both of them. Especially, as most of business data are unstructured text documents, including the Web pages in Internet, we need a Text OLAP solution to perform multidimensional analysis of text documents in the same way as structured relational data. We first survey the representative works selected for demonstrating how the technologies of text mining and information retrieval can be applied for multidimensional analysis of text documents, because they are major technologies handling text data. And then, we survey the representative works selected for demonstrating how we can associate and consolidate both unstructured text documents and structured relation data for obtaining total business intelligence. Finally, we present a future business intelligence platform architecture as well as related research topics. We expect the proposed total heterogeneous business intelligence architecture, which integrates information retrieval, text mining, and information extraction technologies all together, including relational OLAP technologies, would make a better platform toward total business intelligence.

Text-Driven Reasoning and Multi-Structured Data Analytics for Business Intelligence

2016 ◽  
pp. 1-32 ◽  
Author(s):  
Lipika Dey ◽  
Ishan Verma
Keyword(s):  
Business Intelligence ◽  
Data Analytics ◽  
Statistical Significance ◽  
Heterogeneous Data ◽  
Structured Data ◽  
Text Data ◽  
Business Operations ◽  
Unstructured Text ◽  
Business Data ◽  
Heterogeneous Resources

Business Intelligence (BI) refers to an organization's capability to gather and analyze data about business operations and transactions in order to evaluate its performance. The abundance of information both within the enterprise and outside of it has necessitated a change in traditional Business Intelligence practices. There is a need to exploit heterogeneous resources. Text data like news, analyst reports, etc. helps in better interpretation of business data. In this chapter, the authors present a futuristic BI framework that facilitates acquisition, indexing, and analysis of heterogeneous data for extracting business intelligence. It enables integration of unstructured text data and structured business data seamlessly to generate insights. The authors propose methods that can help in extraction of events or significant happenings from both unstructured and structured data, correlate the events, and thereafter reason to generate insights. The insights extracted could be validated as cause-effect pairs based on the statistical significance of co-occurrence of events.

scholarly journals Weighted k-Prototypes Clustering Algorithm Based on the Hybrid Dissimilarity Coefficient

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Ziqi Jia ◽  
Ling Song
Keyword(s):  
Categorical Data ◽  
Clustering Algorithm ◽  
Numerical Data ◽  
Experimental Results ◽  
Cluster Center ◽  
Real Dataset ◽  
Dissimilarity Coefficient ◽  
Initial Cluster ◽  
Data Objects ◽  
Selection Of

The k-prototypes algorithm is a hybrid clustering algorithm that can process Categorical Data and Numerical Data. In this study, the method of initial Cluster Center selection was improved and a new Hybrid Dissimilarity Coefficient was proposed. Based on the proposed Hybrid Dissimilarity Coefficient, a weighted k-prototype clustering algorithm based on the hybrid dissimilarity coefficient was proposed (WKPCA). The proposed WKPCA algorithm not only improves the selection of initial Cluster Centers, but also puts a new method to calculate the dissimilarity between data objects and Cluster Centers. The real dataset of UCI was used to test the WKPCA algorithm. Experimental results show that WKPCA algorithm is more efficient and robust than other k-prototypes algorithms.

scholarly journals Modelling the sensory space of varietal wines: Mining of large, unstructured text data and visualisation of style patterns

2018 ◽  
Vol 8 (1) ◽  
Author(s):  
Carlo C. Valente ◽  
Florian F. Bauer ◽  
Fritz Venter ◽  
Bruce Watson ◽  
Hélène H. Nieuwoudt
Keyword(s):  
Text Data ◽  
Unstructured Text ◽  
Sensory Space

scholarly journals Effective processing of unstructured data using python in Hadoop map reduce

2018 ◽  
Vol 7 (2.21) ◽  
pp. 417
Author(s):  
K Kousalya ◽  
Shaik Javed Parvez
Keyword(s):  
Open Source ◽  
Unstructured Data ◽  
Map Reduce ◽  
Text Data ◽  
Apache Hadoop ◽  
Unstructured Text ◽  
Wide Range ◽  
Two Stages

In present scenario, the growing data are naturally unstructured. In this case to handle the wide range of data, is difficult. The proposed paper is to process the unstructured text data effectively in Hadoop map reduce using Python. Apache Hadoop is an open source platform and it widely uses Map Reduce framework. Map Reduce is popular and effective for processing the unstructured data in parallel manner.  There are two stages in map reduce, namely transform and repository. Here the input splits into small blocks and worker node process individual blocks in parallel. This map reduce generally is based on java. While Hadoop Streaming allows writing mapper and reducer in other languages like Python. In this paper, we are going to show an alternative way of processing the growing unstructured content data by using python. We will also compare the performance between java based and non-java based programs. 

TLabel

2016 ◽  
Vol 12 (4) ◽  
pp. 54-74 ◽  
Author(s):  
Lamia Oukid ◽  
Omar Boussaid ◽  
Nadjia Benblidia ◽  
Fadila Bentayeb
Keyword(s):  
Text Categorization ◽  
Numerical Data ◽  
Semantic Content ◽  
Aggregation Operators ◽  
Structured Data ◽  
Wide Range ◽  
Textual Data ◽  
On Line ◽  
Analytical Processing ◽  
On Line Analytical Processing

Data Warehousing technologies and On-Line Analytical Processing (OLAP) feature a wide range of techniques for the analysis of structured data. However, these techniques are inadequate when it comes to analyzing textual data. Indeed, classical aggregation operators have earned their spurs in the online analysis of numerical data, but are unsuitable for the analysis of textual data. To alleviate this shortcoming, on-line analytical processing in text cubes requires new analysis operators adapted to textual data. In this paper, the authors propose a new aggregation operator named Text Label (TLabel), based on text categorization. Their operator aggregates textual data in several classes of documents. Each class is associated with a label that represents the semantic content of the textual data of the class. TLabel is founded on a tailoring of text mining techniques to OLAP. To validate their operator, the authors perform an experimental study and the preliminary results show the interest of their approach for Text OLAP.

Sign in / Sign up

Export Citation Format

Share Document