The Use of Visual Text Mining to Support the Study Selection Activity in Systematic Literature Reviews: A Replication Study

2013 ◽  
Author(s):  
Katia Romero Felizardo ◽  
Simone R.S. Souza ◽  
Jose Carlos Maldonado
Keyword(s):  
Text Mining ◽  
Replication Study ◽  
Study Selection ◽  
Literature Reviews ◽  
Visual Text

Visual Text Mining: Ensuring the Presence of Relevant Studies in Systematic Literature Reviews

2015 ◽  
Vol 25 (05) ◽  
pp. 909-928 ◽  
Author(s):  
Katia Romero Felizardo ◽  
Ellen Francine Barbosa ◽  
Rafael Messias Martins ◽  
Pedro Henrique Dias Valle ◽  
José Carlos Maldonado
Keyword(s):  
Text Mining ◽  
Replication Study ◽  
Primary Study ◽  
Original Experiment ◽  
Study Selection ◽  
Literature Reviews ◽  
Selection Step ◽  
Visual Text ◽  
And Performance ◽  
Level Of Experience

One of the activities associated with the Systematic Literature Review (SLR) process is the selection review of primary studies. When the researcher faces large volumes of primary studies to be analyzed, the process used to select studies can be arduous. In a previous experiment, we conducted a pilot test to compare the performance and accuracy of PhD students in conducting the selection review activity manually and using Visual Text Mining (VTM) techniques. The goal of this paper is to describe a replication study involving PhD and Master students. The replication study uses the same experimental design and materials of the original experiment. This study also aims to investigate whether the researcher's level of experience with conducting SLRs and research in general impacts the outcome of the primary study selection step of the SLR process. The replication results have confirmed the outcomes of the original experiment, i.e., VTM is promising and can improve the performance of the selection review of primary studies. We also observed that both accuracy and performance increase in function of the researcher's experience level in conducting SLRs. The use of VTM can indeed be beneficial during the selection review activity.

scholarly journals Using Visual Text Mining to Support the Study Selection Activity in Systematic Literature Reviews

2011 ◽  
Author(s):  
Katia R. Felizardo ◽  
Norsaremah Salleh ◽  
Rafael M. Martins ◽  
Emilia Mendes ◽  
Stephen G. MacDonell ◽  
...  
Keyword(s):  
Text Mining ◽  
Study Selection ◽  
Literature Reviews ◽  
Visual Text

scholarly journals Aligning text mining and machine learning algorithms with best practices for study selection in systematic literature reviews

2020 ◽  
Vol 9 (1) ◽  
Author(s):  
E. Popoff ◽  
M. Besada ◽  
J. P. Jansen ◽  
S. Cope ◽  
S. Kanters
Keyword(s):  
Machine Learning ◽  
Text Mining ◽  
Sensitivity And Specificity ◽  
Full Text ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Support Vector ◽  
Study Selection ◽  
Literature Reviews

Abstract Background Despite existing research on text mining and machine learning for title and abstract screening, the role of machine learning within systematic literature reviews (SLRs) for health technology assessment (HTA) remains unclear given lack of extensive testing and of guidance from HTA agencies. We sought to address two knowledge gaps: to extend ML algorithms to provide a reason for exclusion—to align with current practices—and to determine optimal parameter settings for feature-set generation and ML algorithms. Methods We used abstract and full-text selection data from five large SLRs (n = 3089 to 12,769 abstracts) across a variety of disease areas. Each SLR was split into training and test sets. We developed a multi-step algorithm to categorize each citation into the following categories: included; excluded for each PICOS criterion; or unclassified. We used a bag-of-words approach for feature-set generation and compared machine learning algorithms using support vector machines (SVMs), naïve Bayes (NB), and bagged classification and regression trees (CART) for classification. We also compared alternative training set strategies: using full data versus downsampling (i.e., reducing excludes to balance includes/excludes because machine learning algorithms perform better with balanced data), and using inclusion/exclusion decisions from abstract versus full-text screening. Performance comparisons were in terms of specificity, sensitivity, accuracy, and matching the reason for exclusion. Results The best-fitting model (optimized sensitivity and specificity) was based on the SVM algorithm using training data based on full-text decisions, downsampling, and excluding words occurring fewer than five times. The sensitivity and specificity of this model ranged from 94 to 100%, and 54 to 89%, respectively, across the five SLRs. On average, 75% of excluded citations were excluded with a reason and 83% of these citations matched the reviewers’ original reason for exclusion. Sensitivity significantly improved when both downsampling and abstract decisions were used. Conclusions ML algorithms can improve the efficiency of the SLR process and the proposed algorithms could reduce the workload of a second reviewer by identifying exclusions with a relevant PICOS reason, thus aligning with HTA guidance. Downsampling can be used to improve study selection, and improvements using full-text exclusions have implications for a learn-as-you-go approach.

scholarly journals An Approach Based on Visual Text Mining to Support Categorization and Classification in the Systematic Mapping

2010 ◽  
Author(s):  
Katia Romero Felizardo ◽  
Elisa Yumi Nakagawa ◽  
Daniel R.C. Feitosa ◽  
Rosane Minghim ◽  
José Carlos Maldonado
Keyword(s):  
Text Mining ◽  
Systematic Mapping ◽  
Visual Text

A Survey of Selected Software Technologies for Text Mining

2009 ◽  
pp. 1164-1181
Author(s):  
Richard S. Segall ◽  
Qingyu Zhang
Keyword(s):  
Data Analysis ◽  
Text Mining ◽  
Text Analysis ◽  
Web Mining ◽  
Qualitative Data ◽  
Future Trends ◽  
Data Preparation ◽  
Software Packages ◽  
Visual Text ◽  
Key Steps

This chapter presents background on text mining, and comparisons and summaries of seven selected software for text mining. The text mining software selected for discussion and comparison in this chapter are: Compare Suite by AKS-Labs, SAS Text Miner, Megaputer Text Analyst, Visual Text by Text Analysis International, Inc. (TextAI), Magaputer PolyAnalyst, WordStat by Provalis Research, and SPSS Clementine. This chapter not only discusses unique features of these text mining software packages but also compares the features offered by each in the following key steps in analyzing unstructured qualitative data: data preparation, data analysis, and result reporting. A brief discussion of Web mining and its software are also presented, as well as conclusions and future trends.

Visual text mining using association rules

2007 ◽  
Vol 31 (3) ◽  
pp. 316-326 ◽  
Author(s):  
A.A. Lopes ◽  
R. Pinho ◽  
F.V. Paulovich ◽  
R. Minghim
Keyword(s):  
Text Mining ◽  
Association Rules ◽  
Visual Text
Sign in / Sign up

Export Citation Format

Share Document