scholarly journals Bidirectional Language Modeling: A Systematic Literature Review

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Muhammad Shah Jahan ◽  
Habib Ullah Khan ◽  
Shahzad Akbar ◽  
Muhammad Umar Farooq ◽  
Sarah Gul ◽  
...  
Keyword(s):  
Literature Review ◽  
Transfer Learning ◽  
Systematic Literature Review ◽  
Fine Tuning ◽  
Language Models ◽  
Small Step ◽  
Step Size ◽  
Large Dataset ◽  
Small Step Size ◽  
Research Studies

In transfer learning, two major activities, i.e., pretraining and fine-tuning, are carried out to perform downstream tasks. The advent of transformer architecture and bidirectional language models, e.g., bidirectional encoder representation from transformer (BERT), enables the functionality of transfer learning. Besides, BERT bridges the limitations of unidirectional language models by removing the dependency on the recurrent neural network (RNN). BERT also supports the attention mechanism to read input from any side and understand sentence context better. It is analyzed that the performance of downstream tasks in transfer learning depends upon the various factors such as dataset size, step size, and the number of selected parameters. In state-of-the-art, various research studies produced efficient results by contributing to the pretraining phase. However, a comprehensive investigation and analysis of these research studies is not available yet. Therefore, in this article, a systematic literature review (SLR) is presented investigating thirty-one (31) influential research studies published during 2018–2020. Following contributions are made in this paper: (1) thirty-one (31) models inspired by BERT are extracted. (2) Every model in this paper is compared with RoBERTa (replicated BERT model) having large dataset and batch size but with a small step size. It is concluded that seven (7) out of thirty-one (31) models in this SLR outperforms RoBERTa in which three were trained on a larger dataset while the other four models are trained on a smaller dataset. Besides, among these seven models, six models shared both feedforward network (FFN) and attention across the layers. Rest of the twenty-four (24) models are also studied in this SLR with different parameter settings. Furthermore, it has been concluded that a pretrained model with a large dataset, hidden layers, attention heads, and small step size with parameter sharing produces better results. This SLR will help researchers to pick a suitable model based on their requirements.

Computer Science Unplugged: A Systematic Literature Review

2021 ◽  
pp. 004723952110188
Author(s):  
Ali Battal ◽  
Gülgün Afacan Adanır ◽  
Yasemin Gülbahar
Keyword(s):  
Literature Review ◽  
Computer Science ◽  
Systematic Literature Review ◽  
Computational Thinking ◽  
Thinking Skills ◽  
Inclusion Criteria ◽  
Relevant Research ◽  
Tangible Programming ◽  
Review Procedure ◽  
Research Studies

The computer science (CS) unplugged approach intends to teach CS concepts and computational thinking skills without employing any digital tools. The current study conducted a systematic literature review to analyze research studies that conducted investigations related to implementations of CS unplugged activities. A systematic review procedure was developed and applied to detect and subsequently review relevant research studies published from 2010 to 2019. It was found that 55 research studies (17 articles + 38 conference proceedings) satisfied the inclusion criteria for the analysis. These research studies were then examined with regard to their demographic characteristics, research methodologies, research results, and main findings. It was found that the unplugged approach was realized and utilized differently among researchers. The majority of the studies used the CS unplugged term when referring to “paper–pencil activities,” “problem solving,” “storytelling,” “games,” “tangible programming,” and even “robotics.”

scholarly journals Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study (Preprint)

2019 ◽  
Author(s):  
Derek Howard ◽  
Marta M Maslej ◽  
Justin Lee ◽  
Jacob Ritchie ◽  
Geoffrey Woollard ◽  
...  
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Social Media ◽  
Transfer Learning ◽  
Computational Linguistics ◽  
Feature Representation ◽  
Fine Tuning ◽  
Language Models ◽  
Universal Sentence ◽  
Text Feature

BACKGROUND Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data that can be mined to predict mental health states using machine learning methods. OBJECTIVE This study aimed to benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools. We tested on datasets that contain posts labeled for perceived suicide risk or moderator attention in the context of self-harm. Specifically, we assessed the ability of the methods to prioritize posts that a moderator would identify for immediate response. METHODS We used 1588 labeled posts from the Computational Linguistics and Clinical Psychology (CLPsych) 2017 shared task collected from the Reachout.com forum. Posts were represented using lexicon-based tools, including Valence Aware Dictionary and sEntiment Reasoner, Empath, and Linguistic Inquiry and Word Count, and also using pretrained artificial neural network models, including DeepMoji, Universal Sentence Encoder, and Generative Pretrained Transformer-1 (GPT-1). We used Tree-based Optimization Tool and Auto-Sklearn as AutoML tools to generate classifiers to triage the posts. RESULTS The top-performing system used features derived from the GPT-1 model, which was fine-tuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macroaveraged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from metadata or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. In addition, we have presented visualizations that aid in the understanding of the learned classifiers. CONCLUSIONS In this study, we found that transfer learning is an effective strategy for predicting risk with relatively little labeled data and noted that fine-tuning of pretrained language models provides further gains when large amounts of unlabeled text are available.

scholarly journals The RSC Chromatin Remodeling ATPase can Translocate DNA with High Force and Small Step Size

2011 ◽  
Vol 100 (3) ◽  
pp. 354a
Author(s):  
George Sirinakis ◽  
Cedric R. Clapier ◽  
Ying Gao ◽  
Ramya Viswanathan ◽  
Bradley R. Cairns ◽  
...  
Keyword(s):  
Chromatin Remodeling ◽  
Small Step ◽  
Step Size ◽  
Small Step Size

scholarly journals Spam Review Detection:A Systematic Literature Review

2020 ◽  
Author(s):  
Muhammad Shoaib Farooq
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Literature Review ◽  
Systematic Literature Review ◽  
State Of The Art ◽  
The State ◽  
Research Gaps ◽  
Art Research ◽  
Fake Reviews ◽  
Research Studies

In this era of technology, people rely on online posted reviews before buying any product. These reviews are very important for both the consumers and people. Consumers and people use this information for decision making while buying products or investing money in any product. This has inclined the spammers to generate spam or fake reviews so that they can recommend their products and beat the competitors. Spammers have developed many systems to generate the bulk of spam reviews within hours. Many techniques, strategies have been designed and recommended to resolve the issue of spam reviews. In this paper, a complete review of existing techniques and strategies for detecting spam review is discussed. Apart from reviewing the state-of-the-art research studies on spam review detection, a taxonomy on techniques of machine learning for spam review detection has been proposed. Moreover, its focus on research gaps and future recommendations for spam review identification.

scholarly journals Automated Paraphrase Quality Assessment Using Language Models and Transfer Learning

Computers ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 166
Author(s):  
Bogdan Nicula ◽  
Mihai Dascalu ◽  
Natalie N. Newton ◽  
Ellen Orcutt ◽  
Danielle S. McNamara
Keyword(s):  
Quality Assessment ◽  
Transfer Learning ◽  
Network Models ◽  
Learning Task ◽  
Fine Tuning ◽  
Language Models ◽  
Neural Network Models ◽  
Writing Ability ◽  
Four Dimensions ◽  
Timely Feedback

Learning to paraphrase supports both writing ability and reading comprehension, particularly for less skilled learners. As such, educational tools that integrate automated evaluations of paraphrases can be used to provide timely feedback to enhance learner paraphrasing skills more efficiently and effectively. Paraphrase identification is a popular NLP classification task that involves establishing whether two sentences share a similar meaning. Paraphrase quality assessment is a slightly more complex task, in which pairs of sentences are evaluated in-depth across multiple dimensions. In this study, we focus on four dimensions: lexical, syntactical, semantic, and overall quality. Our study introduces and evaluates various machine learning models using handcrafted features combined with Extra Trees, Siamese neural networks using BiLSTM RNNs, and pretrained BERT-based models, together with transfer learning from a larger general paraphrase corpus, to estimate the quality of paraphrases across the four dimensions. Two datasets are considered for the tasks involving paraphrase quality: ULPC (User Language Paraphrase Corpus) containing 1998 paraphrases and a smaller dataset with 115 paraphrases based on children’s inputs. The paraphrase identification dataset used for the transfer learning task is the MSRP dataset (Microsoft Research Paraphrase Corpus) containing 5801 paraphrases. On the ULPC dataset, our BERT model improves upon the previous baseline by at least 0.1 in F1-score across the four dimensions. When using fine-tuning from ULPC for the children dataset, both the BERT and Siamese neural network models improve upon their original scores by at least 0.11 F1-score. The results of these experiments suggest that transfer learning using generic paraphrase identification datasets can be successful, while at the same time obtaining comparable results in fewer epochs.

scholarly journals Prioritizing marketing research in virtual reality: development of an immersion/fantasy typology

2019 ◽  
Vol 53 (8) ◽  
pp. 1585-1611 ◽  
Author(s):  
Kirsten Cowan ◽  
Seth Ketron
Keyword(s):  
Virtual Reality ◽  
Literature Review ◽  
Systematic Literature Review ◽  
New Technologies ◽  
Marketing Research ◽  
Content Type ◽  
Future Studies ◽  
Research Opportunities ◽  
Retail Service ◽  
Research Studies

Purpose Virtual reality (VR) is of increasing interest to marketers because it can be used to explore and proactively shape long-term futures, co-create value with consumers, and foster consumer-brand engagement. Yet, to date, the field lacks a cohesive framework for approaching VR research; thus, the objective of this systematic literature review is to provide such a framework and highlight research opportunities. Design/methodology/approach First, after conducting a systematic literature review, we highlight VR themes instrumental to flow and propose a typology for VR research using realism-fantasy and immersion as dimensions. Next, we review the current state of empirical research for each quadrant. Finally, we synthesize research within each quadrant, specifying criteria and considerations for conducting research. In doing so, we propose an agenda for marketing research, centered on methodological, future studies, and consumer-related contributions. Findings VR themes instrumental to flow include the avatar, application quality, and interactivity. We find, contrary to some conceptualizations of VR, that all applications are capable of producing flow. Conflicting research and gaps are highlighted in the findings section and summarized in Table III. Additionally, while prior research seems to draw from findings of other VR applications in advancing knowledge in general, the results of the literature review suggest that VR applications should be treated uniquely. Finally, we propose highly immersive VR applications as more conducive to future studies research. Research limitations/implications Scholars can utilize the findings to prioritize future research studies in marketing. By following the typology and research opportunities, scholars can advance marketing theory and enhance the external validity of research studies through VR applications. Practical implications Managers can utilize the findings to ascertain consumers and societies‘ responses to various marketing stimuli, with implications for product development, branding, retail/service experiences, adoption of new technologies, tourism, and many other domains. VR applications offer managers more ways of testing concepts and processes in realistic fashion without the costs and risks associated with more traditional methods. Originality/value The objective of this paper is to examine varying opportunities for VR research given flow and fantasy potential and to prioritize VR research.

scholarly journals The RSC chromatin remodelling ATPase translocates DNA with high force and small step size

2011 ◽  
Vol 30 (12) ◽  
pp. 2364-2372 ◽  
Author(s):  
George Sirinakis ◽  
Cedric R Clapier ◽  
Ying Gao ◽  
Ramya Viswanathan ◽  
Bradley R Cairns ◽  
...  
Keyword(s):  
Chromatin Remodelling ◽  
Small Step ◽  
Step Size ◽  
Small Step Size

scholarly journals Qawaid Tafsir dan Ushul Tafsir Siti Aisyah dalam Kitab Sahih Muslim

2021 ◽  
Vol 18 (2) ◽  
pp. 152
Author(s):  
Andri Nirwana AN
Keyword(s):  
Literature Review ◽  
Systematic Literature Review ◽  
Special Section ◽  
Linguistic Rules ◽  
Literature Reviews ◽  
Research Questions ◽  
Level Of Understanding ◽  
Research Studies

Qawaid Tafsir has a purpose, namely the rules needed by commentators in understanding the verses of the Qur'an. The rules needed by the exegetes in understanding the Qur'an include appreciation of its uslubs, understanding of its origins, mastery of its secrets and linguistic rules. Siti Aisyah's contribution in the Interpretation of the Qur'an has become a reference for many mufassirins, whose interpretation results are written in the books of Hadith, there is even a special section of the Muslim hadith books, namely the chapter on the hadith books of interpretation. How qawaid and ushul interpretation siti aisyah in the book of hadith Sahih Muslim is the goal of solving the problem of this article. The method used in this research is Systematic Literature Review (SLR) which is a systematic way to collect, critically evaluate, integrate and present findings from various research studies on research questions or topics of interest. The SLR provides a way to assess the level of quality of existing evidence on a question or topic of interest. The SLR provides a broader and more accurate level of understanding than traditional literature reviews. The results of this study were found five verses interpreted by Siti Aisyah in the book of Hadith Sahih Muslim. The details of the explanation can be seen in the discussion chapter. This research is useful for developing the results of the interpretation of the Companions in mapping the methodology of interpretation, qawaid Tafsir and Usul Tafsir.ABSTRAKQawaid Tafsir mempunyai maksud ialah kaidah-kaidah yang diperlukan oleh para mufasir dalam memahami ayat-ayat Al-Qur’an. Kaidah-kaidah yang diperlukan para mufasir dalam memahami Al-Qur’an meliputi penghayatan uslub-uslub nya, pemahaman asal-asal nya, penguasaan rahasia-rahasia nya dan qaidah-qaidah kebahasaan. Kontribusi Siti Aisyah dalam Penafsiran Al Qur’an banyak menjadi referensi bagi para mufassirin, yang hasil dari penafsiran nya tersebut tertulis dalam kitab kitab Hadis, bahkan ada bagian khusus dari kitab hadis Muslim yatu bab kitab hadis tafsir. Bagaimana qawaid dan ushul tafsir siti aisyah dalam kitab hadis sahih Muslim menjadi tujuan dari penyelesaian masalah artikel ini. Metode yang digunakan dalam penelitian ini adalah Systematic Literature Review (SLR) adalah cara sistematis untuk mengumpulkan, mengevaluasi secara kritis, mengintegrasikan dan menyajikan temuan dari berbagai studi penelitian pada pertanyaan penelitian atau topik yang menarik. SLR menyediakan cara untuk menilai tingkat kualitas bukti yang ada pada pertanyaan atau topik yang menarik. SLR memberikan tingkat pemahaman yang lebih luas dan lebih akurat daripada tinjauan literatur secara tradisional. Adapun hasil dari penelitian ini adalah ditemukan lima ayat yang ditafsirkan oleh siti aisyah dalam kitab hadis Sahih Muslim. Adapun rincian penjelasan nya bisa melihat pada bab pembahasan. Penelitian ini berguna untuk mengambangkan hasil penafsiran para sahabat dalam pemeteaan metodologi tafsir, qawaid Tafsir dan Ushul tafsir..

Sign in / Sign up

Export Citation Format

Share Document