scholarly journals An Appraisal of Incremental Learning Methods

Entropy ◽  
2020 ◽  
Vol 22 (11) ◽  
pp. 1190
Author(s):  
Yong Luo ◽  
Liancheng Yin ◽  
Wenchao Bai ◽  
Keming Mao
Keyword(s):  
Incremental Learning ◽  
Computational Models ◽  
Network Models ◽  
Original Data ◽  
Development Trend ◽  
Research Area ◽  
Neural Network Models ◽  
Learning Methods ◽  
Rehearsal Strategy ◽  
Artificial Intelligence Technology

As a special case of machine learning, incremental learning can acquire useful knowledge from incoming data continuously while it does not need to access the original data. It is expected to have the ability of memorization and it is regarded as one of the ultimate goals of artificial intelligence technology. However, incremental learning remains a long term challenge. Modern deep neural network models achieve outstanding performance on stationary data distributions with batch training. This restriction leads to catastrophic forgetting for incremental learning scenarios since the distribution of incoming data is unknown and has a highly different probability from the old data. Therefore, a model must be both plastic to acquire new knowledge and stable to consolidate existing knowledge. This review aims to draw a systematic review of the state of the art of incremental learning methods. Published reports are selected from Web of Science, IEEEXplore, and DBLP databases up to May 2020. Each paper is reviewed according to the types: architectural strategy, regularization strategy and rehearsal and pseudo-rehearsal strategy. We compare and discuss different methods. Moreover, the development trend and research focus are given. It is concluded that incremental learning is still a hot research area and will be for a long period. More attention should be paid to the exploration of both biological systems and computational models.

Computational Models in Developmental Psychology

2013 ◽  
pp. 476-504
Author(s):  
Thomas R. Shultz
Keyword(s):  
Neural Networks ◽  
Computational Modeling ◽  
Computational Models ◽  
Developmental Psychology ◽  
Probability Distributions ◽  
Network Models ◽  
Developmental Theory ◽  
Negative Evidence ◽  
Developmental Robotics ◽  
Neural Network Models

Computational modeling implements developmental theory in a precise manner, allowing generation, explanation, integration, and prediction. Several modeling techniques are applied to development: symbolic rules, neural networks, dynamic systems, Bayesian processing of probability distributions, developmental robotics, and mathematical analysis. The relative strengths and weaknesses of each approach are identified and examples of each technique are described. Ways in which computational modeling contributes to developmental issues are documented. A probabilistic model of the vocabulary spurt shows that various psychological explanations for it are unnecessary. Constructive neural networks clarify the distinction between learning and development and show how it is possible to escape Fodor’s paradox. Connectionist modeling reveals different versions of innateness and how learning and evolution might interact. Agent-based models analyze the basic principles of evolution in a testable, experimental fashion that generates complete evolutionary records. Challenges posed by stimulus poverty and lack of negative examples are explored in neural-network models that learn morphology or syntax probabilistically from indirect negative evidence.

scholarly journals Modelling MTPL insurance claim events: Can machine learning methods overperform the traditional GLM approach?

2021 ◽  
Vol 4 (2) ◽  
pp. 34-69
Author(s):  
Dávid Burka ◽  
László Kovács ◽  
László Szepesváry
Keyword(s):  
Machine Learning ◽  
Network Models ◽  
Statistical Modelling ◽  
Third Party ◽  
Insurance Companies ◽  
Neural Network Models ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Long Time ◽  
Generalised Linear Modelling

Pricing an insurance product covering motor third-party liability is a major challenge for actuaries. Comprehensive statistical modelling and modern computational power are necessary to solve this problem. The generalised linear and additive modelling approaches have been widely used by insurance companies for a long time. Modelling with modern machine learning methods has recently started, but applying them properly with relevant features is a great issue for pricing experts. This study analyses the claim-causing probability by fitting generalised linear modelling, generalised additive modelling, random forest, and neural network models. Several evaluation measures are used to compare these techniques. The best model is a mixture of the base methods. The authors’ hypothesis about the existence of significant interactions between feature variables is proved by the models. A simplified classification and visualisation is performed on the final model, which can support tariff applications later.

scholarly journals Transfer Learning Methods as a New Approach in Computer Vision Tasks with Small Datasets

2020 ◽  
Vol 45 (3) ◽  
pp. 179-193
Author(s):  
Andrzej Brodzicki ◽  
Michal Piekarski ◽  
Dariusz Kucharski ◽  
Joanna Jaworek-Korjakowska ◽  
Marek Gorgon
Keyword(s):  
Computer Vision ◽  
Transfer Learning ◽  
Network Models ◽  
Classification Problems ◽  
Quality Of Data ◽  
Neural Network Models ◽  
Learning Methods ◽  
New Approach ◽  
Thickness Prediction

AbstractDeep learning methods, used in machine vision challenges, often face the problem of the amount and quality of data. To address this issue, we investigate the transfer learning method. In this study, we briefly describe the idea and introduce two main strategies of transfer learning. We also present the widely-used neural network models, that in recent years performed best in ImageNet classification challenges. Furthermore, we shortly describe three different experiments from computer vision field, that confirm the developed algorithms ability to classify images with overall accuracy 87.2-95%. Achieved numbers are state-of-the-art results in melanoma thickness prediction, anomaly detection and Clostridium di cile cytotoxicity classification problems.

scholarly journals Satellite IoT Edge Intelligent Computing: A Research on Architecture

Electronics ◽  
2019 ◽  
Vol 8 (11) ◽  
pp. 1247 ◽  
Author(s):  
Junyong Wei ◽  
Jiarong Han ◽  
Suzhi Cao
Keyword(s):  
Satellite Data ◽  
Satellite Communication ◽  
Network Models ◽  
Development Trend ◽  
Utilization Rate ◽  
Neural Network Models ◽  
Computing Architecture ◽  
Intelligent Computing ◽  
Speed Up ◽  
The Development Trend

As the number of satellites continues to increase, satellites become an important part of the IoT and 5G/6G communications. How to deal with the data of the satellite Internet of Things is a problem worth considering and paying attention to. Due to the current on-board processing capability and the limitation of the inter-satellite communication rate, the data acquisition from the satellite has a higher delay and the data utilization rate is lower. In order to use the data generated by the satellite IoT more effectively, we propose a satellite IoT edge intelligent computing architecture. In the article, we analyze the current methods of satellite data processing, combined with the development trend of future satellites, and use the characteristics of edge computing and machine learning to describe the satellite IoT edge intelligent computing architecture. Finally, we verify that the architecture can speed up the processing of satellite data. By demonstrating the performance of different neural network models in the satellite edge intelligent computing architecture, we can find that the lightweight of neural networks can promote the development of satellite IoT edge intelligent computing architecture.

scholarly journals Development and Application of Back-Propagation-Based Artificial Neural Network Models in Solving Engineering

2002 ◽  
Vol 7 (1) ◽  
pp. 55
Author(s):  
Saleh Mohammed Al-Alawi
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Model Development ◽  
Back Propagation ◽  
Network Models ◽  
Research Area ◽  
Ann Model ◽  
Neural Network Models ◽  
Fields Of Study ◽  
Artificial Neural

Artificial Neural Networks (ANNs) are computer software programs that mimic the human brain's ability to classify patterns or to make forecasts or decisions based on past experience.  The development of this research area can be attributed to two factors, sufficient computer power to begin practical ANN-based research in the late 1970s and the development of back-propagation in 1986 that enabled ANN models to solve everyday business, scientific, and industrial problems.  Since then, significant applications have been implemented in several fields of study, and many useful intelligent applications and systems have been developed.  The objective of this paper is to generate awareness and to encourage applications development using artificial intelligence-based systems.  Therefore, this paper provides basic ANN concepts, outlines steps used for ANN model development, and lists examples of engineering applications based on the use of the back-propagation paradigm conducted in Oman.  The paper is intended to provide guidelines and necessary references and resources for novice individuals interested in conducting research in engineering or other fields of study using back-propagation artificial neural networks.      

scholarly journals Prediction Models to Control Aging Time in Red Wine

2019 ◽  
Author(s):  
G Astray ◽  
JC Mejuto ◽  
V Martinez-Martinez ◽  
I Nevares ◽  
M Alamo-Sanza ◽  
...  
Keyword(s):  
Aging Time ◽  
Computational Models ◽  
Red Wine ◽  
Prediction Models ◽  
Network Models ◽  
Support Vector ◽  
Red Wines ◽  
Ann Model ◽  
Neural Network Models ◽  
Artificial Neural Network Models

A combination of physical-chemical analysis has been used to monitor the aging of red wines from D.O. Toro (Spain). The changes in the chemical composition of wines that occur along aging time can be permitted to discriminate wine samples collected after one, four, seven and ten months of aging. Different computational models were used to develop a good authenticity tool to certificate wines. In this research different models have developed: Artificial Neural Network models (ANNs), Support Vector Machine (SVM) and Random Forest (RF) models. The results obtained for the ANN model developed with sigmoidal function in the output neuron and the RF model permit to determine the aging time, with an average absolute percentage deviation below 1% and it can conclude that these two models have demonstrated its capacity as a valid tool to predict the wine age.

scholarly journals Deep Learning-Based Energy Disaggregation and On/Off Detection of Household Appliances

2021 ◽  
Vol 15 (3) ◽  
pp. 1-21
Author(s):  
Jie Jiang ◽  
Qiuqiang Kong ◽  
Mark D. Plumbley ◽  
Nigel Gilbert ◽  
Mark Hoogendoorn ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Energy Consumption ◽  
Network Models ◽  
Binary Classifier ◽  
Neural Network Models ◽  
Learning Methods ◽  
Target Values ◽  
Energy Disaggregation

Energy disaggregation, a.k.a. Non-Intrusive Load Monitoring, aims to separate the energy consumption of individual appliances from the readings of a mains power meter measuring the total energy consumption of, e.g., a whole house. Energy consumption of individual appliances can be useful in many applications, e.g., providing appliance-level feedback to the end users to help them understand their energy consumption and ultimately save energy. Recently, with the availability of large-scale energy consumption datasets, various neural network models such as convolutional neural networks and recurrent neural networks have been investigated to solve the energy disaggregation problem. Neural network models can learn complex patterns from large amounts of data and have been shown to outperform the traditional machine learning methods such as variants of hidden Markov models. However, current neural network methods for energy disaggregation are either computational expensive or are not capable of handling long-term dependencies. In this article, we investigate the application of the recently developed WaveNet models for the task of energy disaggregation. Based on a real-world energy dataset collected from 20 households over 2 years, we show that WaveNet models outperforms the state-of-the-art deep learning methods proposed in the literature for energy disaggregation in terms of both error measures and computational cost. On the basis of energy disaggregation, we then investigate the performance of two deep-learning based frameworks for the task of on/off detection which aims at estimating whether an appliance is in operation or not. The first framework obtains the on/off states of an appliance by binarising the predictions of a regression model trained for energy disaggregation, while the second framework obtains the on/off states of an appliance by directly training a binary classifier with binarised energy readings of the appliance serving as the target values. Based on the same dataset, we show that for the task of on/off detection the second framework, i.e., directly training a binary classifier, achieves better performance in terms of F1 score.

scholarly journals Inferring brain-wide interactions using data-constrained recurrent neural network models

2020 ◽  
Author(s):  
Matthew G. Perich ◽  
Charlotte Arlt ◽  
Sofia Soares ◽  
Megan E. Young ◽  
Clayton P. Mosher ◽  
...  
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Large Scale ◽  
Computational Models ◽  
Network Models ◽  
Brain Regions ◽  
Neural Network Models ◽  
Neural Data ◽  
Neural Recordings ◽  
Using Data

ABSTRACTBehavior arises from the coordinated activity of numerous anatomically and functionally distinct brain regions. Modern experimental tools allow unprecedented access to large neural populations spanning many interacting regions brain-wide. Yet, understanding such large-scale datasets necessitates both scalable computational models to extract meaningful features of interregion communication and principled theories to interpret those features. Here, we introduce Current-Based Decomposition (CURBD), an approach for inferring brain-wide interactions using data-constrained recurrent neural network models that directly reproduce experimentally-obtained neural data. CURBD leverages the functional interactions inferred by such models to reveal directional currents between multiple brain regions. We first show that CURBD accurately isolates inter-region currents in simulated networks with known dynamics. We then apply CURBD to multi-region neural recordings obtained from mice during running, macaques during Pavlovian conditioning, and humans during memory retrieval to demonstrate the widespread applicability of CURBD to untangle brain-wide interactions underlying behavior from a variety of neural datasets.

scholarly journals Development of digital twin of CNC unit based on machine learning methods

2019 ◽  
Vol 19 (1) ◽  
pp. 45-55 ◽  
Author(s):  
Yu. G. Kabaldin ◽  
D. A. Shatagin ◽  
M. S. Anosov ◽  
A. M. Kuzmishina
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Dynamic Stability ◽  
Network Models ◽  
Sensor System ◽  
Cnc Machine ◽  
Neural Network Models ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Digital Twins

Introduction. It is shown that the digital twin (electronic passport) of a CNC machine is developed as a cyber-physical system. The work objective is to create neural network models to determine the operation of a CNC machine, its performance and dynamic stability under cutting.Materials and Methods. The development of mathematical models of machining processes using a sensor system and the Industrial Internet of Things is considered. Machine learning methods valid for the implementation of the above tasks are evaluated. A neural network model of dynamic stability of the cutting process is proposed, which enables to optimize the machining process at the stage of work preparation. On the basis of nonlinear dynamics approaches, the attractors of the dynamic cutting system are reconstructed, and their fractal dimensions are determined. Optimal characteristics of the equipment are selected by input parameters and debugging of the planned process based on digital twins.Research Results. Using machine learning methods allowed us to create and explore neural network models of technological systems for cutting, and the software for their implementation. The possibility of applying decision trees for the problem of diagnosing and classifying malfunctions of CNC machines is shown.Discussion and Conclusions. In real production, the technology of digital twins enables to optimize processing conditions considering the technical and dynamic state of CNC machines. This provides a highly accurate assessment of the production capacity of the enterprise under the development of the production program. In addition, equipment failures can be identified in real time on the basis of the intelligent analysis of the distributed sensor system data.

Sign in / Sign up

Export Citation Format

Share Document