scholarly journals A Spline Kernel-Based Approach for Nonlinear System Identification with Dimensionality Reduction

Electronics ◽  
2020 ◽  
Vol 9 (6) ◽  
pp. 940
Author(s):  
Wanxin Zhang ◽  
Jihong Zhu
Keyword(s):  
Nonlinear Systems ◽  
Dimensionality Reduction ◽  
Nonlinear System Identification ◽  
Computational Cost ◽  
Feature Space ◽  
Training Data ◽  
Accurate Identification ◽  
Novel Approach ◽  
New States ◽  
Identification Approach

This paper proposes a novel approach for identification of nonlinear systems. By transforming the data space into a feature space, kernel methods can be used for modeling nonlinear systems. The spline kernel is adopted to produce a Hilbert space. However, a problem exists as the spline kernel-based identification method cannot deal with data with high dimensions well, resulting in huge computational cost and slow estimation speed. Additionally, owing to the large number of parameters to be estimated, the amount of training data required for accurate identification must be large enough to satisfy the persistence of excitation conditions. To solve the problem, a dimensionality reduction strategy is proposed. Transformation of coordinates is made with the tool of differential geometry. The purpose of the transformation is that no intersection of information with relevance to the output will exist between different new states, while the states with no impact on the output are extracted, which are then abandoned when constructing the model. Then, the dimension of the kernel-based model is reduced, and the number of parameters to be estimated is also reduced. Finally, the proposed identification approach was validated by simulations performed on experimental data from wind tunnel tests. The identification result turns out to be accurate and effective with lower dimensions.

scholarly journals Nearest labelset using double distances for multi-label classification

2019 ◽  
Vol 5 ◽  
pp. e242
Author(s):  
Hyukjun Gweon ◽  
Matthias Schonlau ◽  
Stefan H. Steiner
Keyword(s):  
Maximum Likelihood ◽  
Supervised Learning ◽  
Feature Space ◽  
Training Data ◽  
Model Parameters ◽  
Data Sets ◽  
Weighted Sum ◽  
Novel Approach ◽  
Binomial Regression ◽  
F Measure

Multi-label classification is a type of supervised learning where an instance may belong to multiple labels simultaneously. Predicting each label independently has been criticized for not exploiting any correlation between labels. In this article we propose a novel approach, Nearest Labelset using Double Distances (NLDD), that predicts the labelset observed in the training data that minimizes a weighted sum of the distances in both the feature space and the label space to the new instance. The weights specify the relative tradeoff between the two distances. The weights are estimated from a binomial regression of the number of misclassified labels as a function of the two distances. Model parameters are estimated by maximum likelihood. NLDD only considers labelsets observed in the training data, thus implicitly taking into account label dependencies. Experiments on benchmark multi-label data sets show that the proposed method on average outperforms other well-known approaches in terms of 0/1 loss, and multi-label accuracy and ranks second on the F-measure (after a method called ECC) and on Hamming loss (after a method called RF-PCT).

scholarly journals Image Forgery Detection Based on Deep Transfer Learning

2019 ◽  
Vol 3 (5) ◽  
Author(s):  
Younis Abdalla ◽  
M. T. Iqbal ◽  
M. Shehata
Keyword(s):  
Transfer Learning ◽  
Feature Space ◽  
Training Image ◽  
Training Data ◽  
Space Distribution ◽  
Forgery Detection ◽  
Image Forgery ◽  
Digital Revolution ◽  
Image Forgery Detection ◽  
Novel Approach

The recent digital revolution has sparked a growing interest in applying convolutional neural networks (CNNs) and deep learning to the field of image forensics. The proposed methods aim to train algorithms for solving a range of predetermined tasks. However, training a model that has been randomly initialized requires extensive time for computation as well as an enormous pool of training data to draw from. Moreover, such a model needs to be developed and redeveloped from the ground up if there are any alterations to the feature-space distribution. In addressing these problems, the present paper proposes a novel approach to training image forgery detection models. The method applies prior knowledge that has been transferred to the new model from previous steganalysis models. Additionally, because CNN models generally perform badly when transferred to other databases, transfer learning accomplished through knowledge transfer allows the model to be easily trained for other databases. The various models are then evaluated using image forgery techniques such as shearing, rotating, and scaling images. The experimental results, which show an image manipulation detection has validation accuracy of over 94.89%, indicate that the proposed transfer learning approach successfully accelerates CNN model convergence but does not improve image quality.     

scholarly journals Sentiment Analysis based on Soft Clustering through Dimensionality Reduction Technique

2021 ◽  
Vol 40 (3) ◽  
pp. 630-644
Author(s):  
Saba Akmal ◽  
Hafiz Muhammad Shahzad Asif
Keyword(s):  
Dimensionality Reduction ◽  
Sentiment Analysis ◽  
Real World ◽  
Clustering Algorithms ◽  
Computational Cost ◽  
Expectation Maximization Algorithm ◽  
Principal Component ◽  
Training Data ◽  
Machine Learning Techniques ◽  
Soft Clustering

Clustering based sentiment analysis confers new directions to analyze real-world opinions without human participation and pre-tagged training data overhead. Clustering based techniques do not rely on linguistic information and more convenient as compared to other traditional machine learning techniques. Combining the dimensionality reduction techniques with clustering algorithms highly influence the computational cost and improve the performance of sentiment analysis. In this research, we applied Principal Component Analysis technique to reduce the size of features set. This reduced feature set improves binary K-means clustering results of sentiments analysis. In our experiments, we demonstrate the performance of the clustering system with a reduced feature set to provide high-quality sentiment analysis. However, K-mean clustering has its own limitations such as hard assignment and instability of results. To overcome the limitation of traditional K-means algorithm we applied soft clustering (Expectation maximization algorithm) approach which stabilizes clustering accuracy. This approach allows a soft assignment to cluster documents. Consequently, our experimental accuracy is 95% with standard deviation rate of 0.1% which is sufficient to apply the clustering technique in real-world applications.

Accurate and Transferable Multitask Prediction of Chemical Properties with an Atoms-in-Molecule Neural Network

2018 ◽  
Author(s):  
Roman Zubatyuk ◽  
Justin S. Smith ◽  
Jerzy Leszczynski ◽  
Olexandr Isayev
Keyword(s):  
Neural Network ◽  
Molecular System ◽  
Computational Cost ◽  
Chemical Properties ◽  
The State ◽  
Molecular Properties ◽  
Training Data ◽  
Dft Methods ◽  
Benchmark Datasets ◽  
Quantum Phenomena

<p>Atomic and molecular properties could be evaluated from the fundamental Schrodinger’s equation and therefore represent different modalities of the same quantum phenomena. Here we present AIMNet, a modular and chemically inspired deep neural network potential. We used AIMNet with multitarget training to learn multiple modalities of the state of the atom in a molecular system. The resulting model shows on several benchmark datasets the state-of-the-art accuracy, comparable to the results of orders of magnitude more expensive DFT methods. It can simultaneously predict several atomic and molecular properties without an increase in computational cost. With AIMNet we show a new dimension of transferability: the ability to learn new targets utilizing multimodal information from previous training. The model can learn implicit solvation energy (like SMD) utilizing only a fraction of original training data, and archive MAD error of 1.1 kcal/mol compared to experimental solvation free energies in MNSol database.</p>

scholarly journals Siamese Reconstruction Network: Accurate Image Reconstruction from Human Brain Activity by Learning to Compare

2019 ◽  
Vol 9 (22) ◽  
pp. 4749
Author(s):  
Lingyun Jiang ◽  
Kai Qiao ◽  
Linyuan Wang ◽  
Chi Zhang ◽  
Jian Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Human Brain ◽  
Brain Activity ◽  
Feature Space ◽  
Training Data ◽  
Reconstruction Method ◽  
Learning Method ◽  
Training Samples ◽  
Visual Reconstruction ◽  
Relationship Of

Decoding human brain activities, especially reconstructing human visual stimuli via functional magnetic resonance imaging (fMRI), has gained increasing attention in recent years. However, the high dimensionality and small quantity of fMRI data impose restrictions on satisfactory reconstruction, especially for the reconstruction method with deep learning requiring huge amounts of labelled samples. When compared with the deep learning method, humans can recognize a new image because our human visual system is naturally capable of extracting features from any object and comparing them. Inspired by this visual mechanism, we introduced the mechanism of comparison into deep learning method to realize better visual reconstruction by making full use of each sample and the relationship of the sample pair by learning to compare. In this way, we proposed a Siamese reconstruction network (SRN) method. By using the SRN, we improved upon the satisfying results on two fMRI recording datasets, providing 72.5% accuracy on the digit dataset and 44.6% accuracy on the character dataset. Essentially, this manner can increase the training data about from n samples to 2n sample pairs, which takes full advantage of the limited quantity of training samples. The SRN learns to converge sample pairs of the same class or disperse sample pairs of different class in feature space.

scholarly journals Autonomous flight cycles and extreme landings of airliners beyond the current limits and capabilities using artificial neural networks

2021 ◽  
Author(s):  
Haitham Baomar ◽  
Peter J. Bentley
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Weather Conditions ◽  
Extreme Weather ◽  
Training Data ◽  
Aviation Industry ◽  
Control Models ◽  
Novel Approach ◽  
Network Modules ◽  
Artificial Neural

AbstractWe describe the Intelligent Autopilot System (IAS), a fully autonomous autopilot capable of piloting large jets such as airliners by learning from experienced human pilots using Artificial Neural Networks. The IAS is capable of autonomously executing the required piloting tasks and handling the different flight phases to fly an aircraft from one airport to another including takeoff, climb, cruise, navigate, descent, approach, and land in simulation. In addition, the IAS is capable of autonomously landing large jets in the presence of extreme weather conditions including severe crosswind, gust, wind shear, and turbulence. The IAS is a potential solution to the limitations and robustness problems of modern autopilots such as the inability to execute complete flights, the inability to handle extreme weather conditions especially during approach and landing where the aircraft’s speed is relatively low, and the uncertainty factor is high, and the pilots shortage problem compared to the increasing aircraft demand. In this paper, we present the work done by collaborating with the aviation industry to provide training data for the IAS to learn from. The training data is used by Artificial Neural Networks to generate control models automatically. The control models imitate the skills of the human pilot when executing all the piloting tasks required to pilot an aircraft between two airports. In addition, we introduce new ANNs trained to control the aircraft’s elevators, elevators’ trim, throttle, flaps, and new ailerons and rudder ANNs to counter the effects of extreme weather conditions and land safely. Experiments show that small datasets containing single demonstrations are sufficient to train the IAS and achieve excellent performance by using clearly separable and traceable neural network modules which eliminate the black-box problem of large Artificial Intelligence methods such as Deep Learning. In addition, experiments show that the IAS can handle landing in extreme weather conditions beyond the capabilities of modern autopilots and even experienced human pilots. The proposed IAS is a novel approach towards achieving full control autonomy of large jets using ANN models that match the skills and abilities of experienced human pilots and beyond.

MULFE: Multi-Label Learning via Label-Specific Feature Space Ensemble

2021 ◽  
Vol 16 (1) ◽  
pp. 1-24
Author(s):  
Yaojin Lin ◽  
Qinghua Hu ◽  
Jinghua Liu ◽  
Xingquan Zhu ◽  
Xindong Wu
Keyword(s):  
Empirical Studies ◽  
Feature Space ◽  
Training Data ◽  
Data Sets ◽  
Learning Framework ◽  
Feature Spaces ◽  
Public Data ◽  
Margin Distribution ◽  
Label Correlations ◽  
Label Correlation

In multi-label learning, label correlations commonly exist in the data. Such correlation not only provides useful information, but also imposes significant challenges for multi-label learning. Recently, label-specific feature embedding has been proposed to explore label-specific features from the training data, and uses feature highly customized to the multi-label set for learning. While such feature embedding methods have demonstrated good performance, the creation of the feature embedding space is only based on a single label, without considering label correlations in the data. In this article, we propose to combine multiple label-specific feature spaces, using label correlation, for multi-label learning. The proposed algorithm, mu lti- l abel-specific f eature space e nsemble (MULFE), takes consideration label-specific features, label correlation, and weighted ensemble principle to form a learning framework. By conducting clustering analysis on each label’s negative and positive instances, MULFE first creates features customized to each label. After that, MULFE utilizes the label correlation to optimize the margin distribution of the base classifiers which are induced by the related label-specific feature spaces. By combining multiple label-specific features, label correlation based weighting, and ensemble learning, MULFE achieves maximum margin multi-label classification goal through the underlying optimization framework. Empirical studies on 10 public data sets manifest the effectiveness of MULFE.

scholarly journals Experiments of Image Classification Using Dissimilarity Spaces Built with Siamese Networks

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1573
Author(s):  
Loris Nanni ◽  
Giovanni Minchio ◽  
Sheryl Brahnam ◽  
Gianluca Maguolo ◽  
Alessandra Lumini
Keyword(s):  
Vector Space ◽  
Image Classification ◽  
Ad Hoc ◽  
Feature Space ◽  
Medical Data ◽  
Training Data ◽  
Data Sets ◽  
Large Set ◽  
Clustering Methods ◽  
Siamese Networks

Traditionally, classifiers are trained to predict patterns within a feature space. The image classification system presented here trains classifiers to predict patterns within a vector space by combining the dissimilarity spaces generated by a large set of Siamese Neural Networks (SNNs). A set of centroids from the patterns in the training data sets is calculated with supervised k-means clustering. The centroids are used to generate the dissimilarity space via the Siamese networks. The vector space descriptors are extracted by projecting patterns onto the similarity spaces, and SVMs classify an image by its dissimilarity vector. The versatility of the proposed approach in image classification is demonstrated by evaluating the system on different types of images across two domains: two medical data sets and two animal audio data sets with vocalizations represented as images (spectrograms). Results show that the proposed system’s performance competes competitively against the best-performing methods in the literature, obtaining state-of-the-art performance on one of the medical data sets, and does so without ad-hoc optimization of the clustering methods on the tested data sets.

A novel identification approach for shearer running status through integration of rough sets and improved wavelet neural network

2016 ◽  
Vol 230 (16) ◽  
pp. 2792-2805 ◽  
Author(s):  
Lei Si ◽  
Zhongbin Wang ◽  
Xinhua Liu
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Rough Sets ◽  
Particle Swarm Optimization Algorithm ◽  
Wavelet Neural Network ◽  
Decision Table ◽  
Swarm Optimization ◽  
Novel Approach ◽  
Identification Approach ◽  
Simulation Results

In order to accurately and conveniently identify the shearer running status, a novel approach based on the integration of rough sets (RS) and improved wavelet neural network (WNN) was proposed. The decision table of RS was discretized through genetic algorithm and the attribution reduction was realized by MIBARK algorithm to simply the samples of WNN. Furthermore, an improved particle swarm optimization algorithm was proposed to optimize the parameters of WNN and the flowchart of proposed approach was designed. Then, a simulation example was provided and some comparisons with other methods were carried out. The simulation results indicated that the proposed approach was feasible and outperforming others. Finally, an industrial application example of mining automation production was demonstrated to verify the effect of proposed system.

Sign in / Sign up

Export Citation Format

Share Document