scholarly journals A Feature Optimization Approach Based on Inter-Class and Intra-Class Distance for Ship Type Classification

Sensors ◽  
2020 ◽  
Vol 20 (18) ◽  
pp. 5429
Author(s):  
Chen Li ◽  
Ziyuan Liu ◽  
Jiawei Ren ◽  
Wenchao Wang ◽  
Ji Xu
Keyword(s):  
Deep Learning ◽  
Objective Function ◽  
State Of The Art ◽  
Optimization Method ◽  
Classification Algorithms ◽  
Optimization Approach ◽  
Time Frequency ◽  
Real Environment ◽  
Feature Optimization ◽  
Type Classification

Deep learning based methods have achieved state-of-the-art results on the task of ship type classification. However, most existing ship type classification algorithms take time–frequency (TF) features as input, the underlying discriminative information of these features has not been explored thoroughly. This paper proposes a novel feature optimization method which is designed to minimize an objective function aimed at increasing inter-class and reducing intra-class feature distance for ship type classification. The objective function we design is able to learn a center for each class and make samples from the same class closer to the corresponding center. This ensures that the features maximize underlying discriminative information involved in the data, particularly for some targets that usually confused by the conventional manual designed feature. Results on the dataset from a real environment show that the proposed feature optimization approach outperforms traditional TF features.

scholarly journals Representing Deep Neural Networks Latent Space Geometries with Graphs

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 39
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Objective Function ◽  
Learning Process ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The Core ◽  
Learning Tasks ◽  
Latent Space

Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.

scholarly journals Comparison of Attention-based Deep LearningModels for EEG Classification

2021 ◽  
Author(s):  
Giulia Cisotto ◽  
Alessio Zanga ◽  
Joanna Chlebus ◽  
Italo Zoppis ◽  
Sara Manzoni ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Classification Performance ◽  
Classification Problems ◽  
Large Variability ◽  
Eeg Classification ◽  
Time Frequency ◽  
Promising Strategy ◽  
Complex Reasoning

Abstract Deep Learning (DL) has recently shown promising classification performance in Electroencephalography (EEG) in many different scenarios. However, the complex reasoning of such models often prevent the user to explain their classification abilities. Attention, one of the most recent and influential ideas in DL, allows the models to learn which portions of the data are relevant to the final classification output. In this work, we compared three attention-enhanced DL models, the brand-new InstaGATs , an LSTM with attention and a CNN with attention. We used these models to classify normal and abnormal, including artifactual and pathological, EEG patterns in three different datasets. We achieved the state of the art in all classification problems, regardless the large variability of the datasets and the simple architecture of the attention-enhanced models. Additionally, we proved that, depending on how the attention mechanism is applied and where the attention layer is located in the model, we can alternatively leverage the information contained in the time, frequency or space domain of the EEG dataset. Therefore, attention represents a promising strategy to evaluate the quality of the EEG information, and its relevance for classification, in different real-world scenarios.

DHIMAN: A novel algorithm for economic Dispatch problem based on optimization metHod usIng Monte Carlo simulation and Astrophysics coNcepts

2019 ◽  
Vol 34 (04) ◽  
pp. 1950032 ◽  
Author(s):  
Gaurav Dhiman ◽  
Pritpal Singh ◽  
Harsimran Kaur ◽  
Ritika Maini
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
State Of The Art ◽  
Real Life ◽  
Optimization Method ◽  
Optimization Approach ◽  
Effective Number ◽  
Number Of Solutions ◽  
Efficient Prediction ◽  
Competing Models

This paper presents a new model using optimization approach for efficient prediction of load in real-life environment. Monte Carlo simulation and Schrödinger equations provide the effective number of solutions. This technique is useful in representation of relationships between different models. The proposed algorithm is verified and validated with various state-of-the-art approaches for solving economic load power dispatch problem to demonstrate its efficiency. Experimental results signify that the proposed algorithm is more precise than existing competing models.

Fuel Optimization Using Biologically-Inspired Computational Models

2008 ◽  
Author(s):  
Thamar E. Mora ◽  
Abu B. Sesay ◽  
Jo¨rg Denzinger ◽  
H. Golshan ◽  
G. Poissant ◽  
...  
Keyword(s):  
Fuel Consumption ◽  
Computational Models ◽  
State Of The Art ◽  
Optimization Method ◽  
Operating Conditions ◽  
Optimization Approach ◽  
Biologically Inspired ◽  
Natural Gas Pipeline ◽  
Constraints Solving ◽  
Fuel Optimization

This paper presents a method for optimizing the fuel consumption of large and complex natural gas pipeline systems. The optimization method uses a biologically-inspired computational model, namely Particle Swarm Systems. The main objective is to identify the set of operating conditions that minimizes the use of fuel in compressor stations while maintaining the desired throughput and satisfying given system constraints. Solving this fuel optimization problem is non-trivial given the large number of decision variables and constraints in large networks, the nature of the fuel function and the minimum response time imposed by the frequent changes in flow nominations. The experimental evaluation tested on various subnetworks of TransCanada show that the proposed optimization approach meets TransCanada’s time requirements and reliably outperforms the interactive method that is the current state-of-the-art by providing solutions for which the fuel consumption is 12% less than state-of-the-art methods.

scholarly journals Remaining Useful Life Estimation Using Neural Ordinary Differential Equations

2021 ◽  
Vol 12 (2) ◽  
Author(s):  
Marco Star ◽  
Kristoffer McKee
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Differential Equations ◽  
Ordinary Differential Equations ◽  
Image Recognition ◽  
State Of The Art ◽  
Recognition Task ◽  
Remaining Useful Life ◽  
Learning Methods ◽  
Time Frequency

Data-driven machinery prognostics has seen increasing popularity recently, especially with the effectiveness of deep learning methods growing. However, deep learning methods lack useful properties such as the lack of uncertainty quantification of their outputs and have a black-box nature. Neural ordinary differential equations (NODEs) use neural networks to define differential equations that propagate data from the inputs to the outputs. They can be seen as a continuous generalization of a popular network architecture used for image recognition known as the Residual Network (ResNet). This paper compares the performance of each network for machinery prognostics tasks to show the validity of Neural ODEs in machinery prognostics. The comparison is done using NASA’s Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset, which simulates the sensor information of degrading turbofan engines. To compare both architectures, they are set up as convolutional neural networks and the sensors are transformed to the time-frequency domain through the short-time Fourier transform (STFT). The spectrograms from the STFT are the input images to the networks and the output is the estimated RUL; hence, the task is turned into an image recognition task. The results found NODEs can compete with state-of-the-art machinery prognostics methods. While it does not beat the state-of-the-art method, it is close enough that it could warrant further research into using NODEs. The potential benefits of using NODEs instead of other network architectures are also discussed in this work.

scholarly journals Two-level optimization approach with accelerated proximal gradient for objective measures in sparse speech reconstruction

2021 ◽  
Vol 0 (0) ◽  
pp. 0
Author(s):  
Hai Huyen Dam ◽  
Siow Yong Low ◽  
Sven Nordholm
Keyword(s):  
Speech Enhancement ◽  
Quality Measures ◽  
Optimization Method ◽  
Optimization Approach ◽  
Reconstruction Process ◽  
Time Frequency ◽  
Score Improvement ◽  
Frequency Representation ◽  
Objective Quality ◽  
Speech Reconstruction

<p style='text-indent:20px;'>Compressive speech enhancement makes use of the sparseness of speech and the non-sparseness of noise in time-frequency representation to perform speech enhancement. However, reconstructing the sparsest output may not necessarily translate to a good enhanced speech signal as speech distortion may be at risk. This paper proposes a two level optimization approach to incorporate objective quality measures in compressive speech enhancement. The proposed method combines the accelerated proximal gradient approach and a global one dimensional optimization method to solve the sparse reconstruction. By incorporating objective quality measures in the optimization process, the reconstructed output is not only sparse but also maintains the highest objective quality score possible. In other words, the sparse speech reconstruction process is now quality sparse speech reconstruction. Experimental results in a compressive speech enhancement consistently show score improvement in objectives measures in different noisy environments compared to the non-optimized method. Additionally, the proposed optimization yields a higher convergence rate with a lower computational complexity compared to the existing methods.</p>

scholarly journals Experienced Optimization with Reusable Directional Model for Hyper-Parameter Search

2018 ◽  
Author(s):  
Yi-Qi Hu ◽  
Yang Yu ◽  
Zhi-Hua Zhou
Keyword(s):  
State Of The Art ◽  
Optimization Method ◽  
Parameter Selection ◽  
Optimization Approach ◽  
Network Architectures ◽  
Directional Model ◽  
Derivative Free Optimization ◽  
Derivative Free ◽  
Parameter Search ◽  
Difficult Issue

Hyper-parameter selection is a crucial yet difficult issue in machine learning. For this problem, derivative-free optimization has being playing an irreplaceable role. However, derivative-free optimization commonly requires a lot of hyper-parameter samples, while each sample could have a high cost for hyper-parameter selection due to the costly evaluation of a learning model. To tackle this issue, in this paper, we propose an experienced optimization approach, i.e., learning how to optimize better from a set of historical optimization processes. From the historical optimization processes on previous datasets, a directional model is trained to predict the direction of the next good hyper-parameter. The directional model is then reused to guide the optimization in learning new datasets. We implement this mechanism within a state-of-the-art derivative-free optimization method SRacos, and conduct experiments on learning the hyper-parameters of heterogeneous ensembles and neural network architectures. Experimental results verify that the proposed approach can significantly improve the learning accuracy within a limited hyper-parameter sample budget.

scholarly journals Combining compact news representations generated using DistilBERT and topological features to classify fake news

2020 ◽  
Author(s):  
Carlos Abel Córdova Sáenz ◽  
Marcelo Dias ◽  
Karin Becker
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Classification Algorithms ◽  
Fake News ◽  
Topological Features ◽  
Compact Representations ◽  
The Social ◽  
Fact Checking ◽  
Propagation Network

Fake news (FN) have affected people’s lives in unimaginable ways. The automatic classification of FN is a vital tool to prevent their dissemination and support fact-checking. Related work has shown that FN spread faster, deeper, and more broadly than the truth on social media. Besides, deep learning has produced state-of-the-art solutions in this field, mainly based on textual attributes. In this paper, we propose initial experiments to combine compact representations of the textual news properties generated using DistilBERT, with topological metrics extracted from the social propagation network. Using a dataset related to politics and five distinct classification algorithms, our results are encouraging. Regarding the textual attributes, we reached results comparable to state-of-the-art solutions using only the news title and contents, which is useful for FN early detection. The topological attributes were not as effective, but the promising results encourage the investigation of alternative architectures for their combination

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Puhong Duan ◽  
Pedram Ghamisi ◽  
Ferdinand Filip ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Data Science ◽  
State Of The Art ◽  
Science Methods ◽  
Learning Models ◽  
Diverse Range ◽  
Hybrid Machine ◽  
Economics Research

This paper provides a state-of-the-art investigation of advances in data science in emerging economic applications. The analysis was performed on novel data science methods in four individual classes of deep learning models, hybrid deep learning models, hybrid machine learning, and ensemble models. Application domains include a wide and diverse range of economics research from the stock market, marketing, and e-commerce to corporate banking and cryptocurrency. Prisma method, a systematic literature review methodology, was used to ensure the quality of the survey. The findings reveal that the trends follow the advancement of hybrid models, which, based on the accuracy metric, outperform other learning algorithms. It is further expected that the trends will converge toward the advancements of sophisticated hybrid deep learning models.

Sign in / Sign up

Export Citation Format

Share Document