Analysis of a low-dimensional bottleneck neural network representation of speech for modelling speech dynamics

2015 ◽  
Author(s):  
Linxue Bai ◽  
Peter Jančovič ◽  
Martin Russell ◽  
Philip Weber
Keyword(s):  
Neural Network ◽  
Network Representation ◽  
Bottleneck Neural Network ◽  
Low Dimensional

scholarly journals Identifying Disease Related Genes by Network Representation and Convolutional Neural Network

2021 ◽  
Vol 9 ◽  
Author(s):  
Bolin Chen ◽  
Yourui Han ◽  
Xuequn Shang ◽  
Shenggui Zhang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Biological Networks ◽  
Dimensional Space ◽  
Scale Free ◽  
Network Representation ◽  
Disease Related Genes ◽  
Representation Method ◽  
Low Dimensional

The identification of disease related genes plays essential roles in bioinformatics. To achieve this, many powerful machine learning methods have been proposed from various computational aspects, such as biological network analysis, classification, regression, deep learning, etc. Among them, deep learning based methods have gained big success in identifying disease related genes in terms of higher accuracy and efficiency. However, these methods rarely handle the following two issues very well, which are (1) the multifunctions of many genes; and (2) the scale-free property of biological networks. To overcome these, we propose a novel network representation method to transfer individual vertices together with their surrounding topological structures into image-like datasets. It takes each node-induced sub-network as a represented candidate, and adds its environmental characteristics to generate a low-dimensional space as its representation. This image-like datasets can be applied directly in a Convolutional Neural Network-based method for identifying cancer-related genes. The numerical experiments show that the proposed method can achieve the AUC value at 0.9256 in a single network and at 0.9452 in multiple networks, which outperforms many existing methods.

scholarly journals Network Embedding via a Bi-Mode and Deep Neural Network Model

2017 ◽  
Author(s):  
Yang Fang ◽  
Xiang Zhao ◽  
Zhen Tan
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Semantic Information ◽  
Dimensional Space ◽  
Relation Extraction ◽  
Network Embedding ◽  
Structure Information ◽  
Second Mode ◽  
Real World Datasets ◽  
Low Dimensional

Network Embedding (NE) is an important method to learn the representations of network via a low-dimensional space. Conventional NE models focus on capturing the structure information and semantic information of vertices while neglecting such information for edges. In this work, we propose a novel NE model named BimoNet to capture both the structure and semantic information of edges. BimoNet is composed of two parts, i.e., the bi-mode embedding part and the deep neural network part. For bi-mode embedding part, the first mode named add-mode is used to express the entity-shared features of edges and the second mode named subtract-mode is employed to represent the entity-specific features of edges. These features actually reflect the semantic information. For deep neural network part, we firstly regard the edges in a network as nodes, and the vertices as links, which will not change the overall structure of the whole network. Then we take the nodes' adjacent matrix as the input of the deep neural network as it can obtain similar representations for nodes with similar structure. Afterwards, by jointly optimizing the objective function of these two parts, BimoNet could preserve both the semantic and structure information of edges. In experiments, we evaluate BimoNet on three real-world datasets and task of relation extraction, and BimoNet is demonstrated to outperform state-of-the-art baseline models consistently and significantly.

Convolutional Neural Network Representation of the Total Multiple Scattering Cross Section

2021 ◽  
Author(s):  
Thang Tran ◽  
Peter Lai ◽  
Wei-Ching Wang ◽  
Amaris Rosa ◽  
Feruza Amirkulova ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Multiple Scattering ◽  
Cross Section ◽  
Scattering Cross Section ◽  
Scattering Cross ◽  
Network Representation

scholarly journals Exponential Family Graph Embeddings

2020 ◽  
Vol 34 (04) ◽  
pp. 3357-3364
Author(s):  
Abdulkadir Celikkanat ◽  
Fragkiskos D. Malliaros
Keyword(s):  
Random Walk ◽  
Exponential Family ◽  
Representation Learning ◽  
Learning Problems ◽  
Interaction Patterns ◽  
Network Representation ◽  
Learning Tasks ◽  
Learning Techniques ◽  
Real World Datasets ◽  
Low Dimensional

Representing networks in a low dimensional latent space is a crucial task with many interesting applications in graph learning problems, such as link prediction and node classification. A widely applied network representation learning paradigm is based on the combination of random walks for sampling context nodes and the traditional Skip-Gram model to capture center-context node relationships. In this paper, we emphasize on exponential family distributions to capture rich interaction patterns between nodes in random walk sequences. We introduce the generic exponential family graph embedding model, that generalizes random walk-based network representation learning techniques to exponential family conditional distributions. We study three particular instances of this model, analyzing their properties and showing their relationship to existing unsupervised learning models. Our experimental evaluation on real-world datasets demonstrates that the proposed techniques outperform well-known baseline methods in two downstream machine learning tasks.

scholarly journals The Goldilocks Zone: Towards Better Understanding of Neural Network Loss Landscapes

2019 ◽  
Vol 33 ◽  
pp. 3574-3581
Author(s):  
Stanislav Fort ◽  
Adam Scherlis
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Configuration Space ◽  
Loss Function ◽  
Positive Curvature ◽  
Local Convexity ◽  
Convolutional Networks ◽  
Hollow Spherical Shell ◽  
Low Dimensional ◽  
Fully Connected

We explore the loss landscape of fully-connected and convolutional neural networks using random, low-dimensional hyperplanes and hyperspheres. Evaluating the Hessian, H, of the loss function on these hypersurfaces, we observe 1) an unusual excess of the number of positive eigenvalues of H, and 2) a large value of Tr(H)/||H|| at a well defined range of configuration space radii, corresponding to a thick, hollow, spherical shell we refer to as the Goldilocks zone. We observe this effect for fully-connected neural networks over a range of network widths and depths on MNIST and CIFAR-10 datasets with the ReLU and tanh non-linearities, and a similar effect for convolutional networks. Using our observations, we demonstrate a close connection between the Goldilocks zone, measures of local convexity/prevalence of positive curvature, and the suitability of a network initialization. We show that the high and stable accuracy reached when optimizing on random, low-dimensional hypersurfaces is directly related to the overlap between the hypersurface and the Goldilocks zone, and as a corollary demonstrate that the notion of intrinsic dimension is initialization-dependent. We note that common initialization techniques initialize neural networks in this particular region of unusually high convexity/prevalence of positive curvature, and offer a geometric intuition for their success. Furthermore, we demonstrate that initializing a neural network at a number of points and selecting for high measures of local convexity such as Tr(H)/||H||, number of positive eigenvalues of H, or low initial loss, leads to statistically significantly faster training on MNIST. Based on our observations, we hypothesize that the Goldilocks zone contains an unusually high density of suitable initialization configurations.

Neural network representation and optimization of thermoelectric states of multiple interacting quantum dots

2020 ◽  
Vol 22 (28) ◽  
pp. 16165-16173
Author(s):  
Hangbo Zhou ◽  
Gang Zhang ◽  
Yong-Wei Zhang
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Quantum Dots ◽  
Seebeck Coefficient ◽  
Master Equation ◽  
Thermoelectric Properties ◽  
Electrical Conductance ◽  
Thermal Conductance ◽  
Quantum Master Equation ◽  
Network Representation

We perform quantum master equation calculations and machine learning to investigate the thermoelectric properties of multiple interacting quantum dots, including electrical conductance, Seebeck coefficient, thermal conductance and ZT.

Sign in / Sign up

Export Citation Format

Share Document