scholarly journals A primer on high-dimensional data analysis workflows for studying visual cortex development and plasticity

2019 ◽  
Author(s):  
Justin L. Balsor ◽  
David G. Jones ◽  
Kathryn M. Murphy
Keyword(s):  
Big Data ◽  
Visual Cortex ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
R Package ◽  
High Dimensional ◽  
Data Sets ◽  
Data Set ◽  
Dimensional Changes ◽  
Or Genes

AbstractNew techniques for quantifying large numbers of proteins or genes are transforming the study of plasticity mechanisms in visual cortex (V1) into the era of big data. With those changes comes the challenge of applying new analytical methods designed for high-dimensional data. Studies of V1, however, can take advantage of the known functions that many proteins have in regulating experience-dependent plasticity to facilitate linking big data analyses with neurobiological functions. Here we discuss two workflows and provide example R code for analyzing high-dimensional changes in a group of proteins (or genes) using two data sets. The first data set includes 7 neural proteins, 9 visual conditions, and 3 regions in V1 from an animal model for amblyopia. The second data set includes 23 neural proteins and 31 ages (20d-80yrs) from human post-mortem samples of V1. Each data set presents different challenges and we describe using PCA, tSNE, and various clustering algorithms including sparse high-dimensional clustering. Also, we describe a new approach for identifying high-dimensional features and using them to construct a plasticity phenotype that identifies neurobiological differences among clusters. We include an R package “v1hdexplorer” that aggregates the various coding packages and custom visualization scripts written in R Studio.

scholarly journals Constructing plasticity phenotypes to classify experience-dependent development of the visual cortex

2020 ◽  
Author(s):  
Justin L. Balsor ◽  
David G. Jones ◽  
Kathryn M. Murphy
Keyword(s):  
Visual Cortex ◽  
Neural Development ◽  
Large Data ◽  
R Package ◽  
Synaptic Proteins ◽  
High Dimensional ◽  
Data Sets ◽  
Visual Plasticity ◽  
Dependent Plasticity ◽  
Dimensional Changes

AbstractMany neural mechanisms regulate experience-dependent plasticity in the visual cortex (V1) and new techniques for quantifying large numbers of proteins or genes are transforming how plasticity is studied into the era of big data. With those large data sets comes the challenge of extracting biologically meaningful results about visual plasticity from data-driven analytical methods designed for high-dimensional data. In other areas of neuroscience, high-information content methodologies are revealing more subtle aspects of neural development and individual variations that give rise to a richer picture of brain disorders. We have developed an approach for studying V1 plasticity that takes advantage of the known functions of many synaptic proteins for regulating visual plasticity and using that to rebrand the results of high-dimensional analyses into a plasticity phenotype. Here we provide a primer for analyzing experience-dependent plasticity in V1 using example R code to identify high-dimensional changes in a group of proteins. We describe using PCA to classify high-dimensional plasticity features and use them to construct a plasticity phenotype. In the examples, we show how the plasticity phenotype can be visualized and used to identify neurobiological features in V1 that change during development or after different visual rearing conditions. We include an R package “v1hdexplorer” that aggregates the various coding packages and custom visualization scripts written in R Studio.

scholarly journals Subspace Clustering of High-Dimensional Data: An Evolutionary Approach

2013 ◽  
Vol 2013 ◽  
pp. 1-12 ◽  
Author(s):  
Singh Vijendra ◽  
Sahoo Laxman
Keyword(s):  
Clustering Algorithm ◽  
Dimensional Space ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
Subspace Clustering ◽  
High Dimensional ◽  
Data Sets ◽  
Real World Data ◽  
Data Set ◽  
Data Points

Clustering high-dimensional data has been a major challenge due to the inherent sparsity of the points. Most existing clustering algorithms become substantially inefficient if the required similarity measure is computed between data points in the full-dimensional space. In this paper, we have presented a robust multi objective subspace clustering (MOSCL) algorithm for the challenging problem of high-dimensional clustering. The first phase of MOSCL performs subspace relevance analysis by detecting dense and sparse regions with their locations in data set. After detection of dense regions it eliminates outliers. MOSCL discovers subspaces in dense regions of data set and produces subspace clusters. In thorough experiments on synthetic and real-world data sets, we demonstrate that MOSCL for subspace clustering is superior to PROCLUS clustering algorithm. Additionally we investigate the effects of first phase for detecting dense regions on the results of subspace clustering. Our results indicate that removing outliers improves the accuracy of subspace clustering. The clustering results are validated by clustering error (CE) distance on various data sets. MOSCL can discover the clusters in all subspaces with high quality, and the efficiency of MOSCL outperforms PROCLUS.

scholarly journals Data segmentation based on the local intrinsic dimension

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Michele Allegra ◽  
Elena Facco ◽  
Francesco Denti ◽  
Alessandro Laio ◽  
Antonietta Mira
Keyword(s):  
High Dimensional Data ◽  
Large Data ◽  
Large Data Sets ◽  
High Dimensional ◽  
Data Sets ◽  
Imaging Data ◽  
Unsupervised Segmentation ◽  
Real World Data ◽  
Data Set ◽  
Intrinsic Dimension

Abstract One of the founding paradigms of machine learning is that a small number of variables is often sufficient to describe high-dimensional data. The minimum number of variables required is called the intrinsic dimension (ID) of the data. Contrary to common intuition, there are cases where the ID varies within the same data set. This fact has been highlighted in technical discussions, but seldom exploited to analyze large data sets and obtain insight into their structure. Here we develop a robust approach to discriminate regions with different local IDs and segment the points accordingly. Our approach is computationally efficient and can be proficiently used even on large data sets. We find that many real-world data sets contain regions with widely heterogeneous dimensions. These regions host points differing in core properties: folded versus unfolded configurations in a protein molecular dynamics trajectory, active versus non-active regions in brain imaging data, and firms with different financial risk in company balance sheets. A simple topological feature, the local ID, is thus sufficient to achieve an unsupervised segmentation of high-dimensional data, complementary to the one given by clustering algorithms.

Big Data in Cloud Computing

2021 ◽  
pp. 77-84
Author(s):  
Jayashree K. ◽  
Swaminathan B.
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Social Network ◽  
High Dimensional Data ◽  
High Dimensional ◽  
Data Sets ◽  
It Services ◽  
Delivery Model ◽  
Research Challenges ◽  
Future Direction

The huge size of data that has been produced by applications that spans from social network to scientific computing is termed big data. Cloud computing as a delivery model for IT services enhances business productivity by reducing cost. It has the intention of achieving solution for managing big data such as high dimensional data sets. Thus, this chapter discusses the background of big data and cloud computing. It also discusses the various application of big data in detail. The various related work, research challenges of big data in cloud computing, and the future direction are addressed in this chapter.

scholarly journals Visualization of Very Large High-Dimensional Data Sets as Minimum Spanning Trees

2019 ◽  
Author(s):  
Daniel Probst ◽  
Jean-Louis Reymond
Keyword(s):  
Data Visualization ◽  
Particle Physics ◽  
Cancer Biology ◽  
Spanning Trees ◽  
Minimum Spanning Tree ◽  
High Dimensional Data ◽  
Locality Sensitive Hashing ◽  
High Dimensional ◽  
Data Sets ◽  
Data Set

<div>Here, we introduce a new data visualization and exploration method, TMAP (tree-map), which exploits locality sensitive hashing, Kruskal’s minimum-spanning-tree algorithm, and a multilevel multipole-based graph layout algorithm to represent large and high dimensional data sets as a tree structure, which is readily understandable and explorable. Compared to other data visualization methods such as t-SNE or UMAP, TMAP increases the size of data sets that can be visualized due to its significantly lower memory requirements and running time and should find broad applicability in the age of big data. We exemplify TMAP in the area of cheminformatics with interactive maps for 1.16 million drug-like molecules from ChEMBL, 10.1 million small molecule fragments from FDB17, and 131 thousand 3D-structures of biomolecules from the PDB Databank, and to visualize data from literature (GUTENBERG data set), cancer biology (PANSCAN data set) and particle physics (MiniBooNE data set). TMAP is available as a Python package. Installation, usage instructions and application examples can be found at http://tmap.gdb.tools.</div>

Urban green economic development indicators based on spatial clustering algorithm and blockchain

2020 ◽  
pp. 1-12
Author(s):  
Xiaoguang Gao
Keyword(s):  
Development Strategy ◽  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
Large Data ◽  
Experimental Comparison ◽  
High Dimensional ◽  
Density Peak ◽  
Data Set

The unbalanced development strategy makes the regional development unbalanced. Therefore, in the development process, resources must be effectively utilized according to the level and characteristics of each region. Considering the resource and environmental constraints, this paper measures and analyzes China’s green economic efficiency and green total factor productivity. Moreover, by expounding the characteristics of high-dimensional data, this paper points out the problems of traditional clustering algorithms in high-dimensional data clustering. This paper proposes a density peak clustering algorithm based on sampling and residual squares, which is suitable for high-dimensional large data sets. The algorithm finds abnormal points and boundary points by identifying halo points, and finally determines clusters. In addition, from the experimental comparison on the data set, it can be seen that the improved algorithm is better than the DPC algorithm in both time complexity and clustering results. Finally, this article analyzes data based on actual cases. The research results show that the method proposed in this paper is effective.

scholarly journals Cluster Weighted Model Based on TSNE Algorithm for High-Dimensional Data

2021 ◽  
Author(s):  
Kehinde Olobatuyi
Keyword(s):  
Mixture Models ◽  
Dimensional Space ◽  
High Dimensional Data ◽  
Expectation Maximization Algorithm ◽  
Real Data ◽  
R Package ◽  
High Dimensional ◽  
Data Sets ◽  
Dimensionality Reduction Technique ◽  
Weighted Model

Abstract Similar to many Machine Learning models, both accuracy and speed of the Cluster weighted models (CWMs) can be hampered by high-dimensional data, leading to previous works on a parsimonious technique to reduce the effect of ”Curse of dimensionality” on mixture models. In this work, we review the background study of the cluster weighted models (CWMs). We further show that parsimonious technique is not sufficient for mixture models to thrive in the presence of huge high-dimensional data. We discuss a heuristic for detecting the hidden components by choosing the initial values of location parameters using the default values in the ”FlexCWM” R package. We introduce a dimensionality reduction technique called T-distributed stochastic neighbor embedding (TSNE) to enhance the parsimonious CWMs in high-dimensional space. Originally, CWMs are suited for regression but for classification purposes, all multi-class variables are transformed logarithmically with some noise. The parameters of the model are obtained via expectation maximization algorithm. The effectiveness of the discussed technique is demonstrated using real data sets from different fields.

scholarly journals Assessing the shared variation among high-dimensional data matrices: a modified version of the Procrustean correlation coefficient

2019 ◽  
Author(s):  
E. Coissac ◽  
C. Gonindard-Melodelima
Keyword(s):  
Correlation Coefficient ◽  
Partial Correlation ◽  
High Dimensional Data ◽  
Correlation Coefficients ◽  
R Package ◽  
High Dimensional ◽  
Data Sets ◽  
High Dimension Data ◽  
Correct Estimation ◽  
Shared Variation

AbstractMotivationMolecular biology and ecology studies can produce high dimension data. Estimating correlations and shared variation between such data sets are an important step in disentangling the relationships between different elements of a biological system. Unfortunately, classical approaches are susceptible to producing falsely inferred correlations.ResultsHere we propose a corrected version of the Procrustean correlation coefficient that is robust to high dimensional data. This allows for a correct estimation of the shared variation between two data sets and the partial correlation coefficients between a set of matrix data.AvailabilityThe proposed corrected coefficients are implemented in the ProcMod R package available on CRAN. The git repository is hosted at https://git.metabarcoding.org/lecasofts/[email protected]

scholarly journals A new variant of radial visualization for supervised visualization of high dimensional data

2019 ◽  
Vol 70 (3) ◽  
pp. 162-172
Author(s):  
Long Tran Van ◽  
Nguyen Dinh Thi
Keyword(s):  
High Dimensional Data ◽  
Visual Space ◽  
High Dimensional ◽  
Data Sets ◽  
Data Set ◽  
New Variant ◽  
Radial Visualization ◽  
Linear Dimensionality Reduction ◽  
High Dimensional Datasets ◽  
Dimensionality Reduction Method

Radial Visualization technique is a non linear dimensionality reduction method. Radial Visualization projects multivariate data in the 2-dimensional visual space inside the unit circle. Radial Visualization supports display both the samples and the attributes that provides useful information of data structures. In this article, we introduced a new variant of Radial Visualization for visualizing high dimensional data set that named Arc Radial Visualization. The new proposal that modified Radial Visualization supported more space to display high dimensional datasets. Our method provides an improvement in visualizing cluster structures of high dimensional data sets on the Radial Visualization. We present our proposal method with two quality measurements and proved the effectiveness of our approach for several real datasets.

A SOM PROJECTION TECHNIQUE WITH THE GROWING STRUCTURE FOR VISUALIZING HIGH-DIMENSIONAL DATA

2003 ◽  
Vol 13 (05) ◽  
pp. 353-365 ◽  
Author(s):  
ZHENG WU ◽  
GARY G. YEN
Keyword(s):  
Projection Method ◽  
Cell Structure ◽  
High Dimensional Data ◽  
Network Size ◽  
High Dimensional ◽  
Data Sets ◽  
Self Organizing Map ◽  
Data Set ◽  
Growing Cell ◽  
Self Organizing

The Self-Organizing Map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, an intuitive and effective SOM projection method is proposed for mapping high-dimensional data onto the two-dimensional grid structure with a growing self-organizing mechanism. In the learning phase, a growing SOM is trained and the growing cell structure is used as the baseline framework. In the ordination phase, the new projection method is used to map the input vector so that the input data is mapped to the structure of the SOM without having to plot the weight values, resulting in easy visualization of the data. The projection method is demonstrated on four different data sets, including a 118 patent data set and a 399 checical abstract data set related to polymer cements, with promising results and a significantly reduced network size.

Sign in / Sign up

Export Citation Format

Share Document