scholarly journals CostNet: A Concise Overpass Spatiotemporal Network for Predictive Learning

2020 ◽  
Vol 9 (4) ◽  
pp. 209
Author(s):  
Fengzhen Sun ◽  
Shaojie Li ◽  
Shaohua Wang ◽  
Qingjun Liu ◽  
Lixin Zhou
Keyword(s):  
State Of The Art ◽  
Vertical Direction ◽  
Bottom Layer ◽  
Spatiotemporal Data ◽  
Short Term ◽  
The Core ◽  
Predictive Learning ◽  
Fast Gradient ◽  
Unit Output ◽  
Cross Connection

Predicting the futures from previous spatiotemporal data remains a challenging topic. There have been many previous works on predictive learning. However, mainstream models suffer from huge memory usage or the gradient vanishing problem. Enlightened by the idea from the resnet, we propose CostNet, a novel recursive neural network (RNN)-based network, which has a horizontal and vertical cross-connection. The core of this network is a concise unit, named Horizon LSTM with a fast gradient transmission channel, which can extract spatial and temporal representations effectively to alleviate the gradient propagation difficulty. In the vertical direction outside of the unit, we add overpass connections from unit output to the bottom layer, which can capture the short-term dynamics to generate precise predictions. Our model achieves better prediction results on moving-mnist and radar datasets than the state-of-the-art models.

scholarly journals PredCNN: Predictive Learning with Cascade Convolutions

2018 ◽  
Author(s):  
Ziru Xu ◽  
Yunbo Wang ◽  
Mingsheng Long ◽  
Jianmin Wang
Keyword(s):  
State Of The Art ◽  
Spatiotemporal Data ◽  
Challenging Problem ◽  
The Core ◽  
Predictive Learning ◽  
Flow Prediction ◽  
Video Frames ◽  
Memory Footprint ◽  
Core Idea ◽  
Video Prediction

Predicting future frames in videos remains an unsolved but challenging problem. Mainstream recurrent models suffer from huge memory usage and computation cost, while convolutional models are unable to effectively capture the temporal dependencies between consecutive video frames. To tackle this problem, we introduce an entirely CNN-based architecture, PredCNN, that models the dependencies between the next frame and the sequential video inputs. Inspired by the core idea of recurrent models that previous states have more transition operations than future states, we design a cascade multiplicative unit (CMU) that provides relatively more operations for previous video frames. This newly proposed unit enables PredCNN to predict future spatiotemporal data without any recurrent chain structures, which eases gradient propagation and enables a fully paralleled optimization. We show that PredCNN outperforms the state-of-the-art recurrent models for video prediction on the standard Moving MNIST dataset and two challenging crowd flow prediction datasets, and achieves a faster training speed and lower memory footprint.

scholarly journals The New SWOT for a Sustainable World

2021 ◽  
Vol 7 (1) ◽  
pp. 18
Author(s):  
Leandro Pereira ◽  
Miguel Pinto ◽  
Renato Lopes da Costa ◽  
Álvaro Dias ◽  
Rui Gonçalves
Keyword(s):  
Open Innovation ◽  
Swot Analysis ◽  
Business Environment ◽  
Short Term ◽  
The Core ◽  
Essential Tool ◽  
Management Positions ◽  
Forward Looking

In today’s complex and changing business environment the concern with sustainability has gained more notoriety. However, companies still do not have a sustainable perspective, but a short-term one, where their values are constantly forgotten and this concept is no longer welcomed. This research demonstrates the need for companies to adapt and to start acting in this direction. Following a set of interviews conducted with professionals with management positions of high responsibility, findings reveal that although sustainability is on the management mind, strategies and tools need to be adapted to be at the core of the organization’s strategic formulation. To support this process, a new SWOT analysis to fit a forward-looking sustainable world is proposed. Furthermore, due to the aggregative nature of the model, it represents an essential tool for an open innovation. “SWOT i” integrates the concern with sustainability as one of its pillars, placing the values and impacts that each decision can have at the center of the strategic formulation, allowing their performance to leverage.

scholarly journals Representing Deep Neural Networks Latent Space Geometries with Graphs

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 39
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Objective Function ◽  
Learning Process ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The Core ◽  
Learning Tasks ◽  
Latent Space

Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.

Two Birds With One Stone: Boosting Both Search and Write Performance for Tree Indices on Persistent Memory

2021 ◽  
Vol 20 (5s) ◽  
pp. 1-25
Author(s):  
Yongping Luo ◽  
Peiquan Jin ◽  
Zhou Zhang ◽  
Junchen Zhang ◽  
Bin Cheng ◽  
...  
Keyword(s):  
State Of The Art ◽  
Bottom Layer ◽  
The State ◽  
Search Performance ◽  
Persistent Memory

The advance of byte-addressable persistent memory (PM) makes it a hot topic to revisit traditional tree indices such as B+-tree and radix tree, and a few new persistent memory-friendly tree indices have been proposed. However, due to the special features of persistent memory compared to DRAM and the limitations of B+-tree-like indices, it is much harder to optimize both search and write performance for tree indices on persistent memory. As a result, most existing indices for persistent memory, e.g., WB-tree, proposed to improve write performance while sacrificing search performance. Aiming to optimize both write and search performance for tree indices on persistent memory, in this paper, we first propose a novel Two-Layer Architecture (TLA) for constructing tree indices on persistent memory. The key idea, of TLA is to organize the index with a search-optimized top layer and a write-optimized bottom layer, letting the top layer optimize search performance and the bottom layer improve write performance. By adopting efficient structures for the two layers, TLA can boost both write and search performance for tree indices on persistent memory. Following the TLA architecture, we present a new index called TLBtree (Two-Layer B+-tree) offering high search and write performance for persistent memory. Moreover, we develop a concurrent TLBtree to support non-blocking read operations in multi-core environment. We evaluate our proposals under a server equipped with real Intel Optane persistent memory. The results show that TLBtree outperforms the state-of-the-art tree indices, including WB-tree, Fast&Fair, and FPTree, in both search and write performance. Also, the concurrent TLBtree can achieve up to 3.7x speedup than its competitors under the multi-core environment.

A Two-Layer Long Short-Term Memory Network for Bottleneck Prediction in Multi-Job Manufacturing Systems

2018 ◽  
Author(s):  
Xingjian Lai ◽  
Huanyi Shui ◽  
Jun Ni
Keyword(s):  
Manufacturing Systems ◽  
Short Term Memory ◽  
Complex Dynamics ◽  
State Of The Art ◽  
Short Term ◽  
Term Memory ◽  
Future Production ◽  
Effective Manner ◽  
Long Short Term Memory ◽  
Factory Floor

Throughput bottlenecks define and constrain the productivity of a production line. Prediction of future bottlenecks provides a great support for decision-making on the factory floor, which can help to foresee and formulate appropriate actions before production to improve the system throughput in a cost-effective manner. Bottleneck prediction remains a challenging task in literature. The difficulty lies in the complex dynamics of manufacturing systems. There are multiple factors collaboratively affecting bottleneck conditions, such as machine performance, machine degradation, line structure, operator skill level, and product release schedules. These factors impact on one another in a nonlinear manner and exhibit long-term temporal dependencies. State-of-the-art research utilizes various assumptions to simplify the modeling by reducing the input dimensionality. As a result, those models cannot accurately reflect complex dynamics of the bottleneck in a manufacturing system. To tackle this problem, this paper will propose a systematic framework to design a two-layer Long Short-Term Memory (LSTM) network tailored to the dynamic bottleneck prediction problem in multi-job manufacturing systems. This neural network based approach takes advantage of historical high dimensional factory floor data to predict system bottlenecks dynamically considering the future production planning inputs. The model is demonstrated with data from an automotive underbody assembly line. The result shows that the proposed method can achieve higher prediction accuracy compared with current state-of-the-art approaches.

scholarly journals Arabic dialect sentiment analysis with ZERO effort. \\ Case study: Algerian dialect

2020 ◽  
Vol 23 (65) ◽  
pp. 124-135
Author(s):  
Imane Guellil ◽  
Marcelo Mendoza ◽  
Faical Azouaou
Keyword(s):  
Sentiment Analysis ◽  
Short Term Memory ◽  
State Of The Art ◽  
Short Term ◽  
Term Memory ◽  
Ongoing Work ◽  
Long Short Term Memory ◽  
Large Corpus ◽  
Unique Condition

This paper presents an analytic study showing that it is entirely possible to analyze the sentiment of an Arabic dialect without constructing any resources. The idea of this work is to use the resources dedicated to a given dialect \textit{X} for analyzing the sentiment of another dialect \textit{Y}. The unique condition is to have \textit{X} and \textit{Y} in the same category of dialects. We apply this idea on Algerian dialect, which is a Maghrebi Arabic dialect that suffers from limited available tools and other handling resources required for automatic sentiment analysis. To do this analysis, we rely on Maghrebi dialect resources and two manually annotated sentiment corpus for respectively Tunisian and Moroccan dialect. We also use a large corpus for Maghrebi dialect. We use a state-of-the-art system and propose a new deep learning architecture for automatically classify the sentiment of Arabic dialect (Algerian dialect). Experimental results show that F1-score is up to 83% and it is achieved by Multilayer Perceptron (MLP) with Tunisian corpus and with Long short-term memory (LSTM) with the combination of Tunisian and Moroccan. An improvement of 15% compared to its closest competitor was observed through this study. Ongoing work is aimed at manually constructing an annotated sentiment corpus for Algerian dialect and comparing the results

scholarly journals Optimization Learning: Perspective, Method, and Applications

2020 ◽  
Author(s):  
Risheng Liu
Keyword(s):  
Inverse Problems ◽  
Theoretical Analysis ◽  
Iterative Methods ◽  
State Of The Art ◽  
Learning Paradigm ◽  
Rigorous Analysis ◽  
The Core ◽  
Globally Convergent ◽  
Ill Posed ◽  
Art Performance

Numerous tasks at the core of statistics, learning, and vision areas are specific cases of ill-posed inverse problems. Recently, learning-based (e.g., deep) iterative methods have been empirically shown to be useful for these problems. Nevertheless, integrating learnable structures into iterations is still a laborious process, which can only be guided by intuitions or empirical insights. Moreover, there is a lack of rigorous analysis of the convergence behaviors of these reimplemented iterations, and thus the significance of such methods is a little bit vague. We move beyond these limits and propose a theoretically guaranteed optimization learning paradigm, a generic and provable paradigm for nonconvex inverse problems, and develop a series of convergent deep models. Our theoretical analysis reveals that the proposed optimization learning paradigm allows us to generate globally convergent trajectories for learning-based iterative methods. Thanks to the superiority of our framework, we achieve state-of-the-art performance on different real applications.

scholarly journals Asymmetric Distribution Measure for Few-shot Learning

2020 ◽  
Author(s):  
Wenbin Li ◽  
Lei Wang ◽  
Jing Huo ◽  
Yinghuan Shi ◽  
Yang Gao ◽  
...  
Keyword(s):  
State Of The Art ◽  
Asymmetric Distribution ◽  
Query Image ◽  
Local Descriptor ◽  
Innovative Design ◽  
Feature Representations ◽  
The Core ◽  
Measure Function ◽  
Asymmetric Relation ◽  
Core Idea

The core idea of metric-based few-shot image classification is to directly measure the relations between query images and support classes to learn transferable feature embeddings. Previous work mainly focuses on image-level feature representations, which actually cannot effectively estimate a class's distribution due to the scarcity of samples. Some recent work shows that local descriptor based representations can achieve richer representations than image-level based representations. However, such works are still based on a less effective instance-level metric, especially a symmetric metric, to measure the relation between a query image and a support class. Given the natural asymmetric relation between a query image and a support class, we argue that an asymmetric measure is more suitable for metric-based few-shot learning. To that end, we propose a novel Asymmetric Distribution Measure (ADM) network for few-shot learning by calculating a joint local and global asymmetric measure between two multivariate local distributions of a query and a class. Moreover, a task-aware Contrastive Measure Strategy (CMS) is proposed to further enhance the measure function. On popular miniImageNet and tieredImageNet, ADM can achieve the state-of-the-art results, validating our innovative design of asymmetric distribution measures for few-shot learning. The source code can be downloaded from https://github.com/WenbinLee/ADM.git.

scholarly journals Learning Long- and Short-Term User Literal-Preference with Multimodal Hierarchical Transformer Network for Personalized Image Caption

2020 ◽  
Vol 34 (05) ◽  
pp. 9571-9578 ◽  
Author(s):  
Wei Zhang ◽  
Yue Ying ◽  
Pan Lu ◽  
Hongyuan Zha
Keyword(s):  
State Of The Art ◽  
Natural Extension ◽  
Target Image ◽  
Short Term ◽  
Image Representations ◽  
High Level ◽  
Image Descriptions ◽  
Shed Light ◽  
Image Caption

Personalized image caption, a natural extension of the standard image caption task, requires to generate brief image descriptions tailored for users' writing style and traits, and is more practical to meet users' real demands. Only a few recent studies shed light on this crucial task and learn static user representations to capture their long-term literal-preference. However, it is insufficient to achieve satisfactory performance due to the intrinsic existence of not only long-term user literal-preference, but also short-term literal-preference which is associated with users' recent states. To bridge this gap, we develop a novel multimodal hierarchical transformer network (MHTN) for personalized image caption in this paper. It learns short-term user literal-preference based on users' recent captions through a short-term user encoder at the low level. And at the high level, the multimodal encoder integrates target image representations with short-term literal-preference, as well as long-term literal-preference learned from user IDs. These two encoders enjoy the advantages of the powerful transformer networks. Extensive experiments on two real datasets show the effectiveness of considering two types of user literal-preference simultaneously and better performance over the state-of-the-art models.

Sign in / Sign up

Export Citation Format

Share Document