scholarly journals Neural Architecture Search for a Highly Efficient Network with Random Skip Connections

2020 ◽  
Vol 10 (11) ◽  
pp. 3712
Author(s):  
Dongjing Shan ◽  
Xiongwei Zhang ◽  
Wenhua Shi ◽  
Li Li
Keyword(s):  
State Of The Art ◽  
Cell Structure ◽  
Frequency Discrimination ◽  
Search Space ◽  
Short Term ◽  
Cell Parameters ◽  
Neural Architecture ◽  
Proposed Model ◽  
Initialization Scheme

Regarding the sequence learning of neural networks, there exists a problem of how to capture long-term dependencies and alleviate the gradient vanishing phenomenon. To manage this problem, we proposed a neural network with random connections via a scheme of a neural architecture search. First, a dense network was designed and trained to construct a search space, and then another network was generated by random sampling in the space, whose skip connections could transmit information directly over multiple periods and capture long-term dependencies more efficiently. Moreover, we devised a novel cell structure that required less memory and computational power than the structures of long short-term memories (LSTMs), and finally, we performed a special initialization scheme on the cell parameters, which could permit unhindered gradient propagation on the time axis at the beginning of training. In the experiments, we evaluated four sequential tasks: adding, copying, frequency discrimination, and image classification; we also adopted several state-of-the-art methods for comparison. The experimental results demonstrated that our proposed model achieved the best performance.

scholarly journals Where to Go Next: Modeling Long- and Short-Term User Preferences for Point-of-Interest Recommendation

2020 ◽  
Vol 34 (01) ◽  
pp. 214-221 ◽  
Author(s):  
Ke Sun ◽  
Tieyun Qian ◽  
Tong Chen ◽  
Yile Liang ◽  
Quoc Viet Hung Nguyen ◽  
...  
Keyword(s):  
State Of The Art ◽  
User Preferences ◽  
Short Term ◽  
Preference Modeling ◽  
Point Of Interest ◽  
Proposed Model ◽  
Poi Recommendation ◽  
Novel Method ◽  
Real World Datasets

Point-of-Interest (POI) recommendation has been a trending research topic as it generates personalized suggestions on facilities for users from a large number of candidate venues. Since users' check-in records can be viewed as a long sequence, methods based on recurrent neural networks (RNNs) have recently shown promising applicability for this task. However, existing RNN-based methods either neglect users' long-term preferences or overlook the geographical relations among recently visited POIs when modeling users' short-term preferences, thus making the recommendation results unreliable. To address the above limitations, we propose a novel method named Long- and Short-Term Preference Modeling (LSTPM) for next-POI recommendation. In particular, the proposed model consists of a nonlocal network for long-term preference modeling and a geo-dilated RNN for short-term preference learning. Extensive experiments on two real-world datasets demonstrate that our model yields significant improvements over the state-of-the-art methods.

scholarly journals Learning Long- and Short-Term User Literal-Preference with Multimodal Hierarchical Transformer Network for Personalized Image Caption

2020 ◽  
Vol 34 (05) ◽  
pp. 9571-9578 ◽  
Author(s):  
Wei Zhang ◽  
Yue Ying ◽  
Pan Lu ◽  
Hongyuan Zha
Keyword(s):  
State Of The Art ◽  
Natural Extension ◽  
Target Image ◽  
Short Term ◽  
Image Representations ◽  
High Level ◽  
Image Descriptions ◽  
Shed Light ◽  
Image Caption

Personalized image caption, a natural extension of the standard image caption task, requires to generate brief image descriptions tailored for users' writing style and traits, and is more practical to meet users' real demands. Only a few recent studies shed light on this crucial task and learn static user representations to capture their long-term literal-preference. However, it is insufficient to achieve satisfactory performance due to the intrinsic existence of not only long-term user literal-preference, but also short-term literal-preference which is associated with users' recent states. To bridge this gap, we develop a novel multimodal hierarchical transformer network (MHTN) for personalized image caption in this paper. It learns short-term user literal-preference based on users' recent captions through a short-term user encoder at the low level. And at the high level, the multimodal encoder integrates target image representations with short-term literal-preference, as well as long-term literal-preference learned from user IDs. These two encoders enjoy the advantages of the powerful transformer networks. Extensive experiments on two real datasets show the effectiveness of considering two types of user literal-preference simultaneously and better performance over the state-of-the-art models.

scholarly journals Learning from Interventions Using Hierarchical Policies for Safe Learning

2020 ◽  
Vol 34 (06) ◽  
pp. 10352-10360
Author(s):  
Jing Bi ◽  
Vikas Dhiman ◽  
Tianyou Xiao ◽  
Chenliang Xu
Keyword(s):  
Reaction Time ◽  
State Of The Art ◽  
The State ◽  
Policy Framework ◽  
Asymptotic Performance ◽  
Short Term ◽  
Learning From Demonstrations ◽  
Hierarchical Levels ◽  
Long Term Behavior

Learning from Demonstrations (LfD) via Behavior Cloning (BC) works well on multiple complex tasks. However, a limitation of the typical LfD approach is that it requires expert demonstrations for all scenarios, including those in which the algorithm is already well-trained. The recently proposed Learning from Interventions (LfI) overcomes this limitation by using an expert overseer. The expert overseer only intervenes when it suspects that an unsafe action is about to be taken. Although LfI significantly improves over LfD, the state-of-the-art LfI fails to account for delay caused by the expert's reaction time and only learns short-term behavior. We address these limitations by 1) interpolating the expert's interventions back in time, and 2) by splitting the policy into two hierarchical levels, one that generates sub-goals for the future and another that generates actions to reach those desired sub-goals. This sub-goal prediction forces the algorithm to learn long-term behavior while also being robust to the expert's reaction time. Our experiments show that LfI using sub-goals in a hierarchical policy framework trains faster and achieves better asymptotic performance than typical LfD.

scholarly journals Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation

2020 ◽  
Vol 34 (05) ◽  
pp. 7797-7804
Author(s):  
Goran Glavašš ◽  
Swapna Somasundaran
Keyword(s):  
State Of The Art ◽  
Language Transfer ◽  
Text Segmentation ◽  
Word Embeddings ◽  
Neural Architecture ◽  
Text Coherence ◽  
Sentence Level ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Cross Lingual

Breaking down the structure of long texts into semantically coherent segments makes the texts more readable and supports downstream applications like summarization and retrieval. Starting from an apparent link between text coherence and segmentation, we introduce a novel supervised model for text segmentation with simple but explicit coherence modeling. Our model – a neural architecture consisting of two hierarchically connected Transformer networks – is a multi-task learning model that couples the sentence-level segmentation objective with the coherence objective that differentiates correct sequences of sentences from corrupt ones. The proposed model, dubbed Coherence-Aware Text Segmentation (CATS), yields state-of-the-art segmentation performance on a collection of benchmark datasets. Furthermore, by coupling CATS with cross-lingual word embeddings, we demonstrate its effectiveness in zero-shot language transfer: it can successfully segment texts in languages unseen in training.

scholarly journals Long-term target tracking combined with re-detection

2020 ◽  
Author(s):  
Juanjuan Wang ◽  
HaoRan Yang ◽  
Ning Xu ◽  
Chengqin Wu ◽  
ZengShun Zhao ◽  
...  
Keyword(s):  
Correlation Energy ◽  
State Of The Art ◽  
Tracking Algorithm ◽  
Correlation Filters ◽  
Short Term ◽  
Tracking Method ◽  
Tracking Tasks ◽  
Svm Model ◽  
Confidence Degree

Abstract The long-term visual tracking undergoes more challenges and is closer to realistic applications than short-term tracking. However, the performances of most existing methods have been limited in the long-term tracking tasks. In this work, we present a reliable yet simple long-term tracking method, which extends the state-of-the-art Learning Adaptive Discriminative Correlation Filters (LADCF) tracking algorithm with a re-detection component based on the SVM model. The LADCF tracking algorithm localizes the target in each frame and the re-detector is able to efficiently re-detect the target in the whole image when the tracking fails. We further introduce a robust confidence degree evaluation criterion that combines the maximum response criterion and the average peak-to correlation energy (APCE) to judge the confidence level of the predicted target. When the confidence degree is generally high, the SVM is updated accordingly. If the confidence drops sharply, the SVM re-detects the target. We perform extensive experiments on the OTB-2015 and UAV123 datasets. The experimental results demonstrate the effectiveness of our algorithm in long-term tracking.

scholarly journals Single-Cell Tests to Explore the Reliability of Sofc Installations Operating Offshore

Energies ◽  
2020 ◽  
Vol 13 (7) ◽  
pp. 1624
Author(s):  
Nelson Thambiraj ◽  
Ivar Waernhus ◽  
Crina Suciu ◽  
Arild Vik ◽  
Alex C. Hoffmann
Keyword(s):  
Cell Structure ◽  
Oxygen Depletion ◽  
X Ray Diffraction ◽  
Short Term ◽  
Sem Images ◽  
Cathode Degradation ◽  
Air Supply ◽  
Different Temperatures ◽  
Effect Of Oxygen

This paper studies the robustness of off-shore solid oxide fuel cell (SOFC) installations and the nature and causes of possible cell degradation in marine environments. Two important, cathode-related, impediments to ensuring SOFC reliability in off-shore installations are: cathode degradation due to salt contamination and oxygen depletion in the air supply. Short-term and long-term tests show the effect of salt contamination in the cathode feed on cell performance, and reveal the underlying cause of the degradation seen. SEM/X-ray Diffraction/(XRD) analyses made it possible to identify salt taken up in the cathode microstructure after the short-term testing while the macroscopic cell structure remained intact after the short-term tests. The long-term degradation was found to be more severe, and SEM images showed delamination at the cathode/electrolyte interface with salt present, something that was not seen after long-term testing without salt. The effect of oxygen depletion on the performance was also determined at three different temperatures using I-V curves.

scholarly journals An LSTM-Based Method with Attention Mechanism for Travel Time Prediction

Sensors ◽  
2019 ◽  
Vol 19 (4) ◽  
pp. 861 ◽  
Author(s):  
Xiangdong Ran ◽  
Zhiguang Shan ◽  
Yufei Fang ◽  
Chuang Lin
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Traffic Prediction ◽  
Travel Time Prediction ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Departure Time ◽  
Long Short Term Memory

Traffic prediction is based on modeling the complex non-linear spatiotemporal traffic dynamics in road network. In recent years, Long Short-Term Memory has been applied to traffic prediction, achieving better performance. The existing Long Short-Term Memory methods for traffic prediction have two drawbacks: they do not use the departure time through the links for traffic prediction, and the way of modeling long-term dependence in time series is not direct in terms of traffic prediction. Attention mechanism is implemented by constructing a neural network according to its task and has recently demonstrated success in a wide range of tasks. In this paper, we propose an Long Short-Term Memory-based method with attention mechanism for travel time prediction. We present the proposed model in a tree structure. The proposed model substitutes a tree structure with attention mechanism for the unfold way of standard Long Short-Term Memory to construct the depth of Long Short-Term Memory and modeling long-term dependence. The attention mechanism is over the output layer of each Long Short-Term Memory unit. The departure time is used as the aspect of the attention mechanism and the attention mechanism integrates departure time into the proposed model. We use AdaGrad method for training the proposed model. Based on the datasets provided by Highways England, the experimental results show that the proposed model can achieve better accuracy than the Long Short-Term Memory and other baseline methods. The case study suggests that the departure time is effectively employed by using attention mechanism.

scholarly journals Long Short-Term Memory with Dynamic Skip Connections

2019 ◽  
Vol 33 ◽  
pp. 6481-6488 ◽  
Author(s):  
Tao Gui ◽  
Qi Zhang ◽  
Lujun Zhao ◽  
Yaosong Lin ◽  
Minlong Peng ◽  
...  
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Training Data ◽  
Sequential Data ◽  
Short Term ◽  
Term Memory ◽  
Transition Functions ◽  
Proposed Model ◽  
Long Short Term Memory

In recent years, long short-term memory (LSTM) has been successfully used to model sequential data of variable length. However, LSTM can still experience difficulty in capturing long-term dependencies. In this work, we tried to alleviate this problem by introducing a dynamic skip connection, which can learn to directly connect two dependent words. Since there is no dependency information in the training data, we propose a novel reinforcement learning-based method to model the dependency relationship and connect dependent words. The proposed model computes the recurrent transition functions based on the skip connections, which provides a dynamic skipping advantage over RNNs that always tackle entire sentences sequentially. Our experimental results on three natural language processing tasks demonstrate that the proposed method can achieve better performance than existing methods. In the number prediction experiment, the proposed model outperformed LSTM with respect to accuracy by nearly 20%.

scholarly journals Modeling sensitivity indices of industrial enterprise organizational change

2021 ◽  
Vol 10 (45) ◽  
pp. 230-241
Author(s):  
Victoriia Bilyk ◽  
Olena Kolomytseva ◽  
Olha Myshkovych ◽  
Nataliia Tymoshyk ◽  
Denis Shcherbatykh
Keyword(s):  
Organizational Change ◽  
Theory And Practice ◽  
Organizational Changes ◽  
Short Term ◽  
Industrial Enterprises ◽  
Sensitivity Indices ◽  
Proposed Model ◽  
Development Effectiveness ◽  
Made In

Evaluation of sensitivity of commercial enterprises to organizational changes should be made in terms of short-term planning for which it is important to ensure the financial results, as well as in terms of long-term planning, which is important for non-monetary indicators of development effectiveness. To solve this problem, the paper is designed model sensitivity Descriptive indicators of industrial enterprises to organizational changes, reflecting monetary and non-monetary effects of organizational change. The authors determined that the proposed model allows for the analysis of organizational change with regard to their impact on monetary and non-monetary efficiency. This paper contributes to the theory and practice at the border to ensure a balance between short-term and long-term development of industrial enterprises. Convincingly demonstrated the possibility of using research results in practice.

scholarly journals Optimization Model for the Long-Term Operation of an Interprovincial Hydropower Plant Incorporating Peak Shaving Demands

Energies ◽  
2020 ◽  
Vol 13 (18) ◽  
pp. 4804
Author(s):  
Rui Cao ◽  
Jianjian Shen ◽  
Chuntian Cheng ◽  
Jian Wang
Keyword(s):  
Power Generation ◽  
Optimization Model ◽  
Hydropower Plant ◽  
Mixed Integer ◽  
Short Term ◽  
Long Distance ◽  
Peak Shaving ◽  
Term Operation ◽  
Proposed Model

The increasing peak-to-valley load difference in China pose a challenge to long-distance and large-capacity hydropower transmission via high-voltage direct current (HVDC) lines. Considering the peak shaving demands of load centers, an optimization model that maximizes the expected power generation revenue is proposed here for the long-term operation of an interprovincial hydropower plant. A simulation-based method was utilized to explore the relationships between long-term power generation and short-term peak shaving revenue in the model. This method generated representative daily load scenarios via cluster analysis and approximated the real-time electricity price of each load profile with the time-of-use price strategy. A mixed-integer linear programming model with HVDC transmission constraints was then established to obtain moving average (MA) price curves that bridged two time-coupled operations. The MA price curves were finally incorporated into the long-term optimization model to determine monthly generation schedules, and the inflow uncertainty was addressed by discretized inflow scenarios. The proposed model was evaluated based on the operation of the Xiluodu hydropower system in China during the drawdown season. The results revealed a trade-off between long-term energy production and short-term peak shaving revenue, and they demonstrated the revenue potential of interprovincial hydropower transmission while meeting peak shaving demands. A comparison with other long-term optimization methods demonstrated the effectiveness and reliability of the proposed model in maximizing power generation revenue.

Sign in / Sign up

Export Citation Format

Share Document