scholarly journals AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System

2021 ◽  
Author(s):  
Pengyu Zhao ◽  
Kecheng Xiao ◽  
Yuanxing Zhang ◽  
Kaigui Bian ◽  
Wei Yan
Keyword(s):  
Recommender System ◽  
Random Search ◽  
Search Space ◽  
Behavior Modeling ◽  
Model Complexity ◽  
Domain Experts ◽  
Neural Architecture ◽  
Automatic Behavior ◽  
Three Stages ◽  
The One

Recently, deep learning models have been widely explored in recommender systems. Though having achieved remarkable success, the design of task-aware recommendation models usually requires manual feature engineering and architecture engineering from domain experts. To relieve those efforts, we explore the potential of neural architecture search (NAS) and introduce AMEIR for Automatic behavior Modeling, interaction Exploration and multi-layer perceptron (MLP) Investigation in the Recommender system. Specifically, AMEIR divides the complete recommendation models into three stages of behavior modeling, interaction exploration, MLP aggregation, and introduces a novel search space containing three tailored subspaces that cover most of the existing methods and thus allow for searching better models. To find the ideal architecture efficiently and effectively, AMEIR realizes the one-shot random search in recommendation progressively on the three stages and assembles the search results as the final outcome. The experiment over various scenarios reveals that AMEIR outperforms competitive baselines of elaborate manual design and leading algorithmic complex NAS methods with lower model complexity and comparable time cost, indicating efficacy, efficiency, and robustness of the proposed method.

One-Shot Neural Architecture Search by Dynamically Pruning Supernet in Hierarchical Order

2021 ◽  
pp. 2150029
Author(s):  
Jianwei Zhang ◽  
Dong Li ◽  
Lituan Wang ◽  
Lei Zhang
Keyword(s):  
State Of The Art ◽  
Search Space ◽  
Experimental Results ◽  
Research Interest ◽  
Search Process ◽  
Search Efficiency ◽  
Neural Architecture ◽  
Speed Up ◽  
The One

Neural Architecture Search (NAS), which aims at automatically designing neural architectures, recently draw a growing research interest. Different from conventional NAS methods, in which a large number of neural architectures need to be trained for evaluation, the one-shot NAS methods only have to train one supernet which synthesizes all the possible candidate architectures. As a result, the search efficiency could be significantly improved by sharing the supernet’s weights during the candidate architectures’ evaluation. This strategy could greatly speed up the search process but suffer a challenge that the evaluation based on sharing weights is not predictive enough. Recently, pruning the supernet during the search has been proven to be an efficient way to alleviate this problem. However, the pruning direction in complex-structured search space remains unexplored. In this paper, we revisited the role of path dropout strategy, which drops the neural operations instead of the neurons, in supernet training, and several interesting characters of the supernet trained with dropout are found. Based on the observations, a Hierarchically-Ordered Pruning Neural Architecture Search (HOPNAS) algorithm is proposed by dynamically pruning the supernet with a proper pruning direction. Experimental results indicate that our method is competitive with state-of-the-art approaches on CIFAR10 and ImageNet.

A Comprehensive Survey of Neural Architecture Search

2021 ◽  
Vol 54 (4) ◽  
pp. 1-34
Author(s):  
Pengzhen Ren ◽  
Yun Xiao ◽  
Xiaojun Chang ◽  
Po-yao Huang ◽  
Zhihui Li ◽  
...  
Keyword(s):  
Research Work ◽  
Search Space ◽  
Feature Representation ◽  
Future Research ◽  
Related Research ◽  
Neural Architecture ◽  
Comprehensive Survey ◽  
Future Research Directions ◽  
Intuitive Idea ◽  
New Perspective

Deep learning has made substantial breakthroughs in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers’ prior knowledge and experience. And due to the limitations of humans’ inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search ( NAS ) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. In addition, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.

scholarly journals The Small and Efficient Language Network of Polyglots and Hyper-polyglots

2020 ◽  
Vol 31 (1) ◽  
pp. 62-76
Author(s):  
Olessia Jouravlev ◽  
Zachary Mineroff ◽  
Idan A Blank ◽  
Evelina Fedorenko
Keyword(s):  
Foreign Language ◽  
Language Processing ◽  
Default Mode Network ◽  
Individual Level ◽  
Neural Architecture ◽  
Initial Characterization ◽  
The One ◽  
Multiple Languages ◽  
Language Network

Abstract Acquiring a foreign language is challenging for many adults. Yet certain individuals choose to acquire sometimes dozens of languages and often just for fun. Is there something special about the minds and brains of such polyglots? Using robust individual-level markers of language activity, measured with fMRI, we compared native language processing in polyglots versus matched controls. Polyglots (n = 17, including nine “hyper-polyglots” with proficiency in 10–55 languages) used fewer neural resources to process language: Their activations were smaller in both magnitude and extent. This difference was spatially and functionally selective: The groups were similar in their activation of two other brain networks—the multiple demand network and the default mode network. We hypothesize that the activation reduction in the language network is experientially driven, such that the acquisition and use of multiple languages makes language processing generally more efficient. However, genetic and longitudinal studies will be critical to distinguish this hypothesis from the one whereby polyglots’ brains already differ at birth or early in development. This initial characterization of polyglots’ language network opens the door to future investigations of the cognitive and neural architecture of individuals who gain mastery of multiple languages, including changes in this architecture with linguistic experiences.

Agent-based economic models and econometrics

2012 ◽  
Vol 27 (2) ◽  
pp. 187-219 ◽  
Author(s):  
Shu-Heng Chen ◽  
Chia-Ling Chang ◽  
Ye-Rong Du
Keyword(s):  
Financial Markets ◽  
Agent Based Modeling ◽  
Computational Finance ◽  
Agent Based ◽  
Second Stage ◽  
The Past ◽  
Stage Development ◽  
Three Stages ◽  
The One ◽  
Two Stages

AbstractThis paper reviews the development of agent-based (computational) economics (ACE) from an econometrics viewpoint. The review comprises three stages, characterizing the past, the present, and the future of this development. The first two stages can be interpreted as an attempt to build the econometric foundation of ACE, and, through that, enrich its empirical content. The second stage may then invoke a reverse reflection on the possible agent-based foundation of econometrics. While ACE modeling has been applied to different branches of economics, the one, and probably the only one, which is able to provide evidence of this three-stage development is finance or financial economics. We will, therefore, focus our review only on the literature of agent-based computational finance, or, more specifically, the agent-based modeling of financial markets.

Optimization of the Closure-Weld Region of Cylindrical Containers for Long-Term Corrosion Resistance Using the Successive Heuristic Quadratic Approximation Technique*

2003 ◽  
Vol 125 (3) ◽  
pp. 533-539 ◽  
Author(s):  
Zekai Ceylan ◽  
Mohamed B. Trabia
Keyword(s):  
Finite Element ◽  
Random Search ◽  
Nonlinear Problems ◽  
Search Space ◽  
Induction Coil ◽  
Quadratic Approximation ◽  
Approximation Technique ◽  
General Corrosion ◽  
Highly Nonlinear

Welded cylindrical containers are susceptible to stress corrosion cracking (SCC) in the closure-weld area. An induction coil heating technique may be used to relieve the residual stresses in the closure-weld. This technique involves localized heating of the material by the surrounding coils. The material is then cooled to room temperature by quenching. A two-dimensional axisymmetric finite element model is developed to study the effects of induction coil heating and subsequent quenching. The finite element results are validated through an experimental test. The container design is tuned to maximize the compressive stress from the outer surface to a depth that is equal to the long-term general corrosion rate of the container material multiplied by the desired container lifetime. The problem is subject to several geometrical and stress constraints. Two different solution methods are implemented for this purpose. First, an off-the-shelf optimization software is used. The results however were unsatisfactory because of the highly nonlinear nature of the problem. The paper proposes a novel alternative: the Successive Heuristic Quadratic Approximation (SHQA) technique. This algorithm combines successive quadratic approximation with an adaptive random search within varying search space. SHQA promises to be a suitable search method for computationally intensive, highly nonlinear problems.

Unconditional and conditional as categories of the dialectic of law

2021 ◽  
pp. 58
Author(s):  
Grigory N. Utkin
Keyword(s):  
Theoretical Analysis ◽  
Social Life ◽  
The Other ◽  
Other Hand ◽  
Dialectical Relationship ◽  
Three Stages ◽  
The One ◽  
Dialectical Analysis ◽  
Conceptual Meaning

The article reveals the conceptual, meaning-forming role of the categories of the unconditional and conditional in law. At the same time, their dialectical relationship with each other and with other categories is put in the center of attention. The dialectic of the unconditional and conditional is revealed by achieving the unity of the three stages of theoretical analysis, which allows us to present the unconditional and conditional, on the one hand, as the content of all concepts, through which the idea of law is generally expressed in various aspects and elements; on the other hand, the entire set of categories subject to dialectical analysis appears as elements of the content of the unconditional and conditional as semantic units that Express the universal characteristics of law in its features, isolation from other forms of social life.

Swarm Intelligence for Multi-Objective Optimization in Engineering Design

2018 ◽  
pp. 239-250 ◽  
Author(s):  
Janga Reddy Manne
Keyword(s):  
Swarm Intelligence ◽  
Engineering Design ◽  
Random Search ◽  
Search Space ◽  
Population Based ◽  
Design Problems ◽  
Multi Objective ◽  
Hard Constraints ◽  
Decision Variables ◽  
Engineering Design Problems

Most of the engineering design problems are intrinsically complex and difficult to solve, because of diverse solution search space, complex functions, continuous and discrete nature of decision variables, multiple objectives and hard constraints. Swarm intelligence (SI) algorithms are becoming popular in dealing with these kind of complexities. The SI algorithms being population based random search techniques, use heuristics inspired from nature to enable effective exploration of optimal solutions to complex engineering problems. The SI algorithms derived based on principles of co-operative group intelligence and collective behavior of self-organized systems. This chapter presents key principles of multi-optimization, and swarm optimization for solving multi-objective engineering design problems with illustration through few examples.

Philosophy and Common Sense

Philosophy ◽  
1988 ◽  
Vol 63 (244) ◽  
pp. 161-174 ◽  
Author(s):  
Keith Campbell
Keyword(s):  
Twentieth Century ◽  
Common Sense ◽  
Large Scale ◽  
Natural Philosophy ◽  
The Other ◽  
Jerry Fodor ◽  
Rational Acceptability ◽  
Three Stages ◽  
The One ◽  
The Relationship

This paper raises once more the question of the relationship between philosophy on the one hand and common sense on the other. More particularly, it is concerned with the role which common sense can play in passing judgment on the rational acceptability (or otherwise) of large-scale hypotheses in natural philosophy and the cosmological part of metaphysics. There are, as I see it, three stages through which the relationship has passed in the course of the twentieth century. There is the era of G. E. Moore, the Quine–Feyerabend period, and now a new and modest vindication of common sense is emerging in the work of Jerry Fodor.

Sign in / Sign up

Export Citation Format

Share Document