Model reduction based global optimization for large-scale steady state nonlinear systems

Many engineering processes can be accurately modeled using partial differential equations (PDEs), but high dimensionality and non-convexity of the resulting systems pose limitations on their efficient optimization. In this work, a model reduction methodology combining principal component analysis (PCA) and artificial neural networks (ANNs) is employed to construct a reduced surrogate model, which is then utilized by advanced deterministic global optimization algorithms to compute global optimal solutions with theoretical guarantees. However, the optimization framework is still time-consuming due to the high non-convexity of the activation functions inside the reduced ANN structure. To further enhance the capability of our optimization framework, two alternative strategies have been proposed. The first one is a piecewise-affine reformulation while the second one is based on deep rectifier neural networks with ReLU activation function. The performances of the two improved frameworks is demonstrated through two illustrative case studies.

Download Full-text

Pulsar Detection for Wavelets SODA and Regularized Fuzzy Neural Networks Based on Andneuron and Robust Activation Function

International Journal of Artificial Intelligence Tools ◽

10.1142/s0218213019500039 ◽

2019 ◽

Vol 28 (01) ◽

pp. 1950003 ◽

Cited By ~ 10

Author(s):

Paulo Vitor de Campos Souza ◽

Luiz Carlos Bambirra Torres ◽

Augusto Junio Guimarães ◽

Vanessa Souza Araujo

Keyword(s):

Neural Networks ◽

Large Scale ◽

Activation Function ◽

Fuzzy Neural Networks ◽

Activation Functions ◽

Large Scale Problems ◽

Proposed Model ◽

Fuzzy Neural ◽

Learning Machine ◽

Intelligent Models

The use of intelligent models may be slow because of the number of samples involved in the problem. The identification of pulsars (stars that emit Earth-catchable signals) involves collecting thousands of signals by professionals of astronomy and their identification may be hampered by the nature of the problem, which requires many dimensions and samples to be analyzed. This paper proposes the use of hybrid models based on concepts of regularized fuzzy neural networks that use the representativeness of input data to define the groupings that make up the neurons of the initial layers of the model. The andneurons are used to aggregate the neurons of the first layer and can create fuzzy rules. The training uses fast extreme learning machine concepts to generate the weights of neurons that use robust activation functions to perform pattern classification. To solve large-scale problems involving the nature of pulsar detection problems, the model proposes a fast and highly accurate approach to address complex issues. In the execution of the tests with the proposed model, experiments were conducted explanation in two databases of pulsars, and the results prove the viability of the fast and interpretable approach in identifying such involved stars.

Download Full-text

Automatic Construction and Global Optimization of a Multisentiment Lexicon

Computational Intelligence and Neuroscience ◽

10.1155/2016/2093406 ◽

2016 ◽

Vol 2016 ◽

pp. 1-8 ◽

Cited By ~ 5

Author(s):

Xiaoping Yang ◽

Zhongxia Zhang ◽

Zhongqiu Zhang ◽

Yuting Mo ◽

Lianbei Li ◽

...

Keyword(s):

Global Optimization ◽

Large Scale ◽

Language Model ◽

Superior Performance ◽

Optimization Framework ◽

Test Intensity ◽

Sentiment Lexicon ◽

Classification Tasks ◽

Network Language ◽

Accurate Quantification

Manual annotation of sentiment lexicons costs too much labor and time, and it is also difficult to get accurate quantification of emotional intensity. Besides, the excessive emphasis on one specific field has greatly limited the applicability of domain sentiment lexicons (Wang et al., 2010). This paper implements statistical training for large-scale Chinese corpus through neural network language model and proposes an automatic method of constructing a multidimensional sentiment lexicon based on constraints of coordinate offset. In order to distinguish the sentiment polarities of those words which may express either positive or negative meanings in different contexts, we further present a sentiment disambiguation algorithm to increase the flexibility of our lexicon. Lastly, we present a global optimization framework that provides a unified way to combine several human-annotated resources for learning our 10-dimensional sentiment lexicon SentiRuc. Experiments show the superior performance of SentiRuc lexicon in category labeling test, intensity labeling test, and sentiment classification tasks. It is worth mentioning that, in intensity label test, SentiRuc outperforms the second place by 21 percent.

Download Full-text

P-lsGOF: A parallel learning-selection-based global optimization framework

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-200753 ◽

2020 ◽

Vol 39 (5) ◽

pp. 7333-7361

Author(s):

Mingcheng Zuo ◽

Guangming Dai

Keyword(s):

Global Optimization ◽

Differential Evolution ◽

Search Algorithm ◽

Differential Evolution Algorithm ◽

Principal Component ◽

Parallel Learning ◽

Optimization Framework ◽

Engineering Design Problems ◽

Adaptive Differential Evolution ◽

Evolution Algorithm

When optimizing complicated engineering design problems, the search spaces are usually extremely nonlinear, leading to the great difficulty of finding optima. To deal with this challenge, this paper introduces a parallel learning-selection-based global optimization framework (P-lsGOF), which can divide the global search space to numbers of sub-spaces along the variables learned from the principal component analysis. The core search algorithm, named memory-based adaptive differential evolution algorithm (MADE), is parallel implemented in all sub-spaces. MADE is an adaptive differential evolution algorithm with the selective memory supplement and shielding of successful control parameters. The efficiency of MADE on CEC2017 unconstrained problems and CEC2011 real-world problems is illustrated by comparing with recently published state-of-the-art variants of success-history based adaptative differential evolution algorithm with linear population size reduction (L-SHADE) The performance of P-lsGOF on CEC2011 problems shows that the optimized results by individually conducting MADE can be further improved.

Download Full-text

Two-stage based ensemble optimization framework for large-scale global optimization

European Journal of Operational Research ◽

10.1016/j.ejor.2012.12.021 ◽

2013 ◽

Vol 228 (2) ◽

pp. 308-320 ◽

Cited By ~ 27

Author(s):

Yu Wang ◽

Jin Huang ◽

Wei Shan Dong ◽

Jun Chi Yan ◽

Chun Hua Tian ◽

...

Keyword(s):

Global Optimization ◽

Large Scale ◽

Two Stage ◽

Optimization Framework ◽

Ensemble Optimization

Download Full-text

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

Economy of Ukraine ◽

10.15407/economyukr.2020.10.054 ◽

2020 ◽

Vol 2020 (10) ◽

pp. 54-62

Author(s):

Oleksii VASYLIEV ◽

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Statistical Data ◽

Activation Function ◽

Decision Making Process ◽

Neural Network Architecture ◽

Acceptable Accuracy ◽

The Neural Network ◽

Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

Download Full-text