Hierarchical Quasi-Fractional Gradient Descent Method for Parameter Estimation of Nonlinear ARX Systems Using Key Term Separation Principle

Recently, a quasi-fractional order gradient descent (QFGD) algorithm was proposed and successfully applied to solve system identification problem. The QFGD suffers from the overparameterization problem and results in estimating the redundant parameters instead of identifying only the actual parameters of the system. This study develops a novel hierarchical QFDS (HQFGD) algorithm by introducing the concepts of hierarchical identification principle and key term separation idea. The proposed HQFGD is effectively applied to solve the parameter estimation problem of input nonlinear autoregressive with exogeneous noise (INARX) system. A detailed investigation about the performance of HQFGD is conducted under different disturbance conditions considering different fractional orders and learning rate variations. The simulation results validate the better performance of the HQFGD over the standard counterpart in terms of estimation accuracy, convergence speed and robustness.

Remote error estimation of smart meter based on clustering and adaptive gradient descent method

At present, the main way for electric power companies to check the accuracy of electric meters is that professionals regularly bring standard electric meters to the site for verification. With the widespread application of smart meters and the development of data processing technology, remote error estimation based on the operating data of smart meters becomes possible. In this paper, an error estimation method of smart meter based on clustering and adaptive gradient descent method is proposed. Firstly, the fuzzy c-means clustering method is used to preprocess the data to classify the operating conditions of each measurement, and then the adaptive gradient descent method is used to establish the error estimation model. The simulation results show that this method has high error estimation accuracy. This method has a small amount of calculation and high reliability and is suitable for large-scale power grids.

News Video Classification Model Based on ResNet-2 and Transfer Learning

A large amount of useful information is included in the news video, and how to classify the news video information has become an important research topic in the field of multimedia technology. News videos are enormously informative, and employing manual classification methods is too time-consuming and vulnerable to subjective judgment. Therefore, developing an automated news video analysis and retrieval method becomes one of the most important research contents in the current multimedia information system. Therefore, this paper proposes a news video classification model based on ResNet-2 and transfer learning. First, a model-based transfer method was adopted to transfer the commonality knowledge of the pretrained model of the Inception-ResNet-v2 network on ImageNet, and a news video classification model was constructed. Then, a momentum update rule is introduced on the basis of the Adam algorithm, and an improved gradient descent method is proposed in order to obtain an optimal solution of the local minima of the function in the learning process. The experimental results show that the improved Adam algorithm can iteratively update the network weights through the adaptive learning rate to reach the fastest convergence. Compared with other convolutional neural network models, the modified Inception-ResNet-v2 network model achieves 91.47% classification accuracy for common news video datasets.

Minimum-monitor-unit optimization via a stochastic coordinate descent method

Objective: Deliverable proton spots are subject to the minimum monitor-unit (MMU) constraint. The MMU optimization problem with relatively large MMU threshold remains mathematically challenging due to its strong nonconvexity. However, the MMU optimization is fundamental to proton radiotherapy (RT), including efficient IMPT, proton arc delivery (ARC), and FLASH-RT. This work aims to develop a new optimization algorithm that is effective in solving the MMU problem. Approach: Our new algorithm is primarily based on stochastic coordinate decent (SCD) method. It involves three major steps: first to decouple the determination of active sets for dose-volume-histogram (DVH) planning constraints from the MMU problem via iterative convex relaxation method; second to handle the nonconvexity of the MMU constraint via SCD to localize the index set of nonzero spots; third to solve convex subproblems projected to this convex set of nonzero spots via projected gradient descent method. Main results: Our new method SCD is validated and compared with alternating direction method of multipliers (ADMM) for IMPT and ARC. The results suggest SCD had better plan quality than ADMM, e.g., the improvement of conformal index (CI) from 0.51 to 0.71 during IMPT, and from 0.22 to 0.86 during ARC for the lung case. Moreover, SCD successfully handled the nonconvexity from large MMU threshold that ADMM failed to handle, in the sense that (1) the plan quality from ARC was worse than IMPT (e.g., CI was 0.51 with IMPT and 0.22 with ARC for the lung case), when ADMM was used; (2) in contrast, with SCD, ARC achieved better plan quality than IMPT (e.g., CI was 0.71 with IMPT and 0.86 with ARC for the lung case), which is compatible with more optimization degrees of freedom from ARC compared to IMPT. Significance: To the best of our knowledge, our new MMU optimization method via SCD can effectively handle the nonconvexity from large MMU threshold that none of the current methods can solve. Therefore, we have developed a unique MMU optimization algorithm via SCD that can be used for efficient IMPT, proton arc delivery (ARC), FLASH-RT, and other particle RT applications where large MMU threshold is desirable (e.g., for the delivery of high dose rates or/and a large number of spots).

An Efficient Algorithm for Convex Biclustering

We consider biclustering that clusters both samples and features and propose efficient convex biclustering procedures. The convex biclustering algorithm (COBRA) procedure solves twice the standard convex clustering problem that contains a non-differentiable function optimization. We instead convert the original optimization problem to a differentiable one and improve another approach based on the augmented Lagrangian method (ALM). Our proposed method combines the basic procedures in the ALM with the accelerated gradient descent method (Nesterov’s accelerated gradient method), which can attain O(1/k2) convergence rate. It only uses first-order gradient information, and the efficiency is not influenced by the tuning parameter λ so much. This advantage allows users to quickly iterate among the various tuning parameters λ and explore the resulting changes in the biclustering solutions. The numerical experiments demonstrate that our proposed method has high accuracy and is much faster than the currently known algorithms, even for large-scale problems.

Confidence-Controlled Hebbian Learning Efficiently Extracts Category Membership From Stimuli Encoded in View of a Categorization Task

In experiments on perceptual decision making, individuals learn a categorization task through trial-and-error protocols. We explore the capacity of a decision-making attractor network to learn a categorization task through reward-based, Hebbian-type modifications of the weights incoming from the stimulus encoding layer. For the latter, we assume a standard layer of a large number of stimu lus-specific neurons. Within the general framework of Hebbian learning, we have hypothesized that the learning rate is modulated by the reward at each trial. Surprisingly, we find that when the coding layer has been optimized in view of the categorization task, such reward-modulated Hebbian learning (RMHL) fails to extract efficiently the category membership. In previous work, we showed that the attractor neural networks' nonlinear dynamics accounts for behavioral confidence in sequences of decision trials. Taking advantage of these findings, we propose that learning is controlled by confidence, as computed from the neural activity of the decision-making attractor network. Here we show that this confidence-controlled, reward-based Hebbian learning efficiently extracts categorical information from the optimized coding layer. The proposed learning rule is local and, in contrast to RMHL, does not require storing the average rewards obtained on previous trials. In addition, we find that the confidence-controlled learning rule achieves near-optimal performance. In accordance with this result, we show that the learning rule approximates a gradient descent method on a maximizing reward cost function.

Evaluation of Neural Network Model for Estimating Pile Load Capacity

Neural network models based on deep learning algorithms are increasingly used for estimating pile load capacities as supplements of bearing capacity equations and field load tests. A series of hyperparameter tuning is required to improve the performance and reliability of developing a neural network model. In this study, the number of hidden layers and neurons, the activation functions, the optimizing algorithms of the gradient descent method, and the learning rates were tuned. The grid search method was applied for the tuning, which is a hyperpameter optimizer supplied by the developing platform. The cross-validation method was applied to enhance reliability for model validation. An appropriate number of epochs was determined using the early stopping method to prevent the overfitting of the model to the training data. The performance of the tuned optimum model evaluated for the test data set revealed that the model could estimate pile load capacities approximately with an average absolute error of 3,000 kN and a coefficient of determinant of 0.5.

Classification of Benign and Malignant Lung Nodules Based on Deep Convolutional Network Feature Extraction

With the rapid development of detection technology, CT imaging technology has been widely used in the early clinical diagnosis of lung nodules. However, accurate assessment of the nature of the nodule remains a challenging task due to the subjective nature of the radiologist. With the increasing amount of publicly available lung image data, it has become possible to use convolutional neural networks for benign and malignant classification of lung nodules. However, as the network depth increases, network training methods based on gradient descent usually lead to gradient dispersion. Therefore, we propose a novel deep convolutional network approach to classify the benignity and malignancy of lung nodules. Firstly, we segmented, extracted, and performed zero-phase component analysis whitening on images of lung nodules. Then, a multilayer perceptron was introduced into the structure to construct a deep convolutional network. Finally, the minibatch stochastic gradient descent method with a momentum coefficient is used to fine-tune the deep convolutional network to avoid the gradient dispersion. The 750 lung nodules in the lung image database are used for experimental verification. Classification accuracy of the proposed method can reach 96.0%. The experimental results show that the proposed method can provide an objective and efficient aid to solve the problem of classifying benign and malignant lung nodules in medical images.