Image Recognition of Standard Actions in Sports Videos Based on Feature Fusion

Standard actions are crucial to sports training of athletes and daily exercise of ordinary people. There are two key issues in sports action recognition: the extraction of sports action features, and the classification of sports actions. The existing action recognition algorithms cannot work effectively on sports competitions, which feature high complexity, fine class granularity, and fast action speed. To solve the problem, this paper develops an image recognition method of standard actions in sports videos, which merges local and global features. Firstly, the authors combed through the functions and performance required for the recognition of standard actions of sports, and proposed an attention-based local feature extraction algorithm for the frames of sports match videos. Next, a sampling algorithm was developed based on time-space compression, and a standard sports action recognition algorithm was designed based on time-space feature fusion, with the aim to fuse the time-space features of the standard actions in sports match videos, and to overcome the underfitting problem of direct fusion of time-space features extracted by the attention mechanism. The workflow of these algorithms was explained in details. Experimental results confirm the effectiveness of our approach.

LOCAL FEATURE EXTRACTION IN IMAGES

The methods of the local feature point extraction are analyzed. The analysis shows that the most effective detectors are based on the brightness gradient determination. They usually use the Harris angle detector, which is complex in calcu­la­tions. The algorithm complexity minimization contradicts both the detector effective­ness and to the high dynamic range of the analyzed image. As a result, the high-speed methods could not recognize the feature points in the heavy luminance conditions. The modification of the high dynamic range (HDR) image compression algorithm based on the Retinex method is proposed. It contains an adaptive filter, which preserves the image edges. The filter is based on a set of feature detectors perfor­ming the Harris-Laplace transform which is much simpler than the Harris angle detector. A prototype of the HDR video camera is designed which provides sharp images. Its structure simplifies the design of the artificial intelligence engine, which is implemented in FPGA of medium or large size.

PlaneNet: an efficient local feature extraction network

Due to memory and computing resources limitations, deploying convolutional neural networks on embedded and mobile devices is challenging. However, the redundant use of the 1 × 1 convolution in traditional light-weight networks, such as MobileNetV1, has increased the computing time. By utilizing the 1 × 1 convolution that plays a vital role in extracting local features more effectively, a new lightweight network, named PlaneNet, is introduced. PlaneNet can improve the accuracy and reduce the numbers of parameters and multiply-accumulate operations (Madds). Our model is evaluated on classification and semantic segmentation tasks. In the classification tasks, the CIFAR-10, Caltech-101, and ImageNet2012 datasets are used. In the semantic segmentation task, PlaneNet is tested on the VOC2012 datasets. The experimental results demonstrate that PlaneNet (74.48%) can obtain higher accuracy than MobileNetV3-Large (73.99%) and GhostNet (72.87%) and achieves state-of-the-art performance with fewer network parameters in both tasks. In addition, compared with the existing models, it has reached the practical application level on mobile devices. The code of PlaneNet on GitHub: https://github.com/LinB203/planenet.

Fully automatic image colorization based on semantic segmentation technology

Aiming at these problems of image colorization algorithms based on deep learning, such as color bleeding and insufficient color, this paper converts the study of image colorization to the optimization of image semantic segmentation, and proposes a fully automatic image colorization model based on semantic segmentation technology. Firstly, we use the encoder as the local feature extraction network and use VGG-16 as the global feature extraction network. These two parts do not interfere with each other, but they share the low-level feature. Then, the first fusion module is constructed to merge local features and global features, and the fusion results are input into semantic segmentation network and color prediction network respectively. Finally, the color prediction network obtains the semantic segmentation information of the image through the second fusion module, and predicts the chrominance of the image based on it. Through several sets of experiments, it is proved that the performance of our model becomes stronger and stronger under the nourishment of the data. Even in some complex scenes, our model can predict reasonable colors and color correctly, and the output effect is very real and natural.

Memory-Augmented Transformer for Remote Sensing Image Semantic Segmentation

The semantic segmentation of remote sensing images requires distinguishing local regions of different classes and exploiting a uniform global representation of the same-class instances. Such requirements make it necessary for the segmentation methods to extract discriminative local features between different classes and to explore representative features for all instances of a given class. While common deep convolutional neural networks (DCNNs) can effectively focus on local features, they are limited by their receptive field to obtain consistent global information. In this paper, we propose a memory-augmented transformer (MAT) to effectively model both the local and global information. The feature extraction pipeline of the MAT is split into a memory-based global relationship guidance module and a local feature extraction module. The local feature extraction module mainly consists of a transformer, which is used to extract features from the input images. The global relationship guidance module maintains a memory bank for the consistent encoding of the global information. Global guidance is performed by memory interaction. Bidirectional information flow between the global and local branches is conducted by a memory-query module, as well as a memory-update module, respectively. Experiment results on the ISPRS Potsdam and ISPRS Vaihingen datasets demonstrated that our method can perform competitively with state-of-the-art methods.

Hyperspectral Image Classification via a Novel Spectral–Spatial 3D ConvLSTM-CNN

In recent years, deep learning-based models have produced encouraging results for hyperspectral image (HSI) classification. Specifically, Convolutional Long Short-Term Memory (ConvLSTM) has shown good performance for learning valuable features and modeling long-term dependencies in spectral data. However, it is less effective for learning spatial features, which is an integral part of hyperspectral images. Alternatively, convolutional neural networks (CNNs) can learn spatial features, but they possess limitations in handling long-term dependencies due to the local feature extraction in these networks. Considering these factors, this paper proposes an end-to-end Spectral-Spatial 3D ConvLSTM-CNN based Residual Network (SSCRN), which combines 3D ConvLSTM and 3D CNN for handling both spectral and spatial information, respectively. The contribution of the proposed network is twofold. Firstly, it addresses the long-term dependencies of spectral dimension using 3D ConvLSTM to capture the information related to various ground materials effectively. Secondly, it learns the discriminative spatial features using 3D CNN by employing the concept of the residual blocks to accelerate the training process and alleviate the overfitting. In addition, SSCRN uses batch normalization and dropout to regularize the network for smooth learning. The proposed framework is evaluated on three benchmark datasets widely used by the research community. The results confirm that SSCRN outperforms state-of-the-art methods with an overall accuracy of 99.17%, 99.67%, and 99.31% over Indian Pines, Salinas, and Pavia University datasets, respectively. Moreover, it is worth mentioning that these excellent results were achieved with comparatively fewer epochs, which also confirms the fast learning capabilities of the SSCRN.

Application of Multiattention Mechanism in Power System Branch Parameter Identification

Maintaining accuracy and robustness has always been an unsolved problem in the task of power grid branch parameter identification. Therefore, many researchers have participated in the research of branch parameter identification. The existing methods of power grid branch parameter identification suffer from two limitations. (1) Traditional methods only use manual experience or instruments to complete parameter identification of single branch characteristics, but they are only used to identify a single target and cannot make full use of the historical information of power grid data. (2) Deep learning methods can complete model training through historical data, but these methods cannot consider the constraints of power grid topological structure, which is equivalent to identifying connected power grid branches separately. To overcome these limitations, we propose a novel multitask Graph Transformer Network (GTN), which combines a graph neural network and a multiattention mechanism to construct our model. Specifically, we input the global features and topology information of branch nodes into our GTN model. In the process of parameter identification, the multihead attention mechanism is used to fuse the branch feature information of different subspaces, which highlights the importance of different branches and enhances the ability of local feature extraction. Finally, the fitting and prediction of each branch feature are completed through the decoding layer. The experiment shows that our proposed GTN is superior to other machine learning methods and deep learning methods and can still realize accurate branch parameter identification under various noise conditions.