Evolving Deep Architecture Generation with Residual Connections for Image Classification Using Particle Swarm Optimization

Automated deep neural architecture generation has gained increasing attention. However, exiting studies either optimize important design choices, without taking advantage of modern strategies such as residual/dense connections, or they optimize residual/dense networks but reduce search space by eliminating fine-grained network setting choices. To address the aforementioned weaknesses, we propose a novel particle swarm optimization (PSO)-based deep architecture generation algorithm, to devise deep networks with residual connections, whilst performing a thorough search which optimizes important design choices. A PSO variant is proposed which incorporates a new encoding scheme and a new search mechanism guided by non-uniformly randomly selected neighboring and global promising solutions for the search of optimal architectures. Specifically, the proposed encoding scheme is able to describe convolutional neural network architecture configurations with residual connections. Evaluated using benchmark datasets, the proposed model outperforms existing state-of-the-art methods for architecture generation. Owing to the guidance of diverse non-uniformly selected neighboring promising solutions in combination with the swarm leader at fine-grained and global levels, the proposed model produces a rich assortment of residual architectures with great diversity. Our devised networks show better capabilities in tackling vanishing gradients with up to 4.34% improvement of mean accuracy in comparison with those of existing studies.

Continuous Particle Swarm Optimization-Based Deep Learning Architecture Search for Hyperspectral Image Classification

Deep convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the most successful CNN architectures are handcrafted, which need professional knowledge and consume a very significant amount of time. To automatically design cell-based CNN architectures for HSI classification, we propose an efficient continuous evolutionary method, named CPSO-Net, which can dramatically accelerate optimal architecture generation by the optimization of weight-sharing parameters. First, a SuperNet with all candidate operations is maintained to share the parameters for all individuals and optimized by collecting the gradients of all individuals in the population. Second, a novel direct encoding strategy is devised to encode architectures into particles, which inherit the parameters from the SuperNet. Then, particle swarm optimization is used to search for the optimal deep architecture from the particle swarm. Furthermore, experiments with limited training samples based on four widely used biased and unbiased hyperspectral datasets showed that our proposed method achieves good performance comparable to the state-of-the-art HSI classification methods.

Evolving Deep DenseBlock Architecture Ensembles for Image Classification

Automatic deep architecture generation is a challenging task, owing to the large number of controlling parameters inherent in the construction of deep networks. The combination of these parameters leads to the creation of large, complex search spaces that are feasibly impossible to properly navigate without a huge amount of resources for parallelisation. To deal with such challenges, in this research we propose a Swarm Optimised DenseBlock Architecture Ensemble (SODBAE) method, a joint optimisation and training process that explores a constrained search space over a skeleton DenseBlock Convolutional Neural Network (CNN) architecture. Specifically, we employ novel weight inheritance learning mechanisms, a DenseBlock skeleton architecture, as well as adaptive Particle Swarm Optimisation (PSO) with cosine search coefficients to devise networks whilst maintaining practical computational costs. Moreover, the architecture design takes advantage of recent advancements of the concepts of residual connections and dense connectivity, in order to yield CNN models with a much wider variety of structural variations. The proposed weight inheritance learning schemes perform joint optimisation and training of the architectures to reduce the computational costs. Being evaluated using the CIFAR-10 dataset, the proposed model shows great superiority in classification performance over other state-of-the-art methods while illustrating a greater versatility in architecture generation.

Main Stages in the Formation of the Theory of Architecture of the Spanish Renaissance (Based on Treatises of the 16th–17th centuries)

The Spanish treatises of the 16th — early 17th century on the theory of architecture are reviewed in the context of design evolution, stylistic features of text and visual material, sources of theoretical thought of predecessors that influenced the creative rethinking of architecture, generation of new original ideas and the determination of their place in the evolution of the architectural views of the Renaissance as a whole. The appeal to antiquity as a treasury of humanistic values was typical for the Spanish culture of the 16th century. The Spanish theorists interpreted the focus on the classical tradition in architecture as the embodiment of the ideas of Christian architecture, the source of which was the theoretical legacy of the Italian Renaissance. The main stages in the formation of the theory of architecture of the Spanish Renaissance are reflected in the works of Diego de Sagredo “The Medidas del Romano” (1526), Juan de Arfe “Measurements for architecture and sculpture” (1564), Fray Lorenzo de san Nicolás “Art use of architecture” (1639). The authors of the treatises tell about the benefits and the need for study, use the construction and artistic experience of antiquity, and provide a didactic overview of the most significant theoretical works of their predecessors with practical tips for building professional skills. In general, the Vitruvian style as the structural basis of classicism was very popular in Spain during Renaissance, and its components serve as the fundamental method of practical use at each stage of the development of architecture.