Improving Convolutional Neural Networks with Competitive Activation Function

The activation function is the basic component of the convolutional neural network (CNN), which provides the nonlinear transformation capability required by the network. Many activation functions make the original input compete with different linear or nonlinear mapping terms to obtain different nonlinear transformation capabilities. Until recently, the original input of funnel activation (FReLU) competed with the spatial conditions, so FReLU not only has the ability of nonlinear transformation but also has the ability of pixelwise modeling. We summarize the competition mechanism in the activation function and then propose a novel activation function design template: competitive activation function (CAF), which promotes competition among different elements. CAF generalizes all activation functions that use competition mechanisms. According to CAF, we propose a parametric funnel rectified exponential unit (PFREU). PFREU promotes competition among linear mapping, nonlinear mapping, and spatial conditions. We conduct experiments on four datasets of different sizes, and the experimental results of three classical convolutional neural networks proved the superiority of our method.

Object Recognition’s Garden Path: Low Spatial Frequencies

Several experiments, notably one done by Bruner and Potter (1964), have demonstrated delayed object recognition when viewing a blurred image gradually come into focus. Bruner and Potter (1964) suggested that the wrong answer is held until there is an obvious contradiction. Others have hypothesized that “competitive activation” is responsible for delayed recognition. The results of the experiments reported in this paper are consistent with a third hypothesis, that delayed recognition is due to an initial organization of image elements that is incompatible with correct recognition and that the initial grouping and figure-ground perception, among other aspects of organization, drive subsequent perception via top-down cortical pathways. A total of 7 experiments using 3 forms of degradation supported this hypothesis. Images degraded by low-pass filtering produced significant delay in recognition, while degradation by fragmentation did not, and a third form of degradation similar to fragmentation mitigated the effect. The experiments also demonstrate that if images are low-pass filtered delayed recognition occurs with presentations of as little as 100 ms and early presentations lead to delayed recognition over long intertrial intervals, at least 105 seconds. Further, support for the hypothesis that top-down cortical influence is key to this phenomenon came from an experiment showing that masking eliminates delayed recognition for short presentations. Taken together these results support a hypothesis that delayed recognition is due to errors in perceptual organization that lead to incorrect responses and these errors are fostered by low spatial frequencies.

Photolysis of Tp′Rh(CNneopentyl)(PhNCNneopentyl) in the presence of ketones and esters: kinetic and thermodynamic selectivity for activation of different aliphatic C–H bonds

The competitive activation of the various C–H bonds found in ketones and esters by a trispyrazolylborate rhodium complex are compared.