Associative Diffusion and the Pitfalls of Structural Reductionism

2021 ◽  
pp. 000312242110571
Author(s):  
Amir Goldberg
Keyword(s):  
Diffusion Model ◽  
Network Size ◽  
Diffusion Models ◽  
Large Networks ◽  
Cultural Differentiation ◽  
Fully Connected ◽  
Fully Connected Networks

In their insightful comment, DellaPosta and Davoodi argue that our finding (Goldberg and Stein 2018) that segmented networks inhibit cultural differentiation does not generalize to large networks. However, their demonstration rests on an incorrect implementation of the preference updating process in the associative diffusion model. We show that once this discrepancy is corrected, cultural differentiation is more pronounced in fully connected networks, irrespective of network size and even under extreme assumptions about cognitive decay. We use this as an opportunity to discuss the associative diffusion model’s assumptions and scope conditions, as well as to critically reassess prevailing contagion-based diffusion models.

Clique Here: On the Distributed Complexity in Fully-Connected Networks

2016 ◽  
Vol 26 (01) ◽  
pp. 1650004 ◽  
Author(s):  
Benny Applebaum ◽  
Dariusz R. Kowalski ◽  
Boaz Patt-Shamir ◽  
Adi Rosén
Keyword(s):  
Boolean Function ◽  
Boolean Functions ◽  
Message Passing ◽  
Constant Number ◽  
Running Time ◽  
Message Size ◽  
Explicit Function ◽  
Randomized Protocols ◽  
Fully Connected ◽  
Fully Connected Networks

We consider a message passing model with n nodes, each connected to all other nodes by a link that can deliver a message of B bits in a time unit (typically, B = O(log n)). We assume that each node has an input of size L bits (typically, L = O(n log n)) and the nodes cooperate in order to compute some function (i.e., perform a distributed task). We are interested in the number of rounds required to compute the function. We give two results regarding this model. First, we show that most boolean functions require ‸ L/B ‹ − 1 rounds to compute deterministically, and that even if we consider randomized protocols that are allowed to err, the expected running time remains [Formula: see text] for most boolean function. Second, trying to find explicit functions that require superconstant time, we consider the pointer chasing problem. In this problem, each node i is given an array Ai of length n whose entries are in [n], and the task is to find, for any [Formula: see text], the value of [Formula: see text]. We give a deterministic O(log n/ log log n) round protocol for this function using message size B = O(log n), a slight but non-trivial improvement over the O(log n) bound provided by standard “pointer doubling.” The question of an explicit function (or functionality) that requires super constant number of rounds in this setting remains, however, open.

scholarly journals Binary and Multiclass Text Classification by Means of Separable Convolutional Neural Network

Inventions ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 70
Author(s):  
Elena Solovyeva ◽  
Ali Abdullah
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Networks ◽  
Low Cost ◽  
Computational Cost ◽  
High Accuracy ◽  
Activation Functions ◽  
Fully Connected ◽  
Fully Connected Networks

In this paper, the structure of a separable convolutional neural network that consists of an embedding layer, separable convolutional layers, convolutional layer and global average pooling is represented for binary and multiclass text classifications. The advantage of the proposed structure is the absence of multiple fully connected layers, which is used to increase the classification accuracy but raises the computational cost. The combination of low-cost separable convolutional layers and a convolutional layer is proposed to gain high accuracy and, simultaneously, to reduce the complexity of neural classifiers. Advantages are demonstrated at binary and multiclass classifications of written texts by means of the proposed networks under the sigmoid and Softmax activation functions in convolutional layer. At binary and multiclass classifications, the accuracy obtained by separable convolutional neural networks is higher in comparison with some investigated types of recurrent neural networks and fully connected networks.

Intensity and dipole correlation functions for the ν3 band of compressed OCS

1981 ◽  
Vol 59 (11) ◽  
pp. 1563-1568 ◽  
Author(s):  
Nguyen-van- Thanh ◽  
J.-P. Bouanich ◽  
I. Rossi ◽  
H. Strapelias
Keyword(s):  
Dipole Moment ◽  
Diffusion Model ◽  
Ir Spectrum ◽  
Correlation Functions ◽  
Coexistence Curve ◽  
Diffusion Models ◽  
Impact Approximation ◽  
Integrated Intensities ◽  
The Impact ◽  
Dipole Correlation

The IR spectrum of the ν3 band (4.85 μm) of gaseous OCS has been investigated along the liquid–vapor coexistence curve for temperatures varying from 268 to 358 K and densities up to 80 amagat. Measurements of integrated intensities are also reported at room temperature.The dipole moment autocorrelation functions (ACF) deduced from the spectra of compressed OCS are analyzed in terms of a collisional model derived from the impact approximation and in terms of Gordon M and J diffusion models developed classically as well as the semi-classical M diffusion model. For the collisional model which implies uncorrelated rotational levels, a discrepancy between predicted and observed ACF's is noticeable from about 15 amagat. The M and J diffusion models agree well with the experimental ACF's only up to 40 amagat.

scholarly journals Convergence Behavior of DNNs with Mutual-Information-Based Regularization

Entropy ◽  
2020 ◽  
Vol 22 (7) ◽  
pp. 727 ◽  
Author(s):  
Hlynur Jónsson ◽  
Giovanni Cherubini ◽  
Evangelos Eleftheriou
Keyword(s):  
Neural Networks ◽  
Mutual Information ◽  
Low Complexity ◽  
High Dimensional ◽  
Test Accuracy ◽  
Compression Phase ◽  
Hidden Layer ◽  
Low Dimensional ◽  
Fully Connected ◽  
Fully Connected Networks

Information theory concepts are leveraged with the goal of better understanding and improving Deep Neural Networks (DNNs). The information plane of neural networks describes the behavior during training of the mutual information at various depths between input/output and hidden-layer variables. Previous analysis revealed that most of the training epochs are spent on compressing the input, in some networks where finiteness of the mutual information can be established. However, the estimation of mutual information is nontrivial for high-dimensional continuous random variables. Therefore, the computation of the mutual information for DNNs and its visualization on the information plane mostly focused on low-complexity fully connected networks. In fact, even the existence of the compression phase in complex DNNs has been questioned and viewed as an open problem. In this paper, we present the convergence of mutual information on the information plane for a high-dimensional VGG-16 Convolutional Neural Network (CNN) by resorting to Mutual Information Neural Estimation (MINE), thus confirming and extending the results obtained with low-dimensional fully connected networks. Furthermore, we demonstrate the benefits of regularizing a network, especially for a large number of training epochs, by adopting mutual information estimates as additional terms in the loss function characteristic of the network. Experimental results show that the regularization stabilizes the test accuracy and significantly reduces its variance.

scholarly journals On the gene conversion model as a mechanism for maintenance of homogeneity in systems with multiple genomes

1977 ◽  
Vol 30 (1) ◽  
pp. 89-91 ◽  
Author(s):  
Tomoko Ohta
Keyword(s):  
Population Genetics ◽  
Gene Conversion ◽  
Diffusion Model ◽  
Diffusion Models ◽  
Conversion Model ◽  
Multiple Genomes ◽  
New Mutations

SUMMARYThe gene conversion model reported by Birky & Skavaril (1976) has been analytically studied by using the theory of diffusion models of Kimura (1964) in population genetics. It has been shown that the fate of new mutations in systems with multiple genomes may be satisfactorily treated by the diffusion model.

Fully-connected networks with local connections

2009 ◽  
Vol 95 (4) ◽  
pp. 999-1004
Author(s):  
P. E. Kornilovitch ◽  
R. N. Bicknell ◽  
J. S. Yeo
Keyword(s):  
Fully Connected ◽  
Fully Connected Networks

Learning and memory properties in fully connected networks

1986 ◽  
Author(s):  
A. D. Bruce ◽  
A. Canning ◽  
B. Forrest ◽  
E. Gardner ◽  
D. J. Wallace
Keyword(s):  
Learning And Memory ◽  
Fully Connected ◽  
Fully Connected Networks

scholarly journals A Study on Water Pollution Source Localization in Sensor Networks

2016 ◽  
Vol 2016 ◽  
pp. 1-11 ◽  
Author(s):  
Jun Yang ◽  
Xu Luo
Keyword(s):  
Water Pollution ◽  
Diffusion Model ◽  
Source Localization ◽  
Pollution Source ◽  
Good Choice ◽  
Diffusion Models ◽  
Hypotheses Testing ◽  
Localization Algorithm ◽  
Source Detection ◽  
Localization Method

The water pollution source localization is of great significance to water environment protection. In this paper, a study on water pollution source localization is presented. Firstly, the source detection is discussed. Then, the coarse localization methods and the localization methods based on diffusion models are introduced and analyzed, respectively. In addition, the localization method based on the contour is proposed. The detection and localization methods are compared in experiments finally. The results show that the detection method using hypotheses testing is more stable. The performance of the coarse localization algorithm depends on the nodes density. The localization based on the diffusion model can yield precise localization results; however, the results are not stable. The localization method based on the contour is better than the other two localization methods when the concentration contours are axisymmetric. Thus, in the water pollution source localization, the detection using hypotheses testing is more preferable in the source detection step. If concentration contours are axisymmetric, the localization method based on the contour is the first option. And, in case the nodes are dense and there is no explicit diffusion model, the coarse localization algorithm can be used, or else the localization based on diffusion models is a good choice.

Sign in / Sign up

Export Citation Format

Share Document