Studying the Evolution of Neural Activation Patterns During Training of Feed-Forward ReLU Networks

The ability of deep neural networks to form powerful emergent representations of complex statistical patterns in data is as remarkable as imperfectly understood. For deep ReLU networks, these are encoded in the mixed discrete–continuous structure of linear weight matrices and non-linear binary activations. Our article develops a new technique for instrumenting such networks to efficiently record activation statistics, such as information content (entropy) and similarity of patterns, in real-world training runs. We then study the evolution of activation patterns during training for networks of different architecture using different training and initialization strategies. As a result, we see characteristic- and general-related as well as architecture-related behavioral patterns: in particular, most architectures form bottom-up structure, with the exception of highly tuned state-of-the-art architectures and methods (PyramidNet and FixUp), where layers appear to converge more simultaneously. We also observe intermediate dips in entropy in conventional CNNs that are not visible in residual networks. A reference implementation is provided under a free license1.

TIME REPRESENTATION IN SCIENTIFIC DIAGRAMS: GALILEO GALILEI’S SOLUTION TO THE PROBLEM OF THE MOTION OF A FALLING BODY

Рассматриваются способы научного изображения темпоральных явлений на примере чертежей Галилео Галилея, при помощи которых он описывает и исследует равноускоренное движение. Для анализа применяется концептуальная рамка теории изобразительной и неизобразительной репрезентации Грегори Карри. Показано, что в случае научных диаграмм и графиков, представляющих время как одно из измерений пространства, основанием для геометрической изобразимости времени становится полагаемый изоморфизм между временем как континуумом мгновений и линией как континуумом точек. Парадигму такого структурного сопоставления мы находим в математическом мышлении Галилея, наиболее ярко проявляющемся в доказательстве формулы равноускоренного движения, представленном в «Беседах и математических доказательствах». The textbook narrative of the scientific revolution of the 17th century says that the early modern transformation of physics and mechanics was grounded in mathematization, that is, the application of mathematical principles and procedures to physical entities and events. However, such a transformation faces a major obstacle: compared to geometry, mechanics includes an additional dimension, namely, time. When temporality of motion is to be represented geometrically, a question arises on how a temporal succession can be expressed by a static image. The problem of representation of temporal events is not limited to science. In my paper, I apply a conceptual tool elaborated by Gregory Currie for the analysis of temporal representations in art, especially in cinema, to the analysis of scientific diagrams. In his book Image and Mind. Film, Philosophy, and Cognitive Science (1995), Currie distinguishes depictive and nondepictive representations, arguing that depictive representation requires similarity and homomorphism between an object ant its representation. Thus, it seems that any non-temporal image of temporal processes would lack the required similarity and cannot be a depictive representation. However, taking into account explanations given by Galileo Galilei for his famous diagrams of accelerated motion, I argue that the representation of time in scientific diagrams as a geometrical line is grounded in isomorphism between time as a continuous structure and continuous structure of a geometrical line. The main temporal process studied by mechanics is motion. Motion can be represented in two main ways: as a trajectory of a body over some period of time or as a functional relation of various parameters of motion (speed, path, acceleration) versus time. In the latter case, time is usually represented in a diagram as a geometrical line. We can find the origin of this type of representation in the late medieval doctrine of ‘intensio et remissio qualitatum’, intension and remission of qualities, in the context of which first diagrams representing intensity and extension of velocity of nonuniform motion as a changing quality over time were produced (Nicolas Oresme). We can find very similar graphical schemes in Galileo Galilei’s works, especially in Discorsi e dimostrazioni matematiche intorno a due nuove scienze (1638). In this work, Galileo announces with all clarity that he considers time to be the same aggregate of temporal moments as a line is an aggregate of points: every moment of time has a corresponding point on the geometrical line. This allows us to establish a homomorphic similarity between temporal duration and spatial (geometrical) extension. Thus, the essential requirement for depictive representation is met. Concluding, I have to point out that the homomorphic relation in this case is established between not real but abstract entities. The visible line itself is a representation of non-visible abstract geometrical line; in the same way, time consisting of non-divisible moments is just an abstract construction which refers to physical of psychological time-duration. However, the established relation between abstract time and abstract geometrical lines is a grounding event of the modern physical science.

Continuous Arch Bridges Over Lake Tisza, Hungary

<p>The design and construction process of the unique cycling bridges at Lake Tisza are presented in this article. The 4 new bridges are parts of the closing segment of the cycle route around the artificial reservoir, which is a popular tourist destination in Hungary, and part of the UNESCO World Heritage. The proximity of the natural environment motivated the use of organic, flowing shapes.</p><p>The unique Eger- and Szomorka bridges are independent continuous half-through arch bridges, 8 spans with a total length of 308.46m, and 3 spans with a total length of 86.30 m, respectively. The bridge over River Tisza is a 5 span bridge with a total length of 279.47 m, which is placed on the extended piers of the existing roadway bridge. It consists of 2 deck truss bridges on the side-spans and 3 network arch bridges in the mid- spans. A 5.70 m span bascule bridge over one of the draining canals of the lake was also accomplished as part of the project.</p><p>The Eger and Szomorka bridges are internationally unique due to the fact that the Designers have dreamed a continuous sinusoid wave on the supports; which, by twirling under and above the deck, results in a continuous structure. The successful construction of the Tisza River Bridge also required some special and unprecedented construction methods.</p>

Alone in Song (The Poetry of Ada Salas)

A glance at Ada Salas’ poetry (Cáceres, 1965). Her literary personality revitalizes an aesthetics conceived as research and knowledge that veils the autobiographical narrative and justifies the essential thrust of language and its transcendent dimension. The journey assumes from the beginning of a personal approach, a unique proposition without generational debts and it has a unitary and continuous structure that reflects on the identity and light observation of the environment.

Static and dynamic size optimization of nonlinear continuous structure based on fully stressed design

In order to realize the fully stressed criterion, that is, the stress of the nonlinear engineering structure is uniform along the height direction under the action of wind load or ground motion, the section size of the continuous structure is optimized based on mechanical analysis. The engineering structure is simplified as a continuous cantilever with variable cross-section, and its material constitution is nonlinear. The wind load and ground motion are represented by three kinds of horizontal static loads: uniform load, inverted triangle load, and inertia force related load. In order to meet the needs of different projects, hollow circular section and box section are designed, respectively. By establishing different expressions of section moment, the optimal section size distribution is solved. Then the optimal stiffness distribution of nonlinear structure is proposed. The correctness of the theory is verified by the finite element method. The results are suitable for the elastic and elastic-plastic stages of the structure, and are effective for both static loads and dynamic actions. The optimal section size distribution and structural shape are different under different loads. Finally, a practical design example is shown.

Non-Concatenative Morpheme Segmentation in Adults and Children

While most morphemes in the world’s language involve continuous structure or concatenation (e.g., prefixes and suffixes), many languages show some form of non-adjacent, non-concatenative morphology. Non-concatenative morphology poses a challenge for statistical learning approaches to morpheme segmentation because the combinatorial possibilities greatly increase for non-adjacent dependencies. The present study explores the types of dependencies that human learners (school-aged children and adults) are able to extract from exposure to a miniature, artificial non-concatenative system. In Experiment 1, participants were exposed to 12 CCC ‘roots’ that fit into 72 CVCVC skeletons with a high variety of VV ‘residue’. Experiment 2 extended Experiment 1 to school-aged children (with adult controls). Experiment 3 replicated Experiment 1, but with ‘mixed’ consonant-vowel roots and residues. Across all three experiments, participants were able to recognize familiar items compared to novel items, but had limited ability to generalize the CCC roots to novel items, suggesting a limited ability to parse consonantal roots. Adults were better at generalizing to novel items compared to children.

Advanced necklace-like metal organic framework with ultrahighly continuous structure in membrane for superior butanol/water separation

The fabrication of mixed matrix membranes (MMMs) for pervaporation recovery of butanol from aqueous solutions is garnering significant attention. However, constructing highly continuous metal-organic framework (MOF) with reasonable loading as...

Use of Physical Concepts in Musical Instruments and their Use in Cinema

Cinema without music is now a days beyond imagination. Whatever musical instruments used in the making of Cinema are based on Physical Concepts. One of the concepts is frequency. Frequency is nothing but the number of vibrations per second. Different frequencies forms melodious compositions. There are many musical instruments like Pan flute, Sitar, Violin, Harmonium etc are all Physical Concepts dependent musical instruments. How Physical Concepts play important role in the making of such musical instruments is presented in this paper. Unwanted continuous disturbance due to vibrations in air is noise and composition disturbance of different frequencies called sound. Music is sound with a discrete structure. Noise is sound with a continuous structure. Music is composed of sounds with a fundamental frequency and overtones. Noise is composed of sounds with frequencies that range continuously in value from as low as you can hear to as high as you can hear — not necessarily at equal intensity