Information Bottleneck Theory Based Exploration of Cascade Learning

In solving challenging pattern recognition problems, deep neural networks have shown excellent performance by forming powerful mappings between inputs and targets, learning representations (features) and making subsequent predictions. A recent tool to help understand how representations are formed is based on observing the dynamics of learning on an information plane using mutual information, linking the input to the representation (I(X;T)) and the representation to the target (I(T;Y)). In this paper, we use an information theoretical approach to understand how Cascade Learning (CL), a method to train deep neural networks layer-by-layer, learns representations, as CL has shown comparable results while saving computation and memory costs. We observe that performance is not linked to information–compression, which differs from observation on End-to-End (E2E) learning. Additionally, CL can inherit information about targets, and gradually specialise extracted features layer-by-layer. We evaluate this effect by proposing an information transition ratio, I(T;Y)/I(X;T), and show that it can serve as a useful heuristic in setting the depth of a neural network that achieves satisfactory accuracy of classification.

Chunking in tonal contexts: Information compression during serial recall of visually presented musical notes

Chunking is defined as information compression by means of encoding meaningful units. To advance the understanding of chunking in musical memory, the present study tested characteristics of melodic sequences that might enable a parsimonious memory representation, namely, the presence of a clear tonal context and of melodic cells with clear labels. Musical note symbols, which formed either triads (Experiment 1) or cadences (Experiment 2), were presented visually and sequentially to musically experienced participants for immediate serial recall. The melodic sequences were varied on the within-participant factors list length (long vs. short list) and tonal structure (chunking-supportive vs. chunking-obstructive). Chunking-supportive sequences contained tones from a single diatonic key that formed melodic cells with a clear label, such as “C major triad”. Transitional errors showed that participants grouped notes into melodic cells. Mixed logistic regression modeling revealed that recall was more accurate in chunking-supportive sequences and that this advantage was more pronounced for more experienced participants in the long list length condition of Experiment 2. The findings suggest that a clear tonal context and melodic cells with clear labels benefit chunking in melodic processing, but that the subtleties of the process are additionally influenced by type, size, and number of melodic cells.

Data transmission with Simulink on 6-DoF platform on CAN BUS

Use of CAN BUS for data transmission in Real-Time mode with Simulink on control objects is considered (6-DoF a platform).It is revealed that software of CAN_ API.dll adapters, created in the Microsoft Visual Studio (MVS) does not work with TDM-GCC-64 Matlab/Simulink because of different approach in names of the dll functions according to the standard C ++ 11/17. Recompile by the developer of the adapter of its software (dll) in the TDM-GCC-64 environment under Windows is required.It is established that CAN BUS considerably reduces modeling speed by 4.5 times. The way of information compression and fall forward of exchange twice due to byte-by-byte entering of two float values in the data field is offered. Use of identical values of identifiers is applied to two cylinders 6-DoF of a platform and the subsequent their division in the program microcontrollers of cylinders.For implementation of a Real-Time mode in addition to compression it is offered to transfer data with the smaller frequency (quantization) by what a modeling clock period. It was considered that 6-DoF platforms reproduce frequency band to 10–12 Hz. The program of transfer/data exchange with Simulink on stand control devices with quantization is developed. Influence of parameter of quantization for the period of modeling is investigated. It is established that the Real-Time mode of modeling is provided in the range of parameters of quantization (chc=1/350–1/1000). Frequency of exchange with 6 cylinders at the same time corresponds to 230, 150 Hz.

How the SP System May Promote Sustainability in Energy Consumption in IT Systems

The SP System (SPS), referring to the SP Theory of Intelligence and its realisation as the SP Computer Model, has the potential to reduce demands for energy from IT, especially in AI applications and in the processing of big data, in addition to reductions in CO2 emissions when the energy comes from the burning of fossil fuels. The biological foundations of the SPS suggest that with further development, the SPS may approach the extraordinarily low (20 W)energy demands of the human brain. Some of these savings may arise in the SPS because, like people, the SPS may learn usable knowledge from a single exposure or experience. As a comparison, deep neural networks (DNNs) need many repetitions, with much consumption of energy, for the learning of one concept. Another potential saving with the SPS is that like people, it can incorporate old learning in new. This contrasts with DNNs where new learning wipes out old learning (‘catastrophic forgetting’). Other ways in which the mature SPS is likely to prove relatively parsimonious in its demands for energy arise from the central role of information compression (IC) in the organisation and workings of the system: by making data smaller, there is less to process; because the efficiency of searching for matches between patterns can be improved by exploiting probabilities that arise from the intimate connection between IC and probabilities; and because, with SPS-derived ’Model-Based Codings’ of data, there can be substantial reductions in the demand for energy in transmitting data from one place to another.

Applying Wavelet transform to information compression

This paper discusses the application of wavelet transform for information compression, analysis of discrete wavelet transform tools using the software. Algorithms for audio signal compression and image compression using wavelets are considered.

The Potential of the SP System in Machine Learning and Data Analysis for Image Processing

This paper aims to describe how pattern recognition and scene analysis may with advantage be viewed from the perspective of the SP system (meaning the SP theory of intelligence and its realisation in the SP computer model (SPCM), both described in an appendix), and the strengths and potential of the system in those areas. In keeping with evidence for the importance of information compression (IC) in human learning, perception, and cognition, IC is central in the structure and workings of the SPCM. Most of that IC is achieved via the powerful concept of SP-multiple-alignment, which is largely responsible for the AI-related versatility of the system. With examples from the SPCM, the paper describes: how syntactic parsing and pattern recognition may be achieved, with corresponding potential for visual parsing and scene analysis; how those processes are robust in the face of errors in input data; how in keeping with what people do, the SP system can “see” things in its data that are not objectively present; the system can recognise things at multiple levels of abstraction and via part-whole hierarchies, and via an integration of the two; the system also has potential for the creation of a 3D construct from pictures of a 3D object from different viewpoints, and for the recognition of 3D entities.