Prior latent distribution comparison for the RNN variational autoencoder in low-resource language modeling

Probabilistic Bayesian methods are widely used in the machine learning domain. Variational Autoencoder (VAE) is a common architecture for solving the Language Modeling task in a self-supervised way. VAE consists of a concept of latent variables inside the model. Latent variables are described as a random variable that is fit by the data. Up to now, in the majority of cases, latent variables are considered normally distributed. The normal distribution is a well-known distribution that can be easily included in any pipeline. Moreover, the normal distribution is a good choice when the Central Limit Theorem (CLT) holds. It makes it effective when one is working with i.i.d. (independent and identically distributed) random variables. However, the conditions of CLT in Natural Language Processing are not easy to check. So, the choice of distribution family is unclear in the domain. This paper studies the priors selection impact of continuous distributions in the Low-Resource Language Modeling task with VAE. The experiment shows that there is a statistical difference between the different priors in the encoder-decoder architecture. We showed that family distribution hyperparameter is important in the Low-Resource Language Modeling task and should be considered for the model training.

Introduction to the Theory of Imprecise Probability

The theory of imprecise probability is a generalization of classical ‘precise’ probability theory that allows modeling imprecision and indecision. This is a practical advantage in situations where a unique precise uncertainty model cannot be justified. This arises, for example, when there is a relatively small amount of data available to learn the uncertainty model or when the model’s structure cannot be defined uniquely. The tools the theory provides make it possible to draw conclusions and make decisions that correctly reflect the limited information or knowledge available for the uncertainty modeling task. This extra expressivity however often implies a higher computational burden. The goal of this chapter is to primarily give you the necessary knowledge to be able to read literature that makes use of the theory of imprecise probability. A secondary goal is to provide the insight needed to use imprecise probabilities in your own research. To achieve the goals, we present the essential concepts and techniques from the theory, as well as give a less in-depth overview of the various specific uncertainty models used. Throughout, examples are used to make things concrete. We build on the assumed basic knowledge of classical probability theory.

Secondary Teacher Candidates’ Mathematical Modeling Task Design and Revision

The purpose of this study is to investigate secondary teacher candidates’ experience of mathematical modeling task design. In the study, 54 teacher candidates in a university-based teacher education program created modeling tasks and scoring rubrics. Next, the participants pilot-tested the tasks with students and had the opportunity to revise the original tasks and rubrics based on student responses. The data included participants’ statements, in which they described and reflected on the design and revision process of modeling tasks. The study describes six didactic revision strategies in revising modeling tasks and identifies five emerging pedagogical ideas from revising tasks and rubrics. The study also discusses the way modeling task design activities have the potential to support teacher candidates’ learning through a bottom-up modeling curriculum in teacher education.

Command2Vec: Feature Learning of 3D Modeling Behavior Sequence—A Case Study on “Spiral-stair”

In this study, we applied machine learning to mine the event logs generated in modeling process for behavior sequence clustering. The motivation for the study is to develop cognitively intelligent 3D tools through process mining which has been a hot area in recent years. In this study, we develop a novel classification method Command2Vec to perceive, learn and classify different design behavior during 3D-modeling aided design process. The method is applied in a case study of 112 participate students on a ‘Spiral-stair’ modeling task. By extracting the event logs generated in each participate student’s modeling process into a new data structures: ‘command graph’, we classified participants’ behavior sequences from final 99 valid event logs into certain groups using our novel Command2Vec. To verify the effectiveness of our classification, we invited five experts with extensive modeling experience to grade the classification results. The final grading shows that our algorithm performs well in certain grouping of classification with significant features.

IMPLEMENTATION OF THE INDICATOR SYSTEM IN MODELING OF COMPLEX TECHNICAL SYSTEMS

Context. The problem of determining the optimal topology of a neuromodel, which is characterized by a high level of logical transparency in modeling complex technical systems, is considered. The object of research is the process of applying an indicator system to simplify and select the topology of neuromodels. Objective of the work is to develop and use a system of indicators to determine the level of complexity of the modeling problem and gradually select the optimal logically transparent topology of the neuromodel. Method. A method is proposed for selecting an optimal, logically transparent neural network topology for modeling complex technical systems using a system of corresponding indicators. At the beginning, the method determines the overall level of complexity of the modeling task and, using the obtained estimate, determines the method for further optimization of the neuromodel. Then, using Task data and input data characteristics, the method allows to obtain the most optimal structure of the neural model for further modeling of the system. The method reduces trainingvtime and increases the level of logical transparency of neuromodels, which significantly expands the practical use of such models, without using neuroevolution methods, which may not be justified by resource-intensive tasks. Results. The developed method is implemented and investigated in solving the problem of modeling the dynamics of pitting processes of steel alloys. Using the developed method made it possible to reduce the training time of the model by 22%, depending on the computing resources used. The method also increased the level of logical transparency of the model by reducing the number of computing nodes by 50%, which also indicates faster and more efficient use of resources. Conclusions. The conducted experiments confirmed the operability of the proposed mathematical support and allow us to recommend it for use in practice in the design of topologies of neuromodels for further solving modeling, diagnosis and evaluation problems. Prospects for further research may consist in the development of methods for structural optimization of previously synthesized models and the development of new methods for feature selection.

An External Knowledge Enhanced Graph-based Neural Network for Sentence Ordering

As an important text coherence modeling task, sentence ordering aims to coherently organize a given set of unordered sentences. To achieve this goal, the most important step is to effectively capture and exploit global dependencies among these sentences. In this paper, we propose a novel and flexible external knowledge enhanced graph-based neural network for sentence ordering. Specifically, we first represent the input sentences as a graph, where various kinds of relations (i.e., entity-entity, sentence-sentence and entity-sentence) are exploited to make the graph representation more expressive and less noisy. Then, we introduce graph recurrent network to learn semantic representations of the sentences. To demonstrate the effectiveness of our model, we conduct experiments on several benchmark datasets. The experimental results and in-depth analysis show our model significantly outperforms the existing state-of-the-art models.

STEAM Practices to Explore Ancient Architectures Using Augmented Reality and 3D Printing with GeoGebra

In this study, we develop mathematical educational practices for students to explore ancient buildings using GeoGebra, Augmented Reality and 3D printing. It is an interdisciplinary approach, intertwining history, culture, mathematics, and engineering. For example, the 3D modelling of Cheomseongdae in Korea and the Temple of Dendera in Egypt can enable students to practice a multimodal set of traditional and innovative learning approaches. Students might use their mathematical knowledge to reflect on architectural and cultural history in a modeling task.