SHAT: A Novel Asynchronous Training Algorithm That Provides Fast Model Convergence in Distributed Deep Learning

The recent unprecedented success of deep learning (DL) in various fields is underlied by its use of large-scale data and models. Training a large-scale deep neural network (DNN) model with large-scale data, however, is time-consuming. To speed up the training of massive DNN models, data-parallel distributed training based on the parameter server (PS) has been widely applied. In general, a synchronous PS-based training suffers from the synchronization overhead, especially in heterogeneous environments. To reduce the synchronization overhead, asynchronous PS-based training employs the asynchronous communication between PS and workers so that PS processes the request of each worker independently without waiting. Despite the performance improvement of asynchronous training, however, it inevitably incurs the difference among the local models of workers, where such a difference among workers may cause slower model convergence. Fro addressing this problem, in this work, we propose a novel asynchronous PS-based training algorithm, SHAT that considers (1) the scale of distributed training and (2) the heterogeneity among workers for successfully reducing the difference among the local models of workers. The extensive empirical evaluation demonstrates that (1) the model trained by SHAT converges to the higher accuracy up to 5.22% than state-of-the-art algorithms, and (2) the model convergence of SHAT is robust under various heterogeneous environments.

Synchronous-asynchronous distance learning in the process of teaching information technology on the example of Microsoft Access

В статье рассмотрена проблема организации дистанционного обучения на примере Microsoft Access в рамках курса информационных технологий. Автор задался целью представить авторскую методику дистанционного обучения информатике на основе универсальных информационно-технологических сред. Практическая значимость работы заключается в том, что показан способ синхронно-асинхронной организации учебного процесса, выявлены условия проектирования данного метода обучения, продемонстрирована его реализация на конкретном примере, выявлены проблемы, возникающие в процессе использования представленной модели обучения. Также приводятся мнения студентов о данной форме обучения. Теоретическая значимость работы состоит в том, что в ней обосновывается невозможность освоения в очном режиме целого ряда навыков и умений, в которых нуждаются обучающиеся. Но они могут получить необходимые знания в рамках представленной дистанционной модели обучения. Автор утверждает, что дистанционное обучение информационным технологиям является в разы более эффективной формой организации образовательного процесса, чем очное обучение. Основной новизной статьи является представленная методика обучения, основанная на использовании синхронно-асинхронной модели и универсальных информационно-технологических сред. The article considers the problem of organizing distance learning, using the example of Microsoft Access within the course of information technology. The article aims to present the author’s method of distance learning in IT ba ed on universal information technology environments. The practical significance of the work lies in the fact that the method of synchronous-asynchronous organization of training is shown; the conditions for the design of synchronous-asynchronous training have been identified; the implementation of such training is shown by a specific example; the problems that arise in the process of using the presented training model are identified. Moreover, we can see students’ opinions about this educational form; it is shown that, in general, students are positively disposed towards this format of learning. It is impossible to teach a whole range of skills and abilities necessary for students in full-time education unlike, however, using the presented distance learning model. This is the theoretical significance of the work. The author claims that distance learning in information technologies is a much more efficient form of organizing the educational process than full-time education. The novelty of the article lies in the presented teaching methodology based on the use of a synchronous-asynchronous model and universal information technology environments.

The Implementation of Online Training for Library Employees as an Activity Increasing Competencies in the Promotion of Culture on the Internet

Thesis / purpose of the article ‒ The aim of the article is to analyse the process of on-line training for library employees as an activity increasing competencies in the field of promoting culture on the Internet. Research methods ‒ The training process concerning areas that have a direct or indirect impact on the subsequent activities of library employees on the Internet is discussed. Various forms of online training are indicated, among which priority is given to the division into synchronous and asynchronous training. Results and conclusions ‒ Libraries, as cultural institutions, play a special role in providing access to information and cultural events. This scope of activity is of exceptional importance in the case of local communities, in particular those located outside large urban centres. Therefore, business continuity is very important, even in unusual situations such as the COVID-19 pandemic

A Cross Entropy Based Deep Neural Network Model for Road Extraction from Satellite Images

This paper proposes a deep convolutional neural network model with encoder-decoder architecture to extract road network from satellite images. We employ ResNet-18 and Atrous Spatial Pyramid Pooling technique to trade off between the extraction precision and running time. A modified cross entropy loss function is proposed to train our deep model. A PointRend algorithm is used to recover a smooth, clear and sharp road boundary. The augmentated DeepGlobe dataset is used to train our deep model and the asynchronous training method is applied to accelerate the training process. Five salellite images covering Xiaomu village are taken as input to evaluate our model. The proposed E-Road model has fewer number of parameters and shorter training time. The experiments show E-Road outperforms other state-of-the-art deep models with 5.84% to 59.09% improvement, and can give the accurate predictions for the images with complex environment.

Εξ Αποστάσεως Επιμόρφωση Εκπαιδευτικών από το 4ο ΠΕ.Κ.Ε.Σ. Κ. Μακεδονίας. Εφαρμογή, αποτίμηση και προοπτική. Teacher Training with Distance e-Learning. Application, Valuation and Perspective.

Σκοπός της εργασίας είναι η διερεύνηση της επίδρασης της εξ αποστάσεως εκπαίδευσης στη διαδικασία επιμόρφωσης των εν ενεργεία Εκπαιδευτικών Πρωτοβάθμιας και Δευτεροβάθμιας Εκπαίδευσης. Έγινε καταγραφή των απόψεων μετά από τη συμμετοχή τους σε προγράμματα εξ αποστάσεως επιμόρφωσης. Το 4ο Περιφερειακό Κέντρο Εκπαιδευτικού Σχεδιασμού (ΠΕΚΕΣ) Κ. Μακεδονίας σε συνεργασία με το «itlab» εργαστήρι πληροφορικής του Διεθνούς Πανεπιστημίου Ελλάδος (ΔΙ.ΠΑ.Ε.) ανταποκρινόμενο στις απαιτήσεις των εκπαιδευτικών για συνέχιση των εξ αποστάσεως επιμορφωτικών προγραμμάτων του σχολικού έτους 2017-2018 υλοποίησε τέσσερα (4) προγράμματα εξ αποστάσεως επιμόρφωσης, με χρήση της πλατφόρμας ασύγχρονης εκπαίδευσης OpeneClass και της πλατφόρμας σύγχρονης εκπαίδευσης BigBlueButton. Με την ολοκλήρωση σε καθένα από τα προγράμματα υλοποιήθηκε έρευνα με ερωτηματολόγια ερωτήσεων κλειστού και ανοικτού τύπου μέσω της πλατφόρμας, ώστε να καταγραφούν οι στάσεις, οι απόψεις και οι προτάσεις των εκπαιδευτικών για την εμπειρία τους κατά την διάρκεια της τηλεπιμόρφωσης καθώς και τα συμπεράσματά τους για την υλοποίηση εξ αποστάσεως επιμορφώσεων γενικότερα. Έγινε αποτίμηση και επισκόπηση των απαιτούμενων εκπαιδευτικών πόρων στην εφαρμογή εξ αποστάσεως εκπαίδευσης. The aim of this program is to investigate the impact of distance learning on teachers training processes of education of primary and secondary teachers in active education. Τhe opinions of the Teachers were recorded, after their attendance of distance learning courses. The 4th PEKES of Central Macedonia in collaboration with the ITlab Computer Lab of the International Hellenic University in response to the demands of teachers for the continuation distance learning programs of last scholar year (2017-2018) realized four distance learning programs. The courses were developed on the OpeneClass asynchronous training platform, supported by the synchronous BigBlueButton training platform. Α questionnaire of closed-ended questions was answered through the platform in order to capture the opinions of the teachers for their experience of the online training. The required educational resources in distance learning implementation were evaluated and reviewed.