The information complexity of learning tasks, their structure and their distance

Abstract We introduce an asymmetric distance in the space of learning tasks and a framework to compute their complexity. These concepts are foundational for the practice of transfer learning, whereby a parametric model is pre-trained for a task, and then fine tuned for another. The framework we develop is non-asymptotic, captures the finite nature of the training dataset and allows distinguishing learning from memorization. It encompasses, as special cases, classical notions from Kolmogorov complexity and Shannon and Fisher information. However, unlike some of those frameworks, it can be applied to large-scale models and real-world datasets. Our framework is the first to measure complexity in a way that accounts for the effect of the optimization scheme, which is critical in deep learning.

Download Full-text

Discrete Binary Coding based Label Distribution Learning

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/518 ◽

2019 ◽

Author(s):

Ke Wang ◽

Xin Geng

Keyword(s):

Large Scale ◽

Head Pose Estimation ◽

Superior Performance ◽

Binary Codes ◽

Training Time ◽

Binary Coding ◽

Special Cases ◽

Label Distribution Learning ◽

Real World Datasets ◽

Label Distribution

Label Distribution Learning (LDL) is a general learning paradigm in machine learning, which includes both single-label learning (SLL) and multi-label learning (MLL) as its special cases. Recently, many LDL algorithms have been proposed to handle different application tasks such as facial age estimation, head pose estimation and visual sentiment distributions prediction. However, the training time complexity of most existing LDL algorithms is too high, which makes them unapplicable to large-scale LDL. In this paper, we propose a novel LDL method to address this issue, termed Discrete Binary Coding based Label Distribution Learning (DBC-LDL). Specifically, we design an efficiently discrete coding framework to learn binary codes for instances. Furthermore, both the pair-wise semantic similarities and the original label distributions are integrated into this framework to learn highly discriminative binary codes. In addition, a fast approximate nearest neighbor (ANN) search strategy is utilized to predict label distributions for testing instances. Experimental results on five real-world datasets demonstrate its superior performance over several state-of-the-art LDL methods with the lower time cost.

Download Full-text

Unsupervised Representation Learning by Predicting Random Distances

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/408 ◽

2020 ◽

Author(s):

Hu Wang ◽

Guansong Pang ◽

Chunhua Shen ◽

Congbo Ma

Keyword(s):

Neural Networks ◽

Anomaly Detection ◽

Unsupervised Learning ◽

Large Scale ◽

Deep Neural Networks ◽

State Of The Art ◽

Representation Learning ◽

Great Success ◽

Learning Tasks ◽

Real World Datasets

Deep neural networks have gained great success in a broad range of tasks due to its remarkable capability to learn semantically rich features from high-dimensional data. However, they often require large-scale labelled data to successfully learn such features, which significantly hinders their adaption in unsupervised learning tasks, such as anomaly detection and clustering, and limits their applications to critical domains where obtaining massive labelled data is prohibitively expensive. To enable unsupervised learning on those domains, in this work we propose to learn features without using any labelled data by training neural networks to predict data distances in a randomly projected space. Random mapping is a theoretically proven approach to obtain approximately preserved distances. To well predict these distances, the representation learner is optimised to learn genuine class structures that are implicitly embedded in the randomly projected space. Empirical results on 19 real-world datasets show that our learned representations substantially outperform a few state-of-the-art methods for both anomaly detection and clustering tasks. Code is available at: \url{https://git.io/RDP}

Download Full-text

Non-Blind Image Deconvolution Based on “Ringing” Removal Using Convolutional Neural Network

Electronic Imaging ◽

10.2352/issn.2470-1173.2020.10.ipas-180 ◽

2020 ◽

Vol 2020 (10) ◽

pp. 181-1-181-7

Author(s):

Takahiro Kudo ◽

Takanori Fujisawa ◽

Takuro Yamaguchi ◽

Masaaki Ikehara

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Network Architecture ◽

Large Scale ◽

Blind Deconvolution ◽

Training Dataset ◽

Image Deconvolution ◽

Classic Problem ◽

Key Points ◽

Blind Image

Image deconvolution has been an important issue recently. It has two kinds of approaches: non-blind and blind. Non-blind deconvolution is a classic problem of image deblurring, which assumes that the PSF is known and does not change universally in space. Recently, Convolutional Neural Network (CNN) has been used for non-blind deconvolution. Though CNNs can deal with complex changes for unknown images, some CNN-based conventional methods can only handle small PSFs and does not consider the use of large PSFs in the real world. In this paper we propose a non-blind deconvolution framework based on a CNN that can remove large scale ringing in a deblurred image. Our method has three key points. The first is that our network architecture is able to preserve both large and small features in the image. The second is that the training dataset is created to preserve the details. The third is that we extend the images to minimize the effects of large ringing on the image borders. In our experiments, we used three kinds of large PSFs and were able to observe high-precision results from our method both quantitatively and qualitatively.

Download Full-text

Comparison of the acoustic characteristics of large-scale models of several propulsive-lift concepts

10.2514/6.1974-1094 ◽

1974 ◽

Author(s):

M. FALARSKI ◽

T. AIKEN ◽

K. AOYAGI ◽

D. KOENIG

Keyword(s):

Large Scale ◽

Acoustic Characteristics ◽

Scale Models

Download Full-text

DeepSSPred: A Deep Learning Based Sulfenylation site predictor via a novel n-segmented optimize federated feature encoder

Protein and Peptide Letters ◽

10.2174/0929866527666201202103411 ◽

2020 ◽

Vol 27 ◽

Author(s):

Zaheer Ullah Khan ◽

Dechang Pi

Keyword(s):

Large Scale ◽

Computational Models ◽

Research Work ◽

Training Data ◽

Training Dataset ◽

Validation Dataset ◽

Cytokine Signaling ◽

Minority Class ◽

Independent Dataset ◽

Feature Encoding

Background: S-sulfenylation (S-sulphenylation, or sulfenic acid) proteins, are special kinds of post-translation modification, which plays an important role in various physiological and pathological processes such as cytokine signaling, transcriptional regulation, and apoptosis. Despite these aforementioned significances, and by complementing existing wet methods, several computational models have been developed for sulfenylation cysteine sites prediction. However, the performance of these models was not satisfactory due to inefficient feature schemes, severe imbalance issues, and lack of an intelligent learning engine. Objective: In this study, our motivation is to establish a strong and novel computational predictor for discrimination of sulfenylation and non-sulfenylation sites. Methods: In this study, we report an innovative bioinformatics feature encoding tool, named DeepSSPred, in which, resulting encoded features is obtained via n-segmented hybrid feature, and then the resampling technique called synthetic minority oversampling was employed to cope with the severe imbalance issue between SC-sites (minority class) and non-SC sites (majority class). State of the art 2DConvolutional Neural Network was employed over rigorous 10-fold jackknife cross-validation technique for model validation and authentication. Results: Following the proposed framework, with a strong discrete presentation of feature space, machine learning engine, and unbiased presentation of the underline training data yielded into an excellent model that outperforms with all existing established studies. The proposed approach is 6% higher in terms of MCC from the first best. On an independent dataset, the existing first best study failed to provide sufficient details. The model obtained an increase of 7.5% in accuracy, 1.22% in Sn, 12.91% in Sp and 13.12% in MCC on the training data and12.13% of ACC, 27.25% in Sn, 2.25% in Sp, and 30.37% in MCC on an independent dataset in comparison with 2nd best method. These empirical analyses show the superlative performance of the proposed model over both training and Independent dataset in comparison with existing literature studies. Conclusion : In this research, we have developed a novel sequence-based automated predictor for SC-sites, called DeepSSPred. The empirical simulations outcomes with a training dataset and independent validation dataset have revealed the efficacy of the proposed theoretical model. The good performance of DeepSSPred is due to several reasons, such as novel discriminative feature encoding schemes, SMOTE technique, and careful construction of the prediction model through the tuned 2D-CNN classifier. We believe that our research work will provide a potential insight into a further prediction of S-sulfenylation characteristics and functionalities. Thus, we hope that our developed predictor will significantly helpful for large scale discrimination of unknown SC-sites in particular and designing new pharmaceutical drugs in general.

Download Full-text

Leveraging Edge Intelligence for Video Analytics in Smart City Applications

Information ◽

10.3390/info12010014 ◽

2020 ◽

Vol 12 (1) ◽

pp. 14

Author(s):

Aluizio Rocha Neto ◽

Thiago P. Silva ◽

Thais Batista ◽

Flávia C. Delicato ◽

Paulo F. Pires ◽

...

Keyword(s):

Distributed System ◽

Smart City ◽

Large Scale ◽

Facial Recognition ◽

Public Spaces ◽

Hardware Accelerators ◽

Video Streams ◽

Video Analytics ◽

Learning Tasks ◽

Naive Approach

In smart city scenarios, the huge proliferation of monitoring cameras scattered in public spaces has posed many challenges to network and processing infrastructure. A few dozen cameras are enough to saturate the city’s backbone. In addition, most smart city applications require a real-time response from the system in charge of processing such large-scale video streams. Finding a missing person using facial recognition technology is one of these applications that require immediate action on the place where that person is. In this paper, we tackle these challenges presenting a distributed system for video analytics designed to leverage edge computing capabilities. Our approach encompasses architecture, methods, and algorithms for: (i) dividing the burdensome processing of large-scale video streams into various machine learning tasks; and (ii) deploying these tasks as a workflow of data processing in edge devices equipped with hardware accelerators for neural networks. We also propose the reuse of nodes running tasks shared by multiple applications, e.g., facial recognition, thus improving the system’s processing throughput. Simulations showed that, with our algorithm to distribute the workload, the time to process a workflow is about 33% faster than a naive approach.

Download Full-text

UVLens

Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies ◽

10.1145/3463495 ◽

2021 ◽

Vol 5 (2) ◽

pp. 1-26

Author(s):

Longbiao Chen ◽

Chenhui Lu ◽

Fangxu Yuan ◽

Zhihan Jiang ◽

Leye Wang ◽

...

Keyword(s):

Large Scale ◽

Floating Population ◽

Open Government ◽

Residential Areas ◽

Rapid Urbanization ◽

Urban Village ◽

Open Government Data ◽

Government Data ◽

Real World Datasets ◽

Urban Villages

Urban villages refer to the residential areas lagging behind the rapid urbanization process in many developing countries. These areas are usually with overcrowded buildings, high population density, and low living standards, bringing potential risks of public safety and hindering the urban development. Therefore, it is crucial for urban authorities to identify the boundaries of urban villages and estimate their resident and floating populations so as to better renovate and manage these areas. Traditional approaches, such as field surveys and demographic census, are time consuming and labor intensive, lacking a comprehensive understanding of urban villages. Against this background, we propose a two-phase framework for urban village boundary identification and population estimation. Specifically, based on heterogeneous open government data, the proposed framework can not only accurately identify the boundaries of urban villages from large-scale satellite imagery by fusing road networks guided patches with bike-sharing drop-off patterns, but also accurately estimate the resident and floating populations of urban villages with a proposed multi-view neural network model. We evaluate our method leveraging real-world datasets collected from Xiamen Island. Results show that our framework can accurately identify the urban village boundaries with an IoU of 0.827, and estimate the resident population and floating population with R2 of 0.92 and 0.94 respectively, outperforming the baseline methods. We also deploy our system on the Xiamen Open Government Data Platform to provide services to both urban authorities and citizens.

Download Full-text

Cyclic Breathing Simulations in Large Scale Models of the Lung Airway From the Oronasal Opening to the Terminal Bronchioles

Volume 2: Biomedical and Biotechnology ◽

10.1115/imece2012-88901 ◽

2012 ◽

Author(s):

D. Keith Walters ◽

Greg W. Burgreen ◽

Robert L. Hester ◽

David S. Thompson ◽

David M. Lavallee ◽

...

Keyword(s):

Steady State ◽

Boundary Conditions ◽

Large Scale ◽

Whole Body ◽

Patient Specific ◽

Cfd Simulations ◽

Reduced Order ◽

Scale Models ◽

Steady State Simulation ◽

Conducting Zone

Computational fluid dynamics (CFD) simulations were performed for unsteady periodic breathing conditions, using large-scale models of the human lung airway. The computational domain included fully coupled representations of the orotracheal region and large conducting zone up to generation four (G4) obtained from patient-specific CT data, and the small conducting zone (to G16) obtained from a stochastically generated airway tree with statistically realistic geometrical characteristics. A reduced-order geometry was used, in which several airway branches in each generation were truncated, and only select flow paths were retained to G16. The inlet and outlet flow boundaries corresponded to the oronasal opening (superior), the inlet/outlet planes in terminal bronchioles (distal), and the unresolved airway boundaries arising from the truncation procedure (intermediate). The cyclic flow was specified according to the predicted ventilation patterns for a healthy adult male at three different activity levels, supplied by the whole-body modeling software HumMod. The CFD simulations were performed using Ansys FLUENT. The mass flow distribution at the distal boundaries was prescribed using a previously documented methodology, in which the percentage of the total flow for each boundary was first determined from a steady-state simulation with an applied flow rate equal to the average during the inhalation phase of the breathing cycle. The distal pressure boundary conditions for the steady-state simulation were set using a stochastic coupling procedure to ensure physiologically realistic flow conditions. The results show that: 1) physiologically realistic flow is obtained in the model, in terms of cyclic mass conservation and approximately uniform pressure distribution in the distal airways; 2) the predicted alveolar pressure is in good agreement with previously documented values; and 3) the use of reduced-order geometry modeling allows accurate and efficient simulation of large-scale breathing lung flow, provided care is taken to use a physiologically realistic geometry and to properly address the unsteady boundary conditions.

Download Full-text