scholarly journals Deep Learning at Alibaba

2017 ◽  
Author(s):  
Rong Jin
Keyword(s):  
Deep Learning ◽  
Combinatorial Optimization ◽  
Large Scale ◽  
Packing Problem ◽  
Learning Technologies ◽  
Optimization Techniques ◽  
Multimedia Data ◽  
Shipping Industry ◽  
Model Compression ◽  
New Directions

In this talk, I will focus on the applications and the latest development of deep learning technologies at Alibaba. More specifically, I will discuss (a) how to handle high dimensional data in DNN and its application to recommender system, (b) the development of deep learning models for transfer learning and its application to multimedia data analysis, (c) the development of combinatorial optimization techniques for DNN model compression and its application to large-scale image classification, and (d) the exploration of deep learning technique for combinatorial optimization and its application to the packing problem in shipping industry. I will conclude my talk with a discussion of new directions for deep learning that are under development at Alibaba.

scholarly journals A Deep Learning Algorithm for the Max-Cut Problem Based on Pointer Network Structure with Supervised Learning and Reinforcement Learning Strategies

Mathematics ◽  
2020 ◽  
Vol 8 (2) ◽  
pp. 298 ◽  
Author(s):  
Shenshen Gu ◽  
Yue Yang
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Reinforcement Learning ◽  
Combinatorial Optimization ◽  
Supervised Learning ◽  
Learning Strategies ◽  
Large Scale ◽  
Deep Neural Network ◽  
Max Cut Problem ◽  
Cut Problems

The Max-cut problem is a well-known combinatorial optimization problem, which has many real-world applications. However, the problem has been proven to be non-deterministic polynomial-hard (NP-hard), which means that exact solution algorithms are not suitable for large-scale situations, as it is too time-consuming to obtain a solution. Therefore, designing heuristic algorithms is a promising but challenging direction to effectively solve large-scale Max-cut problems. For this reason, we propose a unique method which combines a pointer network and two deep learning strategies (supervised learning and reinforcement learning) in this paper, in order to address this challenge. A pointer network is a sequence-to-sequence deep neural network, which can extract data features in a purely data-driven way to discover the hidden laws behind data. Combining the characteristics of the Max-cut problem, we designed the input and output mechanisms of the pointer network model, and we used supervised learning and reinforcement learning to train the model to evaluate the model performance. Through experiments, we illustrated that our model can be well applied to solve large-scale Max-cut problems. Our experimental results also revealed that the new method will further encourage broader exploration of deep neural network for large-scale combinatorial optimization problems.

scholarly journals Triplet Deep Hashing with Joint Supervised Loss Based on Deep Neural Networks

2019 ◽  
Vol 2019 ◽  
pp. 1-17 ◽  
Author(s):  
Mingyong Li ◽  
Ziye An ◽  
Qinmin Wei ◽  
Kaiyue Xiang ◽  
Yan Ma
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Large Scale ◽  
Selection Method ◽  
Image Features ◽  
Multimedia Data ◽  
Image Feature ◽  
Great Success ◽  
Deep Hashing ◽  
Hash Code

In recent years, with the explosion of multimedia data from search engines, social media, and e-commerce platforms, there is an urgent need for fast retrieval methods for massive big data. Hashing is widely used in large-scale and high-dimensional data search because of its low storage cost and fast query speed. Thanks to the great success of deep learning in many fields, the deep learning method has been introduced into hashing retrieval, and it uses a deep neural network to learn image features and hash codes simultaneously. Compared with the traditional hashing methods, it has better performance. However, existing deep hashing methods have some limitations; for example, most methods consider only one kind of supervised loss, which leads to insufficient utilization of supervised information. To address this issue, we proposed a triplet deep hashing method with joint supervised loss based on the convolutional neural network (JLTDH) in this work. The proposed method JLTDH combines triplet likelihood loss and linear classification loss; moreover, the triplet supervised label is adopted, which contains richer supervised information than that of the pointwise and pairwise labels. At the same time, in order to overcome the cubic increase in the number of triplets and make triplet training more effective, we adopt a novel triplet selection method. The whole process is divided into two stages: In the first stage, taking the triplets generated by the triplet selection method as the input of the CNN, the three CNNs with shared weights are used for image feature learning, and the last layer of the network outputs a preliminary hash code. In the second stage, relying on the hash code of the first stage and the joint loss function, the neural network model is further optimized so that the generated hash code has higher query precision. We perform extensive experiments on the three public benchmark datasets CIFAR-10, NUS-WIDE, and MS-COCO. Experimental results demonstrate that the proposed method outperforms the compared methods, and the method is also superior to all previous deep hashing methods based on the triplet label.

scholarly journals Evaluating Optimal Differentially Private Learning - Shallow and Deep Techniques

2020 ◽  
Vol 9 (10) ◽  
pp. 181-187
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Data Analytics ◽  
Large Scale ◽  
Differential Privacy ◽  
Personal Information ◽  
Optimization Techniques ◽  
Single Individual ◽  
Activity Data ◽  
Online Activity

Data analytics is an evolving arena in today’s technological evolution. Big data, IoT and machine learning are multidisciplinary fields which pave way for large scale data analytics. Data is the basic ingredient in all type of analytical tasks, which is collected from various sources through online activity. Data divulged in these day-to-day activities contain personal information of individuals. These sensitive details may be disclosed when data is shared with data analysts or researchers for futuristic analysis. In order to respect the privacy of individuals involved, it is required to protect data to avoid any intentional harm. Differential privacy is an algorithm that allows controlled machine learning practices for quality analytics. With differential privacy, the outcome of any analytical task is unaffected by the presence or absence of a single individual or small group of individuals. But, it goes without saying that privacy protection diminishes the usefulness of data for analysis. Hence privacy preserving analytics requires algorithmic techniques that can handle privacy, data quality and efficiency simultaneously. Since one cannot be obtained without degrading the other, an optimal solution that balances the attributes is considered acceptable. The work in this paper, proposes different optimization techniques for shallow and deep learners. While evolutionary approach is proposed for shallow learning, private deep learning is optimized using Bayesian method. The results prove that the Bayesian optimized private deep learning model gives a quantifiable trade-off between the privacy, utility and performance.

Automatic Video Event Detection for Imbalance Data Using Enhanced Ensemble Deep Learning

2017 ◽  
Vol 11 (01) ◽  
pp. 85-109 ◽  
Author(s):  
Samira Pouyanfar ◽  
Shu-Ching Chen
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Event Detection ◽  
Large Scale ◽  
Performance Metrics ◽  
Imbalanced Data ◽  
Multimedia Data ◽  
Support Vector ◽  
Svm Classifier ◽  
Imbalance Problem

With the explosion of multimedia data, semantic event detection from videos has become a demanding and challenging topic. In addition, when the data has a skewed data distribution, interesting event detection also needs to address the data imbalance problem. The recent proliferation of deep learning has made it an essential part of many Artificial Intelligence (AI) systems. Till now, various deep learning architectures have been proposed for numerous applications such as Natural Language Processing (NLP) and image processing. Nonetheless, it is still impracticable for a single model to work well for different applications. Hence, in this paper, a new ensemble deep learning framework is proposed which can be utilized in various scenarios and datasets. The proposed framework is able to handle the over-fitting issue as well as the information losses caused by single models. Moreover, it alleviates the imbalanced data problem in real-world multimedia data. The whole framework includes a suite of deep learning feature extractors integrated with an enhanced ensemble algorithm based on the performance metrics for the imbalanced data. The Support Vector Machine (SVM) classifier is utilized as the last layer of each deep learning component and also as the weak learners in the ensemble module. The framework is evaluated on two large-scale and imbalanced video datasets (namely, disaster and TRECVID). The extensive experimental results illustrate the advantage and effectiveness of the proposed framework. It also demonstrates that the proposed framework outperforms several well-known deep learning methods, as well as the conventional features integrated with different classifiers.

scholarly journals Accurate Suitability Evaluation of Large-Scale Roof Greening Based on RS and GIS Methods

2020 ◽  
Vol 12 (11) ◽  
pp. 4375
Author(s):  
Nan Xu ◽  
Jiancheng Luo ◽  
Jin Zuo ◽  
Xiaodong Hu ◽  
Jing Dong ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Green Space ◽  
Learning Technologies ◽  
Pollution Level ◽  
Suitability Evaluation ◽  
Order Preference ◽  
Learning Machine ◽  
Gis Methods ◽  
Roof Greening

Under increasingly low urban land resources, carrying out roof greening to exploit new green space is a good strategy for sustainable development. Therefore, it is necessary to evaluate the suitability of roof greening for buildings in cities. However, most current evaluation methods are based on qualitative and conceptual research. In this paper, a methodological framework for roof greening suitability evaluation is proposed based on the basic units of building roofs extracted via deep learning technologies. The building, environmental and social criteria related to roof greening are extracted using technologies such as deep learning, machine learning, remote sensing (RS) methods and geographic information system (GIS) methods. The technique for order preference by similarity to an ideal solution (TOPSIS) method is applied to quantify the suitability of each roof, and Sobol sensitivity analysis of the score results is conducted. The experiment on Xiamen Island shows that the final evaluation results are highly sensitive to the changes in weight of the green space distance, population density and the air pollution level. This framework is helpful for the quantitative and objective development of roof greening suitability evaluation.

scholarly journals Advances in Deep Learning-Based Medical Image Analysis

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Xiaoqing Liu ◽  
Kunlun Gao ◽  
Bo Liu ◽  
Chengwei Pan ◽  
Kongming Liang ◽  
...  
Keyword(s):  
Image Analysis ◽  
Deep Learning ◽  
Medical Image ◽  
Large Scale ◽  
Medical Image Analysis ◽  
Research Area ◽  
Clinical Applications ◽  
Learning Technologies ◽  
Small Scale ◽  
Great Success

Importance. With the booming growth of artificial intelligence (AI), especially the recent advancements of deep learning, utilizing advanced deep learning-based methods for medical image analysis has become an active research area both in medical industry and academia. This paper reviewed the recent progress of deep learning research in medical image analysis and clinical applications. It also discussed the existing problems in the field and provided possible solutions and future directions. Highlights. This paper reviewed the advancement of convolutional neural network-based techniques in clinical applications. More specifically, state-of-the-art clinical applications include four major human body systems: the nervous system, the cardiovascular system, the digestive system, and the skeletal system. Overall, according to the best available evidence, deep learning models performed well in medical image analysis, but what cannot be ignored are the algorithms derived from small-scale medical datasets impeding the clinical applicability. Future direction could include federated learning, benchmark dataset collection, and utilizing domain subject knowledge as priors. Conclusion. Recent advanced deep learning technologies have achieved great success in medical image analysis with high accuracy, efficiency, stability, and scalability. Technological advancements that can alleviate the high demands on high-quality large-scale datasets could be one of the future developments in this area.

scholarly journals Social Multimedia Security and Suspicious Activity Detection in SDN using Hybrid Deep Learning Technique

2020 ◽  
Vol 2 (2) ◽  
pp. 108-115
Author(s):  
Dr. Joy Iong Zong Chen ◽  
Dr. Smys S.
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
Large Scale ◽  
Multimedia Data ◽  
Support Vector ◽  
Data Delivery ◽  
Energy Aware ◽  
Detection Scheme ◽  
Social Multimedia

Social multimedia traffic is growing exponentially with the increased usage and continuous development of services and applications based on multimedia. Quality of Service (QoS), Quality of Information (QoI), scalability, reliability and such factors that are essential for social multimedia networks are realized by secure data transmission. For delivering actionable and timely insights in order to meet the growing demands of the user, multimedia analytics is performed by means of a trust-based paradigm. Efficient management and control of the network is facilitated by limiting certain capabilities such as energy-aware networking and runtime security in Software Defined Networks. In social multimedia context, suspicious flow detection is performed by a hybrid deep learning based anomaly detection scheme in order to enhance the SDN reliability. The entire process is divided into two modules namely – Abnormal activities detection using support vector machine based on Gradient descent and improved restricted Boltzmann machine which facilitates the anomaly detection module, and satisfying the strict requirements of QoS like low latency and high bandwidth in SDN using end-to-end data delivery module. In social multimedia, data delivery and anomaly detection services are essential in order to improve the efficiency and effectiveness of the system. For this purpose, we use benchmark datasets as well as real time evaluation to experimentally evaluate the proposed scheme. Detection of malicious events like confidential data collection, profile cloning and identity theft are performed to analyze the performance of the system using CMU-based insider threat dataset for large scale analysis.

scholarly journals Artificial Intelligence & Machine Learning in Computer Vision Applications

2020 ◽  
Vol 7 (1) ◽  
pp. 2-3
Author(s):  
Shadi Saleh
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Neural Networks ◽  
Computer Vision ◽  
Deep Learning ◽  
Large Scale ◽  
Short Term Memory ◽  
Graphics Processing Unit ◽  
Multimedia Data ◽  
Processing Unit

Deep learning and machine learning innovations are at the core of the ongoing revolution in Artificial Intelligence for the interpretation and analysis of multimedia data. The convergence of large-scale datasets and more affordable Graphics Processing Unit (GPU) hardware has enabled the development of neural networks for data analysis problems that were previously handled by traditional handcrafted features. Several deep learning architectures such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short Term Memory (LSTM)/Gated Recurrent Unit (GRU), Deep Believe Networks (DBN), and Deep Stacking Networks (DSNs) have been used with new open source software and libraries options to shape an entirely new scenario in computer vision processing.

Multi Disease-Prediction Framework Using Hybrid Deep Learning: An Optimal Prediction Model (Preprint)

2020 ◽  
Author(s):  
Anusha Ampavathi ◽  
Vijaya Saradhi T
Keyword(s):  
Feature Extraction ◽  
Big Data ◽  
Deep Learning ◽  
Weight Function ◽  
Optimization Algorithm ◽  
Large Scale ◽  
Heuristic Algorithms ◽  
Disease Prediction ◽  
Health Care Decisions ◽  
Proposed Model

UNSTRUCTURED Big data and its approaches are generally helpful for healthcare and biomedical sectors for predicting the disease. For trivial symptoms, the difficulty is to meet the doctors at any time in the hospital. Thus, big data provides essential data regarding the diseases on the basis of the patient’s symptoms. For several medical organizations, disease prediction is important for making the best feasible health care decisions. Conversely, the conventional medical care model offers input as structured that requires more accurate and consistent prediction. This paper is planned to develop the multi-disease prediction using the improvised deep learning concept. Here, the different datasets pertain to “Diabetes, Hepatitis, lung cancer, liver tumor, heart disease, Parkinson’s disease, and Alzheimer’s disease”, from the benchmark UCI repository is gathered for conducting the experiment. The proposed model involves three phases (a) Data normalization (b) Weighted normalized feature extraction, and (c) prediction. Initially, the dataset is normalized in order to make the attribute's range at a certain level. Further, weighted feature extraction is performed, in which a weight function is multiplied with each attribute value for making large scale deviation. Here, the weight function is optimized using the combination of two meta-heuristic algorithms termed as Jaya Algorithm-based Multi-Verse Optimization algorithm (JA-MVO). The optimally extracted features are subjected to the hybrid deep learning algorithms like “Deep Belief Network (DBN) and Recurrent Neural Network (RNN)”. As a modification to hybrid deep learning architecture, the weight of both DBN and RNN is optimized using the same hybrid optimization algorithm. Further, the comparative evaluation of the proposed prediction over the existing models certifies its effectiveness through various performance measures.

Sign in / Sign up

Export Citation Format

Share Document