scholarly journals Dress like an Internet Celebrity: Fashion Retrieval in Videos

2020 ◽  
Author(s):  
Hongrui Zhao ◽  
Jin Yu ◽  
Yanan Li ◽  
Donghui Wang ◽  
Jie Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Large Scale ◽  
Deep Neural Network ◽  
Online Shopping ◽  
State Of The Art ◽  
The State ◽  
Video Sharing ◽  
Fashion Products ◽  
Cross Domain ◽  
Object Proposals

Nowadays, both online shopping and video sharing have grown exponentially. Although internet celebrities in videos are ideal exhibition for fashion corporations to sell their products, audiences do not always know where to buy fashion products in videos, which is a cross-domain problem called video-to-shop. In this paper, we propose a novel deep neural network, called Detect, Pick, and Retrieval Network (DPRNet), to break the gap between fashion products from videos and audiences. For the video side, we have modified the traditional object detector, which automatically picks out the best object proposals for every commodity in videos without duplication, to promote the performance of the video-to-shop task. For the fashion retrieval side, a simple but effective multi-task loss network obtains new state-of-the-art results on DeepFashion. Extensive experiments conducted on a new large-scale cross-domain video-to-shop dataset shows that DPRNet is efficient and outperforms the state-of-the-art methods on video-to-shop task.

Precise No-Reference Image Quality Evaluation Based on Distortion Identification

2021 ◽  
Vol 17 (3s) ◽  
pp. 1-21
Author(s):  
Chenggang Yan ◽  
Tong Teng ◽  
Yutao Liu ◽  
Yongbing Zhang ◽  
Haoqian Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Image Quality ◽  
Quality Assessment ◽  
Large Scale ◽  
Quality Evaluation ◽  
Image Quality Assessment ◽  
State Of The Art ◽  
Gaussian White Noise ◽  
The State ◽  
Reference Image

The difficulty of no-reference image quality assessment (NR IQA) often lies in the lack of knowledge about the distortion in the image, which makes quality assessment blind and thus inefficient. To tackle such issue, in this article, we propose a novel scheme for precise NR IQA, which includes two successive steps, i.e., distortion identification and targeted quality evaluation. In the first step, we employ the well-known Inception-ResNet-v2 neural network to train a classifier that classifies the possible distortion in the image into the four most common distortion types, i.e., Gaussian white noise (WN), Gaussian blur (GB), jpeg compression (JPEG), and jpeg2000 compression (JP2K). Specifically, the deep neural network is trained on the large-scale Waterloo Exploration database, which ensures the robustness and high performance of distortion classification. In the second step, after determining the distortion type of the image, we then design a specific approach to quantify the image distortion level, which can estimate the image quality specially and more precisely. Extensive experiments performed on LIVE, TID2013, CSIQ, and Waterloo Exploration databases demonstrate that (1) the accuracy of our distortion classification is higher than that of the state-of-the-art distortion classification methods, and (2) the proposed NR IQA method outperforms the state-of-the-art NR IQA methods in quantifying the image quality.

scholarly journals PATCHCOMM: Using Commonsense Knowledge to Guide Syntactic Parsers

2021 ◽  
Author(s):  
Yida Xin ◽  
Henry Lieberman ◽  
Peter Chin
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Knowledge Bases ◽  
General Purpose ◽  
The State ◽  
Prepositional Phrase ◽  
Syntactic Parsing ◽  
Commonsense Knowledge ◽  
Syntactic Ambiguities

Syntactic parsing technologies have become significantly more robust thanks to advancements in their underlying statistical and Deep Neural Network (DNN) techniques: most modern syntactic parsers can produce a syntactic parse tree for almost any sentence, including ones that may not be strictly grammatical. Despite improved robustness, such parsers still do not reflect the alternatives in parsing that are intrinsic in syntactic ambiguities. Two most notable such ambiguities are prepositional phrase (PP) attachment ambiguities and pronoun coreference ambiguities. In this paper, we discuss PatchComm, which uses commonsense knowledge to help resolve both kinds of ambiguities. To the best of our knowledge, we are the first to propose the general-purpose approach of using external commonsense knowledge bases to guide syntactic parsers. We evaluated PatchComm against the state-of-the-art (SOTA) spaCy parser on a PP attachment task and against the SOTA NeuralCoref module on a coreference task. Results show that PatchComm is successful at detecting syntactic ambiguities and using commonsense knowledge to help resolve them.

scholarly journals RNA-protein binding motifs mining with a new hybrid deep learning based cross-domain knowledge integration approach

2016 ◽  
Author(s):  
Xiaoyong Pan ◽  
Hong-Bin Shen
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Binding Sites ◽  
Domain Knowledge ◽  
Large Scale ◽  
Knowledge Integration ◽  
State Of The Art ◽  
Binding Motifs ◽  
Abstraction Level ◽  
Cross Domain

AbstractBackgroundRNAs play key roles in cells through the interactions with proteins known as the RNA-binding proteins (RBP) and their binding motifs enable crucial understanding of the post-transcriptional regulation of RNAs. How the RBPs correctly recognize the target RNAs and why they bind specific positions is still far from clear. Machine learning-based algorithms are widely acknowledged to be capable of speeding up this process. Although many automatic tools have been developed to predict the RNA-protein binding sites from the rapidly growing multi-resource data, e.g. sequence, structure, their domain specific features and formats have posed significant computational challenges. One of current difficulties is that the cross-source shared common knowledge is at a higher abstraction level beyond the observed data, resulting in a low efficiency of direct integration of observed data across domains. The other difficulty is how to interpret the prediction results. Existing approaches tend to terminate after outputting the potential discrete binding sites on the sequences, but how to assemble them into the meaningful binding motifs is a topic worth of further investigation.ResultsIn viewing of these challenges, we propose a deep learning-based framework (iDeep) by using a novel hybrid convolutional neural network and deep belief network to predict the RBP interaction sites and motifs on RNAs. This new protocol is featured by transforming the original observed data into a high-level abstraction feature space using multiple layers of learning blocks, where the shared representations across different domains are integrated. To validate our iDeep method, we performed experiments on 31 large-scale CLIP-seq datasets, and our results show that by integrating multiple sources of data, the average AUC can be improved by 8% compared to the best single-source-based predictor; and through cross-domain knowledge integration at an abstraction level, it outperforms the state-of-the-art predictors by 6%. Besides the overall enhanced prediction performance, the convolutional neural network module embedded in iDeep is also able to automatically capture the interpretable binding motifs for RBPs. Large-scale experiments demonstrate that these mined binding motifs agree well with the experimentally verified results, suggesting iDeep is a promising approach in the real-world applications.ConclusionThe iDeep framework not only can achieve promising performance than the state-of-the-art predictors, but also easily capture interpretable binding motifs. iDeep is available at http://www.csbio.sjtu.edu.cn/bioinf/iDeep

scholarly journals Abnormal electricity detection with hybrid deep neural network model

2018 ◽  
Vol 189 ◽  
pp. 03001
Author(s):  
Jie Liu ◽  
Xiang Cao ◽  
Diangang Wang ◽  
Kejia Pan ◽  
Cheng Zhang ◽  
...  
Keyword(s):  
Neural Network ◽  
Power Consumption ◽  
Large Scale ◽  
Deep Neural Network ◽  
Electricity Consumption ◽  
The State ◽  
Detection Model ◽  
Gradient Model ◽  
The Arts ◽  
The Difference

This paper tackles a new challenge in abnormal electricity detection: how to promptly detect stealing electricity behavior by a large-scale data from power users. Proposed scheme firstly forms power consumption gradient model by extracting daily trend indicators of electricity consumption, which can exactly reflect the short-term power consumption trend for each user. Furthermore, we design the line-losing model by analyzing the difference between power supplying and actual power consumption. Finally, a hybrid deep neural network detection model is built by combining with the power consumption gradient model and the line-losing model, which can quickly pin down to the abnormal electricity users. Comprehensive experiments are implemented by large-scale user samples from the State Grid Corporation and Tensorflow framework. Extensive results show that comparing with the state-of-the-arts, proposed scheme has a superior detection performance, and therefore is believed to be able to give a better guidance to abnormal electricity detection.

A Deep Neural Network Model for Cross-Domain Sentiment Analysis

2021 ◽  
Vol 12 (2) ◽  
pp. 1-16
Author(s):  
Suman Kumari ◽  
Basant Agarwal ◽  
Mamta Mittal
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Supervised Learning ◽  
Neural Network Model ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Analysis Model ◽  
Cross Domain ◽  
Unstructured Text

Sentiment analysis is used to detect the opinion/sentiment expressed from the unstructured text. Most of the existing state-of-the-art methods are based on supervised learning, and therefore, a labelled dataset is required to build the model, and it is very difficult task to obtain a labelled dataset for every domain. Cross-domain sentiment analysis is to develop a model which is trained on labelled dataset of one domain, and the performance is evaluated on another domain. The performance of such cross-domain sentiment analysis is still very limited due to presence of many domain-related terms, and the sentiment analysis is a domain-dependent problem in which words changes their polarity depending upon the domain. In addition, cross-domain sentiment analysis model suffers with the problem of large number of out-of-the-vocabulary (unseen words) words. In this paper, the authors propose a deep learning-based approach for cross-domain sentiment analysis. Experimental results show that the proposed approach improves the performance on the benchmark dataset.

scholarly journals SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

2021 ◽  
Vol 40 (3) ◽  
pp. 1-13
Author(s):  
Lumin Yang ◽  
Jiajie Zhuang ◽  
Hongbo Fu ◽  
Xiangzhi Wei ◽  
Kun Zhou ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Large Scale ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Structure Information ◽  
Graph Neural Networks ◽  
Node Labels ◽  
Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

scholarly journals Fast Accurate and Automatic Brushstroke Extraction

2021 ◽  
Vol 17 (2) ◽  
pp. 1-24
Author(s):  
Yunfei Fu ◽  
Hongchuan Yu ◽  
Chih-Kuo Yeh ◽  
Tong-Yee Lee ◽  
Jian J. Zhang
Keyword(s):  
Neural Network ◽  
Efficient Algorithm ◽  
Deep Neural Network ◽  
High Efficiency ◽  
State Of The Art ◽  
High Reliability ◽  
The Other ◽  
Manual Annotation ◽  
Stroke Extraction ◽  
Art Research

Brushstrokes are viewed as the artist’s “handwriting” in a painting. In many applications such as style learning and transfer, mimicking painting, and painting authentication, it is highly desired to quantitatively and accurately identify brushstroke characteristics from old masters’ pieces using computer programs. However, due to the nature of hundreds or thousands of intermingling brushstrokes in the painting, it still remains challenging. This article proposes an efficient algorithm for brush Stroke extraction based on a Deep neural network, i.e., DStroke. Compared to the state-of-the-art research, the main merit of the proposed DStroke is to automatically and rapidly extract brushstrokes from a painting without manual annotation, while accurately approximating the real brushstrokes with high reliability. Herein, recovering the faithful soft transitions between brushstrokes is often ignored by the other methods. In fact, the details of brushstrokes in a master piece of painting (e.g., shapes, colors, texture, overlaps) are highly desired by artists since they hold promise to enhance and extend the artists’ powers, just like microscopes extend biologists’ powers. To demonstrate the high efficiency of the proposed DStroke, we perform it on a set of real scans of paintings and a set of synthetic paintings, respectively. Experiments show that the proposed DStroke is noticeably faster and more accurate at identifying and extracting brushstrokes, outperforming the other methods.

scholarly journals ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition

2021 ◽  
Author(s):  
Anil S. Baslamisli ◽  
Partha Das ◽  
Hoang-An Le ◽  
Sezer Karaoglu ◽  
Theo Gevers
Keyword(s):  
Neural Network ◽  
Large Scale ◽  
State Of The Art ◽  
Image Decomposition ◽  
Natural Environments ◽  
Decomposition Algorithms ◽  
Ambient Light ◽  
Fine Grained ◽  
Large Scale Dataset ◽  
Direct Illumination

AbstractIn general, intrinsic image decomposition algorithms interpret shading as one unified component including all photometric effects. As shading transitions are generally smoother than reflectance (albedo) changes, these methods may fail in distinguishing strong photometric effects from reflectance variations. Therefore, in this paper, we propose to decompose the shading component into direct (illumination) and indirect shading (ambient light and shadows) subcomponents. The aim is to distinguish strong photometric effects from reflectance variations. An end-to-end deep convolutional neural network (ShadingNet) is proposed that operates in a fine-to-coarse manner with a specialized fusion and refinement unit exploiting the fine-grained shading model. It is designed to learn specific reflectance cues separated from specific photometric effects to analyze the disentanglement capability. A large-scale dataset of scene-level synthetic images of outdoor natural environments is provided with fine-grained intrinsic image ground-truths. Large scale experiments show that our approach using fine-grained shading decompositions outperforms state-of-the-art algorithms utilizing unified shading on NED, MPI Sintel, GTA V, IIW, MIT Intrinsic Images, 3DRMS and SRD datasets.

scholarly journals Large-scale Semantic Parsing without Question-Answer Pairs

2014 ◽  
Vol 2 ◽  
pp. 377-392 ◽  
Author(s):  
Siva Reddy ◽  
Mirella Lapata ◽  
Mark Steedman
Keyword(s):  
Natural Language ◽  
Large Scale ◽  
Graph Matching ◽  
State Of The Art ◽  
The State ◽  
Semantic Parsing ◽  
Matching Problem ◽  
Weak Supervision ◽  
Benchmark Datasets

In this paper we introduce a novel semantic parsing approach to query Freebase in natural language without requiring manual annotations or question-answer pairs. Our key insight is to represent natural language via semantic graphs whose topology shares many commonalities with Freebase. Given this representation, we conceptualize semantic parsing as a graph matching problem. Our model converts sentences to semantic graphs using CCG and subsequently grounds them to Freebase guided by denotations as a form of weak supervision. Evaluation experiments on a subset of the Free917 and WebQuestions benchmark datasets show our semantic parser improves over the state of the art.

Sign in / Sign up

Export Citation Format

Share Document