scholarly journals Context-Aware Image Inpainting with Learned Semantic Priors

2021 ◽  
Author(s):  
Wendong Zhang ◽  
Junwei Zhu ◽  
Ying Tai ◽  
Yunbo Wang ◽  
Wenqing Chu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Contextual Information ◽  
Image Inpainting ◽  
Context Aware ◽  
Global Context ◽  
Low Level ◽  
Complex Scenes ◽  
Knowledge Distillation ◽  
Image Generator ◽  
High Level

Recent advances in image inpainting have shown impressive results for generating plausible visual details on rather simple backgrounds. However, for complex scenes, it is still challenging to restore reasonable contents as the contextual information within the missing regions tends to be ambiguous. To tackle this problem, we introduce pretext tasks that are semantically meaningful to estimating the missing contents. In particular, we perform knowledge distillation on pretext models and adapt the features to image inpainting. The learned semantic priors ought to be partially invariant between the high-level pretext task and low-level image inpainting, which not only help to understand the global context but also provide structural guidance for the restoration of local textures. Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator. The semantic learner and the image generator are trained in an end-to-end manner. We name the model SPL to highlight its ability to learn and leverage semantic priors. It achieves the state of the art on Places2, CelebA, and Paris StreetView datasets

scholarly journals Context-aware Cross-level Fusion Network for Camouflaged Object Detection

2021 ◽  
Author(s):  
Yujia Sun ◽  
Geng Chen ◽  
Tao Zhou ◽  
Yi Zhang ◽  
Nian Liu
Keyword(s):  
Object Detection ◽  
State Of The Art ◽  
Context Aware ◽  
Global Context ◽  
Feature Representations ◽  
Multi Scale ◽  
Benchmark Datasets ◽  
Multi Level ◽  
High Level ◽  
Level Fusion

Camouflaged object detection (COD) is a challenging task due to the low boundary contrast between the object and its surroundings. In addition, the appearance of camouflaged objects varies significantly, e.g., object size and shape, aggravating the difficulties of accurate COD. In this paper, we propose a novel Context-aware Cross-level Fusion Network (C2F-Net) to address the challenging COD task. Specifically, we propose an Attention-induced Cross-level Fusion Module (ACFM) to integrate the multi-level features with informative attention coefficients. The fused features are then fed to the proposed Dual-branch Global Context Module (DGCM), which yields multi-scale feature representations for exploiting rich global context information. In C2F-Net, the two modules are conducted on high-level features using a cascaded manner. Extensive experiments on three widely used benchmark datasets demonstrate that our C2F-Net is an effective COD model and outperforms state-of-the-art models remarkably. Our code is publicly available at: https://github.com/thograce/C2FNet.

A service-oriented context-awareness reasoning framework and its implementation

2018 ◽  
Vol 36 (6) ◽  
pp. 1114-1134 ◽  
Author(s):  
Xiufeng Cheng ◽  
Jinqing Yang ◽  
Lixin Xia
Keyword(s):  
Context Awareness ◽  
Service Providers ◽  
User Behavior ◽  
Contextual Information ◽  
Sensor Data ◽  
Context Information ◽  
Content Type ◽  
Low Level ◽  
Service Oriented ◽  
High Level

PurposeThis paper aims to propose an extensible, service-oriented framework for context-aware data acquisition, description, interpretation and reasoning, which facilitates the development of mobile applications that provide a context-awareness service.Design/methodology/approachFirst, the authors propose the context data reasoning framework (CDRFM) for generating service-oriented contextual information. Then they used this framework to composite mobile sensor data into low-level contextual information. Finally, the authors exploited some high-level contextual information that can be inferred from the formatted low-level contextual information using particular inference rules.FindingsThe authors take “user behavior patterns” as an exemplary context information generation schema in their experimental study. The results reveal that the optimization of service can be guided by the implicit, high-level context information inside user behavior logs. They also prove the validity of the authors’ framework.Research limitations/implicationsFurther research will add more variety of sensor data. Furthermore, to validate the effectiveness of our framework, more reasoning rules need to be performed. Therefore, the authors may implement more algorithms in the framework to acquire more comprehensive context information.Practical implicationsCDRFM expands the context-awareness framework of previous research and unifies the procedures of acquiring, describing, modeling, reasoning and discovering implicit context information for mobile service providers.Social implicationsSupport the service-oriented context-awareness function in application design and related development in commercial mobile software industry.Originality/valueExtant researches on context awareness rarely considered the generation contextual information for service providers. The CDRFM can be used to generate valuable contextual information by implementing more reasoning rules.

scholarly journals Saliency Detection by Multilevel Deep Pyramid Model

2018 ◽  
Vol 2018 ◽  
pp. 1-11 ◽  
Author(s):  
Hai Wang ◽  
Lei Dai ◽  
Yingfeng Cai ◽  
Long Chen ◽  
Yong Zhang
Keyword(s):  
Background Noise ◽  
State Of The Art ◽  
Saliency Detection ◽  
Saliency Map ◽  
Multiple Features ◽  
Low Level ◽  
Pyramid Model ◽  
High Level ◽  
Different Levels ◽  
Better Than

Traditional salient object detection models are divided into several classes based on low-level features and contrast between pixels. In this paper, we propose a model based on a multilevel deep pyramid (MLDP), which involves fusing multiple features on different levels. Firstly, the MLDP uses the original image as the input for a VGG16 model to extract high-level features and form an initial saliency map. Next, the MLDP further extracts high-level features to form a saliency map based on a deep pyramid. Then, the MLDP obtains the salient map fused with superpixels by extracting low-level features. After that, the MLDP applies background noise filtering to the saliency map fused with superpixels in order to filter out the interference of background noise and form a saliency map based on the foreground. Lastly, the MLDP combines the saliency map fused with the superpixels with the saliency map based on the foreground, which results in the final saliency map. The MLDP is not limited to low-level features while it fuses multiple features and achieves good results when extracting salient targets. As can be seen in our experiment section, the MLDP is better than the other 7 state-of-the-art models across three different public saliency datasets. Therefore, the MLDP has superiority and wide applicability in extraction of salient targets.

Y-Net: Dual-branch Joint Network for Semantic Segmentation

2021 ◽  
Vol 17 (4) ◽  
pp. 1-22
Author(s):  
Yizhen Chen ◽  
Haifeng Hu
Keyword(s):  
Feature Vector ◽  
State Of The Art ◽  
Computational Cost ◽  
Receptive Fields ◽  
Semantic Segmentation ◽  
Global Context ◽  
Multi Level ◽  
The One ◽  
Public Datasets ◽  
High Level

Most existing segmentation networks are built upon a “ U -shaped” encoder–decoder structure, where the multi-level features extracted by the encoder are gradually aggregated by the decoder. Although this structure has been proven to be effective in improving segmentation performance, there are two main drawbacks. On the one hand, the introduction of low-level features brings a significant increase in calculations without an obvious performance gain. On the other hand, general strategies of feature aggregation such as addition and concatenation fuse features without considering the usefulness of each feature vector, which mixes the useful information with massive noises. In this article, we abandon the traditional “ U -shaped” architecture and propose Y-Net, a dual-branch joint network for accurate semantic segmentation. Specifically, it only aggregates the high-level features with low-resolution and utilizes the global context guidance generated by the first branch to refine the second branch. The dual branches are effectively connected through a Semantic Enhancing Module, which can be regarded as the combination of spatial attention and channel attention. We also design a novel Channel-Selective Decoder (CSD) to adaptively integrate features from different receptive fields by assigning specific channelwise weights, where the weights are input-dependent. Our Y-Net is capable of breaking through the limit of singe-branch network and attaining higher performance with less computational cost than “ U -shaped” structure. The proposed CSD can better integrate useful information and suppress interference noises. Comprehensive experiments are carried out on three public datasets to evaluate the effectiveness of our method. Eventually, our Y-Net achieves state-of-the-art performance on PASCAL VOC 2012, PASCAL Person-Part, and ADE20K dataset without pre-training on extra datasets.

scholarly journals Global Context-Aware Progressive Aggregation Network for Salient Object Detection

2020 ◽  
Vol 34 (07) ◽  
pp. 10599-10606 ◽  
Author(s):  
Zuyao Chen ◽  
Qianqian Xu ◽  
Runmin Cong ◽  
Qingming Huang
Keyword(s):  
Object Detection ◽  
Critical Role ◽  
Saliency Map ◽  
Salient Object Detection ◽  
Semantic Features ◽  
Context Aware ◽  
Salient Object ◽  
Deep Convolutional Neural Networks ◽  
Global Context ◽  
High Level

Deep convolutional neural networks have achieved competitive performance in salient object detection, in which how to learn effective and comprehensive features plays a critical role. Most of the previous works mainly adopted multiple-level feature integration yet ignored the gap between different features. Besides, there also exists a dilution process of high-level features as they passed on the top-down pathway. To remedy these issues, we propose a novel network named GCPANet to effectively integrate low-level appearance features, high-level semantic features, and global context features through some progressive context-aware Feature Interweaved Aggregation (FIA) modules and generate the saliency map in a supervised way. Moreover, a Head Attention (HA) module is used to reduce information redundancy and enhance the top layers features by leveraging the spatial and channel-wise attention, and the Self Refinement (SR) module is utilized to further refine and heighten the input features. Furthermore, we design the Global Context Flow (GCF) module to generate the global context information at different stages, which aims to learn the relationship among different salient regions and alleviate the dilution effect of high-level features. Experimental results on six benchmark datasets demonstrate that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively.

scholarly journals Backbone Cannot Be Trained at Once: Rolling Back to Pre-Trained Network for Person Re-Identification

2019 ◽  
Vol 33 ◽  
pp. 8859-8867 ◽  
Author(s):  
Youngmin Ro ◽  
Jongwon Choi ◽  
Dae Ung Jo ◽  
Byeongho Heo ◽  
Jongin Lim ◽  
...  
Keyword(s):  
Network Architecture ◽  
State Of The Art ◽  
Fine Tuning ◽  
Neural Network Architecture ◽  
Large Dataset ◽  
Low Level ◽  
Tuning Method ◽  
Improved Performance ◽  
High Level ◽  
Tuning Strategy

In person re-identification (ReID) task, because of its shortage of trainable dataset, it is common to utilize fine-tuning method using a classification network pre-trained on a large dataset. However, it is relatively difficult to sufficiently finetune the low-level layers of the network due to the gradient vanishing problem. In this work, we propose a novel fine-tuning strategy that allows low-level layers to be sufficiently trained by rolling back the weights of high-level layers to their initial pre-trained weights. Our strategy alleviates the problem of gradient vanishing in low-level layers and robustly trains the low-level layers to fit the ReID dataset, thereby increasing the performance of ReID tasks. The improved performance of the proposed strategy is validated via several experiments. Furthermore, without any addons such as pose estimation or segmentation, our strategy exhibits state-of-the-art performance using only vanilla deep convolutional neural network architecture.

scholarly journals Deep Convolutional Neural Network for Pedestrian Detection with Multi-Levels Features Fusion

2018 ◽  
Vol 232 ◽  
pp. 01061
Author(s):  
Danhua Li ◽  
Xiaofeng Di ◽  
Xuan Qu ◽  
Yunfei Zhao ◽  
Honggang Kong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Deep Convolutional Neural Network ◽  
High Quality ◽  
Low Level ◽  
Features Fusion ◽  
Current State ◽  
High Level

Pedestrian detection aims to localize and recognize every pedestrian instance in an image with a bounding box. The current state-of-the-art method is Faster RCNN, which is such a network that uses a region proposal network (RPN) to generate high quality region proposals, while Fast RCNN is used to classifiers extract features into corresponding categories. The contribution of this paper is integrated low-level features and high-level features into a Faster RCNN-based pedestrian detection framework, which efficiently increase the capacity of the feature. Through our experiments, we comprehensively evaluate our framework, on the Caltech pedestrian detection benchmark and our methods achieve state-of-the-art accuracy and present a competitive result on Caltech dataset.

ConChi

2016 ◽  
pp. 798-820
Author(s):  
Luca Cagliero
Keyword(s):  
Contextual Information ◽  
Extraction Process ◽  
Mobile User ◽  
Mobile Service ◽  
Context Aware ◽  
Time Periods ◽  
Context Aware System ◽  
Mobile Context ◽  
High Level ◽  
Abstraction Levels

Mobile context-aware systems focus on adapting mobile service provisions to the actual user needs. They offer personalized services based on the context in which mobile users' requests have been submitted. Since contextual information changes over time, the application of established itemset change mining algorithms to context-aware data is an appealing research issue. Change itemset discovery focuses on discovering patterns which represent the temporal evolution of frequent itemsets in consecutive time periods. However, the sparseness of the analyzed data may bias the extraction process, because itemsets are likely to become infrequent at certain time periods. This chapter presents ConChI, a novel context-aware system that performs change itemset mining from context-aware data with the aim at supporting mobile expert decisions. To counteract data sparseness itemset change mining is driven by an analyst-provided taxonomy which allows analyzing data correlation changes at different abstraction levels. In particular, taxonomy is exploited to represent the knowledge that becomes infrequent in certain time periods by means of high level (generalized) itemsets. Experiments performed on real contextual data coming from a mobile application show the effectiveness of the proposed system in supporting mobile user and service profiling.

ConChi

2014 ◽  
pp. 263-285
Author(s):  
Luca Cagliero
Keyword(s):  
Contextual Information ◽  
Extraction Process ◽  
Mobile User ◽  
Mobile Service ◽  
Context Aware ◽  
Time Periods ◽  
Context Aware System ◽  
Mobile Context ◽  
High Level ◽  
Abstraction Levels

Mobile context-aware systems focus on adapting mobile service provisions to the actual user needs. They offer personalized services based on the context in which mobile users’ requests have been submitted. Since contextual information changes over time, the application of established itemset change mining algorithms to context-aware data is an appealing research issue. Change itemset discovery focuses on discovering patterns which represent the temporal evolution of frequent itemsets in consecutive time periods. However, the sparseness of the analyzed data may bias the extraction process, because itemsets are likely to become infrequent at certain time periods. This chapter presents ConChI, a novel context-aware system that performs change itemset mining from context-aware data with the aim at supporting mobile expert decisions. To counteract data sparseness itemset change mining is driven by an analyst-provided taxonomy which allows analyzing data correlation changes at different abstraction levels. In particular, taxonomy is exploited to represent the knowledge that becomes infrequent in certain time periods by means of high level (generalized) itemsets. Experiments performed on real contextual data coming from a mobile application show the effectiveness of the proposed system in supporting mobile user and service profiling.

Representing and reasoning with situations for context-aware pervasive computing: a logic programming perspective

2004 ◽  
Vol 19 (3) ◽  
pp. 213-233 ◽  
Author(s):  
SENG W LOKE
Keyword(s):  
Logic Programming ◽  
Pervasive Computing ◽  
Contextual Information ◽  
Physical World ◽  
Context Aware ◽  
Pervasive Systems ◽  
Computational Environment ◽  
Meta Programming ◽  
Context Aware System ◽  
High Level

Context-aware pervasive systems are emerging as an important class of applications. Such systems can respond intelligently to contextual information about the physical world acquired via sensors and information about the computational environment. A declarative approach to building context-aware pervasive systems is presented, and the notion of the situation program is introduced, which highlights the primacy of the situation abstraction for building context-aware pervasive systems. There is also a demonstration of how to manipulate situation programs using meta-programming within an extension of the Prolog logic programming language which is called LogicCAP. Such meta-reasoning enables complex situations to be described in terms of other situations. Furthermore, a discussion is given on how the design of situation programs can affect the properties of a context-aware system. The approach encourages a high-level of abstraction for representing and reasoning with situations, and supports building context-aware systems incrementally by providing modularity and separation of concerns.

Sign in / Sign up

Export Citation Format

Share Document