Contour Extraction of Individual Cattle From an Image Using Enhanced Mask R-CNN Instance Segmentation Method

The paper proposes a novel instance segmentation method for traffic videos devised for deployment on real-time embedded devices. A novel neural network architecture is proposed using a multi-resolution feature extraction backbone and improved network designs for the object detection and instance segmentation branches. A novel post-processing method is introduced to ensure a reduced rate of false detection by evaluating the quality of the output masks. An improved network training procedure is proposed based on a novel label assignment algorithm. An ablation study on speed-vs.-performance trade-off further modifies the two branches and replaces the conventional ResNet-based performance-oriented backbone with a lightweight speed-oriented design. The proposed architectural variations achieve real-time performance when deployed on embedded devices. The experimental results demonstrate that the proposed instance segmentation method for traffic videos outperforms the you only look at coefficients algorithm, the state-of-the-art real-time instance segmentation method. The proposed architecture achieves qualitative results with 31.57 average precision on the COCO dataset, while its speed-oriented variations achieve speeds of up to 66.25 frames per second on the Jetson AGX Xavier module.

Download Full-text

An Efficient Module for Instance Segmentation Based on Multi-Level Features and Attention Mechanisms

Applied Sciences ◽

10.3390/app11030968 ◽

2021 ◽

Vol 11 (3) ◽

pp. 968

Author(s):

Yingchun Sun ◽

Wang Gao ◽

Shuguo Pan ◽

Tao Zhao ◽

Yahui Peng

Keyword(s):

Feature Extraction ◽

Spatial Structure ◽

Semantic Feature ◽

Semantic Features ◽

Segmentation Method ◽

Spatial Dimensions ◽

Feature Pyramid ◽

Multi Level ◽

High Level ◽

Instance Segmentation

Recently, multi-level feature networks have been extensively used in instance segmentation. However, because not all features are beneficial to instance segmentation tasks, the performance of networks cannot be adequately improved by synthesizing multi-level convolutional features indiscriminately. In order to solve the problem, an attention-based feature pyramid module (AFPM) is proposed, which integrates the attention mechanism on the basis of a multi-level feature pyramid network to efficiently and pertinently extract the high-level semantic features and low-level spatial structure features; for instance, segmentation. Firstly, we adopt a convolutional block attention module (CBAM) into feature extraction, and sequentially generate attention maps which focus on instance-related features along the channel and spatial dimensions. Secondly, we build inter-dimensional dependencies through a convolutional triplet attention module (CTAM) in lateral attention connections, which is used to propagate a helpful semantic feature map and filter redundant informative features irrelevant to instance objects. Finally, we construct branches for feature enhancement to strengthen detailed information to boost the entire feature hierarchy of the network. The experimental results on the Cityscapes dataset manifest that the proposed module outperforms other excellent methods under different evaluation metrics and effectively upgrades the performance of the instance segmentation method.

Download Full-text

Accurate Instance Segmentation in Pediatric Elbow Radiographs

Sensors ◽

10.3390/s21237966 ◽

2021 ◽

Vol 21 (23) ◽

pp. 7966

Author(s):

Dixiao Wei ◽

Qiongshui Wu ◽

Xianpei Wang ◽

Meng Tian ◽

Bowen Li

Keyword(s):

Elbow Joint ◽

Segmentation Method ◽

Qualitative And Quantitative ◽

Robust Segmentation ◽

Local Contexts ◽

Good Potential ◽

Quantitative Results ◽

Bounding Boxes ◽

Global And Local ◽

Instance Segmentation

Radiography is an essential basis for the diagnosis of fractures. For the pediatric elbow joint diagnosis, the doctor needs to diagnose abnormalities based on the location and shape of each bone, which is a great challenge for AI algorithms when interpreting radiographs. Bone instance segmentation is an effective upstream task for automatic radiograph interpretation. Pediatric elbow bone instance segmentation is a process by which each bone is extracted separately from radiography. However, the arbitrary directions and the overlapping of bones pose issues for bone instance segmentation. In this paper, we design a detection-segmentation pipeline to tackle these problems by using rotational bounding boxes to detect bones and proposing a robust segmentation method. The proposed pipeline mainly contains three parts: (i) We use Faster R-CNN-style architecture to detect and locate bones. (ii) We adopt the Oriented Bounding Box (OBB) to improve the localizing accuracy. (iii) We design the Global-Local Fusion Segmentation Network to combine the global and local contexts of the overlapped bones. To verify the effectiveness of our proposal, we conduct experiments on our self-constructed dataset that contains 1274 well-annotated pediatric elbow radiographs. The qualitative and quantitative results indicate that the network significantly improves the performance of bone extraction. Our methodology has good potential for applying deep learning in the radiography’s bone instance segmentation.

Download Full-text

Target Location of Industrial Robot Based on Vision

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.945-949.1478 ◽

2014 ◽

Vol 945-949 ◽

pp. 1478-1481

Author(s):

Gui Hong Jia

Keyword(s):

Target Location ◽

Vision System ◽

Industrial Robot ◽

Robot Vision ◽

Contour Extraction ◽

Multiple Target ◽

Segmentation Method ◽

Target Image ◽

Black And White ◽

Centroid Position

Vision is the most important way to obtain information from the word. This paper collected the target image using industrial robot vision system, and We get Black and white images using binary image segmentation method, then the contour of each object in the image can be obtained with edge detection and contour extraction, The centroid position was confirmed using minimum enclosing rectangle method after gaining the outline of target. The experimental results show that this method can quickly and accurately obtain multiple target centroid position.

Download Full-text

Redundant Points Removing of SFS Reconstructed Surface

Key Engineering Materials ◽

10.4028/www.scientific.net/kem.579-580.618 ◽

2013 ◽

Vol 579-580 ◽

pp. 618-625

Author(s):

Feng He Wu ◽

Qing Jing Yin ◽

Jin Fen Wang ◽

Yong Xin Li

Keyword(s):

Operator Method ◽

Shape From Shading ◽

Contour Extraction ◽

Segmentation Method ◽

Background Region ◽

Novel Approach ◽

Shape Precision ◽

Intersection Algorithm ◽

Reconstructed Surface ◽

3D Profile

To remove the redundant points of image background in SFS reconstructed surface, a novel approach which combines the advantages of polygon scan and intersection algorithm and improves contour extraction method is proposed. In one hand, according to the gray scale level of pixels, the 3D profile of object is reconstructed by Shape from Shading method. In the other hand, firstly, the image is transformed into binary image through single-threshold segmentation method, the defects in image are remedied by morphological method, and the 2D contour of image is extracted through differential operator method; secondly, the image is divided into two parts using the polygon scan and intersection algorithm, that is the objective region and the background region; finally, the redundant points of image background in SFS reconstructed surface are removed through the fusing of 3D profile and 2D contour. Experiments shown that the proposed method can effectively remove the redundant points of SFS reconstructed surface, and it not only ensure the shape precision of SFS reconstructed surface, but also improve the universal of Shape from Shading technique.

Download Full-text

Automatic Detection and Segmentation for Group-Housed Pigs Based on PigMS R-CNN

Sensors ◽

10.3390/s21093251 ◽

2021 ◽

Vol 21 (9) ◽

pp. 3251

Author(s):

Shuqin Tu ◽

Weijun Yuan ◽

Yun Liang ◽

Fan Wang ◽

Hua Wan

Keyword(s):

Body Condition Score ◽

Live Weight ◽

Segmentation Method ◽

Feature Maps ◽

Residual Network ◽

Candidate Network ◽

Welfare Evaluation ◽

Condition Score ◽

Feature Pyramid ◽

Instance Segmentation

Instance segmentation is an accurate and reliable method to segment adhesive pigs’ images, and is critical for providing health and welfare information on individual pigs, such as body condition score, live weight, and activity behaviors in group-housed pig environments. In this paper, a PigMS R-CNN framework based on mask scoring R-CNN (MS R-CNN) is explored to segment adhesive pig areas in group-pig images, to separate the identification and location of group-housed pigs. The PigMS R-CNN consists of three processes. First, a residual network of 101-layers, combined with the feature pyramid network (FPN), is used as a feature extraction network to obtain feature maps for input images. Then, according to these feature maps, the region candidate network generates the regions of interest (RoIs). Finally, for each RoI, we can obtain the location, classification, and segmentation results of detected pigs through the regression and category, and mask three branches from the PigMS R-CNN head network. To avoid target pigs being missed and error detections in overlapping or stuck areas of group-housed pigs, the PigMS R-CNN framework uses soft non-maximum suppression (soft-NMS) by replacing the traditional NMS to conduct post-processing selected operation of pigs. The MS R-CNN framework with traditional NMS obtains results with an F1 of 0.9228. By setting the soft-NMS threshold to 0.7 on PigMS R-CNN, detection of the target pigs achieves an F1 of 0.9374. The work explores a new instance segmentation method for adhesive group-housed pig images, which provides valuable exploration for vision-based, real-time automatic pig monitoring and welfare evaluation.

Download Full-text

CONTOUR EXTRACTION OF PLANAR ELEMENTS OF BUILDING FACADES FROM POINT CLOUDS USING GLOBAL GRAPH-BASED CLUSTERING

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-2-w7-211-2019 ◽

2019 ◽

Vol IV-2/W7 ◽

pp. 211-219 ◽

Cited By ~ 1

Author(s):

Y. Xu ◽

U. Stilla

Keyword(s):

Graphical Model ◽

Point Clouds ◽

Contour Extraction ◽

Segmentation Method ◽

Unsupervised Segmentation ◽

Line Segments ◽

Segmentation Methods ◽

Building Facades ◽

Urban Scene ◽

Building Elements

<p><strong>Abstract.</strong> In this work, we present a surface-based method to extract the contours of planar building elements in the urban scene. A bottom-up segmentation method that utilizes global graph-based optimization and supervoxel structure is developed, enabling an automatic and unsupervised segmentation of point clouds. Then, a planarity-based extraction is conducted to segments, and only the planar segments, as well as their neighborhoods, are selected as candidates for the plane fitting. The points of the plane can be identified by the parametric model given by the planarity calculation. Afterward, the boundary points of the extracted plane are extracted by the alpha-shape. Optionally, line segments can be fitted and optimized by the energy minimization with the local graphical model. The experimental results using different datasets reveal that our proposed segmentation methods can be effective and comparable with other method, and the contours of planar building elements can be well extracted from the complex urban scene.</p>

Download Full-text

EmbedMask: Embedding Coupling for Instance Segmentation

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/175 ◽

2021 ◽

Author(s):

Hui Ying ◽

Zhaojin Huang ◽

Shu Liu ◽

Tianjia Shao ◽

Kun Zhou

Keyword(s):

High Resolution ◽

High Speed ◽

State Of The Art ◽

The State ◽

Single Stage ◽

Segmentation Method ◽

Two Stage ◽

Object Proposal ◽

Segmentation Methods ◽

Instance Segmentation

Current instance segmentation methods can be categorized into segmentation-based methods and proposal-based methods. The former performs segmentation first and then does clustering, while the latter detects objects first and then predicts the mask for each object proposal. In this work, we propose a single-stage method, named EmbedMask, that unifies both methods by taking their advantages, so it can achieve good performance in instance segmentation and produce high-resolution masks in a high speed. EmbedMask introduces two newly defined embeddings for mask prediction, which are pixel embedding and proposal embedding. During training, we enforce the pixel embedding to be close to its coupled proposal embedding if they belong to the same instance. During inference, pixels are assigned to the mask of the proposal if their embeddings are similar. This mechanism brings several benefits. First, the pixel-level clustering enables EmbedMask to generate high-resolution masks and avoids the complicated two-stage mask prediction. Second, the existence of proposal embedding simplifies and strengthens the clustering procedure, so our method can achieve high speed and better performance than segmentation-based methods. Without any bell or whistle, EmbedMask outperforms the state-of-the-art instance segmentation method Mask R-CNN on the challenging COCO dataset, obtaining more detailed masks at a higher speed.

Download Full-text

Applying Data Augmentation and Mask R-CNN-based Instance Segmentation Method for Mixed-Type Wafer Maps Defect Patterns Classification

IEEE Transactions on Semiconductor Manufacturing ◽

10.1109/tsm.2021.3118922 ◽

2021 ◽

pp. 1-1

Author(s):

Ming-Chuan Chiu ◽

Tao-Ming Chen

Keyword(s):

Mixed Type ◽

Data Augmentation ◽

Segmentation Method ◽

Instance Segmentation

Download Full-text

Instance Segmentation Method of User Interface Component of Games

Applied Sciences ◽

10.3390/app10186502 ◽

2020 ◽

Vol 10 (18) ◽

pp. 6502

Author(s):

Shinjin Kang ◽

Jong-in Choi

Keyword(s):

Deep Learning ◽

Semantic Information ◽

Data Augmentation ◽

Image Data ◽

Learning Networks ◽

Segmentation Method ◽

Learning Network ◽

Additional Input ◽

Deep Learning Network ◽

Instance Segmentation

On the game screen, the UI interface provides key information for game play. A vision deep learning network exploits pure pixel information in the screen. Apart from this, if we separately extract the information provided by the UI interface and use it as an additional input value, we can enhance the learning efficiency of deep learning networks. To this end, by effectively segmenting UI interface components such as buttons, image icons, and gauge bars on the game screen, we should be able to separately analyze only the relevant images. In this paper, we propose a methodology that segments UI components in a game by using synthetic game images created on a game engine. We developed a tool that approximately detected the UI areas of an image in games on the game screen and generated a large amount of synthetic labeling data through this. By training this data on a Pix2Pix, we applied UI segmentation. The network trained in this way can segment the UI areas of the target game regardless of the position of the corresponding UI components. Our methodology can help analyze the game screen without applying data augmentation to the game screen. It can also help vision researchers who should extract semantic information from game image data.

Download Full-text