Seismic stratigraphy interpretation by deep convolutional neural networks: A semisupervised workflow

Geophysics ◽  
2020 ◽  
Vol 85 (4) ◽  
pp. WA77-WA86 ◽  
Author(s):  
Haibin Di ◽  
Zhun Li ◽  
Hiren Maniar ◽  
Aria Abubakar
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Seismic Data ◽  
Network Architecture ◽  
Domain Knowledge ◽  
Model Building ◽  
Seismic Stratigraphy ◽  
Training Data ◽  
Data Sets ◽  
Deep Convolutional Neural Networks

Depicting geologic sequences from 3D seismic surveying is of significant value to subsurface reservoir exploration, but it is usually time- and labor-intensive for manual interpretation by experienced seismic interpreters. We have developed a semisupervised workflow for efficient seismic stratigraphy interpretation by using the state-of-the-art deep convolutional neural networks (CNNs). Specifically, the workflow consists of two components: (1) seismic feature self-learning (SFSL) and (2) stratigraphy model building (SMB), each of which is formulated as a deep CNN. Whereas the SMB is supervised by knowledge from domain experts and the associated CNN uses a similar network architecture typically used in image segmentation, the SFSL is designed as an unsupervised process and thus can be performed backstage while an expert prepares the training labels for the SMB CNN. Compared with conventional approaches, the our workflow is superior in two aspects. First, the SMB CNN, initialized by the SFSL CNN, successfully inherits the prior knowledge of the seismic features in the target seismic data. Therefore, it becomes feasible for completing the supervised training of the SMB CNN more efficiently using only a small amount of training data, for example, less than 0.1% of the available seismic data as demonstrated in this paper. Second, for the convenience of seismic experts in translating their domain knowledge into training labels, our workflow is designed to be applicable to three scenarios, trace-wise, paintbrushing, and full-sectional annotation. The performance of the new workflow is well-verified through application to three real seismic data sets. We conclude that the new workflow is not only capable of providing robust stratigraphy interpretation for a given seismic volume, but it also holds great potential for other problems in seismic data analysis.

scholarly journals A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

2020 ◽  
Author(s):  
B Wang ◽  
Y Sun ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Learning Task ◽  
Fixed Number ◽  
Learning Rate ◽  
Current Layer ◽  
Training Process ◽  
Deep Convolutional Neural Networks

© 2019, Springer Nature Switzerland AG. Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to automatically evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy.

scholarly journals Understanding unconventional preprocessors in deep convolutional neural networks for face identification

2019 ◽  
Vol 1 (11) ◽  
Author(s):  
Chollette C. Olisah ◽  
Lyndon Smith
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Data Augmentation ◽  
Color Space ◽  
Back Propagation ◽  
Face Identification ◽  
Deep Convolutional Neural Networks ◽  
Softmax Classifier ◽  
Deep Network

Abstract Deep convolutional neural networks have achieved huge successes in application domains like object and face recognition. The performance gain is attributed to different facets of the network architecture such as: depth of the convolutional layers, activation function, pooling, batch normalization, forward and back propagation and many more. However, very little emphasis is made on the preprocessor’s module of the network. Therefore, in this paper, the network’s preprocessing module is varied across different preprocessing approaches while keeping constant other facets of the deep network architecture, to investigate the contribution preprocessing makes to the network. Commonly used preprocessors are the data augmentation and normalization and are termed conventional preprocessors. Others are termed the unconventional preprocessors, they are: color space converters; grey-level resolution preprocessors; full-based and plane-based image quantization, Gaussian blur, illumination normalization and insensitive feature preprocessors. To achieve fixed network parameters, CNNs with transfer learning is employed. The aim is to transfer knowledge from the high-level feature vectors of the Inception-V3 network to offline preprocessed LFW target data; and features is trained using the SoftMax classifier for face identification. The experiments show that the discriminative capability of the deep networks can be improved by preprocessing RGB data with some of the unconventional preprocessors before feeding it to the CNNs. However, for best performance, the right setup of preprocessed data with augmentation and/or normalization is required. Summarily, preprocessing data before it is fed to the deep network is found to increase the homogeneity of neighborhood pixels even at reduced bit depth which serves for better storage efficiency.

scholarly journals Dealing with Lack of Training Data for Convolutional Neural Networks: The Case of Digital Pathology

Electronics ◽  
2019 ◽  
Vol 8 (3) ◽  
pp. 256
Author(s):  
Francesco Ponzio ◽  
Gianvito Urgese ◽  
Elisa Ficarra ◽  
Santa Di Cataldo
Keyword(s):  
Neural Networks ◽  
Transfer Learning ◽  
Convolutional Neural Networks ◽  
Digital Pathology ◽  
Training Data ◽  
Automated Classification ◽  
Computationally Efficient ◽  
Deep Convolutional Neural Networks ◽  
Cad Systems ◽  
Tissue Characteristics

Thanks to their capability to learn generalizable descriptors directly from images, deep Convolutional Neural Networks (CNNs) seem the ideal solution to most pattern recognition problems. On the other hand, to learn the image representation, CNNs need huge sets of annotated samples that are unfeasible in many every-day scenarios. This is the case, for example, of Computer-Aided Diagnosis (CAD) systems for digital pathology, where additional challenges are posed by the high variability of the cancerous tissue characteristics. In our experiments, state-of-the-art CNNs trained from scratch on histological images were less accurate and less robust to variability than a traditional machine learning framework, highlighting all the issues of fully training deep networks with limited data from real patients. To solve this problem, we designed and compared three transfer learning frameworks, leveraging CNNs pre-trained on non-medical images. This approach obtained very high accuracy, requiring much less computational resource for the training. Our findings demonstrate that transfer learning is a solution to the automated classification of histological samples and solves the problem of designing accurate and computationally-efficient CAD systems with limited training data.

scholarly journals Automatic classification of ovarian cancer types from cytological images using deep convolutional neural networks

2018 ◽  
Vol 38 (3) ◽  
Author(s):  
Miao Wu ◽  
Chuanbo Yan ◽  
Huiqiang Liu ◽  
Qian Liu
Keyword(s):  
Neural Networks ◽  
Ovarian Cancer ◽  
Convolutional Neural Networks ◽  
Image Data ◽  
Training Data ◽  
Serous Carcinoma ◽  
Deep Convolutional Neural Networks ◽  
Ovarian Cancers ◽  
Cancer Types

Ovarian cancer is one of the most common gynecologic malignancies. Accurate classification of ovarian cancer types (serous carcinoma, mucous carcinoma, endometrioid carcinoma, transparent cell carcinoma) is an essential part in the different diagnosis. Computer-aided diagnosis (CADx) can provide useful advice for pathologists to determine the diagnosis correctly. In our study, we employed a Deep Convolutional Neural Networks (DCNN) based on AlexNet to automatically classify the different types of ovarian cancers from cytological images. The DCNN consists of five convolutional layers, three max pooling layers, and two full reconnect layers. Then we trained the model by two group input data separately, one was original image data and the other one was augmented image data including image enhancement and image rotation. The testing results are obtained by the method of 10-fold cross-validation, showing that the accuracy of classification models has been improved from 72.76 to 78.20% by using augmented images as training data. The developed scheme was useful for classifying ovarian cancers from cytological images.

scholarly journals ADVERSARIAL DOMAIN ADAPTATION FOR THE CLASSIFICATION OF AERIAL IMAGES AND HEIGHT DATA USING CONVOLUTIONAL NEURAL NETWORKS

2019 ◽  
Vol IV-2/W7 ◽  
pp. 197-204 ◽  
Author(s):  
D. Wittich ◽  
F. Rottensteiner
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Domain Adaptation ◽  
Feature Representation ◽  
Training Data ◽  
Aerial Images ◽  
Remotely Sensed Data ◽  
Target Domain ◽  
Deep Convolutional Neural Networks

<p><strong>Abstract.</strong> Domain adaptation (DA) can drastically decrease the amount of training data needed to obtain good classification models by leveraging available data from a source domain for the classification of a new (target) domains. In this paper, we address deep DA, i.e. DA with deep convolutional neural networks (CNN), a problem that has not been addressed frequently in remote sensing. We present a new method for semi-supervised DA for the task of pixel-based classification by a CNN. After proposing an encoder-decoder-based fully convolutional neural network (FCN), we adapt a method for adversarial discriminative DA to be applicable to the pixel-based classification of remotely sensed data based on this network. It tries to learn a feature representation that is domain invariant; domain-invariance is measured by a classifier’s incapability of predicting from which domain a sample was generated. We evaluate our FCN on the ISPRS labelling challenge, showing that it is close to the best-performing models. DA is evaluated on the basis of three domains. We compare different network configurations and perform the representation transfer at different layers of the network. We show that when using a proper layer for adaptation, our method achieves a positive transfer and thus an improved classification accuracy in the target domain for all evaluated combinations of source and target domains.</p>

scholarly journals Deep Convolutional Neural Networks with Merge-and-Run Mappings

2018 ◽  
Author(s):  
Liming Zhao ◽  
Mingjie Li ◽  
Depu Meng ◽  
Xi Li ◽  
Zhaoxiang Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Information Flow ◽  
Building Block ◽  
Network Architecture ◽  
Time Complexity ◽  
Residual Network ◽  
Deep Convolutional Neural Networks ◽  
Testing Error ◽  
Simple Network

A deep residual network, built by stacking a sequence of residual blocks, is easy to train, because identity mappings skip residual branches and thus improve information flow. To further reduce the training difficulty, we present a simple network architecture, deep merge-and-run neural networks. The novelty lies in a modularized building block, merge-and-run block, which assembles residual branches in parallel through a merge-and-run mapping: average the inputs of these residual branches (Merge), and add the average to the output of each residual branch as the input of the subsequent residual branch (Run), respectively. We show that the merge-and-run mapping is a linear idempotent function in which the transformation matrix is idempotent, and thus improves information flow, making training easy. In comparison with residual networks, our networks enjoy compelling advantages: they contain much shorter paths and the width, i.e., the number of channels, is increased, and the time complexity remains unchanged. We evaluate the performance on the standard recognition tasks. Our approach demonstrates consistent improvements over ResNets with the comparable setup, and achieves competitive results (e.g., 3.06% testing error on CIFAR-10, 17.55% on CIFAR-100, 1.51% on SVHN). 

Application of convolutional neural networks for monitoring of marine objects

2020 ◽  
pp. 53-61
Author(s):  
Н.А. Полковникова ◽  
Е.В. Тузинкевич ◽  
А.Н. Попов
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Research Process ◽  
Probability Of Detection ◽  
Neural Network Architecture ◽  
Deep Convolutional Neural Networks

В статье рассмотрены технологии компьютерного зрения на основе глубоких свёрточных нейронных сетей. Применение нейронных сетей особенно эффективно для решения трудно формализуемых задач. Разработана архитектура свёрточной нейронной сети применительно к задаче распознавания и классификации морских объектов на изображениях. В ходе исследования выполнен ретроспективный анализ технологий компьютерного зрения и выявлен ряд проблем, связанных с применением нейронных сетей: «исчезающий» градиент, переобучение и вычислительная сложность. При разработке архитектуры нейросети предложено использовать функцию активации RELU, обучение некоторых случайно выбранных нейронов и нормализацию с целью упрощения архитектуры нейросети. Сравнение используемых в нейросети функций активации ReLU, LeakyReLU, Exponential ReLU и SOFTMAX выполнено в среде Matlab R2020a. На основе свёрточной нейронной сети разработана программа на языке программирования Visual C# в среде MS Visual Studio для распознавания морских объектов. Программапредназначена для автоматизированной идентификации морских объектов, производит детектирование (нахождение объектов на изображении) и распознавание объектов с высокой вероятностью обнаружения. The article considers computer vision technologies based on deep convolutional neural networks. Application of neural networks is particularly effective for solving difficult formalized problems. As a result convolutional neural network architecture to the problem of recognition and classification of marine objects on images is implemented. In the research process a retrospective analysis of computer vision technologies was performed and a number of problems associated with the use of neural networks were identified: vanishing gradient, overfitting and computational complexity. To solve these problems in neural network architecture development, it was proposed to use RELU activation function, training some randomly selected neurons and normalization for simplification of neural network architecture. Comparison of ReLU, LeakyReLU, Exponential ReLU, and SOFTMAX activation functions used in the neural network implemented in Matlab R2020a.The computer program based on convolutional neural network for marine objects recognition implemented in Visual C# programming language in MS Visual Studio integrated development environment. The program is designed for automated identification of marine objects, produces detection (i.e., presence of objects on image), and objects recognition with high probability of detection.

scholarly journals A mixed-scale dense convolutional neural network for image analysis

2017 ◽  
Vol 115 (2) ◽  
pp. 254-259 ◽  
Author(s):  
Daniël M. Pelt ◽  
James A. Sethian
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Processing ◽  
Network Architecture ◽  
Training Data ◽  
Network Architectures ◽  
Feature Maps ◽  
Deep Convolutional Neural Networks ◽  
Single Set ◽  
Reduced Risk

Deep convolutional neural networks have been successfully applied to many image-processing problems in recent works. Popular network architectures often add additional operations and connections to the standard architecture to enable training deeper networks. To achieve accurate results in practice, a large number of trainable parameters are often required. Here, we introduce a network architecture based on using dilated convolutions to capture features at different image scales and densely connecting all feature maps with each other. The resulting architecture is able to achieve accurate results with relatively few parameters and consists of a single set of operations, making it easier to implement, train, and apply in practice, and automatically adapts to different problems. We compare results of the proposed network architecture with popular existing architectures for several segmentation problems, showing that the proposed architecture is able to achieve accurate results with fewer parameters, with a reduced risk of overfitting the training data.

Sign in / Sign up

Export Citation Format

Share Document