Encoding Health Records into Pathway Representations for Deep Learning

There is a growing trend in building deep learning patient representations from health records to obtain a comprehensive view of a patient’s data for machine learning tasks. This paper proposes a reproducible approach to generate patient pathways from health records and to transform them into a machine-processable image-like structure useful for deep learning tasks. Based on this approach, we generated over a million pathways from FAIR synthetic health records and used them to train a convolutional neural network. Our initial experiments show the accuracy of the CNN on a prediction task is comparable or better than other autoencoders trained on the same data, while requiring significantly less computational resources for training. We also assess the impact of the size of the training dataset on autoencoders performances. The source code for generating pathways from health records is provided as open source.

Download Full-text

Towards machine learning assisted error identification in orbital laser altimetry for tides derivation

10.5194/epsc2021-688 ◽

2021 ◽

Author(s):

Oliver Stenzel ◽

Martin Hilchenbach

Keyword(s):

Neural Network ◽

Machine Learning ◽

Training Dataset ◽

Altimeter Data ◽

Laser Altimetry ◽

Linear Network ◽

Error Identification ◽

Laser Altimeter ◽

Convolutional Network ◽

The Impact

Laser altimetry experiments on the NASA MESSENGER mission [1], and on the currently on cruise ESA/JAXA BepiColombo Mission [2,3] did and are going to yield, respectively, a plethora of range measurements of the surface of Mercury. Orbital laser altimetry can be used to derive tidal parameters, which can in turn be used to infer properties of a body&#8217;s interior [4,5]. The derivation of tidal parameters requires large datasets of precise and accurate measurements. Errors as well as outliers can degrade the quality of the computed tidal parameters. While many outliers can be filtered though conventional automated processes, other errors could only be identified by human supervision. In the face of the amount of data involved, systematic user interaction at the error identification step becomes unpractical. A neural network trained with user expertise could help spotting outliers and errors and would improve the derived parameters in accuracy and precession. We started developing a neural network based on the pytorch framework[6] and compared the performance with a small training dataset form the MESSENGER Laser Altimeter (MLA) for a linear and a convolutional network. The results were much in favour of the linear network [7]. In this presentation we explore the reasons behind bad convolutional network performance with extended training and test datasets. We are going to show our results for the filtered datasets and the impact this has on the derived tidal parameters. The filtering with an artificial neural network might be useful for other applications, as well. 1. Cavanaugh, J. F. et al. The Mercury Laser Altimeter Instrument for the MESSENGER Mission. Space Sci Rev 131, 451&#8211;479 (2007). 2. Benkhoff, J. et al. BepiColombo&#8212;Comprehensive exploration of Mercury: Mission overview and science goals. Planetary and Space Science 58, 2&#8211;20 (2010). 3. Thomas, N. et al. The BepiColombo Laser Altimeter. Space Sci Rev 217, 25 (2021). 4. Thor, R. N. et al. Determination of the lunar body tide from global laser altimetry data. J Geod 95, 4 (2021). 5. Thor, R. N. et al. Prospects for measuring Mercury&#8217;s tidal Love number h2 with the BepiColombo Laser Altimeter. A&A 633, A85 (2020). 6. Paszke, A., et al., PyTorch: An Imperative Style, High-Performance Deep Learning Library, In: Advances in Neural Information Processing Systems 32, pp 8024&#8211;8035, 2019. 7. Stenzel, O., Thor, R., and Hilchenbach, M.: Error identification in orbital laser altimeter data by machine learning, EGU General Assembly 2021, online, 19&#8211;30 Apr 2021, EGU21-14749, https://doi.org/10.5194/egusphere-egu21-14749, 2021.

Download Full-text

An Overview of Convolutional Neural Network: Its Architecture and Applications

10.20944/preprints201811.0546.v1 ◽

2018 ◽

Author(s):

Shadman Sakib ◽

Nazib Ahmed ◽

Ahmed Jawad Kabir ◽

Hridon Ahmed

Keyword(s):

Neural Network ◽

Machine Learning ◽

Artificial Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Learning Strategies ◽

Learning Tasks ◽

Artificial Neural ◽

Artificial Neural Network Ann ◽

The Many

With the increase of the Artificial Neural Network (ANN), machine learning has taken a forceful twist in recent times. One of the most spectacular kinds of ANN design is the Convolutional Neural Network (CNN). The Convolutional Neural Network (CNN) is a technology that mixes artificial neural networks and up to date deep learning strategies. In deep learning, Convolutional Neural Network is at the center of spectacular advances. This artificial neural network has been applied to several image recognition tasks for decades and attracted the eye of the researchers of the many countries in recent years as the CNN has shown promising performances in several computer vision and machine learning tasks. This paper describes the underlying architecture and various applications of Convolutional Neural Network.

Download Full-text

Comparing Machine Learning and Deep Learning Methods for Real-Time Crash Prediction

Transportation Research Record Journal of the Transportation Research Board ◽

10.1177/0361198119841571 ◽

2019 ◽

Vol 2673 (8) ◽

pp. 169-178 ◽

Cited By ~ 12

Author(s):

Athanasios Theofilatos ◽

Cong Chen ◽

Constantinos Antoniou

Keyword(s):

Neural Network ◽

Machine Learning ◽

Deep Learning ◽

Real Time ◽

Naive Bayes ◽

Naïve Bayes ◽

Support Vector ◽

Data Set ◽

Real Time Traffic ◽

The Impact

Although there are numerous studies examining the impact of real-time traffic and weather parameters on crash occurrence on freeways, to the best of the authors’ knowledge there are no studies which have compared the prediction performances of machine learning (ML) and deep learning (DL) models. The present study adds to current knowledge by comparing and validating ML and DL methods to predict real-time crash occurrence. To achieve this, real-time traffic and weather data from Attica Tollway in Greece were linked with historical crash data. The total data set was split into training/estimation (75%) and validation (25%) subsets, which were then standardized. First, the ML and DL prediction models were trained/estimated using the training data set. Afterwards, the models were compared on the basis of their performance metrics (accuracy, sensitivity, specificity, and area under curve, or AUC) on the test set. The models considered were k-nearest neighbor, Naïve Bayes, decision tree, random forest, support vector machine, shallow neural network, and, lastly, deep neural network. Overall, the DL model seems to be more appropriate, because it outperformed all other candidate models. More specifically, the DL model managed to achieve a balanced performance among all metrics compared with other models (total accuracy = 68.95%, sensitivity = 0.521, specificity = 0.77, AUC = 0.641). It is surprising though that the Naïve Bayes model achieved a good performance despite being far less complex than other models. The study findings are particularly useful, because they provide a first insight into performance of ML and DL models.

Download Full-text

An Overview of Convolutional Neural Network: Its Architecture and Applications

10.20944/preprints201811.0546.v4 ◽

2019 ◽

Cited By ~ 1

Author(s):

Shadman Sakib ◽

Nazib Ahmed ◽

Ahmed Jawad Kabir ◽

Hridon Ahmed

Keyword(s):

Neural Network ◽

Machine Learning ◽

Artificial Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Learning Strategies ◽

Learning Tasks ◽

Artificial Neural ◽

Artificial Neural Network Ann ◽

The Many

Download Full-text

Adaptive Information Assimilation using Convolutional Neural Network for Forecast of Breast Cancer from Electronic Health Records

10.21203/rs.2.18006/v1 ◽

2019 ◽

Author(s):

Senthil Kumar Kumar J ◽

Kamala Devi K ◽

Raja Sekar J

Keyword(s):

Breast Cancer ◽

Neural Network ◽

Deep Learning ◽

Electronic Health Records ◽

Convolutional Neural Network ◽

Frequency Vector ◽

Health Records ◽

Electronic Health ◽

Pathology Reports ◽

The Impact

Abstract Purpose Data acquired from cancer based Electronic Health Records (EHRs) shows key statistics on cancer affected persons. To estimate the impact of the cancer on those persons, we need to extract vital information from those pathology health records. It is an exhaustive procedure to carry out because of large volume of records and data acquired for a continuous period of time.Methods This research portrays, the investigation of convolutional neural network (CNN) and Support Vector Machine (SVM) techniques for extracting topographic codes from the pathology reports of breast cancer. Investigations are carried out using conventional frequency vector space method and the deep learning techniques such as CNN. The learning experience of those algorithms were absorbed on a set of 730 pathology reports.Results We perceived that the CNN technique reliably outperformed the conventional frequency vector methods. It is also observed that it causes the micro and macro average performance to increase up to 0.119, and 0.101, while considering the populated class labels for the CNN model. Unambiguously, the top performing CNN approach attained a micro-F score of 0.821 over the considered topography codes.Conclusion These promising outcomes reveals the prospective of deep learning approaches, particularly CNN for estimating the impact of the cancer from the pathology reports compared to conventional SVM approach. More advanced and accurate approaches to effectively improve the accuracy in information extraction are needed.

Download Full-text

Using artificial neural network condensation to facilitate adaption of machine learning in medical settings by reducing computational burden (Preprint)

10.2196/preprints.20767 ◽

2020 ◽

Author(s):

Dianbo Liu

Keyword(s):

Neural Network ◽

Machine Learning ◽

Third World ◽

Mortality Prediction ◽

Neural Net ◽

Medical Settings ◽

Hidden Layer ◽

Applications Of Machine Learning ◽

Computational Resources ◽

Developed Nations

BACKGROUND Applications of machine learning (ML) on health care can have a great impact on people’s lives. At the same time, medical data is usually big, requiring a significant amount of computational resources. Although it might not be a problem for wide-adoption of ML tools in developed nations, availability of computational resource can very well be limited in third-world nations and on mobile devices. This can prevent many people from benefiting of the advancement in ML applications for healthcare. OBJECTIVE In this paper we explored three methods to increase computational efficiency of either recurrent neural net-work(RNN) or feedforward (deep) neural network (DNN) while not compromising its accuracy. We used in-patient mortality prediction as our case analysis upon intensive care dataset. METHODS We reduced the size of RNN and DNN by applying pruning of “unused” neurons. Additionally, we modified the RNN structure by adding a hidden-layer to the RNN cell but reduce the total number of recurrent layers to accomplish a reduction of total parameters in the network. Finally, we implemented quantization on DNN—forcing the weights to be 8-bits instead of 32-bits. RESULTS We found that all methods increased implementation efficiency–including training speed, memory size and inference speed–without reducing the accuracy of mortality prediction. CONCLUSIONS This improvements allow the implementation of sophisticated NN algorithms on devices with lower computational resources.

Download Full-text

Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

Scientific Reports ◽

10.1038/s41598-020-78129-0 ◽

2020 ◽

Vol 10 (1) ◽

Author(s):

Young-Gon Kim ◽

Sungchul Kim ◽

Cristina Eunbee Cho ◽

In Hye Song ◽

Hee Jin Lee ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Frozen Section ◽

Medical Center ◽

External Validation ◽

Model Performance ◽

Classification Model ◽

Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

Download Full-text

Orchard Mapping with Deep Learning Semantic Segmentation

Sensors ◽

10.3390/s21113813 ◽

2021 ◽

Vol 21 (11) ◽

pp. 3813

Author(s):

Athanasios Anagnostis ◽

Aristotelis C. Tagarakis ◽

Dimitrios Kateris ◽

Vasileios Moysiadis ◽

Claus Grøn Sørensen ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Semantic Segmentation ◽

Automated Detection ◽

Aerial Images ◽

Training Dataset ◽

Field Boundary ◽

Different Seasons ◽

Detection And Localization ◽

Different Levels

This study aimed to propose an approach for orchard trees segmentation using aerial images based on a deep learning convolutional neural network variant, namely the U-net network. The purpose was the automated detection and localization of the canopy of orchard trees under various conditions (i.e., different seasons, different tree ages, different levels of weed coverage). The implemented dataset was composed of images from three different walnut orchards. The achieved variability of the dataset resulted in obtaining images that fell under seven different use cases. The best-trained model achieved 91%, 90%, and 87% accuracy for training, validation, and testing, respectively. The trained model was also tested on never-before-seen orthomosaic images or orchards based on two methods (oversampling and undersampling) in order to tackle issues with out-of-the-field boundary transparent pixels from the image. Even though the training dataset did not contain orthomosaic images, it achieved performance levels that reached up to 99%, demonstrating the robustness of the proposed approach.

Download Full-text

Deep Learning Classification of Canine Behavior Using a Single Collar-Mounted Accelerometer: Real-World Validation

Animals ◽

10.3390/ani11061549 ◽

2021 ◽

Vol 11 (6) ◽

pp. 1549

Author(s):

Robert D. Chambers ◽

Nathanael C. Yoder ◽

Aletha B. Carson ◽

Christian Junge ◽

David E. Allen ◽

...

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Real World ◽

Learning Algorithm ◽

Drinking Behavior ◽

True Positive Rate ◽

Training Dataset ◽

Activity Levels ◽

Accelerometer Data ◽

Activity Monitors

Collar-mounted canine activity monitors can use accelerometer data to estimate dog activity levels, step counts, and distance traveled. With recent advances in machine learning and embedded computing, much more nuanced and accurate behavior classification has become possible, giving these affordable consumer devices the potential to improve the efficiency and effectiveness of pet healthcare. Here, we describe a novel deep learning algorithm that classifies dog behavior at sub-second resolution using commercial pet activity monitors. We built machine learning training databases from more than 5000 videos of more than 2500 dogs and ran the algorithms in production on more than 11 million days of device data. We then surveyed project participants representing 10,550 dogs, which provided 163,110 event responses to validate real-world detection of eating and drinking behavior. The resultant algorithm displayed a sensitivity and specificity for detecting drinking behavior (0.949 and 0.999, respectively) and eating behavior (0.988, 0.983). We also demonstrated detection of licking (0.772, 0.990), petting (0.305, 0.991), rubbing (0.729, 0.996), scratching (0.870, 0.997), and sniffing (0.610, 0.968). We show that the devices’ position on the collar had no measurable impact on performance. In production, users reported a true positive rate of 95.3% for eating (among 1514 users), and of 94.9% for drinking (among 1491 users). The study demonstrates the accurate detection of important health-related canine behaviors using a collar-mounted accelerometer. We trained and validated our algorithms on a large and realistic training dataset, and we assessed and confirmed accuracy in production via user validation.

Download Full-text

Audio-Based Drone Detection and Identification Using Deep Learning Techniques with Dataset Enhancement through Generative Adversarial Networks

Sensors ◽

10.3390/s21154953 ◽

2021 ◽

Vol 21 (15) ◽

pp. 4953

Author(s):

Sara Al-Emadi ◽

Abdulla Al-Ali ◽

Abdulaziz Al-Ali

Keyword(s):

Neural Network ◽

Deep Learning ◽

Recurrent Neural Network ◽

Learning Algorithms ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Adversarial Networks ◽

Detection And Identification ◽

Learning Techniques ◽

The Impact

Drones are becoming increasingly popular not only for recreational purposes but in day-to-day applications in engineering, medicine, logistics, security and others. In addition to their useful applications, an alarming concern in regard to the physical infrastructure security, safety and privacy has arisen due to the potential of their use in malicious activities. To address this problem, we propose a novel solution that automates the drone detection and identification processes using a drone’s acoustic features with different deep learning algorithms. However, the lack of acoustic drone datasets hinders the ability to implement an effective solution. In this paper, we aim to fill this gap by introducing a hybrid drone acoustic dataset composed of recorded drone audio clips and artificially generated drone audio samples using a state-of-the-art deep learning technique known as the Generative Adversarial Network. Furthermore, we examine the effectiveness of using drone audio with different deep learning algorithms, namely, the Convolutional Neural Network, the Recurrent Neural Network and the Convolutional Recurrent Neural Network in drone detection and identification. Moreover, we investigate the impact of our proposed hybrid dataset in drone detection. Our findings prove the advantage of using deep learning techniques for drone detection and identification while confirming our hypothesis on the benefits of using the Generative Adversarial Networks to generate real-like drone audio clips with an aim of enhancing the detection of new and unfamiliar drones.

Download Full-text