Embedding Metadata-Enriched Graphs

This paper presents an on-going research where we studythe problem of embedding meta-data enriched graphs, with a focus onknowledge graphs in a vector space with transformer based deep neuralnetworks. Experimentally, we compare ceteris paribus the performance ofa transformer-based model with other non-transformer approaches. Dueto their recent success in natural language processing we hypothesizethat the former is superior in performance. We test this hypothesizesby comparing the performance of transformer embeddings with non-transformer embeddings on different downstream tasks. Our researchmight contribute to a better understanding of how random walks in-fluence the learning of features, which might be useful in the design ofdeep learning architectures for graphs when the input is generated withrandom walks.

A novel deep learning-based 3D cell segmentation framework for future image-based disease detection

Cell segmentation plays a crucial role in understanding, diagnosing, and treating diseases. Despite the recent success of deep learning-based cell segmentation methods, it remains challenging to accurately segment densely packed cells in 3D cell membrane images. Existing approaches also require fine-tuning multiple manually selected hyperparameters on the new datasets. We develop a deep learning-based 3D cell segmentation pipeline, 3DCellSeg, to address these challenges. Compared to the existing methods, our approach carries the following novelties: (1) a robust two-stage pipeline, requiring only one hyperparameter; (2) a light-weight deep convolutional neural network (3DCellSegNet) to efficiently output voxel-wise masks; (3) a custom loss function (3DCellSeg Loss) to tackle the clumped cell problem; and (4) an efficient touching area-based clustering algorithm (TASCAN) to separate 3D cells from the foreground masks. Cell segmentation experiments conducted on four different cell datasets show that 3DCellSeg outperforms the baseline models on the ATAS (plant), HMS (animal), and LRP (plant) datasets with an overall accuracy of 95.6%, 76.4%, and 74.7%, respectively, while achieving an accuracy comparable to the baselines on the Ovules (plant) dataset with an overall accuracy of 82.2%. Ablation studies show that the individual improvements in accuracy is attributable to 3DCellSegNet, 3DCellSeg Loss, and TASCAN, with the 3DCellSeg demonstrating robustness across different datasets and cell shapes. Our results suggest that 3DCellSeg can serve a powerful biomedical and clinical tool, such as histo-pathological image analysis, for cancer diagnosis and grading.

Speech Dereverberation Using Deep Learning Algorithm

This paper focuses on speech derverberation using a single microphone. We investigate the applicability of fully convolutional networks (FCN) to enhance the speech signal represented by short-time Fourier transform (STFT) images in light of their recent success in many image processing applications. We present two variants: a "U-Net," which is an encoder-decoder network with skip connections, and a generative adversarial network (GAN) with the U-Net as the generator, which produces a more intuitive cost function for training. To assess our method, we used data from the REVERB challenge and compared our results to those of other methods tested under the same conditions. In most cases, we discovered that our method outperforms the competing methods.

Application of Reinforcement Learning and Deep Learning in Multiple-Input and Multiple-Output (MIMO) Systems

The current wireless communication infrastructure has to face exponential development in mobile traffic size, which demands high data rate, reliability, and low latency. MIMO systems and their variants (i.e., Multi-User MIMO and Massive MIMO) are the most promising 5G wireless communication systems technology due to their high system throughput and data rate. However, the most significant challenges in MIMO communication are substantial problems in exploiting the multiple-antenna and computational complexity. The recent success of RL and DL introduces novel and powerful tools that mitigate issues in MIMO communication systems. This article focuses on RL and DL techniques for MIMO systems by presenting a comprehensive review on the integration between the two areas. We first briefly provide the necessary background to RL, DL, and MIMO. Second, potential RL and DL applications for different MIMO issues, such as detection, classification, and compression; channel estimation; positioning, sensing, and localization; CSI acquisition and feedback, security, and robustness; mmWave communication and resource allocation, are presented.

A Neural Approach for Modeling Continuous Time Sequences with Intermittent Observations

A large fraction of data generated via human activities such as online purchases, health records, spatial mobility etc. can be represented as continuous-time event sequences (CTES) i.e. sequences of discrete events over a continuous time. Learning neural models over CTES is a non-trivial task as it involves modeling the ever-increasing event timestamps, inter-event time gaps, event types, and the influences between different events within and across different sequences. Moreover, existing sequence modeling techniques consider a complete observation scenario i.e. the event sequence being modeled is completely observed with no missing events – an ideal setting that is rarely applicable in real-world applications. In this paper, we highlight our approach[8] for modeling CTES with intermittent observations. Buoyed by the recent success of neural marked temporal point processes (MTPP) for modeling the generative distribution of CTES, we provide a novel unsupervised model and inference method for learning MTPP in presence of event sequences with missing events. Specifically, we first model the generative processes of observed events and missing events using two MTPP, where the missing events are represented as latent random variables. Then, we devise an unsupervised training method that jointly learns both the MTPP using variational inference. Experiments across real-world datasets show that our modeling framework outperforms state-of-the-art techniques for future event prediction and imputation. This work appeared in AISTATS 2021.

Pansformers: Transformer-Based Self-Attention Network for Pansharpening

Pansharpening is the task of creating a High-Resolution Multi-Spectral Image (HRMS) by extracting and infusing pixel details from the High-Resolution Panchromatic Image into the Low-Resolution Multi-Spectral (LRMS). With the boom in the amount of satellite image data, researchers have replaced traditional approaches with deep learning models. However, existing deep learning models are not built to capture intricate pixel-level relationships. Motivated by the recent success of self-attention mechanisms in computer vision tasks, we propose Pansformers, a transformer-based self-attention architecture, that computes band-wise attention. A further improvement is proposed in the attention network by introducing a Multi-Patch Attention mechanism, which operates on non-overlapping, local patches of the image. Our model is successful in infusing relevant local details from the Panchromatic image while preserving the spectral integrity of the MS image. We show that our Pansformer model significantly improves the performance metrics and the output image quality on imagery from two satellite distributions IKONOS and LANDSAT-8.

Pansformers: Transformer-Based Self-Attention Network for Pansharpening

Pansharpening is the task of creating a High-Resolution Multi-Spectral Image (HRMS) by extracting and infusing pixel details from the High-Resolution Panchromatic Image into the Low-Resolution Multi-Spectral (LRMS). With the boom in the amount of satellite image data, researchers have replaced traditional approaches with deep learning models. However, existing deep learning models are not built to capture intricate pixel-level relationships. Motivated by the recent success of self-attention mechanisms in computer vision tasks, we propose Pansformers, a transformer-based self-attention architecture, that computes band-wise attention. A further improvement is proposed in the attention network by introducing a Multi-Patch Attention mechanism, which operates on non-overlapping, local patches of the image. Our model is successful in infusing relevant local details from the Panchromatic image while preserving the spectral integrity of the MS image. We show that our Pansformer model significantly improves the performance metrics and the output image quality on imagery from two satellite distributions IKONOS and LANDSAT-8.

Remote Cementing Enabled with Automation Drives Reliability, Consistency and Efficiency; A Case History from Norwegian Continental Shelf

Cementing is the fundamental first step and foundation for well construction. The traditional "let's go, mix it, pump it and bump it" cannot be the standard for the current and future offshore cementing operations. As oil and gas operators continue to push the envelope for both innovation and efficiency in well construction operations, to drive energy transition, lower carbon footprint, service providers continue to look for ways to "do more, with less". The latest innovation is redefining offshore cementing operations with a powerful combination of field-proven expertise, equipment, processes, and software. Remote Cementing Operations, the first of its kind in the industry, offers real- time and remote-operation capabilities, controls, and diagnostics of offshore cementing units. While conventional operations would typically involve a cement specialist working in an adjacent room on the rig, Remote Cementing Operations allows all cementing procedures to be controlled offsite by a cementing SME (Subject Matter Expert) from a Remote Operations Center (ROC), miles away from the offshore rig simplifying the operations, minimize errors and improve reliability. As the industry moves forward with a goal to lower carbon footprint, remote cementing enabled by automation will play a key role to implement innovative technologies that will help operators accomplish zonal isolation today and in the future while improving reliability, consistency and driving efficiency. The new implemented process thus results in reduced costs, risks, and non-productive time (NPT) with fewer personnel on-board (POB)—all without sacrificing quality, safety, and performance. A recent success case study is presented, where in an entire offshore well all the cementing operations have been mixed and pumped flawlessly from the ROC in one of the NCS (Norwegian Continental Shelf) rigs. This work explores the relationship between the process of planning, execution and troubleshooting remotely when performing cement operations. By analyzing and reviewing different previous experiences on remote operations, the authors developed a more comprehensive decision support system for remote cementing operations.

Molecular Graph Contrastive Learning with Parameterized Explainable Augmentations

ABSTRACTLearning generalizable, transferable, and robust representations for molecule data has always been a challenge. The recent success of contrastive learning (CL) for self-supervised graph representation learning provides a novel perspective to learn molecule representations. The most prevailing graph CL framework is to maximize the agreement of representations in different augmented graph views. However, existing graph CL frameworks usually adopt stochastic augmentations or schemes according to pre-defined rules on the input graph to obtain different graph views in various scales (e.g. node, edge, and subgraph), which may destroy topological semantemes and domain prior in molecule data, leading to suboptimal performance. Therefore, designing parameterized, learnable, and explainable augmentation is quite necessary for molecular graph contrastive learning. A well-designed parameterized augmentation scheme can preserve chemically meaningful structural information and intrinsically essential attributes for molecule graphs, which helps to learn representations that are insensitive to perturbation on unimportant atoms and bonds. In this paper, we propose a novel Molecular Graph Contrastive Learning with Parameterized Explainable Augmentations, MolCLE for brevity, that self-adaptively incorporates chemically significative information from both topological and semantic aspects of molecular graphs. Specifically, we apply deep neural networks to parameterize the augmentation process for both the molecular graph topology and atom attributes, to highlight contributive molecular substructures and recognize underlying chemical semantemes. Comprehensive experiments on a variety of real-world datasets demonstrate that our proposed method consistently outperforms compared baselines, which verifies the effectiveness of the proposed framework. Detailedly, our self-supervised MolCLE model surpasses many supervised counterparts, and meanwhile only uses hundreds of thousands of parameters to achieve comparative results against the state-of-the-art baseline, which has tens of millions of parameters. We also provide detailed case studies to validate the explainability of augmented graph views.CCS CONCEPTS• Mathematics of computing → Graph algorithms; • Applied computing → Bioinformatics; • Computing methodologies → Neural networks; Unsupervised learning.

Nanovaccine Delivery Approaches and Advanced Delivery Systems for the Prevention of Viral Infections: From Development to Clinical Application

Viral infections causing pandemics and chronic diseases are the main culprits implicated in devastating global clinical and socioeconomic impacts, as clearly manifested during the current COVID-19 pandemic. Immunoprophylaxis via mass immunisation with vaccines has been shown to be an efficient strategy to control such viral infections, with the successful and recently accelerated development of different types of vaccines, thanks to the advanced biotechnological techniques involved in the upstream and downstream processing of these products. However, there is still much work to be done for the improvement of efficacy and safety when it comes to the choice of delivery systems, formulations, dosage form and route of administration, which are not only crucial for immunisation effectiveness, but also for vaccine stability, dose frequency, patient convenience and logistics for mass immunisation. In this review, we discuss the main vaccine delivery systems and associated challenges, as well as the recent success in developing nanomaterials-based and advanced delivery systems to tackle these challenges. Manufacturing and regulatory requirements for the development of these systems for successful clinical and marketing authorisation were also considered. Here, we comprehensively review nanovaccines from development to clinical application, which will be relevant to vaccine developers, regulators, and clinicians.