scholarly journals CP-NAS: Child-Parent Neural Architecture Search for 1-bit CNNs

2020 ◽  
Author(s):  
Li'an Zhuo ◽  
Baochang Zhang ◽  
Hanlin Chen ◽  
Linlin Yang ◽  
Chen Chen ◽  
...  
Keyword(s):  
Computational Cost ◽  
Unified Framework ◽  
Embedded Devices ◽  
Natural Approach ◽  
Neural Architecture ◽  
Resource Limited ◽  
Training Stage ◽  
Comparable Accuracy ◽  
Kernel Level ◽  
High Computational Cost

Neural architecture search (NAS) proves to be among the best approaches for many tasks by generating an application-adaptive neural architectures, which are still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binarized weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a unified framework. To this end, a Child-Parent model is introduced to a differentiable NAS to search the binarized architecture(Child) under the supervision of a full-precision model (Parent). In the search stage, the Child-Parent model uses an indicator generated by the parent and child model accuracy to evaluate the performance and abandon operations with less potential. In the training stage, a kernel level CP loss is introduced to optimize the binarized network. Extensive experiments demonstrate that the proposed CP-NAS achieves a comparable accuracy with traditional NAS on both the CIFAR and ImageNet databases. It achieves an accuracy of 95.27% on CIFAR-10, 64.3% on ImageNet with binarized weights and activations, and a 30% faster search than prior arts.

scholarly journals Survey on Revocation in Ciphertext-Policy Attribute-Based Encryption

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1695 ◽  
Author(s):  
Al-Dahhan ◽  
Shi ◽  
Lee ◽  
Kifayat
Keyword(s):  
Computational Cost ◽  
Sensitive Data ◽  
Resource Limited ◽  
Attribute Based Encryption ◽  
Wide Range ◽  
Secure Cloud Storage ◽  
Iot Devices ◽  
Ciphertext Policy ◽  
Cryptographic Techniques ◽  
High Computational Cost

Recently, using advanced cryptographic techniques to process, store, and share datasecurely in an untrusted cloud environment has drawn widespread attention from academicresearchers. In particular, Ciphertext‐Policy Attribute‐Based Encryption (CP‐ABE) is a promising,advanced type of encryption technique that resolves an open challenge to regulate fine‐grainedaccess control of sensitive data according to attributes, particularly for Internet of Things (IoT)applications. However, although this technique provides several critical functions such as dataconfidentiality and expressiveness, it faces some hurdles including revocation issues and lack ofmanaging a wide range of attributes. These two issues have been highlighted by many existingstudies due to their complexity which is hard to address without high computational cost affectingthe resource‐limited IoT devices. In this paper, unlike other survey papers, existing single andmultiauthority CP‐ABE schemes are reviewed with the main focus on their ability to address therevocation issues, the techniques used to manage the revocation, and comparisons among themaccording to a number of secure cloud storage criteria. Therefore, this is the first review paperanalysing the major issues of CP‐ABE in the IoT paradigm and explaining the existing approachesto addressing these issues.

Transferability of Deep Learning Models for Focus Quality Assessment in Digital Pathology

2021 ◽  
Author(s):  
Adyn Miles ◽  
Mahdi S. Hosseini ◽  
Sheyang Tang ◽  
Zhou Wang ◽  
Savvas Damaskinos ◽  
...  
Keyword(s):  
Deep Learning ◽  
Quality Assessment ◽  
Computational Cost ◽  
Digital Pathology ◽  
Low Complexity ◽  
Fine Tuning ◽  
Neural Architecture ◽  
Clinical Diagnoses ◽  
High Computational Cost ◽  
Whole Slide Images

Abstract Out-of-focus sections of whole slide images are a significant source of false positives and other systematic errors in clinical diagnoses. As a result, focus quality assessment (FQA) methods must be able to quickly and accurately differentiate between focus levels in a scan. Recently, deep learning methods using convolutional neural networks (CNNs) have been adopted for FQA. However, the biggest obstacles impeding their wide usage in clinical workflows are their generalizability across different test conditions and their potentially high computational cost. In this study, we focus on the transferability and scalability of CNN-based FQA approaches. We carry out an investigation on ten architecturally diverse networks using five datasets with stain and tissue diversity. We evaluate the computational complexity of each network and scale this to realistic applications involving hundreds of whole slide images. We assess how well each full model transfers to a separate, unseen dataset without fine-tuning. We show that shallower networks transfer well when used on small input patch sizes, while deeper networks work more effectively on larger inputs. Furthermore, we introduce neural architecture search (NAS) to the field and learn an automatically designed low-complexity CNN architecture using differentiable architecture search which achieved competitive performance relative to established CNNs.

An Alternate Algorithm for (3x3) Median Filtering of Digital Images

2012 ◽  
Vol 2 (1) ◽  
pp. 7-9 ◽  
Author(s):  
Satinderjit Singh
Keyword(s):  
Median Filter ◽  
Computational Cost ◽  
Spatial Coherence ◽  
General Purpose ◽  
Median Filtering ◽  
Basic Algorithm ◽  
Temporal Complexity ◽  
Filter Kernel ◽  
One Step ◽  
High Computational Cost

Median filtering is a commonly used technique in image processing. The main problem of the median filter is its high computational cost (for sorting N pixels, the temporal complexity is O(N·log N), even with the most efficient sorting algorithms). When the median filter must be carried out in real time, the software implementation in general-purpose processorsdoes not usually give good results. This Paper presents an efficient algorithm for median filtering with a 3x3 filter kernel with only about 9 comparisons per pixel using spatial coherence between neighboring filter computations. The basic algorithm calculates two medians in one step and reuses sorted slices of three vertical neighboring pixels. An extension of this algorithm for 2D spatial coherence is also examined, which calculates four medians per step.

Methods for studying dissolved oxygen levels in coastal and estuarine waters receiving combined sewer overflows

1995 ◽  
Vol 32 (2) ◽  
pp. 95-103
Author(s):  
José A. Revilla ◽  
Kalin N. Koev ◽  
Rafael Díaz ◽  
César Álvarez ◽  
Antonio Roldán
Keyword(s):  
Dissolved Oxygen ◽  
Coastal Waters ◽  
Computational Cost ◽  
Oxygen Deficit ◽  
Alternative Methods ◽  
Coastal Zones ◽  
Combined Sewer Overflows ◽  
Sewer Systems ◽  
Combined Sewer ◽  
High Computational Cost

One factor in determining the transport capacity of coastal interceptors in Combined Sewer Systems (CSS) is the reduction of Dissolved Oxygen (DO) in coastal waters originating from the overflows. The study of the evolution of DO in coastal zones is complex. The high computational cost of using mathematical models discriminates against the required probabilistic analysis being undertaken. Alternative methods, based on such mathematical modelling, employed in a limited number of cases, are therefore needed. In this paper two alternative methods are presented for the study of oxygen deficit resulting from overflows of CSS. In the first, statistical analyses focus on the causes of the deficit (the volume discharged). The second concentrates on the effects (the concentrations of oxygen in the sea). Both methods have been applied in a study of the coastal interceptor at Pasajes Estuary (Guipúzcoa, Spain) with similar results.

scholarly journals Visualizing Profiles of Large Datasets of Weighted and Mixed Data

Mathematics ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. 891
Author(s):  
Aurea Grané ◽  
Alpha A. Sow-Barry
Keyword(s):  
Multidimensional Scaling ◽  
Random Sample ◽  
Simulation Study ◽  
Clustering Algorithm ◽  
Computational Cost ◽  
Interpolation Formula ◽  
Large Datasets ◽  
Mixed Data ◽  
Multivariate Techniques ◽  
High Computational Cost

This work provides a procedure with which to construct and visualize profiles, i.e., groups of individuals with similar characteristics, for weighted and mixed data by combining two classical multivariate techniques, multidimensional scaling (MDS) and the k-prototypes clustering algorithm. The well-known drawback of classical MDS in large datasets is circumvented by selecting a small random sample of the dataset, whose individuals are clustered by means of an adapted version of the k-prototypes algorithm and mapped via classical MDS. Gower’s interpolation formula is used to project remaining individuals onto the previous configuration. In all the process, Gower’s distance is used to measure the proximity between individuals. The methodology is illustrated on a real dataset, obtained from the Survey of Health, Ageing and Retirement in Europe (SHARE), which was carried out in 19 countries and represents over 124 million aged individuals in Europe. The performance of the method was evaluated through a simulation study, whose results point out that the new proposal solves the high computational cost of the classical MDS with low error.

Reliability and reliability-based sensitivity analysis of self-centering buckling restrained braces using meta-models

2021 ◽  
pp. 1045389X2110263
Author(s):  
Seyede Vahide Hashemi ◽  
Mahmoud Miri ◽  
Mohsen Rashki ◽  
Sadegh Etedali
Keyword(s):  
Failure Probability ◽  
Limit State ◽  
Computational Cost ◽  
Sensitivity Analyses ◽  
State Function ◽  
Reliability Indices ◽  
Buckling Restrained Brace ◽  
Polynomial Response Surface ◽  
Nonlinear Dynamic Analyses ◽  
High Computational Cost

This paper aims to carry out sensitivity analyses to study how the effect of each design variable on the performance of self-centering buckling restrained brace (SC-BRB) and the corresponding buckling restrained brace (BRB) without shape memory alloy (SMA) rods. Furthermore, the reliability analyses of BRB and SC-BRB are performed in this study. Considering the high computational cost of the simulation methods, three Meta-models including the Kriging, radial basis function (RBF), and polynomial response surface (PRSM) are utilized to construct the surrogate models. For this aim, the nonlinear dynamic analyses are conducted on both BRB and SC-BRB by using OpenSees software. The results showed that the SMA area, SMA length ratio, and BRB core area have the most effect on the failure probability of SC-BRB. It is concluded that Kriging-based Monte Carlo Simulation (MCS) gives the best performance to estimate the limit state function (LSF) of BRB and SC-BRB in the reliability analysis procedures. Considering the effects of changing the maximum cyclic loading on the failure probability computation and comparison of the failure probability for different LSFs, it is also found that the reliability indices of SC-BRB were always higher than the corresponding reliability indices determined for BRB which confirms the performance superiority of SC-BRB than BRB.

scholarly journals An efficient pruning scheme of deep neural networks for Internet of Things applications

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Chen Qi ◽  
Shibo Shen ◽  
Rongpeng Li ◽  
Zhifeng Zhao ◽  
Qing Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Internet Of Things ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Superior Performance ◽  
Compact Structure ◽  
Resource Limited ◽  
Benchmark Datasets ◽  
Iot Devices

AbstractNowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalities like sensing, imaging, classification, recognition, etc. However, the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things (IoT) devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs, by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rate directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (CNN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based IoT framework and establish distributed training of neural networks in both cloud and edge.

scholarly journals Efficient Binarized Convolutional Layers for Visual Inspection Applications on Resource-Limited FPGAs and ASICs

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1511
Author(s):  
Taylor Simons ◽  
Dah-Jye Lee
Keyword(s):  
Neural Networks ◽  
Visual Inspection ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Quality Inspection ◽  
Agricultural Produce ◽  
Resource Limited ◽  
Inspection Tasks ◽  
Computational Resources ◽  
Small Models

There has been a recent surge in publications related to binarized neural networks (BNNs), which use binary values to represent both the weights and activations in deep neural networks (DNNs). Due to the bitwise nature of BNNs, there have been many efforts to implement BNNs on ASICs and FPGAs. While BNNs are excellent candidates for these kinds of resource-limited systems, most implementations still require very large FPGAs or CPU-FPGA co-processing systems. Our work focuses on reducing the computational cost of BNNs even further, making them more efficient to implement on FPGAs. We target embedded visual inspection tasks, like quality inspection sorting on manufactured parts and agricultural produce sorting. We propose a new binarized convolutional layer, called the neural jet features layer, that learns well-known classic computer vision kernels that are efficient to calculate as a group. We show that on visual inspection tasks, neural jet features perform comparably to standard BNN convolutional layers while using less computational resources. We also show that neural jet features tend to be more stable than BNN convolution layers when training small models.

scholarly journals Non-parallel dictionary learning for voice conversion using non-negative Tucker decomposition

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Yuki Takashima ◽  
Toru Nakashika ◽  
Tetsuya Takiguchi ◽  
Yasuo Ariki
Keyword(s):  
Dictionary Learning ◽  
Computational Cost ◽  
Tensor Decomposition ◽  
Gaussian Mixture ◽  
Voice Conversion ◽  
Specific Information ◽  
Learning Method ◽  
Tucker Decomposition ◽  
Parallel Data ◽  
High Computational Cost

Abstract Voice conversion (VC) is a technique of exclusively converting speaker-specific information in the source speech while preserving the associated phonemic information. Non-negative matrix factorization (NMF)-based VC has been widely researched because of the natural-sounding voice it achieves when compared with conventional Gaussian mixture model-based VC. In conventional NMF-VC, models are trained using parallel data which results in the speech data requiring elaborate pre-processing to generate parallel data. NMF-VC also tends to be an extensive model as this method has several parallel exemplars for the dictionary matrix, leading to a high computational cost. In this study, an innovative parallel dictionary-learning method using non-negative Tucker decomposition (NTD) is proposed. The proposed method uses tensor decomposition and decomposes an input observation into a set of mode matrices and one core tensor. The proposed NTD-based dictionary-learning method estimates the dictionary matrix for NMF-VC without using parallel data. The experimental results show that the proposed method outperforms other methods in both parallel and non-parallel settings.

scholarly journals A NOTE ON PHASING LONG GENOMIC REGIONS USING LOCAL HAPLOTYPE PREDICTIONS

2006 ◽  
Vol 04 (03) ◽  
pp. 639-647 ◽  
Author(s):  
ELEAZAR ESKIN ◽  
RODED SHARAN ◽  
ERAN HALPERIN
Keyword(s):  
Large Scale ◽  
Computational Cost ◽  
Nucleotide Polymorphisms ◽  
Single Nucleotide ◽  
Novel Approach ◽  
Maximum Likelihood Criterion ◽  
The Common ◽  
Genomic Regions ◽  
High Computational Cost ◽  
Combining Information

The common approaches for haplotype inference from genotype data are targeted toward phasing short genomic regions. Longer regions are often tackled in a heuristic manner, due to the high computational cost. Here, we describe a novel approach for phasing genotypes over long regions, which is based on combining information from local predictions on short, overlapping regions. The phasing is done in a way, which maximizes a natural maximum likelihood criterion. Among other things, this criterion takes into account the physical length between neighboring single nucleotide polymorphisms. The approach is very efficient and is applied to several large scale datasets and is shown to be successful in two recent benchmarking studies (Zaitlen et al., in press; Marchini et al., in preparation). Our method is publicly available via a webserver at .

Sign in / Sign up

Export Citation Format

Share Document