scholarly journals ResMem-Net: memory based deep CNN for image memorability estimation

2021 ◽  
Vol 7 ◽  
pp. e767
Author(s):  
Arockia Praveen ◽  
Abdulfattah Noorwali ◽  
Duraimurugan Samiayya ◽  
Mohammad Zubair Khan ◽  
Durai Raj Vincent P M ◽  
...  

Image memorability is a very hard problem in image processing due to its subjective nature. But due to the introduction of Deep Learning and the large availability of data and GPUs, great strides have been made in predicting the memorability of an image. In this paper, we propose a novel deep learning architecture called ResMem-Net that is a hybrid of LSTM and CNN that uses information from the hidden layers of the CNN to compute the memorability score of an image. The intermediate layers are important for predicting the output because they contain information about the intrinsic properties of the image. The proposed architecture automatically learns visual emotions and saliency, shown by the heatmaps generated using the GradRAM technique. We have also used the heatmaps and results to analyze and answer one of the most important questions in image memorability: “What makes an image memorable?”. The model is trained and evaluated using the publicly available Large-scale Image Memorability dataset (LaMem) from MIT. The results show that the model achieves a rank correlation of 0.679 and a mean squared error of 0.011, which is better than the current state-of-the-art models and is close to human consistency (p = 0.68). The proposed architecture also has a significantly low number of parameters compared to the state-of-the-art architecture, making it memory efficient and suitable for production.

2021 ◽  
Vol 13 (22) ◽  
pp. 4599
Author(s):  
Félix Quinton ◽  
Loic Landrieu

While annual crop rotations play a crucial role for agricultural optimization, they have been largely ignored for automated crop type mapping. In this paper, we take advantage of the increasing quantity of annotated satellite data to propose to model simultaneously the inter- and intra-annual agricultural dynamics of yearly parcel classification with a deep learning approach. Along with simple training adjustments, our model provides an improvement of over 6.3% mIoU over the current state-of-the-art of crop classification, and a reduction of over 21% of the error rate. Furthermore, we release the first large-scale multi-year agricultural dataset with over 300,000 annotated parcels.


Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1459
Author(s):  
Mirela Kundid Vasić ◽  
Vladan Papić

In this paper, we propose a novel method for person detection in aerial images of nonurban terrain gathered by an Unmanned Aerial Vehicle (UAV), which plays an important role in Search And Rescue (SAR) missions. The UAV in SAR operations contributes significantly due to the ability to survey a larger geographical area from an aerial viewpoint. Because of the high altitude of recording, the object of interest (person) covers a small part of an image (around 0.1%), which makes this task quite challenging. To address this problem, a multimodel deep learning approach is proposed. The solution consists of two different convolutional neural networks in region proposal, as well as in the classification stage. Additionally, contextual information is used in the classification stage in order to improve the detection results. Experimental results tested on the HERIDAL dataset achieved precision of 68.89% and a recall of 94.65%, which is better than current state-of-the-art methods used for person detection in similar scenarios. Consequently, it may be concluded that this approach is suitable for usage as an auxiliary method in real SAR operations.


Author(s):  
Honegzhe Liu ◽  
Zhifang Deng ◽  
Cheng Xu

Gesture recognition aims at understanding dynamic gestures of the human body and is one of the most important ways of human–computer interaction; to extract more effective spatiotemporal features in gesture videos for more accurate gesture classification, a novel feature extractor network, spatiotemporal attention 3D DenseNet is proposed in this study. We extend DenseNet with 3D kernels and Refined Temporal Transition Layer based on Temporal Transition Layer, and we also explore attention mechanism in 3D ConvNets. We embed the Refined Temporal Transition Layer and attention mechanism in DenseNet3D, named the proposed network “spatiotemporal attention 3D DenseNet.” Our experiments show that our Refined Temporal Transition Layer performs better than Temporal Transition Layer and the proposed spatiotemporal attention 3D DenseNet in each modality outperforms the current state-of-the-art methods on the ChaLearn LAP Large-Scale Isolated gesture dataset. The code and pretrained model are released in https://github.com/dzf19927/STA3D .


1988 ◽  
Vol 135 ◽  
Author(s):  
Michael M Thackeray

AbstractConsiderable efforts are in progress to develop rechargeable batteries as alternative systems to the nickel-cadmium battery. In this regard, several advances have been made in ambient-temperature lithium battery technology, and specifically in the engineering of rechargeable lithium/manganese dioxide cells. This paper reviews the current state of the art in rechargeable Li/MnO2battery technology; particular attention is paid to the structural features of various MnO2electrode materials which influence their electrochemical and cycling behaviour in lithium cells.


Author(s):  
Yuheng Hu ◽  
Yili Hong

Residents often rely on newspapers and television to gather hyperlocal news for community awareness and engagement. More recently, social media have emerged as an increasingly important source of hyperlocal news. Thus far, the literature on using social media to create desirable societal benefits, such as civic awareness and engagement, is still in its infancy. One key challenge in this research stream is to timely and accurately distill information from noisy social media data streams to community members. In this work, we develop SHEDR (social media–based hyperlocal event detection and recommendation), an end-to-end neural event detection and recommendation framework with a particular use case for Twitter to facilitate residents’ information seeking of hyperlocal events. The key model innovation in SHEDR lies in the design of the hyperlocal event detector and the event recommender. First, we harness the power of two popular deep neural network models, the convolutional neural network (CNN) and long short-term memory (LSTM), in a novel joint CNN-LSTM model to characterize spatiotemporal dependencies for capturing unusualness in a region of interest, which is classified as a hyperlocal event. Next, we develop a neural pairwise ranking algorithm for recommending detected hyperlocal events to residents based on their interests. To alleviate the sparsity issue and improve personalization, our algorithm incorporates several types of contextual information covering topic, social, and geographical proximities. We perform comprehensive evaluations based on two large-scale data sets comprising geotagged tweets covering Seattle and Chicago. We demonstrate the effectiveness of our framework in comparison with several state-of-the-art approaches. We show that our hyperlocal event detection and recommendation models consistently and significantly outperform other approaches in terms of precision, recall, and F-1 scores. Summary of Contribution: In this paper, we focus on a novel and important, yet largely underexplored application of computing—how to improve civic engagement in local neighborhoods via local news sharing and consumption based on social media feeds. To address this question, we propose two new computational and data-driven methods: (1) a deep learning–based hyperlocal event detection algorithm that scans spatially and temporally to detect hyperlocal events from geotagged Twitter feeds; and (2) A personalized deep learning–based hyperlocal event recommender system that systematically integrates several contextual cues such as topical, geographical, and social proximity to recommend the detected hyperlocal events to potential users. We conduct a series of experiments to examine our proposed models. The outcomes demonstrate that our algorithms are significantly better than the state-of-the-art models and can provide users with more relevant information about the local neighborhoods that they live in, which in turn may boost their community engagement.


Author(s):  
Jwalin Bhatt ◽  
Khurram Azeem Hashmi ◽  
Muhammad Zeshan Afzal ◽  
Didier Stricker

In any document, graphical elements like tables, figures, and formulas contain essential information. The processing and interpretation of such information require specialized algorithms. Off-the-shelf OCR components cannot process this information reliably. Therefore, an essential step in document analysis pipelines is to detect these graphical components. It leads to a high-level conceptual understanding of the documents that makes digitization of documents viable. Since the advent of deep learning, the performance of deep learning-based object detection has improved many folds. In this work, we outline and summarize the deep learning approaches for detecting graphical page objects in the document images. Therefore, we discuss the most relevant deep learning-based approaches and state-of-the-art graphical page object detection in document images. This work provides a comprehensive understanding of the current state-of-the-art and related challenges. Furthermore, we discuss leading datasets along with the quantitative evaluation. Moreover, it discusses briefly the promising directions that can be utilized for further improvements.


2021 ◽  
Vol 43 ◽  
pp. e58283
Author(s):  
Clístenes Williams Araújo do Nascimento ◽  
Caroline Miranda Biondi ◽  
Fernando Bruno Vieira da Silva ◽  
Luiz Henrique Vieira Lima

Soil contamination by metals threatens both the environment and human health and hence requires remedial actions. The conventional approach of removing polluted soils and replacing them with clean soils (excavation) is very costly for low-value sites and not feasible on a large scale. In this scenario, phytoremediation emerged as a promising cost-effective and environmentally-friendly technology to render metals less bioavailable (phytostabilization) or clean up metal-polluted soils (phytoextraction). Phytostabilization has demonstrable successes in mining sites and brownfields. On the other hand, phytoextraction still has few examples of successful applications. Either by using hyperaccumulating plants or high biomass plants induced to accumulate metals through chelator addition to the soil, major phytoextraction bottlenecks remain, mainly the extended time frame to remediation and lack of revenue from the land during the process. Due to these drawbacks, phytomanagement has been proposed to provide economic, environmental, and social benefits until the contaminated site returns to productive usage. Here, we review the evolution, promises, and limitations of these phytotechnologies. Despite the lack of commercial phytoextraction operations, there have been significant advances in understanding phytotechnologies' main constraints. Further investigation on new plant species, especially in the tropics, and soil amendments can potentially provide the basis to transform phytoextraction into an operational metal clean-up technology in the future. However, at the current state of the art, phytotechnology is moving the focus from remediation technologies to pollution attenuation and palliative cares.


2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Juan F. Ramirez Rochac ◽  
Nian Zhang ◽  
Lara A. Thompson ◽  
Tolessa Deksissa

Hyperspectral imaging is an area of active research with many applications in remote sensing, mineral exploration, and environmental monitoring. Deep learning and, in particular, convolution-based approaches are the current state-of-the-art classification models. However, in the presence of noisy hyperspectral datasets, these deep convolutional neural networks underperform. In this paper, we proposed a feature augmentation approach to increase noise resistance in imbalanced hyperspectral classification. Our method calculates context-based features, and it uses a deep convolutional neuronet (DCN). We tested our proposed approach on the Pavia datasets and compared three models, DCN, PCA + DCN, and our context-based DCN, using the original datasets and the datasets plus noise. Our experimental results show that DCN and PCA + DCN perform well on the original datasets but not on the noisy datasets. Our robust context-based DCN was able to outperform others in the presence of noise and was able to maintain a comparable classification accuracy on clean hyperspectral images.


2023 ◽  
Vol 55 (1) ◽  
pp. 1-39
Author(s):  
Thanh Tuan Nguyen ◽  
Thanh Phuong Nguyen

Representing dynamic textures (DTs) plays an important role in many real implementations in the computer vision community. Due to the turbulent and non-directional motions of DTs along with the negative impacts of different factors (e.g., environmental changes, noise, illumination, etc.), efficiently analyzing DTs has raised considerable challenges for the state-of-the-art approaches. For 20 years, many different techniques have been introduced to handle the above well-known issues for enhancing the performance. Those methods have shown valuable contributions, but the problems have been incompletely dealt with, particularly recognizing DTs on large-scale datasets. In this article, we present a comprehensive taxonomy of DT representation in order to purposefully give a thorough overview of the existing methods along with overall evaluations of their obtained performances. Accordingly, we arrange the methods into six canonical categories. Each of them is then taken in a brief presentation of its principal methodology stream and various related variants. The effectiveness levels of the state-of-the-art methods are then investigated and thoroughly discussed with respect to quantitative and qualitative evaluations in classifying DTs on benchmark datasets. Finally, we point out several potential applications and the remaining challenges that should be addressed in further directions. In comparison with two existing shallow DT surveys (i.e., the first one is out of date as it was made in 2005, while the newer one (published in 2016) is an inadequate overview), we believe that our proposed comprehensive taxonomy not only provides a better view of DT representation for the target readers but also stimulates future research activities.


Author(s):  
Arun M. Puthanpurayil ◽  
Rajesh P Dhakal ◽  
Athol J. Carr

A consolidated review of the current-state-of-the-art on optimal damper positioning techniques is presented in this chapter. The inherent assumptions made in previous research are discussed and substantiated with numerical studies. Earlier studies have shown that optimal distribution of dampers is sensitive to in-structure damping. In this chapter the significance of optimal distribution of dampers coupled with the necessity for the use of a more realistic in-structure damping model is qualitatively illustrated using a comparative sensitivity study. The effect of inherent assumption of linearity of the parent frame on the ‘optimality’ is also investigated. It is shown that linearity assumption imposed on the parent frame in a major seismic event may not be justified; thereby raising doubts on the scope of optimality techniques proposed in literature.


Sign in / Sign up

Export Citation Format

Share Document