scholarly journals Subjectively Measured Streetscape Qualities for Shanghai with Large-Scale Application of Computer Vision and Machine Learning

2021 ◽  
pp. 242-251
Author(s):  
Waishan Qiu ◽  
Wenjing Li ◽  
Xun Liu ◽  
Xiaokai Huang
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Urban Design ◽  
Design Theory ◽  
Large Scale ◽  
Image Features ◽  
Subjective Measures ◽  
Visual Elements ◽  
Urban Amenities ◽  
Human Perceptions

AbstractRecently, many new studies emerged to apply computer vision (CV) to street view imagery (SVI) dataset to objectively extract the view indices of various streetscape features such as trees to proxy urban scene qualities. However, human perceptions (e.g., imageability) have a subtle relationship to visual elements which cannot be fully captured using view indices. Conversely, subjective measures using survey and interview data explain more human behaviors. However, the effectiveness of integrating subjective measures with SVI dataset has been less discussed. To address this, we integrated crowdsourcing, CV, and machine learning (ML) to subjectively measure four important perceptions suggested by classical urban design theory. We first collected experts’ rating on sample SVIs regarding the four qualities which became the training labels. CV segmentation was applied to SVI samples extracting streetscape view indices as the explanatory variables. We then trained ML models and achieved high accuracy in predicting the scores. We found a strong correlation between predicted complexity score and the density of urban amenities and services Point of Interests (POI), which validates the effectiveness of subjective measures. In addition, to test the generalizability of the proposed framework as well as to inform urban renewal strategies, we compared the measured qualities in Pudong to other five renowned urban cores worldwide. Rather than predicting perceptual scores directly from generic image features using convolution neural network, our approach follows what urban design theory suggested and confirms various streetscape features affecting multi-dimensional human perceptions. Therefore, its result provides more interpretable and actionable implications for policymakers and city planners.

scholarly journals Subjectively Measured Streetscape Perceptions to Inform Urban Design Strategies for Shanghai

2021 ◽  
Vol 10 (8) ◽  
pp. 493
Author(s):  
Waishan Qiu ◽  
Wenjing Li ◽  
Xun Liu ◽  
Xiaokai Huang
Keyword(s):  
Urban Design ◽  
Design Theory ◽  
Human Perception ◽  
Image Features ◽  
Design Strategies ◽  
Subjective Measures ◽  
Explanatory Variables ◽  
Visual Elements ◽  
Urban Amenities ◽  
Human Perceptions

Recently, many new studies applying computer vision (CV) to street view imagery (SVI) datasets to objectively extract the view indices of various streetscape features such as trees to proxy urban scene qualities have emerged. However, human perception (e.g., imageability) have a subtle relationship to visual elements that cannot be fully captured using view indices. Conversely, subjective measures using survey and interview data explain human behaviors more. However, the effectiveness of integrating subjective measures with SVI datasets has been less discussed. To address this, we integrated crowdsourcing, CV, and machine learning (ML) to subjectively measure four important perceptions suggested by classical urban design theory. We first collected ratings from experts on sample SVIs regarding these four qualities, which became the training labels. CV segmentation was applied to SVI samples extracting streetscape view indices as the explanatory variables. We then trained ML models and achieved high accuracy in predicting scores. We found a strong correlation between the predicted complexity score and the density of urban amenities and services points of interest (POI), which validates the effectiveness of subjective measures. In addition, to test the generalizability of the proposed framework as well as to inform urban renewal strategies, we compared the measured qualities in Pudong to other five urban cores that are renowned worldwide. Rather than predicting perceptual scores directly from generic image features using a convolution neural network, our approach follows what urban design theory has suggested and confirmed as various streetscape features affecting multi-dimensional human perceptions. Therefore, the results provide more interpretable and actionable implications for policymakers and city planners.

scholarly journals Time-lapse imagery is cheap and timely in the fight against colonial species' decline

2021 ◽  
Author(s):  
Tom Hart ◽  
Fiona Jones ◽  
Caitlin Black ◽  
Chris Lintott ◽  
Casey Youngflesh ◽  
...  
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Parameter Estimation ◽  
Citizen Science ◽  
Large Scale ◽  
Individual Recognition ◽  
Spatial Scales ◽  
Time Lapse ◽  
Machine Learning Algorithms ◽  
Series Data

Many of the species in decline around the world are subject to different environmental stressors across their range, so replicated large-scale monitoring programmes, are necessary to disentangle the relative impacts of these threats. At the same time as funding for long-term monitoring is being cut, studies are increasingly being criticised for lacking statistical power. For those taxa or environments where a single vantage point can observe individuals or ecological processes, time-lapse cameras can provide a cost-effective way of collecting time series data replicated at large spatial scales that would otherwise be impossible. However, networks of time-lapse cameras needed to cover the range of species or processes create a problem in that the scale of data collection easily exceeds our ability to process the raw imagery manually. Citizen science and machine learning provide solutions to scaling up data extraction (such as locating all animals in an image). Crucially, citizen science, machine learning-derived classifiers, and the intersection between them, are key to understanding how to establish monitoring systems that are sensitive to – and sufficiently powerful to detect –changes in the study system. Citizen science works relatively ‘out of the box’, and we regard it as a first step for many systems until machine learning algorithms are sufficiently trained to automate the process. Using Penguin Watch (www.penguinwatch.org) data as a case study, we discuss a complete workflow from images to parameter estimation and interpretation: the use of citizen science and computer vision for image processing, and parameter estimation and individual recognition for investigating biological questions. We discuss which techniques are easily generalizable to a range of questions, and where more work is needed to supplement ‘out of the box’ tools. We conclude with a horizon scan of the advances in camera technology, such as on-board computer vision and decision making.

scholarly journals Computer Vision Classification of Barley Flour Based on Spatial Pyramid Partition Ensemble

Sensors ◽  
2019 ◽  
Vol 19 (13) ◽  
pp. 2953 ◽  
Author(s):  
Jessica Fernandes Lopes ◽  
Leniza Ludwig ◽  
Douglas Fernandes Barbin ◽  
Maria Victória Eiras Grossmann ◽  
Sylvio Barbon
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Image Features ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Human Consumption ◽  
Barley Flour ◽  
Quality Aspects ◽  
Spatial Pyramid

Imaging sensors are largely employed in the food processing industry for quality control. Flour from malting barley varieties is a valuable ingredient in the food industry, but its use is restricted due to quality aspects such as color variations and the presence of husk fragments. On the other hand, naked varieties present superior quality with better visual appearance and nutritional composition for human consumption. Computer Vision Systems (CVS) can provide an automatic and precise classification of samples, but identification of grain and flour characteristics require more specialized methods. In this paper, we propose CVS combined with the Spatial Pyramid Partition ensemble (SPPe) technique to distinguish between naked and malting types of twenty-two flour varieties using image features and machine learning. SPPe leverages the analysis of patterns from different spatial regions, providing more reliable classification. Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), J48 decision tree, and Random Forest (RF) were compared for samples’ classification. Machine learning algorithms embedded in the CVS were induced based on 55 image features. The results ranged from 75.00% (k-NN) to 100.00% (J48) accuracy, showing that sample assessment by CVS with SPPe was highly accurate, representing a potential technique for automatic barley flour classification.

scholarly journals A quantum-inspired classifier for clonogenic assay evaluations

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Giuseppe Sergioli ◽  
Carmelo Militello ◽  
Leonardo Rundo ◽  
Luigi Minafra ◽  
Filippo Torrisi ◽  
...  
Keyword(s):  
Machine Learning ◽  
Information Theory ◽  
Quantum Information ◽  
Time Complexity ◽  
Large Scale ◽  
Clonogenic Assay ◽  
Expressive Power ◽  
Quantum Information Theory ◽  
Image Features ◽  
Discriminative Feature

AbstractRecent advances in Quantum Machine Learning (QML) have provided benefits to several computational processes, drastically reducing the time complexity. Another approach of combining quantum information theory with machine learning—without involving quantum computers—is known as Quantum-inspired Machine Learning (QiML), which exploits the expressive power of the quantum language to increase the accuracy of the process (rather than reducing the time complexity). In this work, we propose a large-scale experiment based on the application of a binary classifier inspired by quantum information theory to the biomedical imaging context in clonogenic assay evaluation to identify the most discriminative feature, allowing us to enhance cell colony segmentation. This innovative approach offers a two-fold result: (1) among the extracted and analyzed image features, homogeneity is shown to be a relevant feature in detecting challenging cell colonies; and (2) the proposed quantum-inspired classifier is a novel and outstanding methodology, compared to conventional machine learning classifiers, for the evaluation of clonogenic assays.

scholarly journals A New Data-Enabled Intelligence Framework for Evaluating Urban Space Perception

2021 ◽  
Vol 10 (6) ◽  
pp. 400
Author(s):  
Haohao Ji ◽  
Linbo Qing ◽  
Longmei Han ◽  
Zhengyong Wang ◽  
Yongqiang Cheng ◽  
...  
Keyword(s):  
Urban Planning ◽  
Built Environment ◽  
Urban Environment ◽  
Urban Space ◽  
Large Scale ◽  
Urban Region ◽  
Economic Data ◽  
Economic Dimension ◽  
Visual Elements ◽  
Human Perceptions

The urban environment has a great impact on the wellbeing of citizens and it is of great significance to understand how citizens perceive and evaluate places in a large scale urban region and to provide scientific evidence to support human-centered urban planning with a better urban environment. Existing studies for assessing urban perception have primarily relied on low efficiency methods, which also result in low evaluation accuracy. Furthermore, there lacks a sophisticated understanding on how to correlate the urban perception with the built environment and other socio-economic data, which limits their applications in supporting urban planning. In this study, a new data-enabled intelligence framework for evaluating human perceptions of urban space is proposed. Specifically, a novel classification-then-regression strategy based on a deep convolutional neural network and a random-forest algorithm is proposed. The proposed approach has been applied to evaluate the perceptions of Beijing and Chengdu against six perceptual criteria. Meanwhile, multi-source data were employed to investigate the associations between human perceptions and the indicators for the built environment and socio-economic data including visual elements, facility attributes and socio-economic indicators. Experimental results show that the proposed framework can effectively evaluate urban perceptions. The associations between urban perceptions and the visual elements, facility attributes and a socio-economic dimension have also been identified, which can provide substantial inputs to guide the urban planning for a better urban space.

scholarly journals New Directions in Streaming Algorithms

2020 ◽  
Author(s):  
Ali Vakilian
Keyword(s):  
Machine Learning ◽  
Combinatorial Optimization ◽  
Design Theory ◽  
Large Scale ◽  
Computational Models ◽  
Optimization Problems ◽  
Low Rank ◽  
Set Cover ◽  
Streaming Algorithms ◽  
Combinatorial Optimization Problems

Large volumes of available data have led to the emergence of new computational models for data analysis. One such model is captured by the notion of streaming algorithms: given a sequence of N items, the goal is to compute the value of a given function of the input items by a small number of passes and using a sublinear amount of space in N. Streaming algorithms have applications in many areas such as networking and large scale machine learning. Despite a huge amount of work on this area over the last two decades, there are multiple aspects of streaming algorithms that remained poorly understood, such as (a) streaming algorithms for combinatorial optimization problems and (b) incorporating modern machine learningtechniques in the design of streaming algorithms. In the first part of this thesis, we will describe (essentially) optimal streaming algorithms for set cover and maximum coverage, two classic problems in combinatorial optimization. Next, in the second part, we will show how to augment classic streaming algorithms of the frequency estimation and low-rank approximation problems with machine learning oracles in order to improve their space-accuracy tradeoffs. The new algorithms combine the benefits of machine learning with the formal guarantees available through algorithm design theory.

scholarly journals Inspecting Buildings Using Drones and Computer Vision: A Machine Learning Approach to Detect Cracks and Damages

Drones ◽  
2021 ◽  
Vol 6 (1) ◽  
pp. 5
Author(s):  
Hafiz Suliman Munawar ◽  
Fahim Ullah ◽  
Amirhossein Heravi ◽  
Muhammad Jamaluddin Thaheem ◽  
Ahsen Maqsoom
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Crack Detection ◽  
Conditional Random Fields ◽  
Image Features ◽  
Superior Performance ◽  
Training Procedure ◽  
Generative Adversarial Network ◽  
Hybrid Approaches ◽  
Deep Crack

Manual inspection of infrastructure damages such as building cracks is difficult due to the objectivity and reliability of assessment and high demands of time and costs. This can be automated using unmanned aerial vehicles (UAVs) for aerial imagery of damages. Numerous computer vision-based approaches have been applied to address the limitations of crack detection but they have their limitations that can be overcome by using various hybrid approaches based on artificial intelligence (AI) and machine learning (ML) techniques. The convolutional neural networks (CNNs), an application of the deep learning (DL) method, display remarkable potential for automatically detecting image features such as damages and are less sensitive to image noise. A modified deep hierarchical CNN architecture has been used in this study for crack detection and damage assessment in civil infrastructures. The proposed architecture is based on 16 convolution layers and a cycle generative adversarial network (CycleGAN). For this study, the crack images were collected using UAVs and open-source images of mid to high rise buildings (five stories and above) constructed during 2000 in Sydney, Australia. Conventionally, a CNN network only utilizes the last layer of convolution. However, our proposed network is based on the utility of multiple layers. Another important component of the proposed CNN architecture is the application of guided filtering (GF) and conditional random fields (CRFs) to refine the predicted outputs to get reliable results. Benchmarking data (600 images) of Sydney-based buildings damages was used to test the proposed architecture. The proposed deep hierarchical CNN architecture produced superior performance when evaluated using five methods: GF method, Baseline (BN) method, Deep-Crack BN, Deep-Crack GF, and SegNet. Overall, the GF method outperformed all other methods as indicated by the global accuracy (0.990), class average accuracy (0.939), mean intersection of the union overall classes (IoU) (0.879), precision (0.838), recall (0.879), and F-score (0.8581) values. Overall, the proposed CNN architecture provides the advantages of reduced noise, highly integrated supervision of features, adequate learning, and aggregation of both multi-scale and multilevel features during the training procedure along with the refinement of the overall output predictions.

scholarly journals Computer vision cracks the leaf code

2016 ◽  
Vol 113 (12) ◽  
pp. 3305-3310 ◽  
Author(s):  
Peter Wilf ◽  
Shengping Zhang ◽  
Sharat Chikkerur ◽  
Stefan A. Little ◽  
Scott L. Wing ◽  
...  
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Leaf Shape ◽  
Morphological Characters ◽  
Automated System ◽  
Individual Species ◽  
Diagnostic Features ◽  
Visual Elements ◽  
Vision Algorithm ◽  
Open Question

Understanding the extremely variable, complex shape and venation characters of angiosperm leaves is one of the most challenging problems in botany. Machine learning offers opportunities to analyze large numbers of specimens, to discover novel leaf features of angiosperm clades that may have phylogenetic significance, and to use those characters to classify unknowns. Previous computer vision approaches have primarily focused on leaf identification at the species level. It remains an open question whether learning and classification are possible among major evolutionary groups such as families and orders, which usually contain hundreds to thousands of species each and exhibit many times the foliar variation of individual species. Here, we tested whether a computer vision algorithm could use a database of 7,597 leaf images from 2,001 genera to learn features of botanical families and orders, then classify novel images. The images are of cleared leaves, specimens that are chemically bleached, then stained to reveal venation. Machine learning was used to learn a codebook of visual elements representing leaf shape and venation patterns. The resulting automated system learned to classify images into families and orders with a success rate many times greater than chance. Of direct botanical interest, the responses of diagnostic features can be visualized on leaf images as heat maps, which are likely to prompt recognition and evolutionary interpretation of a wealth of novel morphological characters. With assistance from computer vision, leaves are poised to make numerous new contributions to systematic and paleobotanical studies.

scholarly journals How to Represent Paintings: A Painting Classification Using Artistic Comments

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 1940
Author(s):  
Wentao Zhao ◽  
Dalin Zhou ◽  
Xinguo Qiu ◽  
Wei Jiang
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Language Processing ◽  
Large Scale ◽  
Machine Learning Techniques ◽  
Topological Graph ◽  
Convolutional Network ◽  
Learning Techniques ◽  
Classification Tasks ◽  
State Of Art

The goal of large-scale automatic paintings analysis is to classify and retrieve images using machine learning techniques. The traditional methods use computer vision techniques on paintings to enable computers to represent the art content. In this work, we propose using a graph convolutional network and artistic comments rather than the painting color to classify type, school, timeframe and author of the paintings by implementing natural language processing (NLP) techniques. First, we build a single artistic comment graph based on co-occurrence relations and document word relations and then train an art graph convolutional network (ArtGCN) on the entire corpus. The nodes, which include the words and documents in the topological graph are initialized using a one-hot representation; then, the embeddings are learned jointly for both words and documents, supervised by the known-class training labels of the paintings. Through extensive experiments on different classification tasks using different input sources, we demonstrate that the proposed methods achieve state-of-art performance. In addition, ArtGCN can learn word and painting embeddings, and we find that they have a major role in describing the labels and retrieval paintings, respectively.

Image Classification and Retrieval with Kernel Methods

2011 ◽  
pp. 325-345
Author(s):  
Francesca Odone ◽  
Alessandro Verri
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Image Classification ◽  
Kernel Methods ◽  
Image Features ◽  
Kernel Functions ◽  
Current Work ◽  
Classification Problems ◽  
Image Representations ◽  
Open Issues

In this chapter we review some kernel methods useful for image classification and retrieval applications. Starting from the problem of constructing appropriate image representations, we describe in depth and comment on the main properties of various kernel engineering approaches that have been recently proposed in the computer vision and machine learning literature for solving a number of image classification problems. We distinguish between kernel functions applied to images as a whole and kernel functions looking at image features. We conclude by presenting some current work and discussing open issues.

Sign in / Sign up

Export Citation Format

Share Document