Assessing citizen science data quality: an invasive species case study

Over the last decade, hundreds of thousands of volunteers have contributed to science by collecting or analyzing data. This public participation in science, also known as citizen science, has contributed to significant discoveries and led to publications in major scientific journals. However, little attention has been paid to data quality issues. In this work we argue that being able to determine the accuracy of data obtained by crowdsourcing is a fundamental question and we point out that, for many real-life scenarios, mathematical tools and processes for the evaluation of data quality are missing. We propose a probabilistic methodology for the evaluation of the accuracy of labeling data obtained by crowdsourcing in citizen science. The methodology builds on an abstract probabilistic graphical model formalism, which is shown to generalize some already existing label aggregation models. We show how to make practical use of the methodology through a comparison of data obtained from different citizen science communities analyzing the earthquake that took place in Albania in 2019.

Download Full-text

Using citizen science data to monitor the Sustainable Development Goals: a bottom-up analysis

Sustainability Science ◽

10.1007/s11625-021-01001-1 ◽

2021 ◽

Author(s):

Laura Ballerini ◽

Sylvia I. Bergh

Keyword(s):

Sustainable Development ◽

Citizen Science ◽

Sustainable Development Goals ◽

Comparative Case Study ◽

Science Data ◽

The Sustainable Development ◽

Case Study Analysis ◽

Wide Range ◽

Development Goals

AbstractOfficial data are not sufficient for monitoring the United Nations Sustainable Development Goals (SDGs): they do not reach remote locations or marginalized populations and can be manipulated by governments. Citizen science data (CSD), defined as data that citizens voluntarily gather by employing a wide range of technologies and methodologies, could help to tackle these problems and ultimately improve SDG monitoring. However, the link between CSD and the SDGs is still understudied. This article aims to develop an empirical understanding of the CSD-SDG link by focusing on the perspective of projects which employ CSD. Specifically, the article presents primary and secondary qualitative data collected on 30 of these projects and an explorative comparative case study analysis. It finds that projects which use CSD recognize that the SDGs can provide a valuable framework and legitimacy, as well as attract funding, visibility, and partnerships. But, at the same time, the article reveals that these projects also encounter several barriers with respect to the SDGs: a widespread lack of knowledge of the goals, combined with frustration and political resistance towards the UN, may deter these projects from contributing their data to the SDG monitoring apparatus.

Download Full-text

The impact of data quality filtering of opportunistic citizen science data on species distribution model performance

Ecological Modelling ◽

10.1016/j.ecolmodel.2021.109453 ◽

2021 ◽

Vol 444 ◽

pp. 109453

Author(s):

Camille Van Eupen ◽

Dirk Maes ◽

Marc Herremans ◽

Kristijn R.R. Swinnen ◽

Ben Somers ◽

...

Keyword(s):

Data Quality ◽

Citizen Science ◽

Species Distribution ◽

Species Distribution Model ◽

Model Performance ◽

Distribution Model ◽

Science Data ◽

Quality Filtering ◽

The Impact

Download Full-text

The Verification of Ecological Citizen Science Data: Current approaches and future possibilities

Biodiversity Information Science and Standards ◽

10.3897/biss.5.75506 ◽

2021 ◽

Vol 5 ◽

Author(s):

Emily Baker ◽

Jonathan Drury ◽

Johanna Judge ◽

David Roy ◽

Graham Smith ◽

...

Keyword(s):

Data Quality ◽

Citizen Science ◽

Applied Research ◽

Automated Verification ◽

Ecological Data ◽

Science Data ◽

Spatial And Temporal Scales ◽

The Past ◽

Published Research ◽

Critical Process

Citizen science schemes (projects) enable ecological data collection over very large spatial and temporal scales, producing datasets of high value for both pure and applied research. However, the accuracy of citizen science data is often questioned, owing to issues surrounding data quality and verification, the process by which records are checked after submission for correctness. Verification is a critical process for ensuring data quality and for increasing trust in such datasets, but verification approaches vary considerably among schemes. Here, we systematically review approaches to verification across ecological citizen science schemes, which feature in published research, aiming to identify the options available for verification, and to examine factors that influence the approaches used (Baker et al. 2021). We reviewed 259 schemes and were able to locate verification information for 142 of those. Expert verification was most widely used, especially among longer-running schemes. Community consensus was the second most common verification approach, used by schemes such as Snapshot Serengeti (Swanson et al. 2016) and MammalWeb (Hsing et al. 2018). It was more common among schemes with a larger number of participants and where photos or video had to be submitted with each record. Automated verification was not widely used among the schemes reviewed. Schemes that used automation, such as eBird (Kelling et al. 2011) and Project FeederWatch (Bonter and Cooper 2012) did so in conjunction with other methods such as expert verification. Expert verification has been the default approach for schemes in the past, but as the volume of data collected through citizen science schemes grows and the potential of automated approaches develops, many schemes might be able to implement approaches that verify data more efficiently. We present an idealised system for data verification, identifying schemes where this hierachical system could be applied and the requirements for implementation. We propose a hierarchical approach in which the bulk of records are verified by automation or community consensus, and any flagged records can then undergo additional levels of verification by experts.

Download Full-text

TrIAS, leveraging citizen science data to monitor invasive species in Belgium

Biodiversity Information Science and Standards ◽

10.3897/biss.2.24749 ◽

2018 ◽

Vol 2 ◽

pp. e24749

Author(s):

Quentin Groom ◽

Tim Adriaens ◽

Damiano Oldoni ◽

Lien Reyserhove ◽

Diederik Strubbe ◽

...

Keyword(s):

Invasive Species ◽

Alien Species ◽

Citizen Science ◽

Invasive Alien Species ◽

Science Research ◽

Open Science ◽

Policy Support ◽

Science Data ◽

Common View ◽

Indispensable Tool

Reducing the damage caused by invasive species requires a community approach informed by rapidly mobilized data. Even if local stakeholders work together, invasive species do not respect borders, and national, continental and global policies are required. Yet, in general, data on invasive species are slow to be mobilized, often of insufficient quality for their intended application and distributed among many stakeholders and their organizations, including scientists, land managers, and citizen scientists. The Belgian situation is typical. We struggle with the fragmentation of data sources and restrictions to data mobility. Nevertheless, there is a common view that the issue of invasive alien species needs to be addressed. In 2017 we launched the Tracking Invasive Alien Species (TrIAS) project, which envisages a future where alien species data are rapidly mobilized, the spread of exotic species is regularly monitored, and potential impacts and risks are rapidly evaluated in support of policy decisions (Vanderhoeven et al. 2017). TrIAS is building a seamless, data-driven workflow, from raw data to policy support documentation. TrIAS brings together 21 different stakeholder organizations that covering all organisms in the terrestrial, freshwater and marine environments. These organizations also include those involved in citizen science, research and wildlife management. TrIAS is an Open Science project and all the software, data and documentation are being shared openly (Groom et al. 2018). This means that the workflow can be reused as a whole or in part, either after the project or in different countries. We hope to prove that rapid data workflows are not only an indispensable tool in the control of invasive species, but also for integrating and motivating the citizens and organizations involved.

Download Full-text

Citizen science with colour blindness: A case study on the Forel-Ule scale

PLoS ONE ◽

10.1371/journal.pone.0249755 ◽

2021 ◽

Vol 16 (4) ◽

pp. e0249755

Author(s):

Olivier Burggraaff ◽

Sanjana Panchagnula ◽

Frans Snik

Keyword(s):

Citizen Science ◽

Data Entry ◽

Science Data ◽

Training Materials ◽

Colour Scale ◽

The Social ◽

Uncertainty Estimates ◽

Colour Blindness ◽

The Impact

Many citizen science projects depend on colour vision. Examples include classification of soil or water types and biological monitoring. However, up to 1 in 11 participants are colour blind. We simulate the impact of various forms of colour blindness on measurements with the Forel-Ule scale, which is used to measure water colour by eye with a 21-colour scale. Colour blindness decreases the median discriminability between Forel-Ule colours by up to 33% and makes several colour pairs essentially indistinguishable. This reduces the precision and accuracy of citizen science data and the motivation of participants. These issues can be addressed by including uncertainty estimates in data entry forms and discussing colour blindness in training materials. These conclusions and recommendations apply to colour-based citizen science in general, including other classification and monitoring activities. Being inclusive of the colour blind increases both the social and scientific impact of citizen science.

Download Full-text

Collaborative approaches to accessing and utilising historical citizen science data: a case-study with spearfishers from eastern Australia

Marine and Freshwater Research ◽

10.1071/mf14071 ◽

2015 ◽

Vol 66 (3) ◽

pp. 195 ◽

Cited By ~ 11

Author(s):

Daniel C. Gledhill ◽

Alistair J. Hobday ◽

David J. Welch ◽

Stephen G. Sutton ◽

Matthew J. Lansdell ◽

...

Keyword(s):

Environmental Change ◽

Citizen Science ◽

Marine Species ◽

Scientific Data ◽

Science Data ◽

Coastal Fish ◽

Eastern Australia ◽

Fit For Purpose ◽

The 1960S

Scientists are increasingly utilising non-traditional data to assist with defining biological baselines and for monitoring environmental change. These data present challenges not encountered with traditional, fit-for-purpose scientific data, including engaging with data owners, building trust and maintaining relationships, analysing and interpreting data collected under varying methodologies, and the possibility that data may not suit an intended purpose. Here we describe engagement activities undertaken with recreational spearfishers to collate and examine spearfishing club data collected from competitions held throughout south-eastern Australia from the 1960s until the present, representing one of the most extensive citizen science datasets for marine species in the region. The data proved suitable for demonstrating change in coastal fish communities, some of which were consistent with expectations given a warming climate over the period considered. With an attitudinal survey of divers we also asked about their experience of environmental change, and interaction with management over recent decades. Mutually beneficial outcomes include: collating and archiving significant data that may otherwise have been lost; improved understanding of spearfisher concerns and experiences; improved engagement between collaborators; and recognition of spearfishers’ desire for better engagement in science and management. Lessons learnt may be broadly applicable to improving collaboration between recreational fishers, citizen science groups, researchers and managers.

Download Full-text

Perspectives on Citizen Science Data Quality

Frontiers in Climate ◽

10.3389/fclim.2021.615032 ◽

2021 ◽

Vol 3 ◽

Author(s):

Robert R. Downs ◽

Hampapuram K. Ramapriyan ◽

Ge Peng ◽

Yaxing Wei

Keyword(s):

Quality Control ◽

Data Quality ◽

Quality Assessment ◽

Citizen Science ◽

Data Reuse ◽

Quality Information ◽

Science Data ◽

Conducting Research

Information about data quality helps potential data users to determine whether and how data can be used and enables the analysis and interpretation of such data. Providing data quality information improves opportunities for data reuse by increasing the trustworthiness of the data. Recognizing the need for improving the quality of citizen science data, we describe quality assessment and quality control (QA/QC) issues for these data and offer perspectives on aspects of improving or ensuring citizen science data quality and for conducting research on related issues.

Download Full-text