Algorithmic Bias in Education

Virtual reality (VR) is an emerging technology with the potential to extract significantly more data about learners and the learning process. In this article, we present an analysis of how VR education technology companies frame, use and analyse this data. We found both an expansion and acceleration of what data are being collected about learners and how these data are being mobilised in potentially discriminatory and problematic ways. Beyond providing evidence for how VR represents an intensification of the datafication of education, we discuss three interrelated critical issues that are specific to VR: the fantasy that VR data is ‘perfect’, the datafication of soft-skills training, and the commercialisation and commodification of VR data. In the context of the issues identified, we caution the unregulated and uncritical application of learning analytics to the data that are collected from VR training.

Download Full-text

An Objective Search for Unrecognized Bias in Validated COVID-19 Prediction Models

10.1101/2021.10.28.21265629 ◽

2021 ◽

Author(s):

Hossein Estiri ◽

Zachary Strasser ◽

Sina Rashidian ◽

Jeffrey Klann ◽

Kavishwar Wagholikar ◽

...

Keyword(s):

Predictive Models ◽

Medical Records ◽

Prediction Models ◽

Evaluation Studies ◽

Protected Group ◽

Increased Risk ◽

Using Data ◽

Bias Evaluation ◽

Algorithmic Bias

The growing recognition of algorithmic bias has spurred discussions about fairness in artificial intelligence (AI) / machine learning (ML) algorithms. The increasing translation of predictive models into clinical practice brings an increased risk of direct harm from algorithmic bias; however, bias remains incompletely measured in many medical AI applications. Using data from over 56 thousand Mass General Brigham (MGB) patients with confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), we evaluate unrecognized bias in four AI models developed during the early months of the pandemic in Boston, Massachusetts that predict risks of hospital admission, ICU admission, mechanical ventilation, and death after a SARS-CoV-2 infection purely based on their pre-infection longitudinal medical records. We discuss that while a model can be biased against certain protected groups (i.e., perform worse) in certain tasks, it can be at the same time biased towards another protected group (i.e., perform better). As such, current bias evaluation studies may lack a full depiction of the variable effects of a model on its subpopulations. If the goal is to make a change in a positive way, the underlying roots of bias need to be fully explored in medical AI. Only a holistic evaluation, a diligent search for unrecognized bias, can provide enough information for an unbiased judgment of AI bias that can invigorate follow-up investigations on identifying the underlying roots of bias and ultimately make a change.

Download Full-text

EVERY CLICK YOU MAKE: ALGORITHMIC LITERACY AND THE DIGITAL LIVES OF YOUNG ADULTS

AoIR Selected Papers of Internet Research ◽

10.5210/spir.v2020i0.11233 ◽

2020 ◽

Author(s):

Monica Jean Henderson ◽

Leslie Regan Shade ◽

Katie Mackinnon

Keyword(s):

Young Adults ◽

Digital Literacy ◽

Essential Element ◽

Digital Citizenship ◽

Sustainable Living ◽

Popular Discourse ◽

Document Type ◽

The Everyday ◽

Self Discovery ◽

Algorithmic Bias

Critical digital literacy comprises subsets of medium- and content-related skills necessary for digital privacy and digital citizenship. Frameworks for defining and evaluating digital literacy proliferate in academia and policymaking; however, in a networked climate subsumed by dataveillance, algorithmic bias, political bots, and deep fakes, these frameworks need to be updated. Algorithms may be the greatest determinant in sociopolitical online interactions and information gathering, and without a multivalent literacy of algorithms, nuanced understandings of digital privacy and digital citizenship may be unachievable. We therefore propose ‘algorithmic literacy’ become an essential element for digital literacy in young adult media education. Researchers have highlighted how intersectional aspects of gender, ability, and socioeconomic status are stronger predictors of low digital literacy than age. Following a tradition of participatory (rather than protectionist) research about youth privacy online, our research foregrounds young adults’ practices and perspectives on algorithmic culture in order to co-develop a framework for algorithmic literacy. Our paper shares findings from a participatory project co-designing an algorithmic literacy toolkit with young adults as co-researchers and participants. We created a curriculum focusing on reviewing the current critical scholarly literature, policy, and popular discourse on algorithms. After two weeks of intensive research, our student co-researchers met amongst themselves to devise a sustainable, ‘living-document’ type of toolkit, comprising a website, an Instagram page, and a Medium blog. Reflected in the toolkit's name, The Algorithmic You uses an intersectional lens to facilitate peer-oriented ‘self-discovery’ of how algorithms shape and produce interactions in the everyday lives of young adults.

Download Full-text

Editorial. AI, Arts & Design: Questioning Learning Machines

Artnodes ◽

10.7238/a.v0i26.3390 ◽

2020 ◽

Author(s):

Ruth West ◽

Andrés Burbano

Keyword(s):

Artificial Intelligence ◽

Historical Development ◽

Machine Intelligence ◽

Special Issue ◽

Control Structures ◽

Learning Machines ◽

The Arts ◽

Production Of Culture ◽

The Relationship ◽

Algorithmic Bias

Explorations of the relationship between Artificial Intelligence (AI), the arts, and design have existed throughout the historical development of AI. We are currently witnessing exponential growth in the application of Machine Learning (ML) and AI in all domains of art (visual, sonic, performing, spatial, transmedia, audiovisual, and narrative) in parallel with activity in the field that is so rapid that publication can not keep pace. In dialogue with our contemplation about this development in the arts, authors in this issue answer with questions of their own. Through questioning authorship and ethics, autonomy and automation, exploring the contribution of art to AI, algorithmic bias, control structures, machine intelligence in public art, formalization of aesthetics, the production of culture, socio-technical dimensions, relationships to games and aesthetics, and democratization of machine-based creative tools the contributors provide a multifaceted view into crucial dimensions of the present and future of creative AI. In this Artnodes special issue, we pose the question: Does generative and machine creativity in the arts and design represent an evolution of “artistic intelligence,” or is it a metamorphosis of creative practice yielding fundamentally distinct forms and modes of authorship?

Download Full-text

Algorithmic Bias and the Pastoral: Gender Misrecognition in Light of Natural Knowledge

Cervantes Bulletin of the Cervantes Society of America ◽

10.1353/cer.2020.0018 ◽

2020 ◽

Vol 40 (2) ◽

pp. 73-92

Author(s):

Bradley J. Nelson

Keyword(s):

Natural Knowledge ◽

Algorithmic Bias

Download Full-text

Welcome to AI Matters 7(2)

AI Matters ◽

10.1145/3478369.3478370 ◽

2021 ◽

Vol 7 (2) ◽

pp. 3-4

Author(s):

Iolanda Leite ◽

Anuj Karpatne

Keyword(s):

Natural Language ◽

Regular Education ◽

Neural Models ◽

Natural Language Interfaces ◽

Algorithmic Bias

Welcome to the second issue of this year's AI Matters Newsletter. We start with a report on upcoming SIGAI Events by Dilini Samarasinghe and Conference reports by Louise Dennis, our conference coordination officer. In our regular Education column, Carolyn Rosé discusses the role of AI in education in a post-pandemic reality. We then bring you our regular Policy column, where Larry Medsker covers interesting and timely discussions on AI policy, for example whether governments should play a role in reducing algorithmic bias. This issue closes with an article contribution from Li Dong, one of the runner-ups in the latest AAIS/SIGAI dissertation award, on the use neural models to build natural language interfaces.

Download Full-text