KEEPING REAL WORLD BIAS OUT OF ARTIFICIAL INTELLIGENCE “EXAMINATION OF CODER BIAS IN DATA SCIENCE RECRUITMENT SOLUTIONS”

Against generalisation: Data-driven decisions need context to be human-compatible

Business Information Review ◽

10.1177/02663821211061986 ◽

2021 ◽

pp. 026638212110619

Author(s):

Sharon Richardson

Keyword(s):

Artificial Intelligence ◽

Real World ◽

Data Science ◽

Machine Learning Algorithms ◽

Data Driven ◽

Theoretical Frameworks ◽

The Past ◽

Real World Applications ◽

Contextual Framework ◽

Data Driven Decisions

During the past two decades, there have been a number of breakthroughs in the fields of data science and artificial intelligence, made possible by advanced machine learning algorithms trained through access to massive volumes of data. However, their adoption and use in real-world applications remains a challenge. This paper posits that a key limitation in making AI applicable has been a failure to modernise the theoretical frameworks needed to evaluate and adopt outcomes. Such a need was anticipated with the arrival of the digital computer in the 1950s but has remained unrealised. This paper reviews how the field of data science emerged and led to rapid breakthroughs in algorithms underpinning research into artificial intelligence. It then discusses the contextual framework now needed to advance the use of AI in real-world decisions that impact human lives and livelihoods.

Download Full-text

The Necessity for Collaboration Between Data Scientists and Domain Experts

10.2118/208634-ms ◽

2021 ◽

Author(s):

Patrick Bangert

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Change Management ◽

Real World ◽

Data Science ◽

Effective Communication ◽

Practical Experience ◽

Domain Experts ◽

Science Projects ◽

Domain Expertise

Abstract A practical data science, machine learning, or artificial intelligence project benefits from various organizational and managerial prerequisites. The effective collaboration between various data scientists and domain experts is perhaps the most important, which is discussed here. Based on practical experience, the principal theses put forward here are that (1) data science projects require domain expertise, (2) domain expertise and data science expertise generally cannot be provided by the same individual, (3) effective communication between the various experts is essential for which everyone requires some limited understanding of the others’ expertise and real-world experience, and (4) management must acknowledge these aspects by reserving sufficient project time and budget for communication and change management.

Download Full-text

The 9 Pitfalls of Data Science

10.1093/oso/9780198844396.001.0001 ◽

2019 ◽

Cited By ~ 5

Author(s):

Gary Smith ◽

Jay Cordes

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Big Data ◽

Critical Thinking ◽

Real World ◽

Data Science ◽

Thinking Skills ◽

Critical Thinking Skills ◽

Scientific Rigor ◽

Good Data

Scientific rigor and critical thinking skills are indispensable in this age of big data because machine learning and artificial intelligence are often led astray by meaningless patterns. The 9 Pitfalls of Data Science is loaded with entertaining real-world examples of both successful and misguided approaches to interpreting data, both grand successes and epic failures. Anyone can learn to distinguish between good data science and nonsense. We are confident that readers will learn how to avoid being duped by data, and make better, more informed decisions. Whether they want to be effective creators, interpreters, or users of data, they need to know the nine pitfalls of data science.

Download Full-text

About digital transformation of the transport industry in Russia

Scientific notes of the Russian academy of entrepreneurship ◽

10.24182/2073-6258-2020-19-3-95-105 ◽

2020 ◽

Vol 19 (3) ◽

pp. 95-105

Author(s):

Natalia V. Vysotskaya ◽

T. V. Kyrbatskaya

Keyword(s):

Artificial Intelligence ◽

Business Model ◽

Data Science ◽

Digital Transformation ◽

Digital Culture ◽

Blockchain Technology ◽

Community Approach ◽

Non Linear ◽

Transport Industry ◽

Artificial Intelligence Systems

The article is devoted to the consideration of the main directions of digital transformation of the transport industry in Russia. It is proposed in the process of digital transformation to integrate the community approach into the company's business model using blockchain technology and methods and results of data science; complement the new digital culture with a digital team and new communities that help management solve business problems; focus the attention of the company's management on its employees and develop those competencies in them that robots and artificial intelligence systems cannot implement: develop algorithmic, computable and non-linear thinking in all employees of the company.

Download Full-text

Data and research in the era of artificial intelligence and advanced data science (NSF)

Federal Grants & Contracts ◽

10.1002/fgc.31450 ◽

2020 ◽

Vol 45 (2) ◽

pp. 3-3

Keyword(s):

Artificial Intelligence ◽

Data Science

Download Full-text

Stress testing reveals gaps in clinic readiness of image-based diagnostic artificial intelligence models

npj Digital Medicine ◽

10.1038/s41746-020-00380-6 ◽

2021 ◽

Vol 4 (1) ◽

Author(s):

Albert T. Young ◽

Kristen Fernandez ◽

Jacob Pfau ◽

Rasika Reddy ◽

Nhat Anh Cao ◽

...

Keyword(s):

Artificial Intelligence ◽

Real World ◽

Stress Testing ◽

Operating Characteristic ◽

Characteristic Curve ◽

Skin Lesions ◽

Stress Tests ◽

Performance Metric ◽

Intelligence Models ◽

Artificial Intelligence Models

AbstractArtificial intelligence models match or exceed dermatologists in melanoma image classification. Less is known about their robustness against real-world variations, and clinicians may incorrectly assume that a model with an acceptable area under the receiver operating characteristic curve or related performance metric is ready for clinical use. Here, we systematically assessed the performance of dermatologist-level convolutional neural networks (CNNs) on real-world non-curated images by applying computational “stress tests”. Our goal was to create a proxy environment in which to comprehensively test the generalizability of off-the-shelf CNNs developed without training or evaluation protocols specific to individual clinics. We found inconsistent predictions on images captured repeatedly in the same setting or subjected to simple transformations (e.g., rotation). Such transformations resulted in false positive or negative predictions for 6.5–22% of skin lesions across test datasets. Our findings indicate that models meeting conventionally reported metrics need further validation with computational stress tests to assess clinic readiness.

Download Full-text

Medical artificial intelligence readiness scale for medical students (MAIRS-MS) – development, validity and reliability study

BMC Medical Education ◽

10.1186/s12909-021-02546-6 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Ozan Karaca ◽

S. Ayhan Çalışkan ◽

Kadir Demir

Keyword(s):

Artificial Intelligence ◽

Factor Analysis ◽

Medical Students ◽

Data Science ◽

Factor Model ◽

Health Science ◽

Measurement Tool ◽

Validity And Reliability ◽

Mathematical Concepts ◽

Evaluation And Monitoring

Abstract Background It is unlikely that applications of artificial intelligence (AI) will completely replace physicians. However, it is very likely that AI applications will acquire many of their roles and generate new tasks in medical care. To be ready for new roles and tasks, medical students and physicians will need to understand the fundamentals of AI and data science, mathematical concepts, and related ethical and medico-legal issues in addition with the standard medical principles. Nevertheless, there is no valid and reliable instrument available in the literature to measure medical AI readiness. In this study, we have described the development of a valid and reliable psychometric measurement tool for the assessment of the perceived readiness of medical students on AI technologies and its applications in medicine. Methods To define medical students’ required competencies on AI, a diverse set of experts’ opinions were obtained by a qualitative method and were used as a theoretical framework, while creating the item pool of the scale. Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were applied. Results A total of 568 medical students during the EFA phase and 329 medical students during the CFA phase, enrolled in two different public universities in Turkey participated in this study. The initial 27-items finalized with a 22-items scale in a four-factor structure (cognition, ability, vision, and ethics), which explains 50.9% cumulative variance that resulted from the EFA. Cronbach’s alpha reliability coefficient was 0.87. CFA indicated appropriate fit of the four-factor model (χ2/df = 3.81, RMSEA = 0.094, SRMR = 0.057, CFI = 0.938, and NNFI (TLI) = 0.928). These values showed that the four-factor model has construct validity. Conclusions The newly developed Medical Artificial Intelligence Readiness Scale for Medical Students (MAIRS-MS) was found to be valid and reliable tool for evaluation and monitoring of perceived readiness levels of medical students on AI technologies and applications. Medical schools may follow ‘a physician training perspective that is compatible with AI in medicine’ to their curricula by using MAIRS-MS. This scale could be benefitted by medical and health science education institutions as a valuable curriculum development tool with its learner needs assessment and participants’ end-course perceived readiness opportunities.

Download Full-text

A Survey on Bias and Fairness in Machine Learning

ACM Computing Surveys ◽

10.1145/3457607 ◽

2021 ◽

Vol 54 (6) ◽

pp. 1-35

Author(s):

Ninareh Mehrabi ◽

Fred Morstatter ◽

Nripsuta Saxena ◽

Kristina Lerman ◽

Aram Galstyan

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Deep Learning ◽

Real World ◽

State Of The Art ◽

Future Directions ◽

Discriminatory Behavior ◽

Real World Applications ◽

Near Future ◽

Different Sources

With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.

Download Full-text