A systematic meta-analysis on managing innovation projects in uncertain and complex environments

Objective The aim was to evaluate the relevance of the critique offered by Jamieson and Skraaning (2019) regarding the applicability of the lumberjack effect of human–automation interaction to complex real-world settings. Background The lumberjack effect, based upon a meta-analysis, identifies the consequences of a higher degree of automation—to improve performance and reduce workload—when automation functions as intended, but to degrade performance more, as mediated by a loss of situation awareness (SA) when automation fails. Jamieson and Skraaning provide data from a process control scenario that they assert contradicts the effect. Approach We analyzed key aspects of their simulation, measures, and results which we argue limit the strength of their conclusion that the lumberjack effect is not applicable to complex real-world systems. Results Our analysis revealed limits in their inappropriate choice of automation, the lack of a routine performance measure, support for the lumberjack effect that was actually provided by subjective measures of the operators, an inappropriate assessment of SA, and a possible limitation of statistical power. Conclusion We regard these limitations as reasons to temper the strong conclusions drawn by the authors, of no applicability of the lumberjack effect to complex environments. Their findings should be used as an impetus for conducting further research on human–automation interaction in these domains. Applications The collective findings of both Jamieson and Skraaning and our study are applicable to system designers and users in deciding upon the appropriate level of automation to deploy.

Download Full-text

Managing innovation projects in the global system

Proceedings of Engineering Management Society Conference on Managing Projects in a Borderless World ◽

10.1109/iemc.1993.316476 ◽

2002 ◽

Author(s):

P. Purkayastha

Keyword(s):

Global System ◽

Innovation Projects ◽

Managing Innovation

Download Full-text

Correcting index databases improves metagenomic studies

10.1101/712166 ◽

2019 ◽

Cited By ~ 7

Author(s):

Guillaume Méric ◽

Ryan R. Wick ◽

Stephen C. Watts ◽

Kathryn E. Holt ◽

Michael Inouye

Keyword(s):

Meta Analysis ◽

Fold Increase ◽

Taxonomic Composition ◽

Human Pathogens ◽

Complex Environments ◽

Accessible Information ◽

Two Factors ◽

Ncbi Refseq ◽

Reference Index ◽

Microbiome Data

AbstractAssessing the taxonomic composition of metagenomic samples is an important first step in understanding the biology and ecology of microbial communities in complex environments. Despite a wealth of algorithms and tools for metagenomic classification, relatively little effort has been put into the critical task of improving the quality of reference indices to which metagenomic reads are assigned. Here, we inferred the taxonomic composition of 404 publicly available metagenomes from human, marine and soil environments, using custom index databases modified according to two factors: the number of reference genomes used to build the databases, and the monophyletic strictness of species definitions. Index databases built following the NCBI taxonomic system were also compared to others using Genome Taxonomy Database (GTDB) taxonomic redefinitions. We observed a considerable increase in the rate of read classification using modified reference index databases as compared to a default NCBI RefSeq database, with up to a 4.4-, 6.4- and 2.2-fold increase in classified reads per sample for human, marine and soil metagenomes, respectively. Importantly, targeted correction for 70 common human pathogens and bacterial genera in the index database increased their specific detection levels in human metagenomes. We also show the choice of index database can influence downstream diversity and distance estimates for microbiome data. Overall, the study shows a large amount of accessible information in metagenomes remains unexploited using current methods, and that the same data analysed using different index databases could potentially lead to different conclusions. These results have implications for the power and design of individual microbiome studies, and for comparison and meta-analysis of microbiome datasets.

Download Full-text