Automated Alert Failures and their Impact on Operator Performance and Trust

2021 ◽  
Vol 65 (1) ◽  
pp. 1286-1290
Author(s):  
James C. Ferraro ◽  
Mustapha Mouloua
Keyword(s):  
Performance Measures ◽  
User Behavior ◽  
False Alarms ◽  
Automated Systems ◽  
Alert System ◽  
Error Type ◽  
System Failures ◽  
Monitoring Task ◽  
Operator Performance ◽  
Monitoring Performance

Despite its rapid advancement, automation remains vulnerable to system failures. The reliability of automation may impact users’ trust and how they interact with it. Additionally, the type of error can uniquely redirect user behavior. This study investigated how reliability and error type impact operator trust and monitoring performance. Participants completed a monitoring task at either 50% or 90% reliability, experiencing either misses or false alarms from an automated alert system. It was hypothesized that automation reliability would impact trust, while error type would also impact reliance and compliance behaviors. Results indicated that misses had a greater impact on monitoring performance than false alarms, while reliability did not influence performance. Trust was not influenced by reliability or error type and showed no relationship with performance measures. These results can help further clarify the way automation failures shape how humans interact with automation and inform the design of future automated systems.

Sentence processing in language-impaired children under conditions of filtering and time compression

1995 ◽  
Vol 16 (2) ◽  
pp. 137-154 ◽  
Author(s):  
Rachel E. Stark ◽  
James W. Montgomery
Keyword(s):  
Sentence Processing ◽  
High Frequency ◽  
Response Times ◽  
Language Impaired ◽  
Monitoring Task ◽  
Time Compression ◽  
Lower Accuracy ◽  
On Line ◽  
Impaired Children ◽  
Monitoring Performance

ABSTRACTNineteen language-impaired (LI) and 20 language-normal (LN) children participated in an on-line word-monitoring task. Words were presented in lists and in sentences readily comprehended by younger children. The sentences were unaltered, tow-pass filtered, and time- compressed. Both groups had shorter mean response times (MRTs), but lower accuracy, for words in sentences than words in lists. The LI children had significantly longer MRTs under sentence conditions and lower accuracy overall than the LN children. Filtering had an adverse effect upon accuracy and MRT for both subject groups. Time compression did not, suggesting that the reduction in high-frequency information and the rate of presentation exert different effects. Subject differences in attention, as well as in linguistic competence and motor control, may have influenced word-monitoring performance.

Concurrent Monitoring for Multiple Critical Signals in a Complex Display: A Vigilance Perspective

2005 ◽  
Vol 49 (17) ◽  
pp. 1518-1522
Author(s):  
Nathan R. Bailey ◽  
Mark W. Scerbo
Keyword(s):  
Task Demands ◽  
Complex Environment ◽  
Vigilance Performance ◽  
Digital Readout ◽  
Monitoring Task ◽  
Number Of Factors ◽  
Different Types ◽  
Effort Regulation ◽  
Monitoring Performance ◽  
Complex Display

The present study was designed to examine the monitoring performance of operators in a complex environment requiring concurrent monitoring of multiple displays with different types of critical signals. Participants performed a manual flight task concurrently with three monitoring tasks over three separate 2-hour sessions. The monitoring tasks required operators to detect deviations in the gauge, mode, and digital readout portions of a simulated EICAS display. Results indicated that while performance on the primary flight task degraded within each session, monitoring performance remained constant. Further, intrasession monitoring performance did not degrade across trials. These findings suggest that vigilance performance for complex displays may be influenced by a number of factors including compensatory strategies related to mental effort regulation, the complexity of monitoring task demands, the duration of the monitoring session, and the nature of additional operator responsibilities.

Required Performance Levels for Autocueing Devices

1980 ◽  
Vol 24 (1) ◽  
pp. 511-515 ◽  
Author(s):  
David C. Fulkerson
Keyword(s):  
Synthetic Aperture Radar ◽  
Performance Measures ◽  
Variable Number ◽  
Baseline Condition ◽  
Synthetic Aperture ◽  
False Alarms ◽  
Performance Levels ◽  
Number Of False Alarms ◽  
True Target ◽  
Sar Imagery

Acquisition of tactical targets embedded in synthetic aperture radar (SAR) imagery was examined in this study. Experienced radar operators saw static scenes with either nine/five, two, or zero possible targets annotated, representing the application of three hypothetical autocueing devices and a no-autocueing baseline condition. In the autocueing conditions, the true target was always annotated, along with a variable number of false alarms, based upon the sophistication of the hypothesized autocuer. With time and probabilty performance measures taken, a significant improvement in target acquisition behavior was achieved only with the most sophisticated autocuer. This was most evident for the more difficult target types.

Team Size and Decision Rule in the Performance of Simulated Monitoring Teams

1972 ◽  
Vol 14 (4) ◽  
pp. 309-314 ◽  
Author(s):  
Wayne L. Waag ◽  
Charles G. Halcomb
Keyword(s):  
Decision Rule ◽  
Detection Rate ◽  
False Alarms ◽  
Individual Response ◽  
Team Size ◽  
Parallel Teams ◽  
Monitoring Performance

A methodology is presented for randomly creating “teams” from a data pool of individual response records. Using this approach, the effects of two variables on team monitoring performance were investigated: (1) team size and (2) the decision rule employed in defining the requirements of a “team” response. Size of the simulated teams was varied from two to five members. The decision rule was varied from “parallel” in which a response by any one or more members produced a “team” response to “series” in which a “team” response occurred only if all members responded. “Parallel” teams were found to maximize correct detections while “series” teams eliminated all false alarms. For each decision rule, detection rate increased as a function of team size. For each team size, detection rate deteriorated as the decision rule required more members to respond correctly.

scholarly journals The Automation-by-Expertise-by-Training Interaction

2016 ◽  
Vol 59 (2) ◽  
pp. 204-228 ◽  
Author(s):  
Barry Strauch
Keyword(s):  
Situation Awareness ◽  
Ad Hoc ◽  
Training Programs ◽  
Automated Systems ◽  
Sociotechnical System ◽  
Operator Performance ◽  
Related Operator ◽  
Effective System ◽  
Expertise Level ◽  
And Training

Objective: I introduce the automation-by-expertise-by-training interaction in automated systems and discuss its influence on operator performance. Background: Transportation accidents that, across a 30-year interval demonstrated identical automation-related operator errors, suggest a need to reexamine traditional views of automation. Method: I review accident investigation reports, regulator studies, and literature on human computer interaction, expertise, and training and discuss how failing to attend to the interaction of automation, expertise level, and training has enabled operators to commit identical automation-related errors. Results: Automated systems continue to provide capabilities exceeding operators’ need for effective system operation and provide interfaces that can hinder, rather than enhance, operator automation-related situation awareness. Because of limitations in time and resources, training programs do not provide operators the expertise needed to effectively operate these automated systems, requiring them to obtain the expertise ad hoc during system operations. As a result, many do not acquire necessary automation-related system expertise. Conclusion: Integrating automation with expected operator expertise levels, and within training programs that provide operators the necessary automation expertise, can reduce opportunities for automation-related operator errors. Application: Research to address the automation-by-expertise-by-training interaction is needed. However, such research must meet challenges inherent to examining realistic sociotechnical system automation features with representative samples of operators, perhaps by using observational and ethnographic research. Research in this domain should improve the integration of design and training and, it is hoped, enhance operator performance.

The Role of Payoffs and Signal Ratios in Criterion Changes during a Monitoring Task

1971 ◽  
Vol 13 (3) ◽  
pp. 261-267 ◽  
Author(s):  
Robert C. Williges
Keyword(s):  
Signal Detection Theory ◽  
False Alarms ◽  
Signal Ratio ◽  
Monitoring Task ◽  
Decision Performance ◽  
Signal Rate ◽  
Subject Design ◽  
Long Duration ◽  
Over Time

Forty-eight subjects detected a long-duration (1.7 or 1.3 sec.) change in brightness (from a 5 ft.-l. standard to a 4 ft.-l. level) of an electroluminescent panel during a 60-min. monitoring session. Signal/nonsignal ratios (1/9 or 1/1) and payoffs (lax, neutral, or strict) were combined factorially in a between-subject design. Signal ratios affected both the percent of signal detections and the percent of false-alarm errors. When subjects monitored under the lower signal ratios, a decrease in percent of signal detections occurred over time. Payoffs affected only the percent of false alarms in the higher signal rate conditions. Signal detection theory analyses resulted in a slight decrease in d' and a marked increase in β during the watch period. The change in β was due primarily to the lower signal ratio conditions. Payoffs had no effect on subsequent β change. It was concluded that signal ratios rather than payoffs play the major role in determining decision performance in simple visual monitoring tasks.

Knowing Behavior Helps Insure Models Against Breakage: A Commentary on Kaber’s “Issues in Human–Automation Interaction Modeling”

2017 ◽  
Vol 12 (1) ◽  
pp. 67-69 ◽  
Author(s):  
Evan Byrne
Keyword(s):  
System Design ◽  
System Performance ◽  
User Behavior ◽  
Automated Systems ◽  
Interaction Modeling ◽  
Levels Of Automation

This commentary on Kaber’s review of human–automation interaction (HAI) modeling and levels of automation (LOA) highlights some of the challenges designers of automated systems face as a result of a heterogeneous user base. It advocates for understanding the variability in the intended user base to facilitate decisions on whether to constrain user behavior or the system design to optimize overall system performance and the need to anticipate adaptive user strategies before system deployment. It argues that the predictive efficacy of LOA models depends on the heterogeneity of the user base and an increased understanding of behavior through evaluation of breakdowns in HAI.

scholarly journals Development of an Anomaly Alert System Triggered by Unusual Behaviors at Home

Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5454
Author(s):  
Roxana Rodriguez-Goncalves ◽  
Angel Garcia-Crespo ◽  
Carlos Matheus-Chacin ◽  
Adrian Ruiz-Arroyo
Keyword(s):  
Life Expectancy ◽  
Elderly People ◽  
User Behavior ◽  
Daily Life ◽  
Family Members ◽  
Behavior Patterns ◽  
Behavior Changes ◽  
Alert System ◽  
Design And Development ◽  
Unusual Behavior

In many countries, the number of elderly people has grown due to the increase in the life expectancy of the population, many of whom currently live alone and are prone to having accidents that they cannot report, especially if they are immobilized. For this reason, we have developed a non-intrusive IoT device, which, through multiple integrated sensors, collects information on habitual user behavior patterns and uses it to generate unusual behavior rules. These rules are used by our SecurHome system to send alert messages to the dependent person's family members or caregivers if their behavior changes abruptly over the course of their daily life. This document describes in detail the design and development of the SecurHome system.

scholarly journals Bias Adjusted Precipitation Threat Scores

2008 ◽  
Vol 16 ◽  
pp. 137-142 ◽  
Author(s):  
F. Mesinger
Keyword(s):  
Performance Measures ◽  
Present Author ◽  
Forecast Accuracy ◽  
False Alarms ◽  
Equitable Threat Score ◽  
Bias Score ◽  
Threat Score ◽  
Unit Increase ◽  
Operational Models ◽  
The One

Abstract. Among the wide variety of performance measures available for the assessment of skill of deterministic precipitation forecasts, the equitable threat score (ETS) might well be the one used most frequently. It is typically used in conjunction with the bias score. However, apart from its mathematical definition the meaning of the ETS is not clear. It has been pointed out (Mason, 1989; Hamill, 1999) that forecasts with a larger bias tend to have a higher ETS. Even so, the present author has not seen this having been accounted for in any of numerous papers that in recent years have used the ETS along with bias "as a measure of forecast accuracy". A method to adjust the threat score (TS) or the ETS so as to arrive at their values that correspond to unit bias in order to show the model's or forecaster's accuracy in \\textit{placing} precipitation has been proposed earlier by the present author (Mesinger and Brill, the so-called dH/dF method). A serious deficiency however has since been noted with the dH/dF method in that the hypothetical function that it arrives at to interpolate or extrapolate the observed value of hits to unit bias can have values of hits greater than forecast when the forecast area tends to zero. Another method is proposed here based on the assumption that the increase in hits per unit increase in false alarms is proportional to the yet unhit area. This new method removes the deficiency of the dH/dF method. Examples of its performance for 12 months of forecasts by three NCEP operational models are given.

Sign in / Sign up

Export Citation Format

Share Document