Benchmarking Aided Decision Making in a Signal Detection Task
Loading...
Date
2017-03-15
Authors
Bartlett, Megan
McCarley, Jason S
Journal Title
Journal ISSN
Volume Title
Publisher
Sage
Rights
Copyright © 2017, Human Factors and Ergonomics Society.
Rights Holder
Human Factors and Ergonomics Society.
Abstract
Objective:
A series of experiments examined human operators’ strategies for interacting with highly (93%) reliable automated decision aids in a binary signal detection task.
Background:
Operators often interact with automated decision aids in a suboptimal way, achieving performance levels lower than predicted by a statistically ideal model of information integration. To better understand operators’ inefficient use of decision aids, we compared participants’ automation-aided performance levels with the predictions of seven statistical models of collaborative decision making.
Method:
Participants performed a binary signal detection task that asked them to classify random dot images as either blue or orange dominant. They made their judgments either unaided or with assistance from a 93% reliable automated decision aid that provided either graded (Experiments 1 and 3) or binary (Experiment 2) cues. We compared automation-aided performance with the predictions of seven statistical models of collaborative decision making, including a statistically optimal model and Robinson and Sorkin’s contingent criterion model.
Results and Conclusion:
Automation-aided sensitivity hewed closest to the predictions of the two least efficient collaborative models, well short of statistically ideal levels. Performance was similar whether the aid provided graded or binary judgments. Model comparisons identified potential strategies by which participants integrated their judgments with the aid’s.
Application:
Results lend insight into participants’ automation-aided decision strategies and provide benchmarks for predicting automation-aided performance levels.
Description
Copyright © 2017 Human Factors and Ergonomics Society. Reprinted by permission of SAGE Publications.
Keywords
automated decision aids, binary signal detection task
Citation
Bartlett, M. L., & McCarley, J. S. (2017). Benchmarking Aided Decision Making in a Signal Detection Task. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59(6), 881–900. https://doi.org/10.1177/0018720817700258