Predicting and measuring decision rules for social recognition in a Neotropical frog

Many animals use signals, such as vocalizations, to recognize familiar individuals. However, animals risk making recognition mistakes because the signal properties of different individuals often overlap due to within-individual variation in signal production. To understand the relationship between signal variation and decision rules for social recognition, we studied male golden rocket frogs, which recognize the calls of territory neighbors and respond less aggressively to a neighbor's calls than to the calls of strangers. We quantified patterns of individual variation in acoustic properties of calls and predicted optimal discrimination thresholds using a signal detection theory model of receiver utility that incorporated signal variation, the payoffs of correct and incorrect decisions, and the rates of encounters with neighbors and strangers. We then experimentally determined thresholds for discriminating between neighbors and strangers using a habituation-discrimination experiment with territorial males in the field. Males required a threshold difference between 9% and 12% to discriminate between calls differing in temporal properties; this threshold matched those predicted by a signal detection theory model under ecologically realistic assumptions of infrequent encounters with strangers and relatively costly missed detections of strangers. We demonstrate empirically that receivers group continuous variation in vocalizations into discrete social categories and show that signal detection theory can be applied to investigate evolved decision rules.