Visual perceptual decisions can be altered by recent experience. In the “serial dependence” effect, participants’ responses to visual stimuli appear to be biased toward (i.e., attracted to) recently encountered stimuli. Fischer and Whitney (2014) proposed that serial dependence reflects a “continuity field” that promotes visual stability by biasing perception toward the recent past. However, when participants are relatively accurate on the prior trial, there is no discernible difference between attraction to the prior stimulus and attraction to the prior response. To tease apart these alternative explanations of the attraction effect, we developed two complementary analysis techniques that rely on participants’ naturally occurring errors on a trial-by-trial basis, identifying any effect of the prior stimulus and, separately, any effect of the prior response (i.e., each effect could be attractive, repulsive, or absent). Applying these techniques to serial dependence data from a new experiment and four previously published studies, including Fischer and Whitney’s, we found that serial dependencies reflect an attraction to the previous response and repulsion from the previous stimulus, with these effects cancelling each other to different degrees for different experiments. In no case did we find evidence of an attraction to the prior stimulus. These results are consistent with literatures that predate the serial dependence effect: Attraction to prior responses is routinely observed in a wide variety of paradigms and repulsion from prior stimuli is ubiquitous, such as in the tilt aftereffect.