AbstractA signature feature of the neocortex is the dense network of horizontal connections (HCs) through which pyramidal neurons (PNs) exchange “contextual” information. In primary visual cortex (V1), HCs are thought to facilitate boundary detection, a crucial operation for object recognition, but how HCs modulate PN responses to boundary cues within their classical receptive fields (CRF) remains unknown. We began by “asking” natural images, through a structured data collection and ground truth labeling process, what function a V1 cell should use to compute boundary probability from aligned edge cues within and outside its CRF. The “answer” was an asymmetric 2-D sigmoidal function, whose nonlinear form provides the first normative account for the “multiplicative” center-flanker interactions previously reported in V1 neurons (Kapadia et al. 1995, 2000; Polat et al. 1998). Using a detailed compartmental model, we then show that this boundary-detecting classical-contextual interaction function can be computed with near perfect accuracy by NMDAR-dependent spatial synaptic interactions within PN dendrites – the site where classical and contextual inputs first converge in the cortex. In additional simulations, we show that local interneuron circuitry activated by HCs can powerfully leverage the nonlinear spatial computing capabilities of PN dendrites, providing the cortex with a highly flexible substrate for integration of classical and contextual information.Significance StatementIn addition to the driver inputs that establish their classical receptive fields, cortical pyramidal neurons (PN) receive a much larger number of “contextual” inputs from other PNs through a dense plexus of horizontal connections (HCs). However by what mechanisms, and for what behavioral purposes, HC’s modulate PN responses remains unclear. We pursued these questions in the context of object boundary detection in visual cortex, by combining an analysis of natural boundary statistics with detailed modeling PNs and local circuits. We found that nonlinear synaptic interactions in PN dendrites are ideally suited to solve the boundary detection problem. We propose that PN dendrites provide the core computing substrate through which cortical neurons modulate each other’s responses depending on context.