In higher mammals, thalamic afferents to primary visual cortex (area V1) segregate according to their responses to increases (ON) or decreases (OFF) in luminance1–4. This organization induces columnar, ON/OFF domains postulated to provide a scaffold for the emergence of orientation tuning2,5–15. To further test this idea, we asked whether ON/OFF domains exist in mouse V1 – a species containing orientation tuned, simple cells, like those found in other mammals16–19. Here we show that mouse V1 is indeed parceled into ON/OFF domains. Revealingly, fluctuations in the relative density ON/OFF neurons on the cortical surface mirror fluctuations in the relative density of ON/OFF receptive field centers on the visual field. In each cortical volume examined, the average of simple-cell receptive fields correlates with the difference between the average of ON and OFF receptive fields7. Moreover, the local diversity of simple-cell receptive fields is explained by a model in which neurons linearly combine a small number of ON and OFF signals available in their cortical neighborhoods15,20. Altogether, these findings indicate that ON/OFF domains originate in fluctuations of the spatial density of ON/OFF inputs on the visual field which, in turn, shapes the structure of receptive fields10–13,21–23.