Background: Decreased motion perception has been suggested as a marker for visual pathway demyelination in optic neuritis (ON) and/or multiple sclerosis (MS). Objectives: To examine the influence of neuro-axonal damage on motion perception in MS and neuromyelitis optica spectrum disorders (NMOSD). Methods: We analysed motion perception with numbers-from-motion (NFM), visual acuity, (multifocal (mf)) VEP, optical coherence tomography in patients with MS ( n = 38, confirmatory cohort n = 43), NMOSD ( n = 13) and healthy controls ( n = 33). Results: NFM was lower compared with controls in MS ( B = −12.37, p < 0.001) and NMOSD ( B = −34.5, p < 0.001). NFM was lower in ON than in non-ON eyes ( B = −30.95, p = 0.041) in NMOSD, but not MS. In MS and NMOSD, lower NFM was associated with worse visual acuity ( B = −139.4, p < 0.001/ B = −77.2, p < 0.001) and low contrast letter acuity ( B = 0.99, p = 0.002/ B = 1.6, p < 0.001), thinner peripapillary retinal nerve fibre layer ( B = 1.0, p < 0.001/ B = 0.92, p = 0.016) and ganglion cell/inner plexiform layer ( B = 64.8, p < 0.001/ B = 79.5, p = 0.006), but not with VEP P100 latencies. In the confirmatory MS cohort, lower NFM was associated with thinner retinal nerve fibre layer ( B = 1.351, p < 0.001) and increased mfVEP P100 latencies ( B = −1.159, p < 0.001). Conclusions: Structural neuro-axonal visual pathway damage is an important driver of motion perception impairment in MS and NMOSD.