In this paper, we propose a multichannel active noise control (ANC) system with an optimal reference microphone selector based on the time difference of arrival (TDOA). A multichannel feedforward ANC system using upstream reference signals can reduce various noises such as broadband noise by arranging reference microphones close to noise sources. However, the noise reduction performance of an ANC system degrades when the noise environment changes, such as the arrival direction. This is because some reference microphones do not satisfy the causality constraint that the unwanted noise propagates to the control point faster than the anti-noise used to cancel the unwanted noise. To solve this problem, we propose a multichannel ANC system with an optimal reference microphone selector. This selector chooses the reference microphones that satisfy the causality constraint based on the TDOA. Some experimental results demonstrate that the proposed system can choose the optimal reference microphones and effectively reduce unwanted acoustic noise.