In the study of excavator automation, the conventional contact-type sensors can be easily damaged because of the collision or vibration when measuring the posture of excavator’s manipulator. For this case, a novel vision-based approach for measuring the posture of excavator’s manipulator is proposed. In this approach, an industrial camera with an infrared filter is used to grab the perspective images of the manipulator under the irradiation of a LED infrared light source; and saddle point detection is adopted to detect the man-made targets which fixed on every arm of the manipulator. Then the selection algorithm based on the fixed geometric dimensioning between two targets is optimized to select the saddle point in targets, and the posture angle of the corresponding arm can be derived from the saddle point coordinates. Furthermore, the detection regions are tracked by anticipating the three group targets’ movement separately. Experiments show that with the infrared device, the vision-based posture measurement system of the excavator’s manipulator can better adapt to different illumination intensity conditions; and the optimization of target selection algorithm reduces the false detection rate from 4 times in every 30s to 1 time in every 30s; also, the average processing time for every manipulator movement frame is reduced from about 70 ms to 50ms by tracking the detection regions separately.