Backlash, friction, and servo lag factors often result in protrusion or segment difference phenomenon in the moving speed reversal of a machine tool’s moving table. This phenomenon can be improved by adjusting the backlash control parameters of the machine tool controller, but the control parameters must vary with the feed rate and payload of the moving table. Therefore, this study performed the circular test process for CNC machine tools, and used different feed rate, radius, and payload motion conditions to discuss the effect of backlash control parameters on quadrant protrusions. First, this study used parameter-range reduction combined with the Taguchi method and the binary search algorithm to search for the optimal backlash control parameters in the parameter setting range, so that the machine tool could have preferable quadrant protrusion performance when executing circular tests. Afterward, the correlation of the moving table feed rate, radius, and payload to the quadrant protrusion was analyzed according to the experimental results. The results indicated that the machine tool moving table feed rate had the most apparent effect on quadrant protrusions, and the relationship between the payload and quadrant protrusion was influenced by the moving table feed rate and circular radius simultaneously.