Since the development of the 5-point verbal and 11-point numerical scales for measuring noise annoyance by the ICBEN Team 6, these scales have been widely used in socio-acoustic surveys worldwide, and annoyance responses have been easily compared internationally. However, both the top two categories of the 5–point verbal scale and the top three ones of the 11-point numerical scale are correspond to high annoyance, so it is difficult to precisely compare annoyance responses. Therefore, we calculated differences in day–evening–night-weighted sound pressure levels (Lden) by comparing values corresponding to 10% highly annoyed (HA) on Lden_%HA curves obtained from measurements in 40 datasets regarding surveys conducted in Japan and Vietnam. The results showed that the Lden value corresponding to 10% HA using the 5-point verbal scale was approximately 5 dB lower than that of the 11-point numerical scale. Thus, some correction is required to compare annoyance responses measured by the 5-point verbal and the 11-point numerical scales. The results of this study were also compared with those of a survey in Switzerland.