In this study, event-related potential (ERP) was used to examine whether the brain has an inhibition effect on the interference of audio-visual information in the Chinese interface. Concrete icons (flame and snowflake) or Chinese characters ([Formula: see text] and [Formula: see text]) with opposite semantics were used as target carriers, and colors (red and blue) and speeches ([Formula: see text] and [Formula: see text]) were used as audio-visual intervention stimuli. In the experiment, target carrier and audio-visual intervention were presented in a random combination, and the subjects needed to determine whether the semantics of the two matched quickly. By comparing the overall cognitive performance of two carriers, it was found that the brain had a more significant inhibition effect on audio-visual intervention stimuli with different semantics (SBH/LBH and SRC/LRC) relative to the same semantics (SRH/LRH). The semantic mismatch caused significant N400, indicating that semantic interference in the interface information would trigger the brain’s inhibition effect. Therefore, the more complex the semantic matching of interface information was, the higher the amplitude of N400 became. The results confirmed that the semantic relationship between target carrier and audio-visual intervention was the key factor affecting the cognitive inhibition effect. Moreover, under different intervention stimuli, the ERP’s negative activity caused by Chinese characters in frontal and parietal-occipital regions was more evident than that by concrete icons, indicating that concrete icons had a lower inhibition effect than Chinese characters. Therefore, we considered that this inhibition effect was based on the semantic constraints of the target carrier itself, which might come from the knowledge learning and intuitive experience stored in the human brain.