Biofilm formation is considered as a crucial factor in various oral diseases, such as dental caries. The quorum sensing (QS) signaling system was proved to have a crucial role in the microbial dental plaque biofilm formation of Streptococcus mutans ( S. mutans). LuxS was critical to regulating the QS system and survival of the bacterium, and, therefore, compounds which target LuxS may be a potential therapy for dental caries. The binding activities of 37,170 natural compounds to LuxS were virtually screened in this study. Baicalein and paeonol were chosen for further research of the binding mode and ΔG values with LuxS. Both baicalein and paeonol inhibited the biofilm formation without influence on the growth of S. mutans. Baicalein also distinctly reduced the production of both rhamnolipids and acids. The results provide us with a new approach to combat dental caries instead of the traditional use of antibacterial chemicals.