Research on using nanotechnology to solve shale instability problems in drilling engineering has been increasing. The combination of amphiphilic polymer and silica nanoparticles may be a new way to improve shale stability. Herein, an amphiphilic polymer/nano-silica composite (poly(styrene-methyl methacrylate-acrylamide)/nano-SiO2) was introduced as a novel shale stabilizer SMA/SiO2 for water-based muds, which possessed the advantages of both physical plugging and chemical inhibition during the drilling operations. The SMA/SiO2 was prepared and characterized by Fourier transform infrared spectra (FT-IR), nuclear magnetic resonance (1H-NMR), transmission electron microscope (TEM), particle size distribution (PSD) and thermogravimetric analysis (TGA) experiments, which confirmed that SMA/SiO2 was regularly spherical with nano-scale and showed good high-temperature resistance. To evaluate the plugging capacity of SMA/SiO2, the pressure transmission test and BET analysis were applied. The results indicated SMA/SiO2 was capable of effectively plugging the pores and fractures in shale. To evaluate the hydration inhibition capacity of SMA/SiO2, the rolling dispersion experiment and contact angle test were adopted. The results demonstrated that SMA/SiO2 could reduce the tendency of shale hydration, which was better than potassium chloride (KCl) and polymeric alcohol (JHC). In addition, SMA/SiO2 only created slight variations on the rheological parameters of the water-based muds (WBMs) and showed a significant filtration control performance. Due to the outstanding performance of physical plugging and chemical inhibition, SMA/SiO2 was expected to be a novel shale stabilizer to solve shale instability problems.