This paper investigates the nanoscale effect on the effective bulk modulus of nanoparticle-reinforced polymer. An interface-based model is introduced in this work to study the nanoscale effects on the effective properties of heterogeneous materials. That interface model is able to capture discontinuity of mechanical fields across the surface between the nanoparticle and matrix. A generalized self-consistent scheme is then employed to determine the effective bulk modulus. It has been seen from the results that, in a certain range of limits, the influence of nanoscale effects on effective properties of heterogeneous materials is significant and needs to be taken into account. In particular, when the nanoparticle radius is smaller than 10 nm, the value of effective bulk modulus significantly increases when the characteristic size of nanofillers decreases. Besides, it is seen that the harder the inclusion, the smaller the nanoscale influence effects on the overall behaviors of composite materials. Finally, parametric studies in terms of surface strength and filler’s volume fractions are investigated and discussed, together with a comparison between the proposed model and other contributions in the literature.