Recently, in the solar energy society, several key technologies have been reported to meet a grid parity, such as cost-efficient materials, simple processes, and designs. Among them, the assistive plasmonic of metal nanoparticles (MNPs) integrating with the downshifting on luminescent materials attracts much attention. Hereby, Si-based Schottky junction solar cells are fabricated and examined to enhance the performance. CdSe/ZnS quantum dots (QDs) with different gold nanoparticles (Au NPs) sizes were incorporated on a Si light absorbing layer. Due to the light scattering effect from plasmonic resonance, the sole Au NPs layer results in the overall enhancement of Si solar cell’s efficiency in the visible spectrum. However, the back-scattering and high reflectance of Au NPs lead to efficiency loss in the UV region. Therefore, the QDs layer acting as a luminescent downshifter is deployed for further efficiency enhancement. The QDs layer absorbs high-energy photons and re-emits lower energy photons in 528 nm of wavelength. Such a downshift layer can enhance the overall efficiency of Si solar cells due to poor intrinsic spectral response in the UV region. The optical properties of Au NPs and CdSe QDs, along with the electrical properties of solar cells in combination with Au/QD layers, are studied in depth. Moreover, the influence of Au NPs size on the solar cell performance has been investigated. Upon decreasing the diameters of Au NPs, the blueshift of absorbance has been observed, cooperating with QDs, which leads to the improvement of the quantum efficiency in the broadband of the solar spectrum.