Serological test plays an essential role in monitoring and combating COVID-19 pandemic. Recombinant spike protein (S protein), especially S1 protein is one of the major reagents for serological tests. However, the high cost in production of S protein, and the possible cross-reactivity with other human coronaviruses poses unneglectable challenges. Taking advantage of a peptide microarray of full spike protein coverage, we analyzed 2,434 sera from 858 COVID-19 patients, sera from 63 asymptomatic patients and 610 controls collected from multiple clinical centers. Based on the results of the peptide microarray, we identified several S protein derived 12-mer peptides that have high diagnosis performance. Particularly, for monitoring IgG response, one peptide (aa 1148-1159 or S2-78) has a comparable sensitivity (95.5%, 95% CI 93.7-96.9%) and specificity (96.7%, 95% CI 94.8-98.0%) to that of S1 protein for detection of both COVID-19 patients and asymptomatic infections. Furthermore, the performance of S2-78 IgG for diagnosis was successfully validated by ELISA with an independent sample cohort. By combining S2-78/ S1 with other peptides, a two-step strategy was proposed to ensure both the sensitivity and specificity of S protein based serological assay. The peptide/s identified in this study could be applied independently or in combination with S1 protein for accurate, affordable, and accessible COVID-19 diagnosis.