Improving the school governance structure and establishing a modern school system are the current research focuses in elementary and middle school management. Through a comparative analysis of the school governance structure of four provinces and cities in China (Beijing-Shanghai-Jiangsu-Guangdong, BSJG) and PISA2015 high-scored countries/economies, we found that the school autonomy in seven major aspects including “teacher selection, teacher dismissal, evaluation policy, enrollment policy, textbook selection, curriculum content, and curriculum design” in BSJG schools was significantly lower than that of high-scored countries/economies. The average decision-making of BSJG principals and teachers in various affairs was also substantially lower than the high-scored countries/economies. The multilevel analysis found that the impact of school governance structure on student performance presented different patterns between BSJG and high-scored countries/economies. Therefore, China needs to (i) expand the autonomy of school management further and establish a new government-school relationship; (ii) give priority to curriculum management and ensure its autonomy in schools; (iii) improve the principal accountability system, and strengthen the principal’s power and responsibility in school management; (iv) strengthen democratic management, thereby promoting teachers’ participation in the decision-making of school affairs.