In this study, we determined the factors that influence of the temperature on an asphalt pavement by developing a two-dimensional unsteady temperature numerical calculation model using the finite difference method and Matlab. Based on the temperatures obtained by a buried sensor in a construction project, we collected the temperatures at different depths in the pavement structure in real time, and we then compared and analyzed the calculated and measured data. The results showed that the temperature in the asphalt pavement structure was significantly correlated with meteorological factors, such as the air temperature, but it also exhibited obvious hysteresis. Compared with the measured data, the maximum deviation in the numerical model based on the variations in the atmospheric temperature and solar radiation was 3 °C. Thus, it is necessary to effectively optimize the selection of asphalt pavement materials by simulating the temperature conditions in the asphalt pavement structure.