Port passenger station buildings (PPSD) are an important part of transportation buildings in China, which is characterized by large human flow, long operating time, high load of equipment and lighting. The characteristics and functions of PPSD lead to the high energy consumption. However, the energy consumption analysis of PPSD was deficient. In this paper, the characteristics of energy consumption of port passenger stations in cold regions and hot summer and warm winter regions in China were analyzed. Based on eQUEST, the building models of port passenger stations are established. The influencing factors of the building energy consumption were analyzed through orthogonal experiment with SPSS. Results show that the factors such as summer indoor design temperature, heat source form, air conditioning form, window to wall ratio and lighting control mode are the key factors affecting the energy consumption of port passenger station.