When researching the energy consumption of residential buildings, it is becoming increasingly important to consider how residents use energy. With the advancement of computing power and data analysis techniques, it is now possible to analyze user information using big data techniques. Here, we endeavored to integrate user information with the physical characteristics of residential buildings to analyze how these elements impact energy consumption. Regression analysis was conducted to accurately identify the impact of each element on energy consumption. It was found that six elements were influential in all seasons: the number of exterior walls, housing direction, housing area, number of years occupied, number of household members, and the occupation of the household head. The elements that had an impact in each period were then derived. Based on the results of the regression analysis, input variables for the training of an artificial neural network (ANN) model were selected for each period, and residential energy consumption prediction models were implemented based on actual consumption. The elements identified as those affecting energy consumption, through regression analysis, can be used for implementing prediction models with advanced forms. This study is significant in that we derived influential elements from an integrative perspective.