Efficient and precise three-dimensional (3D) measurement is an important issue in the field of machine vision. In this paper, a measurement method for indoor key points is proposed with structured lights and omnidirectional vision system and the system can achieve the wide field of view and accurate results. In this paper, the process of obtaining indoor key points is as follows: Firstly, through the analysis of the system imaging model, an omnidirectional vision system based on structured light is constructed. Secondly, the full convolution neural network is used to estimate the scene for the dataset. Then, according to the geometric relationship between the scenery point and its reference point in structured light, for obtaining the 3D coordinates of the unstructured light point is presented. Finally, combining the full convolution network model and the structured light 3D vision model, the 3D mathematical representation of the key points of the indoor scene frame is completed. The experimental results proved that the proposed method can accurately reconstruct indoor scenes, and the measurement error is about 2%.