In view of the detection of moving object in video sequence, the traditional moving object detection algorithms are researched. The paper presents a new algorithm for object detection based on initial contour and improved variational GAC model. First, this method built up background model utilizing Gaussian mixture model and background subtraction to extract initial contour of the object; taking initial contour as initial value of curve evolution. Then, an improved restriction item is introduced into variational GAC vector model, the proposed restriction item that is a nonlinear hear equation with normalized diffusion rate, therefore re-initialization procedure of level set function is completely eliminated. Iteration number of curve evolution and run time is reduced. The experimental show that accurate contour of moving object is got and this algorithm is effective and feasible in real video environment.