This paper presents a framework that creates background, foreground and a temporal summarization of the motions in a scene. The method is based on the Dominant Motion Assumption (DMA), where the background has a parametric motion and occupies the main part of the scene. Under this assumption, we present a robust optical flow based method to extract the moving parts of the scene using the clustering capabilities of mixtures of Gaussians. A general mosaicing method to summarize the background, the foreground and the trajectories of objects in the scene is also presented.