In this paper, we propose an automated video surveillance system for kidnapping detection using featurebased characteristics. The localization of moving objects in a video stream and human behavior estimation are key techniques adopted by the proposed system. Some motion characteristics are determined from video streams, and using metrics such as a feature vector, the system automatically classifies the video streams into criminal and non-criminal scenes. The proposed system is called an intelligent security camera. We consider many types of scenarios for the training data set. After constructing the classifier, we use test sequences that are continuous video streams of human behavior consisting of several actions in succession. The experimental results show that the system can effectively detect criminal scenes, such as a kidnapping, by distinguishing human behavior.