With the development of situational awareness in intrusion defense, a proactive response is a realistic and effective approach against the attackers. It is assumed that each player can update knowledge of the opponent and assess possible future scenarios of the dynamic game based on their previous interactions. Therefore, finding the best current move of the defender is modeled as a discrete-time stochastic control problem. An on-line, convergent, scenario based proactive defense (SPD) algorithm considering adaptive learning is developed based on differential dynamic programming (DDP) to solve the associated optimal control problem. Numerical experiment shows that the new algorithm can help the defender in finding the best dynamic strategies quickly and efficiently. Moreover, the SPD algorithm can provide optimal defensive efforts against possible future attacks within an appropriate time window, so the success of the attack in the possible future interactions can be assessed, improving situational awareness in computer network security.