A matching algorithm that evaluates the difference between model and calculated flows for obstacle detection in video sequences is presented. A stabilization method for obstacle detection by median filtering to overcome instability in the computation of optical flow is also presented. Since optical flow is a scene-independent measurement, the proposed algorithm can be applied to various situations, whereas most of existing color- and texture-based algorithms depend on specific scenes, such as roadway and indoor scenes. An experiment is conducted with three real image sequences, in which a static box or a moving toy car appears, to evaluate the performance in terms of accuracy under varying thresholds using a receiver operating characteristic (ROC) curve. For the three image sequences, the ROC curves show, in the best case, that the false positive fraction and the true positive fraction is 19.0% and 79.6%, 11.4% and 84.5%, 19.0% and 85.4%, respectively. The processing time per frame is 19.38msec. on 2.0GHz Pentium 4, which is less than the video-frame rate.