Haptic interaction is a important paradigm to be investigated further in virtual reality technology. Though the human sense of sight is generally more sensitive than that of touch, it is subject to optical illusions. We took experiments to investigate the characteristics of shape recognition based on the sense of sight and of touch or haptics in an optical illusion environment. The result is that the evaluated value of recognition is greatly affected by optical illusion. Furthermore, the differential threshold becomes larger when haptic information was added. Therefore, the design of multi modal interfaces requires much consideration so that the visual environmental setting does not cause optical illusion. Also, two methods for haptic display are considered.