Fire departments in Malaysia always facing difficulties to reach fire scene location on short time due to lack of information regarding congestion condition of the roads that connecting to the location. Another problem faced by them is false calls from unknown callers. This study is looking for an alternative solution to overcome the problems by developing a vision based fire recognition system. The system is operated on MATLAB r2013 environment and based on a set of rules developed to identify the value of R, G, B, Y, Cb and Cr component in an image. However, this study scope is limited to fire recognition only. The performance of the system is verified using one hundred images. The images are taken during morning, afternoon and evening to ensure the system is robust to different surrounding brightness. However, the images will undergo several steps of pre-processing to minimize noises. Experimental results show that the both color spaces, RGB and YCbCr have high detection rate of fire which is 90% and 100% respectively. YCbCr color space shows the best performance compared to RGB because it can separate luminance from chrominance more effectively than RGB color space.