Bayesian Foreground and Shadow Detection in Uncertain Frame Rate Surveillance Videos

We give a new model for foreground-background- shadow separation. Our method extracts the accurate silhouettes of foreground objects even if they have partly background like colors and shadows are observable on the image. It does not need any a priori information about the shapes of the objects, it assumes only they are not point-wise. The method exploits temporal statistics to characterize the background and shadow, and spatial statistics for the foreground. We improved a statistical model for shadows, which is robust regarding the forthcoming artifacts in real-world surveillance scenes, a contains an automatic parameter-update procedure, therefore, the method is able to accommodate to the illumination changes. A Markov Random Field model is used to enhance the accuracy of the separation. We validated our method on outdoor and indoor video sequences captured by the surveillance system of the university campus, and we also tested it on well-known benchmark videos.

Results were published in IEEE Trans. Image Processing, and at the ACCV 2006 conference.

Visit the bechmark database set web page.

Segmentation results 1st column: video image, 2nd: result of a pixel-based preliminary classifier, 3rd: pre. classifier result enhanced by morphology, 4th: proposed MRF result.

 

Different parts of the day on the sequences of an outdoor surveillance camera, segmentation results. Above left: in the morning ('am'), right: at noon, below left: in the afternoon ('pm'), right: wet weather

Demo video for foreground - background - shadow separation

References

[1] Cs. Benedek and T. Szirányi: "Bayesian Foreground and Shadow Detection in Uncertain Frame Rate Surveillance Videos", IEEE Transactions on Image Processing, vol. 17, no. 4, pp. 608-621, 2008,

[2] Cs. Benedek and T. Szirányi: ”Markovian Framework for Foreground-Background-Shadow Segmentation of Real World Video Scenes”, Asian Conference on Computer Vision (ACCV), Lecture Notes in Computer Science, Springer, vol. LNCS 3851, pp. 898-907, Hyderabad, India, January 13-16, 2006