Shadow Detection Based on Regions of Light Sources for Object Extraction in Nighttime Video
<p>Video content analytics algorithm based on background subtraction.</p> "> Figure 2
<p>Foreground region extracted in a video content analytics algorithm: (<b>a</b>) Input video frame; (<b>b</b>) Foreground region extracted.</p> "> Figure 3
<p>Shapes of objects and their shadow regions in nighttime video sequences.</p> "> Figure 4
<p>Video content analytics with the proposed shadow detection algorithm for nighttime video sequences.</p> "> Figure 5
<p>Vertical histogram of a foreground region: (<b>a</b>) A foreground region; (<b>b</b>) vertical histogram.</p> "> Figure 6
<p>Partitioning of a foreground region by scanning the vertical histogram: (<b>a</b>) Foreground regions; (<b>b</b>) Foreground pixel histograms and the partitioned results for three scanning methods.</p> "> Figure 7
<p>Foreground region partitioned into two smaller regions: (<b>a</b>) A foreground region extracted; (<b>b</b>) Partitioned regions after the double column scan partitioning.</p> "> Figure 8
<p>Finding the direction of the major axis of a matched ellipse.</p> "> Figure 9
<p>Calculation of the major axis of each partitioned object after vertical histogram analysis: (<b>a</b>) Partitioned objects; (<b>b</b>) Major axes of the virtual matched ellipses.</p> "> Figure 10
<p>Possible regions of light sources for the proposed shadow detection algorithm.</p> "> Figure 11
<p>Results in each step in the proposed shadow detection algorithm: (<b>a</b>) Sequence S5, frame 1280; (<b>b</b>) Sequence S3, frame 826.</p> "> Figure 12
<p>Shadow removal result 1 for a foreground object (120 × 90 pixels from sequence S3): (<b>a</b>) A foreground object; (<b>b</b>) Partitioned objects; (<b>c</b>) A foreground object after shadow removal.</p> "> Figure 13
<p>Shadow removal result 2 for a foreground object (87 × 84 pixels from sequence S3): (<b>a</b>) A foreground object; (<b>b</b>) Partitioned objects; (<b>c</b>) A foreground object after shadow removal.</p> "> Figure 14
<p>Shadow removal result 3 for multiple foreground objects (140 × 105 pixels from sequence S4): (<b>a</b>) Foreground objects; (<b>b</b>) Partitioned objects; (<b>c</b>) Foreground objects after shadow removal.</p> "> Figure 15
<p>Shadow removal result 4 for multiple foreground objects (140 × 105 pixels from sequence S4): (<b>a</b>) Foreground objects; (<b>b</b>) Partitioned objects; (<b>c</b>) Foreground objects after shadow removal.</p> "> Figure 16
<p>Shadow removal result 5 for a foreground object (240 × 180 pixels from sequence S2): (<b>a</b>) Foreground objects; (<b>b</b>) Partitioned objects (<b>c</b>) Foreground object after shadow removal.</p> "> Figure 17
<p>Shadow removal results for various nighttime and one daytime video sequences.</p> ">
Abstract
:1. Introduction
2. Shadow Removal Using Regions of Light Sources in Nighttime Video
2.1. Overview of the Proposed Shadow Detection Algorithm
2.2. Foreground Partitioning Based on Vertical Histogram
Algorithm 1: PartitionBBox (.) |
2.3. Calculation of the Direction of the Major Axis
2.4. Regions of Light Sources
2.5. Detection and Removal of Shadow Regions
3. Experimental Results
3.1. Experimental Environments
3.2. Configuration of RLS
3.3. Measures for Performance Evaluation
3.4. Step-Wise Results for the Proposed Shadow Detection Algorithm
3.4.1. Shadow Removal for a Single Object
3.4.2. Shadow Detection for Multiple Objects
3.5. Performance Comparison
3.5.1. Output Images
3.5.2. Performance Comparison in Terms of Pixel and Object Levels
3.5.3. Shadow Removal Performance for Multiple Objects
4. Conclusions
Acknowledgments
Author Contributions
Conflicts of Interest
Abbreviations
RLS | regions of light sources |
GMM | Gaussian mixture model |
References
- Held, C.; Krumm, J.; Markel, P.; Schenke, R. Intelligent Video Surveillance. Computer 2012, 45, 83–84. [Google Scholar] [CrossRef]
- Del-Blanco, C.; Jaureguizar, F.; Garcia, N. An efficient multiple object detection and tracking framework for automatic counting and video surveillance applications. IEEE Trans. Consum. Electron. 2012, 58, 857–862. [Google Scholar] [CrossRef] [Green Version]
- Chen, Y.K. Challenges and opportunities of internet of things. In Proceedings of the 17th Asia and South Pacific Design Automation Conference, Sydney, Australia, 30 January–2 February 2012; pp. 383–388.
- Fang, X.; Xia, Z.; Su, C.; Xu, T.; Tian, Y.; Wang, Y.; Huang, T. A system based on sequence learning for event detection in surveillance video. In Proceedings of the 2013 20th IEEE International Conference on Image Processing (ICIP), Melbourne, Australia, 15–18 September 2013; pp. 3587–3591.
- Park, J.; Shin, Y.; Jeong, J.; Lee, M. Detection and Tracking of Intruding Objects based on Spatial and Temporal Relationship of Objects. In Proceedings of the 7th International Conference on Information Security and Assurance, Gammarth, Tunisia, 4–6 December 2013; Volume 21, pp. 271–274.
- Kim, J.S.; Yeom, D.H.; Joo, Y.H. Fast and robust algorithm of tracking multiple moving objects for intelligent video surveillance systems. IEEE Trans. Consum. Electron. 2011, 57, 1165–1170. [Google Scholar] [CrossRef]
- Stauffer, C.; Grimson, W.E.L. Adaptive background mixture models for real-time tracking. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, CO, USA, 23–25 June 1999; Volume 2, p. 252.
- Russell, A.; Zou, J.J. Moving shadow detection based on spatial-temporal constancy. In Proceedings of the 2013 7th International Conference on Signal Processing and Communication Systems (ICSPCS), Carrara, Australia, 16–18 December 2013; pp. 1–6.
- Sun, B.; Li, S. Moving Cast Shadow Detection of Vehicle Using Combined Color Models. In Proceedings of the Chinese Conference on Pattern Recognition (CCPR), Chongqing, China, 21–23 October 2010; pp. 1–5.
- Chen, C.T.; Su, C.Y.; Kao, W.C. An enhanced segmentation on vision-based shadow removal for vehicle detection. In Proceedings of the 2010 International Conference on Green Circuits and Systems (ICGCS), Shanghai, China, 21–23 June 2010; pp. 679–682.
- Cucchiara, R.; Grana, C.; Piccardi, M.; Prati, A. Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans. Pattern Anal. Mach. Intell. 2003, 25, 1337–1342. [Google Scholar] [CrossRef] [Green Version]
- Sanin, A.; Sanderson, C.; Lovell, B.C. Shadow detection: A survey and comparative evaluation of recent methods. Pattern Recognit. 2012, 45, 1684–1695. [Google Scholar] [CrossRef]
- Huang, J.B.; Chen, C.S. Moving cast shadow detection using physics-based features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA, 20–25 June 2009; pp. 2310–2317.
- Amato, A.; Mozerov, M.G.; Bagdanov, A.D.; Gonzalez, J. Accurate Moving Cast Shadow Suppression Based on Local Color Constancy Detection. IEEE Trans. Image Process. 2011, 20, 2954–2966. [Google Scholar] [CrossRef] [PubMed]
- Fang, L.Z.; Qiong, W.Y.; Sheng, Y.Z. A method to segment moving vehicle cast shadow based on wavelet transform. Pattern Recognit. Lett. 2008, 29, 2182–2188. [Google Scholar] [CrossRef]
- Yoneyama, A.; Yeh, C.; Kuo, C.C. Moving cast shadow elimination for robust vehicle extraction based on 2D joint vehicle/shadow models. In Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, Miami, FL, USA, 22–22 July 2003; pp. 229–236.
- Hsieh, J.W.; Hu, W.F.; Chang, C.J.; Chen, Y.S. Shadow elimination for effective moving object detection by Gaussian shadow modeling. Image Vis. Comput. 2003, 21, 505–516. [Google Scholar] [CrossRef]
- Chen, C.C.; Aggarwal, J.K. Human Shadow Removal with Unknown Light Source. In Proceedings of the 2010 20th International Conference on Pattern Recognition (ICPR), Istanbul, Turkey, 23–26 August 2010; pp. 2407–2410.
- Sanin, A.; Sanderson, C.; Lovell, B. Improved Shadow Removal for Robust Person Tracking in Surveillance Scenarios. In Proceedings of the 2010 20th International Conference on Pattern Recognition (ICPR), Istanbul, Turkey, 23–26 August 2010; pp. 141–144.
- Panicker, J.V.; Wilscy, M. Detection of moving cast shadows using edge information. In Proceedings of the 2010 2nd International Conference on Computer and Automation Engineering (ICCAE), Singapore, 26–28 February 2010; Volume 5, pp. 817–821.
- Zhang, W.; Fang, X.Z.; Xu, Y. Detection of moving cast shadows using image orthogonal transform. In Proceedings of the 2006 18th International Conference on Pattern Recognition, Hong Kong, China, 20–24 August 2006; Volume 1, pp. 626–629.
- Zhang, L.; He, X. Fake Shadow Detection Based on SIFT Features Matching. In Proceedings of the WASE International Conference on Information Engineering (ICIE), Beidaihe, China, 14–15 August 2010; Volume 1, pp. 216–220.
- Leone, A.; Distante, C. Shadow detection for moving objects based on texture analysis. Pattern Recognit. 2007, 40, 1222–1233. [Google Scholar] [CrossRef]
- Khan, S.H.; Bennamoun, M.; Sohel, F.; Togneri, R. Automatic Feature Learning for Robust Shadow Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 1939–1946.
- Khan, S.H.; Bennamoun, M.; Sohel, F.; Togneri, R. Automatic Shadow Detection and Removal from a Single Image. IEEE Trans. Pattern Anal. Mach. Intell. 2016, 38, 431–446. [Google Scholar] [CrossRef] [PubMed]
- Lee, J.T.; Lim, K.T.; Chung, Y. Moving Shadow Detection from Background Image and Deep Learning. In Image and Video Technology—PSIVT 2015 Workshops: RV 2015, GPID 2013, VG 2015, EO4AS 2015, MCBMIIA 2015, and VSWS 2015, Auckland, New Zealand, November 23–27, 2015. Revised Selected Papers; Springer: New York, NY, USA, 2016; Volume 9555, pp. 299–306. [Google Scholar]
- Martel-Brisson, N.; Zaccarin, A. Kernel-based learning of cast shadows from a physical model of light sources and surfaces for low-level segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 23–28 June 2008; pp. 1–8.
- Porikli, F.; Thornton, J. Shadow flow: A recursive method to learn moving cast shadows. In Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China, 15–21 October 2005; Volume 1, pp. 891–898.
- Liu, Z.; Huang, K.; Tan, T.; Wang, L. Cast Shadow Removal Combining Local and Global Features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007; pp. 1–8.
- Kalentev, O.; Rai, A.; Kemnitz, S.; Schneider, R. Connected component labeling on a 2D grid using CUDA. J. Parallel Distrib. Comput. 2011, 71, 615–620. [Google Scholar] [CrossRef]
- Computational Vision Group, Reading University. PETS: Performance Evaluation of Tracking and Surveillance. Available online: http://www.cvg.reading.ac.uk/slides/pets.html (accessed on 14 September 2015).
- The Centre for the Protection of National Infrastructure (CPNI). Imagery Library for Intelligent Detection Systems: The i-LIDS User Guide; CPNI: London, UK, 2011.
Seq. Number | Length (Frames) | Location | Time | Interference | RLS |
---|---|---|---|---|---|
1 | 300 | a soccer field | night | multiple | |
2 | 200 | an entrance of a building | night | multiple | |
3 | 200 | a running track | night | single | |
4 | 300 | a futsal field | night | multiple | |
5 | 1030 | an entrance of a building | night | single | |
6 | 210 | an entrance of a building | night | single | |
7 | 490 | a crossroad | day | multiple | |
8 | 200 | a forked road | day | multiple |
Sequence | BSR | Proposed | Chr | Geo | Phy | srTex | lrTex | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
η | η | η | η | η | η | |||||||||
night | S1 | 79.9 | 90.6 | 98.9 | 36.5 | 25.1 | 53.4 | 75.0 | 0.1 | 74.8 | 13.5 | 81.2 | 6.0 | 75.0 |
S2 | 16.9 | 96.0 | 93.4 | 88.1 | 34.9 | 62.3 | 50.0 | 7.0 | 21.2 | 51.1 | 79.0 | 11.0 | 34.9 | |
S3 | 35.8 | 88.7 | 99.0 | 80.1 | 19.4 | 74.0 | 36.8 | 0.5 | 36.3 | 34.3 | 16.9 | 16.3 | 43.3 | |
S4 | 53.5 | 98.0 | 57.7 | 47.6 | 22.7 | 44.8 | 43.1 | 11.2 | 47.4 | 45.2 | 37.7 | 18.0 | 49.2 | |
S5 | 53.4 | 94.9 | 92.4 | 85.1 | 57.5 | 60.2 | 75.3 | 4.1 | 57.3 | 37.1 | 51.4 | 4.7 | 59.7 | |
S6 | 52.1 | 94.4 | 98.1 | 86.0 | 35.1 | 71.9 | 38.4 | 8.0 | 70.1 | 51.3 | 87.7 | 51.6 | 71.1 | |
average | 48.6 | 93.8 | 89.9 | 70.5 | 32.4 | 61.1 | 53.1 | 5.2 | 51.2 | 38.7 | 59.0 | 18.0 | 55.5 | |
day | S7 | 84.8 | 93.8 | 97.1 | 96.2 | 60.4 | 67.5 | 49.3 | 69.9 | 95.6 | 89.0 | 85.2 | 92.3 | 81.5 |
S8 | 35.8 | 80.2 | 98.3 | 88.8 | 62.5 | 67.6 | 63.9 | 60.7 | 95.5 | 69.7 | 81.1 | 13.5 | 50.1 |
Merged Objects by Shadow | Removal Rate of Merged Objects by Shadow (%) | ||
---|---|---|---|
before | after | ||
S1 | 12 | 0 | 100.0 |
S2 | 59 | 0 | 100.0 |
S4 | 55 | 16 | 70.9 |
total | 126 | 16 | 87.3 |
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ( http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, G.-b.; Lee, M.-j.; Lee, W.-K.; Park, J.-h.; Kim, T.-H. Shadow Detection Based on Regions of Light Sources for Object Extraction in Nighttime Video. Sensors 2017, 17, 659. https://doi.org/10.3390/s17030659
Lee G-b, Lee M-j, Lee W-K, Park J-h, Kim T-H. Shadow Detection Based on Regions of Light Sources for Object Extraction in Nighttime Video. Sensors. 2017; 17(3):659. https://doi.org/10.3390/s17030659
Chicago/Turabian StyleLee, Gil-beom, Myeong-jin Lee, Woo-Kyung Lee, Joo-heon Park, and Tae-Hwan Kim. 2017. "Shadow Detection Based on Regions of Light Sources for Object Extraction in Nighttime Video" Sensors 17, no. 3: 659. https://doi.org/10.3390/s17030659