KR102107137B1 - Method and Apparatus for Detecting Event by Using Pan-Tilt-Zoom Camera - Google Patents

Abstract

Disclosed is an event detection method and apparatus using a pan tilt zoom camera.
A fan who acquires a video frame for a predetermined preset image from a PTZ camera and performs event monitoring using a new or existing background model generated based on a confirmation result confirming whether or not a scene change in the preset image for the acquired video frame It relates to a method and device for detecting an event using a tilt zoom camera.

Description

Method and Apparatus for Detecting Event by Using Pan-Tilt-Zoom Camera}

The present embodiment relates to a method for detecting an event using a pan tilt zoom camera and an apparatus therefor.

The contents described in this section merely provide background information for this embodiment, and do not constitute a prior art.

PTZ (Pan-Tilt-Zoom) camera is a structure that combines a camera equipped with a full-auto zoom lens to a pan / tilt drive device that can be rotated up, down, left, and right, and rotates and rotates the camera through external control. Zoom in / out is possible. Therefore, compared to a fixed camera, although it is expensive, it has the advantage of monitoring a large area around the camera through the rotation of the camera, and zooming the distance / near field through the zoom lens and allowing the user to view the object of interest in an enlarged manner.

The event monitoring device using a general PTZ camera generates a new background model for object monitoring whenever an image frame is received, or a new control model for object movement for each preset image by receiving a control signal for movement of the PTZ camera. To perform event monitoring. However, the general event monitoring apparatus has a problem in that the moving object detection performance is deteriorated and the computation amount for generating the background model is increased because a new background model is generated whenever the movement of the PTZ camera occurs.

In the present embodiment, an image frame for a predetermined preset image is obtained from a PTZ camera, and an event is monitored using a new or existing background model generated based on a confirmation result confirming whether or not a scene change of a preset image for the acquired image frame is made. The main objective is to provide a method and apparatus for detecting an event using a pan tilt zoom camera that performs a.

According to an aspect of the present embodiment, an image acquisition unit that acquires a captured image from a camera; A scene change confirmation unit comparing the current image frame and the previous image frame included in the captured image to detect movement of the camera; A background model control unit that generates a background model for a photographing direction of the current image frame when the movement of the camera is detected; And it provides an event detection device comprising an event detection unit for detecting an event using the background model.

In addition, according to another aspect of the present embodiment, an image acquisition process for acquiring a captured image from a camera; A scene change confirmation process of comparing the current image frame and the previous image frame included in the captured image to detect movement of the camera; A background model control process for generating a background model for a shooting direction of the current image frame when the movement of the camera is detected; And an event detection process for detecting an event using the background model.

As described above, according to the present embodiment, by using the updated background model, which extracts a background model similar to the video frame input from the event detection device, for event monitoring, it is possible to improve moving object detection performance for event monitoring and detect moving objects. It has the effect of reducing the amount of computation for.

1 is a block diagram schematically showing an event detection system according to the present embodiment.
2 is a flowchart illustrating a method of detecting an event by controlling a background model for each scene according to the present embodiment.
3 is an exemplary view for explaining an operation of confirming a scene change in the event detection device according to the present embodiment.
4 is an exemplary diagram for comparing an event detection operation and a general event detection operation according to the present embodiment.
5 is an exemplary view showing an area for comparing a background model for event detection according to the present embodiment.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing an event detection system according to the present embodiment.

The event detection system according to the present embodiment includes a PTZ (Pan-Tilt-Zoom) camera 110 and an event detection device 120. The event detection device 120 includes an image acquisition unit 130, a scene change confirmation unit 140, a background model control unit 150, and an event detection unit 180.

The PTZ camera 110 is a camera having functions such as Pan, Tilt, and Zoom, and monitors a user's desired place or area vertically, horizontally, enlarged or reduced. Since the PTZ camera 110 can designate a position to be monitored, the surveillance area of the camera is wider compared to a camera that monitors only a fixed position.

The PTZ camera 110 is connected to a management device (not shown) using wired or wireless communication, and performs a monitoring function in a monitoring mode corresponding to a control command received from the management device (not shown). Here, the monitoring mode includes a manual mode in which the user manually controls the PTZ camera 100 by manipulating the joystick to perform monitoring, and a preset mode in which the user pre-sets the monitoring position at a constant time or period.

The PTZ camera 110 according to the present embodiment may be a speed dome camera disposed at a fixed location in a specific place set by an administrator, and the PTZ camera 110 is an event detection device connected using wired or wireless communication ( 120) to transmit a video frame for the preset image captured in the preset mode.

In other words, the PTZ camera 110 repeatedly takes a preset image for a preset place or area using functions such as Pan, Tilt, and Zoom at a predetermined time or period. For example, the PTZ camera 110 may operate in a preset mode that moves at 120 ° intervals every 20 to 30 seconds, which is a preset period, based on a control signal obtained from a management device (not shown). In this case, the PTZ camera 110 photographs the first preset image (0 ° to 120 °) for the first 20 seconds to 30 seconds, and then the second preset image (120 ° to 240 °) for 20 seconds to 30 seconds. And then, for 20 to 30 seconds, a third preset image (240 ° to 360 °) is photographed. After taking the third preset image, the PTZ camera 110 photographs the first preset image (0 ° to 120 °) again for 20 seconds to 30 seconds, and repeats a predetermined cycle to repeat the first preset image, the second The preset image and the third preset image are photographed.

The event detection device 120 obtains an image frame for a predetermined preset image from the PTZ camera 110, and based on the background model generated based on a result of checking whether a preset image is changed for the obtained image frame. To perform event monitoring.

The image acquisition unit 130 receives an image frame for a preset image taken from the PTZ camera 110. The image acquisition unit 130 preferably receives an image frame for a preset image captured in real time, but is not limited thereto, and may sequentially acquire image frames for a plurality of pre-imaged preset images.

In other words, the image acquisition unit 130 acquires an image frame for a preset image photographed at a predetermined period for a predetermined place or area using the PTZ camera 110. For example, the image acquisition unit 130 receives the image frame of the first preset image photographing the preset first place from the PTZ camera 110. In addition, after receiving all of the image frames for the first preset image, the image acquisition unit 130 continuously moves the image frames of the second preset image that is captured in the second place by moving the PTZ camera 110 according to a preset cycle. To receive.

The image acquisition unit 130 may repeatedly receive a predetermined preset image from the PTZ camera 110 according to a preset period. For example, the image acquisition unit 130 may include a first preset image (eg, 0 ° to 120 °) for the first location, a second preset image (eg, 120 ° to 240 °) for the second location, and a third location. After receiving the third preset image for (for example, 240 ° to 360 °), the first preset image, the second preset image and the third preset image in order of receiving the image frame.

The scene change confirmation unit 140 is a module for determining whether the PTZ camera 110 moves, and checks whether a preset image of the image frame received by the image acquisition unit 130 changes. In other words, the scene change confirmation unit 140 checks whether the image frame for a predetermined preset image received from the image acquisition unit 130 is included in the same preset image as the previous (neighbor) image frame previously received. . For example, the scene change confirmation unit 140 compares an image frame received from the image acquisition unit 130 with a previously received (neighbor) image frame, and when a difference exists above a preset threshold or ratio, It is determined that the preset image including the image frame and the preset image including the previous image frame are different scenes, that is, different preset images.

Meanwhile, the scene change confirmation unit 140 compares an image frame received from the image acquisition unit 130 with a previously received image frame, and when a difference exists below a preset threshold or ratio, an image preset is included. It is determined that the preset image including the preset image and the previous image frame is the same scene, that is, the same preset image with no scene change.

Hereinafter, an operation of determining whether a scene is changed by comparing the current image frame with the previous (neighbor) image frame by the scene change confirmation unit 140 will be described.

The scene change confirmation unit 140 compares the brightness of all or some pixel values of the current image frame and the previous image frame, and checks a scene change of the preset image based on the pixel difference value calculated.

For example, the scene change confirmation unit 140 may include predetermined lines such as vertical, diagonal, and horizontal for each of the current image frame and the previous image frame including w (number of horizontal pixels) × h (number of vertical pixels) pixels. The brightness or color of the pixel positioned on the image is sampled.

The scene change confirmation unit 140 compares the pixel values sampled on a predetermined line of the current image frame and the previous image frame to calculate a pixel difference value, and when the calculated pixel value is equal to or greater than a preset threshold, the current image frame It is determined that the preset image including the has changed. Here, the scene change confirmation unit 140 is described as checking the scene change of the preset image using only pixels on a predetermined line for each of the current image frame and the previous image frame, but this reduces the amount of computation for detecting the scene change. In order to do so, if the scene change confirmation unit 140 can determine the scene change, the number of pixels used in the current image frame and the previous image frame can be changed and applied.

On the other hand, the scene change confirmation unit 140 described in FIG. 1 is described as using a visual rhythm method to determine a scene change of an image frame, but is not limited thereto, and an image frame adjacent to the image frame Any method can be applied if it is possible to determine a scene change of a preset image using.

The background model control unit 150 generates or extracts a background model corresponding to the current image frame when the current image frame is determined as the image frame in which the preset image is changed by the scene change confirmation unit 140 to monitor the event 180 Transfer to. That is, the background model control unit 150 updates the existing background model by extracting the existing background model if there is a pre-stored existing background model for the preset image including the current image frame judged to have changed scene. If there is no existing stored background model, a new background model corresponding to the video frame is generated and the background model is transmitted to the event monitoring unit 180.

The background model control unit 150 according to the present embodiment includes an existing background model processing unit 160 and a new background model processing unit 170.

The existing background model processing unit 160 extracts the existing background model for the preset image including the current image frame. In other words, the existing background model processing unit 160 extracts an existing background model determined to be similar by comparing the current image frame with a plurality of previously generated background models. Here, the existing background model refers to a background model that continuously updates each of a plurality of image frames for a specific preset image received in a new background model of a specific preset image including a current image frame.

For example, when the current background frame is included in the first preset image, the existing background model processing unit 160 checks whether an existing background model corresponding to the first preset image exists and extracts the existing background model. Here, the existing background model corresponding to the first preset image is continuously updated by updating the image frame for the first preset image received based on the background model generated from the first image frame for the first preset image. Refers to a background model created before an image frame for an image is received.

The existing background model processing unit 160 may extract an existing background model from a database (not shown) located outside the event detection device 120, but from a storage unit (not shown) provided inside the event detection device 120. Existing background models can also be extracted.

Hereinafter, an operation of comparing the current image frame with a plurality of previously stored background models in the existing background model processing unit 160 to extract an existing background model corresponding to a preset image including the current image frame will be described.

The existing background model processing unit 160 compares each of the current image frame and a plurality of previously stored background models to calculate a comparison difference value. The existing background model processing unit 160 calculates the comparison difference value using Equation (1).

(e: comparison difference value, m: background model, W: current image frame width, H: current image frame height, x, y: predetermined pixel (coordinate value), I: input current image frame)

The existing background model processing unit 160 is a color value of the background model m _i at a predetermined pixel (x, y) position at I (x, y), which is the color value of the current image frame at a predetermined pixel (x, y) position. A comparison difference value (e (m _i )) is calculated by adding a sequence of difference values minus m _i (x, y) to the width and height of the image frame, respectively.

The existing background model processing unit 160 selects a background model having a minimum value among a plurality of comparison difference values (e (m _i )) calculated by comparing each of the previously stored background models with the current image frame, and selecting the background model with the selected background model. When the comparison difference value (e (m _i )) is smaller than the preset threshold value T, the selected background model is extracted as an existing background model similar to the current image frame.

The existing background model processing unit 160 compares each of a plurality of previously stored background models with the current image frame to calculate a comparison difference value, in order to reduce errors due to objects included in the current image frame, objects among the current image frames A comparison difference value can be calculated using only the background expected area, excluding the area where a number is expected to exist. Here, the description of the background expected area will be described in FIG. 5.

The existing background model processing unit 160 updates (learns) the current image frame to the extracted existing background model to generate an updated background model. Hereinafter, an operation of updating the current image frame to the existing background model by the existing background model processing unit 160 will be described.

The existing background model processor 160 updates the background model by calculating a weighted sum of the existing background model and the newly received image frame. In other words, the existing background model processing unit 160 weights the existing background model m _t (x, y) of the pixel (x, y) with the image frame I _t (x, y) input at t at the current time t. Use to update to update background model m _{t + 1} (x, y) at t + 1 hour. Equation 2 for updating the above-described existing background model to the updated background model is as follows.

(m: background model, x, y: predetermined pixel (coordinate value), α: background model learning rate, t: current time, I: input image frame)

In the above-described Equation 2, the existing background model processing unit 160 adjusts the rate of updating the existing background model to the updated background model by the learning rate α, and the learning rate α is set to a value between 0 and 1.

Meanwhile, the existing background model processing unit 160 may update the current image frame without a change in the preset image from the scene change confirmation unit 140. In other words, when the current image frame without a change in the preset image is received, the existing background model processing unit 160 may update the current image frame to the previously used previous background model to generate an updated background model.

The new background model processing unit 170 generates a new background model for the current image frame. In other words, when there is no existing background model corresponding to the preset image including the current image frame, the new background model processing unit 170 generates a new background model for the current image frame. For example, when the current background frame is included in the first preset image and the existing background model for the second preset image and the third preset image exists in the new background model processing unit 170, for the first preset image A new background model is generated based on the current image frame.

The event detection unit 180 receives a background model from the background model control unit 150 and detects an event for an object moving using a region of interest excluding the background model from the current image frame. In other words, the event detection unit 180 detects an event within a region of interest excluding the generated new background model or the updated background model received from the background model control unit 150. The event detection unit 180 transmits event detection result information in a region of interest to a user or an administrator to be output.

The event detection unit 180 sets a pixel block within a region of interest, performs image segmentation using a method such as edge detection based on the pixel block and comparison of pre-stored reference images to classify objects and perform event detection. However, the present invention is not limited thereto, and object recognition may be performed by sliding a preset window in the region of interest.

2 is a flowchart illustrating a method of detecting an event by controlling a background model for each scene according to the present embodiment.

The event detection device 120 acquires a new video frame from the PTZ camera 110 (S210). The event detection device 120 acquires an image frame for a preset image photographed at a predetermined period for a predetermined place or area using the PTZ camera 110.

The event detection device 120 checks whether a scene is changed for a new video frame (S220). The event detection device 120 checks whether an image frame for a predetermined preset image is included in the same preset image as the previous (neighbor) image frame previously received. The event detection device 120 preferably uses a visual rhythm method to determine a scene change of an image frame, but is not limited thereto.

When the scene for the new video frame changes, the event detection device 120 checks whether an existing background model similar to the new video frame exists among the previously stored background models (S230). When an existing background model similar to the new video frame exists, the event detection device 120 extracts an existing background model similar to the new video frame (S240), and updates the existing background model based on the new video frame (S250). .

When there is no existing background model similar to the new video frame, the event detection device 120 generates a new background model based on the new video frame (S260).

Meanwhile, if the scene for the new video frame has not changed, the event detection device 120 updates the previous background model for the previous video frame based on the new video frame (S250).

The event detection device 120 determines a region of interest based on a new background model or an updated existing background model, detects an event for an object in the region of interest (S270), and detects event detection result information in the region of interest by the user or It is transmitted to the manager to be output (S280).

After detecting an event, the event detection device 120 terminates the event detection and repeats the operation of acquiring a new image frame when the detection end signal is not input (S290).

3 is an exemplary view for explaining an operation of confirming a scene change in the event detection device according to the present embodiment.

As described in FIG. 3, the event monitoring device 120 determines whether the PTZ camera 110 moves using the current video frame and the previous video frame. In FIG. 3, it is assumed that the event monitoring apparatus 120 has received the first image frame 310 for the second preset image and the second image frame 320 for the sixth preset image as neighboring image frames.

The event monitoring device 120 first sampling information 314 that samples the color of n pixels located on the first diagonal line 312 of the first image frame 310, S (O) to S (n). The second sampling information 324 is obtained by sampling the color of n pixels located on S (O) to S (n) which are the second diagonal lines 322 of the second image frame 320.

The event monitoring apparatus 120 calculates the color difference value d for every two columns corresponding to the adjacent time t between the first sampling information 314 and the second sampling information 324.

The event monitoring apparatus 120 may be configured to remove the color difference value (d) calculated between the first sampling information 314 and the second sampling information 324 or more, if a peak value 330 is greater than or equal to a preset threshold. It is determined that there is a change in the preset image by moving the PTZ camera 110 while the second image frame 320 is received in the one image frame 310.

4 is an exemplary diagram for comparing an event detection operation and a general event detection operation according to the present embodiment.

4 (a) shows an operation of generating a preset image for general event monitoring and a background model accordingly. As shown in (a) of FIG. 4, the general event detection device sequentially acquires a first preset image, a second preset image and a third preset image from the PTZ camera 110, and three preset images at a preset cycle. It is obtained repeatedly. Here, the general event detecting device generates a background model for each preset image received from the PTZ camera 110, and performs an operation of detecting an event for a moving object except for the background model generated for each preset image.

4B illustrates an operation of generating a preset image for event monitoring and a background model according to the event monitoring according to the present embodiment. As shown in (b) of FIG. 4, the event monitoring device 120 sequentially orders the first preset image 410, the second preset image 420, and the third preset image 430 from the PTZ camera 110. To acquire. Thereafter, the first repeat preset image 412, the second repeat preset image 422, and the third repeat preset image 432 are repeatedly obtained.

As shown in FIG. 4B, the event monitoring device 120 generates a first new background model when receiving the first preset image 410. Thereafter, when receiving the second preset image 420, the event monitoring device 120 generates a second new background model. Thereafter, when receiving the third preset image 430, the event monitoring device 120 generates a third new background model.

Thereafter, when the event monitoring device 120 repeatedly receives the first repeating preset image 412, the event monitoring device 120 extracts a first new background model determined to be similar to the first repeating preset image 412, and the first new background. Except for the updated background model in which the first repeating preset image 412 is updated in the model, an operation for detecting an event for a moving object is performed. When the second repeating preset image 422 and the third repeating preset image 432 are received by the event monitoring device 120, it is the same as the operation of updating the background model in the first repeating preset image 412. Is omitted.

5 is an exemplary view showing an area for comparing a background model for event detection according to the present embodiment.

The event detection device 120 compares the current video frame input from the PTZ camera 110 with a plurality of previously stored background models to calculate a comparison difference value, and based on the comparison difference value, compares the current video frame to the preset image including the current video frame. Extract the existing background model for.

In calculating the comparison difference value, the event detection device 120 may generate an error due to an object included in the current image frame when all regions of the current image frame are used. In order to reduce the error occurrence rate, the event monitoring apparatus 120 calculates a comparison difference value using only the background prediction area excluding the area where the object is expected to exist in the current image frame.

FIG. 5A shows an area in which a comparison difference value is calculated using only the first background prediction area 510 when it is determined that there are many objects moving in the center area of the current image frame. FIG. 5B shows an area in which a comparison difference value is calculated using only the second background prediction area 520 when it is determined that there are many objects moving in the upper area of the current image frame. 5 (c), it is determined that there are many objects moving in a specific region of the current image frame, and the third background excluding the region corresponding to the object region of interest 530 based on the object region of interest 530 set by the administrator. A region for calculating a comparison difference value using only the predicted region 532 is shown.

The above description is merely illustrative of the technical idea of the present embodiment, and those skilled in the art to which this embodiment belongs may be capable of various modifications and variations without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical spirit of the present embodiment, but to explain, and the scope of the technical spirit of the present embodiment is not limited by these embodiments. The protection scope of the present embodiment should be interpreted by the claims below, and all technical spirits within the equivalent range should be interpreted as being included in the scope of the present embodiment.

As described above, the present embodiment is a useful invention that is applied to the field of event monitoring, which can improve the performance of moving object detection for event monitoring, and reduce the amount of computation for moving object detection.

110: PTZ camera 120: event monitoring device
130: image acquisition unit 140: scene change confirmation unit
150: background model control unit 160: existing background model processing unit
170: new background model processing unit 180: event detection unit

Claims (8)

An image acquiring unit acquiring a photographed image from a camera photographing while moving in a preset photographing direction at preset time intervals;
A scene change confirmation unit comparing the current image frame and the previous image frame included in the captured image to detect movement of the camera;
When movement of the camera is detected, the current image frame is compared with a plurality of background models previously generated, a similar background model similar to the current image frame is extracted from the plurality of background models previously generated, and the plurality of groups are previously generated. If there is no background model similar to the current image frame among the background models of the background model control unit for generating a new background model for the shooting direction of the current image frame; And
When the similar background model exists, an event detection unit that detects an event using the similar background model and detects an event using the new background model if the similar background model does not exist.
Event detection device comprising a. ◈ Claim 2 was abandoned when payment of the registration fee was set.◈ According to claim 1,
The scene change confirmation unit,
A pixel difference value is calculated by comparing the brightness or color of the pixels of the current image frame and the previous image frame, and when the calculated pixel difference value is equal to or greater than a preset threshold, a shooting position of the captured image according to movement of the camera Event detection device characterized in that it detects that the change. ◈ Claim 3 was abandoned when payment of the set registration fee was made.◈ According to claim 2,
The scene change confirmation unit,
Based on the pixel difference value obtained by sampling and comparing the brightness or color of the pixels located on at least one of a vertical line, a diagonal line, and a horizontal line for each of the current image frame and the previous image frame, the movement of the camera Event detection device, characterized in that for detecting the scene change of the captured image. ◈ Claim 4 was abandoned when payment of the set registration fee was made.◈ According to claim 1,
The background model control unit,
A reference background model processing unit for generating an updated background model by updating the current image frame to the existing background model when an existing background model exists for a photographing direction of the current image frame; And
If the existing background model for the shooting direction of the current image frame does not exist, a new background model processing unit generating a new background model for the current image frame
Event detection device comprising a. ◈ Claim 5 was abandoned when payment of the set registration fee was made.◈ The method of claim 4,
The reference background model processing unit,
A background model having a minimum value among a plurality of comparison difference values calculated by comparing each of the plurality of previously stored background models and the current image frame is selected, and when the comparison difference value for the selected background model is less than or equal to a preset threshold, the selected An event detection device characterized in that the background model is extracted as the existing background model. ◈ Claim 6 was abandoned when payment of the registration fee was set.◈ According to claim 1,
The event detection unit,
In the current image frame, an area other than the background model is set as a region of interest and an event is detected using at least one object detection method among edge detection, pre-stored reference image comparison, and window slide within the region of interest. Event detection device characterized in that it characterized in that. An image acquisition process of acquiring a photographed image from a camera photographing while moving in a preset photographing direction at preset time intervals;
A scene change confirmation process of comparing the current image frame and the previous image frame included in the captured image to detect movement of the camera;
When movement of the camera is detected, the current image frame is compared with a plurality of background models previously generated, a similar background model similar to the current image frame is extracted from the plurality of background models previously generated, and the plurality of groups are previously generated. A background model control process of generating a new background model for a photographing direction of the current image frame if a background model similar to the current image frame does not exist among the background models of the; And
An event detection process of detecting an event using the similar background model when the similar background model exists, and detecting an event using the new background model if the similar background model does not exist.
Event detection method comprising a. The method of claim 7,
The background model control process,
A reference background model processing process of generating an updated background model by updating the current image frame in the existing background model when the existing background model exists for the preset image including the current image frame; And
When the existing background model for the preset image including the current image frame does not exist, a new background model processing process for generating a new background model for the current image frame
Event detection method comprising a.

Images