CN111179305B - Object position estimation method and object position estimation device thereof - Google Patents

Abstract

The invention discloses an object position estimation method with a time stamp alignment function, which is applied to an object position estimation device. The object position estimation method includes obtaining a plurality of first frames from a first camera, setting a first preset time point, and extracting from the plurality of first frames a first previous frame and a third frame closest to the first preset time point. A later picture, obtaining the first previous coordinate value of the object in the first previous picture and the first later coordinate value of the object in the first later picture, and using the first previous coordinate value and the first later coordinate value After the coordinate value is calculated, the first estimated coordinate value of the object at the first preset time point is calculated. The invention can overcome the time stamp gap of metadata between different cameras and effectively improve the accuracy of object tracking.

Description

Object position estimation method and object position estimation device

Technical field

The present invention relates to an object position estimating method and an object position estimating device, and in particular to an object position estimating method with a timestamp alignment function and an object position estimating device thereof.

Background technique

With the advancement of technology, multi-camera surveillance systems are widely used in a wide range of areas or environments. The field of view of one camera is not enough to cover the entire monitoring area, so the traditional multi-camera monitoring system points each camera to different areas. When tracking objects, the movement trajectory of the object in the monitoring area can be judged based on the monitoring images of different cameras. . In order to obtain the precise movement trajectory of objects, traditional multi-camera surveillance systems will perform time synchronization settings for all cameras to ensure that each camera can obtain surveillance images at the required time point. However, the time point at which each camera acquires surveillance images and the acquisition frequency are affected by hardware configuration or network environment, resulting in differences in metadata timestamps between different cameras. The larger the difference in timestamps, the less accurate the object tracking will be. Therefore, how to overcome this shortcoming and design an object tracking method that can accurately align the object metadata of multiple cameras is a key development goal of the surveillance industry.

Contents of the invention

The invention relates to an object position estimation method with a time stamp alignment function and an object position estimation device thereof.

The invention further discloses an object position estimation method with a timestamp alignment function, which includes obtaining a plurality of first pictures of the first camera, setting a first preset time point, and extracting the closest to the first picture from the plurality of first pictures. The first previous picture and the first later picture at the first preset time point obtain the first previous coordinate value of the object in the first previous picture and the first later coordinate value of the object in the first later picture, and using the first previous coordinate value and the first later coordinate value to calculate a first estimated coordinate value of the object at the first preset time point.

The invention also discloses an object position estimating device with a time stamp alignment function, which includes a receiver and a processor. The receiver captures the footage produced by the camera. The processor is electrically connected to the receiver and is used to obtain a plurality of first frames from the first camera, set a first preset time point, and retrieve a third frame from the plurality of first frames that is closest to the first preset time point. A previous picture and a first later picture, obtaining the first previous coordinate value of the object in the first previous picture and the first later coordinate value of the object in the first later picture, and using the first previous coordinate value and the first later coordinate value to calculate the first estimated coordinate value of the object at the first preset time point.

The fields of view of multiple cameras of the present invention may partially overlap or not overlap at all, and time synchronization settings are completed in advance. The object position estimation method and the object position estimation device of the present invention set several preset time points according to a specific frequency, and after obtaining each object metadata of each camera before and after the preset time point, use interpolation or other methods to The operation function obtains the metadata of the object at this preset time point and estimates the metadata of the object after time alignment. The invention can overcome the time stamp gap of metadata between different cameras and effectively improve the accuracy of object tracking.

Description of the drawings

FIG. 1 is a functional block diagram of an object position estimation device according to an embodiment of the present invention.

Figure 2 is a flow chart of an object position estimation method according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of images acquired by the camera in different time sequences according to the embodiment of the present invention.

Figure 4 is a schematic diagram of an object position estimation device and a camera according to an embodiment of the present invention.

Figures 5 and 6 show the conversion of previous coordinate values and later coordinate values into estimated coordinate values according to the embodiment of the present invention.

Schematic diagram.

Figure 7 is a schematic diagram of a spliced screen according to an embodiment of the present invention.

Among them, the reference symbols are explained as follows:

10 Object position estimation device

12 receivers

14 processors

16 first camera

18 Second camera

Is1 first picture

Is2 second screen

IP1 first previous screen

In1 first later screen

IP2 second previous screen

In2 second later screen

Ip3 third previous screen

In3 third later screen

Ip4 fourth previous screen

In4 fourth later screen

Iu unused screen

I’, I” splicing screen

O, O’ objects

Cp1 first previous coordinate value

Cn1 first later coordinate value

Ce1 first estimated coordinate value

Cp2 second previous coordinate value

Cn2 second later coordinate value

Ce2 second estimated coordinate value

T1 first preset time point

T2 second preset time point

Steps S200, S202, S204, S206, S208, S210, S212, S214

Detailed ways

Please refer to FIG. 1 , which is a functional block diagram of an object position estimation device 10 according to an embodiment of the present invention. When multiple cameras capture images of the same surveillance area from different angles, the operating systems of these cameras have completed time synchronization settings, but there will still be a slight time difference in each frame captured by different cameras. The object position estimating device 10 has a timestamp alignment function, which is used to align object metadata in the frames obtained by multiple cameras to improve object tracking accuracy. The object position estimating device 10 can be a server that collects picture data from multiple cameras for time stamp alignment; the object position estimating device 10 can also be a camera with special functions that can receive monitoring pictures from other cameras and compare them with its own monitoring pictures. Timestamp alignment.

Please refer to Figures 1 to 4. Figure 2 is a flow chart of an object position estimation method according to an embodiment of the present invention. Figure 3 is a schematic diagram of images acquired by a camera in different time sequences according to an embodiment of the present invention. Figure 4 is a schematic diagram of an object position estimation method according to an embodiment of the present invention. A schematic diagram of the object position estimation device 10 and the camera. The object position estimation device 10 may include a receiver 12 and a processor 14 electrically connected together. The receiver 12 acquires images produced by other cameras. The object position estimation method executed by the processor 14 is applicable to the object position estimation device 10 shown in FIG. 1 . Although the first camera 16 and the second camera 18 have completed the time synchronization setting, due to the different variability of the camera system performance and network connection quality, the number of captured frames (Frames per second, FPS) of each camera will change over time. ,As shown in Figure 3.

Regarding the object position estimation method, step S200 is first performed. The processor 14 can determine the frequency of timestamp alignment. For example, when the number of frames captured by the first camera 16 and the second camera 18 is 60 fps, the time stamp alignment of the object position estimation method is The frequency can be 60fps or 30fps, or other values. Next, steps S202 and S204 are executed. The processor 14 obtains a plurality of first images Is1 and a plurality of second images Is2 of the first camera 16 and the second camera 18 through the receiver 12, and sets the first alignment frequency according to the timestamp. Default time point T1. Then step S206 is executed to find out the first previous picture Ip1 and the first later picture In1 from the first picture Is1 that are closest to the first preset time point T1, and to find out the second picture Is2 that is closest to the first preset time point. The second previous picture Ip2 and the second later picture In2 at the time point T1.

Next, step S208 is performed, using object tracking technology to analyze the objects O and O' in the previous pictures Ip1, Ip2 and later pictures In1, Is2, and obtain the first previous coordinate value Cp1 of the object O in the first previous picture Ip1 and the first previous coordinate value Cp1 in the first previous picture Ip1 and the first previous coordinate value Cp1 in the first previous picture Ip1. A first later coordinate value Cn1 in a later picture In1, a second previous coordinate value Cp2 of the object O' in the second previous picture Ip2, and a second later coordinate value Cn2 in the second later picture In2. Later, step S210 is executed, using the first previous coordinate value Cp1 and the first later coordinate value Cn1 to calculate the first estimated coordinate value Ce1 of the object O at the first preset time point T1, and using the second previous coordinate value Cp2 The second estimated coordinate value Ce2 of the object O' at the first preset time point T1 is calculated with the second later coordinate value Cn2.

After obtaining the first estimated coordinate value Ce1 and the second estimated coordinate value Ce2, steps S212 and S214 are executed to select one of the plurality of first pictures Is1 and the plurality of second pictures Is2 for splicing to generate the spliced picture I', The first estimated coordinate value Ce1 of the object O and the second estimated coordinate value Ce2 of the object O' are displayed on the spliced screen I'. In this way, the object position estimating device 10 can form the trajectories of the objects O and O' tracked by the two cameras 16 and 18 respectively on the spliced image I', which is convenient for the user to observe or for other calculation applications. In this embodiment, the first previous picture Ip1 and the second later picture In2 are preferably used for splicing, but the actual application is not limited to this.

In particular, the first estimated coordinate value Ce1 belongs to the predicted quantity of object O at the first preset time point T1, and the second estimated coordinate value Ce2 belongs to the predicted quantity of object O' at the first preset time point T1. Therefore, The first estimated coordinate value Ce1 and the second estimated coordinate value Ce2 are preferably regarded as the same in time sequence; regardless of whether there is a slight error in the actual situation, the object position estimation method of the present invention can reduce the error between the two to a minimum value. , so it can still be defined as the same.

Please refer to FIG. 3 to FIG. 7 . FIG. 5 and FIG. 6 are schematic diagrams of converting previous coordinate values and later coordinate values into estimated coordinate values according to an embodiment of the present invention. FIG. 7 is a schematic diagram of a spliced screen according to an embodiment of the present invention. The object position estimation method of the present invention can use the interpolation method to obtain the first estimated coordinate value Ce1(x3, y3). Among them, the first previous picture Ip1 is obtained at the time point Tp1, the first later picture In1 is further obtained at the time point Tn1, the coordinate value x3=x1+(x2-x1)*(T1-Tp1)/(Tn1-Tp1), and Coordinate value y3=y1+(y2-y1)*(T1-Tp1)/(Tn1-Tp1). The calculation method for converting the second previous coordinate value Cp2 and the second later coordinate value Cn2 into the second estimated coordinate value Ce2 is the same as the first estimated coordinate value Ce1, so the description will not be repeated. In addition, the calculation method of the estimated coordinate value is not limited to the interpolation method, and can also be obtained through other calculation functions.

The object position estimation method can further set one or more preset time points later than the first preset time point T1, and obtain the estimated coordinate values of the objects O and O' at this preset time point. Taking the second preset time point T2 as an example, the second preset time point T2 can be determined according to the timestamp alignment frequency included in step S200. The first picture Is1 has a third previous picture Ip3 and a third later picture In3 that are closest to the second preset time point T2 before and after the second preset time point T2. The position of the object O in the third previous picture Ip3 is the third previous coordinate value (not shown in the drawing), and the position of the object O in the third later picture In3 is the third later coordinate value (not shown in the drawing). ). The third previous coordinate value and the third later coordinate value may use interpolation or other operation functions to generate a third estimated coordinate value (not shown in the drawing). The previous coordinate value and the later coordinate value of the fourth previous picture Ip4 and the fourth later picture In4 in the second picture Is2 that are closest to the second preset time point T2 of the object O' can also be converted to generate another estimated coordinate value. The third previous picture Ip3 can be spliced with the fourth later picture In4 to generate a spliced picture I", and the third estimated coordinate value of the object O and another estimated coordinate value of the object O' are displayed in the spliced picture I".

If there is one or more unused pictures Iu between the first later picture In1 and the third previous picture Ip3, the object position estimation method of the present invention can directly discard the data of the object O in the unused picture Iu, which means that it is not used. The coordinate value of the object O in the screen Iu will not be used in calculating the third estimated coordinate value. The unused frame Iu is defined as any frame that is not used for object position estimation between two preset time points T1 and T2. In addition, the unused frame Iu may be able to cooperate with the third previous frame Ip3 and the third later frame In3 to calculate the estimated coordinate value of the object O at the second preset time point T2 using a specific operation function; for example, the unused frame Iu can be used to calculate the estimated coordinate value of the object O at the second preset time point T2. Low weight, the third previous picture Ip3 and the third later picture In3 are given higher weight, and all three frames are used to calculate the estimated coordinate values. Alternatively, the object position estimation method can analyze and compare the unused frame Iu and the third previous frame Ip3, and select one of the two to estimate the object position in conjunction with the third later frame In3.

In the embodiment of the present invention, if the object position estimation device 10 is a server electrically connected to multiple cameras, it can set its own timestamp alignment frequency and stably calculate its estimated coordinate values at each preset time point. If the object position estimation device 10 is a camera that receives image data from other cameras, the image capture and object position estimation can be performed in an interleaved manner, for example, the estimated coordinate value of the previous preset time point is first calculated, and then the next predetermined value is calculated after completion. Set the estimated coordinate value of the time point, so the interval of the preset time points may be slightly different. The settings of the timestamp alignment frequency and the preset time point are determined according to the hardware of the object position estimation device 10 and its computing performance, and may vary in many ways, which will not be described separately here.

To sum up, the fields of view of multiple cameras can partially overlap or not overlap at all, and time synchronization settings are completed in advance. The object position estimation method and the object position estimation device of the present invention set several preset time points according to a specific frequency, and after obtaining each object metadata of each camera before and after the preset time point, use interpolation or other methods to The operation function obtains the metadata of the object at this preset time point and estimates the metadata of the object after time alignment. Compared with the existing technology, the present invention can overcome the time stamp gap of metadata between different cameras and effectively improve the accuracy of object tracking.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (6)

1. An object position estimation method, characterized in that the object position estimation method includes: Set the time synchronization between the first camera and the second camera; Obtain the pictures captured by the first camera as multiple first pictures; Obtain the pictures captured by the second camera as multiple second pictures; Set the first preset time point according to the timestamp alignment frequency; Extract the first previous picture and the first later picture closest to the first preset time point from the plurality of first pictures; Obtain the first previous coordinate value of the object O in the first previous picture and the first later coordinate value of the object O in the first later picture; Using the first previous coordinate value and the first later coordinate value, calculate the first estimated coordinate value of the object O at the first preset time point; Extract the second previous picture and the second later picture closest to the first preset time point from the plurality of second pictures; Obtain the second previous coordinate value of the object O' in the second previous picture and the second later coordinate value of the object O' in the second later picture; Using the second previous coordinate value and the second later coordinate value, calculate the second estimated coordinate value of the object O' at the first preset time point; Generate a spliced picture based on the first previous picture and the second later picture; and The first estimated coordinate value of the object O and the second estimated coordinate value of the object O' are displayed on the spliced screen. 2. The object position estimation method according to claim 1, wherein the object position estimation method uses interpolation to calculate the first estimate between the first previous coordinate value and the first later coordinate value. coordinate value. 3. The object position estimation method according to claim 1, characterized in that the object position estimation method further includes: Set a second preset time point later than the first preset time point; Extract the third previous picture and the third later picture closest to the second preset time point from the plurality of first pictures; Obtain the third previous coordinate value of the object O in the third previous picture and the third later coordinate value of the object O in the third later picture; and Using the third previous coordinate value and the third later coordinate value, a third estimated coordinate value of the object O at the second preset time point is calculated. 4. The object position estimation method as claimed in claim 3, characterized in that the object position estimation method further includes: Determine whether there is an unused frame between the first later frame and the third previous frame; and If there is, discard the unused screen of object O. 5. The object position estimation method as claimed in claim 3, wherein the second preset time point is determined based on the timestamp alignment frequency. 6. An object position estimating device, characterized in that the object position estimating device includes: a receiver to obtain the images produced by the camera; and A processor, electrically connected to the receiver, used to execute the object position estimation method according to any one of claims 1 to 5.