CN118628533B - Target tracking method and computer equipment - Google Patents

Abstract

The present application relates to a target tracking method and computer equipment. The method comprises: matching the targets in the common view area in the bird's-eye view of the target road section based on the frame-synchronized output information of each sensor to obtain the target matching result; updating the sensor identification and target identification corresponding to each fused target in the common view area based on the target matching result, and updating the position information of each fused target in the common view area in the bird's-eye view of the target road section; obtaining the continuous display result of each fused target in the target road section based on the frame-synchronized output information of each sensor, the updated sensor identification and updated target identification corresponding to each fused target in the common view area, and the updated position information of each fused target in the common view area in the bird's-eye view of the target road section. The use of this method can solve the problem of continuous tracking difficulty existing in the existing vehicle tracking methods.

Description

A target tracking method and computer device

Technical Field

The present application relates to the field of intelligent transportation technology, and in particular to a target tracking method and computer equipment.

Background Art

With the development of electronic technology, intelligent transportation technology has gradually attracted people's attention. In intelligent traffic management systems such as autonomous driving, it is crucial to efficiently and accurately track the trajectory of the target vehicle in a preset section or tunnel and generate the vehicle driving trajectory of the entire section.

The existing vehicle tracking method mainly adopts the cross-camera multi-target cascade matching method with matching priority. By acquiring the image information of each camera, the motion state of the tracking target is estimated, and the appearance features of all tracking targets are collected. Through the appearance features of all tracking targets, the matching association between cross-camera multi-targets is performed to track the vehicle on the entire road section. However, this method only relies on the appearance features of the vehicle for cross-camera association, which may lead to mismatching, missing matching or failure to match when the camera is deployed at a long distance, the vehicle is blocked, and the license plate is recognized, making continuous tracking difficult.

There is currently no effective solution to the problem of continuous tracking difficulties in existing vehicle tracking methods due to problems such as long camera deployment distance, vehicle occlusion, and license plate recognition.

Summary of the invention

Based on this, it is necessary to provide a target tracking method and computer device to address the above technical issues.

In a first aspect, the present application provides a target tracking method. The method comprises the following steps:

Based on the output information of each sensor after frame synchronization, matching the target in the common view area in the bird's-eye view of the target road section to obtain a target matching result; the common view area is an overlapping area between different view areas of adjacent sensors set in the target road section in the bird's-eye view of the target road section;

Based on the target matching result, the sensor identifier and target identifier corresponding to each fused target in the common view area are updated, and the position information of each fused target in the common view area in the bird's-eye view of the target road section is updated; the target identifier is the identifier of each target in the output information after frame synchronization of each sensor;

Based on the output information of each of the sensors after frame synchronization, the updated sensor identifiers and updated target identifiers corresponding to each of the fused targets in the common viewing area, and the updated position information of each of the fused targets in the common viewing area in the bird's-eye view of the target road section, a continuous display result of each of the fused targets in the target road section is obtained.

In one embodiment, before matching the target within the common view area in the bird's-eye view image of the target road section based on the output information of each sensor after frame synchronization and obtaining the target matching result, the following steps are included:

Performing frame synchronization on the output information of all sensors located on the target road section to obtain the frame-synchronized output information of each sensor; the output information of the sensor includes each target in the field of view captured by the sensor and the attribute information of each target;

Based on the output information of each sensor after frame synchronization and the field of view of each sensor in the bird's-eye view of the target road section, all fusion targets in the target road section are determined to obtain a fusion target set;

Based on the fusion target set, a mapping relationship between each of the fusion targets, the sensor identifier and the target identifier, and position information of each of the fusion targets in the bird's-eye view of the target road section are determined.

In one embodiment, before all fused targets in the target road section are determined based on the output information of each sensor after frame synchronization and the field of view of each sensor in the bird's-eye view of the target road section to obtain the fused target set, the following steps are included:

Based on the position information and calibration information of each sensor located on the target road section, the field of view area of each sensor in the bird's-eye view of the target road section is determined.

In one of the embodiments, before determining the field of view area of each sensor in the bird's-eye view of the target road section based on the position information and calibration information of each sensor located on the target road section, the following steps are included:

Based on the sensor type and the road environment of the target road section, position information of each of the sensors located on the target road section is determined.

In one embodiment, determining all fused targets in the target road section based on the frame-synchronized output information of each sensor and the field of view of each sensor in the bird's-eye view of the target road section to obtain a fused target set includes the following steps:

Based on the frame-synchronized output information of each of the sensors, each target in the field of view of the first sensor of the target road section in the bird's-eye view of the target road section is taken as the first fusion target; the first sensor of the target road section is the first sensor of the target entering the target road section;

The target that first appears in the field of view of the bird's-eye view of the target road section by sensors other than the first sensor in the target road section, excluding the common view area, is used as the second fusion target;

The fusion target set is generated based on the first fusion target and the second fusion target.

In one embodiment, matching targets within a common viewing area in a bird's-eye view of the target road section based on the frame-synchronized output information of each sensor to obtain a target matching result includes:

Based on the output information of each sensor after frame synchronization, from a bird's-eye view perspective, perform intersection-and-union calculation on each target in the common viewing area of the bird's-eye view of the target road section by the adjacent sensors to obtain a first intersection-and-union calculation result; based on the first intersection-and-union calculation result, perform matching on the targets in the common viewing area of the adjacent sensors of the target road section to obtain the target matching result;

Or, based on the output information of each of the sensors after frame synchronization, in the bird's-eye view perspective, determine the first point set features between each target in the common view area determined by the adjacent sensors and each target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view of the target road section; determine the second point set features between each target in the common view area determined by the adjacent sensors and each target in the field of view of the second sensor among the adjacent sensors in the bird's-eye view of the target road section; based on the first point set features and the second point set features, perform similarity matching on the targets in the common view area in the bird's-eye view of the target road section to obtain a similarity matching result; based on the similarity matching result, obtain the target matching result;

Or, based on the output information after frame synchronization of each of the sensors, each target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view is projected onto the field of view of the second sensor among the adjacent sensors in the bird's-eye view to obtain each projected target; an intersection-and-union ratio calculation is performed on each projected target and each target in the field of view of the second sensor among the adjacent sensors in the bird's-eye view to obtain a second intersection-and-union ratio calculation result; based on the second intersection-and-union ratio calculation result, the targets in the common viewing area of the adjacent sensors of the target section are matched to obtain the target matching result.

In one of the embodiments, based on the target matching result, updating the sensor identifier and the target identifier corresponding to each of the fused targets in the common view area, and updating the position information of each of the fused targets in the common view area in the bird's-eye view of the target road section include:

When a target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view exists in the fusion target list, based on the target matching result, the sensor identifier and the target identifier corresponding to each of the fusion targets in the common view area are updated, and the position information of each of the fusion targets in the common view area in the bird's-eye view is updated.

In one of the embodiments, when a target in the field of view of a first sensor among the adjacent sensors in the bird's-eye view exists in the fusion target list, based on the target matching result, updating the sensor identifier and the target identifier corresponding to each of the fusion targets in the common view area, and updating the position information of each of the fusion targets in the common view area in the bird's-eye view, includes:

When a first target in a field of view of a first sensor among the adjacent sensors in the bird's-eye view exists in the fusion target list, determining, based on the target matching result, a second target in a field of view of a second sensor among the adjacent sensors in the bird's-eye view that matches the first target in the field of view of the first sensor among the adjacent sensors;

The sensor identifier corresponding to the fused target in the common viewing area is updated to the identifier corresponding to the second sensor in the adjacent sensors, the target identifier corresponding to the fused target in the common viewing area is updated to the identifier of the second target in the field of view of the second sensor in the adjacent sensors in the bird's-eye view, and the position information of the fused target in the common viewing area in the bird's-eye view is updated to the position information of the second target in the field of view of the second sensor in the adjacent sensors in the bird's-eye view.

In one of the embodiments, after updating the sensor identifier and the target identifier corresponding to each of the fused targets in the common view area based on the target matching result, and updating the position information of each of the fused targets in the common view area in the bird's eye view, the method further includes:

When a fusion target is lost, traverse the frame-synchronized output information of the third sensor corresponding to the field of view where the lost fusion target is located, and determine whether there is a third target having attribute information consistent with the lost fusion target in the frame-synchronized output information of the third sensor;

When the third target having attribute information consistent with the lost fusion target exists in the output information of the third sensor after frame synchronization, the sensor identifier corresponding to the fusion target is updated to the identifier corresponding to the third sensor, the target identifier corresponding to the fusion target is updated to the identifier of the third target in the field of view of the third sensor in the bird's-eye view, and the position information of the fusion target in the bird's-eye view is updated to the position information of the third target in the field of view of the third sensor in the bird's-eye view;

When the third target having the same attribute information as the lost fusion target does not exist in the output information of the third sensor after frame synchronization, traverse the output information of the fourth sensor after frame synchronization to determine whether the fourth target having the same attribute as the lost fusion target exists in the output information of the fourth sensor after frame synchronization; the fourth sensor is a sensor corresponding to the next field of view area that the target enters after leaving the field of view area of the third sensor;

When the fourth target having the same attribute as the lost fusion target exists in the output information of the fourth sensor after frame synchronization, the sensor identifier corresponding to the fusion target is updated to the identifier corresponding to the fourth sensor, the target identifier corresponding to the fusion target is updated to the identifier of the fourth target in the field of view of the fourth sensor in the bird's-eye view, and the position information of the fusion target in the bird's-eye view is updated to the position information of the fourth target in the field of view of the fourth sensor in the bird's-eye view;

When the fourth target whose attribute information is consistent with that of the lost fusion target does not exist in the output information of the fourth sensor after frame synchronization, multiple targets corresponding to the multiple adjacent fusion targets in the output information of the fourth sensor after frame synchronization are determined according to the positional relationship between the fusion target and the multiple adjacent fusion targets when the fusion target is not lost; based on the multiple targets corresponding to the multiple adjacent fusion targets in the output information of the fourth sensor after frame synchronization and the positional relationship between the fusion target and the multiple adjacent fusion targets when the fusion target is not lost, the first position information of the lost fusion target in the field of view of the fourth sensor in the bird's-eye view is estimated; and the position information of the fusion target in the bird's-eye view is updated to the first position information in the field of view of the fourth sensor in the bird's-eye view.

In a second aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the target tracking method described in the first aspect is implemented.

The target tracking method and computer device match the targets in the common viewing area in the bird's-eye view of the target road section based on the output information of each sensor after frame synchronization to obtain the target matching result, and based on the target matching result, update the sensor identification and target identification corresponding to each fused target in the common viewing area, and update the position information of each fused target in the common viewing area in the bird's-eye view of the target road section. It matches the targets in the common viewing area in the bird's-eye view of the target road section to establish the association between the different perspectives of the front and rear sensors, and according to the target matching result, update the sensor identification and target identification corresponding to the fused target in the common viewing area, and update the common viewing area. The position information of each fused target in the bird's-eye view of the target road section is used to associate and fuse the same fused target in different field of view of each sensor, and then, based on the output information of each sensor after frame synchronization, the updated sensor identifier and the updated target identifier corresponding to each fused target in the common view area, and the updated position information of each fused target in the common view area in the bird's-eye view of the target road section, the continuous display of the same target in the same bird's-eye view is realized, and the continuous tracking of the target is realized, which solves the problem of continuous tracking difficulty caused by the long camera deployment distance, vehicle occlusion and license plate recognition in the existing vehicle tracking method.

Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

FIG1 is a hardware structure block diagram of a terminal of a target tracking method provided by an embodiment of the present application;

FIG2 is a flow chart of a target tracking method provided by an embodiment of the present application;

FIG3 is a schematic diagram of various cameras of a target road section provided by an embodiment of the present application;

FIG4 is a schematic diagram of the field of view of a camera D in a bird's-eye view of a target road section provided by an embodiment of the present application;

FIG5 is a schematic diagram of location information of a fusion target in a bird's-eye view of a target road section provided by an embodiment of the present application;

FIG6 is a schematic diagram of target tracking results provided by an embodiment of the present application;

FIG7 is a flow chart of a target tracking method provided by a preferred embodiment of the present application;

FIG8 is a structural block diagram of a target tracking device provided in an embodiment of the present application.

DETAILED DESCRIPTION

In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and/or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character "/" indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, when running on a terminal, FIG1 is a hardware structure block diagram of a terminal of the target tracking method of this embodiment. As shown in FIG1 , the terminal may include one or more (only one is shown in FIG1 ) processors 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above-mentioned terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that the structure shown in FIG1 is only for illustration and does not limit the structure of the above-mentioned terminal. For example, the terminal may also include more or fewer components than those shown in FIG1 , or have a different configuration than that shown in FIG1 .

The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target tracking method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

In this embodiment, a target tracking method is provided. FIG. 2 is a flow chart of the target tracking method of this embodiment. As shown in FIG. 2 , the process includes the following steps:

Step S210, based on the output information of each sensor after frame synchronization, the target in the common view area in the bird's-eye view of the target section is matched to obtain a target matching result; the common view area is the overlapping area between different field of view areas of adjacent sensors set in the target section in the bird's-eye view of the target section.

The above-mentioned sensors are sensors that are set in the target road section according to preset positions. The above-mentioned sensors may be one of an inductive loop sensor, an ultrasonic sensor, an infrared sensor, a radar sensor, a video sensor, and the like. For easy distinction, the sensors of the current road section may be numbered, and a sensor identifier of each sensor may be set. For example, an ID of each sensor may be set. For example, the sensor may be a camera, and each camera may be numbered with English letters according to the order in which the target enters the target road section. FIG3 is a schematic diagram of each camera of the target road section provided in an embodiment of the present application. As shown in FIG3 , a plurality of cameras are set in the target road section in order, namely, camera A, camera B, camera C, camera D, and the like, and each camera has a blind spot, wherein the small triangle formed by the dotted line passing through the point of the camera and the solid line crossing the road surface is the blind spot of the camera.

The output information after the above frame synchronization is the image frame information captured by each sensor at the same time after the image information captured by all sensors set on the target road section is frame synchronized. The above target road section can be a bridge, tunnel or other road section that needs to continuously track the target vehicle. The bird's-eye view of the above target road section can be a stereoscopic image formed by looking down at the target road section from any point above the target road section. Among them, the point above the target road section that forms the bird's-eye view can be specifically set according to the specific situation, as long as it can ensure that each target in the bird's-eye view of the target road section can be clearly displayed.

The above-mentioned matching of the targets in the common view area in the bird's-eye view of the target road section may be performed by using a preset matching algorithm to match the targets in the common view area in the bird's-eye view of the target road section. The above-mentioned preset matching algorithm may be one or more of IOU (Intersection over Union) matching based on the three-dimensional perspective of the bird's-eye view, point set matching based on the three-dimensional perspective of the bird's-eye view, target matching based on a two-dimensional perspective, or trajectory matching. The above-mentioned target matching result may be the matching result of the same target in the common view area captured by two adjacent sensors at the same time.

In order to ensure the continuity of the target object captured in the target section, it is necessary to ensure that the shooting areas of all sensors can cover the target section, that is, it is necessary to ensure that there is an overlapping area between the shooting areas of each adjacent sensor. And because the shooting area of each sensor is the field of view area of each sensor in the bird's-eye view of the target section. For example, Figure 4 is a schematic diagram of the field of view area of camera D in the bird's-eye view of the target section provided in an embodiment of the present application. Based on this, there is an overlapping area between the different fields of view of any two adjacent sensors in the bird's-eye view of the target section, and this overlapping area can be used as the common viewing area of the two adjacent sensors.

Because the same target in the common view area can be photographed by two adjacent sensors at the same time. And because the positions of each sensor are different, the viewing angles of the same target are different. Therefore, the display information of the same target photographed by two adjacent sensors at the same time is not exactly the same. It is necessary to match the targets in the common view area in the bird's-eye view of the target section, and match the same target photographed by different sensors at the same time, that is, to associate and match the same target photographed from different viewing angles into the same fusion target. The above fusion target is a unified representation of the same target photographed by different sensors in the current section. Specifically, a fusion target identifier can be assigned to each fusion target, and each fusion target can be associated with the target corresponding to the fusion target in each sensor, so that the same target seen from different viewing angles in the current section corresponds to the same fusion target identifier, so as to achieve the complete, continuous and unique trajectory of each target in the current section.

Step S220, based on the target matching result, update the sensor identifier and target identifier corresponding to each fused target in the common view area, and update the position information of each fused target in the common view area in the bird's-eye view of the target road section; the target identifier is the identifier of the target in the output information after frame synchronization in each sensor.

In order to facilitate the distinction, it is necessary to set a target identifier for each target in the image frame captured by each sensor. For the same target, the target identifiers in the image frame information captured by different sensors are different. The same targets of adjacent sensors need to be associated. Therefore, based on the target matching result, the sensor identifier and target identifier corresponding to each fused target in the common viewing area are updated, and the position information of each fused target in the common viewing area in the bird's-eye view of the target section is updated. It can be based on the first target corresponding to the fused target in the image frame of the sensor corresponding to the previous perspective, and the target matching result, determine the second target matching with the first target in the image frame of the sensor corresponding to the latter perspective, then the second target in the image frame of the sensor corresponding to the latter perspective also corresponds to the fused target. Therefore, the sensor identifier corresponding to each fused target in the common viewing area can be updated to the sensor identifier of the sensor corresponding to the latter perspective, and the target identifier corresponding to each fused target in the common viewing area can be updated to the second target identifier. The position information of the fused target in the common viewing area in the bird's-eye view of the target section is updated according to the position information of the second target in the bird's-eye view of the target section. It realizes the perspective switching of the fused target by associating the same target photographed from different perspectives, and ensures that the trajectory of the target is continuous and unique during the entire process of the target passing through the target section.

Step S230, based on the output information of each sensor after frame synchronization, the updated sensor identifier and the updated target identifier corresponding to each fusion target in the common viewing area, and the updated position information of each fusion target in the common viewing area in the bird's-eye view of the target section, obtain the continuous display result of each fusion target in the target section.

The above-mentioned continuous display result of each fused target in the target section is obtained based on the output information after frame synchronization of each sensor, the updated sensor identifier and the updated target identifier corresponding to each fused target in the common view area, and the updated position information of each fused target in the common view area in the bird's-eye view of the target section. It can be based on the output information after frame synchronization of each sensor to determine the sensor identifier and the target identifier corresponding to each fused target outside the common view area, and then, based on the sensor identifier and the target identifier corresponding to each fused target outside the common view area, the position information of each fused target outside the common view area in the bird's-eye view of the target section, the updated sensor identifier and the updated target identifier corresponding to each fused target in the common view area, and the updated position information of each fused target in the common view area in the bird's-eye view of the target section, to obtain the continuous display result of each fused target in the target section.

In the above steps S210 to S230, the targets in the common viewing area in the bird's-eye view of the target road section are matched based on the output information of each sensor after frame synchronization to obtain a target matching result, and based on the target matching result, the sensor identifier and target identifier corresponding to each fused target in the common viewing area are updated, and the position information of each fused target in the common viewing area in the bird's-eye view of the target road section is updated. By matching the targets in the common viewing area in the bird's-eye view of the target road section, an association between different perspectives of the front and rear sensors is established, and according to the target matching result, the sensor identifier and target identifier corresponding to the fused target in the common viewing area are updated, and the common viewing area is updated. The position information of each fused target in the domain in the bird's-eye view of the target road section is used to associate and fuse the same fused target in different field of view of each sensor, and then, according to the output information of each sensor after frame synchronization, the updated sensor identifier and the updated target identifier corresponding to each fused target in the common view area, and the updated position information of each fused target in the common view area in the bird's-eye view of the target road section, the continuous display of the same target in the same bird's-eye view is realized, and the continuous tracking of the target is realized, which solves the problem of continuous tracking difficulty caused by the long camera deployment distance, vehicle occlusion and license plate recognition in the existing vehicle tracking method.

In one embodiment, before step S210, the following steps are included:

Step S202, frame synchronization is performed on the output information of all sensors located on the target road section to obtain the frame-synchronized output information of each sensor; the output information of the sensor includes each target in the field of view captured by the sensor and the attribute information of each target.

In this step, the frame synchronization of the output information of all sensors located on the target road section may be performed by using one or more methods such as PTP (Precision Time Protocol), NTP (Network Time Protocol), etc. to perform frame synchronization on the output information of all sensors located on the target road section. It should be noted that the specific frame synchronization method is not specifically limited in this embodiment, as long as the frame synchronization method can be used to ensure that the output information of all sensors located on the target road section is an image frame captured at the same time. The above-mentioned targets may include target identifications of the above-mentioned targets, and the attributes of the above-mentioned targets may include information such as the coordinates of the targets in the bird's-eye view of the target road section, the color of the vehicle, the license plate, the type of vehicle, the two-dimensional frame of the vehicle detection, and the three-dimensional frame of the vehicle detection. Among them, determining the attributes of each target may be to estimate the three-dimensional form of the vehicle based on stereo vision technology or using a deep learning method, and determine the attributes of each target based on the three-dimensional form of the vehicle. Among them, the above-mentioned three-dimensional form of the vehicle estimated by using stereo vision technology may be to use multiple cameras to take vehicle pictures at different angles, and obtain the three-dimensional information of the vehicle through the vehicle pictures taken at different angles.

Step S204, based on the frame-synchronized output information of each sensor and the field of view of each sensor in the bird's-eye view of the target road section, all fused targets in the target road section are determined to obtain a fused target set.

By assigning fusion target identifiers to each target in the field of view of each sensor in the bird's-eye view of the target section, it can be ensured that the fusion target corresponding to the same target in the field of view of the bird's-eye view of the target section is unique. The above-mentioned determination of all fusion targets in the target section based on the output information of each sensor after frame synchronization and the field of view of each sensor in the bird's-eye view of the target section can be based on the output information of each sensor after frame synchronization, taking all targets in the field of view of the first sensor in the bird's-eye view of the target section as fusion targets, and then taking the first appearing targets in the field of view of the other sensors in the bird's-eye view of the target section except the first sensor except the common view area as fusion targets.

Step S206, based on the fusion target set, determining the mapping relationship between each fusion target and the sensor identifier and the target identifier, as well as the position information of each fusion target in the bird's-eye view of the target road section.

The following example shows the mapping relationship between the fusion target, the sensor identifier and the target identifier. For example, the sensor identifier is view_id, the target identifier is obj_id, and the fusion target identifier is fusion_obj_id. The mapping relationship between the fusion target, the sensor identifier and the target identifier can be expressed in the form of {fusion_obj_id, {view_id, obj_id}}.

The position information of each fusion target in the bird's-eye view of the target road section may include the lane line information corresponding to each fusion target in the bird's-eye view, and the positional relationship between each fusion target and the adjacent fusion target in the bird's-eye view. For example, FIG5 is a schematic diagram of the position information of the fusion target in the bird's-eye view of the target road section provided by an embodiment of the present application. As shown in FIG5, the fusion target 15 moves from camera A to camera B. The position information of the fusion target 15 in the bird's-eye view of the target road section may be the lane line information corresponding to the fusion target 15 in the bird's-eye view, and the identifiers of other fusion targets closest to the fusion target 15 in front, behind, left and right (e.g., the identifier of the front fusion target is 3, the identifier of the rear fusion target is 35, the identifier of the left fusion target is 10, and the identifier of the right fusion target is 25) and the distance between the fusion target 15 and other fusion targets.

In the above steps S202 to S204, the output information of all sensors located in the target section is frame synchronized to obtain the output information of each sensor after frame synchronization, thereby determining all fusion targets in the target section, and determining the mapping relationship between each fusion target and the sensor identifier and the target identifier, as well as the position information of each fusion target in the bird's-eye view of the target section. It is convenient to update the sensor identifier and the target identifier corresponding to the fusion target through the mapping relationship between each fusion target and the sensor identifier and the target identifier, as well as the target matching results of adjacent sensors in the common viewing area, so as to realize the association and switching of the same target under different viewing angle cameras. By obtaining the position information of each fusion target in the bird's-eye view of the target section, it is convenient to estimate the position information of the lost fusion target according to the position information of each fusion target in the bird's-eye view of the target section, so as to realize the purpose of continuous tracking of the fusion target according to the estimated result of the position information of the lost fusion target when the fusion target cannot be identified.

In addition, in one embodiment, before step S204, the following steps are included:

Step S203, based on the position information and calibration information of each sensor located on the target road section, determining the field of view area of each sensor in the bird's-eye view of the target road section.

The calibration information of the sensor may be the intrinsic parameters and extrinsic parameters of the sensor, for example, the intrinsic parameters and extrinsic parameters of a camera.

Furthermore, in one embodiment, before step S203, the following steps are included:

Based on the sensor type and the road environment of the target road section, the position information of each sensor located on the target road section is determined.

When deploying each sensor, in addition to considering the type of sensor, it is also necessary to consider the road environment of the target section. Specifically, it can be to conduct an environmental survey and analysis of the target section, and determine the location information of each sensor in the target section while ensuring that the field of view of each sensor can cover the range of all target sections. It should be noted that the determination of the location information of each sensor in the target section also needs to consider the field of view of each position in the target section, as well as the placement angle of each sensor. The sensors are deployed while minimizing the blind spots of the sensors as much as possible and ensuring that there is a common view area between the field of view areas of adjacent sensors.

In one embodiment, the above step S204, based on the output information of each sensor after frame synchronization and the field of view of each sensor in the bird's-eye view of the target road section, determines all fused targets in the target road section to obtain a fused target set, including the following steps:

Step S2042, based on the output information of each sensor after frame synchronization, each target in the field of view of the first sensor of the target section in the bird's-eye view of the target section is taken as the first fusion target; the first sensor of the target section is the first sensor of the target entering the target section.

In order to ensure that the tracking of the same target on the target road section is continuous and unique, it is necessary to use fusion targets to represent the display status of the same target in the field of view of different sensors. For example, when a vehicle enters the field of view of the first sensor in the bird's-eye view of the target road section, the fusion target of the vehicle is marked as fusion_obj_001. When the first sensor captures the vehicle, the target identifier of the vehicle is obj_001 in the output information of the first sensor view_001, and the target identifier of the vehicle is obj_002 in the output information of the second sensor view_002. For this vehicle, the mapping relationship between the fusion target, the sensor identifier and the target identifier can be expressed as {fusion_obj_001, {view_001, obj_001}} and {fusion_obj_001, {view_002, obj_002}}. The display status of the same target in the field of view of different sensors can be represented by the mapping relationship between the fusion target, the sensor identifier and the target identifier.

Step S2044, taking the target that first appears in the field of view of the other sensors of the target road section except the first sensor in the bird's-eye view of the target road section except the common view area as the second fusion target.

In the target section, there may be new targets entering from the middle section, and these new targets are not marked for target fusion. The new targets newly merged into the target section need to be taken as new fusion targets. Among them, the new targets entering from the middle section can be the targets that first appear in the field of view of the target section's bird's-eye view by all sensors except the first sensor, excluding the common view area.

Step S2046: Generate a fusion target set based on the first fusion target and the second fusion target.

In the above steps S2042 to S2046, by determining all fused targets within the target road section, it is convenient to subsequently represent the display status of the same target in the field of view of different sensors through the fused targets.

In one embodiment, based on step S210, based on the output information of each sensor after frame synchronization, matching the targets in the common viewing area in the bird's-eye view of the target road section to obtain the target matching result includes the following steps:

Step S212, based on the output information of each sensor after frame synchronization, from the bird's-eye view perspective, perform intersection-and-union (IOR) calculation on each target in the common viewing area of the bird's-eye view of the target road section of adjacent sensors to obtain a first IOR calculation result; based on the first IOR calculation result, match the targets in the common viewing area of the adjacent sensors of the target road section to obtain a target matching result.

The target matching result is obtained by matching the targets of the adjacent sensors of the target road section in the common viewing area based on the first intersection-and-parallel calculation result, and the target matching result can be obtained based on the first intersection-and-parallel calculation result and the preset intersection-and-parallel threshold. Specifically, when the first intersection-and-parallel calculation result is greater than the preset intersection-and-parallel threshold, it is determined that the targets of the adjacent sensors of the target road section in the common viewing area are matched, and when the first intersection-and-parallel calculation result is less than or equal to the preset intersection-and-parallel threshold, it is determined that the targets of the adjacent sensors of the target road section in the common viewing area are not matched. The preset intersection-and-parallel threshold can be specifically set according to the specific situation, as long as the preset intersection-and-parallel threshold can be used to determine whether the target matching of the adjacent sensors of the target road section in the common viewing area is successful. The target matching result can be the matching relationship, matching number and matching time between the targets of different perspectives captured by the adjacent sensors of the target road section in the common viewing area. Among them, the matching relationship can include matching and mismatching. For example, for vehicle S, the target identifier represented in the common view area captured by sensor A is obj_003, and the target identifier represented in the common view area captured by sensor B is obj_004. Then the target matching result obtained is that obj_003 in sensor A matches obj_004 in sensor B. The representation of the above target matching result can be illustrated by an example, for example, the target matching result can be represented by {src_view_id, {dst_view_id, {src_obj_id, {dst_obj_id, count}}}}. Among them, src_view_id is the original sensor identifier, dst_view_id is the sensor identifier used for matching, src_obj_id is the target identifier under the original sensor identifier, dst_obj_id is the target identifier under the sensor identifier used for matching, and count is the number of matches between the target represented by src_obj_id and the target represented by dst_obj_id.

Step S214, based on the output information of each sensor after frame synchronization, in the bird's-eye view perspective, determine the first point set features between each target in the common view area determined by the adjacent sensors and each target in the field of view of the first sensor in the bird's-eye view of the target road section; determine the second point set features between each target in the common view area determined by the adjacent sensors and each target in the field of view of the second sensor in the bird's-eye view of the target road section; based on the first point set features and the second point set features, perform similarity matching on the targets in the common view area in the bird's-eye view of the target road section to obtain a similarity matching result; based on the similarity matching result, obtain a target matching result.

The target matching result is obtained based on the similarity matching result, and the target matching result can be obtained based on the similarity matching result and a preset similarity threshold. Specifically, when the similarity matching result is greater than the preset similarity threshold, it is determined that the target of the adjacent sensors of the target section in the common viewing area is matched, and when the similarity matching result is less than or equal to the preset similarity threshold, it is determined that the target of the adjacent sensors of the target section in the common viewing area is not matched. The above-mentioned similarity threshold can be specifically set according to the specific situation, as long as it can pass the preset similarity threshold to determine whether the target matching of the adjacent sensors of the target section in the common viewing area is successful.

Step S216, based on the output information of each sensor after frame synchronization, each target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view is projected into the field of view of the second sensor among the adjacent sensors in the bird's-eye view to obtain each projected target; perform intersection-and-union (IOR) calculation on each projected target and each target in the field of view of the second sensor among the adjacent sensors in the bird's-eye view to obtain a second IOR calculation result; based on the second IOR calculation result, match the targets in the common view area of the adjacent sensors of the target section to obtain a target matching result.

The above-mentioned projecting of each target in the field of view of a first sensor among the adjacent sensors in the bird's-eye view into the field of view of a second sensor among the adjacent sensors in the bird's-eye view may be explained by way of example.

For example, the two-dimensional frame of the target on camera B is projected into the image coordinate system of camera A, wherein the coordinate points of the target in camera B become homogeneous coordinates.

The matrix transformation is as follows:

Among them, (u, v) is the coordinate of the target coordinate point in the image taken by camera A, (x, y) is the coordinate of the target coordinate point in the image taken by camera B, and H _AB represents the projection transformation matrix from the perspective of camera B to the perspective of camera A.

Through the above formula, the target coordinate point of the image taken by camera B is obtained, and the position of the target coordinate point in the image taken by camera A is projected.

This step obtains the target matching result by combining the matching under the three-dimensional perspective based on the bird's-eye view with the matching of the target based on the two-dimensional perspective, so that the target matching is more accurate.

In the above steps S212 to S216, the targets in the common viewing area in the bird's-eye view of the target road section are matched by utilizing a matching method under a three-dimensional perspective, or a matching method combining a two-dimensional perspective and a three-dimensional perspective, to obtain a target matching result, so as to facilitate associating the same target photographed from different perspectives based on the matching result, realize perspective switching of the fused target, and ensure that the trajectory of the target is continuous and unique throughout the process of the target passing through the target road section.

In addition, in one embodiment, the above step S220, based on the target matching result, updates the sensor identifier and the target identifier corresponding to each fused target in the common view area, and updates the position information of each fused target in the common view area in the bird's-eye view of the target road section, including the following steps:

Step S222, when the target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view exists in the fusion target list, based on the target matching result, the sensor identifier and target identifier corresponding to each fusion target in the common view area are updated, and the position information of each fusion target in the common view area in the bird's-eye view is updated.

After the target matching is completed, it is necessary to associate the same target photographed by adjacent sensors at different viewing angles, and then update the sensor identifiers and target identifiers corresponding to each fused target in the common viewing area according to the association results, and update the position information of each fused target in the common viewing area in the bird's-eye view. The fused target is switched from the perspective of the first sensor to the perspective of the second sensor to ensure that the trajectory of the target is continuous and unique throughout the process of the target passing through the target section. The above-mentioned first sensor is the sensor corresponding to the field of view that the target enters first among the two adjacent sensors, and the second sensor is the sensor corresponding to the field of view that the target enters after leaving the field of view of the first sensor.

Further, in one embodiment, the above step S222, when the target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view is present in the fusion target list, based on the target matching result, updating the sensor identifier and the target identifier corresponding to each fusion target in the common view area, and updating the position information of each fusion target in the common view area in the bird's-eye view, includes the following steps:

Step S2222, when the first target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view exists in the fusion target list, based on the target matching result, determine the second target in the field of view of the second sensor among the adjacent sensors in the bird's-eye view that matches the first target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view.

The above-mentioned determination of the second target that matches the first target in the field of view of the first sensor in the bird's-eye view in the field of view of the second sensor in the adjacent sensor based on the target matching result may be to determine the number of matches of each target that matches the first target in the field of view of the first sensor in the adjacent sensor in the field of view of the bird's-eye view in the field of view of the second sensor in the adjacent sensor according to the target matching result, determine the target with the largest number of matches and the largest number of matches, and when the largest number of matches is greater than or equal to a preset number of matches threshold, the target in the field of view of the second sensor in the bird's-eye view corresponding to the largest number of matches is used as the second target that matches the first target in the field of view of the first sensor in the adjacent sensor in the bird's-eye view. When the largest number of matches is less than the preset number of matches threshold, the position information of the fused target in the common view area in the bird's-eye view is updated according to the position information of the first sensor in the field of view of the bird's-eye view. The above-mentioned preset number of matches threshold can be set according to the specific situation, as long as it can be determined whether the two targets match by the preset number of matches threshold.

Step S2224, update the sensor identifier corresponding to the fused target in the common view area to the identifier corresponding to the second sensor in the adjacent sensors, update the target identifier corresponding to the fused target in the common view area to the identifier of the second target in the field of view of the second sensor in the adjacent sensors in the bird's-eye view, and update the position information of the fused target in the common view area in the bird's-eye view to the position information of the second target in the field of view of the second sensor in the adjacent sensors in the bird's-eye view.

In the above steps S2222 to S2224, when a target in the field of view of the first sensor among the adjacent sensors in the bird's-eye view exists in the fusion target list, based on the target matching result, the sensor identifier and target identifier corresponding to each fusion target in the common view area are updated, and the position information of each fusion target in the common view area in the bird's-eye view is updated, so as to switch the fusion target from the perspective of the first sensor to the perspective of the second sensor. By switching the perspective of the fusion target of each adjacent sensor, it is ensured that the trajectory of the target is continuous and unique throughout the process of the target passing through the target section.

In one embodiment, after step S220, the following steps are further included:

Step S2201, when a fusion target is lost, traverse the frame-synchronized output information of the third sensor corresponding to the field of view where the lost fusion target is located, and determine whether there is a third target with attribute information consistent with the lost fusion target in the frame-synchronized output information of the third sensor.

During the driving process of the target, the fusion target may be lost due to occlusion, recognition error, target identification change and other problems. Therefore, when the fusion target is lost, it is necessary to traverse the output information of the third sensor after frame synchronization corresponding to the field of view where the lost fusion target is located, and determine whether there is a third target whose attribute information is consistent with the lost fusion target in the output information of the third sensor after frame synchronization. Specifically, when the fusion target is lost, according to the attribute information of the lost fusion target in the previous frame image, traverse the output information of the third sensor after frame synchronization corresponding to the field of view where the lost fusion target is located, and determine whether there is a third target whose attribute information is consistent with the lost fusion target in the previous frame image in the output information of the third sensor after frame synchronization.

Step S2202, when there is a third target consistent with the attribute information of the lost fusion target in the output information of the third sensor after frame synchronization, the sensor identifier corresponding to the fusion target is updated to the identifier corresponding to the third sensor, the target identifier corresponding to the fusion target is updated to the identifier of the third target in the field of view of the third sensor in the bird's-eye view, and the position information of the fusion target in the bird's-eye view is updated to the position information of the third target in the field of view of the third sensor in the bird's-eye view.

Step S2203, when there is no third target having attribute information consistent with the lost fusion target in the output information of the third sensor after frame synchronization, traverse the output information of the fourth sensor after frame synchronization to determine whether there is a fourth target having attributes consistent with the lost fusion target in the output information of the fourth sensor after frame synchronization; the fourth sensor is a sensor corresponding to the next field of view area that the target enters when it moves away from the field of view area of the third sensor.

Step S2204, when there is a fourth target with attributes consistent with the lost fusion target in the output information of the fourth sensor after frame synchronization, the sensor identifier corresponding to the fusion target is updated to the identifier corresponding to the fourth sensor, the target identifier corresponding to the fusion target is updated to the identifier of the fourth target in the field of view of the fourth sensor in the bird's-eye view, and the position information of the fusion target in the bird's-eye view is updated to the position information of the fourth target in the field of view of the fourth sensor in the bird's-eye view.

Step S2205, when there is no fourth target whose attribute information is consistent with the lost fusion target in the output information of the fourth sensor after frame synchronization, determine the multiple targets corresponding to the multiple adjacent fusion targets in the output information of the fourth sensor after frame synchronization according to the positional relationship between the fusion target and the multiple adjacent fusion targets when the fusion target is not lost; estimate the first position information of the lost fusion target in the field of view of the fourth sensor in the bird's-eye view based on the multiple targets corresponding to the multiple adjacent fusion targets in the output information of the fourth sensor after frame synchronization, and the positional relationship between the fusion target and the multiple adjacent fusion targets when the fusion target is not lost; update the position information of the fusion target in the bird's-eye view to the first position information of the fourth sensor in the field of view in the bird's-eye view.

In the above steps S2201 to S2205, when the fusion target is lost, the output information of the third sensor after frame synchronization corresponding to the field of view where the lost fusion target is located is traversed to determine whether there is a third target with the same attribute information as the lost fusion target, and how to obtain the position information of the lost fusion target when there is a third target with the same attribute information as the lost fusion target in the output information of the third sensor after frame synchronization or when there is no third target with the same attribute information as the lost fusion target in the output information of the third sensor after frame synchronization. The sensor identifier, target identifier and position information corresponding to the lost fusion target are obtained by comparing the attribute information of the lost fusion target and the position information of the lost fusion target with the output information of the current sensor after frame synchronization or the attribute information of the sensor after frame synchronization at the next viewing angle. When the fusion target is lost due to problems such as long camera deployment distance, vehicle occlusion, and license plate recognition, the target in the current perspective can be found based on the graph constructed by the fusion targets in front, behind, left, and right of the fusion target. Based on the relative position relationship and lane line information of the high-precision map, as well as the historical trajectory information of the fusion target, modeling is performed to predict the position of the target from the bird's-eye view and update the fusion target state. This solves the problem of continuous tracking difficulties caused by long camera deployment distance, vehicle occlusion, and license plate recognition in existing vehicle tracking methods.

In one embodiment, FIG6 is a schematic diagram of target tracking results provided by an embodiment of the present application. As shown in FIG6, the left area shows a schematic diagram of a bird's-eye view perspective on a high-precision map, wherein each point represents a track point of a vehicle in a road section, and the box in the upper left corner represents the vehicle position status information pointed to by the current arrow, such as the center point position, length, width and driving angle of the vehicle. On the right is the picture information after frame synchronization output by the sensor identified as 01 and the sensor identified as 02. The vehicles in the two frames are the same vehicle in different perspectives, wherein the time information in the upper right corner represents that they are at the same time. The target vehicle identified as 01 drives into the area identified as 02, and the fusion target 51 switches from the box in the picture output by the sensor identified as 01 to the box in the picture output by the sensor identified as 02. At this time, the perspective of the fusion target 51 is switched, and the sensor identified as 02 will be mainly responsible for monitoring the status of the fusion target 51. The target will match and relay the vehicle in the order of the camera to ensure that the target is complete and continuous in the road section.

The present embodiment is described and illustrated below through preferred embodiments.

FIG7 is a flow chart of a target tracking method provided by a preferred embodiment of the present application. As shown in FIG7 , the target tracking method includes the following steps:

Step S701, determining the position information of each sensor located on the target road section based on the sensor type and the road section environment of the target road section;

Step S702, determining the field of view of each sensor in the bird's-eye view of the target road section based on the position information and calibration information of each sensor located on the target road section;

Step S703, performing frame synchronization on the output information of all sensors located on the target road section to obtain the frame-synchronized output information of each sensor;

Step S704, based on the frame-synchronized output information of each sensor and the field of view of each sensor in the bird's-eye view of the target road section, all fusion targets in the target road section are determined to obtain a fusion target set;

Step S705, based on the fusion target set, determining the mapping relationship between each fusion target and the sensor identifier and the target identifier, as well as the position information of each fusion target in the bird's-eye view of the target road section;

Step S706, matching the targets in the common viewing area in the bird's-eye view of the target road section based on the frame-synchronized output information of each sensor to obtain a target matching result;

Step S707, based on the target matching result, updating the sensor identifier and target identifier corresponding to each fused target in the common view area, and updating the position information of each fused target in the common view area in the bird's-eye view of the target road section;

Step S708, based on the frame synchronized output information of each sensor, the updated sensor identifier and the updated target identifier corresponding to each fused target in the common viewing area, and the updated position information of each fused target in the common viewing area in the bird's-eye view of the target section, obtain the continuous display result of each fused target in the target section.

In the above steps S701 to S708, the position information of each sensor located on the target road section is determined, and based on the position information and calibration information of each sensor located on the target road section, the field of view of each sensor in the bird's-eye view of the target road section is determined, and then based on the output information of each sensor after frame synchronization, the targets in the common viewing area in the bird's-eye view of the target road section are matched to obtain the target matching result, and based on the target matching result, the sensor identifier and target identifier corresponding to each fused target in the common viewing area are updated, and the position information of each fused target in the common viewing area in the bird's-eye view of the target road section is updated. By matching the targets in the common viewing area in the bird's-eye view of the target road section, an association between different viewing angles of the two sensors is established, and the target matching result is obtained according to the target matching result. According to the matching results, the sensor identifier and the target identifier corresponding to the fusion target in the common viewing area are updated, and the position information of each fusion target in the common viewing area in the bird's-eye view of the target road section is updated, so that the same fusion target in different field of view of each sensor is associated and fused, and then, according to the output information after frame synchronization of each sensor, the updated sensor identifier and the updated target identifier corresponding to each fusion target in the common viewing area, and the updated position information of each fusion target in the common viewing area in the bird's-eye view of the target road section, the continuous display of the same target in the same bird's-eye view is realized, and the continuous tracking of the target is realized, which solves the problem of continuous tracking difficulty caused by the long camera deployment distance, vehicle occlusion and license plate recognition in the existing vehicle tracking method.

It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, a target tracking device is also provided in this embodiment, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and/or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

In one embodiment, FIG8 is a structural block diagram of a target tracking device provided in an embodiment of the present application. As shown in FIG8 , the target tracking device includes:

The matching module 82 is used to match the target in the common view area in the bird's-eye view of the target road section based on the output information of each sensor after frame synchronization, and obtain the target matching result; the common view area is the overlapping area between different view areas of adjacent sensors set in the bird's-eye view of the target road section;

An updating module 84 is used to update the sensor identifier and target identifier corresponding to each fused target in the common viewing area based on the target matching result, and update the position information of each fused target in the common viewing area in the bird's-eye view of the target road section;

And a display module 86 is used to obtain continuous display results of each fused target on the target section based on the output information of each sensor after frame synchronization, the updated sensor identifier and the updated target identifier corresponding to each fused target in the common viewing area, and the updated position information of each fused target in the common viewing area in the bird's-eye view of the target section.

The target tracking device matches the targets in the common viewing area of the bird's-eye view of the target section based on the output information of each sensor after frame synchronization to obtain the target matching result, updates the sensor identification and target identification corresponding to each fused target in the common viewing area based on the target matching result, updates the position information of each fused target in the common viewing area in the bird's-eye view of the target section, matches the targets in the common viewing area in the bird's-eye view of the target section, establishes the association between the different perspectives of the two sensors, and updates the sensor identification and target identification corresponding to the fused target in the common viewing area according to the target matching result, and updates the position information of each fused target in the common viewing area in the bird's-eye view of the target section. The position information of each fused target in the bird's-eye view of the target road section enables the same fused target in different field of view areas of each sensor to be associated and fused, and then, based on the output information of each sensor after frame synchronization, the updated sensor identifier and the updated target identifier corresponding to each fused target in the common view area, and the updated position information of each fused target in the common view area in the bird's-eye view of the target road section, the continuous display of the same target in the same bird's-eye view is realized, and the continuous tracking of the target is realized, which solves the problem of continuous tracking difficulty caused by the long camera deployment distance, vehicle occlusion and license plate recognition in the existing vehicle tracking method.

It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, any one of the target tracking methods in the above embodiments is implemented.

Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims (9)

1. A method of target tracking, the method comprising:

Performing frame synchronization on the output information of all the sensors positioned on the target road section to obtain output information of each sensor after frame synchronization; the output information of the sensor comprises all targets in the visual field area shot by the sensor and attribute information of all targets; determining all fusion targets in the target road section based on the output information of each sensor after frame synchronization and the visual field area of each sensor in the aerial view of the target road section to obtain a fusion target set; determining mapping relations among the fusion targets, the sensor identifications and the target identifications and position information of the fusion targets in the aerial view of the target road section based on the fusion target set; the fusion target is a unified representation of the same target shot by different sensors in the target section; based on the output information of each sensor after frame synchronization, matching targets in a common area in the aerial view of the target road section to obtain a target matching result; the common view area is an overlapping area between different view areas in the bird's eye view of the target road section, and is an adjacent sensor arranged in the target road section;

Based on the target matching result, updating the sensor identifier and the target identifier corresponding to each fusion target in the common area, and updating the position information of each fusion target in the common area in the aerial view of the target road section; the target identification is the identification of each target in the output information of each sensor after frame synchronization;

Based on the output information of the sensors after frame synchronization, updated sensor identifications and updated target identifications corresponding to the fusion targets in the common area and updated position information of the fusion targets in the aerial view of the target road section in the common area, and a continuous display result of the fusion targets in the target road section is obtained.

2. The target tracking method according to claim 1, wherein before determining all the fusion targets in the target section based on the output information of each sensor after frame synchronization and the field of view of each sensor in the bird's eye view of the target section, obtaining a fusion target set, comprising:

And determining the visual field area of each sensor in the aerial view of the target road section based on the position information and the calibration information of each sensor positioned on the target road section.

3. The target tracking method according to claim 2, characterized by, before determining a field of view of each sensor in a bird's eye view of the target section based on position information and calibration information of each sensor located on the target section, comprising:

position information of each of the sensors located at the target link is determined based on the sensor type and the link environment of the target link.

4. The method according to claim 1, wherein the determining all the fusion targets in the target section based on the output information of each sensor after frame synchronization and the field of view of each sensor in the aerial view of the target section to obtain the fusion target set includes:

Based on the output information of the sensors after frame synchronization, taking each target of the first sensor of the target road section in a visual field area in the aerial view of the target road section as a first fusion target; the first sensor of the target road section is the first sensor of the target entering the target road section;

the method comprises the steps that targets, which are first appeared in the area except the common view area, of the visual field area in the aerial view of the target road section except the first sensor in all the sensors of the target road section are used as second fusion targets;

and generating the fusion target set based on the first fusion target and the second fusion target.

5. The method for tracking a target according to claim 1, wherein the matching the target in the common area in the bird's eye view of the target section based on the output information after the frame synchronization of each sensor to obtain a target matching result includes:

Based on the output information of each sensor after frame synchronization, under the view angle of the aerial view, performing cross-over ratio calculation on each target of the adjacent sensor in the common area in the aerial view of the target road section to obtain a first cross-over ratio calculation result; based on the first intersection ratio calculation result, matching targets of adjacent sensors of the target road section in a common area to obtain a target matching result;

Or, based on the output information of the sensors after frame synchronization, determining a first point set feature between each target in the common area determined by the adjacent sensor and each target in the visual field area of the first sensor in the adjacent sensor in the aerial view of the target road section under the aerial view angle; determining a second point set characteristic between each target in the common area determined by the adjacent sensor and each target in the visual field area of the second sensor in the adjacent sensor in the aerial view of the target road section; performing similarity matching on targets in a common area in the aerial view of the target road section based on the first point set features and the second point set features to obtain a similarity matching result; obtaining the target matching result based on the similarity matching result;

Or, based on the output information of the sensors after frame synchronization, projecting each target in the visual field area in the bird's eye view of the first sensor in the adjacent sensor to each target in the visual field area in the bird's eye view of the second sensor in the adjacent sensor, so as to obtain each projected target; performing cross-over ratio calculation on each projected target and each target in a visual field area of a second sensor in the adjacent sensor in the aerial view to obtain a second cross-over ratio calculation result; and matching targets of adjacent sensors of the target road section in a common area based on the second intersection ratio calculation result to obtain the target matching result.

6. The method according to claim 5, wherein updating the sensor identifier and the target identifier corresponding to each of the fused targets in the common area based on the target matching result, and updating the position information of each of the fused targets in the common area in the aerial view of the target section includes:

When targets of the first sensor in the adjacent sensors in the visual field area in the aerial view exist in a fusion target list, based on the target matching result, updating sensor identifiers and target identifiers corresponding to the fusion targets in the common area, and updating position information of the fusion targets in the aerial view in the common area.

7. The object tracking method according to claim 6, wherein updating the sensor identification and the object identification corresponding to each of the fusion objects in the common area when the object in the field of view of the first sensor in the adjacent sensor exists in the fused object list, based on the object matching result, comprises:

When a first target in the view field area of the adjacent sensor in the bird's eye view exists in a fused target list, determining a second target in the view field area of the adjacent sensor in the bird's eye view based on the target matching result, wherein the second target is matched with the first target in the view field area of the first sensor in the adjacent sensor in the bird's eye view;

Updating a sensor identifier corresponding to the fusion target in the common area as an identifier corresponding to the second sensor in the adjacent sensor, updating a target identifier corresponding to the fusion target in the common area as an identifier of a second target in a field-of-view area of the second sensor in the adjacent sensor in the bird's-eye view, and updating position information of the fusion target in the common area in the bird's-eye view as position information of the second target in the field-of-view area of the second sensor in the adjacent sensor in the bird's-eye view.

8. The object tracking method according to any one of claims 1 to 7, characterized by, after updating the sensor identifications and the object identifications corresponding to the respective fusion objects in the common area based on the object matching result, updating the position information of the respective fusion objects in the common area in a bird's eye view, further comprising:

When the fusion target is lost, traversing output information after frame synchronization of a third sensor corresponding to a field of view area where the lost fusion target is located, and judging whether a third target consistent with attribute information of the lost fusion target exists in the output information after frame synchronization of the third sensor;

When the output information of the third sensor after the frame synchronization has the third target consistent with the attribute information of the lost fusion target, updating the sensor identifier corresponding to the fusion target to be the identifier corresponding to the third sensor, updating the target identifier corresponding to the fusion target to be the identifier of the third target in the visual field area of the third sensor in the bird's-eye view, and updating the position information of the fusion target in the bird's-eye view to be the position information of the third target of the third sensor in the visual field area of the bird's-eye view;

When the third target consistent with the attribute information of the lost fusion target does not exist in the output information after frame synchronization of the third sensor, traversing the output information after frame synchronization of a fourth sensor, and judging whether a fourth target consistent with the attribute of the lost fusion target exists in the output information after frame synchronization of the fourth sensor; the fourth sensor is a sensor corresponding to a next visual field area which is entered by a visual field area of a target far away from the third sensor;

When the fourth target consistent with the attribute of the lost fusion target exists in the output information of the fourth sensor after the frame synchronization, updating the identifier corresponding to the fusion target as the identifier corresponding to the fourth sensor, updating the identifier corresponding to the fusion target as the identifier of the fourth target in the view area of the fourth sensor in the bird's-eye view, and updating the position information of the fusion target in the bird's-eye view as the position information of the fourth target in the view area of the fourth sensor in the bird's-eye view;

When the fourth target consistent with the attribute information of the lost fusion target does not exist in the output information after frame synchronization of the fourth sensor, determining a plurality of targets corresponding to a plurality of adjacent fusion targets in the output information after frame synchronization of the fourth sensor according to the position relation between the fusion target and the adjacent fusion targets when the fusion target is not lost; estimating first position information of a lost fusion target in a visual field area of the fourth sensor in a bird's eye view based on a plurality of targets corresponding to a plurality of adjacent fusion targets in the output information of the fourth sensor after frame synchronization and the position relation between the fusion target and the adjacent fusion targets when the fusion target is not lost; and updating the position information of the fusion target in the aerial view as the first position information of the fourth sensor in the visual field area in the aerial view.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the object tracking method of any one of claims 1 to 8 when the computer program is executed.