CN108833928B - Traffic monitoring video coding method - Google Patents

Abstract

The invention discloses a traffic monitoring video coding method, which is based on a vehicle and a background database to realize traffic monitoring video coding, can effectively remove global redundancy of a traffic monitoring video in a time dimension after paying a certain cost of storage space, and finally has the overall effect of effectively improving the overall coding performance of the traffic monitoring video under the condition of not obviously increasing the complexity of a coding end and a decoding end.

Description

Traffic Surveillance Video Coding Method

technical field

The invention relates to the technical field of video coding, in particular to a video coding method for traffic monitoring.

Background technique

In recent years, with the rapid development of smart transportation, the amount of surveillance video data has exploded. In order to effectively store and transmit surveillance video data, the first problem to be solved is the encoding of surveillance video.

Currently, the compression of surveillance video usually adopts the general video coding standard H.264/AVC or H.265/HEVC. However, considering some characteristics of surveillance video, such as the stillness of surveillance cameras, etc., the general video coding technology is directly used in the coding of surveillance video, which cannot make full use of the inherent characteristics of surveillance video. In order to further improve the performance of surveillance video coding, many researchers have invented a series of coding techniques for surveillance video.

Generally speaking, the content in the surveillance video can be roughly divided into background content and foreground content. Correspondingly, the coding for surveillance video can be designed from two aspects: optimizing background coding and optimizing foreground coding. Considering the static characteristics of surveillance cameras, optimized background encoding usually generates a high-quality background frame first, and then relies on quality transfer to improve the encoding efficiency of the overall surveillance video. In terms of optimizing foreground encoding, researchers have successively proposed some foreground encoding techniques based on models and object segmentation.

Some works also propose other surveillance video coding techniques, such as:

Adaptive prediction techniques based on background modeling (Xianguo Zhang, Tiejun Huang, YonghongTian, and WenGao, “Background-modeling-based adaptive prediction for surveillance video coding,” IEEE Transactions on ImageProcessing, vol.23, no.2, pp.769–784 , 2014.)

Global Vehicle Coding Technology Based on Vehicle 3D Model Database (Jing Xiao, Ruimin Hu, LiangLiao, Yu Chen, Zhongyuan Wang, and ZixiangXiong, "Knowledge-based coding of objects for multisource surveillance video data," IEEE Transactions on Multimedia, vol.18, no.9 , pp.1691–1706, 2016.)

Disadvantages of the above method:

1. The background encoding technology based on high-quality background frames will cause a surge of code streams when generating high-quality background frames, which will cause adverse effects on network transmission, and the encoding performance also needs to be improved.

2. The foreground coding technology based on model and object segmentation is difficult to perform fine pixel-level segmentation of the foreground, and since the segmented foreground may be irregular in shape, the code rate used to represent it is very huge.

3. The adaptive prediction technology based on background modeling subtracts the reconstructed background frame from the current frame and the reference frame at the same time, and then directly uses the obtained foreground pixels of the current frame to perform inter-frame prediction on the foreground pixels of the reference frame when encoding the foreground. When the segmentation effect of foreground pixels is not good, it is easy to cause adverse effects on the improvement of foreground coding efficiency.

4. The global vehicle coding technology based on the vehicle 3D model database does not store the texture information of the vehicle, so the reconstruction quality of the vehicle cannot be improved. In addition, the 3D model of the vehicle, the internal and external parameters of the surveillance camera, and the position and attitude information of the vehicle on the road required by the technology are difficult to obtain or estimate, which brings difficulties to the practical application of the technology.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a traffic monitoring video coding method, which can improve the coding performance of the traffic monitoring video.

The purpose of this invention is to realize through the following technical solutions:

A traffic monitoring video coding method, which mainly comprises the following steps:

Step 1. The original traffic monitoring video sequence is processed by the front and background segmentation method, the vehicle and the background are separated, and the redundancy existing between the separated vehicle and the background is removed respectively, and then put into the database.

Step 2: For the traffic monitoring video to be encoded, the foreground and background segmentation method is also used to separate the vehicle to be encoded and the background to be encoded; for the vehicle to be encoded, the matching vehicle is selected from the database by means of feature matching and rapid motion estimation; Encoded backgrounds are based on absolute differences and matching backgrounds are selected from the database.

Step 3. When using the inter-frame prediction mode or the intra-frame prediction mode, use a predetermined method to determine whether the vehicle to be encoded or the background to be encoded needs to be subjected to rate-distortion optimization processing on the matching vehicle or matching background; Use the corresponding prediction mode for encoding.

It can be seen from the above technical solution provided by the present invention that the traffic monitoring video coding based on the vehicle and the background database can effectively remove the global redundancy existing in the time dimension of the traffic monitoring video after paying a certain storage space cost. Finally, The overall effect is to effectively improve the overall encoding performance of traffic surveillance video without significantly increasing the complexity of encoding and decoding.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a flowchart of a traffic monitoring video encoding method provided by an embodiment of the present invention;

2 is a schematic diagram of a traffic monitoring video coding framework provided by an embodiment of the present invention;

FIG. 3 is a flowchart of removing SIFT features of vehicle area background provided by an embodiment of the present invention;

FIG. 4 is a flow chart of vehicle and background similarity analysis provided by an embodiment of the present invention;

5 is a schematic diagram of reference index bit change information provided by an embodiment of the present invention;

FIG. 6 is a screenshot of a test sequence provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

An embodiment of the present invention provides a traffic monitoring video encoding method, as shown in FIG. 1 , which mainly includes the following steps:

Step 1. The original traffic monitoring video sequence is processed by the front and background segmentation method, the vehicle and the background are separated, and the redundancy existing between the separated vehicle and the background is removed respectively, and then put into the database.

Step 2: For the traffic monitoring video to be encoded, the foreground and background segmentation method is also used to separate the vehicle to be encoded and the background to be encoded; for the vehicle to be encoded, the matching vehicle is selected from the database by means of feature matching and rapid motion estimation; Encoded backgrounds are based on absolute differences and matching backgrounds are selected from the database.

Step 3. When using the inter-frame prediction mode or the intra-frame prediction mode, use a predetermined method to determine whether the vehicle to be encoded or the background to be encoded needs to be subjected to rate-distortion optimization processing on the matching vehicle or matching background; Use the corresponding prediction mode for encoding.

The schematic diagram of the entire coding framework is shown in Figure 2, wherein the offline part is the above-mentioned step 1, and the online part is also the above-mentioned steps 2 to 3.

For ease of understanding, the above three steps are described in detail below.

1. Establishment of vehicle and background database.

In the embodiment of the present invention, for the original traffic monitoring video sequence, a foreground and background segmentation method (for example, the SuBSENSE method) is used to separate the vehicles, the background is extracted from the background model generated when the foreground is separated, and the vehicles and The background is used to build the database. The main implementation process can refer to the following methods:

1. Establish vehicle database.

The preferred embodiment of vehicle database establishment is as follows:

After the vehicles are separated from the front part of the original traffic monitoring video sequence and the redundancy is removed, the vehicles are numbered from 1 to N; N is the number of separated vehicles.

Initially, the vehicle in the database is empty; for a certain vehicle v _i with redundancy removed, the method based on the inverted table is _{used to retrieve similar vehicles {v i1 ,} v _i2 , . .., v _im }, where m is the number of similar vehicles.

In order to determine the size of m, considering the number of SIFT features matched by vehicles v _i and v _j , the way to determine whether the two SIFT features match can be implemented by conventional techniques or by the methods mentioned in the introduction of vehicle matching later.

When retrieving similar vehicles, compare the number of SIFT features matched by vehicle v _i and any vehicle v _j among the other vehicles. When the number of SIFT features matched by vehicle v _i and vehicle v _j satisfies the following formula, put vehicle v _j into {v _i1 , v _i2 , ..., v _im } in:

N _ij ≥β×N _i ;

N _ij ≥ min(N ₀ , N _i );

In the above formula, N _ij is the number of SIFT features matched by vehicle v _i and vehicle v _j , N _i is the number of SIFT features in vehicle v _i , β and N ₀ are constants; exemplarily, β and N ₀ can be set respectively are 0.1 and 4. After processing in the above manner, similar vehicles {v _i1 , v _i2 , . . . , v _im } corresponding to the vehicle v _i can be obtained.

After that, the pixel-level similarity is compared for the vehicles: for the vehicle v _i , if the vehicle in the database is empty, put the vehicle v _i into the database; otherwise, put the vehicle v _i with {v _i1 , v _i2 , .. ., v _im } The vehicles that have been put into the database are compared for pixel-level similarity, and the fast motion estimation method is used when the similarity is compared, and the sum of absolute difference (SAD) is used as the loss function.

The fast motion estimation method mentioned here can be implemented by conventional techniques, or a specific fast motion estimation method used in vehicle matching described later.

If the calculated absolute difference and average value are less than a set value (eg, 5), it is determined that the two vehicles are similar at the pixel level. Those skilled in the art can understand that when the similarity is calculated, the object of each calculation is a vehicle v _i and {v _i1 , v _i2 , . . . , _vim } that has been put into the database. When calculating the absolute difference sum, the vehicle _vi is divided into blocks of a certain size, and a certain block on the vehicle _vi is used for fast motion estimation on the entire image of a vehicle in the database; For each 16x16 block, an SAD value is obtained; the absolute difference and average value considered here are also the average of the absolute difference _sums of all 16x16 blocks in vi.

If the vehicles in {v _i1 , _{v i2 , .} , the vehicle _vi is not put into the database.

If the final decision vehicle v _i is put into the database, compare the pixel-level similarity between the vehicles already put into the database in {v _i1 , v _i2 , ..., _vim } and the vehicle v _i , if there is a If the vehicle v _i is similar to the vehicle at the pixel level, the similar vehicles that have been put into the database will be removed from the database; if the cumulative number of vehicles is not similar to the vehicle v _i at the pixel level, the above checking process will stop.

Each vehicle is processed in the above-mentioned manner, and the final vehicle to be put into the database is determined, encoded and put into the database.

2. Background database establishment

For the redundant background, take a frame of background every time period (for example, 20s) and put it into the database after encoding.

In practical applications, after the installation of the surveillance camera is completed, the encoder first performs the establishment of the vehicle and background database. For the vehicle, the encoder performs high-quality encoding on the vehicle to be put into the database according to the aforementioned steps of establishing the vehicle database, and puts the encoded vehicle into the database. At the same time, the information used to identify these vehicles is also encoded into the code stream. After the decoding end decodes the reconstructed image, the same vehicle database establishment process is performed according to the decoded vehicle identification information; for the background, the encoder follows the previous steps of establishing the background database. , perform high-quality encoding on the generated background frames at regular intervals, and put the encoded background into the database. At the same time, the high-quality encoded backgrounds and the information used to identify these backgrounds are also encoded into the code stream. After the decoder decodes the high-quality background frames according to the above information, it puts them into the database. In this way, the same vehicle and background database can be established on the codec side.

In the embodiment of the present invention, the original traffic monitoring video sequence can be divided, the former part of the data is used to establish a vehicle and background database; the latter part is used as the traffic monitoring video to be encoded. Of course, the traffic surveillance video of the first day can also be used to establish a vehicle and background database, and the data from the second day can be used as the traffic surveillance video to be encoded. The codec performs the codec work of the traffic monitoring video according to the method of the present invention. The general traffic surveillance video is usually saved for a period of several months, and the above-mentioned work is repeated after the saved data is emptied.

2. Vehicle and Background Search

1. Vehicle search.

1) Vehicle and background separation and de-redundancy operations.

In the embodiment of the present invention, the separation of the vehicle and the background and the redundancy removal operation are also required for the traffic surveillance video to be encoded; this part of the operation process is similar to the operation when the vehicle and background database is established. The preferred implementation of this operation process is as follows:

After the vehicle in the surveillance video sequence (original traffic surveillance video sequence or traffic surveillance video to be encoded) is separated by the SuBSENSE method, since the shape of the vehicle may be irregular, the separated vehicle is separated from the upper left corner to the lower right corner of the square area. The pixels of the vehicle are used as the vehicle, and the remaining part is used as the background. The SIFT features of the vehicle are extracted, and the background SIFT features are removed. The process of removing the background SIFT features is shown in Figure 3.

While using the SuBSENSE method to separate vehicles, it will gradually generate relatively clean background frames. While extracting the vehicle from the surveillance video sequence, the background of the corresponding position on the background frame is extracted.

Taking the traffic surveillance video to be encoded as an example, for the separated current vehicle to be encoded and the corresponding background, the SIFT features of the two are extracted respectively. For each SIFT feature extracted from the current vehicle to be encoded, the following formula is used in the corresponding Search within a certain location neighborhood range on the background:

(xs _c -xs _b ) ² +(ys _c -ys _b ) ² ≤d ² ;

Wherein, xs _c and ys _c represent the coordinates of the SIFT features extracted from the current vehicle to be encoded, xs _b and ys _b represent the coordinates of the SIFT features extracted from the corresponding background; d is the bounded range of the location neighborhood; an exemplary , you can set d=5.

If the distance between the retrieved SIFT feature with the smallest Euclidean distance after normalization and a certain SIFT feature of the current vehicle to be encoded is less than a certain threshold: D _min ≤ D ₁ ; where D _min is the normalized Euclidean distance The normalized Euclidean distance between the smallest SIFT feature and a certain SIFT feature of the currently to-be-coded vehicle, D ₁ is the threshold, and D ₁ =1.1 can be set exemplarily; The SIFT feature of the encoded vehicle is similar to the SIFT feature of the vehicle, and the corresponding SIFT feature of the current vehicle to be encoded is the background SIFT feature, which is removed from the vehicle SIFT.

2) Use feature matching for rough retrieval.

In the embodiment of the present invention, the SIFT features of the vehicles (including the vehicles in the database and the vehicles to be encoded) are extracted, the vehicles in the database establish an inverted table index based on the SIFT features, and for the vehicles to be encoded, a rough search is performed from the database based on the SIFT feature matching Select several candidate vehicles. The preferred implementation of this process is as follows:

Use feature matching to roughly select several candidate vehicles from the database: quantify the SIFT features of each vehicle in the database into visual text using the k-means algorithm, and calculate the corresponding mapping mean vector for each visual text; Each SIFT feature of each vehicle in the database is mapped to the nearest neighbor visual text, and the mapped SIFT feature vector is compared with the mapped mean vector corresponding to the nearest neighbor visual text, and the binary representation of each SIFT feature vector is obtained. Each vehicle in the database is represented by the frequency histogram of the visual text corresponding to its SIFT feature, and the frequency histogram of each vehicle in the database is organized by means of an inverted table.

For the current vehicle to be coded, according to the above method of processing vehicles in the database, each SIFT feature is assigned to the nearest visual text to obtain the frequency histogram of the current vehicle to be coded, and the binarization of each SIFT feature is calculated at the same time. characterization.

When comparing the similarity between the current vehicle to be encoded and a vehicle in the database, under the condition that the Hamming distance of the binarized representation of the SIFT feature mapped to the same visual text is less than a certain threshold, tf-idf (term frequency- Inverse document frequency, item frequency - inverse document frequency) item weighted frequency histogram distance is used as the evaluation index of similarity, and the comparison result of the similarity between the current vehicle to be encoded and each vehicle in the database is obtained; according to the calculated similarity The comparison results are sorted, and several vehicles with the highest similarity ranking are selected as candidate vehicles.

Exemplarily, in a specific implementation, 10 candidate vehicles may be retrieved.

3) Vehicle selection using fast motion estimation.

In the embodiment of the present invention, a matching vehicle is selected from several candidate vehicles by means of fast motion estimation; the preferred implementation of this process is as follows:

a. Align the current vehicle to be encoded with each candidate vehicle.

The preferred implementation of alignment is as follows:

For a certain SIFT feature of the current vehicle to be encoded, calculate the distance between it and all SIFT features of each candidate vehicle, sort the calculated distances in ascending order, and determine the current vehicle to be encoded if the following formula is satisfied The corresponding SIFT features of are found matching SIFT features in the corresponding candidate vehicles:

d ₁ ≤ D ₂ ;

d ₁ /d ₂ ≤α;

Among them, d ₁ and d ₂ are the smallest and second smallest distances, respectively, and D ₂ and α are constants;

Calculate each SIFT feature of the current vehicle to be encoded according to the above method, and obtain the SIFT matching pair of the current vehicle to be encoded and each candidate vehicle; position offset, as follows:

Among them, MV _x and MV _y are the horizontal and vertical components of the offset, _n is the number of matched SIFT feature pairs, xci and _yci are the coordinates of the SIFT feature of the current vehicle to be encoded _, and xvi and _yvi are The coordinates of the SIFT feature of the candidate vehicle; i is the sequence number of the SIFT feature matching pair;

Then an iterative method is used to remove abnormal points to obtain the final position offset result; according to the calculated position offset result, the current vehicle to be encoded is aligned with the corresponding candidate vehicle.

The abnormal point can be determined by the following way: if the motion vector calculated by a certain pair of SIFT matching pairs deviates far from the mean motion vector (ie, exceeds the set value), the SIFT feature matching pair is an abnormal point.

b. Divide the current vehicle to be encoded into blocks with a fixed size of 16x16, each 16x16 block searches for the block with the smallest loss function in a candidate vehicle, where the loss function is composed of the absolute difference sum and the encoding code rate of the motion vector ; The search method is to take the position of the current 16x16 block as the starting point, perform an eight-point diamond search within 64 pixels around the starting point, and accumulate the loss functions of all 16x16 blocks as the entire current vehicle to be encoded. The overall loss function on a candidate vehicle; the candidate vehicle with the smallest overall loss function is finally reserved as the matching vehicle.

2. Background search.

In the embodiment of the present invention, for the background to be encoded, the matching background is selected from the database based on the absolute difference and the preferred implementation of this process is as follows:

Taking the absolute difference sum of the current background to be encoded and the pixels corresponding to the background in the database as the similarity evaluation criterion, calculate the absolute difference sum of the current background to be encoded and each background in the database, as shown in the following formula:

SAD = ∑ _k∈B |pc _k -pl _k |;

Wherein, pc _k and pl _k are the current background to be encoded and the value of the kth pixel of the background in the database, respectively, and B is the set of current background pixels to be encoded;

Sort the calculation results from small to large, and use the background with the absolute difference and the smallest as the matching background of the current background to be encoded.

3. Coding.

1. Similarity analysis.

In the embodiment of the present invention, after determining the matching vehicle and background of the current vehicle to be encoded and the background, it is determined whether rate-distortion optimization (RDO) is performed on the matching vehicle and background for the current vehicle and background. When the current vehicle and background are in the inter-frame prediction mode, the matching vehicle and background are compared with the existing reference frame information of the current vehicle and the background for RDO; when the current vehicle and the background are in the intra-frame prediction mode, the candidate vehicle and background are compared with the current vehicle. Compare with RDO with rough background intra-frame prediction. The specific process of vehicle and background similarity analysis is shown in Figure 4. The comparison of RDO in the inter-frame and intra-frame prediction modes will be described in detail below.

1) Comparison of RDO in inter prediction mode.

The comparison criteria for rate-distortion optimization in inter prediction mode are:

Among them, I is the Lagrangian loss function, D is the absolute difference sum of the prediction block and the matching block, R is the number of bits used to represent the pattern information, and λ is the Lagrangian multiplier;

In order to compare the matching vehicle with the background and the existing reference frame, first calculate the Lagrangian loss function of the current vehicle to be encoded and the background and the existing reference frame, and then calculate the matching vehicle and background obtained by considering the retrieval. The updated Lagrangian loss function, compare the Lagrangian loss function before and after the update, and determine whether to do RDO on the matching vehicle and background. The preferred implementation of this process is as follows:

a. Calculate the Lagrangian loss function of the current vehicle to be encoded, the current background to be encoded and the existing reference frame:

For each existing reference frame of the current vehicle to be encoded, first estimate the displacement of the current vehicle on the existing reference frame, then obtain the optimal RDO result of the current vehicle on the existing reference frame, and finally compare it with the current vehicle on the existing reference frame. Compare the optimal RDO results on the candidate matching vehicle to determine whether to do RDO on the candidate matching vehicle. The relevant process is as follows:

Taking a 4×4 block as a unit, obtain the motion vector (Motion Vector, MV) of the 4×4 block using inter-frame prediction and the picture number (POC) information of its reference frame at the corresponding position of the current vehicle to be encoded. Based on this, the motion vector information of the corresponding 4×4 block on the current vehicle to be encoded is estimated. The estimated formula is as follows:

Among them, MVX _ref and MVY _ref are the horizontal component and vertical component of the 4×4 block motion vector of the inter-frame prediction on the existing reference frame, respectively; POC _cur , POC _ref and POC _colref are respectively the frame of the current to-be-encoded vehicle. POC, the POC of the existing reference frame, and the POC of the 4×4 block reference frame for inter-frame prediction on the existing reference frame; MVX _cur and MVY _cur are the estimated values of the 4×4 block motion vector corresponding to the current vehicle to be encoded, respectively. Horizontal component and vertical component; traverse each 4×4 small block in the current vehicle to be encoded, record the number of inter-frame prediction 4×4 blocks and the corresponding motion vector of the 4×4 block of the current vehicle to be encoded, and finally The estimated horizontal component and vertical component of the currently to-be-coded vehicle motion vector are the average value of all inter-frame prediction 4x4 small block motion vectors;

Thereby, the displacement of the current vehicle to be encoded on each existing reference frame is obtained. After that, the current vehicle to be encoded is divided into blocks with a fixed size of 16x16, and each 16x16 block is sequentially searched in all existing reference frames. The minimum loss function The loss function is composed of the absolute difference sum and the coding rate of the motion vector; the search method is to take the position of the current 16x16 block translated by the estimated displacement as the starting point, and 64 pixels around the starting point. Perform an eight-point diamond search within the range; take 16x16 blocks as a unit, record the minimum loss function of all blocks in the current vehicle to be encoded and matching blocks in all existing reference frames; traverse each 16x16 block in the current vehicle to be encoded in turn , accumulate the sum of the minimum loss functions recorded, and obtain the Lagrangian loss function of the current vehicle to be encoded relative to the existing reference frame

For the current background to be encoded, it is divided into 16x16 blocks; for the current 16x16 block, the matching block corresponding to the minimum loss function is searched from all existing reference frames; the search method is to compare the 16x16 blocks in the corresponding positions of all reference frames The sum of absolute differences with the current 16x16 block in the current background to be encoded, and the smallest absolute difference sum is selected as the loss function of the current 16x16 block in the current background to be encoded; traverse all 16x16 blocks in the current background to be encoded, and accumulate all 16x16 blocks The loss function of the block, as the Lagrangian loss function of the current background to be encoded

b. Taking into account the matching vehicle and background, calculate the updated Lagrangian loss function:

计算结果的基础上，采用快速运动估计的方法，计算其与匹配车辆的损失函数；再将每一16x16的块与匹配车辆的损失函数，与计算拉格朗日损失函数

时得到的与现有参考帧的最小损失函数比较，取较小者为相应16x16的块的最小损失函数；遍历当前待编码车辆内的每个16x16的块，累加每个16x16的块的最小损失函数，得到当前待编码车辆的拉格朗日损失函数

同时，对于当前待编码车辆，引起比特数的变化包含了匹配车辆在数据库中的位置索引信息、匹配车辆在参考帧中的位置信息、参考索引(参考帧的索引)比特变化信息和CTU级别的表示信息，将这些比特数变化与拉格朗日损失函数

组合起来，得到更新后的拉格朗日损失函数

For each 16x16 block within the current vehicle to be encoded, the Lagrangian loss function

On the basis of the calculation results, the method of fast motion estimation is used to calculate the loss function of the matching vehicle; then each 16x16 block and the loss function of the matching vehicle are calculated with the Lagrangian loss function.

Compared with the minimum loss function of the existing reference frame, the smaller one is the minimum loss function of the corresponding 16x16 block; traverse each 16x16 block in the current vehicle to be encoded, and accumulate the minimum loss of each 16x16 block. function to get the Lagrangian loss function of the current vehicle to be encoded

At the same time, for the current vehicle to be encoded, the change in the number of bits caused by the matching vehicle includes the position index information of the matching vehicle in the database, the position information of the matching vehicle in the reference frame, the reference index (reference frame index) bit change information and the CTU level To represent the information, combine these bit number changes with the Lagrangian loss function

Combined to get the updated Lagrangian loss function

计算结果的基础上，计算与匹配背景的损失函数；再将每一16x16的块与匹配背景的损失函数，与计算拉格朗日损失函数

时得到的与现有参考帧的最小损失函数比较，取较小者为相应16x16的块的最小损失函数；遍历当前待编码背景内的每个16x16的块，累加每个16x16的块的最小损失函数，得到当前待编码背景的损失函数

同时，对于当前待编码背景，引起比特数的变化包含了匹配背景在数据库中的位置索引信息及参考索引比特变化信息，将这些比特数变化与拉格朗日损失函数

组合起来，得到更新后的拉格朗日损失函数

For each 16x16 block in the current background to be encoded, the Lagrangian loss function

On the basis of the calculation results, calculate and match the loss function of the background; then combine each 16x16 block with the loss function of the matching background, and calculate the Lagrangian loss function

Compared with the minimum loss function of the existing reference frame, the smaller one is the minimum loss function of the corresponding 16x16 block; traverse each 16x16 block in the current background to be encoded, and accumulate the minimum loss of each 16x16 block. function to get the loss function of the current background to be encoded

At the same time, for the current background to be encoded, the change in the number of bits caused by the background includes the position index information and reference index bit change information of the matching background in the database, and these changes in the number of bits are combined with the Lagrange loss function

Combined to get the updated Lagrangian loss function

Take the calculation method of the number of bits of the reference index bit change information as an example to introduce:

As shown in Figure 5, for each 16x16 block in the current vehicle to be encoded and the background, when calculating its minimum loss function with the existing reference frame and matching vehicle and background, if the matching block corresponding to its minimum loss function If the index is n-1, the number of bits is increased by 1, where n is the number of existing reference frames; otherwise, if the matching block corresponding to its minimum loss function is on the matched vehicle or background, the number of bits is increased by n-1- idx, where idx is the matching block index corresponding to the 16x16 block minimum loss function when the matched vehicle and background are not considered. Other than that, the number of bits remains the same. Traverse the current vehicle to be encoded and each 16x16 block in the background, and the final reference index bit change information is the sum of the bit number of each 16x16 block change. Combining the change in the number of bits with the Lagrangian loss function calculated earlier, the updated Lagrangian loss function corresponding to the current vehicle to be encoded and the background is obtained.

与更新后的拉格朗日损失函数

之间的大小，若

则在匹配车辆上进行率失真优化处理；比较拉格朗日损失函数

与更新后的拉格朗日损失函数

之间的大小，若

则在匹配背景上进行率失真优化处理。Finally, compare the Lagrangian loss functions

with the updated Lagrangian loss function

size, if

Then perform rate-distortion optimization on the matched vehicle; compare the Lagrangian loss function

with the updated Lagrangian loss function

size, if

Then, rate-distortion optimization is performed on the matching background.

2. Comparison of RDO in intra prediction mode.

The comparison criterion for rate-distortion optimization in intra prediction mode is similar to that in inter prediction mode, which is also expressed as:

Among them, J is the Lagrangian loss function, D is the absolute difference sum of the prediction block and the matching block, R is the number of bits used to represent the pattern information, and λ is the Lagrangian multiplier.

a. For the current background to be encoded, in the intra-frame prediction mode, rate-distortion optimization is always performed on the matching background.

b. For the current vehicle to be encoded, first, roughly estimate the loss function when the current vehicle to be encoded adopts intra-frame prediction: divide the current vehicle to be encoded into blocks with a fixed size of 16x16, and for each 16x16 block, perform the mean value in turn Mode (DC), smooth mode (planar), horizontal and vertical intra prediction mode estimation, get the absolute difference sum of each 16x16 block corresponding to each mode; when estimating intra prediction mode, the reference of the current 16x16 block The pixel value is derived from the original value of the adjacent 16x16 block; for each 16x16 block, the absolute difference sum estimated in all modes is sorted in ascending order, and the result with the smallest absolute difference sum is used as the current 16x16. The optimal matching result of the block; traverse all 16x16 blocks in the current vehicle to be encoded, accumulate the optimal matching results of each 16x16 block, and obtain the Lagrangian loss function of the current vehicle to be encoded

计算结果的基础上，采用快速运动估计的方法，计算与匹配车辆的损失函数(绝对差和)；再将每一16x16的块与匹配车辆的损失函数，与计算拉格朗日损失函数

时得到的其帧内预测估计出的最小绝对差和比较，取较小者为相应16x16的块的最小损失函数；遍历当前待编码车辆内的每个16x16的块，累加每个16x16的块的最小损失函数，得到当前待编码车辆的损失函数

同时，对于当前待编码车辆，引起比特数的变化包含了匹配车辆在数据库中的位置索引信息、匹配车辆在参考帧中的位置信息和CTU级别的表示信息，将这些比特数变化与拉格朗日损失函数

组合起来，得到更新后的拉格朗日损失函数

Taking matching vehicles into account, compute the updated Lagrangian loss function: For each 16x16 block within the current vehicle to be encoded, the Lagrangian loss function

On the basis of the calculation results, the method of fast motion estimation is used to calculate and match the loss function (sum of absolute difference) of the vehicle; then each 16x16 block and the loss function of the matching vehicle are used to calculate the Lagrangian loss function.

The minimum absolute difference and comparison of the estimated intra-frame prediction obtained when , take the smaller one as the minimum loss function of the corresponding 16x16 block; traverse each 16x16 block in the current vehicle to be encoded, and accumulate the value of each 16x16 block. Minimum loss function to get the loss function of the current vehicle to be encoded

At the same time, for the current vehicle to be encoded, the changes in the number of bits caused by the position index information of the matching vehicle in the database, the position information of the matching vehicle in the reference frame, and the representation information of the CTU level. daily loss function

Combined to get the updated Lagrangian loss function

与更新后的拉格朗日损失函数

之间的大小，若

则在匹配车辆上进行率失真优化处理。Comparing Lagrangian Loss Functions

with the updated Lagrangian loss function

size, if

The rate-distortion optimization process is then performed on the matched vehicle.

2. Coding of vehicles and backgrounds.

1) When the inter-frame prediction mode is adopted, if it is determined that rate-distortion optimization processing is required on the matching vehicle or the matching background, a space for a new reference frame is newly applied, and the matching vehicle or matching background is pasted on the newly applied reference frame. The existing reference frames are used for inter-frame prediction of the current vehicle to be encoded or the background to be encoded; after the inter-frame prediction is over, traverse each 4x4 block covered by the current vehicle to be encoded or the current background to be encoded, if a certain 4x4 block With reference to the information of the current vehicle to be encoded or the current background to be encoded, the corresponding syntax element is encoded into the code stream;

2) When the intra-frame prediction mode is adopted, if it is determined that rate-distortion optimization processing is required on the matching vehicle or the matching background, a space for a new reference frame is newly applied, and the matching vehicle or matching background is pasted on the newly applied reference frame for reference. Intra-frame prediction is performed on the current vehicle to be encoded or the background to be encoded.

In the above two parts, the position where the matching vehicle is attached to the reference frame of the new application is determined by the following formula:

x ₀ =x _c +MV _x ;

y ₀ =y _c +MV _y ;

Among them, x ₀ and y ₀ represent the position where the matching vehicle is attached to the reference frame of the new application, x _c and y _c represent the position of the current vehicle to be encoded in the current frame, MV _x and MV _y are the relative position of the current vehicle to be encoded relative to the matching vehicle the horizontal and vertical components of the vehicle offset (obtained by the aforementioned fast motion estimation);

When the matching background is pasted on the reference frame, it can be aligned with the reference frame position.

3. Encoding code stream structure

In the embodiment of the present invention, the structure of the encoded code stream is divided into two layers: slice and tree coding unit (CTU); wherein:

Slice layer: For the current vehicle to be encoded, the slice layer contains a flag that indicates whether there is a matching vehicle in the current slice layer; traverse the 4x4 blocks covered by all vehicles in the current slice layer to determine whether it refers to a matching vehicle. , if there is a 4x4 block that references a matching vehicle, it is marked as true, otherwise it is marked as false; if it is marked as true, the slice layer also contains a syntax element indicating the number of referenced matching vehicles in the current slice layer; for each matching vehicles, their position index in the database, their position on the reference frame of the new application are encoded into the code stream together, the number of referenced matching vehicles, the index of each matching vehicle, and the labeling of each matching vehicle The position on the reference frame of the new application is coded using fixed-length coding;

For the current background to be coded, the slice layer contains a flag indicating whether there is a matching background in the current slice layer to be referenced; traverse all the 4x4 blocks covered by the background in the current slice layer to determine whether it refers to the matching background, if there is a certain 4x4 If the block refers to the matching background, it is marked as true, otherwise it is marked as false; if it is marked as true, the slice layer also contains the position index syntax element of the referenced matching background in the database, and the syntax element adopts the fixed-length encoding method. to encode;

CTU layer: For the current vehicle to be encoded, the CTU layer contains a flag indicating whether the current CTU layer refers to the matching vehicle pixel; traverse each 4x4 block in the current CTU layer, if there is a 4x4 block that refers to the matching vehicle pixel, If it is marked as true, otherwise it is marked as false; when it is marked as true, the CTU layer also contains a syntax element indicating the matching vehicle index (index);

For the current background to be encoded, the CTU layer contains a flag indicating whether the current CTU layer refers to matching background pixels.

On the other hand, in order to illustrate the coding performance of the above scheme of the present invention, relevant tests are also carried out.

The test conditions include: 1) Inter-frame configuration: random access is Random Access, RA; low-delay B is Low-delayB, LDB; low-delay P is Low-delay P, LDP. 2) The basic quantization step size (QP) is set to {27, 32, 37, 42}, the software based on it is HM16.7, and the test sequence is a 14-segment test sequence taken by myself, as shown in Figure 6. The experimental results are shown in Table 1 and Table 2.

Among them, Table 1 shows the performance comparison results under the RA, LDB, and LDP settings, and Table 2 shows the codec side complexity comparison results under the RA, LDB, and LDP settings.

Table 1. Performance comparison results under RA, LDB, and LDP settings

Table 2 Comparison results of codec complexity under RA, LDB and LDP settings

It can be seen from Table 1 to Table 2 that, compared with HM16.7, the above scheme in the embodiment of the present invention can obtain 35.1%, 31.3% and 28.8.0% of the code rate saving in RA, LDB and LDP modes respectively, and the coding The increase in complexity at the decoding end is within a reasonable range.

From the description of the above embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by software or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the above embodiments may be embodied in the form of software products, and the software products may be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.), including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this. Substitutions should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (9)

1. A traffic monitoring video coding method is characterized by comprising the following steps:

processing an original traffic monitoring video sequence by adopting a foreground and background segmentation method, separating out vehicles and a background, and respectively removing redundancy existing between the separated vehicles and the background and then putting the vehicles and the background into a database;

for the traffic monitoring video to be coded, a foreground and background segmentation method is also adopted to separate the vehicle to be coded and the background to be coded; selecting matched vehicles from a database by adopting a characteristic matching and rapid motion estimation mode for the vehicles to be coded; selecting a matched background from a database on the basis of the sum of absolute differences for the background to be coded;

when an inter-frame prediction mode or an intra-frame prediction mode is adopted, judging whether the vehicle to be coded or the background to be coded needs to perform rate distortion optimization processing on a matched vehicle or a matched background by using a preset mode; performing corresponding processing according to the judgment result, and encoding by using a corresponding prediction mode;

wherein, the processing the original traffic monitoring video sequence by adopting the front background segmentation method to separate the vehicle and the background, and respectively removing the redundancy existing between the separated vehicle and the background and then putting the vehicle and the background into the database comprises the following steps:

numbering vehicles from 1 to N after the redundancy is removed;

initially, the vehicle in the database is empty; for a certain vehicle v with redundancy removed_iRemoving vehicles v by using a method based on an inverted list_iSearching similar vehicles { v ] from all other vehicles_i1,v_i2,…,v_imWhere m is the number of similar vehicles;

comparing vehicles v when retrieving similar vehicles_iAnd any one of the remaining vehicles v_jNumber of SIFT features matched when vehicle v_iAnd a vehicle v_jWhen the matched SIFT feature number meets the following formula, the vehicle v is connected_jPut in { v_i1,v_i2,…,v_imIn the method, the following steps:

N_ij≥β×N_i；

N_ij≥min(N₀,N_i)；

in the above formula, N_ijFor vehicles v_iAnd a vehicle v_jNumber of matched SIFT features, N_iFor vehicles v_iNumber of SIFT features in (1), β and N₀Is a constant;

then, forAnd (3) comparing the similarity of the pixel levels by the vehicle: for vehicle v_iIf the vehicle in the database is empty, the vehicle v is put_iPutting the obtained product into a database; otherwise, the vehicle v is driven_iAnd { v_i1,v_i2,…,v_imComparing the pixel level similarity of vehicles which are put into the database, using a rapid motion estimation mode during similarity comparison, using a loss function as a sum of absolute differences, and judging that the two vehicles are similar at the pixel level if the calculated sum of absolute differences and the average value are smaller than a set value; if { v }_i1,v_i2,…,v_imThe vehicles already put into the database are not connected with the vehicle v continuously_iAt a pixel level, the vehicle v is similar_iPut into the database, otherwise, the vehicle v_iNot putting the data into a database; if the vehicle v is to be finally decided_iPut into the database, then { v }_i1,v_i2,…,v_imThe vehicles and the vehicle v in which the database has been put_iMaking a comparison of the pixel level similarity, if any, to the vehicle v_iRemoving similar vehicles which are put into the database from the database if the vehicles are similar in pixel level; if the number of vehicles and the number of vehicles v are more than the total number_iStopping comparing the pixel level similarity of the vehicles when the pixel levels are not similar;

processing each vehicle in the above manner, determining the vehicle finally put into the database, coding the vehicle and putting the vehicle into the database;

and for the background after removing the redundancy, taking a frame of background at intervals, coding the background and then putting the background into a database.

2. The traffic monitoring video coding method according to claim 1, wherein when a front background segmentation method is adopted to separate the vehicle from the background, pixels in a square region from the upper left corner to the lower right corner of the separated vehicle are used as the vehicle, and the rest part is used as the background;

for the separated current vehicle to be coded and the corresponding background, SIFT features of the vehicle to be coded and the corresponding background are respectively extracted, and for each SIFT feature extracted from the current vehicle to be coded, the following formula is adopted to search in a certain position neighborhood range on the corresponding background:

(xs_c-xs_b)²+(ys_c-ys_b)²≤d²；

wherein, xs_cAnd ys_cCoordinates, xs, representing SIFT features extracted from the vehicle currently to be coded_bAnd ys_bCoordinates representing SIFT features extracted from a corresponding background, and d is a defined range of a position neighborhood;

if the distance between the searched SIFT feature with the minimum Euclidean distance after normalization and a certain SIFT feature of the current vehicle to be coded is smaller than a threshold value, the fact that the SIFT feature similar to the SIFT feature of the current vehicle to be coded exists in the background region is indicated, the corresponding SIFT feature of the current vehicle to be coded is the background SIFT feature, and the SIFT feature is removed from the SIFT of the vehicle.

3. The traffic monitoring video coding method according to claim 1, wherein the selecting matched vehicles from the database by using the methods of feature matching and fast motion estimation for the vehicles to be coded comprises:

firstly, a plurality of candidate vehicles are roughly selected from a database in a characteristic matching mode: quantizing SIFT features of each vehicle in the database into visual words by adopting a k-means algorithm, and calculating a corresponding mapping mean vector for each visual word; mapping each SIFT feature of each vehicle in the database to the nearest neighbor visual character, and comparing the mapped SIFT feature vectors with the mapping mean vector corresponding to the nearest neighbor visual character to obtain the binarization representation of each SIFT feature vector; simultaneously, representing each vehicle in the database by using a frequency histogram of visual characters corresponding to SIFT features of the vehicle, and organizing the frequency histogram of each vehicle in the database in an inverted list mode; for the current vehicle to be coded, distributing each SIFT feature to the nearest visual characters according to the method for processing the vehicle in the database to obtain a frequency histogram of the current vehicle to be coded, and simultaneously calculating the binarization representation of each SIFT feature; when the similarity between a current vehicle to be coded and a certain vehicle in a database is compared, under the condition that the Hamming distance of the binarization representation of the SIFT features mapped to the same visual characters is smaller than a certain threshold value, the distance of a frequency histogram weighted by a tf-idf term is used as an evaluation index of the similarity, and the comparison result of the similarity between the current vehicle to be coded and each vehicle in the database is obtained; sorting according to the comparison result of the calculated similarity, and selecting a plurality of vehicles with the top similarity ranking as candidate vehicles;

then, a matching vehicle is selected from a plurality of candidate vehicles by using a rapid motion estimation mode: aligning a current vehicle to be coded with each candidate vehicle, dividing the current vehicle to be coded into blocks with the fixed size of 16x16, and searching a block with the minimum loss function in a certain candidate vehicle by each block with the size of 16x16, wherein the loss function consists of the sum of absolute differences and the coding code rate of a motion vector; the searching mode is that the position of the current 16x16 block is taken as a starting point, eight-point diamond type searching is carried out in the range of 64 pixels up, down, left and right around the starting point, and the loss functions of all the 16x16 blocks are accumulated to be used as the whole loss function of the whole current vehicle to be coded on a candidate vehicle; and finally, the candidate vehicle with the minimum overall loss function is reserved as the matching vehicle.

4. The traffic monitoring video coding method according to claim 3, wherein the alignment of the current vehicle to be coded with each candidate vehicle is performed as follows:

for a certain SIFT feature of the current vehicle to be coded, calculating the distance between the certain SIFT feature and all SIFT features of each candidate vehicle, sorting the calculated distances from small to large, and if the following formula is met, judging that the corresponding SIFT feature of the current vehicle to be coded finds a matched SIFT feature in the corresponding candidate vehicle:

d₁≤D₂；

d₁/d₂≤α；

wherein d is₁And d₂Respectively, a minimum and a second small distance, D₂And α are constants;

calculating each SIFT feature of the current vehicle to be coded according to the mode to obtain SIFT matching pairs of the current vehicle to be coded and each candidate vehicle; according to the obtained result of SIFT feature matching pairs, calculating the position offset of the current vehicle to be coded and each candidate vehicle, as shown in the following formula:

wherein, MV_xAnd MV_yFor the horizontal and vertical components of the offset, n is the number of matched SIFT feature pairs, xc_iAnd yc_iAs coordinates of the SIFT feature of the vehicle currently to be encoded, xv_iAnd yv_iCoordinates of SIFT features of the candidate vehicle; i is the serial number of SIFT feature matching pair;

removing abnormal points in an iteration mode to obtain a final position offset result; and aligning the current vehicle to be coded with the corresponding candidate vehicle according to the calculated position offset result.

5. The traffic monitoring video coding method according to claim 1, wherein selecting the matching context from the database based on the sum of absolute differences for the context to be coded comprises:

taking the absolute difference sum of the current background to be coded and the pixels at the corresponding positions of the background in the database as a similarity evaluation criterion, calculating the absolute difference sum of the current background to be coded and each background in the database, as shown in the following formula:

SAD＝∑_k∈B|pc_k-pl_k|；

wherein, pc_kAnd pl_kRespectively representing the current background to be coded and the k pixel value of the background in the database, wherein B is the set of the current background pixel to be coded;

and sorting the calculation results from small to large, and taking the background with the minimum absolute difference as the matching background of the current background to be coded.

6. The traffic monitoring video coding method according to claim 1, wherein when the inter-frame prediction mode is adopted, the determining whether the vehicle to be coded or the background to be coded needs to perform rate distortion optimization processing on the matching vehicle or the matching background by using a predetermined mode comprises:

the comparison criterion for rate-distortion optimization in the inter-prediction mode is as follows:

j is a Lagrange loss function, D is the sum of absolute differences of the prediction block and the matching block, R is the bit number used for representing mode information, and lambda is a Lagrange multiplier;

firstly, calculating Lagrange loss functions of a current vehicle to be encoded and a current background to be encoded and a current reference frame:

for each existing reference frame of the current vehicle to be coded, the motion vector of the block adopting inter-frame prediction 4 × 4 at the corresponding position of the current vehicle to be coded and the image number information of the reference frame are obtained by taking the block of 4 × 4 as a unit, and on the basis, the motion vector information of the block corresponding to 4 × 4 on the current vehicle to be coded is estimated, wherein the estimation formula is as follows:

wherein, MVX_refAnd MVY_refHorizontal and vertical components of the block motion vector, POC, of inter prediction 4 × 4 on an existing reference frame, respectively_cur、POC_refAnd POC_colrefThe image number of the frame where the current vehicle to be coded is located, the image number of the existing reference frame and the block reference frame of inter-frame prediction 4 × 4 on the existing reference frameThe image number of (1); MVX_curAnd MVY_curTraversing each 4x4 small block in the current vehicle to be coded, recording the number of blocks of inter-frame prediction 4 × and the motion vector of the corresponding block of the current vehicle to be coded 4 ×, and finally, taking the horizontal component and the vertical component of the finally estimated current vehicle to be coded as the average value of all the inter-frame prediction 4x4 small block motion vectors;

obtaining the displacement of the current vehicle to be coded on each existing reference frame, then dividing the current vehicle to be coded into blocks with the fixed size of 16x16, and sequentially searching the block with the minimum loss function in all the existing reference frames by each block with the size of 16x16, wherein the loss function consists of the sum of absolute differences and the coding code rate of a motion vector; the searching mode is that the position of the current 16x16 block after translation according to the estimated displacement is taken as a starting point, and eight-point diamond type searching is carried out in the range of 64 pixels around the starting point; recording the minimum loss function of all blocks in the current vehicle to be coded and the matching blocks of the blocks in all the existing reference frames by taking a 16x16 block as a unit; sequentially traversing each 16x16 block in the current vehicle to be coded, accumulating the blocks to obtain the minimum loss function sum, and obtaining the Lagrangian loss function of the current vehicle to be coded relative to the current reference frame

For the current background to be coded, dividing the current background into 16x16 blocks; for the current 16x16 block, searching a matching block corresponding to the minimum loss function from all the existing reference frames; the searching mode is that the absolute difference sum of the 16x16 blocks at the corresponding positions of all the reference frames and the current 16x16 block in the current background to be coded is compared, and the minimum absolute difference sum is selected as the loss function of the current 16x16 block in the current background to be coded; traversing all 16x16 blocks in the background to be coded currently, and accumulating the loss functions of all 16x16 blocks as Lagrangian loss functions of the background to be coded currently

Then, taking into account the matching vehicles and the background, an updated lagrangian loss function is calculated:

for each 16x16 block in the vehicle currently to be encoded, the lagrange loss function

On the basis of the calculation result, calculating a loss function of the vehicle and the matched vehicle by adopting a rapid motion estimation method; then, the loss function of each 16x16 block and the matched vehicle is compared with the calculated Lagrange loss function

Comparing the obtained minimum loss function with the minimum loss function of the existing reference frame, and taking the smaller one as the minimum loss function of the corresponding 16x16 block; traversing each 16x16 block in the current vehicle to be coded, and accumulating the minimum loss function of each 16x16 block to obtain the Lagrangian loss function of the current vehicle to be coded

Meanwhile, for the current vehicle to be coded, the change of the bit number comprises the position index information of the matched vehicle in the database, the position information of the matched vehicle in the reference frame, the reference index bit change information and the CTU-level representation information, and the change of the bit number and the Lagrangian loss function are combined

Combined to obtain an updated Lagrangian loss function

For each 16x16 block within the current context to be encoded, the lagrangian loss function

Calculating a loss function of the matched background on the basis of the calculation result; then, the loss function of each 16x16 block and the matched background is combined with the calculated Lagrangian loss function

Comparing the obtained minimum loss function with the minimum loss function of the existing reference frame, and taking the smaller one as the minimum loss function of the corresponding 16x16 block; traversing each 16x16 block in the current background to be coded, and accumulating the minimum loss function of each 16x16 block to obtain the loss function of the current background to be coded

Meanwhile, for the current background to be coded, the change of the bit number comprises the position index information and the reference index bit change information of the matched background in the database, and the bit number change and the Lagrangian loss function are carried out

Combined to obtain an updated Lagrangian loss function

Finally, the Lagrangian loss functions are compared

With updated lagrange loss function

The size between, if

Performing rate distortion optimization processing on the matched vehicle; comparison of Lagrange loss functions

With updated lagrange loss function

The size between, if

Then a rate distortion optimization process is performed on the matching background.

7. The traffic monitoring video coding method according to claim 1, wherein when the intra prediction mode is adopted, the determining whether the vehicle to be coded or the background to be coded needs to perform rate distortion optimization processing on the matching vehicle or the matching background by using a predetermined mode comprises:

the comparison criteria for rate-distortion optimization in intra prediction mode are:

j is a Lagrange loss function, D is the sum of absolute differences of the prediction block and the matching block, R is the bit number used for representing mode information, and lambda is a Lagrange multiplier;

for the current background to be coded, carrying out rate distortion optimization processing on the matched background all the time in an intra-frame prediction mode;

for the current vehicle to be coded, firstly, a loss function when the current vehicle to be coded adopts intra-frame prediction is roughly estimated: dividing a current vehicle to be coded into blocks with a fixed size of 16x16, and sequentially estimating DC, planar, horizontal and vertical intra-frame prediction modes for each block of 16x16 to obtain the sum of absolute differences of each block of 16x16 corresponding to each mode; in intra prediction mode estimation, the reference pixel values of the current 16x16 block are deduced from the original values of the neighboring 16x16 blocks; for each 16x16 block, sorting the sum of absolute differences estimated in all modes in order from small to large, and taking the result with the smallest sum of absolute differences as the optimal matching result of the current 16x16 block; go throughAccumulating the optimal matching result of each 16x16 block of all 16x16 blocks in the current vehicle to be coded to obtain the Lagrangian loss function of the current vehicle to be coded

Then, taking the matching vehicles into account, an updated lagrangian loss function is calculated: for each 16x16 block in the vehicle currently to be encoded, the lagrange loss function

On the basis of the calculation result, calculating and matching a loss function of the vehicle by adopting a rapid motion estimation method; then, the loss function of each 16x16 block and the matched vehicle is compared with the calculated Lagrange loss function

Comparing the obtained minimum absolute difference sum estimated by the intra-frame prediction, and taking the smaller one as the minimum loss function of the corresponding 16x16 block; traversing each 16x16 block in the current vehicle to be coded, and accumulating the minimum loss function of each 16x16 block to obtain the loss function of the current vehicle to be coded

Meanwhile, for the current vehicle to be coded, the change of the bit number comprises the position index information of the matched vehicle in the database, the position information of the matched vehicle in the reference frame and the CTU-level representation information, and the change of the bit number and the Lagrange loss function are combined

Combined to obtain an updated Lagrangian loss function

Finally, the Lagrangian loss functions are compared

With updated lagrange loss function

The size between, if

Then a rate-distortion optimization process is performed on the matching vehicle.

8. The traffic monitoring video coding method according to claim 1, 6 or 7, wherein the corresponding processing is performed according to the judgment result, the coding is performed by using the corresponding prediction mode, and the information of the matched vehicle or the matched background referred to in the coding is coded into the code stream together

When an inter-frame prediction mode is adopted, if the rate distortion optimization processing needs to be carried out on the matched vehicle or the matched background, a reference frame space is newly applied, and the matched vehicle or the matched background is attached to the newly applied reference frame and is used for inter-frame prediction of the current vehicle to be coded or the background to be coded together with the existing reference frame; after the interframe prediction is finished, traversing each 4x4 block covered by the current vehicle to be coded or the current background to be coded, and if a certain 4x4 block refers to the information of the current vehicle to be coded or the current background to be coded, coding a corresponding syntax element into a code stream;

when an intra-frame prediction mode is adopted, if the rate distortion optimization processing needs to be carried out on the matched vehicle or the matched background, a reference frame space is newly applied, and the matched vehicle or the matched background is attached to the newly applied reference frame for intra-frame prediction of the current vehicle to be coded or the background to be coded;

the location of the reference frame where the matching vehicle is attached to the new application is determined by:

x₀＝x_c+MV_x；

y₀＝y_c+MV_y；

wherein x is₀And y₀Indicating the location of the matching vehicle to attach to the newly applied reference frame, x_cAnd y_cIndicating the position, MV, of the current vehicle to be coded in the current frame_xAnd MV_yThe horizontal component and the vertical component of the offset of the current vehicle to be coded relative to the matched vehicle;

when the matched background is pasted on the reference frame, the matched background is aligned with the position of the reference frame.

9. The traffic monitoring video coding method according to claim 8, wherein the structure of the coded stream is divided into two layers, a slice and a tree coding unit CTU; wherein:

slice layer: for the current vehicle to be coded, the slice layer comprises a mark for indicating whether a matched vehicle is referenced in the current slice layer; traversing 4x4 blocks covered by all vehicles in the current slice layer, and judging whether the blocks refer to matched vehicles, if a certain 4x4 block refers to a matched vehicle, marking the block as true, otherwise, marking the block as false; if the mark is true, the slice layer also comprises a syntax element which represents the number of the referenced matched vehicles in the current slice layer; for each matched vehicle, the position index of the matched vehicle in the database and the position of the matched vehicle attached to the reference frame of the new application are coded into a code stream, and the number of the referenced matched vehicles, the index of each matched vehicle and the position of each matched vehicle attached to the reference frame of the new application are coded in a fixed-length coding mode;

for the current background to be coded, the slice layer comprises a mark for indicating whether the matching background is referred in the current slice layer; traversing all 4x4 blocks covered by the background in the current slice layer, and judging whether the blocks refer to a matching background, if a certain 4x4 block refers to the matching background, marking the block as true, otherwise, marking the block as false; if the mark is true, the slice layer also contains a position index syntax element of the referenced matching background in the database, and the syntax element is coded by adopting a fixed-length coding mode;

and (3) CTU layer: for the current vehicle to be coded, the CTU layer comprises a mark for indicating whether the current CTU layer refers to the matched vehicle pixel or not; traversing each 4x4 block in the current CTU layer, if there is some 4x4 block that references a matching vehicle pixel, then marking as true, otherwise marking as false; when the flag is true, the CTU layer further includes a syntax element indicating a matching vehicle index;

for the current background to be encoded, the CTU layer contains a flag indicating whether the current CTU layer references matching background pixels.

Images