CN104301735B - The overall situation coding method of urban transportation monitor video and system - Google Patents

Abstract

The invention discloses a method and system for global encoding of urban traffic monitoring video, comprising steps: step 1, dividing the original monitoring video into vehicle video and removing vehicle video; step 2, adopting an optional differential encoding method to remove the vehicle video Encoding; step 3, extracting the global feature parameter set of the global moving vehicle in the vehicle video; step 4, performing global encoding on the vehicle video based on the global feature parameters. The invention further removes the global redundancy in the monitoring video on the basis of removing the scene redundancy, and effectively improves the encoding and compression efficiency of the urban traffic monitoring video.

Description

Global encoding method and system for urban traffic surveillance video

technical field

The invention belongs to the technical field of urban traffic monitoring video coding, and in particular relates to a global coding method and system for urban traffic monitoring video.

Background technique

The goal of video signal compression coding technology is to represent video information with as few bits as possible under the premise of ensuring a certain reconstruction quality. The traditional video coding method based on Shannon information theory starts from the signal processing level, uses pixels and blocks as the basis, adopts a hybrid coding framework that combines transformation, prediction, and entropy coding, and improves compression performance by mining the spatiotemporal redundancy of the image and video signal itself. However, most of the current video compression technologies are oriented to non-specific applications. In recent years, video compression technologies developed for the characteristics and needs of specialized applications (such as surveillance video) have become a research direction that has attracted much attention, such as surveillance in urban traffic environments. Video coding and transmission technology. AVS-S2 aims at the long-term unchanged characteristics of surveillance video scenes, by modeling the surveillance background and foreground, and selectively using the original mode and differential mode to encode each block, removing a large number of existing "scene redundancy". The efficiency is twice that of H.264/AVC, which is the first international standard for video surveillance. However, AVS-S2 cannot remove the "global redundancy" caused by global object motion, the compression efficiency is limited, and the contradiction between data volume and storage capacity is still very prominent

Different models of vehicles in the surveillance video have similarity in video texture characteristics, vehicles of the same model have the identity of 3D objects, and the same vehicle has long-term stability of appearance characteristics. All kinds of urban operating vehicles with similar, identical, and long-term stability are repeatedly captured by surveillance cameras all over the city, resulting in a large amount of urban surveillance data redundancy. Most of the urban monitoring points are under-covered, and the data generated by the movement of vehicles and people constitutes the main source of urban monitoring data. The redundancy of video monitoring data generated by the same moving vehicle being repeatedly recorded by a large number of surveillance cameras in the city is called global redundancy. There are texture similarities between different moving objects, shape consistency between the same semantic objects, and long-term similarities between specific objects, resulting in a large number of global redundancy of moving objects. Traditional video coding and scene redundancy removal technologies remove local spatio-temporal redundancy, while the global redundancy in surveillance video due to repeated long-term recording of vehicles by cameras provides a huge space for further improvement of video compression efficiency.

Contents of the invention

Aiming at the deficiencies in the prior art, the present invention provides a method and system for global encoding of urban traffic monitoring video in consideration of global redundancy, and the method can further improve the encoding efficiency of urban traffic monitoring video.

In order to solve the problems of the technologies described above, the present invention adopts the following technical solutions:

(1) a global encoding method for urban traffic monitoring video, comprising steps:

Step 1, the original monitoring video is divided into vehicle video and video of removing vehicle;

Step 2, using an optional differential encoding method to encode the video with the vehicle removed;

Step 3, extracting the global feature parameter set of the global moving vehicle in the vehicle video, further including:

S31 extracts the 2D appearance feature of the global moving vehicle;

S32 builds a vehicle 3D model database, the vehicle 3D model database includes general 3D models of various vehicles, fine 3D models and model key description parameter sets, and the model key description parameters are obtained by dimensionality reduction of the model description parameter set;

S33 uses sparse coding to build a global vehicle texture dictionary for global moving vehicles, further including:

by As the cost function, based on the texture information of the global moving vehicle, the first layer of knowledge dictionary is obtained, that is, the common visual texture information knowledge dictionary of all types of global moving vehicles;

Obtain the difference information r _c between all kinds of global moving vehicles reconstructed by the first-level knowledge dictionary and the original global computing vehicles, to As the cost function, based on the difference information r _c , the second-level knowledge dictionary is obtained, that is, the knowledge dictionary of the three-dimensional structure and texture individual information of various global moving vehicles;

Obtain the difference information r _c,m between each individual global moving vehicle reconstructed by the second-layer knowledge dictionary and the original global moving vehicle under various global moving vehicles, to As the cost function, based on the difference information r _c,m , the third layer of knowledge dictionary is obtained, that is, the knowledge dictionary of the personality long-term update information of individual global moving vehicles;

As mentioned above, D ₁ represents the first layer of knowledge dictionary; C represents the number of types of global moving vehicles, c represents the type number of global moving vehicles; y _c represents the texture information of all types of global moving vehicles; a ₁ represents the coding coefficient; τ is the balance factor, According to the actual situation and empirical setting, the larger τ is, the sparser the coding coefficient is; Represents the second-level knowledge dictionary, M represents the number of individual vehicles under a certain type of global motion vehicle, m represents the number of individual vehicles under a certain type of global motion vehicle, a _{2, c} represent coding coefficients; Represents the third layer of knowledge dictionary, N represents the number of individual vehicles under this type of global moving vehicle, i represents the number of individual vehicles under this type of global moving vehicle; a _{3, c, m} represent coding coefficients;

S34 matches the 2D appearance features of the global moving vehicle with the model in the vehicle 3D model database, and obtains the texture of the global moving vehicle and key description parameter information of the model;

S35 extracting the position information of the global moving vehicle according to the 2D appearance feature of the global moving vehicle, and combining the attitude information in the corresponding model key description parameter information to form the position and attitude parameters of the global moving vehicle;

S36 performs lossless compression on the coding coefficients corresponding to the global feature parameter set and the three-level knowledge dictionary, and the global feature parameter set is composed of the texture and model key description parameter information obtained in step S34 and the position and attitude parameters obtained in step S35;

Step 4: Globally encode the vehicle video based on the global feature parameters.

In step 1, background modeling and vehicle detection techniques are used to segment the original surveillance video into vehicle video and vehicle-removed video, including:

S11 converts the original surveillance video image to YUV space, and establishes an automatically updated background model based on the background difference method;

S12 utilizes the vehicle detection method to detect the vehicle in the original surveillance video image, and obtain the vehicle video image;

S13 subtracting the vehicle video image from the original monitoring video image to obtain a video image containing background holes and removing the vehicle;

S14 uses the background model to superimpose and fill the background hole in the video image obtained in S13 to obtain the video image without the vehicle.

Step 2 further includes sub-steps:

S21 generates a background image according to the video image from which the vehicle is removed, and reconstructs the background image after encoding;

S22 performs global motion estimation on the video image from which the vehicle is removed, and obtains a global motion vector;

S23, based on the reconstructed background image and the global motion vector, selectively use the original coding mode or the differential coding mode to code each video block.

Substep S32 further includes:

(1) Construct a general 3D model of a vehicle based on a grid structure;

(2) Obtain a fine 3D model of the vehicle;

(3) obtaining a 3D model description parameter set according to the general 3D model of the vehicle;

(4) Dimensionality reduction is performed on the parameters in the 3D model description parameter set to obtain key description parameters.

Substep S35 further includes:

(1) Determine the position and angle parameters of the global moving vehicle ρ=[x, y, θ] ^T , x, y are the vertical projection coordinates of the center of the global moving vehicle in the world coordinate system, θ is the main moving direction of the global moving vehicle and OX the included angle of the axis;

(2) extracting the motion region in the vehicle video by background modeling;

(3) Using the sparse optical flow method to obtain the two-dimensional motion vector of the global moving vehicle;

(4) Obtain the main motion direction θ and velocity v of the global moving vehicle in the world coordinate system;

(5) Iteratively match the two-dimensional projection of the general 3D model of the global moving vehicle with the size and shape of the moving area, and obtain the position parameters (x, y) of the global moving vehicle in the world coordinate system.

Step 4 further includes sub-steps:

S41 Perform motion estimation and motion compensation on the global moving vehicle based on the global vehicle characteristic parameters, to obtain residual parameter information;

S42 acquires the illumination compensation parameters of the global moving vehicle;

S43 fuses the residual parameter information and the illumination compensation parameter, and performs lossless coding on the residual parameter information and the illumination compensation parameter.

(2) A global coding system for urban traffic monitoring video, comprising:

(1) Video segmentation module, used for dividing original monitoring video into vehicle video and removing the video of vehicle;

(2) An optional differential encoding module, used to encode the video in which the vehicle is removed in an optional differential encoding manner;

(3) The global feature parameter extraction module is used to extract the global feature parameter set of the global moving vehicle in the vehicle video, and this module further includes submodules:

The 2D appearance feature extraction module is used to extract the 2D appearance features of the global moving vehicle;

The vehicle 3D model database building module is used to construct the vehicle 3D model database. The vehicle 3D model database includes general 3D models, fine 3D models and key description parameter sets of various vehicles. The key description parameters of the model are obtained by reducing the dimension of the model description parameter set ;

The global vehicle texture dictionary construction module is used to construct the global vehicle texture dictionary of the global moving vehicle by means of sparse coding, further including:

by As the cost function, based on the texture information of the global moving vehicle, the first layer of knowledge dictionary is obtained, that is, the common visual texture information knowledge dictionary of all types of global moving vehicles;

Obtain the difference information r _c between all kinds of global moving vehicles reconstructed by the first-level knowledge dictionary and the original global computing vehicles, to As the cost function, based on the difference information r _c , the second-level knowledge dictionary is obtained, that is, the knowledge dictionary of the three-dimensional structure and texture individual information of various global moving vehicles;

Obtain the difference information r _c,m between each individual global moving vehicle reconstructed by the second-layer knowledge dictionary and the original global moving vehicle under various global moving vehicles, to As the cost function, based on the difference information r _c,m , the third layer of knowledge dictionary is obtained, that is, the knowledge dictionary of the personality long-term update information of individual global moving vehicles;

As mentioned above, D ₁ represents the first layer of knowledge dictionary; C represents the number of types of global moving vehicles, c represents the type number of global moving vehicles; y _c represents the texture information of all types of global moving vehicles; a ₁ represents the coding coefficient; τ is the balance factor, According to the actual situation and empirical setting, the larger τ is, the sparser the coding coefficient is; Represents the second-level knowledge dictionary, M represents the number of individual vehicles under a certain type of global motion vehicle, m represents the number of individual vehicles under a certain type of global motion vehicle, a _{2, c} represent coding coefficients; Represents the third layer of knowledge dictionary, N represents the number of individual vehicles under this type of global moving vehicle, i represents the number of individual vehicles under this type of global moving vehicle; a _{3, c, m} represent coding coefficients;

The texture and model key description parameter information acquisition module is used to match the 2D appearance features of the global moving vehicle with the model in the vehicle 3D model database, and obtain the texture and model key description parameter information of the global moving vehicle;

The position and attitude parameter acquisition module is used to extract the position information of the global moving vehicle according to the 2D appearance characteristics of the global moving vehicle, and combine the attitude information in the corresponding model key description parameter information to form the position and attitude parameters of the global moving vehicle;

The lossless compression module is used to perform lossless compression on the coding coefficients corresponding to the global feature parameter set and the three-level knowledge dictionary. The global feature parameter set is composed of the texture and model key description parameter information obtained in step S34 and the position and position obtained in step S35. Composition of attitude parameters;

(4) A global encoding module, used to globally encode the vehicle video based on the global feature parameters.

The above-mentioned position and attitude parameter acquisition module further includes:

The position and angle parameter determination module is used to determine the position and angle parameters of the global moving vehicle ρ=[x,y,θ] ^T , where x and y are the vertical projection coordinates of the center of the global moving vehicle in the world coordinate system, and θ is the global moving The angle between the main motion direction of the vehicle and the OX axis;

The motion area extraction module is used to extract the motion area in the vehicle video through background modeling;

The two-dimensional motion vector acquisition module is used to obtain the two-dimensional motion vector of the global moving vehicle by using the sparse optical flow method;

The main motion direction and speed acquisition module is used to obtain the main motion direction θ and speed v of the global moving vehicle in the world coordinate system;

The position parameter acquisition module is used to iteratively match the two-dimensional projection of the universal 3D model matched with the global moving vehicle with the size and shape of the moving area, and obtain the position parameters (x, y) of the global moving vehicle in the world coordinate system.

The above-mentioned global encoding module further includes:

The residual parameter information obtaining module is used to perform motion estimation and motion compensation on the global moving vehicle based on the global vehicle characteristic parameters to obtain the residual parameter information;

The illumination compensation parameter acquisition module is used to acquire the illumination compensation parameters of the global moving vehicle;

The lossless encoding module is used to fuse the residual parameter information and the illumination compensation parameter, and perform lossless encoding on the residual parameter information and the illumination compensation parameter.

The present invention is based on the mechanism of global object redundancy generation in urban monitoring video, and utilizes the large proportion of vehicle information in the monitoring video, the characteristics of strong vehicle structure, similar appearance, and rich texture, and uses vehicle detection technology to segment the original video to generate vehicles. The video and the video after removing the vehicle are encoded separately in different ways. For the vehicle video, the vehicle knowledge dictionary is established through sparse coding technology, etc., and the global feature parameter set is extracted. Since the encoding only describes the texture, attitude and other characteristics of the moving vehicle, the video data of the global moving vehicle is transformed into only containing a small amount of information. The feature description data of the moving vehicle effectively removes the global redundancy of the moving vehicle; while the video after removing the vehicle (including background images and other moving objects) is encoded based on AVS-S2 optional differential coding. The present invention further removes the global redundancy in the monitoring video on the basis of removing the scene redundancy, and effectively improves the encoding and compression efficiency.

Description of drawings

Fig. 1 is the concrete flowchart of the inventive method;

Fig. 2 is the specific flowchart of video segmentation;

Fig. 3 is the concrete flowchart of vehicle detection method;

Fig. 4 is the specific flow chart of extracting the global feature parameter;

Fig. 5 is a schematic diagram of vehicle 2D appearance features, wherein, Fig. a is a 3D model diagram of a vehicle, and Fig. b is a sampling of a vehicle 2D template;

Fig. 6 is a schematic diagram of a general 3D model of a vehicle and a fine 3D model of a vehicle, wherein Fig. a is a general 3D model of a vehicle, Fig. b is a texture schematic diagram of a general 3D model of a vehicle, and Fig. c is a schematic diagram of a fine 3D model of a vehicle;

Fig. 7 is a schematic diagram describing the attitude position and space angle parameters of a moving vehicle;

Fig. 8 is a schematic diagram of vehicle position and attitude parameter extraction;

FIG. 9 is a schematic diagram of a global encoding process of a vehicle video.

detailed description

In order to make the purpose, technical features and advantages of the present invention more obvious and comprehensible, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

The monitoring scene in urban traffic surveillance video is relatively fixed, the vehicle information accounts for a large proportion, and vehicle movement produces a large amount of global redundancy. Based on the above-mentioned characteristics of urban traffic monitoring video and the characteristics of strong vehicle structure, similar appearance and rich texture, the present invention provides a global encoding method and system for urban traffic monitoring video to encode urban traffic monitoring video to remove Scene redundancy and global redundancy.

Firstly, the present invention divides the original monitoring video into the vehicle video and the vehicle-removed video through the video segmentation technology. Then, for the video where the vehicle is removed, the optional differential coding method based on AVS-S2 is used to remove the scene redundancy in the video. Next, for the vehicle video, use sparse coding technology to create a vehicle knowledge dictionary, and then generate a global feature parameter set including the vehicle's position and attitude information as well as texture and parameter information, and encode the global feature parameters globally; The texture, pose and other features of the vehicle are described, and the video data of the global moving vehicle is transformed into feature description data containing only a small amount of information, which effectively removes the global redundancy of the moving vehicle, thereby further improving the coding efficiency.

Fig. 1 is the concrete flowchart of the inventive method, with reference to Fig. 1, the specific steps of the inventive method are as follows:

Step 1, video segmentation: segment the original surveillance video into vehicle video and remove vehicle video.

Video segmentation can be realized by background modeling and vehicle detection technology, and the original surveillance video is divided into two parts: the vehicle video and the video without the vehicle.

See Figure 2, this step is carried out for the video image, and further includes sub-steps:

S11 converts the original surveillance video image to YUV space, and establishes an automatically updated background model based on the background difference method.

In this specific implementation, the background model is established using the ViBe method (visual background extraction method), but the modeling method of the background model is not limited to the ViBe method.

S12 Use the vehicle detection method to detect the vehicle in the original surveillance video image, and obtain the vehicle video image.

The specific implementation process of this step is shown in Figure 3, including steps:

(1) Gaussian filtering is performed on the original surveillance video image, and the background model is used to detect the moving area. Gaussian filtering is used to eliminate Gaussian noise in the image, improve image quality, and further ensure the correctness of subsequent video processing.

(2) Select training samples, extract the SIFT feature (scale-invariant feature transformation) of the training samples, and use the Adaboost classifier to train the vehicle detector; the training samples are a series of original surveillance video images.

(3) Use the trained vehicle detector to classify the SIFT features of the moving area. If the proportion of the SIFT features belonging to the vehicle in the moving area to the total SIFT features is greater than the threshold R, it is judged that the moving area is a vehicle; otherwise, it is not vehicle area. The threshold R is set empirically.

Preferably, after the motion area is acquired, the motion shadow of the motion area needs to be removed.

S13 subtracts the vehicle video image from the original surveillance video image to obtain a video image of the vehicle with background holes removed.

S14 uses the background model to superimpose and fill the background hole in the video image obtained in S13 to obtain the video image without the vehicle.

Step 2, use optional differential coding to process the video that removes the vehicle.

The optional differential coding method is a conventional technology in the field of video coding technology, that is, on the basis of the traditional hybrid coding standard scheme (such as H.264), the predictive coding technology based on the background frame is added, and two kinds of coding, the background reference prediction and the differential prediction, are extended. predictive mode. When the module to be coded is a background block, the background reference prediction is used to make the residual smaller; if the block to be coded is a foreground-background mixed block, the differential prediction mode is used, that is, the foreground part after the background is cut off is used for prediction; while the pure The foreground block continues to use the traditional nearest neighbor prediction mode. In principle, the foreground-background mixing block can also adopt the traditional nearest neighbor prediction mode or differential prediction mode.

In this specific implementation, an optional differential encoding method based on AVS-S2 is used to encode the video without the vehicle. The optional differential coding method based on AVS-S2 aims at the long-term invariable characteristics of the surveillance video scene, and removes a large number of "scene redundancy" by modeling the surveillance background and foreground. The coding efficiency is H.264/AVC coding way twice. The optional differential coding method based on AVS-S2 For the macroblocks of each P frame, in addition to using the existing coding method, you can also selectively use the "difference result between the latest reference frame and the background image" to compare the "current macroblock with its Corresponding background difference result" for predictive coding.

This step encodes the video with the vehicle removed, and further includes the following sub-steps:

S21 background modeling:

The background image is generated by using video image modeling, and the background image is reconstructed after encoding.

S22 global motion estimation:

The global motion estimation with pixel or sub-pixel precision is performed on the video image without the vehicle, and the global motion vector is obtained.

S23 encoding mode selection:

Each video block is selectively encoded using an original encoding mode or a differential encoding mode based on the reconstructed background image and the global motion vector. For the background block, the residual error is smaller through the background reference prediction; for the foreground and background mixed block, the differential prediction mode is used to predict the foreground part after subtracting the background; for the pure foreground block, the traditional neighbor prediction mode is continued.

Step 3, extract the global feature parameters of the global moving vehicle in the vehicle video.

See Figure 4, this step further includes sub-steps:

S31 Extracting 2D appearance features of a global moving vehicle based on the vehicle video.

Extract the 2D appearance features of each global moving vehicle in the vehicle video obtained in step 1. The specific implementation method is as follows: For a certain type of vehicle, pre-calculate its 2D image contours under different viewpoints. For example, for a certain car, it is quantized into 72 segments in the range of 360-degree vehicle direction, and 19 segments in the range of 90-degree elevation angle, and a total of 1368 2D shape templates are given. Fig. 5 is a schematic diagram of 2D appearance features of a vehicle. Among them, Figure a is the 3D model of the car; Figure b is the sampling of the 2D template of the car in Figure a, the first to third lines represent the 2D templates when the camera elevation angle is 0 degrees, 15 degrees and 30 degrees respectively, the first to the fourth The columns are the 2D templates when the vehicle orientation is 0°, 30°, 90° and 120°, respectively.

S32 Establishment of vehicle 3D model database.

When building a vehicle 3D model database, classify vehicles according to vehicle make and model.

Establish the general 3D model and fine 3D model of the vehicle, that is, constitute the vehicle 3D model database. The vehicle 3D model is composed of 5 main parts: the main body of the vehicle body and 4 wheels, which are specifically constructed using a CAD model. The vehicle 3D model is composed of a grid structure, and stores the coordinates of each grid vertex and the grid surface index. Components such as windows and lights play an important role in describing vehicle characteristics and distinguishing vehicle models due to their high identifiability and distinguishability. Such components are called key components of vehicles. The key components are represented in different levels of detail in the model.

The specific implementation of this step is as follows:

(1) Establish a general 3D model of the vehicle

The quadrilateral grid is used to represent the general 3D model of the vehicle, as shown in Figure 6(a), which is the general 3D model of the Audi Q7 model. The quadrilateral grid is highly generalized, and the boundary of the quadrilateral grid does not completely coincide with the boundary of the key components of the vehicle. Therefore, the key components of the vehicle are represented by two-dimensional closed lines attached to the model. The contour lines of all key components are shown on the texture schematic in Fig. 6(a), see Fig. 6(b).

(2) Obtain the fine 3D model of the vehicle

Before each type of vehicle leaves the factory, its CAD-based fine 3D model already exists, which can be downloaded from the relevant website. In the fine 3D model, in order to improve the recognizability of the vehicle model, the key components of the vehicle are not only represented by contour lines, but also retain the appearance characteristics of each component. Figure 6(c) shows the fine mesh 3D model including the Audi Q7 car model and its key components.

(3) Match the general 3D model of the vehicle with the global moving vehicle in the vehicle video based on the general 3D model parameters.

Obtain a general 3D model description parameter set based on the general 3D model, realize the matching between the general 3D model of the vehicle and the global moving vehicle in the vehicle video based on the description parameters, and realize the general 3D model of the vehicle and the global moving vehicle in the vehicle video by adjusting the description parameters of the general 3D model The best fit of the vehicle's general 3D model and the global motion vehicle's matching belongs to the conventional technology in this technical field. In this specific implementation, the general 3D model description parameters include 30 parameters such as the wheelbase of the vehicle, the width of the front, and the height of the hood.

Through the matching of the general 3D model of the vehicle and the global moving vehicle, that is, when the video image is restored, the general 3D vehicle model matched with the global moving vehicle is placed in the real position corresponding to the global moving vehicle in the video.

(4) Dimensionality reduction of general 3D model description parameters

Principal component analysis (PCA) was used to reduce the dimensionality of 30 general 3D model description parameters to obtain key description parameters. The number of parameters is small, the calculation is simple, and the adaptability to noise and low quality is high; but the number of parameters is large, the model has a high degree of expression of vehicle details, and a high degree of matching with the actual vehicle, so a balance needs to be struck between the two. Leotta obtained through experiments that the first six PCA principal components can better express the vehicle model and effectively reduce the amount of calculation. Therefore, the general 3D model description parameters after dimensionality reduction, that is, the key description parameters p=[p ₁ , p ₂ , p ₃ , p ₄ , p ₅ , p ₆ ] ^T .

Construction of S33 Global Vehicle Texture Dictionary

The purpose of constructing the global vehicle texture dictionary is to reconstruct the texture of the general 3D vehicle model according to the global vehicle texture dictionary after the general 3D vehicle model matching the global moving vehicle is placed in the corresponding real position in the video.

According to the global vehicle extraction and recognition results, a global vehicle texture dictionary is constructed for each global moving vehicle in the vehicle video by means of sparse coding, and the global vehicle texture dictionary and the vehicle 3D model database jointly constitute a vehicle knowledge dictionary.

In this step, the common visual texture information knowledge base of global vehicles is constructed first; then, the three-dimensional structure and texture individual information knowledge dictionary of various types of vehicles is constructed; finally, the individual vehicle personality long-term update information knowledge dictionary is constructed. When encoding, only by describing the texture, attitude and other features of the moving vehicle, the video data of the global moving vehicle is transformed into feature description data containing only a small amount of information, effectively removing the global redundancy of the moving vehicle, and further improving the coding efficiency.

The specific implementation of the three-layer knowledge dictionary construction of this step is as follows:

(1) The first layer of knowledge dictionary construction, that is, the construction of the common visual texture information knowledge dictionary of all types of global moving vehicles.

The common visual texture information knowledge dictionary of all types of moving vehicles is constructed by sparse coding, and the cost function is as follows:

Among them, D ₁ represents the first layer of knowledge dictionary; C represents the number of types of global moving vehicles, c represents the type number of global moving vehicles; y _c represents the common visual texture information of global moving vehicles, that is, the texture information of all types of global moving vehicles ; a ₁ represents the coding coefficient; τ is the balance factor, which is set according to the actual situation and experience, the larger the τ, the sparser the coding coefficient.

The cost function (1) is used to calculate the sparsity, prevent over-fitting through the constraint item, and also reduce the non-zero elements in the coding coefficient.

(2) The second layer of knowledge dictionary construction, that is, the construction of the three-dimensional structure and texture individual information knowledge dictionary of various types of global moving vehicles.

Construct the three-dimensional structure and texture individual information knowledge dictionary of various types of global moving vehicles by means of sparse coding, specifically:

First, extract the difference information r _c between various global moving vehicles reconstructed by the first layer of knowledge dictionary and the original global moving vehicles:

Secondly, the second-layer knowledge dictionary is constructed for each type of global moving vehicle, and the cost function is as follows:

in, Represents the second-level knowledge dictionary, M represents the number of individual vehicles under a certain type of global motion vehicle, m represents the number of individual vehicles under a certain type of global motion vehicle, a _{2, c} represent the coding coefficients.

(3) The third layer of knowledge dictionary construction, that is, the long-term update information knowledge dictionary of individual global moving vehicles.

Construct the long-term update information knowledge dictionary of the personality of individual moving vehicles through sparse coding, specifically:

First, extract the difference information r _c,m of each individual vehicle under various global moving vehicles reconstructed by the second-level knowledge dictionary and the original global running vehicle:

Secondly, the third-layer knowledge dictionary is constructed for individual vehicles in various types of global moving vehicles, and the cost function is as follows:

in, Represents the third-level knowledge dictionary, N represents the number of individual vehicles under this type of global moving object, i represents the number of individual vehicles under this type of moving object; a _{3, c, m} represent encoding coefficients.

Vehicle video coding is handled by a global vehicle feature representation based on a three-layer knowledge dictionary. On the one hand, since the three-layer knowledge dictionary changes slowly over time, the video data of the global moving vehicle can be transformed into feature description information containing only a small amount of information, and the feature description information only contains a small amount of information of the video data of the global moving object; on the other hand, In urban public security applications, since the position of the monitoring camera is relatively fixed and the background information is relatively fixed, the frequency of background information transmission can be reduced. For example, background frame information is transmitted once every 100 frames. Then, only a small amount of features need to be transmitted for each frame of background information. Descriptive information can be reconstructed by using a three-layer knowledge dictionary at the decoding end, thereby greatly improving the coding efficiency of video big data.

S34 acquires texture and model parameter information of the global moving vehicle.

The 2D appearance features of the global moving vehicle are matched with the vehicle 3D model to obtain the texture and model parameter information of the global moving vehicle.

The specific steps of feature matching are as follows:

(1) Based on the vehicle 3D model database, pre-generate and establish the index of the 2D orthogonal appearance mask after all viewpoints of the vehicle are quantified (see step S31);

(2) Evaluate selected parameters (type, inclination, direction) to match the foreground model for the detected rectangular area of the vehicle through area-based matching and contour matching.

S35 extracts the position information of the global moving vehicle according to the 2D appearance feature of the global moving vehicle, and describes the position and attitude parameters of the global moving vehicle in combination with the corresponding vehicle attitude information in the vehicle 3D model.

This step further includes sub-steps:

(1) Extract the feature information of each global moving vehicle in the vehicle video.

The feature information F _i of the i-th global moving vehicle includes six parameters of spatial position (x, y, z) and attitude angle (α, β, γ), as shown in Figure 8:

F _i ＝[x _i ,y _i , _zi ,α _i ,β _i ,γ _i ] (6)

The first step is to determine the parameters:

Assuming that the static surveillance camera satisfies the principle of perspective projection, the calibration of the camera and the ground plane parameters of the shooting area are all realized offline preprocessing. Usually, the target pose is described by three position parameters (x, y, z) and three angle parameters (α, β, γ). However, in the vehicle running scenario, it can be considered that the vehicle mainly runs along the ground plane. Using the ground plane constraints, the vehicle attitude parameter ρ can be reduced to three:

ρ=[x,y,θ] ^T (6)

Among them, x and y are the vertical projection coordinates of the vehicle center on the world coordinate system (WCS) with the ground plane as the XOY plane, and θ is the angle between the main motion direction of the vehicle and the OX axis.

The second step is to obtain the attitude parameters of the global moving vehicle:

This step mainly includes the initialization of vehicle attitude parameters based on the optical flow method and the update of vehicle attitude parameters based on predictive tracking. First, based on the 3D model of the vehicle, the motion area is extracted through background modeling; then, the two-dimensional motion vector of the vehicle is calculated using the sparse optical flow method, and the main motion direction θ and velocity v of the vehicle in the world coordinate system are obtained by combining the camera calibration results; Finally, the two-dimensional projection of the vehicle 3D model is iteratively matched with the size and shape of the motion area to obtain the position parameters (x, y) of the vehicle in the world coordinate system.

(2) Extract the index information of primitives in the three-level knowledge dictionary when reconstructing the global moving vehicle, that is, the coding coefficient vector ID _i =[a ₁ ,a _2,c ,a _3.cm ] in the dictionary.

(3) During encoding and transmission, lossless compression is performed on the feature information F _i and the encoding coefficient vector ID _i . Specifically, entropy encoding can be used for lossless compression, which can effectively protect the key information of the global moving vehicle.

The position and attitude parameters of the vehicle obtained in step S35 and the texture and model parameter information obtained in step S34 together constitute a global vehicle characteristic parameter set, and lossless compression is performed on the global vehicle characteristic parameter set.

Step 4: Globally encode the vehicle video based on the global vehicle feature parameters.

The specific process of this step is shown in Figure 9, and the specific steps are as follows:

S41 Perform motion estimation and motion compensation on the global moving vehicle based on the global feature parameters to obtain residual parameter information.

Using multi-frame reference images, 1/4 or 1/8 pixel precision motion estimation method and variable size block motion compensation method, specifically based on MPEG-1/2/4, H.263, H.264/AVC, A motion estimation method and a motion compensation method of H.265/HEVC or AVS, but not limited thereto.

S42 Acquire the illumination compensation parameters of the globally moving vehicle.

Since the lighting change factor has a great influence on the encoding and restoration effect of the vehicle video, the following method is used in this embodiment to determine the lighting compensation parameters:

(1) Establish color-invariant features based on the UV components of the YUV color space, and obtain various vehicle samples from vehicle videos to form a sample set.

(2) Randomly sample N sample points from the sample set, N is usually set to 1/4 of the number of sample sets, preferably N>50.

(3) Obtain illumination compensation parameters based on sample points.

The sample points can be obtained by screening the color features, and can also be screened by other single or combined features, such as gradient features, wavelet features, etc. Any other method of obtaining illumination compensation parameters through image feature screening results should be included in the protection of the present invention. within range.

S43 fuses the residual parameter information and the illumination compensation parameter, and transmits the residual parameter information and the illumination compensation parameter to the decoding end by using entropy coding, so as to restore the video image.

The residual parameter information and the illumination compensation parameter are fused, and entropy coding is used to perform lossless coding on the fused residual parameter information and illumination compensation parameter. Entropy coding can adopt traditional variable-length coding and algorithmic coding, implementations include but not limited to CAVLC (context-based adaptive variable-length coding), CABAC (context-based adaptive binary arithmetic entropy coding), C2DVLC (context-based self-adaptive Adaptive two-dimensional variable length coding) and CBAC (context-based binary algorithm coding).

The present invention utilizes the characteristics of fixed scenes and a large amount of global redundancy in urban surveillance video, and compresses video vehicle objects and remaining parts to eliminate redundancy in video sequences to a greater extent and obtain better compression performance.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the technical field of the present invention can make various modifications or supplements to the described specific embodiments or adopt similar methods to replace them, but they will not deviate from the spirit of the present invention or go beyond the scope defined in the appended claims .

Claims (7)

1. A global coding method for urban traffic monitoring videos is characterized by comprising the following steps:

step 1, dividing an original monitoring video into a vehicle video and a video for removing a vehicle;

step 2, coding the video without the vehicle by adopting an optional differential coding mode;

step 3, extracting a global feature parameter set of a global motion vehicle in the vehicle video, further comprising:

s31, extracting 2D appearance characteristics of the global motion vehicle;

s32, a vehicle 3D model database is built, the vehicle 3D model database comprises general 3D models, fine 3D models and model key description parameter sets of various vehicles, and the model key description parameters are obtained by dimension reduction of the model description parameter sets;

s33, constructing a global vehicle texture dictionary of the global moving vehicle by adopting a sparse coding mode, and further comprising the following steps:

to be provided withObtaining a first-layer knowledge dictionary, namely a common visual texture information knowledge dictionary of all classes of global motion vehicles, based on the texture information of the global motion vehicles as a cost function;

obtaining difference information r between various global motion vehicles and original global operation vehicles after the global motion vehicles are reconstructed by a first-layer knowledge dictionary_cTo do so byAs a cost function, based on the difference information r_cAcquiring a second-layer knowledge dictionary, namely a three-dimensional structure and texture individual information knowledge dictionary of various global motion vehicles;

obtaining difference information r between each individual global motion vehicle under various global motion vehicles and the original global motion vehicle after reconstruction of the second-layer knowledge dictionary_c,mTo do so byAs a cost function, based on the difference information r_c,mObtaining a third-layer knowledge dictionary, namely an individual long-time updated information knowledge dictionary of the individual global motion vehicle;

above, D₁Representing a first level knowledge dictionary; c represents the type number of the global motion vehicle, and C represents the type number of the global motion vehicle; y is_cRepresenting all types of global motion vehicle texture information; a is₁Representing the encoded coefficients; tau is a balance factor, and the larger tau is, the more sparse the coding coefficient is;representing a second-level knowledge dictionary, M representing the number of individual vehicles under a certain type of global motion vehicle, M representing the number of individual vehicles under a certain type of global motion vehicle, a_2,cRepresenting the encoded coefficients;representing a third-layer knowledge dictionary, N representing the number of individual vehicles under the type of global motion vehicle, and i representing the number of individual vehicles under the type of global motion vehicle; a is_3,c,mRepresenting the encoded coefficients;

s34, carrying out feature matching on the 2D appearance features of the global motion vehicle and the models in the vehicle 3D model database to obtain the texture of the global motion vehicle and the key description parameter information of the models;

s35, extracting global motion vehicle position information according to the 2D appearance characteristics of the global motion vehicle, and combining the attitude information in the corresponding model key description parameter information to form the position and attitude parameters of the global motion vehicle;

s36, lossless compression is carried out on the coding coefficients corresponding to the global feature parameter set and the three-level knowledge dictionary, wherein the global feature parameter set is composed of the texture and model key description parameter information obtained in the step S34 and the position and posture parameters obtained in the step S35;

step 4, carrying out global coding on the vehicle video based on the global characteristic parameters;

step 4 further comprises the sub-steps of:

s41, motion estimation and motion compensation are carried out on the global motion vehicle based on the global vehicle characteristic parameters, and residual error parameter information is obtained;

s42, acquiring illumination compensation parameters of the global motion vehicle;

s43 fuses the residual parameter information and the illumination compensation parameter, and losslessly encodes the residual parameter information and the illumination compensation parameter.

2. The global encoding method for urban traffic monitoring video according to claim 1, characterized in that:

in the step 1, the original monitoring video is divided into the vehicle video and the video for removing the vehicle by adopting the background modeling and vehicle detection technology, and the method specifically comprises the following steps:

s11, converting the original monitoring video image into YUV space, and establishing an automatically updated background model based on a background subtraction method;

s12, detecting the vehicle in the original monitoring video image by using a vehicle detection method to obtain a vehicle video image;

s13, subtracting the vehicle video image from the original monitoring video image to obtain a vehicle-removed video image containing a background hole;

and S14, overlaying and filling the background hole in the video image obtained in S13 by adopting the background model, and obtaining the video image without the vehicle.

3. The global encoding method for urban traffic monitoring video according to claim 1, characterized in that:

step 2 further comprises the sub-steps of:

s21, generating a background image according to the video image without the vehicle, and reconstructing the background image after encoding;

s22, carrying out global motion estimation on the video image without the vehicle to obtain a global motion vector;

s23 encodes each video block selectively using the original encoding mode or the differential encoding mode based on the reconstructed background image and the global motion vector.

4. The global encoding method for urban traffic monitoring video according to claim 1, characterized in that:

the sub-step S32 further includes:

(1) constructing a vehicle universal 3D model based on a grid structure;

(2) obtaining a fine 3D model of the vehicle;

(3) obtaining a 3D model description parameter set according to the vehicle general 3D model;

(4) and the key description parameters of the model are obtained by dimension reduction of the model description parameter set.

5. The global encoding method for urban traffic monitoring video according to claim 1, characterized in that:

the sub-step S35 further includes:

(1) determining a global moving vehicle's position and angle parameter ρ ═ x, y, θ]^TX and y are vertical projection coordinates of the center of the global motion vehicle in a world coordinate system, and theta is an included angle between the main motion direction of the global motion vehicle and an OX axis;

(2) extracting a motion area in the vehicle video through background modeling;

(3) obtaining a two-dimensional motion vector of the global motion vehicle by using a sparse optical flow method;

(4) obtaining a main motion direction theta and a speed v of the global motion vehicle in a world coordinate system;

(5) and iteratively matching the two-dimensional projection of the universal 3D model matched with the global moving vehicle with the size and the shape of the moving area to obtain the position parameters (x, y) of the global moving vehicle under a world coordinate system.

6. A global coding system for urban traffic monitoring video is characterized by comprising:

(1) the video segmentation module is used for segmenting the original monitoring video into a vehicle video and a video for removing the vehicle;

(2) the optional differential coding module is used for coding the video without the vehicle by adopting an optional differential coding mode;

(3) the global characteristic parameter extraction module is used for extracting a global characteristic parameter set of a global motion vehicle in the vehicle video, and further comprises sub-modules:

the 2D appearance characteristic extraction module is used for extracting 2D appearance characteristics of the global motion vehicle;

the vehicle 3D model database construction module is used for constructing a vehicle 3D model database, the vehicle 3D model database comprises general 3D models, fine 3D models and model key description parameter sets of various vehicles, and the model key description parameters are obtained by reducing the dimensions of the model description parameter sets;

the global vehicle texture dictionary building module is used for building a global vehicle texture dictionary of a global moving vehicle in a sparse coding mode, and further comprises:

to be provided withObtaining a first-layer knowledge dictionary, namely a common visual texture information knowledge dictionary of all classes of global motion vehicles, based on the texture information of the global motion vehicles as a cost function;

obtaining difference information r between various global motion vehicles and original global operation vehicles after the global motion vehicles are reconstructed by a first-layer knowledge dictionary_cTo do so byAs a cost function, based on the difference information r_cAcquiring a second-layer knowledge dictionary, namely a three-dimensional structure and texture individual information knowledge dictionary of various global motion vehicles;

obtaining difference information r between each individual global motion vehicle under various global motion vehicles and the original global motion vehicle after reconstruction of the second-layer knowledge dictionary_c,mTo do so byAs a cost function, based on the difference information r_c,mObtaining a third-layer knowledge dictionary, namely an individual long-time updated information knowledge dictionary of the individual global motion vehicle;

above, D₁Representing a first level knowledge dictionary; c represents the type number of the global motion vehicle, and C represents the type number of the global motion vehicle; y is_cRepresenting all types of global motion vehicle texture information; a is₁Representing the encoded coefficients; tau is a balance factor, and the larger tau is, the more sparse the coding coefficient is;representing a second-level knowledge dictionary, M representing the number of individual vehicles under a certain type of global motion vehicle, M representing the number of individual vehicles under a certain type of global motion vehicle, a_2,cRepresenting the encoded coefficients;representing a third-layer knowledge dictionary, N representing the number of individual vehicles under the type of global motion vehicle, and i representing the number of individual vehicles under the type of global motion vehicle; a is_3,c,mRepresenting the encoded coefficients;

the texture and model key description parameter information acquisition module is used for carrying out feature matching on the 2D appearance features of the global motion vehicle and the models in the vehicle 3D model database to acquire the texture and model key description parameter information of the global motion vehicle;

the position and attitude parameter acquisition module is used for extracting global moving vehicle position information according to the 2D appearance characteristics of the global moving vehicle and combining attitude information in the corresponding model key description parameter information to form position and attitude parameters of the global moving vehicle;

the lossless compression module is used for carrying out lossless compression on the coding coefficients corresponding to the global feature parameter set and the three-level knowledge dictionary, wherein the global feature parameter set is composed of the texture and model key description parameter information obtained in the step S34 and the position and posture parameters obtained in the step S35;

(4) the global coding module is used for carrying out global coding on the vehicle video based on the global characteristic parameters;

the global encoding module further comprises sub-modules:

the residual parameter information acquisition module is used for carrying out motion estimation and motion compensation on the global motion vehicle based on the global vehicle characteristic parameters to obtain residual parameter information;

the illumination compensation parameter acquisition module is used for acquiring illumination compensation parameters of the global motion vehicle;

and the lossless coding module is used for fusing the residual parameter information and the illumination compensation parameter and carrying out lossless coding on the residual parameter information and the illumination compensation parameter.

7. The global urban traffic monitoring video coding system according to claim 6, wherein:

the position and posture parameter obtaining module further comprises:

a position and angle parameter determination module to determine a position and angle parameter ρ ═ x, y, θ for the globally moving vehicle]^TX and y are vertical projection coordinates of the center of the global motion vehicle in a world coordinate system, and theta is an included angle between the main motion direction of the global motion vehicle and an OX axis;

the motion region extraction module is used for extracting a motion region in the vehicle video through background modeling;

the two-dimensional motion vector obtaining module is used for obtaining a two-dimensional motion vector of the global motion vehicle by utilizing a sparse optical flow method;

the main motion direction and speed obtaining module is used for obtaining a main motion direction theta and a speed v of the global motion vehicle in a world coordinate system;

and the position parameter obtaining module is used for iteratively matching the two-dimensional projection of the universal 3D model matched with the global moving vehicle with the size and the shape of the moving area to obtain the position parameters (x, y) of the global moving vehicle under a world coordinate system.