CN102982311B - Based on automobile video frequency Feature Extraction System and the method for video structural description - Google Patents

Abstract

The invention relates to a vehicle video feature extraction system and method based on video structured description, and belongs to the technical field of video processing. The system includes a video stream encoding parser, a vehicle underlying feature extractor, a vehicle video standard feature extractor, a license plate and car logo segmenter, a license plate and car logo recognizer, and a visual task support server. In this method, firstly, the vehicle features in the traffic monitoring video can be extracted by using the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and vehicle logo segmenter, and the license plate and vehicle logo recognizer, and then The visual task support server uses the timestamp as a unified identifier to store the information output by other modules to realize the dynamic association of information; and assembles the stored information, generates a message and outputs it, so as to facilitate the subsequent video retrieval and processing. Retrieval and positioning can realize real-time intelligent supervision of motor vehicle behavior in road management.

Description

Vehicle Video Feature Extraction System and Method Based on Video Structured Description

technical field

The present invention relates to the technical field of video processing, in particular to the technical field of video feature extraction, in particular to a vehicle video feature extraction system and method based on video structured description.

Background technique

In recent years, the large-scale popularization of automobiles and the increase of urban vehicles have brought enormous pressure to urban traffic. While bringing convenience to people's lives, it also leads to more serious problems such as road congestion and traffic accidents. Various vehicle violations are also common, and the casualties caused by traffic accidents have remained high year after year.

In order to deal with the above problems, it has become one of the important contents of traffic management to collect statistics on the traffic flow of key roads and expressways, monitor and collect traffic violations such as speeding and overloading, as well as road condition information. The establishment of an automatic monitoring system for vehicle management can significantly reduce the labor intensity of traffic police, and can enhance the ability to deal with emergencies, deter a small number of drivers with illegal driving habits, and effectively guarantee traffic safety. The key technology of the automatic vehicle monitoring system is to extract the vehicle features from the video data.

In the prior art, Wang Haiyan proposed a video-based vehicle detection and tracking system in her invention patent application "Video-based Vehicle Detection and Tracking System" [CN200710163606.7]. According to the vehicle's binary projection model parameters, from The vehicle is identified in the video, and the MAD algorithm is used to realize vehicle tracking. In their invention patent application "Vehicle Tracking Method and System Based on Intelligent Video Recognition Technology" [CN200810037793.9], Lin Weici proposed a vehicle tracking system based on video recognition technology, which extracts vehicle templates from two consecutive frames of video Carry out matching discrimination, when the matching is successful, control the rotation of the camera so that the vehicle is always at the center of the video image. In their invention patent "a vehicle detection method based on video image features" [CN200710704558.9], Gao Lei et al. proposed a vehicle detection method based on video image features, using a panoramic digital camera for multi-lane video surveillance image acquisition , through the vehicle detection algorithm based on the characteristics of the video image, analyze each frame of digital video image acquired by the panoramic digital camera, and locate the position and time of appearance of the vehicle. In her master thesis "Research on Vehicle Detection and Tracking Technology in Surveillance Video" [Nanjing University of Posts and Telecommunications. 2012], Yang Mengjie used the CodeBook method under the framework of the background difference method to detect the target foreground, and realized it in the real environment by improving the particle resampling method. Real-time vehicle tracking of variable-sized single targets. In her master thesis "Research on Vehicle Detection and Tracking Technology in Surveillance Video" [Nanjing University of Posts and Telecommunications. 2012], Liu Hanyan studied two sets of tracking and counting algorithms during the day and night, and developed a video-based vehicle detection and tracking system. Algorithm switching is performed according to the average brightness information of the video frame and the system time information.

Although the above-mentioned systems and methods in the prior art explore and apply vehicle detection and tracking methods, they do not involve a practical traffic supervision system with complete functions, and cannot realize the association of vehicle attributes in video with video frames. When using video data, it still needs to consume a lot of manpower for video retrieval and comparison, which limits the extended application of the traffic supervision system.

Contents of the invention

The purpose of the present invention is to overcome the above-mentioned shortcoming in the prior art, provide a kind of efficient stage-by-stage extraction of vehicle features in the traffic monitoring video, and establish the time-space correlation between vehicle features and video source data, which is convenient for subsequent video retrieval and retrieval. Rapid retrieval and positioning during processing, and then realize real-time intelligent supervision of motor vehicle behavior in road management, and a vehicle video feature extraction system and method based on video structured description with simple structure, simple implementation method, and low application cost.

In order to achieve the above object, the vehicle video feature extraction system based on video structured description of the present invention has the following components:

The system includes a video stream encoding parser, a vehicle underlying feature extractor, a vehicle video standard feature extractor, a license plate and car logo segmenter, a license plate and car logo recognizer, and a visual task support server.

Wherein, the video stream coder is used to obtain the video stream from the connected external video capture device, transcode the video stream according to the preset video format and add a time stamp, and output the video stream containing the time stamp Transcoded video stream; the vehicle bottom layer feature extractor is connected to the output end of the video stream encoding parser, in order to obtain the vehicle bottom layer feature information from the transcoded video stream, and output; the vehicle video standard feature extractor Connect the output end of the vehicle bottom layer feature extractor, in order to obtain the vehicle video standard feature information according to the vehicle bottom layer feature information, and output; the license plate and car logo segmenter connect the output of the vehicle bottom layer feature extractor The end is used to segment and obtain the license plate and car logo image of the vehicle according to the vehicle bottom layer feature information, and output it; License plate and car logo image recognition to obtain the license plate and car logo information of the vehicle, and output; and the visual task support server is respectively connected to the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter and license plate and car logo recognizer, in order to coordinate and control the described video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter and license plate and car Operation of the mark recognizer; Utilize the time stamp as a unified identification to store the described video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter and license plate and car logo recognition The information output by the device realizes the dynamic association of the output information; and assembles the stored information to generate a message for network communication transmission.

In the vehicle video feature extraction system based on video structured description, the video stream encoding parser is a video stream encoding parser that provides support for video-on-demand functions.

In the vehicle video feature extraction system based on video structured description, the vehicle underlying feature extractor includes: video stream analysis unit, camera depth of field calibration and vehicle speed calculation unit, intersection photography and vehicle ROI (regionofinterest, video sensitive area) information Extract unit. Wherein, the video stream analysis unit is connected to the output end of the video stream encoding parser for background modeling and foreground extraction; the camera depth of field calibration and vehicle speed calculation unit is connected to the video stream analysis unit for background and foreground extraction. The depth of field is calibrated in the foreground to realize the calculation of the vehicle speed; the intersection camera and vehicle ROI information extraction unit is connected to the camera depth of field calibration and vehicle speed calculation unit to control the intersection camera to take pictures, extract vehicle ROI information, and perform traffic flow statistics.

In the vehicle video feature extraction system based on video structured description, the vehicle underlying feature information includes vehicle partial picture information, vehicle coordinate position information in the image frame, vehicle type information, body color information, driving direction information and violation type information.

In the vehicle video feature extraction system based on video structured description, the vehicle video standard feature extractor is a vehicle MPEG-7 feature extractor, and the vehicle video standard feature information is vehicle MPEG-7 feature information. The vehicle MPEG-7 feature information includes: color layout information, color space information, color structure information, main color information, edge histogram information, scalable color information, texture browsing information, GoFGoP color information and color quantization information.

In the vehicle video feature extraction system based on video structured description, the license plate and vehicle logo segmenter includes: a license plate and vehicle logo positioning unit and a license plate and vehicle logo image conversion unit. The license plate and car logo location unit is connected to the output end of the described vehicle bottom layer feature extractor, in order to determine the license plate position and the car logo position according to the described vehicle bottom layer feature information, obtain the license plate and the car logo image; the license plate and the car logo image The transformation unit is connected to the license plate and vehicle logo positioning unit, performs affine transformation according to the license plate and vehicle logo images, obtains a corrected standard view, and outputs it.

In the vehicle video feature extraction system based on video structured description, the license plate and vehicle logo recognizer includes: a license plate recognition unit and a vehicle logo recognition unit. Wherein, the license plate recognition unit is connected to the output end of the license plate and the car logo splitter, in order to divide the characters of the license plate image, identify the divided characters, obtain the vehicle license plate information, and output; the car logo recognition unit is connected to the The output end of the license plate and car logo splitter is used to match the car logo image with the data in the car logo image library to obtain the vehicle logo information and output it.

In the vehicle video feature extraction system based on video structured description, the visual task support server includes: a hybrid information storage pool, a network communication controller, a message parsing distributor, a support logic controller and a hybrid data assembler . Wherein, the hybrid information storage pool is respectively connected to the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer to utilize the described The timestamp is used as a unified identification to store the information output by the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter, and license plate and car logo recognizer; the network communication controller uses To connect the Ethernet to realize the sending and receiving of network messages; the message analysis distributor is connected to the output end of the network communication controller to analyze the control messages obtained from the network communication controller, and after analysis, obtain control information or query information; the support logic controller is connected to the output end of the message parsing distributor, so as to implement the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate according to the control information And the coordination and control of the car logo splitter and the license plate and car logo recognizer, and output data assembly instructions according to the query information; the hybrid data assembler is respectively connected to the hybrid information storage pool and the output end of the support logic controller , and connected to the input end of the network communication controller, to obtain corresponding information from the mixed-structure information storage pool according to the data assembly instruction, and assemble and generate a message for network communication transmission via the The network communication controller described above sends to Ethernet.

In the vehicle video feature extraction system based on video structured description, the network communication controller is connected to an external semantic analysis system, streaming media server and multi-function debugger through Ethernet.

The present invention also provides a method for realizing vehicle video feature extraction based on the video structured description using the system, the method comprising the following steps:

(1) The video stream encoding parser obtains the video stream from the connected external video acquisition device, transcodes the video stream according to the preset video format and adds a time stamp, and sends the video stream to the vehicle The underlying feature extractor and the visual task support server output the transcoded video stream containing the time stamp;

(2) The vehicle bottom layer feature extractor obtains the vehicle bottom layer feature information from the transcoded video stream, and provides the vehicle video standard feature extractor, the license plate and car logo segmenter, and the visual task support server Outputting the vehicle bottom layer feature information;

(3) The vehicle video standard feature extractor acquires vehicle video standard feature information according to the vehicle underlying feature information, and outputs the vehicle video standard feature information to the visual task support server;

(4) The license plate and logo splitter obtains the license plate and logo images of the vehicle according to the underlying feature information of the vehicle, and outputs them to the license plate and logo recognizer and the visual task support server The said license plate and logo image;

(5) The license plate and logo recognizer acquires the license plate and logo information of the vehicle according to the license plate and logo image recognition, and outputs the license plate and logo information to the visual task support server;

(6) The visual task support server uses the time stamp as a unified identifier to store the video stream encoding parser, vehicle underlying feature extractor, vehicle video standard feature extractor, license plate and vehicle logo splitter, and license plate and the information output by the vehicle logo recognizer, and dynamically associate it; and assemble the stored information according to the query information obtained through the external Ethernet, generate a message that can be transmitted by network communication, and then output it through the Ethernet.

In the method for realizing vehicle video feature extraction based on video structured description, the vehicle bottom layer feature extractor includes a video stream analysis unit, camera depth of field calibration and vehicle speed calculation unit, intersection photographing and vehicle ROI information extraction unit, the steps (2) Specifically include the following steps:

(21) The video stream analysis unit performs background modeling and foreground extraction according to the transcoded video stream;

(22) The camera depth of field calibration and vehicle speed calculation unit calculates the vehicle speed based on the background and foreground calibration depth of field;

(23) The intersection photographing and vehicle ROI information extraction unit controls the intersection camera to photograph, extract vehicle ROI information, and perform traffic flow statistics.

In the method for realizing vehicle video feature extraction based on video structured description, the license plate and car logo segmenter includes: a license plate and car logo positioning unit and a license plate and car logo image conversion unit, and the step (4) includes the following step:

(41) The license plate and vehicle logo positioning unit determines the position of the license plate and the vehicle logo according to the underlying characteristic information of the vehicle, and acquires the images of the license plate and the vehicle logo;

(42) The license plate and logo image conversion unit performs affine transformation according to the license plate and logo images to obtain a corrected standard view, and outputs it to the license plate and logo recognizer and the Vision task support server.

In the method for extracting vehicle video features based on video structured description, the license plate and logo recognizer includes: a license plate recognition unit and a car logo recognition unit, and the step (5) specifically includes the following steps:

(51) The license plate recognition unit divides the license plate image into characters, recognizes the divided characters, obtains the vehicle license plate information, and outputs it to the visual task support server;

(52) The vehicle logo recognition unit matches the car logo image with the data in the car logo image library, obtains the vehicle logo information, and outputs it to the visual task support server.

In the method for extracting vehicle video features based on video structured description, the visual task support server includes: a hybrid information storage pool, a network communication controller, a message analysis distributor, a support logic controller, and a hybrid data assembly device, the step (6) specifically includes the following steps:

(61) The hybrid information storage pool uses the timestamp as a unified identifier to store the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter and The information output by the license plate and logo recognizer, and dynamic information association;

(62) The network communication controller receives the network message through Ethernet;

(63) The message analysis distributor analyzes the control message obtained from the network communication controller to obtain control information or query information;

(64) According to the control information, the supporting logic controller implements the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer coordination and control, and output data assembly instructions to the hybrid data assembler according to the query information;

(65) The hybrid data assembler obtains corresponding information from the hybrid information storage pool according to the data assembly instruction, assembles and generates a message for network communication transmission, and then passes through the network Communication controller sends to Ethernet.

The vehicle video feature extraction system based on the video structured description of the invention is adopted, which includes a video stream encoding parser, a vehicle bottom layer feature extractor, a vehicle video standard feature extractor, a license plate and car logo splitter, and a license plate and car logo recognition server and vision task support server. In the vehicle video feature extraction method using this system, firstly, the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer can be used to realize traffic surveillance video. Efficient stage-by-stage extraction of vehicle features, and then the visual task support server uses the time stamp as a unified identifier to store the video stream encoding parser, vehicle underlying feature extractor, vehicle video standard feature extractor, license plate and vehicle logo The information output by the segmenter and the license plate and logo recognizer is dynamically correlated based on the time stamp; and the stored information is assembled according to the query information obtained through the external Ethernet to generate a message for network communication transmission. Through the Ethernet output, it is convenient for quick retrieval and positioning in subsequent video access and processing, and then can realize real-time intelligent supervision of motor vehicle behavior in road management, and the vehicle video feature extraction system based on video structured description of the present invention And the method, the system structure is simple, the method realization process is simple, the realization cost is low, and the application range is relatively wide.

Description of drawings

FIG. 1 is a schematic structural diagram of a vehicle video feature extraction system based on video structured description according to the present invention.

FIG. 2 is a flow chart of the steps of the method for extracting vehicle video features based on video structured description in the present invention.

FIG. 3 is a schematic structural diagram of a visual task support server in the vehicle video feature extraction system based on video structured description of the present invention.

detailed description

In order to understand the technical aspects of the present invention more clearly, the following examples are given in detail.

Please refer to FIG. 1 , which is a schematic structural diagram of a vehicle video feature extraction system based on video structured description according to the present invention.

In one embodiment, the system includes a video stream code parser, a vehicle bottom layer feature extractor, a vehicle video standard feature extractor, a license plate and logo segmenter, a license plate and logo recognizer, and a visual task support server.

Wherein, the video stream encoding parser is a video stream encoding parser that supports the video on demand function, and is used to obtain the video stream from the connected external video capture device, and transcode and add the video stream according to the preset video format. Timestamp, and output the transcoded video stream containing the timestamp; the vehicle bottom layer feature extractor is connected to the output end of the video stream encoding parser, in order to obtain the vehicle from the transcoded video stream Bottom layer feature information, and output; Vehicle video standard feature extractor connects the output end of described vehicle bottom layer feature extractor, in order to obtain vehicle video standard feature information according to described vehicle bottom layer feature information, and output; License plate and car logo The splitter is connected to the output end of the vehicle bottom layer feature extractor, in order to segment and obtain the license plate and the car logo image of the vehicle according to the described vehicle bottom layer feature information, and output; The output terminal of the car logo splitter is used to obtain the license plate and car logo information of the vehicle according to the license plate and car logo image recognition, and output; and the visual task support server is respectively connected to the video stream encoding parser, vehicle The underlying feature extractor, the vehicle video standard feature extractor, the license plate and vehicle logo segmenter and the license plate and vehicle logo recognizer are used to coordinate and control the video stream encoding parser, the vehicle bottom layer feature extractor, and the vehicle video standard feature Operation of the extractor, license plate and car logo splitter and license plate and car logo recognizer; using the time stamp as a unified identification to store the described video stream encoding parser, vehicle bottom layer feature extractor, and vehicle video standard feature extractor , the license plate and logo splitter and the information output by the license plate and logo recognizer to realize the dynamic association of the output information; and assemble the stored information to generate a message for network communication transmission.

Utilize the system described in this embodiment to realize the method for vehicle video feature extraction based on video structured description, as shown in Figure 2, comprising the following steps:

(1) The video stream encoding parser obtains the video stream from the connected external video acquisition device, transcodes the video stream according to the preset video format and adds a time stamp, and sends the video stream to the vehicle The underlying feature extractor and the visual task support server output the transcoded video stream containing the time stamp;

(2) The vehicle bottom layer feature extractor obtains the vehicle bottom layer feature information from the transcoded video stream, and provides the vehicle video standard feature extractor, the license plate and car logo segmenter, and the visual task support server Outputting the vehicle bottom layer feature information;

(3) The vehicle video standard feature extractor acquires vehicle video standard feature information according to the vehicle underlying feature information, and outputs the vehicle video standard feature information to the visual task support server;

(4) The license plate and logo splitter obtains the license plate and logo images of the vehicle according to the underlying feature information of the vehicle, and outputs them to the license plate and logo recognizer and the visual task support server The said license plate and logo image;

(5) The license plate and logo recognizer acquires the license plate and logo information of the vehicle according to the license plate and logo image recognition, and outputs the license plate and logo information to the visual task support server;

(6) The visual task support server uses the time stamp as a unified identifier to store the video stream encoding parser, vehicle underlying feature extractor, vehicle video standard feature extractor, license plate and vehicle logo splitter, and license plate and the information output by the vehicle logo recognizer, and dynamically associate it; and assemble the stored information according to the query information obtained through the external Ethernet, generate a message that can be transmitted by network communication, and then output it through the Ethernet.

In a more preferred embodiment, the vehicle bottom layer feature extractor includes: a video stream analysis unit, a camera depth of field calibration and vehicle speed calculation unit, and an intersection photographing and vehicle ROI information extraction unit. Wherein, the video stream analysis unit is connected to the output end of the video stream encoding parser for background modeling and foreground extraction; the camera depth of field calibration and vehicle speed calculation unit is connected to the video stream analysis unit for background and foreground extraction. The depth of field is calibrated in the foreground to realize the calculation of the vehicle speed; the intersection camera and vehicle ROI information extraction unit is connected to the camera depth of field calibration and vehicle speed calculation unit to control the intersection camera to take pictures, extract vehicle ROI information, and perform traffic flow statistics.

Using this preferred embodiment, in the method for extracting vehicle video features based on video structured description, the step (2) specifically includes the following steps:

(21) The video stream analysis unit performs background modeling and foreground extraction according to the transcoded video stream;

(22) The camera depth of field calibration and vehicle speed calculation unit calculates the vehicle speed based on the background and foreground calibration depth of field;

(23) The intersection photographing and vehicle ROI information extraction unit controls the intersection camera to photograph, extract vehicle ROI information, and perform traffic flow statistics.

In another preferred embodiment, the vehicle bottom-level feature information includes vehicle partial picture information, vehicle coordinate position information in the image frame, vehicle type information, body color information, driving direction information, and violation type information. The vehicle video standard feature extractor is a vehicle MPEG-7 feature extractor, and the vehicle video standard feature information is vehicle MPEG-7 feature information. The vehicle MPEG-7 feature information includes: color layout information, color space information, color structure information, main color information, edge histogram information, scalable color information, texture browsing information, GoFGoP color information and color quantization information.

In another preferred embodiment, the license plate and logo segmenter includes: a license plate and logo positioning unit and a license plate and logo image conversion unit. The license plate and car logo location unit is connected to the output end of the described vehicle bottom layer feature extractor, in order to determine the license plate position and the car logo position according to the described vehicle bottom layer feature information, obtain the license plate and the car logo image; the license plate and the car logo image The transformation unit is connected to the license plate and vehicle logo positioning unit, performs affine transformation according to the license plate and vehicle logo images, obtains a corrected standard view, and outputs it.

Using this preferred embodiment, in the method for extracting vehicle video features based on video structured description, the step (4) includes the following steps:

(41) The license plate and vehicle logo positioning unit determines the position of the license plate and the vehicle logo according to the underlying characteristic information of the vehicle, and acquires the images of the license plate and the vehicle logo;

(42) The license plate and logo image conversion unit performs affine transformation according to the license plate and logo images to obtain a corrected standard view, and outputs it to the license plate and logo recognizer and the Vision task support server.

In a further preferred embodiment, the license plate and logo recognizer includes: a license plate recognition unit and a car logo recognition unit. Wherein, the license plate recognition unit is connected to the output end of the license plate and the car logo splitter, in order to divide the characters of the license plate image, identify the divided characters, obtain the vehicle license plate information, and output; the car logo recognition unit is connected to the The output end of the license plate and car logo splitter is used to match the car logo image with the data in the car logo image library to obtain the vehicle logo information and output it.

Using this further preferred embodiment, in the method for extracting vehicle video features based on video structured description, the step (5) specifically includes the following steps:

(51) The license plate recognition unit divides the license plate image into characters, recognizes the divided characters, obtains the vehicle license plate information, and outputs it to the visual task support server;

(52) The vehicle logo recognition unit matches the car logo image with the data in the car logo image library, obtains the vehicle logo information, and outputs it to the visual task support server.

In a more preferred embodiment, the visual task support server includes, as shown in FIG. 3 , a hybrid information storage pool, a network communication controller, a message parsing distributor, a supporting logic controller, and a hybrid data assembler. Wherein, the hybrid information storage pool is respectively connected to the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer to utilize the described The timestamp is used as a unified identification to store the information output by the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter, and license plate and car logo recognizer; the network communication controller uses To connect the Ethernet to realize the sending and receiving of network messages; the message analysis distributor is connected to the output end of the network communication controller to analyze the control messages obtained from the network communication controller, and after analysis, obtain control information or query information; the support logic controller is connected to the output end of the message parsing distributor, so as to implement the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate according to the control information And the coordination and control of the car logo splitter and the license plate and car logo recognizer, and output data assembly instructions according to the query information; the hybrid data assembler is respectively connected to the hybrid information storage pool and the output end of the support logic controller , and connected to the input end of the network communication controller, to obtain corresponding information from the mixed-structure information storage pool according to the data assembly instruction, and assemble and generate a message for network communication transmission via the The network communication controller described above sends to Ethernet. The network communication controller is connected with an external semantic analysis system, a streaming media server and a multifunctional debugger through Ethernet.

Using this more preferred embodiment, in the method for extracting vehicle video features based on video structured description, the step (6) specifically includes the following steps:

(61) The hybrid information storage pool uses the timestamp as a unified identifier to store the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter and The information output by the license plate and logo recognizer, and dynamic information association;

(62) The network communication controller receives the network message through Ethernet;

(63) The message analysis distributor analyzes the control message obtained from the network communication controller to obtain control information or query information;

(64) According to the control information, the supporting logic controller implements the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer coordination and control, and output data assembly instructions to the hybrid data assembler according to the query information;

(65) The hybrid data assembler obtains corresponding information from the hybrid information storage pool according to the data assembly instruction, assembles and generates a message for network communication transmission, and then passes through the network Communication controller sends to Ethernet.

In the application of the present invention, as shown in Figure 1, the vehicle video feature extraction system of the present invention includes a video stream encoding parser 102, a vehicle bottom layer feature extractor 103, a vehicle MPEG-7 feature extractor 104, license plate/vehicle logo segmentation device 105, license plate/logo recognizer 106 and visual task support server 107. Peripheral devices and systems include video capture device 101 , semantic analysis system 108 and streaming media server 109 . These systems cooperate to complete the vehicle feature extraction and video-on-demand support functions in the video data.

Video acquisition equipment 101 is installed in traffic road monitoring point, road traffic live is photographed, produces real-time video flow data, adds the GPS position of the monitoring place where video acquisition equipment is located, road name, monitoring lane number, sends together to the design of the present invention The video stream encoding and analyzing module 102 in the vehicle video feature extraction system.

The video stream coder module 102 parses the video stream from the video capture device, transcodes the collected data according to a specified video format, and adds a time stamp according to the current time. The time stamp is used as the global uniform identifier of the video frame to provide support for the video on demand function.

The vehicle bottom-level feature extractor module 103 is mainly responsible for obtaining vehicle activity and basic features in outdoor videos, and extracting vehicle bottom-level features (partial pictures of vehicles, coordinate positions of vehicles in image frames, vehicle types, body colors, driving directions, and types of violations). It mainly includes the following functions: ① video stream analysis, background modeling and foreground extraction; ② camera depth of field calibration and vehicle speed calculation; ③ intersection photography, vehicle ROI information extraction, and traffic flow statistics. The output vehicle bottom layer features of the vehicle bottom layer feature extractor are divided into three paths in parallel, one path is sent to the hybrid data storage pool sub-module 302 in the visual task support server module 107 for storage, and one path is sent to the vehicle MPEG-7 feature extractor module 104 for further processing. MPEG-7 feature extraction, the other way is sent to the license plate/vehicle logo splitter 105.

The vehicle MPEG-7 feature extractor module 104 receives the vehicle bottom layer feature information from the vehicle bottom layer feature extractor module 107 output, performs bottom layer feature (MPEG-7) extraction according to the vehicle ROI, and obtains the vehicle MPEG-7 feature (color layout, color Space, Color Structure, Primary Color, Edge Histogram, Scalable Color, Texture Browsing, GoFGoP Color and Color Quantization). The vehicle MPEG-7 feature is output to the hybrid data storage pool sub-module 302 in the visual task support server module 107 for storage.

The license plate/vehicle logo splitter module 105 receives the vehicle bottom layer feature information output from the vehicle bottom layer feature extractor module 107, is responsible for locating the license plate ROI according to the basic vehicle information, obtaining the position of the car logo according to the license plate position and other information, and dividing the license plate according to the prior knowledge. Perform affine transformation with the car logo image to obtain the corrected standard view. The license plate/vehicle logo splitter module 105 separates two partial pictures of the license plate and the car logo from the partial vehicle picture, and marks its coordinate position in the vehicle picture, and the license plate/license plate segmentation information is divided into two outputs. One output is sent to the hybrid data storage pool sub-module 302 in the visual task support server module 107 for storage, and the other is sent to the license plate/vehicle logo recognizer module 106 .

The license plate/vehicle logo recognizer module 106 receives the license plate/license plate segmentation information output from the license plate/vehicle logo splitter module 105, is responsible for dividing characters from the standard face-to-face license plate, and recognizes each character to obtain the license plate information; The content of the car logo is obtained by matching the car logo image library. License plate/vehicle logo recognizer module 106 outputs license plate and car logo information (including manufacturer logo, license plate text, license plate color, license plate structure) and recognition process information (recognition reliability, license plate recognition time-consuming, car logo recognition time-consuming), and sends It is stored in the hybrid data storage pool sub-module 302 in the visual task support server module 107.

The visual task support server module 107 is responsible for organically combining other modules and providing the step-by-step debugging function of each module: ① realize network communication with the semantic analysis system 108, execute message commands or configuration; Retrieve the data, generate query results and return them to the semantic analysis system 108; ③ Independently/jointly test each visual task module; ④ Manually assist mark debugging; ⑤ Simultaneously display the source image and the analysis result. The visual task support server 107, as shown in Figure 3, consists of a support logic controller 301, a mixed data storage pool 302, a mixed data assembler 303, a message analysis distributor 304, a network communication controller 305 and a multifunctional debugger 306 consists of six sub-modules. Among them, the sub-module of the support logic controller 301 implements the main process of the entire visual task support server 107, and dispatches and controls the running sequence and function of each functional module of the visual task support server 107 to match. The main functions of the supporting logic controller 301 include: ①Execute configuration commands; ②Control (or debug) the video stream encoding parser 102, the vehicle underlying feature extractor 103, the vehicle MPEG-7 feature extractor 104, and the license plate/logo splitter 105 and the operation of each functional module of the license plate/vehicle logo recognizer 106; ③ query the mixed data in the mixed data storage pool 302 according to the message requirements, generate the required sending data subset and push it into the mixed data storage pool 302 Send buffer; ④ Send response message data (error message and execution feedback information) to the hybrid data assembler 303; ⑤ Notify the hybrid data assembler 303 to send data. The mixed structure data storage pool sub-module 302 is responsible for storing the source video stream with the time stamp of the video frame as the global unified identifier. The output generated after the processing of the vehicle logo segmenter 105 and the license plate/vehicle logo recognizer 106 includes: compressed video in a standardized format with a time stamp, vehicle bottom layer features, vehicle MPEG-7 features, license plate and car logo segmentation information, and license plate and car logo recognition result. The hybrid data assembler sub-module 303 is responsible for assembling the hybrid data from the hybrid data storage pool 302 and the supporting logic controller 301 according to the communication protocol, and generating a message body that can be sent by the network communication controller 305 . The message analysis distributor sub-module 304 analyzes the message received by the network communication controller according to the self-defined communication protocol, generates a message sequence, and sends it to the supporting logic controller for processing. The network communication controller submodule 305 is responsible for receiving the TCP request command from the peripheral system (semantic analysis system 108, streaming media server 109) or the multi-function debugger module 306, and sends it to the message analysis distributor 304 to analyze and process, and then The response result from the hybrid data assembler 303 is sent back to the peripheral system or the multi-function debugger module 306 .

The vehicle video feature extraction system based on the video structured description of the invention is adopted, which includes a video stream encoding parser, a vehicle bottom layer feature extractor, a vehicle video standard feature extractor, a license plate and car logo splitter, and a license plate and car logo recognition server and vision task support server. In the vehicle video feature extraction method using this system, firstly, the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer can be used to realize traffic surveillance video. Efficient stage-by-stage extraction of vehicle features, and then the visual task support server uses the time stamp as a unified identifier to store the video stream encoding parser, vehicle underlying feature extractor, vehicle video standard feature extractor, license plate and vehicle logo The information output by the segmenter and the license plate and logo recognizer is dynamically correlated based on the time stamp; and the stored information is assembled according to the query information obtained through the external Ethernet to generate a message for network communication transmission. Through the Ethernet output, it is convenient for quick retrieval and positioning in subsequent video access and processing, and then can realize real-time intelligent supervision of motor vehicle behavior in road management, and the vehicle video feature extraction system based on video structured description of the present invention And the method, the system structure is simple, the method realization process is simple, the realization cost is low, and the application range is relatively wide.

In this specification, the invention has been described with reference to specific embodiments thereof. However, it is obvious that various modifications and changes can be made without departing from the spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive.

Claims (11)

1. A vehicle video feature extraction system based on video structured description, is characterized in that, described system comprises: A video stream encoding parser, used to obtain a video stream from the connected external video capture device, transcode the video stream according to a preset video format and add a time stamp, and output the video containing the time stamp Transcode the video stream; The vehicle bottom layer feature extractor is connected to the output end of the video stream coder and parser, in order to obtain the vehicle bottom layer feature information from the transcoded video stream, and output it; The vehicle video standard feature extractor is connected to the output end of the vehicle bottom layer feature extractor, in order to obtain the vehicle video standard feature information according to the vehicle bottom layer feature information, and output it; License plate and car logo splitter, connected to the output of the vehicle bottom layer feature extractor, in order to segment and obtain the license plate and car logo image of the vehicle according to the vehicle bottom layer feature information, and output; The license plate and car logo recognizer is connected to the output end of the license plate and car logo splitter, and is used for obtaining the license plate and car logo information of the vehicle according to the described license plate and car logo image recognition, and outputting it; and The visual task support server is respectively connected to the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer to coordinate and control the The operation of the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and the car logo splitter and the license plate and the car logo recognizer; use the time stamp as a unified identification to store the video Stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter, and license plate and car logo recognizer output information to realize dynamic association of output information; and assemble the stored information , generating a message available for network communication transmission; the video stream encoding parser is a video stream encoding parser that provides video-on-demand function support; Described visual task supporting server comprises: The hybrid information storage pool is respectively connected to the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter and the license plate and car logo recognizer, in order to use the described The time stamp is used as a unified identification to store the information output by the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer; The network communication controller is used to connect to the Ethernet to realize the sending and receiving of network messages; The message parsing distributor is connected to the output end of the network communication controller to analyze the control message obtained from the network communication controller, and obtain control information or query information after analysis; The supporting logic controller is connected to the output end of the message parsing distributor, so as to implement the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and the vehicle according to the control information. Coordination and control of the logo splitter and the license plate and car logo recognizer, and output data assembly instructions according to the query information; The hybrid data assembler is respectively connected to the output of the hybrid information storage pool and the support logic controller, and connected to the input of the network communication controller, so as to obtain from the Obtain corresponding information from the mixed-structure information storage pool, and assemble and generate a message that can be transmitted by network communication and send it to Ethernet through the network communication controller; The network communication controller is connected to an external semantic analysis system, a streaming media server and a multi-function debugger through Ethernet. 2. the vehicle video feature extraction system based on video structured description according to claim 1, is characterized in that, described vehicle bottom layer feature extractor comprises: A video stream analysis unit, connected to the output end of the video stream encoding parser, for background modeling and foreground extraction; The camera depth of field calibration and vehicle speed calculation unit is connected to the video stream analysis unit to calibrate the depth of field based on the background and foreground to realize the vehicle speed calculation; The intersection camera and vehicle ROI information extraction unit is connected to the camera depth of field calibration and vehicle speed calculation unit to control the intersection camera to take pictures, extract vehicle ROI information, and perform traffic flow statistics. 3. The vehicle video feature extraction system based on video structured description according to claim 2, wherein the vehicle bottom layer feature information includes vehicle partial picture information, vehicle coordinate position information in the image frame, vehicle type information , body color information, driving direction information and violation type information. 4. the vehicle video feature extraction system based on video structured description according to claim 1, wherein said vehicle video standard feature extractor is a vehicle MPEG-7 feature extractor, and said vehicle video standard feature extractor The information is vehicle MPEG-7 characteristic information. 5. The vehicle video feature extraction system based on video structured description according to claim 4, wherein said vehicle MPEG-7 feature information comprises: color layout information, color space information, color structure information, main color Information, Edge Histogram Information, Scalable Color Information, Texture Browsing Information, GoFGoP Color Information, and Color Quantization Information. 6. The vehicle video feature extraction system based on video structured description according to claim 1, wherein said license plate and car logo segmenter include: The license plate and car logo positioning unit is connected to the output end of the vehicle bottom layer feature extractor, in order to determine the license plate position and the car logo position according to the described vehicle bottom layer feature information, and obtain the license plate and car logo images; The license plate and vehicle logo image transformation unit is connected to the license plate and vehicle logo positioning unit, performs affine transformation according to the license plate and vehicle logo images, obtains a corrected standard view, and outputs it. 7. The vehicle video feature extraction system based on video structured description according to claim 1, wherein said license plate and vehicle logo recognizer comprises: The license plate recognition unit is connected to the output end of the license plate and the car logo divider, and is used to divide the license plate image into characters, recognize the divided characters, obtain the vehicle license plate information, and output it; The vehicle logo recognition unit is connected to the output end of the license plate and the car logo splitter, and is used to match the car logo image with the data in the car logo image database, obtain the vehicle logo information, and output it. 8. A method utilizing the system according to claim 1 to realize vehicle video feature extraction based on video structured description, characterized in that said method comprises the following steps: (1) The video stream encoding parser obtains the video stream from the connected external video acquisition device, and transcodes the video stream according to the preset video format and adds a time stamp to the vehicle The underlying feature extractor and the visual task support server output the transcoded video stream containing the time stamp; (2) The vehicle bottom layer feature extractor obtains the vehicle bottom layer feature information from the transcoded video stream, and provides the vehicle video standard feature extractor, license plate and car logo splitter and visual task support server Outputting the vehicle bottom layer feature information; (3) The vehicle video standard feature extractor obtains the vehicle video standard feature information according to the vehicle bottom layer feature information, and outputs the vehicle video standard feature information to the visual task support server; (4) The license plate and car logo splitter obtains the license plate and car logo images of the vehicle according to the bottom layer feature information of the vehicle, and outputs them to the license plate and car logo recognizer and the visual task support server The said license plate and logo image; (5) The license plate and logo recognizer obtains the license plate and logo information of the vehicle according to the license plate and logo image recognition, and outputs the license plate and logo information to the visual task support server; (6) The visual task support server uses the time stamp as a unified identifier to store the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and vehicle logo splitter, and the license plate and the information output by the vehicle logo recognizer to dynamically correlate; and assemble the stored information according to the query information obtained through the external Ethernet, generate a message that can be transmitted through the network communication, and then output it through the Ethernet; The visual task support server includes: a mixed-structure information storage pool, a network communication controller, a message parsing distributor, a support logic controller and a mixed-structure data assembler, and the step (6) specifically includes the following steps: (61) The hybrid information storage pool uses the time stamp as a unified identifier to store the video stream encoding parser, vehicle bottom layer feature extractor, vehicle video standard feature extractor, license plate and car logo splitter and The information output by the license plate and logo recognizer, and dynamic information association; (62) The network communication controller receives the network message through the Ethernet; (63) After the message analysis distributor analyzes the control message obtained from the network communication controller, it obtains control information or query information; (64) According to the control information, the support logic controller implements the video stream encoding parser, the vehicle bottom layer feature extractor, the vehicle video standard feature extractor, the license plate and car logo splitter, and the license plate and car logo recognizer coordination and control, and output data assembly instructions to the hybrid data assembler according to the query information; (65) The hybrid data assembler obtains corresponding information from the hybrid information storage pool according to the data assembly instruction, assembles and generates a message for network communication transmission, and then passes through the network Communication controller sends to Ethernet. 9. The method for realizing vehicle video feature extraction based on video structured description according to claim 8, wherein the vehicle bottom layer feature extractor includes a video stream analysis unit, camera depth of field calibration and vehicle speed calculation unit and intersection photographing And vehicle ROI information extracting unit, described step (2) specifically comprises the following steps: (21) The video stream analysis unit performs background modeling and foreground extraction according to the transcoded video stream; (22) The camera depth of field calibration and vehicle speed calculation unit is based on background and foreground calibration depth of field, and performs vehicle speed calculation; (23) The intersection photographing and vehicle ROI information extraction unit controls the intersection camera to take photographs, extract vehicle ROI information, and perform traffic flow statistics. 10. The method for realizing vehicle video feature extraction based on video structured description according to claim 8, characterized in that, said license plate and car logo segmenter comprises: a license plate and car logo positioning unit and a license plate and car logo image transformation unit, described step (4) comprises the following steps: (41) The license plate and vehicle logo positioning unit determines the license plate position and the vehicle logo position according to the vehicle bottom layer feature information, and obtains the license plate and vehicle logo images; (42) The license plate and logo image conversion unit performs affine transformation according to the license plate and logo images to obtain a corrected standard view, and outputs it to the license plate and logo recognizer and the Vision task support server. 11. The method for realizing vehicle video feature extraction based on video structured description according to claim 8, wherein said license plate and vehicle logo recognizer comprises: a license plate recognition unit and a car logo recognition unit, and said steps (5) Concretely include the following steps: (51) The license plate recognition unit divides the license plate image into characters, recognizes the divided characters, obtains the vehicle license plate information, and outputs it to the visual task support server; (52) The vehicle logo recognition unit matches the car logo image with the data in the car logo image library, obtains the vehicle logo information, and outputs it to the visual task support server.