CN108322636A - A kind of speed estimator, vehicle and method of estimation based on machine vision - Google Patents

Abstract

The present invention provides a kind of speed estimator, vehicle and method of estimation based on machine vision, the speed estimator is in the side mirror of vehicle, it is made of prism and camera, laser emitter, drive module, speed estimation module and communication module, laser emitter sends out light source and exposes to prism, on the one hand laser light source enters camera after prism reflects, on the other hand it is irradiated to ground through prism, ground image information reaches camera after prism reflects；Drive module is connect with laser emitter and camera signals respectively, and the filming frequency of camera is identical as the tranmitting frequency of laser emitter, and camera is connect with speed estimator signal；Speed estimates realization signal communication of the mould by communication module and vehicle ECU；The vehicle speed estimation method calculates three parts by the acquisition of image, the processing of image and velocity information and forms.The problems such as present invention can effectively solve the problem that existing speed estimation is difficult or estimation is inaccurate.

Description

A vehicle speed estimator, vehicle and estimation method based on machine vision

technical field

The invention belongs to the technical field of automobile assisted driving, and relates to a vehicle speed estimator based on machine vision, a vehicle and an estimating method, which are suitable for self-driving cars or mid-to-high-end passenger vehicles equipped with advanced driving assistance systems.

Background technique

With the progress of the automobile industry, automobiles have become affordable consumer goods for ordinary families. The number of vehicles is increasing continuously. On the one hand, it has caused a serious burden on traffic and the environment. On the other hand, traffic accidents have seriously threatened people. safety of life and property. According to the survey, at present, more than 90% of traffic accidents are caused by human error or negligence. In order to reduce traffic accidents caused by human factors, researchers from various countries have vigorously researched and developed advanced driver assistance systems (ADAS) to improve driving safety.

Due to the improvement of advanced driver assistance systems and their continuous release to the low-end market, the number of cars equipped with driver assistance systems is increasing day by day, and driving safety has been significantly improved. However, many driving assistance systems require information such as vehicle speed and body posture as input to realize the assisted driving function. Accurate vehicle speed information is of great significance for the evaluation of slip ratio, and slip ratio is crucial to many active safety systems, especially chassis-related active safety systems. Most of the methods currently used are estimated by the maximum wheel speed method, but this method is not accurate in estimating the speed of low-speed vehicles, and there is a lack of a suitable sensor in the prior art to accurately detect the speed of the vehicle.

At present, the computing speed and camera imaging quality of consumer electronics products (such as smart phones) have made a qualitative leap. Billions of transistors can be integrated in a small CPU, and the computing speed of mobile terminals is already comparable to that of desktop terminals. Moreover, with the development of machine vision technology, small integrated systems are fully capable of performing high-computing tasks such as machine vision processing.

The so-called machine vision technology is a branch of artificial intelligence that is rapidly developing. Simply put, it is to use machines instead of human eyes for measurement and judgment. The machine vision system converts the ingested target into an image signal through machine vision products, transmits it to a dedicated image processing system, obtains the shape information of the captured target, and converts it into a digital signal according to the pixel distribution, brightness, color and other information; dedicated The image system performs various calculations on these signals to extract the characteristics of the target, and then controls the on-site equipment actions according to the results of the discrimination.

When applying machine vision technology to the vehicle speed estimation system, the two crucial issues are image acquisition and image processing. At present, except for the depth-of-field dual-shot of self-driving vehicles that use high-quality cameras, other vehicle-mounted cameras use relatively low-end cameras under the premise of considering cost, and relatively low-end cameras cannot meet high frame rate shooting. Image processing technology is only used in some autonomous driving test vehicles and high-end passenger vehicles. Therefore, the use of mature imaging circuit integration technology and machine vision technology can not only provide accurate speed information for vehicles, but also further realize functions such as lane line detection and blind spot monitoring.

Contents of the invention

The present invention provides a machine vision-based vehicle speed estimator, vehicle and estimation method to solve the existing problems such as difficult or inaccurate estimation of vehicle speed. In combination with the accompanying drawings, the technical solution of the present invention is as follows:

A vehicle speed estimator based on machine vision, the vehicle speed estimator is embedded in the rear view mirror of the vehicle, and is composed of a prism and a camera, a laser transmitter, a drive module, and a vehicle speed estimation module integrated on a PCB printed circuit board and communication module;

The prism is arranged on the front end of the PCB printed circuit board, and is fixed on the bottom of the rear view mirror; the laser emitter emits a light source to irradiate the prism, and the laser light source enters the camera after being refracted by the prism on the one hand, and irradiates to the prism through the prism on the other hand. On the ground, the ground image information reaches the camera after being reflected by the prism; the drive module is connected with the laser transmitter and the camera signal respectively; the image signal output port of the camera is connected with the image signal receiving port signal of the vehicle speed estimator; the vehicle speed estimation The signal output port of the module is connected to the signal receiving port of the communication module, and the signal output port of the communication module is connected to the signal receiving port of the vehicle ECU. The vehicle speed estimation module processes the received image information and passes the processing result through The communication module transmits to the vehicle ECU;

The shooting frequency of the camera is the same as that of the laser transmitter.

Further, the camera is a narrow-angle camera.

Further, the communication module uses CAN bus technology to communicate the data signal sent by the vehicle speed estimation module with the vehicle electronic control system ECU.

A vehicle, the vehicle speed estimator based on machine vision as claimed in claim 1 is installed on both sides of the vehicle.

A kind of vehicle speed estimation method based on machine vision, described vehicle speed estimation method is based on above-mentioned vehicle, described vehicle speed estimation method is made up of image acquisition, image processing and speed information calculation three parts, specifically as follows:

1. The image acquisition process is as follows:

First detect the ambient light, and then collect the data;

The detection process of ambient light is as follows:

A1: When the vehicle is started, the camera collects an image, and sends the image information to the vehicle speed estimation module to calculate the ambient light intensity;

A2: The vehicle speed estimation module extracts the light intensity information of all pixels in the image, filters out noise points whose intensity is significantly different from the light intensity of surrounding pixels, and averages the light intensity of the remaining pixels to obtain an ambient light intensity value P _env ;

A3: Compare the ambient light intensity value P _env with the preset ambient light intensity threshold, and then judge whether to turn on the laser transmitter for image-assisted acquisition;

The data collection process is as follows:

Control the camera to take high-definition shots of the ground image information below it at a preset frequency, and the camera transmits the data information of the first image to the vehicle speed estimation module in the form of a data packet;

The data structure of the data packet is as follows: it includes an unsigned integer variable timestamp stamp in milliseconds for recording the shooting time; an unsigned integer two-dimensional array data is used to store image LBP information, and the data is image The intensity value of each pixel; an integer variable dir is used to record the direction of the vehicle, dir<0 represents the forward direction, dir>0 represents the backward direction; a floating-point variable hight is used to store the height information of the camera from the ground, and the unit is Meter;

Two, the image processing process is as follows:

The image processing process is carried out in the vehicle speed estimation module, and the specific processing process is as follows:

B1: Analyze the data packets obtained during the aforementioned image acquisition process, and extract the four variables of stamp, data, dir and hight of the data packets into the memory;

B2: The LBP algorithm is used to extract local features of the image to obtain LBP information, and the edge area is ignored in the process of feature extraction;

B3: If the image collected by the camera is the first image taken, that is, there is no data in the cache at this time, then directly store the LBP information in the cache; if the image collected by the camera is not the first image taken, that is, at this time If there is data in the cache, extract the local image information in the memory and compare it with the image feature information in the current cache;

Suppose the data structure of the first data packet is (stamp ₁ , data ₁ , dir ₁ , hight ₁ ), and the data structure of the i-th data packet is (stamp _i , data _i , dir _i , hight _i ); and assume this At the same time, the i-th data packet is in the memory, and the i-1-th data packet is in the cache; perform a backward or forward traversal search according to the value of dir _i-1 , find the area with the most similar features, and finally get the image Offsets offset_x and offset_y, where offset_x is the vertical offset and offset_y is the horizontal offset.

3. The calculation process of the speed information is as follows:

Estimate the speed of the vehicle based on the difference of the time stamp information in the data packet, the offset of the image, and the relative position information of the cameras on both sides of the vehicle, which mainly includes: lateral velocity estimation, longitudinal velocity estimation and yaw angular velocity estimation;

The specific process of estimating the lateral velocity of the vehicle is as follows:

Δt=stamp _i -stamp _i-1

In the above formula:

Δt is the time difference between two image samples;

offset_y _l and offset_y _r are the lateral offsets of similar areas on the left and right sides of the vehicle, respectively;

Finally, take the average value of the left and right lateral velocity estimates as the lateral velocity of the vehicle;

The specific process of vehicle longitudinal velocity estimation is as follows:

Δt=stamp _i -stamp _i-1

In the above formula:

Δt is the time difference between two image samples;

offset_x _l and offset_x _r are the longitudinal offsets of similar areas on the left and right sides of the vehicle, respectively;

Finally, take the average value of the left and right longitudinal velocity estimates as the longitudinal velocity of the vehicle;

The specific process of estimating the vehicle yaw rate is as follows:

Δt=stamp _i -stamp _i-1

In the above formula:

ω is the yaw rate of the vehicle;

offset_x _l and offset_x _r are the longitudinal offsets of similar areas on the left and right sides of the vehicle, respectively;

l is the distance between the vehicle speed estimators on the left and right sides of the vehicle;

In the above vehicle speed estimation method, the LBP processing and extraction process of the image information and the calculation process of the four offsets of offset_x _l , offset_x _r , offset_y _l and offset_y _r are all completed in the vehicle speed estimation module, and the rest of the calculation process is performed in the vehicle ECU completed.

Further, in the detection process of the ambient light, in step A3, the ambient light intensity value P _env is compared with the preset ambient light intensity threshold value, and then the specific process of judging whether to turn on the laser transmitter for image-assisted acquisition is as follows :

The preset ambient light intensity upper limit threshold is P _high , and the ambient light intensity lower limit threshold is P _low , where P _high >P _low ; when the vehicle is started for the first acquisition, the captured image is the first During the image, compare the obtained ambient light intensity value P _env with (P _high +P _low )/2, if P _env is greater than (P _high +P _low )/2, turn off the laser transmitter, otherwise turn on the laser Transmitter; Next, when the camera performs the second acquisition or subsequent acquisition, and the image collected is not the first image, when the laser transmitter is turned off at this time, if P _env is less than P _low , that is, P _env < P _low , the laser transmitter is turned on, otherwise the laser transmitter remains off; when the laser transmitter is turned on at this time, if P _env is greater than P _high , that is, P _env > P _high , the laser transmitter is turned off, otherwise the laser emits The device remains open.

Further, in the image processing process, in step B3, the specific process of extracting the partial image information in the memory and comparing the image feature information in the current cache is as follows:

Take inten=0xffffffff, offset_x=offset_y=0; extract the local feature information of the image in the current cache, get the LBP histogram, store the data in the variable list, that is, store the local information of data _i-1 in the cache data, and wait for the next step Contrast use;

In the vehicle coordinate system, move dir _i-1 pixels in the vertical direction, move horizontally to the top of the image, and obtain the area to be searched; extract the LBP histogram information of the area to be searched, and store the data in the variable list', and list' is in In memory; find the standard deviation inten' of the difference between list and list', that is, compare the image feature data in the memory with the image feature data in the cache, and the smaller the standard deviation, it proves that the image feature data in the memory and the image feature in the cache The higher the data similarity is; compare inten' and inten, if inten'<inten, set inten=inten', and record the current offset into the current offset_x and offset_y, otherwise do not process; after the processing is completed, the area to be compared Move down |dir _i-1 | units and perform the above process again until the bottom of the image is reached;

After a column of vertical comparison is completed, move the area to be searched by dir _i-1 pixels horizontally, start from the top of the image again, and search and compare to the bottom of the image; and so on, and finally get the image feature information in the memory and the cache. The offset_x and offset_y of the closest area of the local feature information of the image;

At this point, a cycle is completed, the data in the cache is cleared, and the image feature data of the most similar region is stored in the cache for comparison in the next cycle.

Compared with prior art, the beneficial effect of the present invention is:

1. The machine vision-based vehicle speed estimator and estimation method of the present invention can solve problems such as difficult or inaccurate estimation of the existing vehicle speed;

2. The machine vision-based vehicle speed estimator and estimation method of the present invention give full play to the advantages of circuit integration technology and machine vision technology, adopt an independent vehicle speed estimation module to communicate with the CAN bus, and directly send speed signals, reducing the speed of traditional vehicle systems. The technical threshold for docking with the present invention also reduces the calculation load of the vehicle ECU and improves the calculation efficiency of vehicle speed estimation;

3. The vehicle speed estimator based on machine vision of the present invention rationally integrates and arranges its components, so that the components are all embedded in the side mirrors on both sides of the vehicle, so as to avoid excessive protrusions on the vehicle body and avoid damaging the overall streamlined shape;

4. In the vehicle speed estimator based on machine vision of the present invention, a prism is used to deflect the optical path, and the focal length can be increased as much as possible without increasing the longitudinal volume of the reversing mirror, so that the road surface information collected by the camera is more detailed ;In addition, the prism can protect the camera to a certain extent and make the integrated circuit PCB easier to fix;

5. The vehicle speed estimator based on machine vision of the present invention uses a laser transmitter embedded in the rear view mirror to emit laser light, and irradiates the ground through a prism so that the camera can work normally under poor lighting conditions, including driving at night. round-the-clock operation;

6. The vehicle speed estimator based on machine vision of the present invention is equipped with an independent drive module to drive the laser transmitter, and the shooting frequency of the camera is equal to the shooting frequency of the laser transmitter, and the exposure time is shortened by using high frame rate shooting technology, Avoid imaging blur caused by high-speed driving of vehicles;

7. The vehicle speed estimation method based on machine vision of the present invention enables the drive module to perform intermittent charging and discharging, which meets the energy demand of the laser transmitter under the condition of low-power energy supply;

8. The camera in the machine vision-based vehicle speed estimator of the present invention can also be used to detect lane lines or detect blind spots;

9. The vehicle speed estimator based on machine vision of the present invention can use a camera with a narrow viewing angle to reduce graphic distortion.

Description of drawings

Fig. 1 is a schematic diagram of the installation layout of the vehicle speed estimator based on machine vision in the vehicle rear view mirror according to the present invention;

Fig. 2 is the logic connection block diagram of each constituent device in the vehicle speed estimator based on machine vision of the present invention;

Fig. 3 is in the vehicle speed estimation method based on machine vision described in the present invention, the flow block diagram of ambient light detection;

Fig. 4 is in the vehicle speed estimation method based on machine vision described in the present invention, the flowchart of image processing and speed computing process;

Fig. 5a is a schematic diagram of extracting LPB feature information of the image in the current buffer during the image feature information comparison process of the machine vision-based vehicle speed estimation method of the present invention;

Fig. 5b is a schematic diagram of the image LBP feature information in the memory and the area to be searched during the image feature information comparison process of the machine vision-based vehicle speed estimation method of the present invention, wherein the shaded part is the area to be searched;

Fig. 5c is a schematic diagram of traversing the area to be searched, finding the closest similar area, and obtaining the lateral offset and vertical offset of the similar area during the image feature information comparison process of the machine vision-based vehicle speed estimation method of the present invention.

Detailed ways

In order to further illustrate the technical solution of the present invention and the technical effects brought by it, in conjunction with the accompanying drawings, the specific implementation of the present invention is as follows:

The present invention provides a kind of vehicle speed estimator based on machine vision, as shown in Figure 1, described vehicle speed estimator is made up of prism, camera, laser emitter, drive module, vehicle speed estimation module and communication module; Said vehicle speed estimator All components are embedded in the rear view mirror of the vehicle; the camera, laser transmitter, drive module, vehicle speed estimation module and communication module are all integrated and arranged on a PCB printed circuit board, and the triangular prism is arranged on the PCB printed circuit board. The front end of the board is fixed on the bottom of the reversing mirror. On the one hand, the light source emitted by the laser emitter is deflected by the prism and then enters the camera to lengthen the focal length of the camera so that the camera can collect road information in more detail. On the other hand, the laser The light source emitted by the transmitter is irradiated to the ground through the prism, so that the camera can work normally under poor lighting conditions and realize all-weather operation including driving at night.

As shown in Figure 2, the laser transmitter emits a light source and irradiates the prism. The laser light source enters the camera after being refracted by the prism on the one hand, and irradiates the ground through the prism on the other hand. The ground image information reaches the camera after being reflected by the prism; Two drive signal output ports are respectively connected with the drive signal input ports of the laser transmitter and the camera, and the drive module drives the laser transmitter and the camera to work respectively; the image signal output port of the camera is connected to the image signal receiving port of the vehicle speed estimator Signal connection, the camera transmits the captured image information to the vehicle speed estimation module; the signal output port of the vehicle speed estimation module is connected to the signal receiving port of the communication module, and the signal output port of the communication module is connected to the signal receiving port of the vehicle ECU Signal connection, the vehicle speed estimation module processes the received image information and transmits the processing result to the vehicle ECU through the communication module, for the vehicle ECU to further analyze and process the vehicle driving state.

The drive module drives the laser transmitter to send out a laser signal that meets the frequency requirements. In addition, the drive module also drives the camera to drive the camera to perform shooting work; the drive module uses intermittent charging Discharge to meet the energy demand of the laser transmitter under the condition of low power supply.

The laser emission frequency of the laser emitter is 200Hz, and the shooting frequency of the camera is equal to the emission frequency of the laser emitter, that is, the shooting frequency of the camera also adopts 200Hz (200 frames/second). It should be noted that the laser emission frequency and camera acquisition frequency in the vehicle speed estimator of the present invention are not limited to 200Hz, that is, the image acquisition frequency of the vehicle speed estimator of the present invention is not limited to 200Hz; 200Hz is used as the embodiment of the present invention It is an image data acquisition frequency value selected according to the current consumer-grade integrated high-speed image sensor (refer to mobile phone cameras, such as Sony IMAX series). Today's professional high-speed cameras have been able to achieve a shooting frame rate of 1,000 to 10,000 frames per second. Based on considerations of product cost and layout space, the embodiments of the present invention can achieve corresponding technical objectives by adopting the technical parameters of a consumer-grade integrated high-speed image sensor. But this does not rule out that the present invention uses high-end image sensor technology with a higher frame rate in actual operation. In fact, using a shooting technology with a higher frame rate to shorten the exposure time can better avoid the imaging blur caused by the high-speed driving of the vehicle. Phenomenon.

The laser transmitter emits laser light under the drive of the drive module. On the one hand, the laser light is irradiated on the ground through the prism, so that the camera can work normally under the conditions of insufficient light conditions, and realizes all-weather operation including driving at night; on the other hand On the one hand: the laser passes through the prism, using the principle of total reflection of the prism, the laser light path is deflected and then reflected to the camera, so that the camera can take high-definition shots of the ground image below it to collect the image information of the road below the camera, and the camera will collect the collected road image information It is transmitted to the vehicle speed estimation module.

The camera adopts a narrow viewing angle camera. What needs to be explained here is that most of the cameras carried on existing vehicles are wide-angle cameras. The advantage of the wide-angle camera is that it has a wider field of view. The distortion is serious, that is, the edge of the image is greatly deformed, which will affect the accuracy of the image processing in the later stage, because the key step of the vehicle speed estimator in the present invention is to identify and process the image taken by the camera in the process of estimating the vehicle speed. Therefore, the accuracy of the image taken by the camera is crucial to the accuracy of the speed estimation, so the selected camera in the vehicle speed estimator of the present invention abandons the wide-angle camera commonly used by most of the existing vehicles, and uses the wide-angle camera instead. Other narrow-angle cameras to reduce graphic distortion and improve the accuracy of image processing.

In addition, in the vehicle speed estimator based on machine vision of the present invention, the function of the camera can be further developed, and can be used for detection of lane lines and detection of blind spots.

The vehicle speed estimation module is an independent small image processing unit, and its function is to process machine vision signals including the estimation of ambient light and image signals. Specifically, the vehicle speed estimation module receives the road surface image information collected by the camera, Carry out machine vision signal processing on the road surface image information, extract road surface features, and compare the road surface image information collected at different time points, and accurately estimate the vehicle speed according to the comparison results of the road surface image information captured by the camera before and after. In addition, adopting the independent vehicle speed estimation module in the present invention will effectively reduce the technical threshold for docking the traditional vehicle system with the present invention, and also reduce the calculation load of the ECU of the vehicle electronic control system, so as to improve the calculation efficiency of vehicle speed estimation.

The vehicle speed estimation module sends the estimated lateral speed of one side of the vehicle and the longitudinal speed information of one side of the vehicle to the communication module, and the communication module uses CAN bus technology to transmit the received lateral speed of one side of the vehicle and the information of the longitudinal speed of one side of the vehicle to the communication module. The longitudinal speed information on the side communicates with the ECU of the vehicle electronic control system as an input signal of the ECU of the vehicle electronic control system to realize assisted driving control of the vehicle.

The present invention makes full use of the space inside the rear view mirror, and closely arranges relatively large cameras, laser transmitters, drive modules, vehicle speed estimation modules and communication modules in the rear view mirror of the vehicle, so that the rear view mirror does not have obvious protrusions. Lifting or other deformations hardly affect the appearance of the vehicle. The camera, laser transmitter, drive module, vehicle speed estimation module and communication module are all integrated and arranged on a PCB printed circuit board, the logical connection relationship of each part is as described above, and the layout of each part on the PCB can be seen in detail. Depending on the situation, the layout adopted in this specific embodiment is: the vehicle speed estimation module is located above one side of the PCB printed circuit board, the driving module and the vehicle speed estimation module are located side by side above the other side of the PCB printed circuit board, and the laser emitter The controller and the communication module are located side by side under the driver module, and the camera is located under the communication module.

The vehicle speed estimator based on machine vision of the present invention is symmetrically installed in the side view mirrors on both sides of the vehicle along the longitudinal axis of the vehicle. When the vehicle is in a working condition with yaw rate such as turning, the longitudinal velocity measured by the vehicle speed estimator on the inside of the turn should be smaller than the longitudinal velocity measured by the vehicle speed estimator on the outside of the turn. For the specific measurement process, please refer to the description of the estimation method of the vehicle speed estimator below.

It should be noted that the machine vision-based vehicle speed estimator of the present invention is a single-channel black and white sensor, that is, the machine vision-based vehicle speed estimator of the present invention can only sense light intensity information, but cannot collect information such as color.

Combining the composition and logical connection relationship of the above-mentioned vehicle speed estimator based on machine vision, the present invention also provides a vehicle speed estimation method based on machine vision. The vehicle speed estimation method is composed of three parts: image acquisition, image processing and speed information calculation.

1. The image acquisition process is as follows:

In an environment with insufficient light (such as driving at night), in order to ensure that the vehicle speed estimator can still operate normally, the camera needs to be used to detect the surrounding ambient light. If the ambient light is detected to be insufficient, the laser transmitter is turned on and the camera shooting frequency is turned on. A laser with the same frequency is used to assist in image acquisition. Therefore, the image collection process includes: firstly detecting the ambient light, and then collecting the data;

As shown in Figure 3, the detection process of ambient light is as follows:

A1: When the vehicle is started, the camera collects an image, and sends the image information to the vehicle speed estimation module to calculate the ambient light intensity;

A2: The vehicle speed estimation module extracts the light intensity information of all pixels in the image, filters out noise points whose intensity is significantly different from the light intensity of surrounding pixels, and averages the light intensity of the remaining pixels to obtain an ambient light intensity value P _env ;

A3: Compare the ambient light intensity value P _env with the preset ambient light intensity threshold, and then judge whether to turn on the laser transmitter for image-assisted acquisition;

The comparison and judgment process are as follows: the upper limit threshold of ambient light intensity is preset as P _high , and the lower limit threshold of ambient light intensity is P _low , wherein, P _high >P _low ; when the vehicle starts to collect for the first time, that is When the collected image is the first image, compare the obtained ambient light intensity value P _env with (P _high +P _low )/2, if P _env is greater than (P _high +P _low )/2, turn off The laser transmitter, otherwise the laser transmitter will be turned on; next, when the camera performs the second acquisition or subsequent acquisition, and the image collected is not the first image, when the laser transmitter is turned off at this time, if P _env is less than P _low , that is, P _env < P _low , then the laser transmitter is turned on, otherwise the laser transmitter remains off; when the laser transmitter is turned on at this time, if P _env is greater than P _high , that is, P _env > P _high , then The laser transmitter is off, otherwise the laser transmitter remains on.

The above comparison and judgment method can effectively avoid the problem that the laser sensor frequently switches between the on and off states when the light intensity point is near the preset ambient light intensity upper limit threshold or the ambient light intensity lower limit threshold, so that the laser sensor The working state of the sensor tends to be stable.

The data collection process is as follows:

Control the camera to take high-definition shots of the ground image information below it at a frequency of 200Hz, and the camera transmits the data information of the first image to the vehicle speed estimation module in the form of a data packet;

The data structure of the data packet is as follows: it includes an unsigned integer variable timestamp stamp in milliseconds for recording the shooting time; an unsigned integer two-dimensional array data is used to store image LBP information, and the data is image The intensity value of each pixel; an integer variable dir is used to record the direction of the vehicle, dir<0 represents the forward direction, dir>0 represents the backward direction; a floating-point variable hight is used to store the height information of the camera from the ground, and the unit is Meter.

The role of the four variables stamp, data, dir, and hight in the above packet structure will be described in detail below.

Two, the image processing process is as follows:

The image processing process is carried out in the vehicle speed estimation module, as shown in Figure 4, and the specific processing process is as follows:

B1: Analyze the data packets obtained during the aforementioned image acquisition process, and extract the four variables of stamp, data, dir and hight of the data packets into the memory;

B2: The LBP algorithm is used to extract local features of the image to obtain LBP information, and the edge area is ignored in the process of feature extraction;

B3: If the image collected by the camera is the first image taken, that is, there is no data in the cache at this time, then directly store the LBP information in the cache; if the image collected by the camera is not the first image taken, that is, at this time If there is data in the cache, extract the local image information in the memory and compare it with the image feature information in the current cache;

Suppose the data structure of the first data packet is (stamp ₁ , data ₁ , dir ₁ , hight ₁ ), and the data structure of the i-th data packet is (stamp _i , data _i , dir _i , hight _i ); and assume this At the same time, the i-th data packet is in the memory, and the i-1-th data packet is in the cache; perform a backward or forward traversal search according to the value of dir _i-1 , find the area with the most similar features, and finally get the image Offsets offset_x and offset_y, where offset_x is the vertical offset and offset_y is the horizontal offset.

The specific process of comparing the above image feature information is as follows:

Get inten=0xffffffff (the maximum number that can be represented by 32-bit binary numbers), offset_x=offset_y=0; extract the local feature information of the image in the current cache, as shown in Figure 5a, obtain the LBP histogram, and store the data in the variable list, That is, the partial information of data _i-1 is stored in the cache data, and will be used for comparison in the next step;

In the vehicle coordinate system, move dir _i-1 pixels vertically, and move horizontally to the top of the image, as shown in Figure 5b, to obtain the area to be searched; extract the LBP histogram information of the area to be searched, and store the data in variables list', list' is in the memory; find the standard deviation inten' of the difference between list and list', that is, compare the image feature data in the memory with the image feature data in the cache, the smaller the standard deviation, the image in the memory The higher the similarity between the feature data and the image feature data in the cache; compare inten' and inten, if inten'<inten, set inten=inten', and record the current offset into the current offset_x and offset_y, otherwise do not process; process After completion, move the area to be compared downward |dir _i-1 | units and perform the above process again until it reaches the bottom of the image;

After a column of vertical comparison is completed, move the area to be searched by dir _i-1 pixels horizontally, start from the top of the image again, and search and compare to the bottom of the image; and so on, and finally get the image feature information in the memory and the cache. The offsets offset_x and offset_y of the region where the local feature information of the image is closest, as shown in Figure 5c.

At this point, a cycle is completed, the data in the cache is cleared, and the image feature data (LBP data) of the most similar region is stored in the cache for comparison in the next cycle.

It should be noted _that the value of dir _i-1 can be positive or negative. When the vehicle is moving forward, the image captured at the previous moment will move backward at the next moment. After the search; under the condition of the vehicle moving backwards, the image at the previous moment will move forward at the next moment, so the search step size of dir _i-1 is greater than zero for backward search. The size of the search area can be selected by yourself, but it is not recommended that the search area be too small, preferably one tenth of the entire image. The absolute value of dir _i-1 must be an integer greater than zero, and the smaller the better, the smaller the dir _i-1 , the more accurate the similar graphics will be, but at the same time it will increase the amount of calculation. It is recommended to take ±(5～15 ) is appropriate.

In addition, in the above-mentioned process of "using LBP algorithm to extract local features of images", the LBP feature information is a texture operator with invariant gray scale, which is obtained from the general definition of texture in the local field. Since LBP is very mature The image texture feature extraction algorithm belongs to the scope of the prior art, so it will not be repeated in the present invention.

3. The calculation process of the speed information is as follows:

Estimate the speed of the vehicle based on the difference of the time stamp information in the data packet, the offset of the image, and the relative position information of the cameras on both sides of the vehicle, which mainly includes: lateral velocity estimation, longitudinal velocity estimation and yaw angular velocity estimation, each estimation process details as follows:

1. Lateral velocity estimation:

Taking the lateral velocity value estimated by the vehicle speed estimator in the side-view mirror as an example, the specific process is as follows:

Δt=stamp _i -stamp _i-1

In the above formula:

Δt is the time difference between two image samples;

offset_y _l and offset_y _r are the lateral offsets of similar areas on the left and right sides of the vehicle, respectively.

It should be noted that, theoretically, the lateral velocity estimated by the vehicle speed estimator located in the side mirrors on both sides of the vehicle should be the same, but there will inevitably be a certain deviation, so the average value of the left and right lateral velocity estimates is finally taken as the lateral velocity of the vehicle.

2. Longitudinal velocity estimation:

The longitudinal velocity estimation of the vehicle is basically the same as the lateral velocity estimation method, as follows:

Δt=stamp _i -stamp _i-1

In the above formula:

Δt is the time difference between two image samples;

offset_x _l and offset_x _r are the longitudinal offsets of similar areas on the left and right sides of the vehicle, respectively.

The difference from the estimation of the lateral velocity of the vehicle is that, theoretically, the longitudinal velocity estimated by the vehicle speed estimator located in the rear view mirror on both sides of the vehicle is not necessarily the same, but the average of the left and right longitudinal velocity estimates is still taken here Value as the longitudinal velocity of the vehicle.

3. Estimation of yaw rate:

The specific process of estimating the vehicle yaw rate is as follows:

Δt=stamp _i -stamp _i-1

In the above formula:

ω is the yaw rate of the vehicle;

offset_x _l and offset_x _r are the longitudinal offsets of similar areas on the left and right sides of the vehicle, respectively;

l is the distance of the vehicle speed estimator on the left and right sides of the vehicle, which can be obtained by measurement.

It should be noted that, in the above vehicle speed estimation method, the LBP processing extraction process of the image information and the calculation process of the four offsets of offset_x _l , offset_x _r , offset_y _l and offset_y _r are all completed in the vehicle speed estimation module, and the rest of the calculation The process is completed in the vehicle ECU.

Claims (7)

1. a kind of speed estimator based on machine vision, it is characterised in that：

The speed estimator by prism and integrated is arranged in one piece of PCB printed circuit in the side mirror of vehicle Camera, laser emitter, drive module, speed estimation module on plate and communication module composition；

The prism is arranged in the front end of PCB printed circuit boards, and is fixed on the bottom of side mirror；The laser emitter hair Go out light source and expose to prism, on the one hand laser light source enters camera after prism reflects, on the other hand shone through prism It is mapped to ground, ground image information reaches camera after prism reflects；The drive module respectively with laser emitter and Camera signals connect；The image signal output end mouth of the camera and the picture signal receiving port signal of speed estimator Connection；The signal output port of the speed estimation module and the receiver port signal of communication module connect, the communication The signal output port of module is connect with the receiver port signal of vehicle ECU, and speed estimation module is to the image that is received Handling result is transmitted to vehicle ECU by information after being handled through communication module；

The filming frequency of the camera is identical as the tranmitting frequency of laser emitter.

2. a kind of speed estimator based on machine vision as described in claim 1, it is characterised in that：

The camera is narrow angle camera.

3. a kind of speed estimator based on machine vision as described in claim 1, it is characterised in that：

The communication module uses the data-signal that CAN bus technology sends out speed estimation module and Car Electronic Control system ECU is communicated.

4. a kind of vehicle, it is characterised in that：

The vehicle both sides are equipped with the speed estimator based on machine vision as described in claim 1.

5. a kind of vehicle speed estimation method based on machine vision, it is characterised in that：

The vehicle speed estimation method is acquisition of the vehicle speed estimation method by image, image based on vehicle described in claim 4 Processing and velocity information calculate three parts composition, it is specific as follows：

One, the gatherer process of described image is as follows：

Ambient light is detected first, then data are acquired；

The detection process of ambient light is specific as follows：

A1：Vehicle launch, camera acquires an image, and image information is sent to automobile speedestimate module and carries out ambient lighting The calculating of intensity；

A2：Automobile speedestimate module extracts the illumination intensity information of all pixels of image, filters out intensity and surrounding pixel illumination The apparent noise of intensity difference, residual pixel carry out intensity of illumination averaging and handle to obtain an environment illumination intensity value P_env；

A3：By environment illumination intensity value P_envIt is compared with preset environment illumination intensity threshold values, and then it is determined whether to enable swash Optical transmitting set carries out visual aids acquisition；

The gatherer process of data is as follows：

It controls camera and high-definition shooting is carried out to ground image information below with preset frequency, camera is by first image Data information speed estimation module is transmitted in the form of a data packet；

The data structure of the data packet is as follows：Include a unsigned integer variables timestamp stamp as unit of millisecond Time for records photographing；One unsigned int two-dimensional array data is that image is each for storing image LBP information, data The intensity value of a pixel；One integer variable dir is used to record the direction of vehicle, and dir ＜ 0 represent direction of advance, 0 generations of dir ＞ Table direction of retreat；For one floating type variable hight for storing elevation information of the camera apart from ground, unit is rice；

Two, the processing procedure of described image is as follows：

Described image processing procedure carries out in speed estimation module, and concrete processing procedure is as follows：

B1：The data packet obtained in aforementioned image acquisition process is parsed, extract data packet stamp, data, dir and Tetra- variables of hight enter memory；

B2：The extraction for carrying out local feature to image using LBP algorithms, obtains LBP information, wherein being neglected during extraction feature Slightly fringe region；

B3：If camera acquired image is the image of first shooting, i.e., no data in caching at this time then directly will LBP information deposit caching；If camera acquired image is not the image of first shooting, i.e., there is number in caching at this time According to the topography's information then extracted in memory is compared with the image feature information in current cache；

Assuming that the data structure of first data packet is (stamp₁, data₁, dir₁, hight₁), the data knot of i-th of data packet Structure is (stamp_i, data_i, dir_i, hight_i)；And assume to be i-th of data packet in memory at this time, it is (i-1)-th number in caching According to packet；According to dir_i-1Value carry out posteriorly or anteriorly traversal formula search, find the most like region of feature, finally obtain figure The offset offset_x and offset_y of picture, wherein offset_x is vertical misalignment amount, and offset_y is transversal displacement.

Three, the calculating process of the velocity information is as follows：

Relative position information according to the difference of timestamp information, the offset of image and vehicle both sides camera in data packet Speed is estimated, wherein including mainly：Lateral velocity estimation, longitudinal velocity estimation and yaw velocity estimation；

The lateral velocity estimation detailed process of vehicle is as follows：

Δ t=stamp_i-stamp_i-1

In above-mentioned formula：

Δ t is the time difference of two image samplings；

offset_y_lAnd offset_y_rThe respectively transversal displacement of left and right vehicle wheel both sides similar area；

The final lateral velocity for taking the average value of the two lateral velocity estimated values in left and right as vehicle；

The longitudinal velocity estimation detailed process of vehicle is as follows：

Δ t=stamp_i-stamp_i-1

In above-mentioned formula：

Δ t is the time difference of two image samplings；

offset_x_lAnd offset_x_rThe respectively vertical misalignment amount of left and right vehicle wheel both sides similar area；

The final longitudinal velocity for taking the average value of the two longitudinal velocity estimated values in left and right as vehicle；

The method of estimation detailed process of yaw rate is as follows：

Δ t=stamp_i-stamp_i-1

In above-mentioned formula：

ω is the yaw velocity of vehicle；

offset_x_lAnd offset_x_rThe respectively vertical misalignment amount of left and right vehicle wheel both sides similar area；

L is the distance of the speed estimator of left and right vehicle wheel both sides；

In above-mentioned vehicle speed estimation method, the LBP processing extraction process and offset_x of image information_l、offset_x_r、offset_ y_lAnd offset_y_rThe calculating process of four offsets is completed in the speed estimation module, remaining calculating process is in vehicle It is completed in ECU.

6. a kind of vehicle speed estimation method based on machine vision as claimed in claim 5, it is characterised in that：

In the detection process of the ambient light, in step A3, by environment illumination intensity value P_envWith preset environment illumination intensity valve Value is compared, and then it is determined whether to enable the detailed process of laser emitter progress visual aids acquisition is as follows：

Default environment illumination intensity upper limit threshold values is P_high, environment illumination intensity lower limit threshold values is P_low, wherein P_high＞ P_low；When When vehicle launch acquire for the first time, i.e., when acquired image is first image, by obtained environmental light intensity Angle value P_envWith (P_high+P_low)/2 are compared, if P_envMore than (P_high+P_lowLaser emitter is then closed in)/2, otherwise will Open laser emitter；Next, when camera carries out second of acquisition or subsequent acquisition, several acquired images are non-first When opening image, when laser emitter is closed at this time, if P_envLess than P_low, i.e. P_env＜ P_low, then laser emitter opening, no Then laser emitter continues to remain off；When laser emitter is opened at this time, if P_envMore than P_high, i.e. P_env＞ P_high, then laser emitter closing, otherwise laser emitter continues to be kept open.

7. a kind of vehicle speed estimation method based on machine vision as claimed in claim 5, it is characterised in that：

In the processing procedure of described image, in step B3, topography's information in memory and the image in current cache are extracted The detailed process that characteristic information is compared is as follows：

Take inten=0xffffffff, offset_x=offset_y=0；Extract the local feature of the image in current cache Information, obtains LBP histograms, and data are stored in variable list, i.e. data_i-1Local message deposit it is data cached, wait in next step do Comparison uses；

Under vehicle axis system, in longitudinal movement dir_i-1A pixel moves laterally to the top of image, obtains area to be searched Domain；The extraction that region of search carries out LBP histogram informations is treated, data are stored in variable list ', list ' in memory；Seek list With the standard deviation inten ' of the difference of two lists of list ', i.e., by image feature data in memory and characteristics of image number in caching According to being compared, image feature data and image feature data similarity in caching are higher in the smaller proof memory of standard deviation；It is right Than inten ' and inten, inten=inten ' is enabled if inten ' ＜ inten, and current offset is charged to currently Offset_x and offset_y, does not otherwise deal with；After the completion of processing, region to be compared moves down | dir_i-1| a unit is again It is secondary to proceed as described above, until proceeding to image lowermost end；

After the completion of the longitudinal comparison of one row, then by transverse shifting dir in region to be searched_i-1A pixel is opened from image the top again Begin, scans for comparing to image bottom；It and so on carries out, finally obtains in the image feature information and caching in memory Image the immediate region of local feature information offset offset_x and offset_y；

So far one cycle is completed, by the data dump in caching, and the image feature data in most like region is stored in and is cached Comparison for recycling next time.