CN107563371B

CN107563371B - Method for dynamically searching interesting region based on line laser light strip

Info

Publication number: CN107563371B
Application number: CN201710573668.9A
Authority: CN
Inventors: 刘巍; 叶帆; 张致远; 赵海洋; 兰志广; 张洋; 马建伟; 贾振元
Original assignee: Dalian University of Technology
Current assignee: Dalian University of Technology
Priority date: 2017-07-17
Filing date: 2017-07-17
Publication date: 2020-04-07
Anticipated expiration: 2037-07-17
Also published as: CN107563371A

Abstract

The invention discloses a method for dynamically searching an interesting area based on a line laser light strip, belongs to the technical field of computer vision measurement, and relates to a method for dynamically searching an interesting area based on a line laser light strip. The method predicts the interesting region of the optical strip by adopting a method of quickly calculating the motion parameters of the optical strip in groups, firstly shoots a group of time sequence laser optical strip scanning images, extracts the interesting region of a measured object by using a polygon, and extracts the interesting region of a first optical strip by using a rectangle. And then grouping all the images based on the laser rotation angle, adopting transverse edge detection on the mixed differential image, quickly calculating the inter-frame pixel speed of the light bars in each group of images, and dynamically extracting the interested area of the light bars in the images according to the inter-frame pixel speed. The method can accurately define the uniform motion range of the line laser light strip by grouping the sequence images, adapts to the condition of variable-speed motion of the light strip and improves the calculation efficiency of the motion parameters of the light strip. The method has high efficiency and high reliability.

Description

Method for dynamically searching interesting region based on line laser light strip

Technical Field

The invention belongs to the technical field of computer vision measurement, and relates to a method for dynamically searching an interested area based on a line laser light strip.

Background

Generally, in a computer vision measurement system, image feature extraction can be decomposed into three parts: target detection, searching system region of interest (ROI); image segmentation, separating the target from the background; and extracting target features. The target detection is used as a precondition for image feature extraction, and the subsequent image processing is directly influenced by the search quality of the target detection. For a continuously moving object in a large number of time sequence images, the rapid and accurate ROI is the key to improve the quality and efficiency of image feature extraction.

The existing target detection methods mainly comprise a background segmentation method, an adjacent frame difference method, an optical flow method, a wavelet method and the like. The background segmentation method is used for segmenting a background and a target by establishing a background model and comparing characteristic parameters of an image sequence with the background model so as to obtain a moving target, but the method is only suitable for occasions with fixed moving scenes and simpler occasions because no high-performance rule defines the target. The adjacent interframe difference method is to perform difference operation on two adjacent frames of images, and judge whether motion exists or not according to the absolute value of the difference. The optical flow method infers the moving speed and direction of an object by detecting the change condition of the gray value of the image pixel points along with time, and is not suitable for a low frame rate camera or a high-speed moving object. The wavelet method is to perform wavelet transformation on an image and then process the wavelet image by adopting methods such as band-pass filtering and the like to obtain a target area, and has the advantages that a weak target in a complex scene can be detected, but the efficiency and the reliability are poor. A dynamic ROI method based on a K-means clustering algorithm is provided in Feng L, Po L M, XuX, et al. The invention discloses a method and a system for setting traffic information monitoring equipment, which is invented by Zhao Shi Yuan et al and has a patent number of CN201710156308.9, wherein the method and the system are used for sequentially combining the traffic information monitoring equipment based on the proximity relation among initial interested areas to obtain new interested units and calculating the number of the monitoring equipment required by vehicle information monitoring.

Disclosure of Invention

The invention provides a method for dynamically searching an interested area based on line laser light bars, aiming at solving the technical problems that a large number of line laser light bar images are high in noise, complex and small in target area, and the problems of low extraction efficiency, low fault tolerance rate, poor robustness and the like of a traditional interested area extraction method. According to the method, for a time sequence image of a group of shot vertical line laser light bars moving transversely, a region of a measured object is extracted based on a polygonal ROI method, the light bars of an initial image are extracted based on a rectangular ROI method, then the images are uniformly grouped to ensure that the speed between the light bars of the same group of images is constant, transverse edges of mixed differential images are convoluted, the moving speed of each group of light bars is rapidly calculated, finally the positions of the light bars are predicted based on the moving parameters of the light bars in the region of the measured object, and dynamic region-of-interest extraction is carried out. The method can adapt to the condition of line laser variable speed scanning by grouping the sequence images, and improves the correctness and robustness of the extraction of the light bar region; the differential image after the transverse edge convolution is quickly analyzed, so that the calculation efficiency and the reliability of the light strip motion parameters are greatly improved, and the efficiency of the method is ensured.

The technical scheme adopted by the invention is a method for dynamically searching the interesting region based on a line laser light bar, which is characterized in that the interesting region of the light bar is predicted by adopting a method of quickly calculating the motion parameters of the light bar in groups; firstly, shooting a group of time sequence laser light bar scanning images, extracting an interested area of a measured object by using a polygon, and extracting an interested area of a first light bar by using a rectangle; then grouping all the images based on the laser turning angle, adopting transverse edge detection on the mixed differential image, and quickly calculating the inter-frame pixel speed of light bars in each group of images; finally, dynamically extracting the interest area of the optical strips in the image according to the inter-frame pixel speed; the method comprises the following specific steps:

first step global polygon ROI and initial ROI of light bar

The method comprises the steps that a vision measuring system based on line laser light bars is built, a line laser 2 installed on a rotary table transversely scans a measured object 1 through rotary motion, the line laser bars are vertically projected onto the measured object 1, the laser rotates along with the rotary table at an angular velocity of omega, the laser light bars on the measured object move from the position of a first light bar 3 to the position of a last light bar 4, the movement velocity of the light bars is v, the rotation angle of the laser in a corresponding process is theta, and a camera shoots a group of time sequence images;

for the time-series images, the number of images is N, the size of each image is U × V, and the ith image in the group of images is g (i), i is 1,2, …, N; the arithmetic mean is carried out on the first image and the last image, and the image G containing the background and the first light bar and the last light bar is obtained by calculation according to the formula (1)_r；

Determination of G_rUsing a polygon to roughly extract the edge of the object to be measured to obtain a polygon region of interest, and recording the polygon region of interest as ROI _ poly; crude extraction of G_rNeglecting the vertical length and the horizontal coordinate of the rectangle in the image to obtain the horizontal coordinate u of the upper left corner point of the two rectangles₁And u₂Width of rectangle in image₁And width₂(ii) a Calculating an initial ROI area RECV of the light bars, i.e. the left border of the first light bar is u, according to equation (2)₁Width w equals max (width)₁,width₂)；

RECV＝{u₁,max(width₁,width₂)} (2)

Second step of calculating line laser light stripe motion parameters based on grouped images

When the light bars move at a uniform speed in a small rotation angle range, N time sequence images are averagely divided into theta groups, wherein the j group comprises images G (j,1), G (j,2), … and G (j, N)_j) J is 1,2, …, θ. Wherein n is_jIs the number of images in the jth group and

theta degree is the total rotation angle of the line laser and theta is a positive integer;

for the jth image, the 1 st and nth images are processed according to the formula (3)_jDifferencing the images to obtain a mixed differential image G containing only two light bars_j；

G_j＝abs(G(j,n_j)-G(j,1)) (3)

For image G according to equation (4)_jConvolution, calculation G_jTransverse gradient image G of_jx；

Obtaining G_jxThe gray values of the pixels in the middle 0.4V,0.45V,0.5V,0.55V and 0.6V lines are marked as G_jx(K, l), wherein l ═ 1,2, …, U, K ∈ K, K ═ {0.4V,0.45V,0.5V,0.55V,0.6V }; k takes five different values of K, with l as abscissa and G_jx(k, l) drawing five histograms for the vertical coordinate, and recording as the k-th row histogram according to the value of k; searching the average abscissa corresponding to two peak clusters in the k-th row histogram

And

calculating the pixel speed v between adjacent frames of the jth group of image light bars according to the formula (5)_j；

Third step light bar dynamic ROI

From the result of the first step, the left boundary of the ROI of the first image light bar in the first group is known as u₁₁＝u₁Width w, right boundary u₁₁+ w; calculating according to formula (6)ROI left boundary u of light bar of p-th image in j groups_jpThen the right boundary is u_jp+w；

Extracting a polygonal interested area of the p image in the j group according to the ROI _ poly, and enabling the gray values of all pixels in other areas to be 0; calculating the dynamic ROI parameters of the light bar image, selecting the region of interest as a rectangle, and four corner points as (u)_jp,0)、(u_jp+w,0)、(u_jp+ w, V) and (u)_jpV); and a rectangle formed by the four corner points is used as the interested area of the jth image in the jth group. The region of interest of the light bars in all N images can thus be obtained.

The method has the advantages that by grouping the sequence images, the uniform motion range of the line laser light bars can be accurately defined, the method adapts to the situation of variable-speed motion of the light bars, and the fault tolerance rate and the reliability of the light bar extraction method are improved; on the basis of the traditional difference method idea, motion prediction information is added, and a difference image after transverse edge convolution is quickly analyzed, so that the calculation efficiency and reliability of light stripe motion parameters are improved, the extraction efficiency of the region of interest is improved, and the extraction accuracy and robustness of the light stripe region are improved.

Drawings

Fig. 1 is a schematic view of a vision measurement system based on vertical line laser light bars. In the figure, 1 is the object to be measured, 2 is a line laser, 3 is a first optical stripe, 4 is a last optical stripe, ω is the rotation angular velocity of the laser, θ is the laser rotation angle, and v is the movement velocity of the optical stripe on the object to be measured.

FIG. 2 is a flow chart of a line laser light bar based dynamic ROI method.

Detailed Description

The following detailed description of the invention refers to the accompanying drawings.

In this embodiment, the surface of the object to be measured 1 is a t800 composite material plate of 2.5m × 3.0m, a blue-violet laser with a wavelength of 460nm is installed on a turntable, the laser is vertically projected onto the object to be measured 1, and the laser scans transversely through rotary motion. The laser 2 rotates with the turntable at an angular velocity of ω, the laser light stripe on the object to be measured 1 moves from the position of the first light stripe 3 to the position of the last light stripe 4, the moving speed of the light stripe is v, the rotation angle of the laser in the corresponding process is θ, and a group of time series images are captured by the camera, as shown in fig. 1.

The invention adopts a camera provided with a wide-angle lens to shoot light bar images. The camera model is a view works VC-12 MC-M/C65 camera, and the resolution is as follows: 4096 × 3072, image sensor: CMOS, frame rate: full frame, maximum 64.3fps, weight: 420 g. The wide-angle lens is EF 16-35mm f/2.8L II USM, the parameters are as follows, and the lens focal length is as follows: f is 16-35mm, APS focal length: 25.5-52.5, aperture: f2.8, lens size: 82X 106. The shooting conditions were as follows: the picture pixels are 4096 × 3072, the focal length of the lens is 25mm, the object distance is 750mm, and the field of view is about 850mm × 450 mm.

FIG. 2 is a flow chart of a line laser light bar based dynamic ROI method. According to the operation flow, the whole target detection process comprises three steps of a global polygon ROI and an initial ROI of an optical strip, calculation of line laser optical strip motion parameters based on a grouped image, dynamic ROI of the optical strip and the like.

First step global polygon ROI and initial ROI of light bar

First, a vision measuring system based on a line laser light bar is built, and as shown in fig. 1, a group of time series images are shot by a camera. The background of all the images is unchanged, the laser light bar in each image crosses the upper and lower boundaries of the measured object, and the shooting time difference of any two adjacent images is a constant value.

For the group of composite material laser light bar images obtained by shooting, the number of the images is 200, and the size of each image is 4096 × 3072. Note that the ith image in the group of images is g (i), where i is 1,2, …, 200. Calculating an image G containing background and head and tail light bars according to formula (1)_r. Manual extraction of G Using polygonal ROI_rThe detected area of the medium composite board is stored as ROI _ poly. Manual extraction of G Using rectangular ROI_rRectangular areas of the middle head light bar and the tail light bar are obtained to obtain the upper left corner point u of the light bars₁₁＝u₁Obtaining the width of the two rectangular areas in the transverse direction of the image₁And width₂The larger value is taken as the lateral width w of the dynamic ROI as max (width)₁,width₂)。

It is known that the line laser rotation angle is 20 ° when the above-mentioned 200 time-series images are taken. These images are divided into 20 groups, and the j-th group includes images G (j,1), G (j,2), …, G (j,10), j being 1, 2.

For the jth group of images, calculating a transverse gradient image G of head-tail difference images in the group according to formulas (3) and (4)_jx. With G_jxThe gray scale value of 4096 pixels in the 1536 th row is ordinate, and each pixel is in G_jxThe value of the middle horizontal coordinate l is plotted as the horizontal coordinate (l ═ 1, 2.., 4096). As shown in fig. 1, the histogram includes four peaks, which represent the positions of the beginning and the end of the jth group of two light bars at the left and right boundaries of the 1536 th row, and the two peaks with close distances are used as peak clusters to calculate the average value of the abscissa of the two peaks in each peak cluster

And

drawing a k-th row histogram according to the method, and searching an average abscissa corresponding to two peak clusters in the k-th row histogram

And

where K is equal to K, K is {1229,1382,1536,1690,1843 }. Calculating the pixel speed v between adjacent frames of the jth group of image light bars according to the formula (5)_j. V is calculated according to the method described above₁,v₂,...,v₁₀。

Third step light bar dynamic ROI

Knowing the first image in the first groupROI left boundary of light bar is u₁₁＝u₁And the width is w. Calculating the ROI left boundary of the light bar of the p image in the j group as u according to the formula (6)_jpThen the right boundary is u_jp+w。

And extracting a polygonal interested area of the p image in the j group according to the ROI _ poly, and enabling the gray values of all pixels in other areas to be 0. Calculating the dynamic ROI parameters of the light bar image, selecting the region of interest as a rectangle, and four corner points as (u)_jp,0)、(u_jp+w,0)、(u_jp+ w,3072) and (u)_jp3072); and a rectangle formed by the four corner points is used as the interested area of the jth image in the jth group. The region of interest of the light bars in all images can thus be obtained.

Aiming at a large number of line laser light bar images with high noise, complexity and small target areas, the region where the target is located cannot be found out quickly and accurately by the conventional interested area extraction method. The method comprises the steps of transversely scanning a group of images of a measured object based on shooting of vertical line laser light bars, extracting light bars of a measured object region and an initial image by adopting a polygonal ROI (region of interest) and rectangular ROI (region of interest) method, then grouping the images, rapidly calculating the movement speed of each group of light bars, finally predicting the positions of the light bars in the measured object region based on the movement parameters of the light bars, and extracting a dynamic region of interest. The invention can adapt to the condition of line laser variable speed scanning, improves the calculation efficiency of the movement parameters of the optical strips, and has the characteristics of high fault tolerance rate, high efficiency, high reliability and the like.

Claims

1. a method for dynamically searching a region of interest based on a line laser light bar, is characterized in that, the method adopts the method of grouping calculation light bar motion parameters to predict the region of interest of the light bar; at first photograph a group of time-series laser light bars Scan the image, use polygons to extract the area of interest of the object to be measured, and use rectangles to extract the area of interest of the first light bar; then group all images based on the laser corners, and use lateral edge detection for the mixed difference image, and calculate the The inter-frame pixel speed of the light bar; finally, the region of interest of the light bar in the image is dynamically extracted according to the inter-frame pixel speed; the specific steps of the method are as follows:

The first step is the global polygon ROI and the initial ROI of the light bar

A visual measurement system based on a line laser light bar is built. The line laser (2) installed on the turntable scans the measured object (1) horizontally through a rotating motion, and the line laser bar is projected vertically on the measured object (1). As the turntable rotates at the angular velocity of ω, the laser light bar on the measured object moves from the position of the first light bar (3) to the position of the last light bar (4). The rotation angle is θ, and a set of time-series images are captured by the camera;

For this time series image, the number of images is N, the size of each image is U×V, and the ith image in this group of images is denoted as G(i), where i=1,2,...,N; The first and last two images are arithmetically averaged, as shown in formula (1), to obtain an image _Gr including the background and the first and last light bars;

Determine the edge of the object to be measured in Gr, use polygon to roughly extract the edge to obtain a polygonal region of interest, denoted as _{ROI_poly} _; roughly extract the rectangular boundaries of the first and last light bars on the object to be measured in Gr, ignore the rectangle in the image The vertical length and abscissa in , the horizontal coordinates u ₁ and u ₂ of the upper left corner of the two rectangles and the horizontal lengths width ₁ and width ₂ of the rectangle in the image are obtained; calculate the initial ROI area of the light bar according to formula (2) RECV, that is, the left border of the first light bar is u ₁ , and the width is w=max(width ₁ , width ₂ );

RECV={u ₁ ,max(width ₁ ,width ₂ )} (2)

The second step calculates the motion parameters of the line laser light bar based on the grouped images

When the light bar moves at a uniform speed in the small turning angle range, the N time-series images are evenly divided into θ groups, and the jth group includes images G(j,1), G(j,2),...,G(j,n _j ), j=1,2,...,θ; where _nj is the number of images in the jth group and

θ° is the total rotation angle of the line laser and θ is a positive integer;

For the jth group of images, according to formula (3), the first image and the _njth image are obtained by difference, to obtain a mixed difference image _Gj that only contains two light strips;

G _j =abs(G(j,n _j )-G(j,1)) (3)

Convolve the image G _j according to formula (4), and calculate the lateral gradient image G _jx of G _j ;

Obtain the pixel gray value of row 0.4V, 0.45V, 0.5V, 0.55V, 0.6V in G _jx , denoted as G _jx (k,l), where l=1,2,...,U,k∈ K, K={0.4V, 0.45V, 0.5V, 0.55V, 0.6V}; k takes five different values in K, and draws five different values with l as the abscissa and G _jx (k, l) as the ordinate The histogram, according to the value of k, is recorded as the k-th row histogram; find the average abscissa corresponding to the two peak clusters in the k-th row histogram

and

Calculate the pixel speed v _j between the adjacent frames of the jth group of image light bars according to formula (5);

The third step light bar dynamic ROI

According to the result of the first step, it is known that the left boundary of the ROI of the first image light strip in the first group is u ₁₁ =u ₁ , the width is w, and the right boundary is u ₁₁ +w; The left border u _jp of the ROI of the light bar of the p-th image, then the right border is u _jp +w;

Extract the polygonal region of interest of the pth image in the jth group according to ROI_poly, and set the gray value of all pixels in other regions to 0; calculate the dynamic ROI parameters of the light strip image, select the region of interest as a rectangle, and the four corners are (u _jp ,0), (u _jp +w,0), (u _jp +w,V) and (u _jp ,V); the rectangle formed by these four corners is used as the Region of interest; thus obtains the region of interest of the light bars in all N images.