CN104835175B - Object detection method in a kind of nuclear environment of view-based access control model attention mechanism - Google Patents

Abstract

The invention discloses object detection methods in a kind of nuclear environment of view-based access control model attention mechanism, comprising: extracts to the brightness of general camera acquired image, color, direction character, obtains three feature significance figures；Fusion is weighted to above-mentioned Saliency maps, obtains weighting notable figure；Area-of-interest is obtained according to weighting notable figure and feature extraction is carried out to it；Extract the feature of γ camera combination chart；Property region interested is merged with combination chart with SIFT method, detects target position.The present invention extracts several area-of-interests using data-driven attention model from bottom to top, substantially reduces the calculation amount of later period matching process；It is combined with task-driven attention model from top to bottom again and establishes two-way visual attention model, the precision and treatment effeciency of objective area in image detection can be greatly improved, and matching process eliminates the interference in uncorrelated region in scene, and the operative goals extracted is made to have better robustness and accuracy.

Description

A Visual Attention Mechanism Based Object Detection Method in Nuclear Environment

technical field

The invention relates to the technical field of image information processing, in particular to a method for detecting objects in a nuclear environment based on a visual attention mechanism. The method specifically adopts a top-down and bottom-up bidirectional visual attention model, which can be used in special environments target detection.

Background technique

With the rapid development of information technology and the expansion of data, people have higher and higher requirements for the speed and accuracy of information processing. The ideal information processing process is that only part of the information related to the task needs to be processed, but the actual information processing process needs to process a lot of information that is not related to the task. Therefore, how to quickly find and process only the part of the information related to the task becomes very important.

The human visual attention mechanism provides a new research idea for quickly finding and processing the part of the information related to the task. Studies have shown that human vision has super information processing capabilities. Faced with various information that changes in real time, it can always respond to the most relevant parts in a timely manner, while automatically ignoring irrelevant parts. Apply the human visual attention mechanism to the field of image processing to form a visual attention mechanism in the field of image processing, which can process image information efficiently and accurately. Therefore, how to imitate the human visual attention mechanism in the process of image processing and quickly find the target area in the image is of great significance to the real-time performance of image processing.

At present, more and more researchers have invested in the research on how to quickly and accurately detect salient regions, and have developed many models, some of which are typical models:

1) Itti model: its main process is to extract various features from the input image, such as color, direction, brightness, etc., and form an attention map of each feature through the high-level pyramid and central peripheral operation operators, and then normalize and combine to obtain Significant figure. On this basis, the neural networks compete with each other through the winner-take-all neural network, so that the salient region wins. This method provides a good measure of local saliency. But it does not fully consider the global information of the image; and does not consider the needs of the actual task, and only belongs to the bottom-up unidirectional attention model.

2) Stentiford model: This method represents the salience of the image with a visual attention map. The basic idea is that when the feature of a certain region of the image appears less frequently in other regions of the image, the salience of the region will be higher; Regions of the same pattern get a visual attention map, which is used to represent saliency. This method considers the integrity of the target and measures the global saliency of the image, but the model still only belongs to the bottom-up attention model, which has no definite physical meaning, and does not judge the importance of the target according to the task.

3) HOAM model: This model uses intensity and orientation as early features to guide visual attention. The attention unit is not a certain point or a certain area in space, but a complete object with definite physical meaning. This method first needs to assume that the image has been divided into several physically meaningful objects or object combinations, so manual intervention is required.

Contents of the invention

Aiming at the above-mentioned problems existing in the prior art, the present invention provides a method for detecting objects in a nuclear environment based on a visual attention mechanism. Using this method, the accuracy of object region detection in an image can be greatly improved, and the extracted salient objects have Better robustness and accuracy, but also greatly improve the image processing efficiency.

In order to achieve the above object, the technical scheme that a specific embodiment of the present invention takes is:

S1. Obtain an image of the target collected by a common camera, extract brightness, color, and direction features of the image, and obtain a brightness feature map, a color feature map, and a direction feature map, respectively;

S2. Calculate the luminance feature map, color feature map and direction feature map through the Gaussian pyramid and the method of the central peripheral operator, and obtain 6 luminance parallax maps, 12 color parallax maps and 24 directional parallax maps;

S3. Perform normalization processing on 6 luminance disparity maps, 12 color disparity maps and 24 directional disparity maps respectively to obtain a luminance saliency map color saliency map and directional saliency map

S4. Select the most significant point from the saliency map, take the point as the salient point, and segment the corresponding feature saliency map by means of region growth to obtain the region of interest;

S5. Obtain an image containing radiation intensity distribution information collected by the gamma camera;

S6, respectively extracting the key points of the region of interest of the mixed image and the common camera image;

S7. Generating feature vectors from the key points respectively;

S8. Match the eigenvectors of the key points of the region of interest with the eigenvectors of the key points of the mixed image, and if the matching condition is met, the target is the job target.

The beneficial effect that method of the present invention produces is:

Firstly, by taking advantage of the bottom-up data-driven attention model, several regions of interest are directly extracted from images collected by common cameras, which greatly reduces the amount of computation in the later matching process; The mixed image of the scene grayscale is used for feature matching, and a two-way visual attention model combining top-down and bottom-up is established. Therefore, the accuracy of target area detection in the image is greatly improved, and the matching process eliminates the interference of irrelevant areas in the scene, so that the extracted salient targets have better robustness and accuracy, and also greatly improve Processing efficiency.

Description of drawings

Fig. 1 shows the flow chart of a technical scheme of a kind of target detection method in the nuclear environment based on visual attention mechanism of the present invention;

Fig. 2 shows the schematic diagram of the DOG difference pyramid formation process of the present invention;

FIG. 3 is a schematic diagram of extreme point detection of the DOG function of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings.

Referring to FIG. 1 , FIG. 1 is a flowchart of an embodiment S100 of a method for detecting salient objects in a nuclear environment based on a visual attention mechanism of the present invention; the embodiment S100 includes the following steps S1 to S8.

In step S1, the image of the target collected and preprocessed by the ordinary camera is obtained, the brightness, color and direction features of the image are extracted, and the brightness feature map, color feature map and direction feature map are respectively obtained.

In step S2, the brightness feature map, color feature map and direction feature map are calculated by the method of Gaussian pyramid and central peripheral operator, and 6 brightness disparity maps, 12 color disparity maps and 24 direction disparity maps are obtained respectively.

In one embodiment of the present invention, the brightness, color and direction features of the image are obtained by using the sampling method of the difference between the center and the periphery. Each feature is computed by a set of linear "middle-periphery" operators similar to the visual perception area.

In one embodiment of the present invention, the central and peripheral scales can be set as follows: the central scale c∈{2,3,4}, the peripheral scale is correspondingly s=c+δ, δ∈{3,4} . The intersecting scale difference of the two graphs is obtained by fine-scale interpolation and point-to-point subtraction, using different scales to obtain multi-scale feature extraction.

Use r, g, b to represent the pixel values of the red, green, and blue color channels of the input image, which are used to build a Gaussian pyramid I(x), and x=[0...8] represents the scale level. The Gaussian pyramid image is a series of image collections formed by Gaussian filtering of an image, and the resolution will gradually decrease as the number of Gaussian filtering increases. The bottom layer of the pyramid is the unfiltered image, with the highest resolution, while the top layer is a lower resolution representation of the image. In this way, the image pyramid is obtained by taking the mean value of the pyramid images of the three color channels, as shown in formula (1):

Through the difference between the central and peripheral images at different scales, six brightness maps of the central and peripheral difference structures are obtained, as shown in formula (2):

I(c,s)=|I(c)-I(s)| (2)

where c∈{2,3,4}, δ∈{3,4}, s=c+δ;

Then obtain the color feature map, first extract the components of the four color channels red, green, blue, and yellow from the image, as shown in formulas (3) to (6):

Red R=r-(g+b)/2 (3)

Green G＝g-(r+b)/2 (4)

Blue B=b-(r+g)/2 (5)

Yellow Y=(r+g)/2-|r-g|/2-b (6)

12 mid-peripheral structural color maps are obtained through their differences, as shown in formulas (7)-(8):

RG(c,s)=|(R(c)-G(c))-(G(s)-R(s))| (7)

BY(c,s)=|(B(c)-Y(c))-(Y(s)-B(s))|; (8)

Then obtain the direction feature map, the brightness pyramid image I(x) is convolved with the commonly used Gabor direction filter, the direction of the image can be obtained, and 24 middle-peripheral structure direction maps are obtained through the difference, as shown in formula (9) :

O(c,s,θ)=|O(c,θ)-O(s,θ)| (9)

where θ = {0°, 45°, 90°, 135°}.

So far, 6 brightness feature maps, 12 color feature maps and 24 direction feature maps have been obtained from the pictures collected by ordinary cameras according to the saliency map method.

In step S3, normalize the 6 luminance disparity maps, 12 color disparity maps and 24 directional disparity maps respectively to obtain the luminance saliency map color saliency map and directional saliency map The fusion of feature maps provides a bottom-up input to the saliency map, which can be modeled as a dynamic neural network.

Since the difference between the center and the periphery can best reflect the saliency of the image, after obtaining the attention map, there will be a situation where there are multiple contrast maxima for a certain feature, and a large number of significant peaks will appear at this time. Therefore, in one embodiment of the present invention, the three sets of feature maps are normalized separately before merging the attention maps to generate the saliency map. Through normalization and cross-scale addition, the feature maps are integrated into three saliency maps of brightness, color and direction to obtain To measure the saliency of objects in an image, it is necessary to synthesize the saliency images of the three channels. Here we normalize the saliency maps of the integrated three channels.

In one embodiment of the present invention, based on the top-down idea, a weighted feature map fusion algorithm is proposed, as shown in formula (10):

Wherein, w _t +w _c +w _o =1, so that the value range of S is still normalized to a certain range. When the feature of a certain channel of the predicted image is sensitive, the weights w _t , w _c and w _o can be adjusted adaptively. This model uses a salient region detection algorithm based on local comparison to obtain the region of interest, selects the most salient point from the saliency map, takes this point as a salient point, and uses the region growing method to segment the corresponding feature saliency map to obtain the sense The region of interest, the obtained saliency map region is more targeted.

In one embodiment of the present invention, the specific process of normalizing the 6 brightness disparity maps, 12 color disparity maps and 24 direction disparity maps is as follows:

Set a normalization operator N(.) to improve the quality of the graph. The calculation process of the normalization operator N(.) is as follows:

Normalize the pixel values of each channel feature map to a fixed interval [0, M], where M is a positive integer;

Find the position of the global maximum M in the graph, and calculate the mean value of the local maximum of all other feature maps

The feature map is globally multiplied by

Through the normalization operator N(.) and cross-scale addition, the feature map is integrated into three saliency maps of color, brightness and direction;

Color normalized feature map:

Brightness normalized feature map:

Orientation normalized feature map:

in, The feature map of each feature is down-sampled on different scale layers to obtain the highest main scale layer, and then the addition operation is performed to obtain the saliency map on the three features of color, brightness, and direction.

In step S4, the most salient point is selected from the saliency map, and this point is taken as a salient point, and the corresponding feature saliency map is segmented by region growing to obtain the region of interest.

In an embodiment of the present invention, in order to enhance the stability of matching, in addition to using the main direction, an auxiliary direction may also be selected. The auxiliary direction is defined as: in the histogram, when the value of a certain direction is greater than or equal to 80% of the main peak value, then this direction is taken as the auxiliary direction of the key point. A key point generally has a main direction and multiple auxiliary directions.

In step S5, the image collected by the gamma camera and containing the radiation intensity distribution information is acquired.

In step S6, the key points of the region of interest of the gamma camera image and the ordinary camera image are extracted respectively.

In one embodiment of the present invention, for a two-dimensional grayscale image (such as the grayscale image of the region of interest of an ordinary camera and the mixed image obtained by a gamma camera), the representation of the scale space at different scales can be represented by the image and the Gaussian kernel Convolution is obtained, as shown in formula (14):

L(x,y,σ)=G(x,y,σ)*I(x,y) (14)

In the formula, G(x, y, σ) is a variable-scale Gaussian function, as shown in formula (15):

Among them, x and y are the horizontal and vertical coordinates of the image, and σ represents variable scale.

The scale-normalized Laplacian function σ ² ▽ ² G is scale-invariant and produces the most stable image features. The scale-normalized Laplacian of Gaussian is shown in formula (16):

Let DOG(x,y,σ)=G(x,y,kσ)-G(x,y,σ), then

DOG(x,y,σ)=G(x,y,kσ)-G(x,y,σ)≈(k-1)σ ² ▽ ² G

The left side of the equation is the Gaussian difference operator (DOG), and the scaling factor (k-1) does not affect the position of the extreme point, so the Gaussian difference operator is similar to the scale-normalized Laplacian difference operator.

In one embodiment of the present invention, Gaussian difference operators of different scales are used to convolve the image, as shown in formula (17):

D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y ,σ) (17)

Fig. 2 is a schematic diagram of the formation process of the DOG differential pyramid of the present invention, the left part of the figure is a Gaussian pyramid image obtained at different scales, and the right is a differential pyramid image obtained by difference between adjacent scales.

It can be seen from the above that obtaining the local extremum point of the DOG space can obtain stable image features, and the extremum point of the DOG function can be compared with the 26 adjacent points around it to judge whether it is a local extremum point.

FIG. 3 is a schematic diagram of extreme point detection of the DOG function of the present invention. The detected point in the middle is compared with its 8 adjacent points of the same scale and 9×2 points corresponding to the upper and lower adjacent scales, a total of 26 points, to ensure that extreme points are detected in both the scale space and the two-dimensional image space.

The method of deriving the extreme value of the spatial scale function is adopted, and the threshold is set to eliminate some low-contrast and unstable points, so as to determine the position of the extreme point.

Use the DOG function to expand Taylor in the scale space, as shown in formula (18):

Among them, X=(x,y,σ) ^T is the coordinates of extreme points detected in the previous step.

Take the derivative of the above formula and make it zero to get the position coordinates of the key points:

and put it into the Taylor expansion:

set by Threshold, eliminate points smaller than the threshold.

The DOG function has a strong edge response at the edge of the image, so it also needs to exclude the edge response. The edge response can be excluded by calculating the Hessian matrix in the 3×3 window around the scale of the point’s location, and its calculation is shown in formula (19):

Let α be the largest eigenvalue, and β be the smallest eigenvalue, then α=rβ, as shown in formulas (20)~(22):

Det(H)＝D _xx D _yy -(D _xy ) ² ＝αβ (21)

Tr(H)= _Dxx + _Dyy =α+β (22)

(r+1) ² /r reaches the minimum when the two eigenvalues are equal, and increases with the increase of r. Therefore, it only needs to be limited after setting r, as shown in formula (23):

In step S7, the key points are respectively generated into feature vectors.

In one embodiment of the present invention, the generation process of feature vector is as follows:

1) First determine the transformation parameters for extracting key point images;

2) Move the coordinates of key points to the main direction;

3) In the 16*16 area centered on the key point, calculate the gradient histogram in 8 directions for each 4*4 area, count the cumulative value of each gradient, form a seed point, and generate 16 seeds in total point, 128-dimensional vector;

4) Thresholding and vector normalization are performed on the obtained feature vectors, and the normalized feature vectors are as follows:

L=(l ₁ ,l ₂ ,...,l ₁₂₈ )

In one embodiment of the present invention, using the gradient direction distribution characteristics of the pixels in the neighborhood of the key point, a direction is specified for each key point, and the key point descriptor is characterized relative to this direction, so that the key point descriptor rotates the image is immutable.

In one embodiment of the present invention, utilize gradient mathematical model to specify direction for key point, gradient mathematical model is as shown in formula (24):

The magnitude of the gradient is shown in formula (25):

The direction of the gradient is shown in formula (26):

In one embodiment of the present invention, sampling is performed in a neighborhood centered on the key point, and the direction of the neighborhood pixels is counted using a histogram method. The direction range of the gradient histogram after statistics is 0° to 360°, and every 10° is regarded as a column for analysis, including 36 columns in total. The histogram peak represents the main direction of the gradient at the key point, which is taken as the direction of the key point.

So far, the key points of the image have been detected, and each key point has three information: position, scale, and direction.

In step S8, match the eigenvectors of the key points of the region of interest in the ordinary camera image with the eigenvectors of the key points of the mixed image, and if the matching conditions are met, the target is a salient target.

The grayscale image of the region of interest of the ordinary camera and the feature points of the mixed image obtained by the gamma camera have been characterized as feature vectors, so the matching of the gray image of the interest region and the feature points of the mixed image obtained by the gamma camera can be It is judged by the similarity of two feature vectors.

In one embodiment of the present invention, key point description subsets are respectively established for the grayscale image of the region of interest and the mixed image obtained through the gamma camera. Target recognition is accomplished by comparing key point descriptors in two point sets. The similarity measure of keypoint descriptors with 128 dimensions adopts Euclidean distance.

Descriptor of key points in the template graph, R _i ＝(r _i1 ,r _i2 ,...,r _i128 )

Descriptor of key points in the real-time graph, S _i =(s _i1 ,s _i2 ,...,s _i128 )

The definition of similarity between any two descriptors is shown in formula (27):

To get the paired feature point descriptor, it needs to meet:

When R _j is the matching point of S _i , (R _j is the closest point to S _i in the real-time graph, and R _p is the closest point to S _i in the real-time graph)

So far, all the work of image matching has been completed. Through the above image matching algorithm, the matching point cloud in the γ image and the ordinary camera image is found, and then the transformation matrix between them is calculated, and finally the pollution source is mapped to the image of the ordinary camera to realize the detection of the pollution source target.

Although specific embodiments of the present invention have been described in detail in conjunction with specific examples, they are not intended to limit the protection scope of this patent. Within the scope defined in the claims, various modifications or adjustments that can be made by those skilled in the art without creative work are still protected by this patent.

Claims (7)

1. a target detection method in a nuclear environment based on visual attention mechanism, is characterized in that, comprises the steps: S1. Obtain an image of the target collected by a common camera, extract brightness, color and direction features of the image, and obtain a brightness feature map, a color feature map, and a direction feature map, respectively; S2. Calculate the luminance feature map, color feature map, and direction feature map by means of a Gaussian pyramid and a central peripheral operator, and obtain 6 luminance parallax maps, 12 color parallax maps, and 24 directional parallax maps; S3. Perform normalization processing on the 6 luminance disparity maps, 12 color disparity maps and 24 directional disparity maps respectively to obtain a luminance saliency map color saliency map and directional saliency map Luminance saliency map using weighted fusion algorithm color saliency map and directional saliency map Perform weighted fusion to obtain a feature saliency map; S4. Selecting the most prominent point from the feature saliency map, using the point as a salient point, segmenting the corresponding feature saliency map by region growing to obtain a region of interest; S5. Obtain an image containing radiation intensity distribution information collected by the gamma camera; S6, respectively extracting the key points of the region of interest of the image containing the radiation intensity distribution information and the common camera image; S7. Generate feature vectors for the key points respectively; S8. Match the feature vector of the key point of the region of interest with the feature vector of the key point of the image containing radiation intensity distribution information, and if the matching condition is met, the target is an operation target; The weighted fusion algorithm in the S3 step is specifically: Among them, w _t +w _c +w _o ＝1, S refers to the feature saliency map after weighted fusion of brightness saliency map, color saliency map and direction saliency map, w _t , w _c , w _o refer to brightness saliency map, Weights for color saliency map and directional saliency map. 2. The method according to claim 1, characterized in that: the specific process of performing parallax calculation to the three feature maps in the step S2 is: Use r, g, b to represent the pixel values of the red, green, and blue color channels of the image of the target collected by the ordinary camera, and use the pixel values to establish a Gaussian pyramid model, and carry out the process through images of different scales in the center and the periphery. difference to obtain 6 luminance disparity maps; Obtain the components on the four color channels red, green, blue, and yellow of the image of the target collected by an ordinary camera, and obtain 12 color parallax maps through differential calculation; The brightness map is convolved with the Gabor direction filter to obtain the direction feature of the image, and 24 direction disparity maps are obtained through difference calculation. 3. The method according to claim 1, characterized in that: the specific process of normalizing the 6 luminance parallax maps, 12 color parallax maps and 24 directional parallax maps respectively is: Set a normalization operator N(.) to improve the quality of the brightness feature map, color feature map and direction feature map. The normalization operator N(.) calculation process is as follows: Normalize the pixel values of the brightness feature map, color feature map and direction feature map of the four channels of R, G, B, and Y into a fixed interval [0, M], where M is a positive integer; Find the position of the global maximum M in the brightness feature map, color feature map, and direction feature map, and calculate the mean value of the local maximum of all other brightness feature maps, color feature maps, and direction feature maps The brightness feature map, color feature map and direction feature map are globally multiplied by Through the normalization operator N(.) and cross-scale addition, the brightness feature map, color feature map and direction feature map are integrated into three saliency maps of color, brightness and direction; Color normalized feature map: Brightness normalized feature map: Orientation normalized feature map: in, It is to downsample the feature map of each feature on different scale layers to obtain the highest main scale layer, and then perform addition operation to obtain the saliency map on the three features of color, brightness, and direction; RG(c, s) refers to the color disparity map of the red-green channel, BY(c,s) refers to the color disparity map of the blue-yellow channel, I(c,s) refers to the brightness disparity map, O(c,s,θ) refers to the direction Disparity map, θ represents the direction angle {0°, 45°, 90°, 135°}. 4. The method according to claim 1, characterized in that: the specific process of extracting respectively the image containing radiation intensity distribution information and the key point of the common camera image region of interest is: Gaussian pyramid images obtained at different scales are obtained by using Gaussian difference operators of different scales to convolve with the image; Carrying out adjacent scale difference operations on the Gaussian pyramid image to obtain a differential pyramid image; A local extremum judgment is performed on the points on the differential pyramid image, and if the points are local extremum points, then the points are key points of the region of interest. 5. The method according to claim 4, characterized in that: the method for judging the local extremum point is: The detected points on the differential pyramid image are compared with 26 points corresponding to the same dimension and the upper and lower adjacent dimensions, so as to ensure that extreme points are detected in both the scale space and the two-dimensional image space. 6. method according to claim 1, is characterized in that: the concrete process that described key point is respectively generated feature vector is: Utilizing the gradient direction distribution characteristic of the neighborhood pixels of the key point, specifying a direction for each key point, and the key point descriptor is characterized relative to this direction, so that the key point descriptor is invariant to image rotation; Determine the transformation parameters for extracting key point images; Move the coordinates of key points to the main direction; In the 16*16 area centered on the key point, calculate the gradient histogram in 8 directions for each 4*4 area, count the cumulative value of each gradient, form a seed point, and generate 16 seed points in total. 128-dimensional vector; Thresholding and vector normalization are performed on the obtained feature vectors to obtain normalized feature vectors. 7. The method according to claim 1, characterized in that: the specific process of matching the feature vector of the key point of the region of interest of the common camera image with the feature vector of the key point of the image containing the radiation intensity distribution information is: Establish a key point descriptor set for the grayscale image of the region of interest and the image containing the radiation intensity distribution information obtained by the γ camera; The similarity of the key point descriptors in the two point sets is compared, and the similarity measure of the key point descriptors with 128 dimensions adopts the Euclidean distance to obtain the paired feature point descriptors; Find the matching point cloud in the γ image and the ordinary camera image, and then calculate the transformation matrix between the matching point cloud.