CN115035446B - A kitchen rat detection and recognition method based on local MSR and target matching - Google Patents

Abstract

The present invention provides a kitchen rat detection and identification method based on local MSR and target matching, including monitoring image extraction, image low-light enhancement, rat activity area detection, rat template preparation, rat matching and judgment, and is characterized in that: the monitoring image extraction is to transfer the video image in the camera to the computer through a video image acquisition card, and then convert the video image into an image sequence of continuous frames.

Description

A kitchen rat detection and recognition method based on local MSR and target matching

Technical Field

Computer vision includes image processing, pattern recognition and target detection. Target detection is widely used in the field of video surveillance. This paper uses the background difference method in moving target detection to detect and identify moving targets in surveillance videos.

Background Art

As people's living standards improve, my country's catering industry has also flourished. The emergence of rats has brought serious food safety problems to the catering industry. The use of video imaging technology can find the movement trajectory of rats, and this problem can be solved by deploying rat traps on the rat path.

With the rapid development of image detection and recognition, video surveillance technology has also been widely used in the field of surveillance. Using computer vision technology to establish a video surveillance system, real-time detection of images under surveillance, identification of rats that endanger food, analysis of rat activity trajectories, and arrangement of rat-catching equipment based on known route trajectories can reduce the harm of rats to kitchen food, reduce unnecessary food safety disputes and the labor cost of rat-catching. This technology uses background difference method to separate moving targets from background, quickly find and identify moving targets. Before background difference, based on the characteristics of fixed rat activity areas, only MSR processing is performed on areas where rats may appear, reducing the amount of calculation and improving efficiency. The learning rate of model update of traditional mixed Gaussian model is fixed, which will cause problems such as holes or smears on the differential image. This paper designs a method that can update the background modeling at different stages according to different learning rates, improves the traditional mixed Gaussian model, and can solve this problem well. This paper designs a matching judgment method for target rats to identify rats. According to the morphological characteristics of rats, four rat templates at different angles are designed, and the templates are matched with the detection image to determine whether the rat exists.

Summary of the invention

The purpose of the present invention is to provide a method for detecting and identifying mice by combining local MSR and specific target matching.

The present invention implements the steps as follows:

① Surveillance image extraction (101):

The video images in the camera are transferred to the computer through the video image acquisition card, and then the video images are converted into an image sequence of continuous frames.

② Image low-light enhancement (102):

Each frame of the image is split horizontally using a non-uniform division method to obtain two images, the upper and lower images, and the lower image is enhanced using the MSR method.

③ Mouse activity area detection (103):

Construct K (K = 3, 5) Gaussian distribution models for each pixel of the preprocessed image to form a mixed Gaussian background model. Match the K Gaussian distributions of the current frame pixel with the K Gaussian distributions of the background pixel. If the matched Gaussian distribution meets the background requirements, then the pixel is the background, otherwise it is the foreground. If no Gaussian distribution matches the background, a new Gaussian distribution appears at this time, and then the distribution replaces the Gaussian distribution with the smallest weight in the background Gaussian distribution. After the background difference is completed, the background model needs to be updated. To update the background model, the Gaussian distribution of the current pixel needs to be compared with the existing Gaussian distribution. If the Gaussian distribution matches a certain Gaussian distribution, the mean, weight and variance of the Gaussian distribution are updated accordingly. If none of them match, a new Gaussian distribution appears, and the Gaussian distribution with the smallest weight needs to be replaced by this Gaussian distribution.

The background modeling process is divided into an initial stage and a stable stage. A larger learning rate is required in the initial stage of the background, and a smaller rate is required in the stable stage of the background.

Based on the established mixed Gaussian model as the background, the foreground image is differentiated from each frame of the preprocessed image.

The differential image is subjected to morphological processing, and then edge detection is performed on the processed image to extract the contour information of the differential image.

④ Mouse template production (104):

According to the shape differences of mice at different angles, four mouse contour images at different angles were designed, and the contour information of the extracted moving target was matched with the four mouse contour images. According to the characteristics of mice being small in size and occupying fewer pixels, the designed contour image was reduced and the contour areas at different angles were calculated.

⑤ Mouse matching and determination (105):

The contour information of the differential image is matched with four mouse templates, and the contour area corresponding to the mouse contour image with the higher match is compared with the contour area of the extracted moving target. If the difference between the two is small, the moving target is determined to be a mouse.

The present invention has the following advantages and positive effects:

In order to solve the problem of low brightness and high noise in images obtained from rats' nighttime activities, in the low-light enhancement stage of images, each frame of the image is first divided into two parts, upper and lower, based on the characteristics that the range of rats' activities is located at the lower part of the monitoring area. Only the lower image is enhanced by MSR, which can not only eliminate image noise and improve image brightness, but also reduce the computational workload and improve computational efficiency. According to the characteristics of the fixed kitchen background and the small rat target, when using the mixed Gaussian background modeling method to establish a background model, the learning rate is changed to an adaptive value to adapt to the update speed of the background at different stages, so as to extract more accurate rat differential images, speed up the detection speed, and reduce unnecessary resource waste. The differential image is binarized, morphologically processed, and edge detected to extract the contour information required for target matching.

In the target matching stage, we analyze specific problems based on specific situations. Based on the characteristics of a fixed kitchen scene and different angles of mouse activity, we produce mouse outline images at different angles and match them with the outline image after edge detection. We determine whether there is a mouse in the frame image by checking whether the shape and area of the outline image after edge detection matches the shape and area of the outline at a certain angle. We use a double matching mechanism to improve recognition accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 Main flow chart

Figure 2 Mouse activity area detection

Figure 3 Kitchen background model update learning rate flow chart

Figure 4 Mouse outline images observed from four angles

101 Surveillance image extraction 102 Image low illumination enhancement 103 Mouse activity area detection 104 Mouse template creation 105 Mouse matching and judgment 201 Establishing a mixed Gaussian kitchen background model 202 Kitchen background update 203 Kitchen background modeling learning rate update 204 Mouse kitchen foreground and background difference 205 Difference image binarization 206 Morphological processing 207 Edge detection

DETAILED DESCRIPTION

1. Methods

1. Steps of this method

① Monitoring image extraction;

② Image low illumination enhancement;

③ Detection of rat activity areas;

④ Mouse template production;

⑤ Mouse matching and determination;

2. Working mechanism

The working mechanism of the present invention is briefly described below:

The video image in the input device is input into the computer memory through the computer input device, and the image is converted into an image that can be recognized and processed by the computer. Each frame of the image is divided into upper and lower blocks, and the lower image is processed by MSR. Multiple Gaussian distribution models are established for each pixel value of the preprocessed image, and the background model is updated according to the extracted frame image to gradually form a relatively stable background model. Then, the foreground pixels and background pixels of the image are determined according to the algorithm of mixed Gaussian model background modeling, so as to achieve the purpose of background and foreground segmentation. The image after background difference is binarized and morphologically processed, and the processed image is edge detected to extract the contour information in the image. According to the morphological characteristics of the mouse, the mouse contour images of different angles are made, and they are matched with the differential image after edge detection to determine whether there is a mouse in the image.

Substeps

i. Surveillance image extraction:

The video images in the camera are transferred to the computer through a digital acquisition card, and then the video is converted into an image sequence.

ii. Image low illumination enhancement:

The time for detecting mice in the kitchen is after get off work in the evening. The exposure conditions are insufficient and the obtained images are low-light images. It is necessary to first perform image enhancement processing on each frame of the image to eliminate image noise and improve image quality.

Since most mice in the kitchen move around on the stove and on the ground, which are below the surveillance camera, only the image below needs to be enhanced to reduce unnecessary calculations and improve computing efficiency.

First, the pixel size of each frame of the image is calculated. Since each image has the same size, only the pixel size of the first frame of the image needs to be calculated, and the vertical pixels of the image are divided into two images, the upper and the lower, by rounding off.

The segmented lower image is subjected to MSR (Multi-Scale Retienx) image enhancement. Three Gaussian filter functions of different scales are used. Each filter function has a different scale parameter. The three Gaussian filter functions are used to perform convolution operations on the low-light image. After the operation is completed, illumination images of different scales can be obtained. Then, the multiple illumination images of different scales are weighted averaged to obtain the final illumination image, as shown in formula (1) and formula (2):

In formula (1) and formula (2), n is 3, N represents the number of Gaussian S _i (x, y) convolution functions, which is generally 3, respectively, used to represent small, medium and large scales, W _n represents the weight of each scale, k _n represents the normalization factor of different scales, c _n represents different scale parameters, G _n (x, y, c) represents n different Gaussian functions, S _i (x, y) represents the component of the i-th channel in the low-light image.

After the operation is completed, illumination images of different scales can be obtained, and then the three illumination images of different scales are weighted averaged to obtain the final image with illumination enhancement and noise elimination.

iii. Mouse activity area detection:

1) Establishing a mixed Gaussian kitchen background model (201):

The mixed Gaussian background modeling uses the method of establishing K Gaussian models for pixels to describe the state of the pixel at a certain moment. Assuming that the pixel value of a pixel at the initial moment is Xt, the expression of the probability function is as follows:

In this formula: X _t ——the observed value of the sample pixel at time t;

K – total number of models;

I——model number;

μ _i,t —mean of the model;

w _i,t ——weight of the model;

∑ _{i, t} —covariance matrix;

η(X _t , μ _i,t , ∑ _i,t )——probability density function;

P(X _t )——Probability function of X _t ;

Through the calculation of the above formula, the initial mixed Gaussian kitchen background model is established to facilitate the subsequent updating and differentiation of the background model.

2) Kitchen background update (202):

Each pixel is matched with each Gaussian distribution. The matching operation formula is as follows:

|X _t -μ _i,t-1 |≤λσ _i,t-1 (5)

In the formula: λ is the confidence parameter, μ _i,t-1 is the i-th Gaussian distribution, and σ _i,t-1 is the mean and variance at time t-1.

If the Gaussian distribution of the pixel matches it, the parameters that match the Gaussian parameters are changed as follows:

In the formula: a is the learning rate, ρ is the expected and variance learning rate.

If there is no match, it means that a new Gaussian distribution has appeared. A new Gaussian distribution needs to be created to match this Gaussian distribution. Since the number of Gaussian distributions for background modeling is fixed, this Gaussian distribution needs to replace the Gaussian distribution with a smaller weight. The initialization parameters of the new Gaussian distribution are to assign the image value of the pixel to be processed to the mean of the Gaussian distribution, and to assign a larger variance and a smaller weight.

3) Learning rate update (203):

According to the characteristics of fixed kitchen background and small mouse target, the kitchen background modeling is divided into the initial stage and the stable stage, and different learning rates are adopted according to different time periods.

a) In the initial stage of background modeling (n<N), the background modeling is in the process of gradual formation. At this time, a larger learning rate is needed to accelerate the formation of the kitchen background model. However, the kitchen background has not yet been fully formed, and the variance and expectation are not easy to converge too quickly. As the kitchen background gradually forms, the learning rate gradually decreases, so that the background model gradually stabilizes:

Where: λ ₁ is the attenuation coefficient; n is the number of frames currently flowing; N is the dividing line between the initial stage and the stable stage of background modeling; α ₁ is the learning rate in the initial stage; ρ ₁ is the learning rate of the expectation and variance in the initial stage;

b) In the background stabilization stage (n≥N), the kitchen background model has been basically formed. Since the activities of mice in the kitchen may cause certain damage to the fixed kitchen background, the background changes, and the background needs to be maintained and updated in real time. In the background maintenance and update stage, the number of matches and mismatches of modal pixel changes are used as feedback to correct the learning rate of the model. When there is a mismatch, the feedback is positive, which increases the learning rate and strengthens the learning of the scene; when there is a match, the feedback is negative, which reduces the learning rate and weakens the learning of the scene to ensure the stability of the model:

Where: λ ₂ is the learning rate reference coefficient, ΔF = (2f-t) × 10 ^-2 is the feedback amount, f is the number of mismatches, t is the number of matches, α ₂ is the learning rate in the stable stage, and ρ ₃ is the learning rate of the expectation and variance in the stable stage.

4) Mouse kitchen foreground and background difference (204):

Arrange the Gaussian distribution priorities from large to small according to the value of w _i,t /σ _i,t , and then start from the model at the front and take B Gaussian distributions as the background model of the mixed Gaussian distribution

Where: T is the proportion of the background.

Since the target mouse is a moving target and the background is a kitchen, which is an indoor scene and is not affected by outdoor lighting, and the items in the kitchen rarely move, the background changes little. Based on the above characteristics, the ratio T is set to a moderate value to reduce the possibility of the target mouse being detected as noise.

When performing foreground detection, after selecting B Gaussian models as the background, the image value of the current frame is compared with the B Gaussian background models respectively. If the image value of the frame has a Gaussian distribution corresponding to the background model, the pixel point is a background point, otherwise it is a foreground point.

5) Binarization of the difference image (205):

In order to better distinguish the foreground and background, it is necessary to color the foreground and background differently after background difference, and binarize the difference image, that is, change the pixel points of the target foreground to the maximum gray value and the background pixel points to the minimum gray value.

6) Morphological processing (206)

In order to remove the tiny spikes and small particle noise points in the binary image, the binarized image is subjected to morphological processing to obtain the complete target mouse image. First, the background noise is eroded, and then the foreground noise is expanded to eliminate noise points and spikes.

7) Edge Detection (207)

The differential image after morphological processing is subjected to edge detection, the contour image of the detected target is extracted, and the area of the contour is calculated.

iv. Mouse template production (104):

According to the specific shape characteristics of the mouse, the outline images of the mouse are produced from the front, side, back and oblique angles. Since the mouse observed in the camera is a small target with small pixels, the pixel ratio of the mouse outline image is designed to be 8×8 based on this feature, and the inner area of the outline of the mouse outline image observed from different angles is calculated based on the pixel ratio.

v. Mouse matching and determination (105):

First, the contour image after edge detection is matched with the mouse contour images observed at four different angles designed previously. If the similarity value between the image and the mouse contour image observed at a certain angle is less than 0.1, it is determined that the image may contain the target mouse and enters the next stage.

The area inside the image contour is compared with the area corresponding to the mouse contour image at a certain angle matched in the previous stage. If the area error between the two is less than 10%, the moving target in the image is determined to be a mouse.

2. Innovation

1. According to the characteristics of low light intensity at night in the kitchen and the mouse activity range in the lower image, a local image enhancement method based on MSR is designed to improve the computational efficiency while eliminating noise and improving image quality. Based on the mixed Gaussian background modeling method, according to the characteristics of fixed kitchen background and small mouse target, when using the mixed Gaussian background modeling method to establish a background model, the learning rate is changed to an adaptive value to adapt to the update speed of the background at different stages, so as to extract more accurate mouse differential images, speed up the detection speed, and reduce unnecessary resource waste.

2. In the target matching stage, we analyze specific problems based on specific situations. Based on the characteristics of a fixed kitchen scene and different angles of mouse activity, we produce mouse outline images at different angles and match them with the outline image after edge detection. We determine whether there is a mouse in the frame image by whether the shape and area of the outline image after edge detection matches the shape and area of the outline at a certain angle. We use a double matching mechanism to improve recognition accuracy.

Claims (4)

1. A kitchen mouse detection and recognition method based on local MSR and target matching comprises the steps of monitoring image extraction, image low illumination enhancement, mouse activity area detection, mouse template manufacturing, mouse matching and judgment, and is characterized in that: the monitoring image extraction is to transmit video images in a video camera to a computer through a video image acquisition card, and then convert the video images into an image sequence of continuous frames;

The mouse activity area detection is as follows: constructing K Gaussian distribution models for each pixel point of the preprocessed image, wherein K is equal to 3 or 5, so that a mixed Gaussian background model is formed; matching the K Gaussian distributions of the pixel points of the current frame with the K Gaussian distributions of the background pixel points, if the matched Gaussian distribution meets the background requirement, the pixel points are the background, otherwise, the pixel points are the foreground, if no Gaussian distribution is matched with the background, a new Gaussian distribution appears at the moment, and then the Gaussian distribution with the minimum weight in the background Gaussian distribution is replaced by the distribution; after the background difference is finished, the background model is required to be updated, the Gaussian distribution of the current pixel point is required to be compared with the existing Gaussian distribution to update the background model, if the Gaussian distribution is matched with a certain Gaussian distribution, the mean value, the weight and the variance of the Gaussian distribution are correspondingly updated, and if the Gaussian distribution is not matched with the certain Gaussian distribution, a new Gaussian distribution appears, and the Gaussian distribution with the minimum weight is required to be replaced by the Gaussian distribution; dividing the background modeling process into an initial stage and a stable stage, wherein a larger learning rate is required in the initial stage of the background, and a smaller learning rate is required in the stable stage of the background; differentiating a foreground image from the preprocessed image of each frame according to the established Gaussian mixture model as a background; carrying out morphological processing on the image after the difference, then carrying out edge detection on the processed image, and extracting outline information of the difference image;

The mouse template manufacturing comprises the following steps: designing four mouse contour images with different angles according to shape differences of mice with different angles, and performing shape matching on the extracted contour information of the moving object and the four mouse contour images; according to the characteristic of small mouse body size and small occupied pixel points, reducing designed contour images and calculating contour areas with different angles;

The mouse matching and determination includes: matching the outline information of the differential image with four mouse templates, comparing the outline area corresponding to the outline image of the mouse with higher matching with the outline area of the extracted moving object, and judging the moving object as the mouse if the difference between the outline area and the outline area of the extracted moving object is smaller;

The image low-illumination enhancement comprises the steps that after the time for detecting mice in a kitchen is in the night and after the time is out of work, exposure conditions are insufficient, the obtained image is a low-illumination image, and image enhancement processing is needed to be carried out on each frame of image, so that image noise is eliminated, and image quality is improved; because most mice in the kitchen move on the cooking bench and the ground and are positioned below the monitoring, only the images below the mice are subjected to enhancement processing, so that unnecessary operation is reduced, and the operation efficiency is improved; firstly, the pixel size of each frame of image is the same, and the pixel size of the first frame of image is only needed to be calculated, and the vertical pixels of the image are divided into an upper image and a lower image by a rounding method; MSR image enhancement is carried out on the segmented lower image, three Gaussian filter functions with different scales are adopted, each filter function has different scale parameters, convolution operation is carried out on the low-light image by using the three Gaussian filter functions, illumination images with different scales can be obtained after the operation is finished, and then a plurality of illumination images with different scales are weighted and averaged to obtain a final illumination image, such as formula (1) and formula (2):

In the formula (1) and the formula (2), n is 3, n represents the number of gaussian S _i (x, y) convolution functions, typically 3 are used to represent the small, medium and large three scales respectively, W _n represents the weight of each scale, k _n represents the normalization factor of different scales, c _n represents different scale parameters, G _n (x, y, c) represents n different gaussian functions, and S _i (x, y) represents the component of the i-th channel in the low-light image;

After the operation is finished, illumination images with different scales can be obtained, and then three illumination images with different scales are weighted and averaged to obtain a final image with illumination enhancement and noise elimination.

2. The kitchen mouse detection and recognition method based on local MSR and target matching according to claim 1, wherein the image low-illumination enhancement is to divide each frame of image in a horizontal direction by adopting a non-uniform division method to obtain an upper image and a lower image, and the lower image is subjected to image enhancement by adopting an MSR method.

3. The method for kitchen mouse detection and recognition based on local MSR and object matching of claim 1, wherein the mouse activity area detection comprises learning rate update: according to the characteristics of kitchen background fixation and small mouse targets, kitchen background modeling is divided into an initial stage and a stable stage, and different learning rates are adopted according to different time periods.

4. The method for kitchen mouse detection and recognition based on local MSR and object matching according to claim 1, wherein the mouse template making comprises: according to the specific shape characteristics of the mice, the contour images of the mice are observed at four angles of the front, the side, the back and the inclined upper surface; since the rat is observed in the camera to belong to a small target, the pixel is smaller, the pixel ratio of the mouse outline image is designed to be 8 multiplied by 8 according to the characteristic, and the outline inner area of the mouse outline image is observed at different angles according to the pixel ratio.