CN101976350B - Grain storage pest detection and identification method based on video analytics and system thereof - Google Patents

Abstract

The invention provides a video analysis-based detection and recognition method for stored grain pests and a corresponding detection and recognition device. The method of the present invention comprises the following steps: S1: Acquire multiple frames of continuous images of the stored grain samples to be tested in a moving state; S2: Segment each frame of image into a grain insect area and a background area; S3: After each frame is segmented In the image, the grain worm search area is defined centering on the grain worm area, and the grain worm search area is composed of M×M grain worm areas; S4: Based on the preset matching standard, the N+1 frame image Search for an area that matches the grain insect area of the Nth frame image in the grain insect search area, and record the matching value between the two; S5: identify the grain insect area with the cumulative matching degree exceeding the predetermined threshold as the grain insect area. Insect area. This method can overcome the defects of misdetection and missed detection which are common in the existing detection and identification methods, and can realize the survival and type determination of grain insects.

Description

Detection and recognition method and system of stored grain pests based on video analysis

technical field

The invention relates to the technical field of detection, in particular to a method and system for detecting and identifying stored grain pests based on video analysis.

Background technique

my country is the largest grain storage country in the world, and the loss of grain stored in the treasury due to the damage of grain insects is as high as more than 1 billion yuan every year. The current detection methods for stored grain pests (grain insects) mainly include near-infrared spectroscopy, X-ray methods, image processing and machine vision methods. Among them, the near-infrared spectroscopy method and the X-ray method identify the types of grain insects by scanning the grains one by one. Although the identification effect is good, the efficiency is too low and the cost of the equipment is high.

The detection and recognition technology of stored grain pests based on image processing and machine vision has become a research hotspot in recent years because of its advantages of high recognition efficiency and low cost. However, there are generally two problems in the existing detection and recognition methods based on image processing and machine vision: first, because a static image needs to be processed, image noise will lead to false detection and missed detection of food insects; second, the existing The method cannot judge whether the grain worm survives or not.

Contents of the invention

(1) Technical problems to be solved

The technical problem to be solved by the present invention is: how to overcome the defects of misdetection and missed detection commonly existing in the existing methods for detecting and identifying stored grain pests based on image processing and machine vision, and how to realize the survival and type determination of grain insects.

(2) Technical solution

In order to solve the above technical problems, the technical solution of the present invention provides a method for detecting and identifying stored grain pests based on video analysis, comprising the following steps:

S1: Acquire multiple frames of continuous images of the stored grain sample to be tested in a moving state;

S2: Segment each frame of image into grain insect area and background area;

S3: In the divided image of each frame, define a grain insect search area with a size of M×M grain insect areas centered on the grain insect area;

S4: Based on the preset matching standard, search for an area block that matches the grain insect area of the Nth frame image in the grain insect search area of the N+1 frame image, and record the matching degree values between the two;

S5: Recognize the grain worm area whose cumulative matching degree value exceeds a predetermined threshold as the area where grain worm exists.

Further, after step S5, the method also includes: detecting the respective motion vector values of the region with grain insects in the multi-frame continuous images relative to its background region, and calculating its average motion vector value, by The average motion vector value judges whether the grain insects in the area where grain insects are present are living grain insects.

Further, after step S5, the method also includes: extracting the image features of the area where grain insects exist, and obtaining classification results and type probability values through an SVM classifier; judging the existence of The type of food worms in the area of food worms.

Further, the motion vector value of the region where grain worms exist relative to its background region is obtained through the following steps:

S401: Construct a six-parameter affine model according to the motion state of the stored grain to be measured, and obtain the motion vector value of the background stored grain;

S402: Obtain the displacement value between the area where grain worms exist and the matching area block, as the absolute motion vector value of the area where grain worms exist;

S403: Obtain the motion vector value of the area where grain insects exist relative to its background area according to the motion vector value of the background stored grain and the absolute motion vector value of the area where grain insects exist.

Preferably, the step S2 is realized by a custom threshold method.

Preferably, the step S4 is realized by a three-step search method.

Preferably, in the step S4, the preset matching standard is to minimize the sum of the squares of gray value differences between the food insect search area of the N+1th frame image and the grain insect area of the Nth frame image The region blocks identified as the matching region blocks.

Correspondingly, the technical solution of the present invention also provides a stored-grain pest detection and identification system based on video analysis, including a stored-grain video acquisition device and a stored-grain pest detection and identification device, wherein,

The stored grain video acquisition device acquires the video of the stored grain sample in motion, and transmits the video to the stored grain pest detection and identification device;

Storage grain pest detection and identification device, including:

The video analysis module analyzes the video transmitted by the grain storage video acquisition device into multiple frames of continuous images;

The grain worm detection module is used to segment and extract the grain worm area for each image, and limit the search area based on the grain worm area; for each grain worm area, search and match in the search area of the multi-frame image ; Identify and locate areas where grain worms exist based on search matching results.

Further, the stored grain pest detection and identification device also includes:

The living insect identification statistics module detects the respective motion vector values of the area where grain insects exist in the multi-frame continuous images relative to its background area, and calculates its average motion vector value, and judges by the average motion vector value Whether the grain worms in the area where grain worms exist are living grain worms.

Further, the stored grain pest detection and identification device also includes:

Grain worm type identification module extracts the image features of the region where the grain worms exist, and obtains classification results and type probability values through the SVM support vector machine classifier; judges the type of grain worms according to the statistical results of the probability values.

(3) Beneficial effects

Compared with the prior art, the method and system for detecting and identifying stored grain pests based on video analysis according to the present invention have the following advantages:

The detection of stored grain pests is based on multi-frame image verification, and the accuracy of grain insect detection is improved through continuous multi-frame search and comparison, avoiding missed and false detections of grain insects;

Detection of stored grain live insects based on video motion analysis, judging the living status of grain insects through the movement characteristics of grain insects, and realizing the detection and real-time statistics of stored grain live insects;

Grain insect type identification is based on statistics. Aiming at the shape instability of live insects, by counting multi-frame grain insect recognition results, the type with the largest average probability is selected as the type of grain insect, which improves the accuracy of grain insect identification.

Description of drawings

Fig. 1 is a flowchart of a method for detecting and identifying stored grain pests based on video analysis according to an embodiment of the present invention;

Fig. 2 is a schematic diagram of the search and matching process in the method for detecting and identifying stored grain pests based on video analysis according to an embodiment of the present invention;

Fig. 3 is a structural block diagram of a system for detecting and identifying stored grain pests based on video analysis according to an embodiment of the present invention.

Detailed ways

The method and system for detecting and identifying stored grain pests based on video analysis proposed by the present invention are described as follows in conjunction with the accompanying drawings and embodiments.

Fig. 1 is a flowchart of a method for detecting and identifying stored grain pests based on video analysis according to an embodiment of the present invention. As can be seen from Figure 1, the method includes the following steps:

S1: Acquire multiple frames of continuous images of the stored grain sample to be tested in a moving state

During the implementation process, the video of the stored grain samples placed on the conveyor belt can be captured and analyzed to obtain multiple frames of continuous images.

Specifically, various grain samples can be regularly taken from the grain depot through the grain sampling device according to needs; the grain samples are scattered on the conveying device, thereby forming a single layer of grain samples on the conveyor belt of the conveying device; The single-layer grain sample moves with the conveyor belt and forms a video; the video is parsed into multiple frames of continuous grain images. Preferably, the above-mentioned grain conveying device can be provided with additional lighting to keep the image brightness of the video stable. Video parsing is an existing technology in the art, and details are not described here.

S2: Segment each frame of image into grain insect area and background area

Specifically, the grain insect area refers to a small image block containing pests, as shown in the small box on the left side of Figure 2; and the area except the grain insect area in each frame of image is the background area. Before segmenting each frame of stored grain image, it can be processed by smoothing and sharpening techniques to enhance the contrast between the grain worm and the background in the image and the clarity of the whole image, for example, using the Robert operator or the Sobel algorithm The edge extraction of the stored grain image is carried out to obtain the grayscale image of grain insects; then, according to the pixel value of the grayscale image, a K-means clustering adaptive threshold method is used to determine a method to divide the stored grain image into grain insect regions and The optimal threshold value of the background area; binarize the stored grain image through the obtained optimal threshold value; finally, the complete grain insect area is obtained through the erosion and expansion operation of mathematical morphology. Of course, any other image processing method that can obtain the same processing effect can also be used.

S3: Defining a grain worm search area centered on the grain worm area in each frame of the segmented image;

Specifically, for each frame of the segmented stored grain image, centering on the grain worm area, expand the area with the size of M×M unit blocks around it, thereby dividing a grain worm search area around it, as shown in the figure 2 shown in the big box on the right part (that is, the search window); wherein, the size of the unit block (the small window on the right part in Fig. 2) should be the same or approximately the same as the size of the grain insect area; M is 3. Preferably, M is 8.

S4: Based on the preset matching standard, search for an area that matches the grain insect area of the Nth frame image in the grain insect search area of the N+1 frame image, and record the matching value between the two;

In the specific implementation process, for the grain insect area in the Nth frame image, the grain insect area in the Nth frame image can be searched based on the preset matching criteria in the grain insect search area in the Nth frame image Matching area blocks, and record the matching value of the two. The matching value here refers to the sum of the squares of the gray value difference between the grain worm area of the Nth frame image and the grain worm search area of the N+1 frame image; the preset matching standard here is: Take the area block with the smallest difference square sum in the grain insect search area of the N+1 frame image as the area matching the grain insect area of the Nth frame image.

Preferably, as shown in Figure 2, the above-mentioned matching process can be realized by a three-step search method: S401, starting from the center point of the search window in the right part of Figure 2, select half of the maximum search length as the step size, and match the center point The eight unit blocks whose point distance is the step size carry out block matching, and the matching object is the grain worm area shown in the small box in the left part of Figure 2, and the matching degree values are calculated respectively, and the area block with the highest matching degree value is determined after comparison; S402, halving the step size, taking the area block with the highest matching value obtained in S401 as the center, performing block matching on the eight unit blocks whose distance from the center point is the step size, and calculating the matching value respectively, after comparison Determine the area block with the highest matching degree; S403, repeat step S402 until the step size is 1, then the currently obtained area block with the highest matching degree is the matching area block, and the search ends. At this time, the matching degree value in step S403 is recorded.

S5: Identify the grain insect area whose cumulative matching degree value exceeds a predetermined threshold as the grain insect.

In the specific implementation process, step S4 is performed sequentially on the multiple frames of continuous images, and all grain insect regions of each frame of image are searched for matching and matching degree value calculation; if the cumulative matching degree value of a certain grain insect region exceeds the given If the threshold is set, it is considered that grain insects have been detected.

Preferably, the predetermined threshold is 0.6.

Through the above steps S1-S5, this embodiment provides a stored grain pest detection method based on multi-frame image verification, which improves the accuracy of grain pest detection through continuous multi-frame search and comparison, and avoids missed detection of grain pests and false positives.

Preferably, the method for detecting and identifying stored grain pests based on video analysis of the present invention can further identify whether the grain pests are alive after detecting the grain pests.

Specifically, according to the detected area of grain insects, it is further judged whether the grain insects are alive, and the number of surviving grain insects is counted. The identification process includes the following steps:

S11: Mark several feature points on the stored grain sample transfer device, and obtain a six-parameter affine model through feature point matching to describe the global motion law of the camera device; the construction of the six-parameter affine model belongs to the prior art, here I won't go into details.

S12: Obtain the motion vector value of each area where grain worms exist relative to its background stored grain;

Specifically, the motion vector value of the background stored grain can be obtained through the affine model constructed in step S11; in step S4, an area block matching the area where grain insects exist will be obtained, and the displacement value between the two That is, the absolute motion vector of the area where grain worms exist; subtracting the motion vector value of the background stored grain from the absolute motion vector value of the area where grain worms exist is The motion vector value of the background stored grain; the relative motion vector is the criterion for judging whether the grain worm survives or not.

S13: Calculate the average motion vector value of each grain insect in multiple frames of continuous images, and identify the grain insect whose average motion vector value is greater than zero as live insects.

Through the above steps S11-S13, the method for detecting and identifying stored grain pests based on video analysis in this embodiment also provides a method for detecting live insects in stored grains based on video motion analysis, judging the survival status of grain insects through the movement characteristics of grain insects , Realize the detection and real-time statistics of stored grain live insects.

Preferably, the stored-grain pest detection and recognition method based on video analysis of the present invention can further determine the type of pest it belongs to after detecting the grain pest. The identification process includes the following steps:

S21: Use the grain insect areas of all the grain insects detected as the area to be identified. For the area to be identified of each grain insect, image features such as its grayscale, perimeter, area, and Fourier operator are extracted;

S22: Input the image features such as grayscale, perimeter, area, and Fourier operator into a support vector machine (support vector machine, SVM) classifier to identify the type of grain insects, and obtain the classification results of grain insects and the category they belong to. probability value;

S23: Calculate the average value of the probability values of each type in the recognition results of step S22, and select the type with the largest average value as the type of the grain worm, so as to avoid the low accuracy rate of single-frame image recognition.

Through the above steps S21-S23, the method for detecting and identifying stored grain pests based on video analysis in this embodiment also provides a method for identifying types of grain insects based on statistics. According to the results of insect identification, the type with the largest mean value of probability is selected as the type of grain insect, which improves the accuracy of grain insect identification.

According to the method for detecting and identifying stored grain pests based on video analysis described in the above embodiments, the present invention also provides a system for detecting and identifying stored grain pests based on video analysis.

As shown in FIG. 3 , the system of this embodiment includes two parts: a stored-grain video acquisition device and a stored-grain pest detection and identification device.

Specifically, the grain storage video acquisition device is used to realize the acquisition of continuous images in step S1. Accordingly, it includes:

The stored grain sampling unit is used to take various grain samples from the granary;

The grain storage conveying unit is used for conveying grain samples in a single layer, which is convenient for the camera unit to take pictures; preferably, the grain storage conveying unit is a conveyor belt;

The camera unit is used to shoot the motion video of the stored grain moving with the conveying device, and transmit it to the stored grain pest detection and identification device.

Preferably, the stored-grain video collection device of this embodiment may further include a lighting unit for illuminating the stored-grain conveying unit, so as to ensure stable image brightness of the stored-grain video.

Specifically, the device for detecting and identifying stored grain pests is used to realize the video analysis and processing process described in steps S2-S5. Accordingly, it includes:

The video analysis module analyzes the video transmitted by the grain storage video acquisition device into multiple frames of continuous images; for example, software such as Directshow can analyze the video stream into multiple frames of continuous images for further processing;

The grain worm detection module is used to segment and extract the grain worm area for each image, and limit the search area based on the grain worm area; for each grain worm area, search and match in the search area of the multi-frame image ; Identify and locate areas where grain worms exist based on search matching results.

Preferably, the stored grain pest detection and identification device of this embodiment may also include:

The living insect identification and statistics module detects the respective motion vector values of the region where grain insects exist in the multi-frame continuous images, and identifies whether the grain insects therein are living grain insects through their average motion vector values; and

Grain worm type identification module extracts the image features of the area where the grain worm exists, and obtains the classification result and type probability value through the SVM classifier; judges the type of the grain worm according to the statistical result of the probability value.

It should be noted that, regarding the working principle of each part of the system for detecting and identifying stored grain pests based on video analysis in this embodiment, you can refer to the corresponding description of the method for detecting and identifying stored grain pests based on video analysis, and details will not be repeated here.

The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Those of ordinary skill in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all Equivalent technical solutions also belong to the category of the present invention, and the scope of patent protection of the present invention should be defined by the claims.

Claims (8)

1. A method for detecting and identifying stored grain pests based on video analysis, characterized in that, comprising the following steps: S1: Acquire multiple frames of continuous images of the stored grain sample to be tested in a moving state; The step S1 specifically includes the steps of: Regularly take various grain samples from the grain depot through the grain sampling device; dropping the grain sample onto the conveyor, thereby forming a single layer of the grain sample on the conveyor belt; Additional lighting on the grain conveyor to keep the image brightness of the video stable; Use a video camera to shoot the process of the single-layer grain sample moving along the conveyor belt and form a video; Parse the video into multi-frame continuous grain storage images; S2: Segment each frame of image into grain insect area and background area; S3: In the divided image of each frame, define a grain insect search area centered on the grain insect area, and the grain insect search area is composed of M×M grain insect areas; S4: Based on the preset matching standard, search for an area block that matches the grain insect area of the Nth frame image in the grain insect search area of the N+1 frame image, and record the matching degree values between the two; S5: identifying the grain insect area whose cumulative matching degree value exceeds a predetermined threshold as an area where grain insects exist; S6: Detect the respective motion vector values of the area with grain insects in the multi-frame continuous images relative to its background area, and calculate its average motion vector value, and judge the presence of grain insects by the average motion vector value Whether the grain worms in the worm area are living grain worms. 2. The method for detecting and identifying stored grain pests based on video analysis as claimed in claim 1, characterized in that, after step S5, the method also includes: extracting the image features of the region where grain insects exist, and supporting them through SVM The vector machine classifier obtains the classification result and the type probability value; According to the statistical result of the probability value, the type of the grain worm in the area where the grain worm exists is judged. 3. the stored-grain pest detection and recognition method based on video analysis as claimed in claim 1, is characterized in that, the motion vector value of the area of described existence grain insect with respect to its background area is obtained by the following steps: S401: Construct a six-parameter affine model according to the motion state of the stored grain to be measured, and obtain the motion vector value of the background stored grain; S402: Obtain the displacement value between the area where grain worms exist and the matching area block, as the absolute motion vector value of the area where grain worms exist; S403: Obtain the motion vector value of the area where grain insects exist relative to its background area according to the motion vector value of the background stored grain and the absolute motion vector value of the area where grain insects exist. 4. The method for detecting and identifying stored grain pests based on video analysis according to any one of claims 1-3, wherein said step S2 is realized by a self-defined threshold method. 5. The method for detecting and identifying stored grain pests based on video analysis according to any one of claims 1-3, wherein the step S4 is realized by a three-step search method. 6. The method for detecting and identifying stored grain pests based on video analysis according to any one of claims 1-3, characterized in that, in the step S4, the preset matching standard is: the N+1th The area block with the smallest sum of squares of gray value differences between the grain worm search area of the frame image and the grain worm area of the Nth frame image is identified as the matching area block. 7. A stored grain pest detection and identification system based on video analysis, characterized in that it includes a stored grain video acquisition device and a stored grain pest detection and identification device, wherein, The stored grain video acquisition device acquires the video of the stored grain sample in motion, and transmits the video to the stored grain pest detection and identification device; The grain storage video acquisition device includes: The stored grain sampling unit is used to take various grain samples from the granary; The grain storage transmission unit is used for single-layer transmission of grain samples, which is convenient for the camera unit to take pictures, and the storage grain transmission unit is a conveyor belt; The camera unit is used to shoot the motion video of the stored grain moving with the conveying device, and transmit it to the stored grain pest detection and identification device; The lighting unit is used to illuminate the grain storage transfer unit, so as to ensure that the image brightness of the grain storage video is stable; Storage grain pest detection and identification device, including: The video analysis module analyzes the video transmitted by the grain storage video acquisition device into multiple frames of continuous images; The grain insect detection module performs segmentation and extraction of the grain insect area on each image, and limits the search area based on the grain insect area; then, based on the preset matching criteria, the grain insect search of the N+1th frame image Search for the area blocks that match the grain insect area of the Nth frame image in the area, and record the matching value between the two; finally, identify the grain insect area with the cumulative matching degree value exceeding the predetermined threshold as the grain insect area area; The living insect identification statistics module detects the respective motion vector values of the area where grain insects exist in the multi-frame continuous images relative to its background area, and calculates its average motion vector value, and judges by the average motion vector value Whether the grain worms in the area where grain worms exist are living grain worms. 8. The stored-grain pest detection and identification system based on video analysis as claimed in claim 7, wherein said stored-grain pest detection and identification device further comprises: Grain worm type identification module extracts the image features of the region where the grain worms exist, and obtains classification results and type probability values through the SVM support vector machine classifier; judges the type of grain worms according to the statistical results of the probability values. the

Images