CN104751199B - A kind of cotton splits bell phase automatic testing method - Google Patents

Abstract

The invention discloses an image-based cotton boll-cracking stage detection method. It includes the following steps: (1) Obtain the positions of all bolls in the cotton field picture sequence; (2) Segment the boll images at all recorded boll positions for the last cotton field image; (3) Judge whether the cotton field has entered Bell cracking period. The method takes the important parameters characterizing the growth status of cotton as the judgment basis, and judges the cotton boll splitting stage in real time, and the detection result has high accuracy. Judging the start time of cotton boll splitting stage is of great significance for analyzing the relationship between cotton development period and meteorological conditions, identifying the agrometeorological conditions for cotton growth, and guiding farmers to carry out agricultural activities in time.

Description

A method for automatic detection of cotton boll splitting stage

technical field

The invention belongs to the intersecting field of digital image processing and agricultural meteorological observation, and in particular relates to an automatic detection method for cotton boll-opening stage.

Background technique

Cotton is one of the main economic crops in China, and China's cotton output is also in the leading position in the world. The boll splitting period of cotton is an important developmental period in the growth of cotton. This developmental period is the key period for the formation of cotton yield and quality, so it is also an important period for cotton field management. During this period, the activity of cotton roots gradually weakens, and the ability to absorb nutrients decreases significantly. The focus of production is to preserve roots and leaves, prevent premature aging, increase boll weight, and prevent pests and diseases. Therefore, the monitoring and identification of cotton boll splitting stage is very important. Cotton farmers can timely fertilize cotton fields and prevent diseases and insect pests according to the growth of cotton, which has positive significance for ensuring and increasing the total cotton output. All in all, the cotton boll splitting period is an important part of agricultural meteorological observation.

For a long time, the relevant information of the cotton development period has been recorded mainly by manual observation and recording. The observation results will be affected by the subjective factors of the observers, resulting in relatively large errors; at the same time, due to the long growth cycle of cotton, cotton planting The range is relatively wide, and the method of only using manual observation is time-consuming and labor-intensive. At present, there is no method to automatically observe and predict the cotton boll opening period.

Contents of the invention

Aiming at the above defects or improvement needs of the prior art, the present invention provides an image-based method for automatic recognition of the cotton boll-opening stage, the purpose of which is to accurately detect whether cotton has entered the boll-opening stage using the cotton digital images acquired in real time in the field , thereby solving the technical problem that there is no automation at present.

In order to achieve the above object, according to the present invention, a kind of cotton boll-breaking stage automatic detection method is provided, comprising the following steps:

(1) Obtain the position of cotton bolls in the picture: collect the cotton field image, and include the image into the cotton field image sequence; split all the images in the cotton field image sequence into M×N pixel sub-images according to a certain step size; Image, use the boll classifier to judge whether it contains bolls: for the sub-image that contains bolls, record its position in the original image as the final boll position; the boll classifier is established as follows: select a picture of M×N pixels as the training Positive and negative samples, where the positive samples are pictures of cotton bolls in different poses, and the negative samples are pictures of non-cotton bolls in the cotton field, extract SIFT feature quantities, and train support vector machines to make the support vector machine have the lowest false positive rate, thereby constructing a boll classifier;

(2) Segment the boll image: In the cotton field image collected in step (1), obtain all the sub-images of the final boll positions, convert them into grayscale images using RGB color mode or HSI color mode, and then convert the grayscale images into continuous The binary image of , using the morphological image processing method to detect the edge of the boll, and segment the boll image;

(3) Judging the boll cracking stage: For the boll image segmented in step (2), perform white crack detection and extract white cracks. According to the shape characteristics of the white cracks, judge whether it is a boll crack; if a boll crack appears, it is considered to have entered the crack. Bell period, otherwise it is considered that it has not entered the bell-splitting period.

Preferably, in the method for automatic detection of cotton boll splitting stage, the positive samples of the boll classifier are images of bolls with clear edges, plump bolls that can be observed by human eyes.

Preferably, the method for automatic detection of cotton boll splitting stage performs local constrained linear coding on the SIFT feature quantity.

Preferably, the step (1) of the automatic detection method for cotton boll-breaking stage includes the following sub-steps:

(1-1) Collect cotton field images, and incorporate the images into the cotton field image sequence;

(1-2) Coarse search for cotton boll position:

First, according to a certain rough search order, split all the images in the cotton field image sequence into M×N pixel sub-images with a coarse search step; then, use the cotton boll classifier to judge the split sub-images, and get The tag value of each sub-image; finally, record the sub-image position whose tag value exceeds the coarse search threshold, as the coarse search boll position;

(1-3) Fine search for the location of cotton bolls

First, for each coarse search boll position, a certain size range near the coarse search boll position is used as the fine search range; then, within the fine search range, according to a certain fine search sequence, the image is split into M ×N pixel sub-image; Next, use the cotton boll classifier to judge the split sub-image to obtain the tag value of each sub-image; finally, record the position of the sub-image whose tag value exceeds the fine search threshold, as the fine search boll position;

(1-4) Record and track boll position

For the finely searched boll positions recorded in step (1-3), use non-maximum value suppression to remove redundant boll positions, keep only the position with the largest boll classifier mark value as the final boll position, and record this position as the boll position ;

(1-5) Get all cotton boll positions

Record the boll position of all images in the sequence as the final boll position. In the cotton field image collected in step (1-1), select the sub-image of the final boll position as the subsequent boll segmentation process.

Preferably, the cotton field image in the step (1-1) of the automatic detection method for cotton boll splitting stage is a longitudinal front view of the cotton field, and its light intensity is the same as that of the cotton boll classifier training samples, which consists of images of no less than 4 million pixels Camera collection, the camera is 0.3 meters above the ground, with a focal length of 14 mm, set up to the north, and shoots horizontally; the preferred image collection time is 20:00.

Preferably, in the method for automatic detection of cotton boll-cracking stage, the order of rough splitting in step (1-2) is from left to right and from top to bottom; the rough search step is 30 pixels; the The coarse search threshold is the boll classifier threshold that makes the false positive rate 6%-9%.

Preferably, in the method for automatic detection of cotton cracking and boll stage, the fine search range described in step (1-3) is an image area that expands 5 pixels from the coarse search boll position to the right and downward; the fine split sequence is From left to right, from top to bottom; the fine search step size is 1 pixel; the fine search threshold is the boll classifier threshold that makes the false positive rate within 1%.

Preferably, the step (2) of the automatic detection method for cotton boll-breaking stage includes the following sub-steps:

(2-1) Determine the position of the cotton boll: when the position of the boll is within 1/3 of the entire image, keep the sub-image of the boll position in the RGB color mode; otherwise, convert it to the sub-image of the boll position in the HSI color mode;

(2-2) Obtain the grayscale image of cotton bolls: For the cotton boll position subimage in RGB color mode, select the green component image in RGB color mode to convert it into a grayscale image; for the cotton boll position subimage in HSI color mode, select HSI color mode The saturation component image in is converted into a grayscale image;

(2-3) Obtain the binary image of cotton peach: set the threshold, convert the grayscale image of the cotton peach position sub-image into a binary image;

(2-4) Obtain a preliminary segmentation image of cotton bolls: first, fill the intermediate holes in the multiple connected domains in the binarized image to obtain connected areas; then, use an edge detector to obtain the grayscale image of the cotton boll position sub-image Carry out edge detection, obtain cotton boll image edge; Finally, carry out intersection operation to the connected area of described binary image and cotton boll image edge, obtain cotton boll preliminary segmentation image;

(2-5) Obtain finely segmented images of cotton bolls: For the preliminary segmented images of cotton bolls, use the morphological image processing method to perform fine image segmentation, and finally obtain the boll segmented images.

Preferably, the edge detector in the step (2-4) of the automatic detection method for cotton boll-breaking stage is a canny edge detector.

Preferably, the step (3) of the automatic detection method for the cotton boll stage includes the following sub-steps:

(3-1) Extract the white image in the segmented boll internal area: in the already segmented boll area, adopt methods such as environment adaptive segmentation method, super green operator segmentation method, crop image segmentation method based on Mean Shift, etc. Carry out white segmentation again;

(3-2) Detect white cracks: For the white image segmented in step (3-1), extract the aspect ratio of its smallest circumscribing rectangle or the ratio of the length and short axis of its smallest circumscribing ellipse, as the shape feature descriptor; set the threshold , retaining white images whose shape feature descriptors are greater than or equal to the threshold as white cracks;

(3-3) Judging whether the cotton has reached the boll splitting stage: in a cotton boll position sub-image, count the number of white cracks, if the number of white cracks is greater than or equal to 1, it is judged that the cotton field has entered the boll splitting stage; otherwise, the next time For the captured cotton field image, repeat steps (1) to (3).

Generally speaking, compared with the prior art, the above technical solutions conceived by the present invention, because the present invention automatically performs image processing on the collected real-time forward-looking cotton field images, utilizes the method of machine learning to construct a cotton boll classifier, and combines Changes in the morphological characteristics of cotton bolls during the boll splitting stage, using color mode conversion and morphological image processing methods, and then judging whether cotton has entered the boll splitting stage. The method uses the cracking status of cotton bolls as the judgment basis, and judges the cotton growth period in real time. The detection result has a high accuracy rate, and has important guiding significance for cotton farming activities.

At the same time, the present invention uses the SIFT feature quantity as the local feature of the image, which maintains invariance to rotation, scaling, and brightness changes, and maintains a certain degree of stability to viewing angle changes, affine transformations, and noise; not only that, the SIFT feature is unique It has good performance and rich information, and is suitable for fast and accurate matching in massive feature databases.

The preferred solution is to select cotton boll images with clear edges, full bolls, and boll cracking that can be observed by human eyes as positive samples, which can significantly improve the classification effect of the classifier, so that bolls can be accurately detected from complex cotton field images position subimage.

The preferred solution is to use the LLC encoding method to encode the SIFT features, which can greatly shorten the detection time of cotton bolls.

The optimal scheme adopts the method of combining coarse search and fine search, which can shorten the detection time of cotton bolls under the premise of ensuring the detection accuracy of bolls.

The preferred solution is to use different color modes for image processing according to the position of the boll, which can ensure the accuracy of the boll image segmentation to the greatest extent.

The preferred solution is to detect the white cracks inside the bolls, which can avoid the image interference of the white cracks in the non-boll area, reduce the false positive rate, and thus accurately observe the boll cracking stage of the bolls.

Description of drawings

Fig. 1 is the cotton boll-cracking stage automatic detection method flowchart provided by the present invention;

Fig. 2 is the longitudinal front view of the cotton field taken at 20:00 on August 24, 2012;

Fig. 3 is embodiment 2 rough search cotton boll position figure;

Fig. 4 is the final boll position figure of embodiment 2;

Fig. 5 is the cotton boll image to be divided in embodiment 2;

Fig. 6 is a result figure of cotton boll position sub-image processing under RGB color mode;

Fig. 7 is the result figure that embodiment 2 detects white crack;

Fig. 8 is the longitudinal front view of the cotton field taken at 20:00 on August 31, 2012;

Fig. 9 is the coarse search cotton boll position map of embodiment 3;

Fig. 10 is the final boll position figure of embodiment 3;

Fig. 11 is the cotton boll image to be divided in embodiment 3;

Fig. 12 is the result figure of cotton boll position sub-image processing under HSI color mode;

Fig. 13 is a diagram showing the results of detecting white cracks in Example 2.

Wherein, Fig. 6 (a) is the edge detection result image of embodiment 2, Fig. 6 (b) is the cotton boll image that embodiment 2 has initially segmented, Fig. 6 (c) is the boll image after embodiment 2 morphological corrosion operation, Fig. 6 (d) is the image that embodiment 2 obtains the largest connected domain, Fig. 6 (e) is the binary image of bolls that are finally segmented in embodiment 2, and Fig. 6 (f) is the image of bolls that are finally segmented in embodiment 2;

Fig. 7 (a) is the white segmentation result image of embodiment 2, and Fig. 7 (b) is the white crack detection result image of embodiment 2;

Figure 12(a) is the image of the edge detection result in Example 3, Figure 12(b) is the image of the cotton boll that has been initially segmented in Example 2, and Figure 12(c) is the image of the cotton boll after the morphological erosion operation in Example 3, Figure 12 (d) is the image of the largest connected domain obtained in embodiment 3, Fig. 12 (e) is the binary image of the final segmented boll in embodiment 3, and Fig. 12 (f) is the final segmented boll image in embodiment 3;

Fig. 13(a) is an image of the white segmentation result of Example 3, and Fig. 13(b) is an image of the white crack detection result of Example 3.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

A kind of cotton boll-cracking stage automatic detection method provided by the invention comprises the following steps:

(1) Obtain the position of cotton bolls in the picture

The position of the boll refers to the sub-image of the cotton field containing the boll, preferably the range of the sub-image of the cotton field of 90×90.

(1-1) Collect images of cotton fields

The preferred cotton field image sequence in the present invention is the longitudinal front view image of the cotton field at 20:00 (ie 8:00 p.m.) every day. Image acquisition requirements: the camera is 0.3 meters above the ground, the focal length is 14 mm, the horizontal shooting direction is north, the angle with the horizon is 0 degrees, and the resolution is not less than 4 million pixels. Since the growth of cotton requires strong light, the smooth surface of bolls and leaves will produce specular reflections, which will negatively affect the identification of bolls and white cracks. Therefore, in the longitudinal front view taken at different times of the day at a fixed position and attitude, we chose to perform image processing on the images collected at 20:00 every day, so as to improve the recognition accuracy and reduce the difficulty of cotton boll-opening stage recognition.

The collected cotton field images are incorporated into the cotton field image sequence.

(1-2) Coarse search for the position of cotton bolls.

First, according to a certain rough search sequence, all the images in the cotton field image sequence are split into sub-images of M×N pixels with a coarse search step. Use the cotton boll classifier to judge the sub-images obtained by splitting, and obtain the tag value of each sub-image. Record the sub-image positions whose marker values exceed the set coarse search threshold as coarse search boll positions.

The rough search order affects the search program design, and the rough search order is preferably from left to right, from top to bottom; the coarse search step affects the time and accuracy of cotton boll search, the larger the step, the faster the splitting speed and the lower the accuracy. The smaller the length, the slower the splitting speed, the higher the accuracy, the coarse search step size is preferably 30 pixels; the coarse search threshold affects the performance of the coarse search, the higher the coarse search threshold, the higher the specificity, the lower the sensitivity, the coarse search threshold The lower the value, the lower the specificity and the higher the sensitivity. In order to balance the sensitivity and specificity, the boll classifier threshold with a false positive rate of 6% to 9% was selected as the coarse search threshold. The threshold can be used when the false positive rate is 6% to 9%, and the specific determination method of the threshold can be adjusted according to the image size and accuracy requirements.

(1-3) Fine search for the location of cotton bolls

For each coarse search boll position, a certain size range around it is used as the fine search range. In the fine search range, according to the fine search sequence, the image is split into M×N pixel sub-images with a fine search step. Use the cotton boll classifier to judge the sub-images obtained by splitting, and obtain the tag value of each sub-image. Set a fine search threshold, and record the subimage positions whose marker values exceed the fine search threshold as the fine search boll positions.

The fine search range affects the search speed of cotton bolls. The optimal fine search range is the image area that expands 5 pixels from the right to the bottom of the coarse search cotton boll position; the fine search order affects the search program design, and the fine search order is preferably from left to right, from top to bottom To the next; the fine search step length affects the time and precision of cotton boll search, the larger the step size, the faster the splitting speed, the lower the precision, the smaller the step size, the slower the splitting speed, the higher the precision, because the fine search requires higher search precision , and the search range is small, so the fine search step size is preferably 1 pixel; the fine search threshold affects the performance of the fine search, the higher the fine search threshold, the higher the specificity, the lower the sensitivity, and the lower the fine search threshold, The lower the specificity, the higher the sensitivity. In order to ensure that the searched bolls are real bolls, the boll classifier threshold that makes the false positive rate within 1% is selected as the fine search threshold. The threshold of the boll classifier that makes the false positive within 1% is acceptable, and the specific determination method of the threshold can be adjusted according to the image size and accuracy requirements.

(1-4) Record and track boll positions.

Due to the impact of boll position offset caused by wind direction, leaf occlusion and camera shake, and in order to reduce the missed detection rate of bolls, the set fine search step size should not be small, so as to avoid the same boll being detected multiple times in a picture It is detected that, for the results of the fine search, the non-maximum value suppression method is used to remove redundant boll positions, that is, the boll positions recorded within the fine search range, and only the position with the largest marker value of the boll classifier is reserved as the final boll position. And record this position as the boll position.

(1-5) Get all boll positions.

Record the boll position in all images in the image sequence as the final boll position, and select the sub-image of the final boll position in the images collected in step (1-1) as the subsequent boll segmentation process.

Since the camera is fixed, the growth of bolls is relatively stable. Therefore, for a sequence of images that take the same scene as the framing object, choosing to track and observe the positions of all detected bolls can further ensure the detection rate of bolls. , reduce the missed detection rate, and can comprehensively track and observe the growth and development of cotton bolls, and reduce the influence of wind direction, shading caused by leaf growth and other factors.

The boll classifier is constructed according to the following method:

First, M×N pixel images are used as positive and negative samples for training. Among them, the positive samples are pictures of cotton bolls in different postures. In order to reduce the false positive rate, only choose pictures of cotton bolls with clearer edge outlines, fuller, and human eyes can observe the state of boll cracking, so that the cracks of cotton bolls can be observed after cracking; Select sub-images of non-cotton bolls in the longitudinal front view of the cotton field as negative samples for training. The positive and negative samples for training should be compatible with the cotton boll image and can clearly and completely reflect the shape of the boll. The preferred image size is 90×90 pixels.

Then, the SIFT feature quantity in the obtained positive and negative sample pictures is extracted, and the SIFT feature quantity is subjected to local constraint linear coding to obtain the positive training sample and negative training sample feature quantity input to the classifier. SIFT (scale invariant feature transform) feature quantity, that is, scale invariant feature transformation feature quantity, see the literature David G.Lowe.Object recognition from local scale-invariant features[C].International Conference on Computer Vision,Corfu,Greece,1999(9): 1150-1157. and David G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints [C]. International Journal of Computer Vision, 60, 2(2004): 91-110. In order to improve the efficiency of the algorithm, LLC (Locality-constrained Linear Coding) is used to encode the extracted SIFT features to obtain the reconstructed local features after encoding. For feature coding, there are methods such as hard voting and sparse coding. Among them, LLC coding is a method of coding features based on the bag-of-features model or codebook model. In the classification task of natural images Excellent performance. See literature Wang J J, Yang J C, Yu K, et al.Locality-constrained linear coding for image classification［C］.IEEEConference on Computer Vision and Pattern Recognition(CVPR).2010:3360-3367. The features obtained by SIFT features through LLC encoding are only local features, which is equivalent to the reconstruction of the original SIFT features by LLC encoding.

Finally, use the support vector machine (SVM) as the classifier, select the radial basis (RBF) kernel function, input the feature quantities of positive training samples and negative training samples, train the classifier, and obtain the cotton boll classifier. Support Vector Machine SVM (SupportVector Machine) was first proposed by Vapnik in 1995. As a trainable machine learning method, SVM has certain advantages in solving small sample, nonlinear and high-dimensional patterns. The support vector machine method is to seek the best compromise between the complexity of the model and the learning ability according to the limited sample information, in order to obtain the best generalization ability (generalization ability). Due to the limitation of the image, the number of positive samples of bolls with multiple poses obtained is not much, and the appropriate boll classifier can be obtained by using the SVM method for training. If the capacity of positive samples and negative samples is enlarged, a cotton boll classifier with better classification effect will be obtained.

(2) Segment the cotton boll image

The cotton field image obtained in step (1-1) contains rich content such as the sky, cotton plants, land, weeds, etc. The local environment of cotton bolls grown in different locations is also relatively complex and different. The environmental factors in the image will still affect the segmentation effect. Therefore, we segment the bolls in the sub-images in different color modes to obtain boll images.

(2-1) Determine the position of bolls.

Put the bolls in different positions in different color modes for image processing.

If the boll position is in the 1/3 area of the entire image, then the boll image is considered to be in contact with the sky, and the boll position sub-image is processed in the RGB color mode, and the boll position sub-image of the RGB color mode is retained; otherwise, Considering that the boll image is not in contact with the sky, the boll position sub-image is converted from the RGB color mode to the HSI color mode for processing, that is, it is converted into the boll position sub-image of the HSI color mode.

(2-2) Obtain the grayscale image of cotton bolls.

Segment the bolls in RGB space:

Convert the sub-image in the rectangular frame of the cotton boll position to RGB color mode, that is, convert the color image into red, green, and blue component images. In order to achieve the best effect of image segmentation processing, the green component image in the RGB color mode is selected for processing, that is, the RGB color image of the cotton peach position sub-image is converted into a grayscale image.

Segment the bolls in HSI space:

Convert the sub-image within the rectangular frame of the boll position to the HSI color mode, that is, the hue, saturation, and brightness component images. In order to achieve the best effect of image segmentation processing, the saturation component image in the HSI color mode is selected for processing, that is, the color image of the sub-image of the cotton peach position is converted into a grayscale image and the histogram equalization operation is performed on it to obtain the gray level of the cotton peach image.

(2-3) Obtain the binary image of cotton bolls.

Convert the grayscale image of the boll position sub-image into a binary image according to the set threshold: Since the boll area we want is a large connected domain, in order to ensure the integrity of the boll connected domain and the convenience of subsequent operations, For any pixel whose gray value is greater than the threshold, the gray value of the pixel is changed to 0, otherwise the gray value of the pixel is changed to 255, so that the large connected area is a white image as a cotton boll binary image, otherwise it will be The image is inverted as a binary image of cotton bolls.

(2-4) Obtain a preliminary segmentation image of cotton bolls.

First, fill the holes in the middle of multiple connected regions in the binarized image to obtain connected regions; then, use an edge detector, such as a canny edge detector, to perform edge detection on the grayscale image of the cotton peach position sub-image to obtain The edge of the cotton boll image; finally, the intersecting operation is performed on the connected area of the binary image and the edge of the boll image to obtain a preliminary segmentation image of the boll.

(2-5) Obtain finely segmented images of bolls.

For the preliminary segmented image of cotton bolls, the morphological image processing method is used for fine image segmentation, and finally the segmented images of cotton bolls are obtained.

(3) Judging the bell-cracking period.

(3-1) Extract the white image in the segmented inner region of the boll.

Carry out white segmentation again in the already segmented cotton boll area. The specific segmentation methods can use the environment adaptive segmentation method, the super green operator segmentation method, the crop image segmentation method based on Mean Shift and so on. (See [1] Lei F.Tian. Environmentally adaptive segmentation algorithm for outdoor image segmentation. Computers and electronics in agriculture, 1998, 21:153～168); [2] D.M.Woebbecke, G.E.Meyer, K.Von Bargen, D.A.Mortensen. Color Indices for weed identification under various soil, residue, and lighting conditions.Transactions of the ASAE,1995,38(1):259～269); [3]Zheng L, Zhang J, Wang Q.Mean-shift-based color segmentation of images containing greenvegetation. Computers and Electronics in Agriculture, 2009, 65:93-98.)

(3-2) Detection of white cracks.

The segmentation method is used to detect the white color on the finely segmented image of the boll, and the boll region is segmented with the help of the "long and narrow" shape characteristics of the white crack itself.

Due to the high reflection of leaves and the influence of segmentation accuracy, the white detected after the white segmentation operation is not all the white cracks that appear after the bolls are cracked. Therefore, the image should be processed according to the shape characteristics of the connected domain to be segmented.

For the white image segmented in step (3-1), extract the aspect ratio of the smallest circumscribing rectangle or the ratio of the length and the short axis of the smallest circumscribing ellipse as the shape feature descriptor; keep its shape feature descriptor greater than or equal to the set A white image with a given threshold is used as a white crack.

Shape feature descriptors can use Fourier descriptors, eccentricity, the aspect ratio (ratio of long and short axes) of the smallest circumscribed rectangle (or ellipse) of the connected domain, and so on.

The preferred solution is to segment the detected white area according to the shape characteristics of the white connected domain (white crack) to be detected by using the "narrowness" of the white crack itself, and the corresponding selected shape descriptor is the length of the smallest circumscribed ellipse of the connected domain Axis ratio, remove the white connected domains whose long-short axis ratio is less than the threshold, and keep the connected domains whose long-short axis ratio is greater than or equal to the set threshold.

(3-3) Judging whether cotton has reached the boll splitting stage.

After obtaining the final segmented white crack image, count the number of white cracks (white narrow and long connected domains). Since each positive sample used for training contains only one boll, when using the classifier obtained after training to detect images, there is only one boll in the detected rectangular area containing bolls each time. There are at most 5 cracks in a boll, but due to the influence of the shooting angle of the boll, there may be only 1 or 2 cracks that can be observed from the image. Therefore, if a white crack is detected, it is judged that the cotton has entered the boll splitting stage; otherwise, it is judged that the cotton has not entered the boll splitting stage.

The following are examples:

Example 1

Build a boll classifier:

First, a picture with a size of 90×90 pixels is selected as positive and negative samples for training. Among them, the positive samples are pictures of cotton bolls in different postures. The edges of the bolls in the boll pictures are clear and full, and the boll cracking can also be observed by the human eye; the negative samples are sub-image pictures of non-cotton bolls in the longitudinal front view of the cotton field.

Then, the SIFT feature quantity in the obtained positive and negative sample pictures is extracted, and the SIFT feature quantity is subjected to local constrained linear (LLC) encoding to obtain the positive training sample and negative training sample feature quantity input to the classifier.

Finally, use the support vector machine (SVM) as the classifier, select the radial basis (RBF) kernel function, input the feature quantities of positive training samples and negative training samples, train the classifier, and obtain the cotton boll classifier.

Example 2

Use the method provided by the invention to judge whether the cotton in Fig. 2 enters the boll splitting stage:

(1) Obtain the position of cotton bolls in the picture.

(1-1) Collect longitudinal front view image sequences of cotton fields.

The camera is 0.3 meters above the ground, the focal length is 14 mm, the horizontal shooting direction is north, the angle with the horizon is 0 degrees, and the resolution is 4 million pixels. The longitudinal front image of the cotton field is collected at 20:00 (that is, 8:00 p.m.), as shown in Figure 2 Show.

(1-2) Coarse search for the position of cotton bolls.

Firstly, the image obtained in step (1-1) is split to obtain sub-images of 90×90 pixels according to the order from top to bottom and from left to right, with a step size of 30 pixels. The boll classifier obtained in Example 1 is used to judge the sub-images to obtain the tag value of each sub-image. Set the coarse search threshold to be the boll classifier threshold when the false positive rate is between 6% and 9%. Record the position of the sub-image whose tag value exceeds the coarse search threshold as the coarse search boll position, as shown in Figure 3.

(1-3) Fine search for the location of cotton bolls

For each coarse search boll position in Figure 3, the image area extending 5 pixels to the right and downward near the coarse search boll position is taken as the fine search range. Within the fine search range, the image is split into sub-images of 90 × 90 pixels with a step size of 1 pixel in order from left to right and top to bottom. Use the cotton boll classifier obtained in Example 1 to judge the sub-images obtained by splitting, and obtain the tag value of each sub-image. Set the fine search threshold to be the boll classifier threshold when the false positive rate is equal to 1%, and record the sub-image positions whose marker values exceed the fine search threshold as the fine search boll positions.

(1-4) Record and track boll positions.

For each coarse search boll position obtained by the coarse search in step (1-2), all the fine search boll positions within the extended search range are subject to maximum value constraints, that is, all the fine search boll positions are given according to their boll classifiers. Sort the marked values, and only retain the position of the finely searched boll with the largest marked value as the final position of the boll, and record this position as the boll position.

(1-5) Get all boll positions.

In the longitudinal front-view image of the cotton field, the subsequent image processing of the segmented bolls is performed on all the final boll position sub-images detected before the recorded detection day, as shown in Figure 4.

(2) Segment the boll image.

(2-1) Determine the position of bolls.

As shown in Figure 5, the boll to be detected is in the black frame, and the boll position is within the upper third of the entire image. It is considered that the boll is in contact with the sky, and the sub-image of the boll position is processed in the RGB color mode.

(2-2) Obtain the grayscale image of cotton bolls.

Convert the sub-image in the rectangular frame of the cotton boll position to RGB color mode, that is, convert the color image into red, green, and blue component images. Select the green component image in the RGB color mode for processing, and convert the RGB color image of the cotton peach position sub-image into a grayscale image.

(2-3) Obtain the binary image of cotton bolls.

Set the threshold to 240 to convert the grayscale image of the boll position sub-image into a binary image: Since the boll area we want is a large connected domain, in order to ensure the integrity of the boll connected domain and the convenience of subsequent operations, any If the grayscale value of the pixel is greater than the threshold, the grayscale value of the pixel is changed to 0, otherwise the grayscale value of the pixel is changed to 255.

(2-4) Obtain a preliminary segmentation image of cotton bolls.

Fill the holes in the middle of multiple connected domains in the binarized image to obtain large connected regions. Use the canny edge detector to detect the edge of the gray image of the boll position sub-image, as shown in Figure 6(a), and perform intersection operation to obtain the preliminary segmentation image of the boll, as shown in Figure 6(b).

(2-5) Obtain finely segmented images of bolls.

For the preliminary segmented image of cotton bolls, the morphological image processing method is used for fine image segmentation, and finally the segmented images of cotton bolls are obtained.

First, the image is corroded, and a rectangular box with a size of 3×3 pixels is selected as the structural element used for corrosion. This is because the image after the intersection operation cannot distinguish two different connected domains well. After the image is corroded, Although the area of the connected domain will be reduced, the largest cotton boll connected domain can be separated from other small connected domains as much as possible, as shown in Figure 6(c).

Then find the largest connected domain obtained by corroding the image, mark the gray value of the pixel in the entire connected domain as 0, and mark the gray value of other pixels in the rectangular area as 1, as shown in Figure 6(d) .

Then, the expansion operation is performed on the image, and a rectangular box with a size of 4×4 pixels is selected as the structural element used for expansion, and the holes in the maximum connected domain are filled. In order to keep the shape of the target object after the erosion and expansion operations unchanged as much as possible, the structural elements used in the expansion operation should be the same as those used in the erosion operation, and a rectangular box with a size of 4×4 pixels is selected as the structural element used in the erosion. A binary image with a gray value of 0 in the cotton boll area and a gray value of 1 in other positions in the rectangular area is obtained, as shown in Figure 6(e).

Finally, the obtained binary image is dot-multiplied with each channel of the RGB image of the cotton boll position sub-image, and finally the segmented boll image is obtained, as shown in Figure 6(f).

(3) Judging the bell-cracking period.

(3-1) Detect white cracks in the segmented inner region of the boll.

In the already segmented cotton boll area, the environment adaptive segmentation method is used to perform white segmentation again, the purpose is to detect and extract the white cracks in the boll, and the segmentation result is shown in Figure 7(a).

(3-2) Detection of white cracks.

According to the shape characteristics of the white connected domain (white crack) to be detected, the detected white area is segmented by using the "narrowness" of the white crack itself, and the corresponding selected shape descriptor is the ratio of the major axis to the smallest circumscribed ellipse of the connected domain Set the threshold to 5.8, remove the white connected domains whose long-short axis ratio is less than the threshold, and keep the connected domains whose long-short axis ratio is greater than or equal to the threshold.

(3-2) Judging whether the cotton has reached the boll splitting stage.

After the final segmented white crack image is obtained, the number of white cracks (white narrow and long connected domains) is counted, as shown in Figure 7(b). No cracks were detected and the cotton did not enter the boll stage.

Example 3

Use the method provided by the invention to judge whether the cotton in Fig. 8 enters the boll splitting stage:

(1) Obtain the position of cotton bolls in the picture:

(1-1) Collect images of cotton fields

The camera is 0.3 meters above the ground, the focal length is 14 mm, the horizontal shooting direction is north, the angle with the horizon is 0 degrees, and the resolution is 4 million pixels. The longitudinal front image of the cotton field is collected at 20:00 (that is, 8:00 p.m.), as shown in Figure 8 Show.

(1-2) Coarse search for the position of cotton bolls.

Firstly, the image obtained in step (1-1) is split to obtain sub-images of 90×90 pixels in the order of top-down and left-to-right with a step size of 30 pixels. The boll classifier obtained in Example 1 is used to judge the sub-images to obtain the tag value of each sub-image. Set the coarse search threshold to be the boll classifier threshold when the false positive rate is equal to 9%. Record the position of the sub-image whose tag value exceeds the coarse search threshold as the coarse search boll position, as shown in Figure 9.

(1-3) Fine search for the location of cotton bolls

For each coarse search boll position in Fig. 9, the image area extending 5 pixels to the right and downward near the coarse search boll position is taken as the fine search range. Within the fine search range, the image is split into sub-images of 90 × 90 pixels with a step size of 1 pixel in order from left to right and top to bottom. Use the cotton boll classifier obtained in Example 1 to judge the sub-images obtained by splitting, and obtain the tag value of each sub-image. Set the fine search threshold to be the boll classifier threshold when the false positive rate is equal to 1%, and record the sub-image positions whose marker values exceed the fine search threshold as the fine search boll positions.

(1-4) Record and track boll positions.

For each coarse search boll position obtained by the coarse search in step (1-2), all the fine search boll positions within the extended search range are subject to maximum value constraints, that is, all the fine search boll positions are given according to their boll classifiers. Sort the marked values, and only retain the position of the finely searched boll with the largest marked value as the final position of the boll, and record this position as the boll position.

(1-5) Get all boll positions.

In the longitudinal front-view image of the cotton field, the sub-images of all the final boll positions detected before the recorded detection day are subjected to subsequent segmentation boll image processing, as shown in Figure 10.

(2) Segment the boll image.

(2-1) Determine the position of bolls.

As shown in Figure 11, the bolls to be detected are in the black frame, and the bolls are located within the lower two-thirds of the entire image. It is considered that the bolls are not in contact with the sky, and the bolls position sub-image is processed in the HSI color mode. .

(2-2) Obtain the grayscale image of cotton bolls.

Convert the sub-image within the rectangular frame of the boll position to the HSI color mode, that is, the hue, saturation, and brightness component images. In order to achieve the best effect of image segmentation processing, the saturation component image in the HSI color mode is selected for processing, that is, the color image of the sub-image of the cotton peach position is converted into a grayscale image and the histogram equalization operation is performed on it to obtain the gray level of the cotton peach image.

(2-3) Obtain the binary image of cotton bolls.

Set the threshold to 240 to convert the grayscale image of the boll position sub-image into a binary image: Since the boll area we want is a large connected domain, in order to ensure the integrity of the boll connected domain and the convenience of subsequent operations, any If the grayscale value of the pixel is greater than the threshold, the grayscale value of the pixel is changed to 0, otherwise the grayscale value of the pixel is changed to 255.

(2-4) Obtain a preliminary segmentation image of cotton bolls.

Fill the holes in the middle of multiple connected domains in the binarized image to obtain large connected regions. Use the canny edge detector to detect the edge of the gray image of the boll position sub-image, as shown in Figure 12(a), and perform intersection operation to obtain the preliminary segmentation image of the boll, as shown in Figure 12(b).

(2-5) Obtain finely segmented images of bolls.

For the preliminary segmented image of cotton bolls, the morphological image processing method is used for fine image segmentation, and finally the segmented images of cotton bolls are obtained.

First, the image is corroded, and a rectangular box with a size of 3×3 pixels is selected as the structural element used for corrosion. This is because the image after the intersection operation cannot distinguish two different connected domains well. After the image is corroded, Although the area of the connected domain will be reduced, the largest cotton boll connected domain can be separated from other small connected domains as much as possible, as shown in Figure 12(c).

Then find the largest connected domain obtained by corroding the image, mark the gray value of the pixels in the entire connected domain as 0, and mark the gray value of other pixels in the rectangular area as 1, as shown in Figure 12(d) .

Then, the expansion operation is performed on the image, and a rectangular box with a size of 4×4 pixels is selected as the structural element used for expansion, and the holes in the maximum connected domain are filled. In order to keep the shape of the target object after the erosion and expansion operations unchanged as much as possible, the structural elements used in the expansion operation should be the same as those used in the erosion operation, and a rectangular box with a size of 4×4 pixels is selected as the structural element used in the erosion. A binary image with a gray value of 0 in the cotton boll area and a gray value of 1 in other positions in the rectangular area is obtained, as shown in Figure 12(e).

Finally, the obtained binary image is dot-multiplied with each channel of the RGB image of the cotton boll position sub-image, and finally the segmented boll image is obtained, as shown in Figure 12(f).

(3) Judging the bell-cracking period.

(3-1) Detect white cracks in the segmented inner region of the boll.

In the already segmented cotton boll area, the environment adaptive segmentation method is used to perform white segmentation again, the purpose is to detect and extract the white cracks in the boll, and the segmentation result is shown in Figure 13(a).

(3-2) Detection of white cracks.

According to the shape characteristics of the white connected domain (white crack) to be detected, the detected white area is segmented by using the "narrowness" of the white crack itself, and the corresponding selected shape descriptor is the long-short axis ratio of the smallest circumscribed ellipse of the connected domain, Set the threshold to 5.8, remove the white connected domains whose long-short axis ratio is less than the threshold, and keep the connected domains whose long-short axis ratio is greater than or equal to the threshold.

(3-2) Judging whether cotton has reached the boll splitting stage.

After the final segmented white crack image is obtained, the number of white cracks (white narrow and long connected domains) is counted, as shown in Figure 13(b). A crack is detected and the cotton enters the boll splitting stage.

It is easy for those skilled in the art to understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention.

Claims (10)

1. a kind of cotton splits bell phase automatic testing method, it is characterised in that comprises the following steps：

(1) cotton boll position in picture is obtained：Cotton field image is gathered, and described image is included into cotton field image sequence；By cotton field figure As all images in sequence, according to a fixed step size, M × N pixel subgraphs are split into；To all subgraphs, cotton boll point is used Class device judges whether to include cotton boll：For the subgraph comprising cotton boll, its position in original image is recorded as final cotton boll Position；The cotton boll grader is established as follows：Positive negative sample of the picture of M × N pixels as training is selected, wherein Positive sample be different postures cotton boll picture, negative sample be cotton field in non-cotton boll picture, extract SIFT feature amount, training support to Amount machine so that SVMs false positive rate is minimum, so as to build cotton boll grader；

(2) cotton boll image is split：In step (1) in the cotton field image of collection, the final corresponding subgraph in cotton boll position is obtained, Gray level image is converted to using RGB color pattern or HSI color modes, then by greyscale image transitions into continuous binary map Picture, cotton boll edge is detected using morphological images processing method, splits cotton boll image；

(3) judge to split the bell phase：The cotton boll image being partitioned into for step (2), carries out white Crack Detection, and extracts white and split Seam, according to the shape facility in white crack, determines whether cotton boll crack；Then think that cotton enters if there is cotton boll crack to split The bell phase, otherwise it is assumed that cotton, which does not enter into, splits the bell phase.

2. cotton as claimed in claim 1 splits bell phase automatic testing method, it is characterised in that the cotton boll grader positive sample For edge contour clearly, full, human eye also observe that the cotton boll picture for splitting bell-shaped condition.

3. cotton as claimed in claim 1 or 2 splits bell phase automatic testing method, it is characterised in that SIFT feature amount is carried out Local restriction uniform enconding.

4. cotton as claimed in claim 1 splits bell phase automatic testing method, it is characterised in that the step (1) includes following Sub-step：

(1-1) gathers cotton field image, and described image is included into cotton field image sequence；

(1-2) coarse search cotton boll position：

First, according to certain coarse search order, all images in the image sequence of cotton field are split into by M × N with coarse search step-length The subgraph of pixel；Then, the subgraph obtained using cotton boll grader to fractionation is judged, obtains the mark of each subgraph Note value；Finally, the subgraph image position that its mark value exceedes coarse search threshold value is recorded, as coarse search cotton boll position；

(1-3) fine searching cotton boll position：

Firstly, for each coarse search cotton boll position, using a certain size scope near coarse search cotton boll position as carefully searching Rope scope；Then, in the range of fine searching, according to certain fine searching order, M × N pictures are splitted the image into fine searching step-length The subgraph of element；Next, the subgraph obtained using cotton boll grader to fractionation is judged, the mark of each subgraph is obtained Note value；Finally, the subgraph image position that its mark value exceedes fine searching threshold value is recorded, as fine searching cotton boll position；

(1-4) is recorded and is tracked cotton boll position：

To the fine searching cotton boll position of record in step (1-3), the cotton boll position of redundancy is removed using non-maxima suppression, is only protected The maximum position of cotton boll grader mark value is stayed as cotton boll final position, and records the position as cotton boll position；

(1-5) obtains all cotton boll positions：

The cotton boll position of all images is as final cotton boll position in records series, in the cotton field image of step (1-1) collection, The subgraph of final cotton boll position is selected as the processing of subsequent singulation cotton boll.

5. cotton as claimed in claim 4 splits bell phase automatic testing method, it is characterised in that the cotton field described in step (1-1) Image, it is cotton field longitudinal direction front view, its luminous intensity is identical with cotton boll classifier training sample, by the phase not less than 4,000,000 pixels Machine gathers, and the camera is high 0.3 meter from the ground, 14 millimeters of focal length, northwards sets, level shooting；When the IMAQ moment is 20.

6. cotton as claimed in claim 4 splits bell phase automatic testing method, it is characterised in that slightly tearing open described in step (1-2) It is from left to right, from top to bottom to divide order；The coarse search step-length is 30 pixels；The coarse search threshold value is so that false positive rate For 6%~9% cotton boll grader threshold value.

7. cotton as claimed in claim 4 splits bell phase automatic testing method, it is characterised in that carefully searching described in step (1-3) Rope scope is the image-region that coarse search cotton boll position respectively extends downwards to the right 5 pixels；The thin fractionation order for from left to right, From top to bottom；The fine searching step-length is 1 pixel；The fine searching threshold value is so that false positive rate is the cotton boll point within 1% Class device threshold value.

8. cotton as claimed in claim 1 or 2 splits bell phase automatic testing method, it is characterised in that the step (2) include with Lower sub-step：

(2-1) judges cotton boll position：When cotton boll position is in entire image in 1/3 region, then retain RGB color pattern Cotton boll position subgraph；Otherwise, it is converted into the cotton boll position subgraph of HSI color modes；

(2-2) obtains cotton boll gray level image：For the cotton boll position subgraph of RGB color pattern, select in RGB color pattern Green component image is converted into gray level image；For the cotton boll position subgraph of HSI color modes, select in HSI color modes Saturation degree component image be converted into gray level image；

(2-3) obtains cotton boll bianry image：By cotton boll position subgraph gray level image, according to the threshold value of setting, two-value is converted into Image；

(2-4) obtains cotton boll primary segmentation image：First, by holes filling among multiple connected domains in the bianry image, Obtain connected region；Then, using edge detector, rim detection is carried out to cotton boll position subgraph gray level image, obtains cotton Peach image border；Finally, the connected region to the bianry image and cotton boll image border carry out shipping calculation, and it is preliminary to obtain cotton boll Segmentation figure picture；

(2-5) obtains cotton boll subdivision and cuts image：For cotton boll primary segmentation image, carried out using morphological images processing method thin Image is split, and finally gives cotton boll segmentation figure picture.

9. cotton as claimed in claim 8 splits bell phase automatic testing method, it is characterised in that the edge described in step (2-4) Detector is canny edge detectors.

10. cotton as claimed in claim 1 splits bell phase automatic testing method, it is characterised in that step (3) includes following sub-step Suddenly：

(3-1) obtains white image inside cotton boll segmentation figure picture：In cotton boll segmentation figure picture, using environment self-adaption segmentation side Method, super green operator dividing method or the crop image partition method based on Mean Shift, split again, obtain in cotton boll image White image；

(3-2) detects white crack：For the white image obtained in step (3-1), the length and width of its minimum enclosed rectangle are extracted Than or its minimum external oval axial ratio, son is described as shape facility；Retain its shape facility and describe son and be more than or wait In given threshold white image as white crack；

(3-3) judges whether cotton reaches and splits the bell phase：In a cotton boll position subgraph, white crack number is counted, if There is white crack, then it is assumed that cotton field, which enters, splits the bell phase；Otherwise it is assumed that cotton field, which does not enter into, splits the bell phase.