CN117351430A - Stacked cage-raising laying hen zone-position egg-laying monitoring method and system based on computer vision - Google Patents

Abstract

The invention discloses a stacked cage-raising layer zone egg laying monitoring method based on computer vision, which comprises the following steps: step 1, obtaining video data of two layers of laying hen positions through inspection equipment; step 2, labeling the acquired video data, and forming a data set by the video image set and the label; step 3, a monitoring algorithm based on an improved deep Sort framework is constructed, wherein the monitoring algorithm comprises a target detection module, a track matching module, a track generation module and a positioning counting module; training a monitoring algorithm by adopting a data set to obtain a monitoring model; and 5, acquiring video data of the laying hen positions to be counted, and inputting the video data into a monitoring model to obtain the egg yield of the corresponding laying hen positions. The invention also provides a layered cage-raising laying hen zone-position egg production monitoring system. The method provided by the invention is designed aiming at the multilayer multi-row stacked cage-raising laying hen zone bit, so that the egg yield monitoring is more visual and accurate and the monitoring efficiency is higher.

Description

A computer vision-based egg production monitoring method for stacked caged laying hens and system

Technical field

The invention belongs to the field of animal husbandry and breeding technology, and in particular relates to a computer vision-based method and system for positional egg production monitoring of stacked caged laying hens.

Background technique

The development of deep learning has provided strong support for computer vision technology, which has significantly improved the accuracy and stability of monitoring methods based on computer vision. In particular, the wide application of deep learning technology has provided many advantages for image processing and target detection. The task provides a more accurate and efficient method to help reduce labor costs and monitoring errors. In recent years, some scholars have applied computer vision technology to the monitoring of egg production in caged laying hens. Zhao Chunjiang and others used QR codes to Labels and improved YOLO are used to locate and count single-layer chicken cages, but this method has certain limitations; on the one hand, due to the breeding process, the QR code label is easily exposed to chicken feces, food residues and other foreign matter and becomes stained or Tear it up, making it impossible to scan; on the other hand, the content in the QR code is fixed and cannot be modified. If you change the chicken coop information, you can only make it again. Pre- and post-maintenance requires a lot of labor costs. This method can only be applied to a single layer, the detection efficiency is low.

Patent document CN114842470A discloses an egg counting method and positioning system in a stacked cage mode. The method includes: an egg target tracking and counting module: based on the egg multi-category target monitoring model and the multi-target tracking DeepSort algorithm to count eggs, and Different types of eggs are distinguished to achieve separate counting; the speed display positioning and display identification module: using the convolution monitoring algorithm to design and train the speed display semantic segmentation model based on the UNet algorithm and the speed display target classification model based on the CNN algorithm . They introduced appearance features in the multi-target tracking process of egg recognition to improve the accuracy of tracking and recognition. This method has high requirements for egg video data. A speed sensor needs to be configured for every two conveyor belts. For multi-layer and multi-column chicken cages, It is troublesome to install the equipment and the positioning of chicken cages on site is not intuitive. In the breeding scene, the initial investment is large and the daily maintenance in the later period is troublesome.

Patent document CN116091473A discloses a method and system for monitoring the precise egg production performance of multi-channel areas in laying hen houses. The method includes multiple chicken cages arranged side by side, and each layer of chicken cages corresponds to a conveyor belt. After the laying hens place an order, the eggs are transported along the chicken cages. The bottom slope slides down to the conveyor belt. After the conveyor belt is opened, the eggs are transported forward to the egg collector. The monitoring system detects the eggs at the egg crossing position on the video screen to monitor the location and quantity of eggs. This method uses computer vision technology to measure egg speed, which has high requirements on the monitoring system. Moreover, for large chicken farms, multiple monitoring systems need to be deployed. The positioning of chicken cages is not intuitive enough and building the system is time-consuming and laborious.

Contents of the invention

The main purpose of the present invention is to provide a computer vision-based method and system for monitoring the egg production of laminated caged laying hens. This method is designed for the location of multi-layered and multi-column laminated caged laying hens, and each chicken cage is The egg production monitoring is more intuitive and accurate and the monitoring efficiency is higher. It requires less monitoring equipment and does not use any external auxiliary tags. It is very simple to maintain in the future and can also ensure efficient unmanned monitoring.

In order to achieve the first object of the present invention, a computer vision-based method for monitoring egg production in stacked caged laying hens is provided, including:

Step 1. Obtain video data of two layers of laying hens through inspection equipment. The inspection equipment includes two monocular cameras with the same overlooking angle but different heights;

Step 2. Intercept the acquired video data frame by frame to construct a video image collection, label the video image collection with the cage columns and eggs in the image, and combine the video image collection and labels to form a data set;

Step 3. Construct a monitoring algorithm based on the improved Deepsort framework. The monitoring algorithm includes a target detection module, a trajectory matching module, a trajectory generation module and a positioning counting module. The target detection module includes a target detection module. The target detection module is used to Calibrate the number of the cage column in the input video data and identify the input video data to generate detection information in each frame of the image. The detection information includes a detection frame obtained by identification or a prediction frame obtained by prediction, and the detection frame includes an egg. Matching trajectories and corresponding shape features and trajectory motion parameters, the trajectory matching module performs multiple rounds of IOU matching based on the detection frame of the current frame and the prediction frame obtained by prediction of the previous frame, and performs cost matrix analysis based on the matching results to output Corresponding linear matching relationship, the trajectory generation module performs cascade matching based on the linear matching relationship between the detection frame and all prediction frames, combined with the corresponding shape features, trajectory motion parameters and trajectories of the detection frame to generate the trajectory image of the egg , the positioning and counting module locates each laying hen location according to the number of the cage column, and combines the egg trajectory image to generate the egg production number corresponding to the egg location;

Step 4. Use the data set to train the monitoring algorithm to obtain a monitoring model for monitoring egg production in laying hen locations;

Step 5: Obtain the video data of the laying hen location to be counted, and input it into the monitoring model to obtain the egg production volume of the corresponding laying hen location.

The present invention only needs two industrial cameras to perform real-time monitoring of multi-layer and multi-row chicken cages in a top-down manner. At the same time, the target tracking part of the model is updated according to the selection of hardware. By replacing the special target detection module for the laying hen location and adding the third The secondary IOU matching can greatly improve the accuracy and speed of number positioning of multi-layer and multi-column laying hens and egg counting of corresponding numbered laying hens.

Specifically, before the inspection equipment is put into use, the two monocular cameras need to be calibrated and de-distorted to ensure the accuracy of egg production monitoring in different layers of laying hens.

Specifically, the expression of the calibration dedistortion process is as follows:

In the formula, Z _c is the scale factor, is the distance from the optical center to the image plane of the laying hen location, u and v are the horizontal and vertical coordinates of the laying hen location in the pixel coordinate system, f _x and f _y are the normalized focal lengths on the x-axis and y-axis of the laying hen location, R _3×3 is the rotation matrix obtained by camera calibration, X _W , Y _W and Z _W are the world coordinate system,/> are the laying hen location distortion correction coordinates, k ₁ and k ₂ are radial distortions, p ₁ and p ₂ are tangential distortions, r is the radius of curvature, /> are the distorted coordinates of the laying hen location.

Specifically, the construction process of the video image set is as follows:

Use the VideoCapture function to intercept the acquired video data with a preset number of frames to obtain the initial image collection;

Perform data cleaning and data expansion on the acquired initial image set to obtain a video image set containing the location of laying hens.

The data cleaning includes data deduplication, data filtering and data repair.

The data augmentation includes HSV data augmentation, Brightness data augmentation and Mixup data augmentation.

Specifically, the monitoring algorithm also includes a global attention module, which is used to operate the attention mechanism in multiple dimensions of each frame of the video image of the input video data, and to perform feature extraction in different dimensions. Multiply in turn to obtain feature-enhanced video data and input it to the target detection module. The introduction of a global attention mechanism operation can better capture the dependence between channels and spatial dimensions in the cage column and egg feature maps, thereby significantly improving the model. performance.

Specifically, the target detection module also introduces the Ghost module, which is used to reduce the number of parameters of the model and generate more feature maps with fewer parameters through grouped convolution and linear operations, which greatly reduces It reduces the complexity of the model, reduces the number of network parameters, and improves the model computing speed.

Specifically, the trajectory matching module uses the Hungarian algorithm to solve the cost matrix to obtain the corresponding linear matching relationship. The linear matching relationship includes trajectory mismatch, the detection frame is not matched, and the detection frame and the prediction frame are successfully matched;

If the matching result is a trajectory mismatch or the detection frame is not matched, the IOU matching is repeated until the end condition is reached.

Specifically, in the case of trajectory mismatch, trajectories with more than 50 consecutive mismatched frames will be deleted directly.

Specifically, for the case where the detection frames are not matched, these unmatched detection frames are initialized as new trajectories.

Specifically, the positioning and counting module generates the area of interest LROI1 for the upper layer hens and the area of interest LROI2 for the lower layer hens through the coordinates of the upper and lower cage columns and numbers them in real time, and then counts the eggs in the LROI1 and LROI2 areas. After the video ends, the egg count values of all frames in each LROI1 and LROI2 area are calculated to obtain the final laying hen location egg laying observation results.

Specifically, during training, the WIoU loss function is used to train the target detector of the monitoring algorithm to update the parameters of the target detector.

Specifically, the expression of the loss function is as follows:

L _WIoUv1 ＝R _WIoU L _IoU (3)

In the formula, x _g and y _g represent the width and height of the predicted box, x _gt and y _gt represent the width and height of the real box, W _g and H _g represent the width and length of the minimum rectangle formed by the combination of the predicted box and the real box, above The mark * represents not participating in the calculation, effectively eliminating factors that hinder convergence. R _WIoU represents the penalty term of WIoU that will enhance the loss of ordinary quality anchor boxes. β represents outliers, which is used to assign a gradient weight to high-quality anchor boxes. Gain, r′ is the gradient gain, α and δ are hyperparameters, and L _IoU represents the IOU loss.

In order to achieve the second object of the present invention, a laminated caged laying hen positional egg production monitoring system is provided, which is realized by the above laminated caged layering hens positional egg production monitoring method, including a chicken farm management unit, and an egg production monitoring system. units and system units;

The chicken farm management unit includes chicken farm environment monitoring, chicken farm feeding management and chicken farm personnel management;

The egg production monitoring unit includes chicken cage monitoring, egg production monitoring and abnormal alarm;

The system management unit includes a system setting part for deploying a chicken farm management unit and an egg production monitoring unit, a system user management part for managing accounts, and an overall data management part for storing daily monitoring results.

Compared with the existing technology, the beneficial effects of the present invention are:

The monitoring model provided by the present invention combines the tracking algorithm with the external characteristics (cage pillars) of the laying hen location to position and number the laying hen location. It does not require any external auxiliary labels, is simple to implement, and requires almost no labor cost. The real-time egg production monitoring is more intuitive and efficient, and the equipment maintenance is very simple. At the same time, the tracking algorithm has been redesigned according to the existing scenario, the special target detection module for the laying hen location has been replaced, and the second IOU matching has been added to monitor multi-layer and multi-row laying hens. The accuracy and speed of location number positioning and egg counting of corresponding numbered laying hens have been greatly improved. In addition, only two industrial cameras are needed to conduct real-time monitoring of multi-layer and multi-column cages. With less equipment, monitoring is faster and more efficient.

Description of drawings

Figure 1 is a flow chart of a computer vision-based egg production monitoring method for layered caged laying hens provided in this example;

Figure 2 is a schematic structural diagram of the inspection equipment provided in this embodiment;

Figure 3 is a schematic structural diagram of the monitoring algorithm provided in this embodiment;

Figure 4 is a work flow chart of positioning counting provided by this embodiment;

Figure 5 is a schematic structural diagram of the laying hen location egg production monitoring system provided in this embodiment;

Figure 6 is a histogram of monitoring data for test number 1 by the monitoring module provided in this embodiment;

Figure 7 is a histogram of the wrong detection chicken cage data of test number 1 by the monitoring module provided in this embodiment;

Figure 8 is a histogram of monitoring data for test number 2 by the monitoring module provided in this embodiment;

Figure 9 is a histogram of incorrectly detected chicken cage data for test number 2 by the monitoring module provided in this embodiment;

Figure 10 is a histogram of monitoring data for test number 3 by the monitoring module provided in this embodiment;

Figure 11 is a histogram of incorrectly detected chicken cage data for test number 3 by the monitoring module provided in this embodiment;

Figure 12 is a histogram of monitoring data for test number 4 by the monitoring module provided in this embodiment;

Figure 13 is a histogram of the wrong detection chicken cage data of test number 4 by the monitoring module provided in this embodiment.

Detailed ways

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. As long as there is no inventive step, all other embodiments obtained by ordinary technical managers belong to the protection scope of the present invention.

As shown in Figure 1, the laying hen location egg production monitoring method provided in this embodiment includes:

As shown in Figure 2, the stacked cage laying hen location includes a multi-layered and multi-row laying hen cage 2 characterized by two cage columns 1, and the eggs produced by the laying hens roll through a certain slope of the rolling angle 3 to the laying hens. Metal bottom mesh platform 4 under the chicken coop.

Step 1. Place two X001 monocular industrial cameras 5 on the second and fourth layer of the camera bracket 6 0.58m away from the laying hen cage, and look down at the non-overlapping images at a 45-degree angle. The two-layer laying hen position field of view method is used with Raspberry Pi 7 and inspection robot 8 to build inspection equipment and calibrate and dedistort its two monocular industrial cameras to ensure accurate monitoring of egg production in different layers of laying hens. accuracy.

In this embodiment, the calculation formula for calibrating and de-distorting two monocular industrial cameras is:

In the formula, Z _c is the scale factor, is the distance from the optical center to the image plane of the laying hen location, u and v are the horizontal and vertical coordinates of the laying hen location in the pixel coordinate system, f _x and f _y are the normalized focal lengths on the x-axis and y-axis of the laying hen location, R _3×3 is the rotation matrix obtained by camera calibration, X _W , Y _W and Z _W are the world coordinate system,/> are the laying hen location distortion correction coordinates, k ₁ and k ₂ are radial distortions, p ₁ and p ₂ are tangential distortions, r is the radius of curvature, /> are the distorted coordinates of the laying hen location.

Step 2. Use the inspection equipment constructed in step 1 to obtain the video data of the two layers of laying hens along the prescribed path at a speed of 0.3m/s, and perform frame interception processing to obtain characteristic picture information and perform data preprocessing to sort out applicable Recognition data set for YOLO model and tracking data set for DeepSort model. Data set categories include cage columns and eggs.

In this embodiment, it specifically includes the following steps:

Step 2.1. Collect the videos of laminated laying hens at different heights, and use the VideoCapture function to intercept the videos in 8 frames to obtain 754 pictures;

Step 2.2. Perform data cleaning on the data set intercepted in step 2.1, including data deduplication, data filtering, and data repair, and obtain 743 laying hen location pictures;

Step 2.3. In order to improve the generalization ability of the model, the 743 laying hen location pictures in step 2.2 were expanded using HSV data enhancement, Brightness data enhancement and Mixup data enhancement. A total of 2229 image data were obtained. data set;

Step 2.4. Use LabelImg for manual annotation to generate a txt file with cage column and egg characteristic coordinate information. The picture and the corresponding annotated txt file are integrated into a cage column and egg identification data set. Then, based on the manually labeled data set, the cage The column and egg areas are intercepted and saved as cage column and egg tracking data sets according to categories;

Step 2.5. Divide the cage column and egg recognition data set into a training set, a verification set and a test set according to 8:1:1; during the training phase of the model, the model calculates the loss function by comparing the predicted target frame with the actual candidate frame, thereby Carry out weight optimization; in the verification phase of the model, the performance of the model is evaluated in real time to avoid overfitting, and hyperparameters are adjusted to further optimize the performance of the model; finally, in the testing phase of the model, test data is used Set to verify the robustness of the trained model and its ability to perform in different situations;

Step 2.6. Divide the cage column and egg tracking data set into a training set and a test set according to 9:1; the model learns the patterns and characteristics of target tracking by analyzing the data in the training set, and gradually improves its performance through repeated training and optimization. To better capture the characteristics of the target object such as movement, appearance, and behavior; the model uses the test set to determine whether the model can successfully track the target object in the actual process.

Step 3. Construct a monitoring algorithm based on the improved DeepSort framework. The monitoring algorithm includes a target detection module, a trajectory matching module, a trajectory generation module and a positioning counting module. The target detection module includes a target detection module. The target detection module is used to Calibrate the number of the cage column in the input video data and identify the input video data to generate detection information in each frame of the image. The detection information includes a detection frame obtained by identification or a prediction frame obtained by prediction, and the detection frame includes an egg. The matching trajectory and the corresponding shape features and trajectory motion parameters are obtained by predicting the unmatched trajectory using Kalman filtering. The trajectory matching module is based on the detection frame of the current frame and the prediction frame obtained by prediction of the previous frame. Two IOU matches are performed, and cost matrix analysis is performed based on the matching results to output the corresponding linear matching relationship. The trajectory generation module is based on the linear matching relationship between the detection frame and all prediction frames, and combines the corresponding shape features of the detection frame , trajectory motion parameters and trajectories are cascade matched to generate the trajectory image of the egg. The positioning and counting module locates the location of each laying hen according to the number of the cage column, and combines the trajectory image of the egg to generate the egg laying position corresponding to the egg location. Measure the number.

In this embodiment, the structure of the monitoring algorithm is shown in Figure 3:

First, the laying hen location-specific target detection module of the improved YOLO algorithm replaces the ordinary two-dimensional convolution Conv of several CBS modules, C3_x modules and C3_1_F modules in the backbone layer and neck layer of the YOLO algorithm with lightweight convolutions by improving the YOLO algorithm. Ghostconv obtains the GhostCBS module, ChostC3_x and GhostC3_1f modules, which greatly reduces the computational cost of the network model. It then uses the backbone layer combined with the GAM global attention mechanism to enhance the feature perception of cage columns and eggs in space and channels, and cooperates with the WIOUv3 loss function to target detection; secondly, the second IOU matching is added to the DeepSort algorithm to reduce the problem of tracking target loss and enable more accurate multi-target tracking of targets.

The algorithm flow of the layer-specific target detection module that improves the YOLO algorithm is to input the layer-location image with 3 channels and a length and width of 640×640. After passing through the CBS module of the 6×6 convolution kernel with a step size of 2, 32×320 is obtained. ×320 feature map. After GhostCBS with a 3×3 convolution kernel with a stride of 2, a 64×160×160 feature map is obtained. After the GhostC3_1 module, a 64×160×160 feature map is obtained. After a 3×3 convolution with a stride of 2 The GhostCBS with accumulation kernel obtains a 128×80×80 feature map, and the 128×80×80 feature map is obtained through the GHOSTC3_2 module. The GhostCBS with a 3×3 convolution kernel with a step size of 2 obtains a 256×40×40 feature map, and the GHOSTC3_3 module is used to obtain a 256×40×40 feature map. The module obtains a 256×40×40 feature map. After GhostCBS with a 3×3 convolution kernel with a stride of 2, a 512×20×20 feature map is obtained. The GhostC3_1 module obtains a 512×20×20 feature map, and then the GAM global attention is obtained. The module and the SPPF spatial pyramid pooling module obtain a 512×20×20 feature map; the GhostCBS module with a 1×1 convolution kernel with a stride of 1 obtains a 256×20×20 feature map, and the Upsample upsampling module obtains a 256×40 ×40 feature map, the 256×40×40 feature map obtained through the Concat module and the backbone layer is connected to obtain a 512×40×40 feature map, the 256×40×40 feature map is obtained through the GhostC3_1f module, and the 1× with a step size of 1 is obtained. The GhostCBS module with 1 convolution kernel obtains a 128×40×40 feature map, the Upsample upsampling module obtains a 128×80×80 feature map, and the 128×80×80 feature map obtained by the Concat module and the backbone layer is connected to obtain 256×80 ×80 feature map, the 128×80×80 feature map is obtained through the GhostC3_1f module, the 128×40×40 feature map is obtained through the GhostCBS module with a 3×3 convolution kernel with a stride of 2, and the 128 is obtained through the Concat module and the backbone layer. The ×40×40 feature map is connected to obtain a 128×40×40 feature map. The 256×40×40 feature map is obtained through the GhostC3_1f module. The 256×20×20 feature map is obtained through the 3×3 GhostCBS module with a step size of 2. After The 256×40×40 feature map obtained by the Concat module and the backbone layer is connected to obtain a 512×20×20 feature map. The 512×20×20 feature map is obtained through the GhostC3_1f module. The 128×80×80 feature map obtained from the neck, The 256×40×40 feature map and 512×20×20 feature map are converted into 256×80×80 feature map, 256×40×40 feature map and 256×20×20 feature map through GhostConv with output channel 256 to head After the layer, use the preset a priori box to calculate the bounding box regression loss using the WIOUv3 loss function for each pixel in the feature map, and obtain a multi-dimensional array containing information such as object category, category confidence, bounding box coordinates, width and height, etc. The confidence threshold and IOU threshold are used to screen the redundant information in the array to filter out invalid content, and the non-maximum suppression algorithm is applied to eliminate redundant prediction boxes to obtain the detection information of cage columns and eggs.

The GhostConv convolution algorithm process in the GhostC3 module, ChostC3_x and GhostC3_1f modules is to perform a 1×1 convolution on the input feature 1 to obtain feature 2, and linearly transform the feature 2 through a 5×5 convolution to obtain feature 3. Finally, Feature 2 and input feature 1 are combined and output to obtain feature 4; the addition of the GhostConv module generates more feature maps with fewer parameters through grouped convolution and linear operations, greatly reducing the complexity of the model and reducing The number of network parameters improves the model operation speed.

The algorithm flow of the GAM global attention module is that the input channel is C, and the image feature 1 with a length and width of W×H is dimensionally converted into a W×H×C feature map to obtain feature 2. After two layers of multi-layer perceptrons, it is converted into The feature map of C×W×H is obtained as feature 3. The output feature map and the input feature map are multiplied bit by bit by Sigmoid processing to obtain feature 4. The input feature 4 is convolved with a convolution kernel of 7×7 to obtain C/r. Feature 5. The r parameter determines the compression ratio of the channel branch and the spatial branch. After convolution with a convolution kernel of 7×7, the feature 6 of C×W×H is obtained. Finally, the Sigmoid output feature map is compared with feature 4 bit by bit. The multiplication results in feature 7; through these steps, the GAM global attention module can better capture the dependence between channels and spatial dimensions in the cage column and egg feature maps, thereby significantly improving the performance of the model.

Step 4. Use the data set to train the monitoring algorithm to obtain a monitoring model for monitoring egg production in laying hen locations.

The expression of the loss function used in this embodiment is as follows:

L _WIoUv1 ＝R _WIoU L _IoU (3)

In the formula, x _g and y _g represent the width and height of the predicted box, x _gt and y _gt represent the width and height of the real box, W _g and H _g represent the width and length of the minimum rectangle formed by the combination of the predicted box and the real box, above The mark * represents not participating in the calculation, effectively eliminating factors that hinder convergence. R _WIoU represents the penalty term of WIoU that will enhance the loss of ordinary quality anchor boxes. β represents outliers, which is used to assign a gradient weight to high-quality anchor boxes. Gain, r′ is the gradient gain, α and δ are hyperparameters, and L _IoU represents the IOU loss.

The training hyperparameters of the target detection module: the training device is RTX3080; the input image size is 640×640; the batch size is 8; the Ad _a m optimizer is used for model gradient optimization; the number of early stopping rounds is set to 150 rounds; initial learning The rate is 0.01; the momentum factor is 0.937; the optimizer weight decay is 0.001; the number of training rounds is 300 rounds.

The training hyperparameters of the monitoring model: the minimum confidence threshold is 0.2; the maximum IOU matching threshold is 0.7; the number of frames maintained in the trajectory during the initialization phase is 3; the maximum number of frames that can be continuously lost is 50.

In addition, Table 1 shows the performance comparison results between the monitoring model of this embodiment and other models.

Method P/% R/% mAP@0.5/% F1_score Params FPS SSD 95.9 86.3 96.3 90.08 13.64 77.5 Faster R-CNN 96.2 86.8 96.5 91.26 136.2 56.1 YOLOv5s 97.8 97.2 98.3 97.50 7.01 91.5 YOLOv8 98.8 97.6 98.6 98.20 11.7 79.3 DETR 97.6 97.3 97.6 97.45 27.98 64.1 Ours 99.1 98 99.2 98.55 5.96 113.4

Step 5: Obtain the video data of the laying hen location to be counted, and input it into the monitoring model to obtain the positioning number and egg production volume of the corresponding laying hen location.

In this embodiment, the positioning and counting process as shown in Figure 4 is as follows:

Step 5.1. Obtain video frames from the chicken farm site. The monitoring model tracks the cage columns and eggs on the upper and lower floors and makes conditional judgments for LROI generation. If more than two cage column features are tracked on the upper and lower floors, proceed to step 5.2. , otherwise go directly to step 5.3;

Step 5.2: Obtain the upper left corner of the two cage column tracking numbers (p _n1 , p _n1+1 ) of the upper layer hen area of the current frame and the two cage column tracking numbers (p _n2 , p _n2+1 ) of the lower layer hen area respectively. The mean value of the horizontal and vertical coordinates and the upper right horizontal and vertical coordinates And draw the area of interest LROI1 for the upper layer hens and the area of interest LROI2 for the lower layer hens. The drawing formula is:

Among them, LROI1 (X _{min , X max} ), LROI1 (Y _{min ,} Y _max ), LROI2 (X _min , max is the maximum and minimum value function, and sum is the summation function.

In addition, the calculation formulas of h ₁ and h ₂ are:

Among them, H is the actual height of the laying hens, f is the focal length, d is the actual distance between the lens and the laying hens, Q ₁ and Q ₂ are the angles of the lens from the lower laying hens, and E is the actual size of one pixel.

Step 5.3, then judge the cage column characteristics of the generated LROI1 and LROI2. If no new cage column number is detected, that is, the tracking number p _t,n of the last cage column of the current frame t exists in the cage of the previous frame. Column tracking number list P _t-1 , the number will not be updated, otherwise the generated LROI1 and LROI2 will be incrementally numbered. For the nth LROI1 and LROI2 number l _t,n of the upper and lower layers of the current frame t, the calculation formula is:

Among them, p _t,n is the tracking number of the n-th cage column in the t-th frame, and P _t ={p _t,1 ,p _t,2 ,…,p _t,n } is the tracking number assigned to the cage column in the t-th frame. The list, 1_L _t ={1_l _t, 1,1_l _t,2 ,...,1_l _t,n } is the cage position number list of the tth frame of the first layer, 2_L _t ={2_l _t, 1,2_l _t,2 , …,2_l _t,n } is the list of cage numbers in the tth frame of the second layer. Calculate the number of eggs in LROI1 and LROI2 respectively. The formula for the number of eggs e _{t,n in LROI1 and LROI2 in the tth frame number n} is: :

in, Represents the LROI area numbered n in the t-th frame, x is the total number of eggs detected in the current frame image, x _t,i , y _t,i are the center point coordinates of the i-th egg in the t-th frame,/> and/> is an indicator function, indicating that x _t,i and y _t,i are within the LROIn area.

Step 5.4. In order to better estimate the number of eggs in each laying hen location so that frames with large changes have less impact on the predicted value, after the video ends, count the eggs of all frames in each LROI1 and LROI2 area. A weighted average calculation is performed to obtain the predicted egg production value of each numbered laying hen location.

As shown in Figure 5, the laminated caged laying hens location egg production monitoring system provided in this embodiment is implemented by the cascaded caged laying hens location egg production monitoring method provided in the above example, which includes a chicken farm management unit, Egg production monitoring unit and system unit; the chicken farm management unit includes chicken farm environment monitoring, chicken farm feeding management and chicken farm personnel management; the egg production monitoring unit includes chicken cage monitoring, egg production monitoring and abnormal alarm; The system management unit includes a system setting part for deploying a chicken farm management unit and an egg production monitoring unit, a system user management part for managing accounts, and an overall data management part for storing daily monitoring results.

In this embodiment, the cascading caged laying hen location egg production monitoring system adopts the SpringBoot architecture, the front end is implemented based on the Vue framework, and the database uses Mysql. Jpa is a specification used to persist entity objects into a relational database; The system architecture consists of the View layer, the Controller layer, the Service layer, and the Dao layer. The View layer is used to visualize the data of egg production in each laying hen location and interact with the chicken farm managers. The Controller layer is responsible for receiving the data from the chicken farm managers. The Service layer handles requests from chicken farm managers. The Dao layer is the data access layer. It is responsible for interacting with the database. At the same time, it writes the monitoring model into the edge computing device of the inspection equipment, and monitors the scene through the X001 monocular industrial camera. The video of the cascading cage laying hens is acquired and transmitted to the Raspberry Pi through the CSI interface for real-time egg production statistics to obtain the egg production results of each laying hen location, and the results are sent to the MySQL database for chicken farm managers. Use the intelligent laying hen location egg production monitoring system to monitor and analyze the egg production results of each laying hen location in the database in real time, formulate a more scientific and effective chicken farm management strategy, remove abnormal laying hens, and improve the egg production of laying hens. quantity and egg quality.

In order to verify the robustness of the present invention in real-life scenarios, on-site experiments were carried out in commercial farms, and tests were conducted on caged layer hens breeding environments at different times and with different illumination. Table 2 shows the monitoring model of the present invention during the breeding process. Overall real-time monitoring results of egg production in stacked caged laying hens.

As shown in Figure 6 and Figure 7, it is the data histogram of egg production in the location of test number 1 and the data histogram of the wrong chicken cage. The horizontal axis of the histogram represents different chicken cage numbers, and the vertical axis represents the corresponding number of eggs. There were 505 chicken cages tested, and 505 chicken cages were actually measured. 0 chicken cages were missed, and 6 cages were incorrectly detected for egg production.

As shown in Figure 8 and Figure 9, the data histogram of egg production in the second sampling video and the data histogram of wrongly detected chicken cages are shown. The horizontal axis of the histogram represents different chicken cage numbers, and the vertical axis represents the corresponding eggs. Quantity: 544 chicken cages were tested, 544 cages were actually measured, 0 chicken cages were missed, and 4 cages were incorrectly detected for egg production;

As shown in Figure 10 and Figure 11, it is the data histogram of egg production in the third sampling video and the data histogram of wrongly detected chicken cages. The horizontal axis of the histogram represents different chicken cage numbers, and the vertical axis represents the corresponding eggs. In terms of quantity, 524 chicken cages were tested, 525 chicken cages were actually measured, 1 chicken cage was missed, and 7 chicken cages were incorrectly detected for egg production.

As shown in Figure 12 and Figure 13, it is the data histogram of egg production in the fourth sampling video and the data histogram of wrongly detected chicken cages. The horizontal axis of the histogram represents different chicken cage numbers, and the vertical axis represents the corresponding eggs. Quantity: 577 chicken cages were tested, 578 chicken cages were actually measured, 1 chicken cage was missed, and 4 cages were incorrectly detected for egg production.

The monitoring results of the present invention under the specified typical "124" production mode show very high accuracy. The accuracy of counting the location of laying hens reaches 99%, and the accuracy of location positioning of laying hens reaches 99.9%. This shows that the accuracy proposed by the present invention The method can also maintain a high level of monitoring quality in practical applications.

Claims (10)

1. The method for monitoring the laying of the layered cage-raising laying hen in the zone based on computer vision is characterized by comprising the following steps:

step 1, obtaining video data of two layers of laying hen positions through inspection equipment, wherein the inspection equipment comprises two monocular cameras with the same overlooking angle and different heights;

step 2, capturing the acquired video data frame by frame to construct a video image set, labeling the video image set by using cage columns and eggs in the image, and forming a data set by the video image set and the labels;

step 3, a monitoring algorithm based on an improved deep sort framework is constructed, the monitoring algorithm comprises a target detection module, a track matching module, a track generation module and a positioning counting module, the target detection module comprises a target detection module, the target detection module is used for calibrating the serial numbers of cage columns in input video data and identifying the input video data to generate detection information in each frame of image, the detection information comprises an obtained detection frame or a predicted prediction frame, the detection frame comprises tracks matched with eggs and corresponding outline characteristics and track motion parameters, the prediction frame predicts the unmatched tracks by adopting Kalman filtering, the track matching module performs two-round IOU matching according to the detection frame of the current frame and the predicted frame obtained by the previous frame and performs cost matrix analysis based on the matching result, the track generation module performs cascading matching according to the linear matching relation between the detection frame and all the predicted frames and combining the outline characteristics, the track motion parameters and the tracks corresponding to the detection frame so as to generate track images of eggs, the positioning counting module performs positioning position quantity combination on the egg production positions of each egg column according to the serial numbers of the cage columns;

training a monitoring algorithm by adopting a data set to obtain a monitoring model for monitoring the egg yield of the laying hen zone bit;

and step 5, acquiring video data of the laying hen positions to be counted, and inputting the video data into the monitoring model to obtain the positioning numbers and the egg yield of the corresponding laying hen positions.

2. The method for monitoring the regional egg laying of the stacked cage-raising layers based on computer vision according to claim 1, wherein calibration de-distortion treatment is required for the two monocular cameras before the inspection equipment is put in.

3. The method for monitoring the regional egg laying of the stacked cage laying hens based on computer vision according to claim 2, wherein the expression of the calibration de-distortion treatment is as follows:

wherein Z is _c As a scale factor of the dimensions of the device,the distance from the optical center to the image plane of the layer location is shown as u and v, and is the abscissa and ordinate of the layer location in the pixel coordinate system, f _x And f _y For the normalized focal length of the laying hen on the x axis and the y axis, R _3×3 The rotation matrix obtained for camera calibration, X _W 、Y _W And Z _W Is the world coordinate system>Correcting coordinates, k for zone bit distortion of laying hens ₁ 、k ₂ For radial distortion, p ₁ 、p ₂ Is tangential distortion, r is radius of curvature, +.>The coordinates of the laying hen after the zone position is distorted.

4. The method for monitoring the regional egg laying of the stacked cage-raising layers based on computer vision according to claim 1, wherein the construction process of the video image set is as follows:

intercepting the acquired video data by adopting a video capture function for a preset frame number to acquire an initial image set;

and performing data cleaning and data expansion on the obtained initial image set to obtain a video image set containing the laying hen zone bit.

5. The method for monitoring the regional egg laying of the stacked cage-raising layers based on computer vision according to claim 1, wherein the target detection module of the monitoring algorithm introduces a global attention module, and the global attention module is used for performing attention mechanism operation under multiple dimensions on each frame of video image of input video data and multiplying the features under different dimensions bit by bit so as to obtain feature-enhanced image data output.

6. The method for monitoring the regional egg laying of the stacked cage-raising layers based on computer vision according to claim 1, wherein the track matching module adopts a hungarian algorithm to solve a cost matrix to obtain a corresponding linear matching relationship, the linear matching relationship comprises track mismatch, unmatched detection frames and successful matching of the detection frames and the prediction frames;

and if the matching result is that the track is mismatched or the detection frame is not matched, repeating IOU matching until the ending condition is reached.

7. The method for monitoring the laying hen zone bit egg production based on computer vision according to claim 1, wherein the positioning counting module generates an upper layer hen zone bit interest area LROI1 and a lower layer hen zone bit interest area LROI2 through upper layer cage column coordinates, numbers the upper layer hen zone bit interest area LROI1 and the lower layer hen zone bit interest area LROI2 in real time, counts eggs in the LROI1 and LROI2 areas, and after the video is finished, performs weighted average calculation on egg count values of all frames in each LROI1 and LROI2 area to obtain a final hen zone bit egg production observation result.

8. The method for monitoring the regional egg production of stacked caged layer based on computer vision according to claim 1, wherein the target detector of the monitoring algorithm is trained using a WIOU loss function to update parameters of the monitoring algorithm.

9. The computer vision-based stacked caged layer location egg production monitoring method of claim 8, wherein the expression of the WIOU loss function is as follows:

L _WIoUv1 ＝R _WIoU L _IoU (3)

wherein x is _g And y _g Representing prediction frame width and height, x _gt And y _gt Representing the width and height of a real frame, W _g And H _g Representing the width and length of the smallest rectangle formed by the combination of the predicted frame and the real frame, and superscript represents that the calculation is not participated, effectively eliminating the factor of preventing convergence, R _WIoU Penalty terms representing WIoU will enhance the loss of normal quality anchor boxes, β represents outliers, for assigning a gradient weight gain, r, to high quality anchor boxes ^′ For gradient gain, alpha and delta are superparameters, L _IoU Representing the IOU penalty.

10. A layered cage-raising layer-location egg-laying monitoring system, which is characterized by comprising a chicken farm management unit, an egg-laying monitoring unit and a system unit, wherein the layered cage-raising layer-location egg-laying monitoring method is realized by the layered cage-raising layer-location egg-laying monitoring method as claimed in any one of claims 1 to 9;

the chicken farm management unit comprises chicken farm environment monitoring, chicken farm raising management and chicken farm personnel management;

the egg laying monitoring unit comprises chicken coop monitoring, egg yield monitoring and abnormal alarm;

the system management unit comprises a system setting part for allocating the chicken farm management unit and the egg production monitoring unit, a system user management part for managing accounts and a total data management part for storing daily monitoring results.