Research on an Identification and Grasping Device for Dead Yellow-Feather Broilers in Flat Houses Based on Deep Learning
<p>The interior scene of the experimental broiler house.</p> "> Figure 2
<p>The structure of the dead broiler identification and grabbing device.</p> "> Figure 3
<p>The image acquisition and processing system.</p> "> Figure 4
<p>Binocular camera imaging model.</p> "> Figure 5
<p>The dead broiler grasping manipulator: (<b>a</b>) mechanical arm backbone; (<b>b</b>) end effector.</p> "> Figure 6
<p>Diagram of coordinate system transformation.</p> "> Figure 7
<p>The checkerboard calibration board. (<b>a</b>) Original image; (<b>b</b>) result.</p> "> Figure 8
<p>Hand–eye calibration operation.</p> "> Figure 9
<p>The transition coordinate system.</p> "> Figure 10
<p>Different behavior images of yellow-feather broilers: (<b>a</b>) walking; (<b>b</b>) pecking; (<b>c</b>) resting; (<b>d</b>) inactive; (<b>e</b>) dead.</p> "> Figure 11
<p>The SE module’s structure diagram.</p> "> Figure 12
<p>Schematic diagram of the improved YOLOv6 network structure.</p> "> Figure 13
<p>Simulation model of manipulator.</p> "> Figure 14
<p>The simulation result graph: (<b>a</b>) acceleration simulation results; (<b>b</b>) velocity simulation result; (<b>c</b>) position simulation result graph. P.s.: The blue line represents the movement of joint 1, the red line represents the movement of joint 2, the purple line represents the movement of joint 3, and the orange line represents the movement of joint 4.</p> "> Figure 15
<p>Speed comparison of YOLOv6, SSD and Faster-RCNN algorithms detection models.</p> "> Figure 16
<p>Recognition result of the YOLOv6 model.</p> "> Figure 17
<p>Comparison of the results of ablation experiments with different labeling categories.</p> "> Figure 17 Cont.
<p>Comparison of the results of ablation experiments with different labeling categories.</p> "> Figure 18
<p>Speed comparison of YOLOv6, YOLOv6 + SE and YOLOv6 + CBAM algorithms detection models.</p> "> Figure 19
<p>Speed comparison of YOLOv6, YOLOv6 + SE and Improved YOLOv6 + SE algorithms detection models.</p> "> Figure 20
<p>Recognition result of the improved YOLOv6 + SE.</p> "> Figure 21
<p>Training loss curves of the improved algorithms: (<b>a</b>) SSD; (<b>b</b>) Faster-RCNN; (<b>c</b>) improved YOLOv6 + SE.</p> "> Figure 22
<p>The real-time detection results.</p> "> Figure 23
<p>(<b>a</b>) Identifying the dead broiler. (<b>b</b>) The device is moving. (<b>c</b>) The device is grabbing the dead broiler. (<b>d</b>) The device is receiving the dead broiler. (<b>e</b>) The device is transporting the dead broiler.</p> "> Figure 24
<p>The different parts grasped: (<b>a</b>) back grab; (<b>b</b>) back and chest grab; (<b>c</b>) chest grab.</p> ">
Abstract
:1. Introduction
2. Materials and Methods
2.1. Experimental Base
2.2. Moving Chassis
2.3. Image Acquisition and Processing System
2.4. Mechanical Arm
2.5. Camera Calibration and Hand–Eye Calibration
2.6. Data Collection and Image Data Set Production
2.7. Recognition Model of Dead Yellow-Feather Broiler
2.7.1. Model Training Parameters
2.7.2. Model Evaluation Metrics
2.8. The Design of the Dead Broiler Grasping Manipulator
3. Results and Analysis
3.1. The Different Model’s Performance
3.2. The Real-Time Detection Effect of the Model
3.3. The Design of the Dead Broiler Grasping Experiment
4. Conclusions
- (1)
- The experimental results demonstrated that the mobile device developed for identifying and grasping dead broilers proved capable of fulfilling the task of removing dead broilers. It achieved a success rate of over 77%, with an average success rate of 81.3%. However, the success rate was lowest when grasping parts of the broilers such as the neck, feet, or other areas with small contact surfaces. Even though the presence of the mobile device itself exerted a certain influence on the success rate, this aspect will be the focus of future improvements. Additionally, the success rate of grasping deceased broilers decreased in densely populated areas, which can be attributed to the device’s limited ability to collect information on dead broilers. Nevertheless, the mobile device could disperse the broilers during movement, capture the necessary information, and complete the grasping task.
- (2)
- This study proposes an enhanced deep learning-based approach for identifying broilers. The YOLOv6 algorithm, with its superior comprehensive performance, was selected as the basic network and underwent in-depth optimization. Specifically, a YOLOv6 network structure based on the SE attention mechanism and ASPP was proposed to address the existing issues found in broiler houses. The experimental outcomes indicated that the recognition accuracy of the improved algorithm model reached 86.1%.
- (3)
- This study designed a mechanical arm for positioning and grasping dead broilers. A model joint simulation of the manipulator was conducted, and the motion trajectory was planned. The experimental results verified that the manipulator model passed the test, the transmission was stable, and the trajectory met the requirements, thereby providing the essential conditions for achieving stable grasping and attaining the design objective.
- (1)
- Limitations of applicability—The current study was tested and optimized mainly with respect to yellow-feathered broilers. Given the wide variety of broiler breeds available in the market, further verification of the applicability of the device for other breeds is required. If the recognition effect is found to be poor, a breed-specific image database needs to be established as a benchmark for recognition.
- (2)
- Efficiency and energy-saving considerations—When there are multiple dead broilers in the coop at the same time, although the device is able to effectively detect and remove them, further research is needed into how to optimize path planning for a more efficient operation from the point of view of improving efficiency and energy use.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Zhou, S.; Watcharaanantapong, P.; Yang, X.; Thornton, T.; Gan, H.; Tabler, T.; Zhao, Y. Evaluating broiler welfare and behavior as affected by growth rate and stocking density. Poult. Sci. 2024, 103, 103459. [Google Scholar] [CrossRef]
- Li, G.; Zhao, Y.; Porter, Z.; Purswell, J. Automated measurement of broilers under four stocking densities via faster region-based convolutional neural network. Animal 2021, 15, 100059. [Google Scholar] [CrossRef]
- Chen, G.; Ling, X.; Xie, M.; Xiong, Y.; Li, T.; Shui, C.; Li, C.; Xu, B.; Ma, F. Systematic evaluation of the meat qualities of free-range chicken (Xuan-Zhou) under different ages explored the optimal slaughter age. Poult. Sci. 2024, 103, 104019. [Google Scholar] [CrossRef]
- Park, J.; Kwon, O.; Lee, K.; Heo, Y.; Yoon, C. Ammonia and Hydrogen Sulfide Monitoring in Broiler Barns and Cattle Barns. J. Environ. Health Sci. 2015, 41, 277–288. [Google Scholar] [CrossRef]
- Charman, W. Visual standards for driving. Ophthal. Physiol. Opt. 2015, 5, 211–220. [Google Scholar] [CrossRef]
- Li, X.; Yan, F.; Hu, K.; He, X.; Ma, Z.; Yuan, Q.; Mirzoev, S.I. Design and test of the integrated machine for self-propelled manure collection and bagging in flat chicken coops. Trans. Chin. Soc. Agric. Eng. 2024, 40, 251–261. [Google Scholar]
- Shi, Z.; Xi, L.; Ji, Z.; Cheng, P. LED illuminant improving broilers house environment and growth performance. Trans. Chin. Soc. Agric. Eng. 2017, 33, 222–227. [Google Scholar]
- Hinojosa, C.; Caldwell, D.; Byrd, J.; Ross, M.; Stringfellow, K.; Fowlkes, E.; Lee, J.; Stayer, P.; Farnell, Y.; Farnell, M. Use of a foaming disinfectant and cleaner to reduce aerobic bacteria on poultry transport coops. J. Appl. Poult. Res. 2015, 24, 364–370. [Google Scholar] [CrossRef]
- Okada, H.; Itoh, T.; Suzuki, K.; Tsukamoto, K. Wireless sensor system or detection of avian influenza outbreak farms at an early stage. In Proceedings of the 8th IEEE Conference on Sensors, Christchurch, New Zealand, 25–28 October 2009; pp. 1374–1377. [Google Scholar]
- Aydin, A.; Bahr, C.; Viazzi, S.; Exadaktylos, V.; Buyse, J.; Berckmans, D. A novel method to automatically measure the feed intake of broiler chickens by sound technology. Comput. Electron. Agric. 2008, 101, 17–23. [Google Scholar] [CrossRef]
- Cao, Y.; Teng, G.; Yu, L.; Li, Q. Comparison of different de-noising methods in vocalization environment of laying hens including fan noise. Trans. Chin. Soc. Agric. Eng. 2014, 30, 212–218. [Google Scholar]
- Jacob, F.; Baracho, M.; Nääs, I.; Souza, R.; Salgado, D. The use of infrared thermography in the identification of pododermatitis in broilers. Eng. Agric. 2016, 36, 253–259. [Google Scholar] [CrossRef]
- Wang, J.; Wang, N.; Li, L.; Ren, Z. Real-time behavior detection and judgement of egg breeders based on YOLO v3. Neural Comput. Appl. 2020, 32, 5471–5481. [Google Scholar] [CrossRef]
- Zhang, Q. Application and Innovation of New Media Technology in Visual Communication Design. Agro Food Ind. Hi Tech. 2017, 28, 3170–3173. [Google Scholar]
- Shafay, M.; Ahmad, R.; Salah, K.; Yaqoob, I.; Jayaraman, R.; Omar, M. Blockchain for deep learning: Review and open challenges. Clust. Comput. 2023, 26, 197–221. [Google Scholar] [CrossRef]
- Lu, C. Study on Dead Birds Detection System Based on Machine Vision in Modern Chicken Farm. Master’s Thesis, JiangSu University, Zhenjiang, China, 2009. [Google Scholar]
- Zheng, S.; Wang, L. Development of Monitoring System for Layers Rearing in Multi-Tier Vertical Cages Using Machine Vision. J. Jilin Agric. Univ. 2009, 31, 476–480. [Google Scholar]
- Qu, Z. Study on Detection Method of Dead Chicken in Unmanned Chicken Farm. Master’s Thesis, Jilin University, Changchun, China, 2009. [Google Scholar]
- Veera, V.; Yang, Z. Automatic Identification of Broiler Mortality Using Image Processing Technology. ASABE Meet. Present. 2018, 18, 3–5. [Google Scholar]
- Sequeira, J.; Ribeiro, M. Human-robot interaction and robot control. Robot. Motion Control Recent. 2006, 35, 375–390. [Google Scholar]
- Liu, H.; Chen, C.; Tsai, Y.; Hsieh, K.; Lin, H. Identifying images of dead chickens with a chicken removal system integrated with adeep learning algorithm. Sensor 2021, 21, 3579. [Google Scholar] [CrossRef]
- Hu, Z. Research on Underactuated End Effector of Dead Chicken Picking Robot. Master’s Thesis, HeBei Agricultural University, Baoding, China, 2021. [Google Scholar]
- Zhao, W.; Chen, Y.; Cao, T. Design and kinematics analysis of robotic arm used for picking up dead chickens. J. Chin. Agric. Mech. 2023, 44, 131–136. [Google Scholar]
- Li, Y.; Liu, Q.; Li, T.; Wu, Y.; Niu, Z.; Hou, J. Design and experiments of garlic bulbil orientation adjustment device using Jetson Nano processor. Trans. Chin. Soc. Agric. Eng. 2021, 37, 35–42. [Google Scholar]
- Wang, S.; Hu, Y. Binocular visual positioning under inhomogeneous, transforming and fluctuating media. Trait. Du Signal 2018, 35, 253–276. [Google Scholar] [CrossRef]
- Li, X.; Zhang, E.; Fang, X.; Zhai, B. Calibration Method for Industrial Robots Based on the Principle of Perigon Error Close. IEEE Access 2022, 10, 48569–48576. [Google Scholar] [CrossRef]
- Yue, W.; Hua, S.; Ying, D. A Complete Analytical Solution to Hand-Eye Calibration Using Quaternions and Eigenvector-Eigenvalue Identity. J. Intell. Robot. Syst. 2023, 109, 54. [Google Scholar]
- Shang, Y.; Shen, J.; Wei, W.; Zheng, B. Optimization of Ball Mill Cylinder Structure Based on Response Surface Optimization Module and Multi-objective Genetic Algorithm. J. Mech. Sci. Technol. 2024, 38, 3631–3640. [Google Scholar] [CrossRef]
- Feng, X.; Tian, D.; Wu, H. A matrix-solving hand-eye calibration method considering robot kinematic errors. J. Manuf. Process. 2023, 99, 618–635. [Google Scholar] [CrossRef]
- Cui, G.; Qiao, L.; Li, Y.; Chen, Z.; Liang, Z.; Xin, C.; Xiao, M.; Zou, X. Division of Cow Production Groups Based on SOLOv2 and Improved CNN-LSTM. Agriculture 2023, 13, 1562. [Google Scholar] [CrossRef]
- Lao, F.; Teng, G.; Li, J.; Yu, L.; Li, Z. Behavior recognition method for individual laying hen based on computer vision. Trans Chin. Soc. Agric. Eng. 2018, 28, 157–163. [Google Scholar]
- Guo, J.; He, G.; Deng, H.; Fan, W.; Xu, L.; Cao, L.; Feng, D.; Li, J.; Wu, H.; Lv, J.; et al. Pigeon cleaning behavior detection algorithm based on lightweight network. Comput. Electron. Agric. 2022, 199, 107032. [Google Scholar] [CrossRef]
- Bist, B.; Subedi, S.; Yang, X.; Chai, L. A Novel YOLOv6 Object Detector for Monitoring Piling Behavior of Cage-Free Laying Hens. AgriEngineering 2023, 5, 905–923. [Google Scholar] [CrossRef]
- Xie, W.; Kimura, M.; Takaki, K.; Asada, Y.; Iida, T.; Jia, X. Interpretable Framework of Physics-Guided Neural Network with Attention Mechanism: Simulating Paddy Field Water Temperature Variations. Water Resour. Res. 2022, 58, e2021WR030493. [Google Scholar] [CrossRef]
- Yun, Y.; Choi, J.; Chung, H.; Bae, K.; Moon, J. Performance evaluation of an occupant metabolic rate estimation algorithm using activity classification and object detection models. Build. Environ. 2024, 252, 252111299. [Google Scholar] [CrossRef]
- Sun, F.; Wang, Y.; Lan, P.; Zhang, X.; Chen, X.; Wang, Z. Identification of apple fruit diseases using improved YOLOv5s and transfer learning. Trans Chin. Soc. Agric. Eng. 2022, 38, 171–179. [Google Scholar]
- Corke, P. MATLAB toolboxes: Robotics and vision for students and teachers. IEEE Robot. Autom. Mag. 2007, 14, 16–17. [Google Scholar] [CrossRef]
- Vila-Rosado, D.; Domínguez-López, J. A MATLAB toolbox for robotic manipulators. In Proceedings of the Sixth Mexican International Conference on Computer Science, Puebla, Mexico, 26–30 September 2005; pp. 256–263. [Google Scholar]
- Liu, Y.; He, Y.; Wu, X.; Wang, W.; Zhang, L.; Lv, H. Potato Sprouting and Surface Damage Detection Method Based on Improved Faster R-CNN. Trans Chin. Soc. Agric. Mach. 2024, 55, 371–378. [Google Scholar]
- Biswas, D.; Su, H.; Wang, C.; Stevanovic, A.; Wang, W. An automatic traffic density estimation using Single Shot Detection (SSD) and Mobile Net-SSD. Phys. Chem. Earth 2019, 110, 176–184. [Google Scholar] [CrossRef]
Behavior Classification | Classification Definition |
---|---|
Dead | The yellow-feather broiler’s eyes close, the body clings to the ground, or the body is weak and stiff. |
Others | This includes motions such as walking, pecking, inactivity, and resting. |
Label Name and Number | Label Definition |
---|---|
Walking (20,864) | Actions such as standing, walking, and arranging feathers |
Pecking (15,648) | Yellow-feathered broilers with their heads touching the ground, troughs or water troughs, or tails cocked up |
Resting (10,432) | Yellow-feathered broilers lying on the ground |
Inactive (5216) | Yellow-feathered broilers lying on their backs with their bodies curled up or their tails drooping |
Dead (32,166) | Yellow-feathered broilers lying flat on the ground with their bodies in a rigid state |
Models | Exact Deep Learning Method | Pros | Cons |
---|---|---|---|
SSD | Single deep network for both object classification and localization using default boxes | Fast, simple, and effective for a wide range of object sizes | Less accurate on very small objects |
Faster-RCNN | Two-stage detector with a region proposal network (RPN) for generating candidate regions | High accuracy, flexible backbone networks, faster-than-previous R-CNN version | Slower than one-stage detectors, more complex to train |
YOLOv6 | Single-pass detector with efficient backbones and improved training techniques | Very fast, simple architecture, competitive accuracy | Limited information available, may struggle with small objects |
Models | Precision | Recall | F1 Score | maP |
---|---|---|---|---|
YOLOv6 | 0.80 | 0.81 | 0.80 | 0.86 |
SSD | 0.78 | 0.78 | 0.78 | 0.80 |
Faster-RCNN | 0.81 | 0.81 | 0.81 | 0.87 |
Models | Standard Errors of Precision | Standard Errors of Recall | Standard Errors of F1 Score |
---|---|---|---|
YOLOv6 | 0.62% | 0.51% | 0.68% |
Faster-RCNN | 0.81% | 0.77% | 0.73% |
Models | Precision | Recall | F1 Score | maP |
---|---|---|---|---|
YOLOv6 + SE | 0.84 | 0.88 | 0.83 | 0.90 |
YOLOv6 + CBAM | 0.82 | 0.86 | 0.84 | 0.90 |
Models | Precision | Recall | F1 Score | maP |
---|---|---|---|---|
YOLOv6 + SE | 0.84 | 0.88 | 0.83 | 0.90 |
Improved YOLOv6 + SE | 0.86 | 0.89 | 0.87 | 0.92 |
Experimental Serial Number | Grab Times | Success Rate |
---|---|---|
1 | 30 | 0.87 |
2 | 30 | 0.80 |
3 | 30 | 0.77 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xin, C.; Li, H.; Li, Y.; Wang, M.; Lin, W.; Wang, S.; Zhang, W.; Xiao, M.; Zou, X. Research on an Identification and Grasping Device for Dead Yellow-Feather Broilers in Flat Houses Based on Deep Learning. Agriculture 2024, 14, 1614. https://doi.org/10.3390/agriculture14091614
Xin C, Li H, Li Y, Wang M, Lin W, Wang S, Zhang W, Xiao M, Zou X. Research on an Identification and Grasping Device for Dead Yellow-Feather Broilers in Flat Houses Based on Deep Learning. Agriculture. 2024; 14(9):1614. https://doi.org/10.3390/agriculture14091614
Chicago/Turabian StyleXin, Chengrui, Hengtai Li, Yuhua Li, Meihui Wang, Weihan Lin, Shuchen Wang, Wentian Zhang, Maohua Xiao, and Xiuguo Zou. 2024. "Research on an Identification and Grasping Device for Dead Yellow-Feather Broilers in Flat Houses Based on Deep Learning" Agriculture 14, no. 9: 1614. https://doi.org/10.3390/agriculture14091614