In Field Detection of Downy Mildew Symptoms with Proximal Colour Imaging
<p>Instrumentation: an autonomous in-field imaging sensor, (<b>a</b>) a device embedded on a vine tractor and (<b>b</b>) details and elements of the device.</p> "> Figure 2
<p>Typical image resulting from the device. The image presents various symptoms of downy mildew circled in red. Two examples of typical “oil spots” are detailed.</p> "> Figure 3
<p>Image processing pipeline for the reconstruction of downy mildew foliar symptoms.</p> "> Figure 4
<p>Graphical resume of the processing pipeline.</p> "> Figure 5
<p>Discriminative properties of LST: separability of visible foliar symptoms of downy mildew on the basis of the <math display="inline"><semantics> <msub> <mi>λ</mi> <mn>1</mn> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>λ</mi> <mn>2</mn> </msub> </semantics></math> eigenvalues of the tensor. Eigenvalues are displayed on a normalised 8-bit colour scale. Three “textbook” cases are presented, (<b>a</b>) a large circular “oilspot”, (<b>b</b>) a “crown of oilspots” and (<b>c</b>) irregular symptoms on the edges of the limbus.</p> "> Figure 6
<p>Adequacy of log-determinant distributions (blue) with Gaussian probability functions of same mean and variance (in red) and with Gaussian mixture models or order K = 3 (in teal). Distributions are shown for healthy leaves (<b>a</b>), healthy berries (<b>b</b>), foliar downy mildew (<b>c</b>) and symptoms on berries (<b>d</b>).</p> "> Figure 7
<p>Adequacy of the distribution of log-determinants of <span class="html-italic">Z</span> (<span class="html-italic">TC-LEST</span> colour component) (red) and comparison with Gaussian mixture (teal) for foliar downy mildew symptoms (<b>a</b>). Details of the K = 3 Gaussian mixture (<b>b</b>).</p> "> Figure 8
<p>Labelling healthy, symptomatic or abnormal grapevine tissues (<b>b</b>) from original (<b>a</b>). (<b>b</b>) A representative amount of pixel sampled to cover the whole image. Samples are annotated with a colour code corresponding to their class according to photo-interpretation.</p> "> Figure 9
<p>Joint evolution of recall–precision (for mildew pixels) for different combinations of thresholds for the determination of seeds. Each cross represents a value of <math display="inline"><semantics> <mrow> <mi>α</mi> <mo>∈</mo> <mo>[</mo> <mn>0.01</mn> <mo>,</mo> <mn>0.30</mn> <mo>]</mo> </mrow> </semantics></math> with low values starting top-left to bottom-right.</p> "> Figure 10
<p>Performances of the reconstruction of downy mildew foliar symptoms for 100 images, depending on the pairs of relaxed thresholds (<math display="inline"><semantics> <mrow> <msub> <mi>R</mi> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>,</mo> <mi>α</mi> </mrow> </semantics></math>) from seeds determined with <math display="inline"><semantics> <mrow> <mo>[</mo> <msub> <mi>R</mi> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>=</mo> <mn>2.5</mn> <mo>,</mo> <mi>α</mi> <mo>=</mo> <mn>0.1</mn> <mo>]</mo> <mo>.</mo> </mrow> </semantics></math></p> "> Figure 11
<p>Reconstruction results with the thresholds: (seeds) [<math display="inline"><semantics> <mrow> <msub> <mi>R</mi> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>=</mo> <mn>2.5</mn> </mrow> </semantics></math>,<math display="inline"><semantics> <mrow> <mi>α</mi> <mo>=</mo> <mn>0.1</mn> <mo>]</mo> </mrow> </semantics></math>⟼ [<math display="inline"><semantics> <mrow> <msub> <mi>R</mi> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>=</mo> <mn>1</mn> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mi>α</mi> <mo>=</mo> <mn>0.2</mn> </mrow> </semantics></math>] (expansion). Blue circles indicate the seeds corresponding to annotated symptoms. Red frames indicate false positives.</p> "> Figure 12
<p>Detailed view of the reconstructed symptoms after growth: from seeds [<math display="inline"><semantics> <mrow> <msub> <mi>R</mi> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>=</mo> <mn>2.5</mn> </mrow> </semantics></math>,<math display="inline"><semantics> <mrow> <mi>α</mi> <mo>=</mo> <mn>0.1</mn> <mo>]</mo> </mrow> </semantics></math>⟼ [<math display="inline"><semantics> <mrow> <msub> <mi>R</mi> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> </msub> <mo>=</mo> <mn>1</mn> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <mi>α</mi> <mo>=</mo> <mn>0.2</mn> </mrow> </semantics></math>] growth. Seed pixels are represented in yellow, while pixels added during the growth phase are represented in pink. These four examples correspond to the numbered symptoms indicated in <a href="#sensors-20-04380-f011" class="html-fig">Figure 11</a>, (<b>a</b>) Large coalesced oilspot, (<b>b</b>) crown of necrosing oilspots, (<b>c</b>) early adaxial symptoms and (<b>d</b>) early abaxial symptoms.</p> ">
Abstract
:1. Introduction
2. Plant Material and Instrumentation
2.1. Cultivation Environment
2.2. Instrumentation
3. Image Processing Pipeline
3.1. Joint Structure–Colour Features
3.1.1. Local Structure Tensor: A Tool to Extract and Represent Textural Information
3.1.2. Log-Euclidean (LE) Mapping of LST’s
3.1.3. Rotation Invariance
3.1.4. Describing Grapevine Healthy and Symptomatic Tissues with LSTs
3.2. Joint Representation of Structure and Colour
Tensorial Representation of Colour in the HSL Colour Space: TC-LEST Representation
3.3. Modelling Structure-Colour Features
3.4. Seed Growth Segmentation
3.4.1. Within Criteria: Retaining the Most Relevant Pixels of Downy Mildew Symptoms
3.4.2. “Between” Criteria: Discarding Uncertain Pixels
3.4.3. The Seed-Growth Process
4. Results & Discussion
4.1. Data Set and Validation Protocol
4.2. Determination of the Seeds
4.3. Seeds Rowth
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Abbreviations
BBCH | Biologische Bundesanstalt, Bundessortenamt und Chemische Industrie |
CELEST | Colour Extended Log-Euclidean Structure Tensor |
CNN | Convolutional Neural Network |
HSL | Hue Saturation Luminance |
LE | Log-Euclidean |
LST | Local Structure Tensor |
RCNN | Regional Convolutional Neural Network |
RGB | Red Blue Green |
SIFT | Scale Invariant Feature Transform |
SPD | Symmetric Positive Definite |
SVM | Support Vector Machine |
TC-LEST | Tensorial Colour Log-Euclidean Structure Tensor |
References
- Pertot, I.; Caffi, T.; Rossi, V.; Mugnai, L.; Hoffmann, C.; Grando, M.; Gary, C.; Lafond, D.; Duso, C.; Thiery, D.; et al. A critical review of plant protection tools for reducing pesticide use on grapevine and new perspectives for the implementation of IPM in viticulture. Crop. Prot. 2017, 97, 70–84. [Google Scholar] [CrossRef]
- Tona, E.; Calcante, A.; Oberti, R. The profitability of precision spraying on specialty crops: A technical–economic analysis of protection equipment at increasing technological levels. Precis. Agric. 2018, 19, 606–629. [Google Scholar] [CrossRef] [Green Version]
- Lamichhane, J.R. Pesticide use and risk reduction in European farming systems with IPM: An introduction to the special issue. Crop. Prot. 2017, 97, 1–6. [Google Scholar] [CrossRef]
- Mahlein, A.K. Plant Disease Detection by Imaging Sensors Parallels and Specific Demands for Precision Agriculture and Plant Phenotyping. Plant Dis. 2015, 100, 241–251. [Google Scholar] [CrossRef] [Green Version]
- Mahlein, A.K.; Rumpf, T.; Welke, P.; Dehne, H.W.; Plümer, L.; Steiner, U.; Oerke, E.C. Development of spectral indices for detecting and identifying plant diseases. Remote Sens. Environ. 2013, 128, 21–30. [Google Scholar] [CrossRef]
- Al Saddik, H.; Simon, J.C.; Cointault, F. Development of Spectral Disease Indices for ‘Flavescence Dorée’ Grapevine Disease Identification. Sensors 2017, 17, 2772. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Al Saddik, H.; Laybros, A.; Billiot, B.; Cointault, F. Using Image Texture and Spectral Reflectance Analysis to Detect Yellowness and Esca in Grapevines at Leaf-Level. Remote. Sens. 2018, 10, 618. [Google Scholar] [CrossRef] [Green Version]
- Albetis de la Cruz, J.L. Potentiel des Images Multispectrales Acquises Par Drone Dans la détection des Zones InfectéEs par la Flavescence DoréE de la Vigne. Ph.D. Thesis, Université de Toulouse, Paul Sabatier, France, 2018. [Google Scholar]
- Nouri, M. Détection PréCoce D’une Maladie Fongique « la Tavelure du Pommier » par Imagerie Hyperspectrale. Ph.D. Thesis, Université de Montpellier, Montpellier SupAgro, France, 2018. [Google Scholar]
- Stewart, E.; McDonald, B. Measuring Quantitative Virulence in the Wheat Pathogen Zymoseptoria tritici Using High-Throughput Automated Image Analysis. Phytopathology 2014, 104. [Google Scholar] [CrossRef] [Green Version]
- Arnal Barbedo, J.G. Digital image processing techniques for detecting, quantifying and classifying plant diseases. SpringerPlus 2013, 2, 660. [Google Scholar] [CrossRef] [Green Version]
- Vamsidhar, E.; Rani, P.J.; Babu, K.R. Plant Disease Identification and Classification using Image Processing. Int. J. Eng. Adv. Technol. 2019, 8, 2249–8958. [Google Scholar]
- Anthonys, G.; Wickramarachchi, N. An image recognition system for crop disease identification of paddy fields in Sri Lanka. In Proceedings of the 2009 International Conference on Industrial and Information Systems (ICIIS), Sri Lanka, Sri Lanka, 28–31 December 2009; pp. 403–407. [Google Scholar]
- Bock, C.; Parker, P.; Cook, A.; Gottwald, T. Visual Rating and the Use of Image Analysis for Assessing Different Symptoms of Citrus Canker on Grapefruit Leaves. Plant Dis. 2008, 92, 530–541. [Google Scholar] [CrossRef] [Green Version]
- Huang, K.Y. Application of Artificial Neural Network for Detecting Phalaenopsis Seedling Diseases Using Color and Texture Features. Comput. Electron. Agric. 2007, 57, 3–11. [Google Scholar] [CrossRef]
- Boulent, J.; Foucher, S.; Théau, J.; St-Charles, P.L. Convolutional Neural Networks for the Automatic Identification of Plant Diseases. Front. Plant Sci. 2019, 10, 941. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Rançon, F.; Bombrun, L.; Keresztes, B.; Germain, C. Comparison of SIFT Encoded and Deep Learning Features for the Classification and Detection of Esca Disease in Bordeaux Vineyards. Remote Sens. 2019, 11, 1. [Google Scholar] [CrossRef] [Green Version]
- Boulent, J.; Beaulieu, M.; St-Charles, P.L.; Théau, J.; Foucher, S. Deep learning for in-field image-based grapevine downy mildew identification. In Proceedings of the 12th European Conference on Precision Agriculture (ECPA), Montpellier, France, 8–11 July 2019; pp. 141–148. [Google Scholar] [CrossRef]
- Abdelghafour, F.; Rançon, F.; Keresztes, B.; Germain, C.; Da Costa, J.P. On-board colour imaging for the detection of downy mildew. In Proceedings of the 12th European Conference on Precision Agriculture (ECPA), Montpellier, France, 8–11 July 2019; pp. 195–202. [Google Scholar] [CrossRef]
- Abdelghafour, F.; Rosu, R.; Keresztes, B.; Germain, C.; Da Costa, J.P. A Bayesian framework for joint structure and colour based pixel-wise classification of grapevine proximal images. Comput. Electron. Agric. 2019, 158, 345–357. [Google Scholar] [CrossRef]
- Lorenz, D.H.; Eichhorn, K.W.; Bleiholder, H.; Klose, R.; Meier, U.; Weber, E. Growth Stages of the Grapevine: Phenological growth stages of the grapevine (Vitis vinifera L. ssp. vinifera)—Codes and descriptions according to the extended BBCH scale. Aust. J. Grape Wine Res. 1995, 1, 100–103. [Google Scholar] [CrossRef]
- Otsu, N. A Threshold Selection Method from Gray-Level Histograms. IEEE Trans. Syst. Man, Cybern. 1979, 9, 62–66. [Google Scholar] [CrossRef] [Green Version]
- Knutsson, H. Representing Local Structure Using Tensors; Technical report; Computer Vision Laboratory, Linkoping University: Linköping, Sweden, 1989. [Google Scholar]
- Bigün, J.; Granlund, G.H.; Wiklund, J. Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 1991, 13, 775–790. [Google Scholar] [CrossRef]
- Rosu, R.G.; Da Costa, J.P.; Donias, M. Structure tensor Log-Euclidean statistical models for texture analysis. In Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016; pp. 3553–3557. [Google Scholar] [CrossRef]
- Arsigny, V.; Fillard, P.; Pennec, X.; Ayache, N. Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magn. Reson. Med. 2006, 56, 411–421. [Google Scholar] [CrossRef] [PubMed]
- Rosu, R.; Donias, M.; Bombrun, L.; Said, S.; Regniers, O.; Da Costa, J.P. Structure Tensor Riemannian Statistical Models for CBIR and Classification of Remote Sensing Images. IEEE Trans. Geosci. Remote Sens. 2017, 55, 248–260. [Google Scholar] [CrossRef]
- Trussell, H.J.; Lin, J.; Shamey, R. Effects of texture on color perception. In Proceedings of the 2011 IEEE 10th IVMSP Workshop: Perception and Visual Signal Analysis, Ithaca, NY, USA, 16–17 June 2011; pp. 7–11. [Google Scholar] [CrossRef]
- Di Zenzo, S. A note on the gradient of a multi-image. Comput. Vision, Graph. Image Process. 1986, 33, 116–125. [Google Scholar] [CrossRef]
- De Luis-Garcia, R.; Deriche, R.; Alberola-Lopez, C. Texture and color segmentation based on the combined use of the structure tensor and the image components. Signal Process. 2008, 88, 776–795. [Google Scholar] [CrossRef]
- Rittner, L.; Flores, F.C.; Lotufo, R.A. A tensorial framework for color images. Pattern Recognit. Lett. 2010, 31, 277–296. [Google Scholar] [CrossRef]
- Angulo, J. Quaternion Colour Representations and Derived Total Orderings for Morphological Operators. In Proceedings of the CGIV 2008/MCS’08 4th European Conference on Colour in Graphics, Imaging, and Vision and 10th International Symposium on Multispectral Colour Science; Society for Imaging Science and Technology, Terrassa, Barcelona, Spain, 9–13 June 2008; pp. 417–422, ISBN 978-0-89208-262-6. [Google Scholar]
- Said, S.; Bombrun, L.; Berthoumieu, Y.; Manton, J.H. Riemannian Gaussian Distributions on the Space of Symmetric Positive Definite Matrices. IEEE Trans. Inf. Theory. 2017, 63, 2153–2170. [Google Scholar] [CrossRef] [Green Version]
- Ilea, I.; Bombrun, L.B.; Said, S.; Berthoumieu, Y. Covariance Matrices Encoding Based on the Log-Euclidean and Affine Invariant Riemannian Metrics. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 18–22 June 2018; pp. 393–402. [Google Scholar] [CrossRef] [Green Version]
- Julesz, B.; Bergen, J.R. Human factors and behavioral science: Textons, the fundamental elements in preattentive vision and perception of textures. Bell Syst. Tech. J. 1983, 62, 1619–1645. [Google Scholar] [CrossRef]
- Cocquerez, J.P.; Bolon, P. Analyse D’images: Filtrage et Segmentation; Enseignement de la physique traitement du signal; Masson: Paris, France, 1995. [Google Scholar]
- Mahalanobis, P.C. On the Generalised Distance in Statistics. Natl. Inst. Sci. 1936, 2, 49–55. [Google Scholar]
- Davis, J.; Goadrich, M. The Relationship Between Precision-Recall and ROC Curves. In Proceedings of the 23rd international conference on Machine learning, Pittsburgh, PA, USA, 25–29 June 2006; pp. 233–240. [Google Scholar] [CrossRef] [Green Version]
0.01 | 0.02 | 0.05 | 0.1 | 0.15 | 0.20 | 0.25 | 0.30 | |
% Symptoms with seeds | 49 | 66 | 83 | 98.1 | 98.5 | 99.3 | 99.3 | 99.3 |
Min | Max | Mean | ||
---|---|---|---|---|
Precision | 0.68 | 0.91 | 0.83 | 0.08 |
Recall | 0.48 | 0.98 | 0.76 | 0.16 |
-score | 0.75 | 0.89 | 0.79 | 0.10 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Abdelghafour, F.; Keresztes, B.; Germain, C.; Da Costa, J.-P. In Field Detection of Downy Mildew Symptoms with Proximal Colour Imaging. Sensors 2020, 20, 4380. https://doi.org/10.3390/s20164380
Abdelghafour F, Keresztes B, Germain C, Da Costa J-P. In Field Detection of Downy Mildew Symptoms with Proximal Colour Imaging. Sensors. 2020; 20(16):4380. https://doi.org/10.3390/s20164380
Chicago/Turabian StyleAbdelghafour, Florent, Barna Keresztes, Christian Germain, and Jean-Pierre Da Costa. 2020. "In Field Detection of Downy Mildew Symptoms with Proximal Colour Imaging" Sensors 20, no. 16: 4380. https://doi.org/10.3390/s20164380