DE102023122228A1 - METHOD, EVALUATION SYSTEM, EVALUATION DEVICE AND SYSTEM OR DEVICE FOR EVALUATING SPECTRAL DATA - Google Patents

Abstract

To carry out a status analysis of a plant or another eukaryote, an evaluation device or an evaluation system (30) has an evaluation circuit (34) which is set up to carry out an evaluation (36) of spectral data. The evaluation has at least one attention mechanism.

Description

TECHNICAL FIELD

The invention relates to methods and evaluation systems that can be used in connection with a spectral analysis of a plant or other eukaryotes. The invention particularly relates to such methods and evaluation systems in which an optical quantity recorded with wavelength resolution is evaluated. The invention particularly relates to such devices, systems and methods that can be used with useful or cultivated plants.

BACKGROUND

The analysis of eukaryotes, such as plants or parts of plants, as well as food products that can be obtained from plant or animal substances, is of great importance. It can be used, for example, to ensure adequate nutrient supply and/or to detect possible diseases.

Conventional methods for analyzing the condition of plants have traditionally relied on the assessment of a human expert. Techniques that use measuring devices to objectively and quantitatively analyze plant conditions are becoming increasingly important. For example, such techniques can be used to detect an oversupply or undersupply of certain nutrients.

The WO 2019/169434 A1 , the US 7 804 588 B2 , the US 11 320 307 B2 and the DE 10 2018 103 509 B3 reveal exemplary techniques for this.

To understand the invention disclosed here, features of specific models of artificial intelligence (AI) are helpful, namely the so-called attention mechanism (or attention module) and the so-called transformer model. Basics can be found, for example, in A. Vaswani et al., “Attention Is All You Need,” 2017, Advances in Neural Information Processing Systems 30 (NIPS 2017), pp. 6000-6010 (https://arxiv.org/abs/1706.03762 ) revealed. Transformer models with their attention mechanisms are often used in speech recognition. Examples are in US 10 789 427 B2 , US 11 494 561 B2 , US 2022/0005465 A1 and US 2022/0108689 A1 Another application of transformer models with their attention mechanisms in image processing is in A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” International Conference on Learning Representations (IPCLR) 2021 These techniques are referred to below, even though they have not previously been discussed in connection with a condition analysis of plants by analyzing spectral data recorded on the plant.

The application not yet published on the priority date of the present application DE 10 2023 113 716.6 discusses the application of a ViT (Vision Transformer) to image data of a plant.

There is still a need for methods, devices and systems that offer mechanical support, optionally even further automation, in the evaluation for the condition analysis of plants, other eukaryotes or food products derived from them.

SUMMARY

The invention is based on the object of specifying improved methods, devices and systems with which information about the condition of plants, other eukaryotes or food products obtained therefrom can be obtained. The invention is particularly based on the object of specifying methods, devices and systems that provide improvements with regard to condition analysis based on recorded spectra. The invention is also based on the object of specifying such methods, devices and systems that can be used for evaluating spectra obtained from crops or cultivated plants.

According to the invention, a method, an evaluation device or an evaluation system and a system are specified as defined in the independent claims. The dependent claims define preferred and advantageous embodiments.

A method for evaluating spectral data according to one aspect of the invention comprises the following: receiving the spectral data by at least one evaluation circuit, wherein the spectral data comprises spectral data recorded on a eukaryote, a part of a eukaryote or a food product, and analyzing the spectral data by the at least one evaluation circuit. Analyzing the spectral data by the at least one evaluation circuit comprises: dividing the spectral data or data generated therefrom by preprocessing into several parts, analyzing the several parts using an analysis logic, wherein an input of the analysis logic receives the several parts, and wherein a Output of the analysis logic provides codings of the plurality of parts, wherein the analysis logic has at least one attention mechanism, and determining at least one evaluation result based on the codings.

The method provides various technical effects and advantages. Spectral data can be evaluated using at least one attention mechanism. This makes it possible to determine a status analysis of the eukaryote (for example a plant), a part of a eukaryote (for example a leaf of a plant) or a food product using the attention mechanism. The evaluation can thus be carried out using a data-driven analysis logic. Subjective assessment criteria and potentially caused errors in the evaluation can be reduced.

The method may be or comprise a method for determining a state of a plant or a plant part by analyzing the spectral data, wherein the evaluation result is an evaluation result representing the state.

This allows the state of a plant or part of a plant to be determined using the analysis logic that can be generated in a data-driven manner. By using an analysis logic that has at least one attention mechanism, relative relationships of different parts of the spectral data (for example, characteristic sequences of local maxima and/or minima that are characteristic of a state) can be efficiently recognized during the evaluation and used for the state analysis.

The at least one evaluation result may comprise at least one quantitative analysis result.

This makes it possible to determine evaluation techniques not only for determining qualitative results (e.g., identification of present and absent pathological conditions or situations) but also for quantitative evaluation.

The at least one quantitative analysis result may have a continuously variable value from a range of values.

This allows continuously variable characteristics (e.g. nutrient concentrations) to be determined and output objectively from the spectral data.

The at least one quantitative analysis result may have a value selected from an ordinal scale.

This can be used, for example, to express membership in different states represented by the ordinal scale. The ordinal scale can, for example, be determined and output for different nutrient concentration ranges (for example, nutrient deficiency / acceptable concentration / excess supply or a value from another scale that maps the nutrient concentration to values on an ordinal scale).

The at least one quantitative analysis result may comprise a concentration of at least one chemical element or at least one chemical compound.

This means that the evaluation techniques can be used to determine concentration based on the spectral data. This type of concentration determination is of great importance in agricultural engineering, for example.

The at least one evaluation result may comprise at least one nutrient concentration.

This means that the evaluation techniques can be used to determine the concentration of nutrients based on the spectral data. This type of determination of the concentration of nutrients is of great importance in agricultural engineering, for example.

The at least one evaluation result may have a value of a nominal scale. The nominal scale may, for example, represent the presence or absence of different types of biotic and/or abiotic stress.

This means that the analysis logic can be used, for example, for a qualitative analysis.

The analysis logic may include an encoder. The encoder may include a stack of multiple attention blocks, wherein the multiple attention blocks each include at least one attention mechanism.

This allows artificial intelligence (AI) models to be used to evaluate the spectral data in order to determine the codings that are known, for example, from text analysis or the analysis of image data. The encoder provides the codings.

The at least one attention mechanism may comprise at least one self-attention mechanism.

This allows AI models to be used to evaluate the spectral data in order to determine the codings known, for example, from text analysis or the analysis of image data.

The at least one attention mechanism may comprise at least one multi-head attention mechanism.

This allows AI models to be used to evaluate the spectral data to determine the codings known from text analysis or image data analysis, for example. The use of a multi-head attention mechanism can provide particularly reliable results.

The method can have a wavelength-dependent position coding. Due to the wavelength-dependent position coding, relative position information of various features of the spectral data is retained for the evaluation, for example information about the wavelength-dependent sequence in which data points of an optical characteristic of the spectral data were recorded.

This allows an evaluation of the spectral data taking into account the relative position of the various features (such as local maxima and/or local minima) of the spectral data.

The method may include preprocessing of the spectral data prior to the analysis logic. During preprocessing, various data may be used in addition to the spectral data, for example spectra of standards and/or calibration measurements.

Preprocessing allows “features” to be determined from the spectral data. These features can then be further processed using an encoder that has at least one attention mechanism.

Preprocessing may include filtering of the spectral data.

This allows the spectral data to be prepared in such a way that they can be further processed particularly efficiently by the analysis logic with at least one attention mechanism.

Filtering parameters can be determined data-driven.

This makes it possible to carry out preprocessing in a data-driven and therefore objective manner.

Preprocessing can be done using another Kl model.

This makes it possible to carry out preprocessing in a data-driven and therefore objective manner.

The method may comprise training the analysis logic or at least a portion of the analysis logic using training data.

This makes it possible to carry out the evaluation in a data-driven and therefore objective manner.

The training may comprise training a transformer encoder having at least one self-attention mechanism.

This makes it possible to carry out the evaluation in a data-driven and therefore objective manner.

The training may involve joint training of the transformer encoder and preprocessing prior to the transformer encoder.

This makes it possible to carry out the evaluation in a data-driven and therefore objective manner, whereby the functionality of the transformer encoder and the preprocessing are inherently coordinated through training.

The training data can contain multiple spectra and associated annotations. This allows an analysis logic to be trained whose input receives the spectral data.

The training data can contain multiple spectrally derived data and associated annotations.

This allows an analysis logic to be trained whose input receives data derived from the spectral data.

The training data may include training data derived from the spectra by augmentation.

This allows a training data set to be generated that meets specific requirements is sufficient. This is particularly desirable if the analysis logic, for example, provides results in the form of levels of an ordinal scale and the augmentation ensures that the different levels of the ordinal scale are sufficiently represented in the training data. Similarly, the augmentation can be used advantageously if the analysis logic, for example, provides values from at least one continuous value range in order to ensure that the at least one continuous value range is sufficiently represented in the training data (for example with a sufficient density per value interval).

The spectral data may comprise spectral data collected on a plant, a part of a plant, a plant-like eukaryote, or a plant or animal food.

This enables a spectrum-based determination of the condition of plants, plant parts, plant-like eukaryotes (such as algae) or a food, whereby the evaluation is made possible using a data-driven analysis logic.

The method may comprise outputting the evaluation result via a human-machine interface.

This makes it possible to provide a result of the evaluation, for example status information concerning a plant.

The method may comprise outputting the evaluation result or data derived therefrom via a data interface.

This allows a result of the evaluation, for example status information concerning a plant, to be provided in a form that is particularly suitable for storage or further automatic use.

The method may include executing a control function based on the evaluation result.

This means that the result of the evaluation, for example status information concerning a plant, can be used for automatic control functions. This can be of particular interest in agricultural engineering.

The control function can be executed automatically.

This means that a result of the evaluation, for example status information concerning a plant, can be used for automatic control functions.

According to a further aspect of the invention, a method is provided for generating an analysis logic of an evaluation system or an evaluation device for evaluating spectral data. The analysis logic has at least one AI model that has at least one attention mechanism. The method comprises training the AI model using training data by a computing device or a computing system and providing parameters of the trained AI model by the computing device or the computing system to the evaluation system or the evaluation device for application to the spectral data, wherein the spectral data is recorded on a eukaryote (for example a plant or a plant part) or a food product (for example a plant or animal food product).

This allows the analysis logic to be generated in a data-driven manner. The objectivity of analysis results is improved.

The training may comprise training a transformer encoder having at least one self-attention mechanism.

This makes it possible to provide the analysis logic for evaluation in a data-driven and therefore objective manner.

The training may involve joint training of the transformer encoder and preprocessing prior to the transformer encoder.

This makes it possible to provide the analysis logic for evaluation in a data-driven and thus objective manner, whereby the functionality of the transformer encoder and the preprocessing are inherently coordinated through training.

The training data can contain multiple spectra and associated annotations. This allows an analysis logic to be trained whose input receives the spectral data.

The training data can contain multiple spectrally derived data and associated annotations.

This allows an analysis logic to be trained whose input receives data derived from the spectral data.

The training data may include training data derived from the spectra by augmentation.

This makes it possible to generate a training data set that meets certain requirements. This is particularly desirable if the analysis logic provides classification results, for example, and the augmentation ensures that the different classes are sufficiently represented in the training data.

The spectral data may comprise spectral data collected on a plant, a part of a plant, a plant-like eukaryote, or a plant or animal food.

This enables a spectrum-based determination of the condition of plants, plant parts, plant-like eukaryotes (such as algae) or a food, whereby the evaluation is made possible using a data-driven analysis logic.

The provision of parameters of the trained AI model for use by the evaluation system or the evaluation device may comprise an output via a data interface and/or via a data network.

This makes it possible to provide the parameters that enable the analysis logic to be applied by the evaluation device or the evaluation system, even if the evaluation device or the evaluation system is located remotely from the computing device carrying out the training.

The method may comprise the use of the parameters of the trained Kl model by the evaluation device.

This allows the data-based trained analysis logic to be applied to spectral data, for example to perform a condition analysis of plants or plant-like organisms (e.g. algae).

According to a further aspect of the invention, machine-readable data are provided which are determined using the method for training the analysis logic and can be used by the evaluation device or the evaluation system in order to use the analysis logic to evaluate spectral data which have been acquired on a eukaryote, a part of a eukaryote or a food product.

This makes it possible to achieve the effects described with reference to the methods.

According to a further aspect of the invention, an evaluation system or an evaluation device for evaluating spectral data is provided, which comprises: an interface for receiving the spectral data, wherein the spectral data comprises spectral data recorded on a eukaryote, a part of a eukaryote or a food product; and at least one evaluation circuit which is configured to carry out at least the following processing steps for analyzing the spectral data: dividing the spectral data or data generated therefrom by preprocessing into several parts, analyzing the several parts using an analysis logic, wherein an input of the analysis logic receives the several parts, and wherein an output of the analysis logic provides codings of the several parts, wherein the analysis logic has at least one attention mechanism, and determining at least one evaluation result based on the codings.

Various technical effects are achieved with the evaluation system or the evaluation device. Spectral data can be evaluated using at least one attention mechanism. This makes it possible to determine a status analysis of the eukaryote (for example a plant), part of a eukaryote (for example a leaf of a plant) or a food product using the attention mechanism. The evaluation can thus be carried out using a data-driven analysis logic. Subjective assessment criteria and potentially resulting errors in the evaluation can be reduced.

The at least one evaluation circuit, the evaluation system or the evaluation device can be set up to carry out the method according to one of the embodiments. The effects achieved in each case correspond to the effects explained with reference to optional method features.

According to a further aspect of the invention, a system or a device is provided which has the evaluation system or the evaluation device according to the invention and a spectral analytical detection device for detecting the spectral data.

The spectral analytical detection device can have one or more of the following configurations: a spectrometer, a hyperspectral camera, a controllable multispectral Illumination source in combination with an optoelectrical converter (for example a camera chip), a multispectral camera. The spectral analytical detection device can have a measuring system that enables the recording of several wavelengths. For this purpose, the spectral analytical detection device can have, for example, a spectrometer (in which light is split depending on the wavelength), filter-based systems (by blocking light components), single-wavelength diodes or other designs. For example, a detector with multi-sensor channels that are sensitive to different spectral ranges can be used.

This enables the application of analysis logic, which includes at least one attention mechanism, to spectral data acquired with such modalities.

The spectral analytical detection device can be configured to collect the spectral data in a reflection arrangement (in which the illumination and wavelength-resolved detection device are arranged on the same side relative to the object) or a transmission arrangement (in which the illumination and wavelength-resolved detection device are arranged on different sides relative to the object) or a transflectance measurement (in which radiation from the illumination penetrates the sample under investigation, is reflected at the back, penetrates the sample a second time and is detected by a wavelength-resolved detection device on the same side as the illumination).

This enables the application of the analysis logic, which has at least one attention mechanism, to spectral data acquired in a reflection or transflection or transmission geometry.

The spectral analytical detection device can have a positioning system and can be set up to determine and speak position data relating to a position and optionally an orientation of the spectral analytical detection device in association with the detected spectral data. The positioning system can, for example, have a navigation satellite system (GNSS) and/or acceleration sensors.

This makes it possible to assign evaluation results obtained using the analysis logic to different positions in a cultivation area (e.g. a field).

In the method and system, the spectral analytical detection device can be mounted on a vehicle or an aircraft or a robot. The method can comprise controlling at least one actuator for positioning the spectral analytical detection device.

This allows spectral data to be collected efficiently and partially or fully automatically and a condition analysis to be carried out associated with the respective collection position.

In the method and system, the spectral analytical detection device can be mounted on a sample conveying system, for example a conveyor belt. The method can comprise controlling at least one actuator for positioning the sample conveying system and/or the sample on the sample conveying system.

This allows spectral data to be collected efficiently and partially or fully automatically and a condition analysis to be carried out associated with the respective collection position.

According to a further aspect of the invention, a use of the methods, evaluation devices, evaluation systems or systems according to embodiments is provided for determining cultivation measures. Using a method, an evaluation device, an evaluation system or a system according to an embodiment, a condition analysis for plants or plant-like organisms (for example algae) can be carried out, wherein, depending on the condition analysis, it is determined which measures are to be taken to improve plant health.

As a result, the methods, evaluation devices, evaluation systems or systems disclosed here can be used to improve cultivation measures.

The methods, evaluation devices, evaluation systems, systems and system components according to embodiments of the invention achieve various effects. In particular, the acquisition and/or evaluation of spectral analytical information can be supported using one or more images that are acquired with a near-field camera.

The devices, processes, systems and system components can be used in various areas. These include state analysis lysis of crops or cultivated plants in agricultural technology, but is not limited to this.

BRIEF DESCRIPTION OF THE CHARACTERS

• 1 ist eine schematische Darstellung eines Systems, das eine Auswerteeinrichtung nach einem Ausführungsbeispiel aufweist.
• 2 zeigt eine schematische Darstellung einer Analyselogik der Auswerteeinrichtung.
• 3 zeigt eine schematische Darstellung einer weiteren Analyselogik der Auswerteeinrichtung.
• 4 zeigt eine schematische Darstellung einer weiteren Analyselogik der Auswerteeinrichtung.
• 5 zeigt eine schematische Darstellung einer weiteren Analyselogik der Auswerteeinrichtung.
• 6 zeigt eine schematische Darstellung einer weiteren Analyselogik der Auswerteeinrichtung.
• 7 ist ein Flussdiagramm eines Verfahrens.
• 8 ist ein Flussdiagramm eines Verfahrens.
• 9 zeigt Ergebnisse einer quantitativen Nährstoffkonzentrationsbestimmung, wobei sowohl ein erfindungsgemäß bestimmtes Ergebnis als auch ein mit einer herkömmlichen Technik bestimmtes Ergebnis dargestellt sind.
• 10 zeigt Ergebnisse einer quantitativen Nährstoffkonzentrationsbestimmung, wobei sowohl ein erfindungsgemäß bestimmtes Ergebnis als auch ein mit einer herkömmlichen Technik bestimmtes Ergebnis dargestellt sind.
• 11 zeigt Ergebnisse einer quantitativen Nährstoffkonzentrationsbestimmung, wobei sowohl ein erfindungsgemäß bestimmtes Ergebnis als auch ein mit einer herkömmlichen Technik bestimmtes Ergebnis dargestellt sind.
• 12 ist eine schematische Darstellung eines Systems nach einem Ausführungsbeispiel.
• 13 ist ein Flussdiagramm eines Verfahrens.
• 14 ist ein Flussdiagramm eines Verfahrens.
• 15 ist ein Blockdarstellung einer Vorrichtung mit einer spektralanalytischen Erfassungseinrichtung.
• 16 ist ein Blockdarstellung einer Vorrichtung mit einer spektralanalytischen Erfassungseinrichtung.
• 17 ist ein Blockdarstellung einer Vorrichtung mit einer spektralanalytischen Erfassungseinrichtung.
• 18 ist ein Blockdarstellung einer Vorrichtung mit einer spektralanalytischen Erfassungseinrichtung.
• 19 ist ein Blockdarstellung einer Vorrichtung mit einer spektralanalytischen Erfassungseinrichtung.
• 20 ist ein Flussdiagramm eines Verfahrens.
• 21 zeigt schematisch ein Zwischenergebnis des Verfahrens von 20.
• 22 illustriert weitere Aspekte des Verfahrens von 20.
• 23 zeigt eine schematische Darstellung eines Systems.
• 24 ist ein Flussdiagramm eines Verfahrens.
• 25 zeigt eine Ausgestaltung einer spektralanalytischen Erfassungseinrichtung bei Systemen und Verfahren nach Ausführungsbeispielen.
• 26 zeigt eine weitere Ausgestaltung einer spektralanalytischen Erfassungseinrichtung bei Systemen und Verfahren nach Ausführungsbeispielen.
• 27 zeigt eine weitere Ausgestaltung einer spektralanalytischen Erfassungseinrichtung bei Systemen und Verfahren nach Ausführungsbeispielen.
• 28 zeigt eine weitere Ausgestaltung einer spektralanalytischen Erfassungseinrichtung bei Systemen und Verfahren nach Ausführungsbeispielen.
• 29 ist ein Flussdiagramm eines Verfahrens nach einem Ausführungsbeispiel.

Embodiments of the invention are described with reference to the figures. In the figures, similar or identical reference symbols designate elements with similar or identical design and/or function.

• 1 is a schematic representation of a system having an evaluation device according to an embodiment.
• 2 shows a schematic representation of an analysis logic of the evaluation device.
• 3 shows a schematic representation of a further analysis logic of the evaluation device.
• 4 shows a schematic representation of a further analysis logic of the evaluation device.
• 5 shows a schematic representation of a further analysis logic of the evaluation device.
• 6 shows a schematic representation of a further analysis logic of the evaluation device.
• 7 is a flow chart of a process.
• 8 is a flow chart of a process.
• 9 shows results of a quantitative nutrient concentration determination, showing both a result determined according to the invention and a result determined using a conventional technique.
• 10 shows results of a quantitative nutrient concentration determination, showing both a result determined according to the invention and a result determined using a conventional technique.
• 11 shows results of a quantitative nutrient concentration determination, showing both a result determined according to the invention and a result determined using a conventional technique.
• 12 is a schematic representation of a system according to an embodiment.
• 13 is a flow chart of a process.
• 14 is a flow chart of a process.
• 15 is a block diagram of a device with a spectral analytical detection device.
• 16 is a block diagram of a device with a spectral analytical detection device.
• 17 is a block diagram of a device with a spectral analytical detection device.
• 18 is a block diagram of a device with a spectral analytical detection device.
• 19 is a block diagram of a device with a spectral analytical detection device.
• 20 is a flow chart of a process.
• 21 shows schematically an intermediate result of the procedure of 20 .
• 22 illustrates further aspects of the procedure of 20 .
• 23 shows a schematic representation of a system.
• 24 is a flow chart of a process.
• 25 shows an embodiment of a spectral analytical detection device in systems and methods according to embodiments.
• 26 shows a further embodiment of a spectral analytical detection device in systems and methods according to embodiments.
• 27 shows a further embodiment of a spectral analytical detection device in systems and methods according to embodiments.
• 28 shows a further embodiment of a spectral analytical detection device in systems and methods according to embodiments.
• 29 is a flowchart of a method according to an embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of the invention are described with reference to the figures. In the figures, similar or identical reference symbols designate elements with similar or identical design and/or function.

While embodiments are described in connection with a nutrient analysis or other condition analysis of a useful or cultivated plant, the embodiments are not limited thereto.

The features of the embodiments can be combined with each other, provided that this is the following description is not expressly excluded.

Methods, evaluation devices, evaluation systems and systems according to embodiments of the invention are designed to evaluate at least one spectral data recorded on a eukaryote (in particular a plant or a plant-like organism such as an alga), a part of a eukaryote (for example a leaf or another plant part) or a food product. An evaluation result can be, for example, a state variable of the sample examined by spectral analysis, for example a nutrient concentration.

The terms spectral data and spectrum, as used here, refer to an optical quantity recorded for a plurality of wavelengths (for example reflection, transflection, transmission, etc.), which are present in a form that enables an assignment of the optical quantity recorded in each case to the associated wavelength. The spectral data and spectra can have the optical quantity for wavelengths in at least one wavelength range, wherein the wavelengths for which the optical quantity was recorded can differ from one another by a constant or wavelength-dependently variable wavelength spacing.

The spectral data and spectra can be recorded using a spectral analytical recording unit. The term spectral analytical recording device, as used here, includes a device that is capable of and designed to record an optical measurement variable for multiple wavelengths in the same measurement area of the plant. The spectral analytical recording device can have a spectrometer. However, this is not necessarily required. For example, the spectral analytical recording device can be configured, alternatively or in addition to using a spectrometer, to record the optical measurement variable for the multiple wavelengths sequentially in time, for example by actively irradiating with different wavelengths and determining the intensity of the scattered or reflected light detected in each case.

At least one attention mechanism is used to evaluate the spectral data. The attention mechanism is known per se, but has so far been used primarily in applications such as text analysis and image analysis. The expert can obtain further information about the attention mechanism and its implementation, for example, from the documents in A. Vaswani et al., “Attention Is All You Need,” 2017, Advances in Neural Information Processing Systems 30 (NIPS 2017), pp. 6000-6010 (https://arxiv.org/abs/1706.03762 ) and A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” International Conference on Learning Representations (IPCLR) 2021 remove.

The applicant has found that these techniques, which have not previously been discussed in connection with a condition analysis of plants by analysis of spectral data recorded on the plant, can surprisingly also be applied to the condition analysis of plants, in particular to the analysis of spectral data recorded on the plant, whereby the measures disclosed in detail here are carried out to adapt the conventional methods.

In general, the attention mechanism is a computational method for analyzing data in an artificial intelligence (AI) model that, after training, is able to recognize relationships in the input data (relevant for a task defined by training data).

The attention mechanism transforms individual parts of the input data (also called tokens) or representations of these parts (also called embedded tokens) into a new representation, whereby information from the entire input data can be taken into account in this transformation. To transform a token, the token is compared with one, several, optionally all other tokens of the input data by applying a comparison function. The tokens are then weighted using an aggregation function and transferred to the new representation, whereby the weighting is determined from the result of applying the comparison function. An attention mechanism therefore generally has an aggregation function and a comparison function.

An attention mechanism can process the tokens before applying the comparison function and/or the aggregation function. The processing of the tokens before applying the comparison function and/or the aggregation function can have trainable parameters.

One possible form is the so-called “scaled dot-product attention” according to A. Vaswani et al., “Attention Is All You Need,” 2017, Advances in Neural Information Processing Systems 30 (NIPS 2017), pp. 6000-6010 (https://arxiv.org/abs/1706.03762 ), which determines at least one query (Q) for each token, at least one key (K) for each token, and based on the pair By comparing the at least one query and the keys, weights are determined for aggregating the values (V) obtained from each token. The determination of query, key and value can have trainable parameters. The trainable parameters that are trained in training can define how input data is processed to obtain query, key and value. This processing can be determined, for example, by linear projection matrices W _Q , W _K and W _V , whereby these projection matrices can be trained in training.

When used in the context considered here for evaluating spectral data, an analysis logic having the at least one attention mechanism is set up to process several parts of spectral data (which can cover different wavelength ranges, for example) or input data determined therefrom by preprocessing. The at least one attention mechanism can be at least one self-attention mechanism, in particular also a multi-head self-attention mechanism.

The use of an analysis logic that has at least one attention mechanism enables the recognition of extensive pairwise relationships in spectral data. Parameter values of the comparison operations are data-dependent. This has surprisingly proven to be particularly advantageous for the analysis of spectral data. In particular, after training, a local or global context in the spectral data is taken into account during processing by the analysis logic, which has at least one attention mechanism. This is in contrast to, for example, conventional filters in CNNs ("convolutional neural networks"), which have unchanging weights after training and are limited to the recognition of local relationships depending on the size of the learned filters.

Before and/or after the attention mechanism, other processing can be carried out (for example, a normalization and a FF NN (“Feed Forward Neural Network”, hereinafter and in this field of technology also referred to as a “feed forward network” or “feed forward layer”) and a residual calculation). A block that can have additional processing in addition to the attention mechanism is also referred to as an attention block. Thus, the analysis logic, as in 3 and 4 shown, have several consecutively executed attention blocks, each of which can have at least one attention mechanism and optionally further processing before and/or after the attention mechanism. The attention mechanism can have a self-attention mechanism, in particular a multi-head self-attention mechanism.

The analysis logic having the attention mechanism can have several attention blocks. In particular, the analysis logic can have an encoder with a stack of several attention blocks. The attention blocks can each have further processing before and/or further processing after the attention mechanism, for example a feed forward network and/or a normalization layer such as layer normalization and/or other normalization functions and/or residual calculation.

The analysis logic can be trained using training data such that the output of the analysis logic provides a state variable associated with the state of the eukaryote, part of the eukaryote or food product. In particular, the analysis logic can be trained such that its output provides a quantitative state variable (for example a nutrient concentration) from a continuous value range, an ordinal value scale or a nominal scale.

The term eukaryotes as used here includes plants and plant-like organisms, in particular algae. The devices, systems and methods disclosed here can be designed in particular for analyzing the condition of plants or parts of plants.

1 shows a system 10 that is set up to evaluate spectral data recorded on a plant 2. The system 10 has a device 20 with a spectral analytical recording device for recording spectral data on the plant 2 and an evaluation device 30. The functions of the evaluation device 30 can also be implemented by a system, for example a server system.

The evaluation device 30 is set up to evaluate the spectral data acquired by the device 20. The evaluation device 30 has one or more interfaces 31 for receiving the spectral data. The evaluation device 30 has a storage system 33 in which parameters (in particular the parameters trained in a training) of an analysis logic for carrying out an evaluation 36 of the spectral data can be stored in a non-volatile manner. The evaluation device has at least one evaluation circuit 34 which is set up to carry out the evaluation 36 of the spectral data. Based on an evaluation result, an interface controller 35 can generate an output and transmit it via a data interface and/or provide a human-machine interface 32.

The at least one evaluation circuit 34 used to carry out the various control and processing functions can have one or more integrated circuits to carry out the control of the interface(s) 31 and the human-machine interface 32 and the evaluation of the spectral data. The one or more integrated circuits can, for example, have any or any combination of the following circuits or circuit components: an application-specific special circuit (ASIC), a field-programmable gate array (FPGA), a processor (for example a CPU ("central processing unit"), GPU ("graphic processor unit") or TPU ("tensor processor unit")), a controller, one or more quantum gates, a circuit for quantum information processing, other integrated circuits.

The memory system 33 can store machine-readable command code which, when executed by the evaluation circuit 34, causes the functions and steps disclosed here to be carried out. The memory system 33 also stores parameters which enable an evaluation of the spectral data, for example for a state analysis. The parameters stored in the memory system 25 for evaluating the spectral data can, for example, have one or more of the following parameters: parameters for processing tokens of an attention block or of several attention blocks of a stack of attention blocks (for example parameters which define the dependency of query, key and value on the input data); parameters of at least one further trainable part of the AI model, for example of neural networks, normalization layers and/or feed forward layers which can be connected upstream and/or downstream of the attention mechanism or the attention mechanisms.

Even if in 1 the evaluation system 30 is shown separately from the device 20, the acquisition of the spectral data and its evaluation can take place in an integrated device. In particular, the evaluation circuit 34 can be arranged in the same housing as the spectral analytical detection device.

2 , 3 , 4 , 5 and 6 each show embodiments of an analysis logic that can be used to evaluate 36 the spectral data. The analysis logic can have an input to receive spectral data 40 or data derived therefrom by preprocessing. The analysis logic in each case has at least one attention mechanism. The analysis logic can in particular have at least one self-attention mechanism. The analysis logic can have at least one encoder of a transformer, which can have at least one self-attention mechanism and in particular a stack of attention blocks, each with at least one (multi-head) self-attention mechanism.

In the 2 , 3 , 4 and 5 In the embodiments of the spectrum evaluation 36 shown, the spectral data 40 or input data obtained therefrom by preprocessing are divided into several parts. The parts can cover different wavelength ranges. The parts can cover interleaved or overlapping wavelength ranges. The evaluation circuit 34 is set up to generate 42 the parts.

For those in 2 , 3 , 4 and 5 In the embodiments of the spectrum evaluation 36 shown, a position coding 43 is carried out. This is done in such a way that an optical measurement variable recorded in the spectral data for a specific wavelength or a specific wavelength range (which can be recorded, for example, in a reflection or transflection or transmission arrangement) remains assignable to the corresponding wavelength or the corresponding wavelength range. The position coding 43 is particularly technically advantageous because common variants of attention mechanisms are invariant with respect to an order of tokens in the input. In order to make the information about relative positions available to the AI model, position information is therefore added to the tokens by the position coding.

For example, a non-learned position encoding can be added by adding the (optionally processed) tokens with oscillating functions, such as sine functions, which may have different frequencies. For example, techniques based on p. 6 section 3.5 of the publication A. Vaswani et al., “Attention Is All You Need,” 2017, Advances in Neural Information Processing Systems 30 (NIPS 2017), pp. 6000-6010 (https://arxiv.org/abs/1706.03762 ) can be used. The position coding 43 can be learned or specified. When learning the position coding, the position coding can be trained together with the transformer.

In the 2 , 3 , 4 and 5 In the embodiments of the spectrum evaluation 36 shown, the parts of the spectral data 40 or the parts of the spectral data are selected by processing of the input data obtained by a transformer encoder 44 and/or transformer decoder 44' ( 5 ) is further processed. The transformer encoder 44 and/or transformer decoder 44' has at least one attention mechanism 45, 55. An output of the transformer encoder 44 and/or transformer decoder 44' provides a code for each of the parts of the spectral data 40 or input data obtained therefrom by preprocessing.

Both 2 , 3 , 4 and 5 In the embodiment of the spectrum evaluation 36 shown, the codings are mapped to a target value by a subsequent further processing 49. This can be done, for example, by a linear or non-linear combination of the codings. More complex pre- and/or post-processing steps are possible and are described in more detail below.

3 shows an embodiment in which the transformer encoder 44 has a stack 46 of several attention blocks 47. Several of the attention blocks 47 of the stack 46 can have identical configurations. The attention blocks 47 can thus be run through one after the other. Each attention block 47 has at least one attention mechanism 45. Optionally, before and/or after each execution of the attention mechanism 45 in the respective attention block 47, further operations 48, 48' can be carried out and/or further processing layers can be executed, for example one, several or all of: a feed forward network 48', a normalization layer 48, a residual calculation.

4 shows an embodiment in which the transformer encoder has a multi-head attention mechanism 55. In the embodiment shown, the analysis logic has several attention blocks 57 in a stack 56. The attention blocks 57 can be run through one after the other. Each attention block 57 has a multi-head attention mechanism 55. The multi-head attention mechanisms 55 are in 4 each designed as a multi-head self-attention mechanism. Optionally, before and/or after each execution of the multi-head attention mechanism 55 in the respective attention block 57, further operations 58, 58' can be carried out and/or further processing layers can be executed, for example one, several or all of: a feed forward network 58', a normalization layer 58, a residual calculation.

5 shows an embodiment in which the spectral data 40 are preprocessed. Preprocessing can be carried out, for example, by convolution with one or more filters 50. Other preprocessings are possible that generate features from the spectral data. The features advantageously remain assigned to wavelengths or wavelength ranges. The filtering 50 is accordingly advantageously carried out with a local filter kernel. The preprocessing, for example convolution with a filter kernel, can be carried out using a further AI model that is applied to the spectral data 40 and whose output can be further processed as already described above, for example by a transformer decoder 44', which can have at least one attention block 47, 57 and advantageously a stack of attention blocks 47, 57. The parameters of the preprocessing can be learnable.

The preprocessing may include the use of additional data other than the spectral data. The additional data may include, for example, standards (e.g., measurements on a white standard) and/or calibration measurements. The additional data may be specific to the particular spectral analytical detection device used and may be stored in a non-volatile manner for use in the preprocessing to be performed for the particular spectral analytical detection device.

6 illustrates that the spectrum evaluation 36 can have several AI models. For example, a preprocessing 50, if provided, can be carried out using a first AI model 61, while a second AI model 62 has at least one attention block 47, 57, which in turn has at least one attention mechanism 45, 55. The second AI model 62 can be designed as a transformer encoder and can have any of the embodiments already explained (in particular any of the embodiments described with reference to 2 , 3 , 4 and 5 explained embodiments) or can be designed as a transformer decoder.

In the case of 2 , 3 , 4 , 5 and 6 In the spectrum evaluation 36 explained above, the spectral data are viewed as a one-dimensional vector, which is then further processed using at least one AI model 44, 44', 61, 62. The at least one AI model has at least one attention mechanism. Advantageously, the at least one AI model has a stack of attention blocks, each of which has an attention mechanism in order to recognize relations within the spectral data or the input data obtained therefrom by preprocessing and to use them, for example, for a condition analysis of a plant.

Trained parameters of the AI models can be stored non-volatilely in a storage system of the evaluation device or the evaluation system. The trained parameters can vary depending on the implementation of the attention mechanisms. Further features of the training are described with reference to 8 , 12 , 13 and 14 described.

7 is a flowchart of a method 70. The method may be performed by the system 10 according to one embodiment.

At 71, a wavelength-resolved detection of at least one optical measurement variable is carried out on a sample, which can be a eukaryote, a part of a eukaryote or a food product. The detection can in particular comprise a detection of an optical measurement variable on a plant or a part of a plant. The detection can also comprise a detection of an optical measurement variable on a plant-like organism (for example an alga). The detection can be carried out in a reflection arrangement or in a transflective arrangement or in a transmission arrangement.

At 72, the spectral data recorded at 71 are evaluated. The evaluation can include a condition analysis of the plant based on the recorded spectral data. During the evaluation, the spectral data are processed using an analysis logic that has at least one attention mechanism.

The analysis logic can be set up in such a way that it provides a result of a quantitative state analysis at its output. The at least one attention mechanism is used to recognize relations within the spectral data and to use them for the quantitative state analysis. The quantitative state analysis can have a quantitative determination of at least one nutrient concentration. A result of the quantitative determination can represent the nutrient concentration(s) on a continuous or ordinal value scale.

At 73, a result of the evaluation is provided. The result can be output via a data interface (for example, for executing a control function) or via a human-machine interface.

The methods, evaluation devices, evaluation systems and systems according to embodiments can be set up such that several different trained AI models are provided and used. The several different trained AI models can be assigned to one, several or all of the following: a plant variety, a plant species, at least one plant stage, at least one leaf stage, a position of the leaf within the plant, a region (for example, the USA or Europe), a type of cultivation (for example, outdoors or in a building), a type of cultivation substrate (for example, soil-based or soilless such as coconut substrate or hydroponics).

The methods, evaluation devices, evaluation systems and systems can be designed such that a selection of the trained AI model to be used is made depending on a user input that specifies one, several or all of the following: a plant variety, a plant species, a plant stage, a leaf stage, a position of the leaf within the plant, a region, a cultivation type, a cultivation substrate type. The methods, evaluation devices, evaluation systems and systems can have a control of a human-machine interface to enable an input that specifies the plant variety, the plant species, the plant stage, the leaf stage, the position of the leaf within the plant, the region, the cultivation type and/or the cultivation substrate type. The methods, evaluation devices, evaluation systems and systems can be set up such that the user input can be made via a human-machine interface on the device. The methods, evaluation devices, evaluation systems and systems can be set up such that the user input can also be received via an interface from a storage system. The latter is useful, for example, when user input is not made in the field environment in which data collection is performed.

The methods, evaluation devices, evaluation systems and systems can be designed such that a selection of the trained AI model to be used is made depending on an image evaluation of an image that was captured from the sample to be analyzed (for example a plant or another eukaryote) and that is analyzed by the evaluation circuit in order to automatically determine the plant variety, the plant species, the plant stage and/or the leaf stage and, based on this, to retrieve and use the AI model to be used to analyze the spectral data.

8 is a flow chart of a method 80 with which the evaluation circuit 34 can be configured for the evaluation of the spectral data.

At 81, training data is generated. The generation of training data can include the acquisition of training spectra. The training spectra are acquired using a spectral analytical acquisition device that can be identical to the spectral analytical acquisition device used to acquire the spectral data to be analyzed. The training spectra or data derived from them (for example by convolution with a filter or by preprocessing with a first AI model) can be annotated.

The generation of the annotations that are part of the training data can, for example, involve determining nutrient concentrations. The determination of the nutrient concentrations can be done in a laboratory (for example, using chemical and/or physical methods) or expert-based. The training spectra can be annotated with the nutrient concentrations. This can also be done implicitly by grouping training spectra into groups with similar nutrient concentrations and then annotating the groups.

Depending on the desired application, the annotations can also be generated in such a way that the training spectra are provided with annotations that are different from a nutrient concentration.

The training data may comprise spectra of at least 100 plants of different plant and/or leaf stages. The training data may comprise spectra of at least 1000 leaves of different plant and/or leaf stages. As shown by 9 until 11 As explained, good results are achieved by using a sufficiently large number of training spectra.

At 82, an analysis logic is trained. This may include training with the training data. A portion of the training data may be used for validation and another portion of the training data may be used for testing. The training may include techniques known to those skilled in the art, such as gradient-based optimization methods, e.g. gradient descent. The training optimizes an objective function (e.g. an L2 loss) over a training set (typically by minimizing the objective function). The training may be terminated on a validation set (determined for the respective training stage) depending on the value of the objective function.

The training at 82 comprises training at least one attention block having an attention mechanism. During the training of the at least one attention block, parameters of a processing of tokens can be determined, for example parameters of a mapping that determine how K, Q and V depend on the input data.

At 83, the trained analysis logic is stored in a storage system for use by the evaluation circuit 34 in evaluating the spectral data. The evaluation system or device can use the trained analysis logic to perform a state analysis of a eukaryote based on the spectral data.

9 , 10 and 11 show results of the inventive technique in the quantitative status analysis (in this specific case in the quantitative determination of nutrient concentration) for different nutrients, namely nitrogen (N) and phosphorus (P). Bars 91, 93, 95 show the results for an inventive technique. Bars 92, 94, 96 show the results for a conventional technique in which a PLS (“partial least squares”) method was applied to spectral data. In 9 is an R2 measure of agreement, i.e. a coefficient of determination where 100% means perfect agreement and 0% means no agreement. In 10 and 11 A mean absolute error (MAE) of agreement is shown, i.e. a deviation measure where 0 means perfect agreement and larger values represent poorer agreement. The comparison was made using laboratory-determined nutrient concentrations. The uncertainties are also shown as error bars at the top of the result bars.

As can be seen, the nutrient concentrations determined using the techniques of the invention are better than the results obtained using conventional techniques in all cases presented.

In the 9 , 10 and 11 The nutrients N and P shown here are particularly important nutrients in agricultural technology.

Below, details of the training of the analysis logic, the design of the analysis logic and its use for the 9 , 10 and 11 described results.

collection of training data

Creating a dataset: Investigations were carried out with 587 groups in strawberry plants, whereby for each group several individual measurements were recorded spectrally from leaves of several plants in the immediate vicinity, so that a total of 17686 individual measurements in the data set. The leaves were removed and analyzed in the laboratory to determine nutrient concentrations. These laboratory values serve as target values for training.

Filtering of the dataset: Training data of poor quality were excluded. The filtering related to both the quality of the spectra and the quality of the target values (i.e. nutrient concentrations).

Division into training, validation and test data: The data was divided into training data (70%), validation data (15%) and test data (15%). The proportion of validation data is optional, but it is helpful for finding good hyperparameters of the AI model and training procedure.

Cross-folds: The division into training, validation and test data sets was carried out multiple times (6 times) into so-called folds. This allows statistically more reliable statements to be obtained.

machine learning model

Two different machine learning models were trained and tested. In a first preferred embodiment, the spectral data is divided into several spectral ranges (i.e., converted into tokens), processed linearly ("embedded"), the processed tokens are position-encoded, and then processed in a transformer encoder that has a sequence of attention blocks with a (self-)attention mechanism. In a second embodiment, the processing by the sequence of attention blocks takes place downstream of processing by a deep learning (DL) architecture.

• - Schichtnormierung („layernorm“)
• - Multi-head self-attention (mit „scaled dot-product attention“ und mit linearen Projektionen für das Prozessieren der Eingaben, um Q, K, V zu erhalten)
• - Feed Forward Netz bestehend aus Schichtnormierung, linearem NN („linear neural network“, in der Technik auch als „Linear Model“ bezeichnet), GELU („Gaussian Error linear unit“) als nicht-lineare Aktivierungsfunktion, linearem NN

Specifically, the transformer encoder has a sequence of attention blocks that can be implemented as follows:

• - layer standardization ("layernorm")
• - Multi-head self-attention (with scaled dot-product attention and with linear projections for processing the inputs to obtain Q, K, V)
• - Feed forward network consisting of layer normalization, linear NN (“linear neural network”, also known in technology as “linear model”), GELU (“Gaussian Error linear unit”) as non-linear activation function, linear NN

The above-mentioned attention block is executed several times in succession.

• - Mittelung der transformierten Token,
• - Schichtnormierung,
• - lineares NN.

The mapping to the target value (block 49 in 3 and 4 ) was realized by

• - Averaging of the transformed tokens,
• - layer standardization,
• - linear NN.

• - Convolution
• - Dropout (im Training)
• - Rectified Linear Unit (ReLU) als nicht-lineare Aktivierungsfunktion
• - Convolution
• - Dropout (im Training)
• - Rectified Linear Unit (ReLU) als nicht-lineare Aktivierungsfunktion

wobei diese Abfolge mehrfach durchlaufen wurde.In the second variant of the machine learning model, in which the processing is carried out by the sequence of attention blocks of a DL architecture, the DL architecture was implemented as follows:

• - Convolution
• - Dropout (in training)
• - Rectified Linear Unit (ReLU) as non-linear activation function
• - Convolution
• - Dropout (in training)
• - Rectified Linear Unit (ReLU) as non-linear activation function

This sequence was repeated several times.

Both variants have hyperparameters that include model hyperparameters and training hyperparameters. These were first optimized separately for each fold on the respective validation set using hyperparameter optimization. Afterwards, a common hyperparameter set for all folds was formed from these fold-specific hyperparameters, with each value in it roughly corresponding to the median of the fold-specific parameters. It should be noted that these parameter sets differ between the predictions of different nutrients.

(A) Using the spectral data to generate the input of the transformer encoder

Such a design is in 2 , 3 and 4 The spectral data are understood as a sequence of several spectral ranges and these spectral ranges (i.e. parts of the spectral data) are analyzed together with attention mechanisms or attention blocks.

The spectral data was divided into several vectors of equal length. These are also referred to here as parts of the spectral data or partial spectra. In technology, the term "patches" or "tokens" is also common. Optionally, the spectral data was transformed to a predetermined length before being divided, for example by cutting off the last and/or first entries or by adding additional entries at the beginning or end, for example by periodic continuation.

These parts of the spectral data (i.e. the multiple vectors of the same length) serve directly as input for a transformer encoder. There they are first processed (“embedded”). In order not to lose the information about the position of a part of the spectrum relative to the overall spectrum, position coding is used. The tokens processed and position-coded in this way are transformed according to the definition of the attention mechanism, whereby the resulting coding (which in technology is also referred to as “transformed tokens” or “transformed inputs”) of each individual partial spectrum can simultaneously encode relevant information about all other partial spectra. The local and global context of the spectral data is thus taken into account in the evaluation.

The final mapping to a target value (in this case to a nutrient content), as described in 2 , 3 and 4 in the mapping operation 49 can be achieved, for example, by a linear mapping of the coding of the first partial spectrum to precisely that target value. Other embodiments are possible, for example by averaging all token codings followed by a linear mapping.

• - Da das Spektrum aus 636 Einzelwerten besteht, wird dieses zunächst in drei Vektoren der Länge 212 geteilt, welche jeweils als ein Teilspektrum der Eingabe dienen. Jedes Token ist, wie bereits erläutert, prozessiert („embedded“) und positionskodiert.
• - Die Tiefe des Transformers wird auf 5 gesetzt, sodass die Teilspektren jeweils 5-fach durch je einen Attention-Block, der einen Multi-Head Attention-Mechanismus aufweist, transformiert werden (wobei die Attention-Blöcke zusätzliche Operationen aufweisen, wie oben erläutert). Die interne Dimensionalität betrug 1024.
• - Bei der Analyse mit einem Attention-Block, der einen Multi-Head Attention-Mechanismus aufweist, wird jedes Spektrum gleichzeitig mehrfach parallel betrachtet, wobei die Ausgaben konkateniert werden und in jedem Block als Eingabe eines MLP mit versteckter Dimension 512 dienen.
• - Die erhaltenen Kodierungen der parallel verarbeiteten Teilspektren werden am Schluss gemittelt, normiert und der dadurch entstehende Vektor dient als Eingabe eines linearen Regressors, welcher die Ausgabe auf einen konkreten Nährstoffkonzentrationswert abbildet.

As a concrete design, which is for the 9 , 10 and 11 generated data, several multi-head attention mechanisms were executed one after the other. Specifically, this was implemented as follows:

• - Since the spectrum consists of 636 individual values, it is first divided into three vectors of length 212, each of which serves as a sub-spectrum of the input. As already explained, each token is processed ("embedded") and position-coded.
• - The depth of the transformer is set to 5, so that the subspectra are each transformed 5-fold by an attention block that has a multi-head attention mechanism (where the attention blocks have additional operations, as explained above). The internal dimensionality was 1024.
• - When analyzing with an attention block that has a multi-head attention mechanism, each spectrum is considered multiple times in parallel, with the outputs being concatenated and serving as input to an MLP with hidden dimension 512 in each block.
• - The obtained codings of the parallel processed partial spectra are finally averaged, normalized and the resulting vector serves as input of a linear regressor, which maps the output to a specific nutrient concentration value.

(B) Using a transformer to process data generated from the spectral data by another AI model

Here, at least one attention block is applied to output data of another Kl model, which may, for example, have a DL model. The at least one attention block can be part of a transformer decoder.

The spectral data are first convolved with one or more filters in order to detect essential, primarily locally observable features. The feature space can be extended by one dimension by applying a filter bank in order to obtain a plurality of convolution results for a 1D spectrum.

The results of the convolutions can then serve as input for an attention mechanism, which finally maps them to a target value.

• - Zuerst wird ein faltungsbasierter Encoder (hier: CNN=„convolutional neural network“) durchlaufen. Dieser besteht aus mehreren Ebenen, die wiederum durch eine Abfolge von Faltungs-, Dropout- und Nichtlinearitätsschichten (z.B. ReLU = „rectified linear unit“) gebildet wird. Optional kann die Dimensionalität mit Poolingschichten verkleinert werden.
• - Nach dem Encoder wird ein Transformer-basierter Decoder durchlaufen. Dieser weist wenigstens einen Attention-Mechanismus auf. Im konkreten Fall war der Decoder aus genau einer Decoder-Schicht aufgebaut, die wie folgt zusammengesetzt ist:
  • ◯ Multi-Head Self-attention
  • ◯ Normierung
  • ◯ Multi-Head- Cross-Attention mit gelerntem Query
  • ◯ Normierung
  • ◯ Feed Forward Netz bestehend aus linearem NN, ReLU, DropOut im Training, linearem NN, DropOut im Training
  • ◯ Normierung
• - Der dadurch entstehende Vektor dient als Eingabe eines linearen Regressors, welcher die Ausgabe auf einen konkreten Nährstoffkonzentrationswert abbildet.

As a concrete design, this can be implemented, for example, as follows:

• - First, a convolution-based encoder (here: CNN = "convolutional neural network") is run through. This consists of several levels, which in turn are formed by a sequence of convolution, dropout and non-linearity layers (e.g. ReLU = "rectified linear unit"). Optionally, the dimensionality can be reduced using pooling layers.
• - After the encoder, a transformer-based decoder is run through. This has at least one attention mechanism. In this specific case, the decoder was made up of exactly one decoder layer, which is composed as follows:
  • ◯ Multi-Head Self-attention
  • ◯ Standardization
  • ◯ Multi-Head Cross-Attention with learned query
  • ◯ Standardization
  • ◯ Feed forward network consisting of linear NN, ReLU, DropOut in training, linear NN, DropOut in training
  • ◯ Standardization
• - The resulting vector serves as input of a linear regressor, which maps the output to a specific nutrient concentration value.

The use of such a decoder can be particularly useful for the simultaneous quantitative determination of the nutrient concentration of several nutrients.

Training the machine learning model

The machine learning model was trained by optimizing an objective function that minimizes a deviation (e.g., the mean square deviation (i.e., an L2 loss)) between the laboratory target values and the respective baseline value of the machine learning model over a training set. The progress of the training was monitored on a validation set.

The training was carried out on a training set from the groups of training spectra determined as described above.

application phase of the trained machine learning model

Newly recorded spectra from strawberry plant leaves were evaluated using the trained machine learning model. The leaves on which the newly recorded spectra were recorded were examined in the laboratory to determine the R2 and MAE values of 9 until 11 to determine.

The results of the transformer-based architecture are superior to those of PLS for all nutrients investigated. Transformer-based architectures represent an attractive alternative to existing chemometric models for the analysis of spectral data.

12 shows the procedure for providing the trained analysis logic. In a particularly simple case, a set of training spectra is recorded using a device 20' that has the same configuration of a spectral analytical recording device as the device 20. The training spectra can be recorded, for example, on plants 101 with different nutrient supplies. The method can include a targeted induction of different nutrient supplies by applying different fertilizers or by planting the plants 101 in substrates with different nutrient supply levels.

The leaves 102 (or other plant areas) from which the training spectra were recorded can be analyzed. In particular, a chemical, physical, or physico-chemical analysis 103 can be carried out.

The training spectra 104 and the analysis results for the corresponding leaves 102 (or other plant areas) are used by a computer or server system 105 to train the analysis logic. The training can include training at least one attention block, which has at least one attention mechanism, or a Kl model with a stack of attention blocks. Different analysis logics can be trained for different state analyses (for example different nutrients). For the simultaneous analysis of several nutrients, the last attention block can advantageously be executed as a cross-attention block and a query can be trained for each nutrient, and these can be processed together.

Parameters and optionally also hyperparameters of the trained analysis logic can be stored for use by the evaluation circuit 34, for example in the storage system 33 of the evaluation system 30. Spectral data acquired by the device 20 can be analyzed using this analysis logic. A result can be output via a data interface and/or provided via a human-machine interface, for example via a human-machine interface of the device 20.

13 is a flowchart of a method 110 that may be used to implement step 81 of the method 80.

At 111, training spectra are acquired from leaves. The training spectra can be acquired in central leaf areas, at leaf tips, or at other areas of leaves that can be reliably identified in both the training phase and the application phase.

At 112, the leaves are analyzed. This may include determining nutrient concentrations using chemical, physical or physicochemical techniques. Alternatively or additionally, an expert-based analysis may be performed.

At 113, the training spectra can be grouped based on the nutrient concentrations determined at 112. The different groups can be annotated with the respective target laboratory values from step 112.

14 is a flowchart of a procedure 120 that may be used to implement step 82 of the method 80.

At 121, the training data can optionally be augmented. Augmenting training ning data can be dependent on random parameters.

Training takes place at 122. As mentioned, training can be carried out using techniques known to those skilled in the art, such as gradient-based optimization methods, for example gradient descent. Training can optimize (typically minimize) an objective function (for example an L2 loss) over a training set. During training 122, outliers can optionally be filtered out. The optional filtering out of outliers can also take place before training.

With reference to 15 , 16 , 17 , 18 and 19 exemplary embodiments of the device 20 with the spectral analytical detection device are described. These devices can be used alone or in combination with the evaluation system or the evaluation device 30.

The devices each have a spectral analytical detection device, which can be designed in different ways. The spectral analytical detection device is designed to record spectral data on a eukaryote (in particular a plant), a part of a eukaryote or a food product.

The device 20 can have a storage system 24 for temporarily storing the spectral data. The device 20 can have a data interface 22 to transmit recorded spectral data or data derived therefrom by preprocessing to the evaluation system 30 or the evaluation device 30. The device 20 can have a human-machine interface 23. A result of the evaluation of the spectral data can be output via the human-machine interface 23. This result can either be determined locally in the device 20 or received by it from the evaluation device or the evaluation system 30.

The device 20 has at least one processing circuit 25. The at least one processing circuit 25 can carry out an optional preprocessing 26 of acquired spectral data. The optional preprocessing can, for example, comprise compression and/or removal of low-quality spectra. The at least one processing circuit 25 can execute an interface control 27 in order to store the spectral data acquired by the spectral analysis acquisition device in the storage system 24 and/or to control the data interface 22 for transmitting the spectral data and/or to control the human-machine interface 23 for outputting an evaluation result.

The device 20 of 15 has a spectral analytical detection device which has a spectrometer 21.

The device 20 of 16 has a spectral analytical detection device which has a hyperspectral camera 21'. The spectral data recorded at a specific point on the test object can be transmitted and analyzed as spectral data.

The device 20 of 17 has a spectral analytical detection device which has a controllable multispectral light source 29 and a camera 21'' or another optoelectric sensor. A light source controller 28 can control the multispectral light source 29 so that light with different wavelengths can be emitted sequentially in time and a corresponding optical parameter can be detected with the optoelectric sensor or the camera 21''. Alternatively or additionally, the sample can be illuminated with multispectral mixed light, with detection taking place by a detector with a filter arrangement. In this case, a filter arrangement can be used which only transmits a specific wavelength to the underlying pixel, or a filter arrangement which randomly lets through several light wavelengths in order to increase the sensitivity.

As in 18 As shown, the device 20 can have a positioning system 121. The positioning system 121 can have a satellite-based positioning system (in particular a GNSS system). The positioning system 121 can have an acceleration sensor-based positioning system. Results of a position determined by the positioning system 121 can be stored in association with recorded spectral data. This enables the results of the condition analysis to be associated with geopositions. In this way, it is possible to determine, for example, in which areas of a cultivation area (for example a field) there is an undersupply or oversupply of plants with nutrients. Such an embodiment can also be used if the spectral analytical detection device has a different configuration (for example the one described with reference to 16 or 17 configuration explained).

As in 19 As shown, the device 20 may include a camera 122. The camera 122 may be designed as a near-field camera that is configured to be in direct contact with the device device 20 to capture images on a sheet. The images can be captured, for example, using the method described in the DE patent application 10 2023 113 704.2 by Carl Zeiss AG. In particular, the captured images can be used to determine at which positions on a plant the spectral analytical data acquisition should take place.

In each of the disclosed embodiments, the spectrum evaluation 36 or at least a part of the spectrum evaluation 36 can be carried out locally in the device 20. In this case, it is not necessarily necessary (but still possible) to output the spectral data for evaluation in a computer system separate from the device 20 via a data interface. This is exemplified in 19 but also applies to the other disclosed embodiments of the device 20. The parameters required for the execution of the analysis logic can then be stored in a memory system 24 from which the processing circuit 25 can retrieve them.

The processing circuit may comprise one or more circuits to perform the functions explained. The one or more integrated circuits may comprise, for example, any one or any combination of the following circuits or circuit components: an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a processor, a CPU, a GPU, a TPU, a controller, one or more quantum gates, a quantum information processing circuit, other integrated circuits.

20 shows a flow chart of a method 130. The method 130 can be carried out automatically by the device 20 and/or an evaluation device or an evaluation system 30 coupled to the device 20.

At 131, a near-field image is received. At least one near-field image of a plant section directly adjacent to the device 20 can be received.

At 132, the at least one near-field image is evaluated to identify possible sources of error such as foreign matter or mechanical damage. The evaluation may include determining a suitability assessment that indicates whether plant-immanent or measurement-related sources of error are present. The suitability assessment may quantify the suitability of the at least one image based on the presence or absence of possible sources of error in a binary manner (suitable/unsuitable), on an ordinal suitability scale or on a continuous suitability scale. The evaluation may include determining a pixel-by-pixel or area-by-area segmentation of the at least one image to identify which areas of the at least one image are free of sources of error and are therefore suitable for further analysis based on spectral data.

At 133, a result of the suitability assessment is used for further analysis based on spectral data. This can be done by selectively performing spectral data acquisition depending on the suitability assessment and/or by generating metadata representing the suitability.

21 shows an example of a leaf 3. A leaf section of the leaf 3 is in use of the device 20 in contact with the device 20. The leaf 3 can have areas 141, 142 in which a nutrient concentration determination based on spectral data entails an increased risk of falsification of the analysis results due to foreign substances such as dirt, pollen or liquids or living organisms such as insects. The leaf 3 can have areas 143 in which a nutrient concentration determination based on spectral data entails a risk of falsification of the analysis results due to mechanical damage, necrosis, chlorosis, anthocyanosis or spoiled leaf areas. The suitability assessment detects whether such areas exist and/or whether sufficiently large areas exist that are free of the checked sources of error and are sufficiently large for further analysis based on spectral data.

22 is a schematic representation 150 for further illustrating the processing of the at least one near-field image. The at least one near-field image can be processed to evaluate the suitability of a plant section adjacent to the device 20 for further analysis based on spectral data.

The suitability rating 152 may be determined based on the absence of various possible sources of error. The plant section may be rated as suitable if all checked sources of error are absent. More complex techniques may be used. For example, the plant section may be rated as suitable if a proportion of the area of the plant section surrounded by the contact surface of the device 20 that is free of all checked sources of error is greater than a given threshold.

The suitability assessment 152 can influence the spectrum-based analysis 151. This can be done in various ways, as already explained in detail. For example, the suitability assessment 152 can be used to ensure that a spectrum-based analysis is only carried out on plant sections that are assessed as suitable. Alternatively or additionally, the suitability assessment 152 can be used to mark the result of the spectrum-based analysis (for example as reliable / limited reliability) or to evaluate it (for example to weight it).

• - Fremdstofferkennung 153: Erkennung von Flüssigkeit und/oder festen Fremdstoffen (wie Staub oder Pollen) im Pflanzenabschnitt, der im wenigstens einen Bild abgebildet ist;
• - Nekrosenerkennung 154: Erkennung nekrotischer Stellen im Pflanzenabschnitt, der im wenigstens einen Bild abgebildet ist;
• - Chlorosenerkennung und/oder Anthocyanosenerkennung: Erkennung von Stellen mit Chlorosen oder Abnormitäten im Hinblick auf Anthocyane im Pflanzenabschnitt, der im wenigstens einen Bild abgebildet ist;
• - Erkennung verdorbener Stellen 155 im Pflanzenabschnitt, der im wenigstens einen Bild abgebildet ist;
• - Schädlingserkennung 156: Erkennung von Schädlingen oder Insekten im Pflanzenabschnitt, der im wenigstens einen Bild abgebildet ist;
• - Positionsverifizierung 157: Erkennung einer inkorrekten Positionierung der Vorrichtung an der Pflanze. Hierzu kann beispielsweise anhand einer automatischen Erkennung des Pflanzenabschnitts ermittelt werden, ob die Vorrichtung an einem Pflanzenabschnitt des richtigen Typs (z.B. einem Blatt in einem gewünschten Blattstadium) positioniert ist;
• - Riss/Bruch-Erkennung 158: Erkennung einer mechanischen Beschädigung im Pflanzenabschnitt, der im wenigstens einen Bild abgebildet ist.
• - Pilzerkennung 159: Erkennung einer von Pilzen (beispielsweise Mehltau) befallenen Stelle im Pflanzenabschnitt, der im wenigstens einen Bild abgebildet ist.

To determine the suitability assessment, one, several or all of the following checks can be carried out using at least one near-field image:

• - Foreign matter detection 153: Detection of liquid and/or solid foreign matter (such as dust or pollen) in the plant section depicted in at least one image;
• - Necrosis detection 154: Detection of necrotic areas in the plant section depicted in at least one image;
• - Chlorosis detection and/or anthocyanosis detection: detection of areas of chlorosis or abnormalities with respect to anthocyanins in the plant section depicted in at least one image;
• - detecting spoiled areas 155 in the plant section depicted in at least one image;
• - Pest detection 156: detecting pests or insects in the plant section depicted in at least one image;
• - Position verification 157: Detection of incorrect positioning of the device on the plant. This can be done, for example, by automatically detecting the plant section to determine whether the device is positioned on a plant section of the correct type (eg a leaf at a desired leaf stage);
• - Crack/break detection 158: Detection of mechanical damage in the plant section shown in at least one image.
• - Fungal detection 159: Detection of a location infected by fungi (e.g. mildew) in the plant section shown in at least one image.

The image-based check for suitability for further analysis, the realization of the evaluations carried out for this purpose and/or the use of the result (i.e. suitability) can be carried out by the techniques described in the German patent applications filed on May 25, 2023 10 2023 113 704.2 Carl Zeiss AG. Please refer to this application for further details.

Alternatively or additionally, automatic control or user guidance with regard to the acquisition of spectral data can also be carried out using all techniques described in the German patent applications filed on 25 May 2023 10 2023 113 709.3 Carl Zeiss AG. Reference is also made to this application for further details. An exemplary implementation is described with reference to 23 described.

23 is a schematic representation of a system 10 that has the device 20, the evaluation system 30 and a camera 160. The camera 160 can be provided separately from the device 20 or can be structurally installed in the same housing as the spectral analysis detection device of the device 20. The various system components can communicate with each other via data connections, in particular a communication network 4. The communication network 4 can have a wireless network, a cellular network and/or other wired or wireless communication paths.

In operation, the camera 160 can capture an image of a plant or part of a plant. The evaluation system 30 can evaluate the image to determine whether the plant and/or which parts thereof are suitable for capturing the spectral data. The determination of suitability can be made using the methods described in the German patent applications filed on May 25, 2023 10 2023 113 709.3 Carl Zeiss AG. A measurement recommendation can be provided via a human-machine interface of the device 20 to assist the user in acquiring the spectral data. The measurement recommendation can indicate which plant and/or plant part is particularly suitable for acquiring the spectral data. The measurement recommendation can depend on conditions to be analyzed (for example, nutrients to be analyzed), which, as in the German patent application 10 2023 113 709.3 Carl Zeiss AG can be customized.

In a further embodiment, the system 10 comprises the device 20 and the evaluation system 30, which can be in communication via a communication network or another communication connection, as described under reference to 1 explained. The camera 160 is therefore optional.

24 is a flowchart of a method 170 that may be performed by the system 10.

In 171, an image-based measurement recommendation is generated that indicates where spectral data should be recorded. This can be done as in the German patent application 10 2023 113 709.3 of Carl Zeiss AG.

At 172, a wavelength-resolved detection of at least one optical measurement variable takes place using the device 20.

At 173, the spectral data acquired at 172 are analyzed. At least one attention mechanism is used for this purpose. The analysis of the spectral data can be carried out as already described in detail here.

Below, various additional features are described that can optionally be used to support the technical effects already described.

Augmentation to enlarge the training data set: To enlarge the training data and cover possible variations, the training spectra can be augmented for training.

• - Nachbilden von Variationen, die bei der Fertigung der spektralanalytischen Erfassungseinrichtung erwartbar sind. Hierzu gehört beispielsweise eine Verschiebung von Pixeln des Bildsensors zwischen unterschiedlichen Bildsensoren gleicher Auslegung, wobei die Verschiebung im Bereich von Nanometern oder Sub-Nanometern liegen kann. Die Pixelverschiebung führt zu Verschiebungen des erfassten Spektrums.
• - Nachbilden von erwartbaren Variationen der Lichtquelle (beispielsweise einer Degradierung über die Zeit oder einer temperaturabhängigen Variation von Intensität und spektraler Signatur)
• - Multiplikative und/oder additive Variationen von Amplituden
• - Verschiebung des Spektrums in Richtung kürzerer oder längerer Wellenlängen
• - Hinzufügen von Rauschen auf das Spektrum
• - Nachbilden von Variationen des Abstands zwischen Probe und Sensor
• - Nachbilden von Variationen des Abstands zwischen Probe und Lichtquelle
• - Punktuelles Glätten des Spektrums

The augmentation may take into account or use one, several or all of the following modifications:

• - Reproducing variations that are to be expected during the manufacture of the spectral analysis detection device. This includes, for example, a shift in pixels of the image sensor between different image sensors of the same design, whereby the shift can be in the range of nanometers or sub-nanometers. The pixel shift leads to shifts in the detected spectrum.
• - Reproducing expected variations of the light source (e.g. degradation over time or temperature-dependent variation of intensity and spectral signature)
• - Multiplicative and/or additive variations of amplitudes
• - Shift of the spectrum towards shorter or longer wavelengths
• - Adding noise to the spectrum
• - Simulating variations in the distance between sample and sensor
• - Simulating variations in the distance between sample and light source
• - Selective smoothing of the spectrum

Optimization of hyperparameters during training: Optimization of hyperparameters of the AI model (e.g., a depth of an encoder or decoder) and/or the training procedure (e.g., a learning rate) is beneficial.

Use of specific hardware in the training phase: Training is computationally intensive. Therefore, the use of suitable hardware such as GPUs, TPUs, etc. is advantageous.

Use of specific hardware in the analysis phase: The application of the trained analysis logic can be significantly accelerated with special hardware. Examples of this are GPUs or special integrated circuits that are optimized for executing neural networks or other ML models. Accordingly, it is advantageous if the evaluation circuit 34 has at least one such hardware component that is particularly suitable for executing the AI model or models.

Use of the methods, evaluation systems, evaluation devices and/or systems for various types of condition analyses: In the context of embodiments, reference was made to the determination of nutrient concentrations by evaluating the spectral data, wherein the spectral data is recorded on plant leaves. The techniques disclosed are particularly suitable for this application, since the relationship between spectral data and nutrient concentrations can be complex, particularly in the case of micronutrients. Therefore, non-linear techniques, in particular using at least one attention mechanism, are particularly suitable.

However, the disclosed techniques have numerous other possible applications, such as the investigation of nutrient content in water or other applications of analytical technology. Even in such applications, the use of AI models comprising transformer-based architectures is possible and useful in order to recognize complex relationships.

Further applications include further condition analyses of plants, plant parts or plant-like organisms, which are described below.

Modification of the architecture of the analysis logic: Several specific examples of architectures with at least one attention mechanism have been explained. Further modifications are possible, for example by varying the various model hyper-parameters (number of attention blocks, inner dimension, activation function, etc.). Furthermore, various variants of the implementation of the attention mechanisms and/or their approximation are also possible, for which the expert can obtain further information, for example from the specialist literature cited here.

Output (e.g. visualization) of uncertainties: In addition to the spectrum-based state analysis, for example to determine the nutrient concentration, an (in)certainty of the analysis can also be determined and used. The uncertainties can be determined in various ways and output as qualitative or quantitative uncertainty measures. For example, the uncertainties can be determined based on initial values of classifiers, based on uncertainties determined during training, based on a scatter of results of the spectral data with several trained machine learning models, based on a scatter of the nutrient concentrations determined spectrum-based on different leaves or different plants, or in another way. The uncertainties can be output via a human-machine interface and/or used to automatically control the spectral analytical acquisition device.

The results of the spectrum-based state analysis can be filtered or weighted based on the associated confidence or uncertainty. For example, only those results whose confidence is above a threshold can be used further (for example for visualization). Only the k most certain results can also be output. The threshold or the number k can be user-defined.

Design and mounting of the device 20: The device 20 can be designed as a portable, in particular as a manually held device 20.

25 shows schematically an embodiment of the device 20 as a manually held device having a structure 180 for holding the device 20.

26 shows a further embodiment in which a vehicle 182 according to an embodiment has a device 20 mounted thereon or several devices 20 mounted thereon. Accordingly, the device 20 or the devices 20 can be designed with a holding structure for attachment to an agricultural vehicle.

27 shows a further embodiment in which a robot 184 has the device 20. A base 186 of the robot 184 can be stationary or movable, for example along a rail system. A controller 188 can control one or more actuators for positioning the device 20.

A relative movement between the device 20 and the material to be examined (eukaryotes or food products) can also be achieved by moving the material to be examined relative to the device 20 by a conveyor or another device. The device 20 and in particular the spectral analytical detection device can be mounted in a stationary manner.

28 shows a further embodiment in which a flying object 190 has the device 20. The flying object 190 can be remotely controllable and/or configured for autonomous flight operation.

In yet another embodiment, the device 20 can be designed as a stationary system. The device 20 can be mounted with a stationary holder near a conveyor on which plants or other eukaryotes or food products are transported past the device 20.

Areas of application: The disclosed devices, evaluation devices and evaluation systems, systems and methods can be used for different applications. In particular, the devices, evaluation devices and evaluation systems, systems and methods can be used on crops or cultivated plants. The plants on which the techniques can be used can in particular also include forestry plants (for example trees), hybrids and/or genetically modified plants.

• • Nutzpflanzen
• • Futterpflanzen
• • Faserpflanzen
• • Ölpflanzen
• • Zierpflanzen
• • Industriekulturpflanzen, wie beispielsweise
  • ◯ Ölsaaten
  • ◯ Tabak
  • ◯ Hanf
  • ◯ Hopfen
  • ◯ aromatische, kulinarische und medizinische Pflanzen
  • ◯ Saatgut für krautige Ölpflanzen
  • ◯ Samen für Leinsamen (und folglich Faserflachs)
  • ◯ Energiepflanzen
  • ◯ Pflanzen, die für die Erzeugung von Ausgangsstoffen zur Energieerzeugung erneuerbarer Energie verwendet werden

Examples of such plants can be:

• • Crops
• • fodder plants
• • Fiber plants
• • Oil plants
• • Ornamental plants
• • Industrial crops, such as
  • ◯ Oilseeds
  • ◯ tobacco
  • ◯ Hemp
  • ◯ Hops
  • ◯ aromatic, culinary and medicinal plants
  • ◯ Seeds for herbaceous oil plants
  • ◯ Seeds for linseed (and consequently fiber flax)
  • ◯ Energy crops
  • ◯ Plants used to produce raw materials for the production of renewable energy

• - Nährstoffversorgungsprobleme, beispielsweise
  • o Nährstoffmangel
  • o Toxizität
• - biotischer Stress, beispielsweise aufgrund von
  • o Nematoden
  • o Insekten
  • o Spinnentieren
  • o Pilzen
  • o Bakterien
  • o Viren
• - abiotischer Stress, beispielsweise aufgrund von
  • o Dürre
  • o Übermäßiger Wasserzufuhr und/oder Staunässe
  • o Salzgehalt
  • o temperaturbedingtem Stress (Kälte, Frost, Hitze)
  • o UV-Licht
  • o Metalltoxizität
  • o mechanischer Belastung (z. B. Druck, Quetschen, Pressen).

The analysis of the spectral data may include, but is not limited to, the qualitative and/or quantitative determination of nutrient concentrations. For example, the condition analysis may include analyses with respect to one, several or all of the following plant conditions:

• - Nutrient supply problems, for example
  • o nutrient deficiency
  • o Toxicity
• - biotic stress, for example due to
  • o Nematodes
  • o insects
  • o arachnids
  • o mushrooms
  • o Bacteria
  • o viruses
• - abiotic stress, for example due to
  • o drought
  • o Excessive water supply and/or waterlogging
  • o salinity
  • o temperature-related stress (cold, frost, heat)
  • o UV light
  • o Metal toxicity
  • o mechanical stress (e.g. pressure, squeezing, pressing).

The result of the evaluation of the spectral data can be used to improve conditions for the plants. This can be done by automatically or semi-automatically adjusting artificially added substances (such as approved fertilizers or approved insecticides) and artificially added water. Numerous other applications are possible, for example for monitoring the health of plants with regard to environmental and regulatory issues.

Further areas of application: Even if embodiments have been described in the context of a condition analysis of plants, the devices, evaluation devices and evaluation systems and methods can also be used elsewhere, for example for spectrum-based analysis of an agricultural product.

29 is a flowchart of a method 200 that can be carried out using the device 20 and/or the evaluation device 30 or the evaluation system 30.

In 201, spectral data from a plant or another eukaryote are collected and evaluated. The evaluation involves the use of at least one attention mechanism.

At 202, a result of the evaluation is used to determine measures to improve conditions for the plants. The measures can then be implemented to improve the conditions.

The disclosed methods, devices, evaluation devices, evaluation systems and systems provide various technical effects. In particular, they offer improvements with regard to the recognition of complex non-linear relationships based on spectral data, for example for analyzing the condition of plants. The methods, devices and systems can be used to improve an evaluation of spectral data on crops or cultivated plants.

Compared to techniques in which plant parts must first be separated, then sent to a laboratory and examined there, the techniques disclosed here offer the advantage that the evaluation can be carried out quickly, for example while the user with the device 20 is still on site at the plant.

Embodiments of the invention can be used in particular in connection with plant cultivation in agriculture, which is a central economic or industrial sector of humanity. Global trends such as the increasing population and simultaneously decreasing Availability of agricultural land leads to a constantly growing need for crop yields per area. At the same time, the unsustainable exploitation of soils leads to an increased need for targeted fertilization in order to maintain or even increase previous yields. In addition, a growing awareness of sustainable management and increasing regulatory pressure are creating new requirements for precisely tailored fertilization, in particular to avoid over-fertilization.

Of particular importance for the growth of every crop is the appropriate supply of nitrogen (N), as this is the most important element of chlorophyll and is therefore essential for optimal metabolism. In addition to N, other macronutrients are also relevant, such as phosphorus (P) and potassium (K), which can often be added to the soil together with N as so-called NPK fertilizers, as well as calcium (Ca), magnesium (Mg) and sulfur (S). Mg is the central element in the chlorophyll ring.

In addition, possibly depending on the plant type, other nutrients may also be relevant for optimal growth and yield, but these are often required in much smaller quantities and are therefore also referred to as micronutrients. These include, for example, boron (B), molybdenum (Mo), copper (Cu), manganese (Mn), zinc (Zn), iron (Fe) and chlorine (Cl).

Embodiments of the invention can help to ensure that optimal conditions are created with regard to the supply of nutrients throughout the entire growth phase of a plant. This is the only way to react appropriately to nutrient deficiencies or nutrient oversupply by applying appropriate fertilizers (through a specific fertigation program in the substrate or soil growth or as additional soil and/or leaf fertilization). Embodiments of the invention allow plants to be monitored regularly. Compared to laboratory leaf analysis, the condition analysis can be carried out quickly. This is particularly desirable for fast-growing plants and plant species with rapid fruit ripening (for example strawberries and lettuce). Embodiments of the invention therefore also address the objective that there should not be several days between the measurement and the availability of the evaluation result, as otherwise the measurement result is invalid again and does not allow for any meaningful adjustment of the nutrient supply. Rapid evaluation is also desirable when the environmental factors change quickly (for example: temperature, precipitation, etc.), as otherwise the condition would change significantly between the measurement and the time of analysis. The rapid evaluation is also advantageous in hydroponic cultivation, where rapid adjustment of applied fertilizers is possible.

Embodiments of the invention enable data collection of the spectral data at the intended application site without the need to transmit physical samples. Embodiments of the invention can thus serve to provide a non-destructive, automated and at the same time rapid (almost real-time) solution for detecting deficiencies and/or oversupply of several nutrients in plants.

While embodiments have been described with reference to the figures, modifications can be implemented in further embodiments. For example, the modifications already explained can be used cumulatively or alternatively. The evaluation device 30 does not necessarily have to be provided separately from the device 20, but can also be structurally connected to it or integrated into the device 20.

While embodiments have been described that can be used with crops, the disclosed techniques can also be used in other applications.

The present disclosure also includes embodiments with any combination of features that are mentioned or shown for different embodiments. It also includes individual features in the figures, even if they are shown there in connection with other features and/or are not mentioned above or below. The alternatives of embodiments described in the figures and the description and individual alternatives of their features can also be excluded from the subject matter of the invention or from the disclosed subject matters.

The terms "comprising" and "having" and derivatives thereof indicate a non-exhaustive relationship and do not exclude the presence of other elements or steps. The indefinite article "a" or "an" and derivatives thereof do not exclude the presence of a plurality of the corresponding elements. The functions of several features listed in the claims can be fulfilled by one unit or one step.

A machine-readable instruction code that can be executed by a programmable circuit to carry out methods according to embodiments can be stored on a suitable medium, such as an optical storage medium or a solid-state medium provided with or as part of other hardware. The instruction code may also be distributed in other forms, such as a modulated data signal sequence.

Embodiments of the invention provide improved techniques for evaluating spectral data and are particularly applicable to the condition analysis of plants.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• WO 2019/169434 A1 [0004]
• US 7 804 588 B2 [0004]
• US 11 320 307 B2 [0004]
• DE 10 2018 103 509 B3 [0004]
• US 10 789 427 B2 [0005]
• US 11 494 561 B2 [0005]
• US 2022/0005465 A1 [0005]
• US 2022/0108689 A1 [0005]
• DE 10 2023 113 716.6 [0006]
• DE 10 2023 113 704.2 [0200, 0212]
• DE 10 2023 113 709.3 [0213, 0215, 0218]

Cited non-patent literature

• A. Vaswani et al., “Attention Is All You Need,” 2017, Advances in Neural Information Processing Systems 30 (NIPS 2017), pp. 6000-6010 (https://arxiv.org/abs/1706.03762 [0005, 0107, 0112, 0127]
• A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” International Conference on Learning Representations (IPCLR) 2021 [0005, 0107]

Claims (17)

Method for evaluating spectral data (40), comprising: Receiving the spectral data (40) by at least one evaluation circuit (34), wherein the spectral data (4) comprises spectral data (40) recorded on a eukaryote (2), a part of a eukaryote (3) or a food product; Analyzing the spectral data (40) by the at least one evaluation circuit (34), comprising: Dividing the spectral data or data generated therefrom by preprocessing into several parts, Analyzing the several parts using an analysis logic (44), wherein an input of the analysis logic (44) receives the several parts, and wherein an output of the analysis logic (44) provides codings of the several parts, wherein the analysis logic has at least one attention mechanism (45, 55), and Determining at least one evaluation result based on the codings. procedure according to claim 1 , wherein the at least one evaluation result comprises at least one quantitative analysis result. procedure according to claim 2 , wherein the at least one quantitative analysis result has a continuously variable value from a range of values. procedure according to claim 2 or claim 3 , wherein the at least one quantitative analysis result has a value selected from an ordinal scale. Method according to one of the Claims 2 until 4 , wherein the at least one quantitative analysis result comprises a concentration of at least one chemical element or at least one chemical compound. Method according to one of the Claims 2 until 5 , wherein the at least one evaluation result comprises at least one nutrient concentration. Method according to one of the preceding claims, wherein the analysis logic comprises: an encoder (44) comprising a stack of a plurality of attention blocks, wherein the plurality of attention blocks each comprise an attention mechanism (45, 55). Method according to one of the preceding claims, wherein the at least one attention mechanism (45, 55) comprises at least one multi-head attention mechanism (55). Method according to one of the preceding claims, further comprising a wavelength-dependent position coding (43). Method according to one of the preceding claims, wherein the at least one attention mechanism (45, 55) has at least one learnable parameter. procedure according to claim 10 , wherein the at least one learnable parameter defines a processing of an input of the respective attention mechanism (45, 55). Method according to one of the preceding claims, further comprising preprocessing (50, 61) preceding the analysis logic. procedure according to claim 12 , where the preprocessing is carried out using another artificial intelligence, Kl, model (61). Method according to one of the preceding claims, further comprising training the analysis logic or at least a part of the analysis logic using training data, the training data comprising: a plurality of spectra (104) and annotations associated therewith or a plurality of data derived from spectra (104) and annotations associated therewith. Method according to one of the preceding claims, wherein the spectral data (40) comprise spectral data (40) recorded on a plant (2), a plant part (3), a plant-like eukaryote or a plant or animal food. Evaluation system or evaluation device for evaluating spectral data (40), comprising: an interface (31) for receiving the spectral data (40), wherein the spectral data (40) comprises spectral data (40) recorded on a eukaryote (2), a part (3) of a eukaryote or a food product; at least one evaluation circuit (34) which is set up to carry out at least the following processing steps for analyzing the spectral data (40): dividing the spectral data (40) or data generated therefrom by preprocessing into several parts, analyzing the several parts using an analysis logic (44), wherein an input of the analysis logic (44) receives the several parts, and wherein an output of the analysis logic (44) provides codings of the several parts, wherein the analysis logic (44) has at least one attention mechanism (45, 55), and Determining at least one evaluation result based on the codings. System (1) or device (20), comprising: the evaluation system (30) or the evaluation device according to claim 16 and a spectral analytical detection device (21; 21';21'', 29) for detecting the spectral data (40).

Images