JPH0520426A - Fruit hue judgement device using neural network - Google Patents

Abstract

PURPOSE:To provide a judgment device for selecting the quality of fruit, in which a processing is speedily executed and whose workability is superior. CONSTITUTION:An image pick-up camera 2 for color is provided facing the moving line of a palette on which fruit is loaded and which moves along a fruit-selection line. A detector 3 is provided facing the moving line of the palette to detect that the palette enters in an image pickup range. When a picture processing part 44 receives a detection signal from the detector 3, red, blue and green picture signals from the image pickup camera 2 are fetched and the picture signals are converted into picture signals showing H(color), S(chroma) and I(lightness). Unnecessary picture elements are removed from the converted picture signals and a previously decided statistical amount is operated. A neural network judgement means 45 executes the algorithm of the neural network based on the statistical amount and outputs information on class based on a hue.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement of a fruit color judging device for automatically judging which grade a fruit belongs to based on the color of the fruit.

[0002]

2. Description of the Related Art Heretofore, a worker has visually determined the color of fruits such as apples. But,
Judgments that depend on human senses often vary due to individual differences and environmental differences, and there is a considerable problem with their reliability. As a technique for automatically determining the hue of the surface of a fruit in order to solve such a problem, there is a hue determining device of Japanese Patent Application No. 62-32656 for apples. This apparatus is composed of an image pickup means and a color matching judgment processing means. The image capturing means captures images of fruits having different colors. Further, the hue determination processing means divides the image captured by the imaging seed stage into pixels, obtains the absolute value or ratio of the hue components of the red component, the green component, and the blue component of each pixel, and obtains each pixel. It has a function of evaluating the hue and determining the hue evaluation value of the entire image from the evaluation. The hue of the whole fruit is determined by obtaining the evaluation value of the whole fruit from the evaluation value of each hue.

[0003]

However, in the case of this device, when the same fruit such as an apple is used, but the varieties are different, an absolute value or a threshold for obtaining a ratio or a hue is used. There is a problem in workability because the threshold value for judgment must be manually set again. An object of the present invention is to provide a determination device for fruit quality selection, which uses a neural network to quickly perform a color matching determination process and has excellent workability.

[0004]

A fruit color judging apparatus using a neural network according to the present invention is a pallet for mounting fruits and moving along a selection line, and a color provided facing a movement path of the pallets. Image pickup camera, a detector provided facing the movement path of the pallet to detect that the pallet has entered the image pickup range, and red and blue from the image pickup camera when a detection signal from the detector is received. , Green image signals are taken in and these image signals are set to H (chromaticity), S
(Saturation) and I (brightness) are converted to image signals, and unnecessary pixels are removed from each of the converted image signals to calculate a predetermined statistical amount. And a neural network judgment means for executing a neural network algorithm and outputting grade discrimination information based on a color.

The neural network judging means obtains a statistic corresponding to the grade from the image processing unit, which is obtained by imaging a plurality of kinds of fruits representing each grade of fruit in advance, as data for each grade. It is a probabilistic neural network that performs learning operation so that it is set in an area.

[0006]

In the fruit color judging device of the present invention, the learning operation is performed before the grade judging work. This learning operation causes image processing and neural network processing to be performed using images obtained by picking up fruits of known grades in advance. The grade data indicating whether it belongs is given as teacher data. The neural network determination means sets a statistic corresponding to the grade from the image processing unit in the data area for each grade. When the learning operation is completed, the actual grade discrimination work is started. This time,
Upon receiving the statistic amount from the image processing unit, the neural network determination means performs a determination processing operation of which grade the statistic amount belongs to.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a schematic structure of a color matching determination device according to the present invention. The pallet 1 on which the fruits are placed is connected to the front and rear and is appropriately driven by a known drive mechanism (for example, a conveyor chain) to move at a predetermined speed along the fruit selection line. An imaging camera 2 such as an ITV provided with an illumination light 5 is directed from one side with respect to the moving path of the pallets 1. Further, on the other side of the moving path of the pallets 1, a detector 3 for detecting that each pallet 1 has entered the image pickup range is provided. The imaging camera 2 and the detector 3 are connected to a processing unit 4, and when the processing unit 4 receives a detection signal from the detector 3, the processing unit 4 determines the hue of fruit based on the image information from the imaging camera 2. And determine the grade.

FIG. 2 shows an example of a concrete configuration of the processing unit 4. This processing unit 4 is a red from the imaging camera 2,
Corresponding to the image information of the green and blue components, it has three A / D converters 41R, 41G, 41B in parallel, and these A / D converters receive the analog image information from the imaging camera 2. Convert to digital image information. These A / D converters 41R, 41G, and 41B respectively have an image memory 42.
The image memories 42R, 42G, and 42B are connected to R, 42G, and 42B, and latch and store corresponding image information. An image processing device 44 is connected to the output side of each image memory.

The image processing device 44 is composed of, for example, a microcomputer, and the image information from the image memories 42R, 42G, and 42B is converted into three elements of color, H (chromaticity) and S (color). Degree) and I (brightness) image signals, and calculates and outputs a statistic necessary for determining the hue of fruit based on these converted image signals, for example, a median value or an average value. . Here, as is well known,
The chromaticity H represents basic color attributes such as red and yellow, the saturation S represents the vividness of colors such as dull and vivid, and the lightness I represents the brightness of colors such as bright and dark. Chromaticity H, saturation S,
The lightness I is represented by the following formula 1, formula 2, and formula 3, respectively.

[0010]

[Equation 1]

[0011]

[Equation 2]

[0012]

[Equation 3]

It should be noted that R, G, and B each represent the level of the converted image signal, and F in Expression 3 is expressed by Expression 4 below.

[0014]

[Equation 4]

The detector 3 is connected to the image processing device 44 via a gate circuit 43. The gate circuit 43 outputs an image pickup timing signal based on the detection signal from the detector 3, and the image pickup timing of the image pickup camera 2, the image memory 42R,
The timing of capturing image information of 42G and 42B is controlled.

A neural network determination unit 45 is connected to the output side of the image processing device 44. The neural network determination unit 45 determines the color of fruit based on the statistics from the image processing device 44 to determine the grade, and is realized by a probabilistic neural network (PNN) circuit. The neural network judgment unit 45 is provided with a keyboard 46 for giving grade information as teacher data to the neural network judgment unit 45 at the time of a learning operation described later.

Next, the learning operation and the judgment operation by the color judgment device having the above-mentioned structure will be described by taking the case of using apples as fruits as an example. Learning operation 1. First, as sample data, several to several tens of apples with typical colors are prepared for each grade. Then, the apple is placed on the pallet 1 on the selection line, and the grade of the apple is input by the keyboard 46. This input information is sent to the neural network determination unit 45. 2. When the pallet 1 on which apples are placed moves along the selection line and reaches the imaging range of the imaging camera 2, the detector 3
Detects this and sends the detection signal to the image processing device 44. 3. At the same time, the imaging camera 2 images the apple,
Image information of green and blue components is converted into A / D converters 41R and 41R.
Image memory 42R, 42 via G, 41B
Send to G, 42B. Each image memory latches and stores these image information.

4. The image processing device 44 performs the following processing based on these image information. First, image memory 42
The image information of the red, green, and blue components for each pixel read from R, 42G, and 42B is converted into an image of chromaticity H, saturation S, and lightness I. This conversion process is shown in FIG. Next, unnecessary pixels are removed from these converted images. Since the color of the background in the image, the brightness of the light source, and the hue of the unnecessary pixel such as the vine part of the fruit are known in advance,
It is determined whether or not each pixel of the image is an unnecessary pixel by using the values of the chromaticity H, the saturation S, and the lightness I obtained in 3 to 3, and those determined as the unnecessary pixels are removed from the image. . The area remaining after the unnecessary pixel removal processing is the hue determination area. Further, the statistical amount in the determination area is calculated using the converted image of chromaticity H, saturation S, and lightness I from which the unnecessary pixels are removed. The calculated three types of statistics are stored in the neural network determination unit 4
Sent to 5.

5. Neural network determination unit 45
Sets these three types of statistics in the data area of the grade input from the keyboard 46. 6. The above processes 1 to 5 are performed for all apples prepared. It should be noted that when the type of fruit has changed, or when the variety of the same fruit has changed, it is possible to perform the determination operation for the new fruit simply by performing the above learning operation.

Judgment operation 1. First, put apples on pallet 1 on the sorting line,
Activate the sorting line. 2. When the pallet 1 on which apples are placed moves along the selection line and reaches the imaging range of the camera 2, the detector 3 detects this and sends the detection signal to the image processing device 44. 3. At the same time, the imaging camera 2 images the apple,
Image information of green and blue components is converted into A / D converters 41R and 41R.
Image memory 42R, 42 via G, 41B
Send to G, 42B. The image memory latches and stores these pieces of image information. The data amount of the image information is, for example, an image composed of 256 × 256 pixels,
Assuming that the red, green and blue components are represented by 8 bits per pixel, the total data amount is about 192 kilobytes.

4. Based on these image information, the image processing device 44 firstly outputs the image memories 42R, 42G, 42B.
The image information of the red, green, and blue components for each scanning line read from is converted into an image of chromaticity H, saturation S, and lightness I. Next, unnecessary pixels are removed from these converted images. Further, the statistical amount in the determination area is calculated using the converted image of chromaticity H, saturation S, and lightness I from which the unnecessary pixels are removed. The three types of calculated statistics are stored in the neural network determination unit 45.
Sent to. 5. The learned neural network determination unit 45 is
Using the input statistics as input data, the hue of the apple imaged is determined to determine the grade. The determination result is displayed visually or audibly by a known method as appropriate.

Here, the neural network will be briefly described with reference to FIG. The neural network used in this embodiment is for pattern classification, and is a so-called PNN (Probabilistic Ne).
ural Network) type,
Input layer IS (Input Slab), pattern layer PS
(Pattern Slab), summation layer SS (Summ
ationSlab) and output layer WS (Win Sl)
ab) and. A node in each layer is called a process element (PE), and when the type of pattern to be classified is N, there are N process elements in the summation layer SS.

An unknown pattern having a certain input vector is given to the input layer IS. Each process element of the pattern layer PS has a weight vector ω _kj (j is 1 to p).
Of the input vector and p is the dimension of the input vector), and at the time of operation, the Euclidean distance d _k between the input vector and the weight vector is calculated based on the following Equation 5.

[0024]

[Equation 5]

In the pattern layer PS, the Euclidean distance d _k calculated by the equation 5 is also used to calculate g by the following equation 6.
_k is calculated and output to the summation layer SS.

[0026]

[Equation 6]

However, σ is a smoothing coefficient. This g _k is
It is the value of the Gaussian distribution of d _k when ω _k is centered. Next, in the summation layer SS, the following Equation 7 is calculated using g _k .

[0028]

[Equation 7]

However, mc is the number of processing elements of the pattern layer PS connected to each processing element of the summation layer SS. In the output layer WS, the class that maximizes the value V obtained by the following formula is selected and used as the determination result. _{V = h c P c (f} c (d k)) where, h _c and P _c are adjustment parameters for each class.

The image processing device 44 and the neural network judging section 45 may be realized by a computer.

[0031]

As described above, according to the present invention, since H, S and I images are used, unnecessary pixels can be easily removed. Further, when the type or variety of the fruit to be judged is changed, this can be dealt with by performing only one learning operation, so the workability is greatly improved. Furthermore, PN
Since the N circuit is used, it is possible to determine even when the statistical value of the fruit for each grade has a distribution that cannot be linearly separated. Moreover, unlike a back-propagation type neural network, learning does not take time, so that the determination operation can be performed immediately even when sample data is added or deleted.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing a schematic configuration of a fruit color determination device according to the present invention.

FIG. 2 is a block diagram showing an example of a processing device used in the color matching determination device of FIG.

FIG. 3 is a diagram showing an example of a fruit image for explaining an evaluation value calculation method by the device of the present invention.

FIG. 4 is a diagram showing an example of a neural network used in the present invention.

[Explanation of symbols]

1 pallet 2 imaging camera 3 detectors 4 processing units 41R, 41G, 41B A / D converter 42R, 42G, 42B image memory 43 gate circuit 44 Image processing device 45 Neural network determination unit 46 keyboard

Claims (2)

[Claims] 1. A pallet which carries fruits and moves along a selection line, a color image pickup camera which faces the moving path of the pallet, and a pallet which faces the moving path of the pallet and picks up an image of the pallet. When a detector for detecting that the image has entered the range and a detection signal from the detector are received, red, blue, and green image signals from the imaging camera are taken in and these image signals are set to H (chromaticity), An image processing unit that converts into image signals representing S (saturation) and I (brightness), removes unnecessary pixels from each of the converted image signals, and calculates a predetermined statistic; A fruit color judgment device using a neural network, comprising: a neural network judgment means for executing a neural network algorithm based on the above to output grade judgment information based on a color. 2. The fruit color determination device according to claim 1, wherein the neural network determination means obtains the plurality of types of fruits representing each grade of the fruit in advance by the image processing unit. Is a probabilistic neural network that performs a learning operation so as to set a statistic corresponding to the above in the data area for each grade, and a fruit color determination device using a neural network.