JP2019133311A - Image processing sensor and image processing method and computer program - Google Patents

Abstract

To provide an image processing sensor, an image processing method executable by the image processing sensor, and a computer program that can conduct quality determination with high accuracy when a character with low contrast was scanned.SOLUTION: A method has an imaging means imaging an inspection object, an image procession means conducting image procession of an image of the inspection object imaged by the imaging means, a quality determination means conducting quality determination of the inspection object based on a processing result of the image procession means, and an output means outputting a determination result of the quality determination means to outside. The image procession means identifies a rectangular area containing a character or a character string that is a recognition object included in the image of the imaged inspection object, calculates an evaluation value indicating a shade degree in the rectangular area as well as recognizing a character in the identified rectangular area and evaluates the shade degree based on the calculated evaluation value and a prescribed threshold. The quality determination of the inspection object is conducted based on the recognized character and evaluated shade degree.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an image processing sensor capable of performing pass / fail determination with high accuracy even when a character with low contrast is read, an image processing method and a computer program that can be executed by the image processing sensor.

  When executing an appearance inspection of an inspection object, an image of the surface of the inspection object flowing on the production line is taken, and the presence / absence of a displacement, the presence / absence of a defect, etc. are inspected based on the taken image. For example, OCR inspection is performed in which characters imprinted on the surface of the inspection object are read using ink, laser, or the like, and the quality of the inspection object is determined.

  For example, in Patent Document 1, when a low-contrast character such as a thin character on a white background is read, even a character engraved under adverse conditions can be read with high accuracy by performing contrast expansion processing or the like. An imaging device that can be used for OCR processing is disclosed. By expanding the contrast, it is possible to improve the recognition accuracy of information such as characters stamped on the surface of an object for which it is difficult to obtain the contrast.

JP 2013-134176 A

  In the OCR processing in the imaging apparatus disclosed in Patent Document 1, even a character with a very low contrast is relieved and the character is successfully read. However, the user cannot tolerate character fading due to insufficient ejection of ink for printing or insufficient output of laser light. In spite of such a printing state, when the character reading is successful, all are determined to be non-defective products and are not determined to be defective products.

  There are also devices that perform pass / fail determination by outputting character quality as a degree of coincidence (stability, recognition level, etc.) and setting determination conditions, but the user cannot adjust the evaluation value used. As a result of character recognition internal processing, it is a numerical value that is calculated. Therefore, it has been difficult to appropriately determine whether the print state is good or bad in accordance with the human senses.

  The present invention has been made in view of the above problems, and an image processing sensor capable of performing pass / fail determination with high accuracy even when a character with low contrast is read, and is executed by the image processing sensor. An object of the present invention is to provide an image processing method and a computer program.

  In order to achieve the above object, an image processing sensor according to a first aspect of the present invention is an image processing unit that images an inspection object, and image processing that performs image processing of the inspection object from an image of the inspection object captured by the imaging unit. An image processing sensor comprising: means; and a pass / fail determination unit that determines pass / fail of an inspection target based on a processing result of the image processing unit; and an output unit that outputs the determination result of the pass / fail determination unit to the outside. The image processing means includes an area specifying means for specifying a rectangular area including a character or character string to be recognized, which is included in the captured image of the inspection object, and character recognition for recognizing a character in the specified rectangular area. Means for calculating an evaluation value indicating the degree of shading in the rectangular area, and evaluating the degree of shading based on the calculated evaluation value and a predetermined threshold, wherein the pass / fail judgment means comprises the character And performing quality determination of the inspection object based on the recognized characters and shade degree evaluated by said shading evaluation means by identification means.

  The image processing sensor according to a second invention is characterized in that, in the first invention, the density evaluation means calculates an evaluation value by analyzing a luminance value histogram in the rectangular area.

  The image processing sensor according to a third aspect of the present invention is the image processing sensor according to the first or second aspect, further comprising image display means for displaying an image of the imaged inspection object, wherein the image display means is used in the shade evaluation means. It has a threshold value setting means for setting a threshold value.

  According to a fourth aspect of the present invention, there is provided the image processing sensor according to any one of the first to third aspects, wherein the first mode for executing the density evaluation unit and the second mode for not executing the mode are selected. Means are provided.

  An image processing sensor according to a fifth aspect of the present invention is the image processing sensor according to any one of the first to third aspects, wherein the selection is made between a first mode in which the density evaluation unit is executed and a second mode in which the density evaluation unit is not executed. A selection receiving unit is provided.

  An image processing sensor according to a sixth aspect of the present invention is the inspection setting data generation means for generating inspection setting data including at least the predetermined threshold, the trigger condition, and the image processing tool according to any one of the first to fifth aspects. And inspection setting data storage means for storing the generated inspection setting data, a setting mode for accepting input by the user to generate inspection setting data, and an inspection for an image repeatedly acquired based on a trigger condition An image that is repeatedly acquired based on the trigger condition by reading the stored inspection setting data when the mode switching unit switches from the setting mode to the execution mode. On the other hand, the image processing means is executed using the image processing tool.

  Next, in order to achieve the above object, an image processing method according to a seventh aspect of the present invention includes an imaging means for imaging an inspection object, and image processing of the inspection object from an image of the inspection object imaged by the imaging means. An image processing sensor comprising: an image processing unit to perform; a pass / fail determination unit for determining pass / fail of an inspection object based on a processing result of the image processing unit; and an output unit for outputting the determination result of the pass / fail determination unit to the outside In the image processing method that can be executed, the image processing sensor is identified with a step of identifying a rectangular area including a character or a character string to be recognized, which is included in the captured image of the inspection object. Recognizing characters in the rectangular area, calculating an evaluation value indicating the degree of shading in the rectangular area, and evaluating the degree of shading based on the calculated evaluation value and a predetermined threshold, And performing quality determination of the inspection object based on the identified characters and evaluated shading degree.

  An image processing method according to an eighth aspect of the present invention is the image processing method according to the seventh aspect, wherein the image processing sensor calculates an evaluation value by analyzing a luminance value histogram in the rectangular area.

  The image processing method according to a ninth aspect of the present invention is the seventh or eighth aspect, wherein the image processing sensor includes a step of displaying a captured image of the inspection object and a step of setting the threshold value. It is characterized by.

  An image processing method according to a tenth aspect of the invention is characterized in that, in any one of the seventh to ninth aspects, a step of selecting whether or not to evaluate the degree of shading.

  An image processing method according to an eleventh aspect of the present invention is the image processing method according to any one of the seventh to tenth aspects, wherein the image processing sensor generates inspection setting data including at least the predetermined threshold, a trigger condition, and an image processing tool. A step of storing the generated inspection setting data, a setting mode for receiving the input by the user and generating the inspection setting data, and an inspection for inspecting an image repeatedly acquired based on a trigger condition When switching from the setting mode to the execution mode, the stored examination setting data is read out, and the image processing tool is used for an image repeatedly acquired based on the trigger condition. And image processing.

  Next, in order to achieve the above object, a computer program according to a twelfth aspect performs image processing of an inspection object from an imaging means for imaging the inspection object and an image of the inspection object imaged by the imaging means. An image processing sensor comprising: an image processing unit; a pass / fail determination unit that determines pass / fail of an inspection object based on a processing result of the image processing unit; and an output unit that outputs the determination result of the pass / fail determination unit to the outside. In the computer program that can be executed, the image processing means is specified as an area specifying means for specifying a rectangular area including a character or a character string to be recognized, which is included in the image of the imaged inspection object. Character recognition means for recognizing characters in the rectangular area, and an evaluation value indicating the degree of shading in the rectangular area are calculated, and the degree of shading is based on the calculated evaluation value and a predetermined threshold value Functions as a shade evaluation unit for evaluation, and causes the quality determination unit to function as a unit for determining pass / fail of an inspection object based on the characters recognized by the character recognition unit and the shade degree evaluated by the shade evaluation unit. It is characterized by that.

  In the first invention, the seventh invention, and the twelfth invention, a rectangular area including a character or a character string to be recognized, which is included in the captured image of the inspection object, is specified, and characters in the specified rectangular area are specified. recognize. On the other hand, an evaluation value indicating the degree of shading in the rectangular area is calculated, and the shading degree is evaluated based on the calculated evaluation value and a predetermined threshold value. The quality of the inspection object is determined based on the recognized character and the evaluated shade level. As a result, even when the character recognition is successful, when the contrast is low, it is possible to correctly detect the thinness of the printed character due to insufficient ink ejection and insufficient laser output. Can be performed correctly.

  In the second and eighth inventions, the evaluation value is calculated by analyzing the luminance value histogram in the rectangular area, so that the contrast level can be objectively evaluated according to the difference in the luminance value, and the ink It becomes possible to correctly detect the thinness of the printed characters due to insufficient ejection and insufficient laser output.

  In the 3rd invention and the 9th invention, the state which a user wants to make NG determination can be certainly set as NG determination by displaying the image of the imaged inspection object and setting a threshold in the displayed state. A threshold value can be set, and it is possible to execute pass / fail judgment that matches the sensitivity judged by the user.

  In the fourth and tenth inventions, by selecting whether or not to evaluate the degree of shading, it is possible to correctly perform NG determination according to the contrast level of the printed characters, and to improve usability. It becomes possible.

  In the fifth invention, since selection of the first mode for evaluating the degree of shading and the second mode for not evaluating is accepted, it is possible to select whether or not to incorporate the shading degree into the NG judgment according to the user's judgment. It becomes possible.

  In the sixth invention and the eleventh invention, inspection setting data including at least a predetermined threshold, a trigger condition, and an image processing tool is generated and stored. When the setting mode for receiving the input by the user and generating the inspection setting data and the inspection mode for inspecting the image repeatedly acquired based on the trigger condition are switched and the setting mode is switched to the execution mode, The stored inspection setting data is read out, and an image processing tool is used to perform image processing on an image that is repeatedly acquired based on the trigger condition. As a result, the inspection setting data for image processing can be stored as a series of data groups, so even if a failure occurs in the image processing sensor, the pass / fail determination is made under the same conditions in the alternative machine. Can be done.

  In the present invention, even if the character recognition is successful, when the contrast is low, it is possible to correctly detect the thinness of the printed character due to insufficient ink ejection and insufficient laser output. Judgment can be made correctly.

It is a schematic diagram which shows the structure of the image processing sensor which concerns on embodiment of this invention. 1 is an outline view showing a configuration of an imaging device of an image processing sensor according to an embodiment of the present invention. It is a block diagram which shows the hardware constitutions of the imaging device of the image processing sensor which concerns on embodiment of this invention. It is a front view which shows the structure of the display apparatus of the image processing sensor which concerns on embodiment of this invention. It is an illustration figure of the mode switching screen of the display apparatus of the image processing sensor which concerns on embodiment of this invention. It is an illustration figure of the mode selection reception screen of the display apparatus of the image processing sensor which concerns on embodiment of this invention. It is an illustration figure of the character printed on the surface of the test object. It is a functional block diagram of the image processing sensor which concerns on Embodiment 1 of this invention. It is a flowchart which shows the flow of the test | inspection setting data generation process of DSP of the image processing sensor which concerns on Embodiment 1 of this invention. It is an illustration figure of the character string area | region extraction of the image processing sensor which concerns on Embodiment 1 of this invention. It is explanatory drawing of the identification method of the inclination angle of the character string of the cut-out object of the image processing sensor which concerns on Embodiment 1 of this invention. It is an illustration figure of the method of cutting out the character from the character string of the cutting target of the image processing sensor which concerns on Embodiment 1 of this invention. It is an illustration figure of the production | generation range of the luminance value histogram of the image processing sensor which concerns on Embodiment 1 of this invention. It is an illustration figure of the evaluation value calculation method from the luminance value histogram of the image processing sensor which concerns on Embodiment 1 of this invention. It is an illustration figure of the threshold value setting screen of the image processing sensor which concerns on Embodiment 1 of this invention. It is a flowchart which shows the flow of the quality determination process of DSP of the image processing sensor which concerns on Embodiment 1 of this invention. It is a functional block diagram of the image processing sensor which concerns on Embodiment 2 of this invention. It is an illustration figure of character string area | region extraction of the image processing sensor which concerns on Embodiment 2 of this invention. It is an illustration figure of the luminance value histogram generation | occurrence | production in the rectangular area of the image processing sensor which concerns on Embodiment 2 of this invention. It is explanatory drawing which shows the state which cut out the character from the part in which the specified character or character string of the image processing sensor which concerns on Embodiment 2 of this invention exists. It is a flowchart which shows the flow of the quality determination process of DSP of the image processing sensor which concerns on Embodiment 2 of this invention.

  Hereinafter, an image processing sensor according to an embodiment of the present invention will be described with reference to the drawings. Note that elements having the same or similar configuration or function are denoted by the same or similar reference numerals throughout the drawings referred to in the description of this embodiment, and detailed description thereof is omitted.

  FIG. 1 is a schematic diagram showing a configuration of an image processing sensor according to an embodiment of the present invention. As shown in FIG. 1, the image processing sensor according to the present embodiment includes an imaging device (imaging means) 1 and a display device (display) connected by a connection cable 3 so as to be able to perform data communication with the imaging device 1. Means) 2. Of course, instead of the display device 2, an external computer having a display may be used. Note that the imaging device 1 and the display device 2 may be configured integrally.

  The imaging apparatus 1 includes an FPGA, a DSP, and the like that execute image processing therein, and includes a camera module having an imaging device that images the inspection target, and an illumination unit that irradiates the inspection target with light. ing. In order to make the imaging device 1 compact, for example, as shown in FIG. 1, a lens 12 is arranged near the center of the front of the imaging device 1, and a plurality of LEDs 11 are arranged as an illumination unit so as to surround the lens 12. It is. Note that external illumination (ring illumination or the like) may be provided separately from the imaging device 1.

  FIG. 2 is an outline view showing the configuration of the imaging device 1 of the image processing sensor according to the embodiment of the present invention. 2A is a front view showing the configuration of the imaging device 1 of the image processing sensor according to the embodiment of the present invention, and FIG. 2B is the imaging of the image processing sensor according to the embodiment of the present invention. FIG. 2C is a plan view showing the configuration of the apparatus 1, and FIG. 2C is a rear view showing the configuration of the imaging apparatus 1 of the image processing sensor according to the embodiment of the present invention.

  As shown in FIG. 2A, the imaging apparatus 1 has a lens 12 disposed in the vicinity of the center of the front surface, and a plurality of LEDs 11 are disposed so as to surround the lens 12. At the time of imaging, by turning on the plurality of LEDs 11, it is possible to irradiate the inspection object with light and to clearly image the inspection object.

  As shown in FIGS. 2B and 2C, the rear surface of the imaging device 1 is in data communication with the display device 2 and a power connector 102 that connects a power cable that receives power supplied from an external power source. A connection connector 103 to which the connection cable 3 can be connected is provided. A focus adjustment screw 101 that can manually adjust the focus is also provided on the back surface.

  FIG. 3 is a block diagram showing a hardware configuration of the imaging device 1 of the image processing sensor according to the embodiment of the present invention. In FIG. 3, the connector board 16 is supplied with power from an external power source via a power connector 102 (see FIGS. 2B and 2C) provided in the power interface 161. The power supply board 18 supplies the supplied power to each board. In the present embodiment, power is supplied to the camera module 14 via the main board 13. The motor driver 181 of the power supply board 18 supplies driving power to the motor 141 of the camera module 14 to realize autofocus.

  The communication board 17 transmits an OK / NG signal (determination signal), image data, and the like indicating the quality of the inspection object to the display device 2 based on whether or not a defect output from the main board 13 is detected. The display device 2 that has received the determination signal displays the determination result. In this embodiment, an OK / NG signal (determination signal) is output via the communication board 17, but an OK / NG signal (determination signal) is output via the connector board 16, for example. Anyway.

  The illumination board 15 is provided with a plurality of LEDs 11 that irradiate light to an imaging region for imaging the inspection object, and a reflector (not shown) is provided in front of the LEDs 11. The lens 12 can be replaced as a lens unit for short distance or long distance.

  The camera module (imaging means) 14 can control the autofocus operation when the motor 141 is driven. The inspection object is imaged according to the imaging instruction signal from the main board 13. In the present embodiment, a CMOS substrate 142 is provided as an image sensor, and the captured color image is converted into an HDR image based on conversion characteristics that widen the dynamic range in the CMOS substrate 142 and is transferred to the FPGA 131 on the main substrate 13. Is output.

  The main board 13 controls the operation of each connected board. For example, a control signal for controlling turning on / off of the plurality of LEDs 11 is transmitted to the LED driver 151 with respect to the illumination board 15. The LED driver 151 adjusts, for example, lighting / extinguishing of the LED 11, the amount of light, and the like according to a control signal from the FPGA 131. Further, a control signal for controlling the autofocus operation is transmitted to the motor 141 of the camera module 14 via the motor driver 181 of the power supply substrate 18, and an imaging instruction signal and the like are transmitted to the CMOS substrate 142. .

  A portable disk drive 19 may be connected to the main board 13. Various setting files 100 can also be downloaded from a portable recording medium 90 such as a DVD-ROM via the portable disk drive 19.

  The FPGA 131 of the main board 13 performs illumination control and imaging control, and executes image processing on the acquired image data (image processing means). In addition, the DSP 132 of the main board 13 performs edge detection processing, pattern search processing, and the like on the image data. As a result of the pattern search process, an OK / NG signal (determination signal) indicating whether the inspection object is good or not is output to the communication board 17 based on whether or not a defect is detected (good / bad determination means, output means). The arithmetic processing result and the like are stored in the memory 133. In the present embodiment, the FPGA 131 executes illumination control, imaging control, and the like, but the DSP 132 may execute it. Further, a circuit in which the FPGA 131 and the DSP 132 are integrated, that is, a main control circuit (main control unit) may be provided. In short, a control signal for controlling turning on / off of the plurality of LEDs 11 is transmitted to the LED driver 151, a control signal for controlling the autofocus operation is transmitted to the motor 141 of the camera module 14, and an imaging instruction signal or the like is transmitted to the CMOS substrate. 142 may be provided, or a main control unit having the functions of both the FPGA 131 and the DSP 132 may be provided.

  FIG. 4 is a front view showing the configuration of the display device (display unit) 2 of the image processing sensor according to the embodiment of the present invention. As shown in FIG. 4, a touch panel 21 is provided in the central portion of the front surface of the display device 2, displays a color image of the imaged inspection object on the screen, and accepts a selection input by the user.

  The display device 2 includes a power connector 24 that connects a power cable supplied with power from an external power source, and a connection connector 25 that can connect the connection cable 3 that performs data communication with the imaging device 1. Yes. Further, a USB port 22 that can be connected to a USB memory or the like is provided on the front surface.

  The user controls the operation of the image processing sensor by selecting a button displayed on the screen of the touch panel 21 of the display device 2. It is also possible to switch between an “inspection mode” for inspecting an inspection object and a “setting mode” for setting conditions of the image processing sensor. In other words, the image processing sensor according to the present embodiment sets an inspection mode (Run mode) for determining the quality of an inspection object and various parameters (imaging parameters, illumination parameters, image processing parameters, etc.) used for the inspection. A mode switching unit (see FIG. 8 described later) for switching between setting modes (non-Run mode) to be performed is provided. FIG. 5 is an exemplary view of a mode switching screen of the display device 2 of the image processing sensor according to the embodiment of the present invention.

  FIG. 5A is a view showing an example of the screen display of “inspection mode”. As shown in FIG. 5A, the inspection object image captured by the imaging apparatus 1 is displayed in the inspection object display area 51. The lower left “sensor setting” button 52 functions as a mode switching unit, and when the “sensor setting” button 52 is selected, the mode is switched to “setting mode”, and the screen transitions to the screen shown in FIG.

  FIG. 5B is an exemplary view of a screen display of “setting mode”. As shown in FIG. 5B, in the program selection area 53, the type of inspection object or the inspection environment is selected. Here, the “program” means a series of data groups (combination of parameter values) set in accordance with the type of inspection object or the inspection environment, and varies depending on the type of inspection object or the inspection environment. A group of data can be stored as a program.

  When a master image (reference image) serving as a reference to be compared with the inspection object is stored, the master image is displayed in the master image display area 54. When the “setting navigation” button 55 is selected, the screen transitions to a setting screen for performing detailed settings. The “start operation” button 56 in the lower left functions as a mode switching unit. When the “start operation” button 56 is selected, the mode is switched to “inspection mode” and the screen transitions to the screen shown in FIG.

  In addition, the user selects a button displayed on the screen of the touch panel 21 of the display device 2, and does not evaluate the lightness and darkness evaluation mode (first mode) for evaluating the lightness and darkness. Switching to the “inspection mode (second mode)” can also be performed. In other words, in the image processing sensor according to the present embodiment, the “gradation evaluation mode (first mode)” that evaluates the shade degree in the pass / fail determination of the inspection object, and the “inspection mode that does not evaluate the shade degree”. A mode selection receiving unit 713 (described later) shown in FIG. FIG. 6 is an exemplary view of a mode selection reception screen of the display device 2 of the image processing sensor according to the embodiment of the present invention.

  FIG. 6A is an exemplary diagram of an extended setting screen for confirming whether or not to evaluate the degree of shading. When the “Confirm dark / dark” button 61 shown in FIG. 6A is selected, the mode selection acceptance screen shown in FIG.

  FIG. 6B is an exemplary view of a mode selection reception screen for determining whether or not the evaluation of the degree of shading is valid. As shown in FIG. 6B, the mode selection button 62 accepts mode selection. That is, when “valid” is selected, “gradation evaluation mode (first mode)” that evaluates the degree of shading is selected, and when “invalid” is selected, “inspection mode that does not evaluate the shading degree” (Second mode) "is selected.

  In addition, without having the mode selection reception part 713, the mode selection part (after-mentioned) shown in FIG. 8 which switches a 1st mode and a 2nd mode, for example according to the signal from an external sensor, PLC, etc. You may have. In any case, in the present embodiment, the second mode is set as the default, and when the user selects the “valid” selection, or when the switching signal to the first mode is received from the outside, Switch to the second mode.

  FIG. 7 is an exemplary view of characters printed on the surface of the inspection object. When the uppermost character string “ABC123” in FIG. 7 is used as a reference character, in the conventional pass / fail judgment, a character string “missing” in which a part of the character is missing, and a part of the character are stained. For the “dirty” text string, the degree of match was low. However, since the degree of coincidence is determined by a plurality of factors, it is not possible to specify whether the degree of coincidence has decreased due to “missing” or “dirt”, and a threshold value is set in accordance with the sense of pass / fail judgment of a person. It was difficult.

  In other words, in order to make an OK determination for a character string of “missing” in which a part of the character is missing and a character string of “dirty” in which a part of the character is soiled, they are matched as before. The degree of coincidence needs to be calculated so that the degree of coincidence does not fall, not the degree.

  On the other hand, the character string “light and dark” printed thin is not judged as NG even though the degree of coincidence is lowered. However, in reality, it is caused by ink ejection failure, laser output shortage, and the like, and there are many cases where NG determination should be made originally. Such shading is easily quantified as a visibility essentially in line with human senses.

  As described above, depending on the user, even if the characters are the same, it is difficult to set the pass / fail judgment threshold for the degree of “missing” or “dirt”. In some cases, it may be desirable to make an NG determination. In such a case, the conventional quality determination method cannot cope. In view of this, the image processing sensor according to the embodiment of the present invention has been devised so that the quality determination can be performed only in accordance with the contrast in the region where the character or the character string exists.

(Embodiment 1)
FIG. 8 is a functional block diagram of the image processing sensor according to Embodiment 1 of the present invention. In FIG. 8, the image display unit 701 of the image processing unit 700 accepts input of inspection setting data. Here, the inspection setting data includes, for example, a trigger condition set through GUI input, an image processing tool such as contour extraction and area calculation, in addition to a threshold value for evaluating the degree of shading described later.

  Specifically, the mode switching unit 711 of the image display unit 701 includes an inspection mode (Run mode) for determining the quality of the inspection object, and various parameters (imaging parameters, illumination parameters, image processing parameters, etc.) used for the inspection. Switch between setting mode (non-run mode) for setting.

  When switched to the setting mode, the inspection setting data generation unit 702 receives input of inspection setting data. Of course, the threshold value for evaluating the degree of shading described later is also received by the threshold value setting unit 712 of the image display unit 701 and added to the inspection setting data.

  The generated inspection setting data is stored in the memory 133 by the inspection setting data storage unit 703. Of course, it may be stored in a removable external memory. As a result, even if the model is changed, it is possible to perform pass / fail determination with the same setting.

  FIG. 9 is a flowchart showing a flow of inspection setting data generation processing of the DSP 132 of the image processing sensor according to Embodiment 1 of the present invention. In FIG. 9, the DSP 132 of the image processing sensor accepts input from the input screen corresponding to each inspection setting data (step S901).

  The DSP 132 generates each inspection setting data that has received the input as inspection setting data that is a series of data groups (step S902), and stores the data in the memory 133 (step S903).

  Returning to FIG. 8, when the mode switching unit 711 switches to the inspection mode, the inspection setting data reading unit 704 reads the inspection setting data stored from the memory 133, and the color image acquisition unit 705 reads the inspection that has been read. Based on the setting data, the operation of the CMOS substrate 142 of the camera module (imaging means) 14 is controlled to obtain the pixel value (luminance value) of the captured color image. In addition, the captured image is displayed on the display device 2.

  The area specifying unit 706 specifies a rectangular area including characters or character strings to be recognized, which are included in the acquired image data. FIG. 10 is a view showing an example of character string region cutout by the image processing sensor according to Embodiment 1 of the present invention.

  As shown in FIG. 10A, an area where the character string exists is cut out as a rectangular area from an image obtained by capturing the character string “ABC123”. That is, as shown in FIG. 10B, a rectangular area surrounding the area where the character string exists is cut out. The region cutout method is not particularly limited as long as it is a known method.

  Depending on the application, image data may be acquired in a state where a rectangular area including a character string is cut out in advance. In this case, the area specifying unit 706 is not necessary.

  FIG. 11 is an explanatory diagram of a method for specifying the inclination angle of a character string to be cut out by the image processing sensor according to the first embodiment of the present invention. A case where a character string “ABC123” as shown in FIG. 11A is acquired as image data will be described.

  First, pixel values in a region including a character string are projected at predetermined projection angles, and a luminance value histogram is created for each projection angle. In the example of FIG. 11, the luminance value histogram in the case of the projection angle 15 degrees shown in FIG. 11B, the projection angle 0 degrees shown in FIG. 11C, and the projection angle −15 degrees shown in FIG. These are shown in FIGS. 11 (e), 11 (f), and 11 (g), respectively.

  As can be seen by comparing FIG. 11E, FIG. 11F, and FIG. 11G, there is a large difference in the separation of the luminance value distribution for each character depending on the projection angle. The higher the projection angle, the higher the degree of coincidence with the inclination angle of the character string. In the example of FIG. 11, since the separation degree of the luminance value histogram shown in FIG. 11E is the highest, the inclination angle of the character string can be determined to be 15 degrees.

  FIG. 12 is an exemplary diagram of a method of cutting out characters from a character string to be cut out by the image processing sensor according to Embodiment 1 of the present invention. When the inclination angle of the character string is determined by the procedure shown in FIG. 11, the delimiter position for each character can be estimated as shown in FIG. Moreover, since the area | region where a luminance value exists can be estimated for every divided | segmented area | region, as shown in FIG.12 (b), the position of the upper and lower ends of a character can also be estimated. This makes it possible to determine a rectangular area for each character.

  Note that the character cutout method is not particularly limited to this. For example, the entire background image including the character string may be binarized so that the character string can be easily distinguished from the other portion. By appropriately setting the threshold value for the binarization process, it is possible to appropriately cut out an area where characters exist.

  In addition to this, for example, by applying an expansion / contraction filter and evaluating connectivity for each region, it is possible to determine whether or not the characters are the same character element, or the character regions do not overlap significantly. A rectangular area circumscribing the character may be specified based on a constraint condition such that the character areas are aligned in parallel.

  Returning to FIG. 8, the character recognition unit 707 recognizes a character in the specified rectangular area. Specifically, characters are recognized by the following procedure. First, a circumscribed rectangular area is cut out for each character within the specified rectangular area.

  Specifically, when a rectangular area for each character is cut out, OCR recognition is executed to recognize a character string and store it as text data. If the character string cannot be recognized and cannot be converted into text data, the process may be performed again after cutting out the rectangular area for each character.

  The shading evaluation unit 708 calculates an evaluation value indicating the shading degree in the specified rectangular area, and evaluates the shading degree based on the calculated evaluation value and a predetermined threshold value. Specifically, an evaluation value is calculated by analyzing a luminance value histogram in a rectangular area for each extracted character.

  FIG. 13 is a view showing an example of the generation range of the luminance value histogram of the image processing sensor according to Embodiment 1 of the present invention. As shown in FIG. 13A, when a rectangular area is cut out for each character, for example, a luminance value histogram as shown in FIG. 13B is generated for each rectangular area 1301 of the cut out character. Alternatively, a luminance value histogram may be generated for the rectangular region 1302 of the entire character string.

  FIG. 14 is an illustration of an evaluation value calculation method from the luminance value histogram of the image processing sensor according to the first embodiment of the present invention. As shown in FIG. 14, the luminance value histogram displays the distribution of luminance values as a frequency distribution of pixels.

  In the first embodiment, the evaluation value (contrast) is calculated based on (Expression 1) using the minimum value Lmin and the maximum value Lmax in the luminance value histogram.

  Evaluation value (contrast) = (Lmax−Lmin) / (Lmax + Lmin) (Expression 1)

  Of course, the evaluation value calculation method is not particularly limited to this. For example, when there are a plurality of frequency peak values, the maximum peak frequency may be calculated as (Pmax) and the minimum peak frequency as Pmin (Formula 2).

  Evaluation value (contrast) = (Pmax−Pmin) / (Pmax + Pmin) (Expression 2)

  Of course, even a simple difference (Pmax−Pmin) does not matter in terms of showing contrast.

  Furthermore, on the premise that there is little variation in the luminance value of the background area, the character portion appears as a difference in luminance value. In other words, the area where characters are present can be considered as a high-frequency area compared to the background area, and the variance of the luminance value of the entire image is calculated, or the total of high-frequency components is calculated by Fourier transforming the entire image, etc. Even with this method, the evaluation value (contrast) can be calculated.

  Returning to FIG. 8, the image display unit 701 includes a threshold setting unit 712 that sets a threshold to be compared with the evaluation value calculated by the density evaluation unit 708 in the pass / fail determination unit 720. Here, the threshold value is a contrast threshold value that determines whether the user makes an NG determination or an OK determination.

  FIG. 15 is a view for showing an example of a threshold setting screen of the image processing sensor according to the first embodiment of the present invention. As shown in FIG. 15A, a slider 150 for setting a threshold value is displayed, and a numerical value of the threshold value corresponding to the position of the slider is displayed in the numerical value display area 152.

  As shown in FIG. 15B, the numerical value displayed in the numerical value display area 152 changes by dragging the slider 150 with a mouse or the like to change the position. In the example of FIG. 15, the threshold value is changed from 46% in FIG. 15A to 66% as shown in FIG. 15B.

  Of course, the operation is not limited to dragging the slider 150, and the threshold value is increased or decreased by selecting the “+” button 154 and the “−” button 153 arranged on the left and right of the slider 151 with a mouse or the like. May be. Further, the threshold value may be directly input as a numerical value.

  FIG. 15C is an exemplary diagram of a threshold value numerical value input screen. The numerical value input screen is displayed in a pop-up by selecting the “downward black triangle” button displayed in the numerical value display area 152 of FIGS. 15A and 15B with a mouse or the like.

  As shown in FIG. 15C, a desired threshold value is input to the input area 155 by selecting the numeric keypad display unit 156 with a mouse or the like. Since the degree of shading to be evaluated varies depending on the threshold, an image of the inspection object corresponding to the variation in contrast may be displayed in the image display area 157. When it is determined that the input of an appropriate threshold is completed, the “OK” button 158 is selected with a mouse or the like to determine the threshold.

  Note that the screen shifts from the “setting mode” screen shown in FIG. 5B on the mode switching screen shown in FIG. 5 to the threshold setting screen shown in FIG. Therefore, the threshold value for the evaluation of the degree of shading is included in the inspection setting data (combination of parameter values), which is a series of data groups stored in the memory 133 as the “program”.

  As described above, when the inspection mode (second mode) is selected, the inspection setting data stored in the memory 133 is read out, and a new image is automatically captured based on the trigger condition. On the other hand, the image processing tool included in the inspection setting data is applied, and further, the OCR inspection is executed using the threshold value of the degree of gray level evaluation included in the inspection setting data.

  As described above, the image display unit 701 may be provided with the mode selection receiving unit 713, or the “tone evaluation mode (first mode)” for evaluating the shade degree by receiving the control signal. And a mode selection unit 714 that can switch between “inspection mode (second mode)” that does not evaluate the degree of shading. In this case, the control signal may be received by data communication from an external sensor or PLC.

  The pass / fail determination unit 720 determines pass / fail of the inspection object based on the character recognition result in the character recognition unit 707 and the evaluation value of the shade degree in the shade evaluation unit 708. Since the evaluation threshold of the degree of shading is selected in accordance with the user's NG determination, it is possible to perform pass / fail determination that matches the user's sense. The output unit 730 outputs the determination result of the pass / fail determination unit 720 to the outside.

  FIG. 16 is a flowchart showing a flow of pass / fail judgment processing of the DSP 132 of the image processing sensor according to the first embodiment of the present invention. As shown in FIG. 16, the DSP 132 of the image processing sensor reads the inspection setting data stored from the memory 133 (step S1601), and the CMOS substrate 142 of the camera module (imaging means) 14 based on the read inspection setting data. The inspection object is controlled to image the inspection object, and the pixel value (luminance value) of the captured color image is acquired (step S1602).

  The DSP 132 specifies a rectangular area including characters or character strings to be recognized, which are included in the acquired image data (step S1603). The DSP 132 cuts out a rectangular area circumscribing each character within the specified rectangular area (step S1604), executes OCR recognition, and recognizes a character (character string) (step S1605).

  The DSP 132 calculates an evaluation value (contrast) indicating the degree of shading in the specified rectangular area (step S1606), and evaluates the shading degree based on the calculated evaluation value and a predetermined threshold value. Specifically, an evaluation value is calculated by analyzing a luminance value histogram in a rectangular area for each extracted character.

  The DSP 132 determines whether the character recognition in step S1605 has failed (step S1607). When the DSP 132 determines that the character recognition has failed (step S1607: YES), the DSP 132 sets the pass / fail determination result as NG (step S1609).

  When the DSP 132 determines that the character recognition is successful (step S1607: NO), the DSP 132 determines whether or not the evaluation value of the degree of shading in step S1606 is smaller than the threshold (step S1608). When the DSP 132 determines that the evaluation value of the degree of shading is smaller than the threshold value (step S1608: YES), the DSP 132 sets the pass / fail determination result as NG determination (step S1609).

  When the DSP 132 determines that the evaluation value of the degree of shading is greater than or equal to the threshold (step S1608: NO), the DSP 132 sets the pass / fail determination result as OK (step S1610). The DSP 132 outputs the pass / fail determination result to the outside (step S1611).

  In the first embodiment, the order of the character recognition processing in the character recognition unit 707 and the evaluation processing of the degree of shading in the shading evaluation unit 708 is not particularly limited, and any processing may precede and simultaneously. May be executed.

  As described above, according to the first embodiment, even when the character recognition is successful, when the contrast is low, the thinness of the printed character due to insufficient ink ejection and insufficient laser output is correctly detected. Therefore, it is possible to correctly perform the NG determination in the pass / fail determination.

(Embodiment 2)
FIG. 17 is a functional block diagram of an image processing sensor according to Embodiment 2 of the present invention. In FIG. 17, functional blocks that execute the same processing as in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. The second embodiment is different from the first embodiment in that the density evaluation process is executed before the character segmenting process for executing the character recognition process.

  The image display unit 701 of the image processing unit 700 receives input of inspection setting data. Here, the inspection setting data includes an image processing tool such as a trigger condition set through a GUI input, contour extraction, and area calculation, in addition to a threshold for evaluation of the degree of shading described later.

  Specifically, the mode switching unit 711 of the image display unit 701 includes an inspection mode (Run mode) for determining the quality of the inspection object, and various parameters (imaging parameters, illumination parameters, image processing parameters, etc.) used for the inspection. Switch between setting mode (non-run mode) for setting.

  Since the processing when the mode is switched to the setting mode is the same as that of the first embodiment, detailed description thereof is omitted. When the mode is switched to the inspection mode, the inspection setting data reading unit 704 reads the inspection setting data stored from the memory 133, and the color image acquisition unit 705 uses the camera module (imaging unit) based on the read inspection setting data. The pixel values (luminance values) of the captured color image are acquired by controlling the operations of the 14 CMOS substrates 142. In addition, the captured image is displayed on the display device 2.

  The area specifying unit 706 specifies a rectangular area including characters or character strings to be recognized, which are included in the acquired image data. FIG. 18 is a view showing an example of character string region cutout by the image processing sensor according to Embodiment 2 of the present invention.

  As illustrated in FIG. 18A, an area where the character string exists is cut out as a rectangular area from an image obtained by capturing the character string “ABC123”. That is, as shown in FIG. 18B, a rectangular area surrounding the area where the character string exists is cut out. The region cutout method is not particularly limited as long as it is a known method.

  As in the first embodiment, depending on the application, image data may be acquired in a state where a rectangular area including a character string is cut out in advance. In this case, the area specifying unit 706 is not necessary.

  Returning to FIG. 17, the shading evaluation unit 708 calculates an evaluation value indicating the degree of shading in the cut-out rectangular area including the character or character string to be character-recognized, and the calculated evaluation value and a predetermined threshold value Based on the above, the degree of shading is evaluated. Specifically, an evaluation value is calculated by analyzing a luminance value histogram in the cut out rectangular area. When the area specifying unit 706 does not exist, the shade degree of the entire image is evaluated.

  FIG. 19 is a view for illustrating generation of a luminance value histogram in a rectangular area of the image processing sensor according to the second embodiment of the present invention. As shown in FIG. 19A, when a rectangular area including a character or a character string to be character-recognized is cut out, the luminance value as shown in FIG. Generate a histogram. Thereby, it can be estimated that the character or character string exists in the part with many pixels with the same luminance value.

  Since the evaluation value calculation method from the luminance value histogram is the same as that in the first embodiment, detailed description thereof is omitted. In order to identify the portion where the character or the character string exists with higher accuracy, the variance of the luminance value in the rectangular region is calculated because the region where the character exists is a high-frequency region compared to other background regions. Alternatively, by calculating the evaluation value (contrast) by, for example, calculating the sum of high-frequency components by performing Fourier transform in the rectangular area, it is possible to specify a portion where a character or a character string exists.

  As shown in FIG. 17, the image display unit 701 includes a threshold setting unit 712 that sets a threshold value to be compared with the evaluation value of the shade degree calculated by the shade evaluation unit 708. The threshold setting screen is the same as in the first embodiment.

  The image display unit 701 may be provided with a mode selection receiving unit 713, and by receiving a control signal, a “shading evaluation mode (first mode)” for evaluating the shading degree, and a shading degree degree. A mode selection unit 714 that can switch between “inspection mode (second mode)” in which evaluation is not performed may be provided separately. In this case, the control signal may be received by data communication from an external sensor or PLC. In the case of the second mode, the shade degree is not evaluated.

  In the second embodiment, the density evaluation unit 708 compares the calculated evaluation value of the degree of density with a threshold value and determines whether it is NG determination or OK determination. That is, from the viewpoint of the user, the inspection object that is determined as NG from the viewpoint of the degree of shading can be excluded from the target of subsequent processing at this point.

  Returning to FIG. 17, the character recognizing unit 707 recognizes a character in the specified rectangular area with respect to the inspection target that has been determined to be OK by the density evaluation unit 708. Specifically, first, a rectangular area for each character is cut out from a rectangular area in which a character or character string is estimated to exist. FIG. 20 is an explanatory diagram showing a state in which a character is cut out from a portion where the specified character or character string exists in the image processing sensor according to the second embodiment of the present invention. As shown in FIG. 20 (a), since the region where the character or the character string exists is estimated, in the same manner as in FIG. 12 of the first embodiment, each character as shown in FIG. 20 (b). Determine the rectangular area.

  Note that the character cutout method is not particularly limited to this. For example, the entire background image including the character string may be binarized so that the character string and the non-character string portion can be easily distinguished. By appropriately setting the threshold value for the binarization process, it is possible to appropriately cut out an area where characters exist.

  In addition, for example, by applying an expansion / contraction filter and evaluating the integrity of each region, it is possible to determine whether or not the character elements are the same, or to cut out characters, or the character regions do not overlap significantly A rectangular area circumscribing the character may be specified based on a constraint condition such that the character areas are aligned in parallel.

  Returning to FIG. 17, the character recognition unit 707 recognizes a character in the specified rectangular area. Since the character recognition method is the same as that of the first embodiment, detailed description thereof is omitted.

  As described above, when the inspection mode (second mode) is selected, the inspection setting data stored in the memory 133 is read out, and a new image is automatically captured based on the trigger condition. On the other hand, the image processing tool included in the inspection setting data is applied, and the OCR inspection is executed using a threshold value for evaluating the degree of shading included in the inspection setting data.

  The pass / fail judgment unit 720 judges pass / fail of the inspection object based on the character recognition result in the character recognition unit 707. Only when the evaluation value of the degree of shading is equal to or greater than the threshold value, the pass / fail determination may be performed based on the character recognition result, and the calculation processing load as a whole can be reduced. The output unit 730 outputs the determination result of the pass / fail determination unit 720 to the outside.

  FIG. 21 is a flowchart showing the flow of pass / fail judgment processing of the DSP 132 of the image processing sensor according to the second embodiment of the present invention. As shown in FIG. 21, the DSP 132 of the image processing sensor reads the inspection setting data stored from the memory 133 (step S2101), and the CMOS substrate 142 of the camera module (imaging means) 14 based on the read inspection setting data. The pixel value (luminance value) of the captured color image is acquired by controlling the operation (step S2102).

  The DSP 132 specifies a rectangular area including a character or character string to be recognized, which is included in the acquired image data (step S2103). The DSP 132 calculates an evaluation value (contrast) indicating the degree of shading in the specified rectangular area (step S2104), and evaluates the shading degree based on the calculated evaluation value and a predetermined threshold value. Specifically, an evaluation value is calculated by analyzing a luminance value histogram in a rectangular area for each extracted character.

  The DSP 132 determines whether or not the calculated evaluation value of the degree of shading is equal to or greater than a threshold value (step S2105). When the DSP 132 determines that the evaluation value of the degree of shading is smaller than the threshold value (step S2105: NO), the DSP 132 sets the pass / fail determination result as NG determination (step S2106).

  When the DSP 132 determines that the evaluation value of the degree of shading is equal to or greater than the threshold (step S2105: YES), the DSP 132 cuts out a rectangular area circumscribing for each character in the specified rectangular area (step S2107), and OCR Recognition is executed to recognize a character (character string) (step S2108).

  The DSP 132 determines whether character recognition is successful (step S2109). When the DSP 132 determines that character recognition has failed (step S2109: NO), the DSP 132 sets the pass / fail determination result to NG (step S2106).

  When the DSP 132 determines that the character recognition is successful (step S2109: YES), the DSP 132 sets the OK / NG determination result as OK (step S2110). The DSP 132 outputs the pass / fail determination result to the outside (step S2111).

  In the second embodiment, the density evaluation process in the density evaluation unit 708 is performed prior to the character recognition process in the character recognition unit 707, but may be performed simultaneously. In this case, the pass / fail determination is executed based on both the pass / fail determination result based on the degree of shading and the pass / fail determination result based on character recognition.

  As described above, according to the second embodiment, even when the character recognition is successful, when the contrast is low, the thinness of the printed character due to insufficient ink ejection or laser output is correctly detected. Therefore, it is possible to correctly perform the NG determination in the pass / fail determination.

  The present invention is not limited to the above-described embodiments, and various changes and improvements can be made within the scope of the present invention. For example, the imaging device 1 and the display device 2 are not limited to the form directly connected by the connection cable 3, and needless to say, they may be connected via a network network such as a LAN or a WAN. In the first and second embodiments, the imaging device 1 and the display device 2 are separate from each other, but may be an image processing sensor configured as a single unit.

1 Imaging device (imaging means)
2. Display device (display means)
3 Connection Cable 13 Main Board 14 Camera Module 15 Illumination Board 21 Touch Panel 132 DSP
700 Image processing unit 701 Image display unit 702 Inspection setting data generation unit 703 Inspection setting data storage unit 704 Inspection setting data reading unit 705 Color image acquisition unit 706 Area specifying unit 707 Character recognition unit 708 Color evaluation unit 711 Mode switching unit (Setting / Inspection)
712 Threshold setting unit 713 Mode selection receiving unit 714 Mode selection unit 720 Pass / fail judgment unit 730 Output unit

Claims (12)

An imaging means for imaging an inspection object;
Image processing means for performing image processing of the inspection object from the image of the inspection object imaged by the imaging means;
A pass / fail determination means for determining pass / fail of the inspection object based on the processing result of the image processing means;
An image processing sensor having output means for outputting the determination result of the pass / fail determination means to the outside;
The image processing means includes
An area specifying means for specifying a rectangular area including a character or a character string to be recognized, which is included in the image of the imaged inspection object;
Character recognition means for recognizing characters in the specified rectangular area;
A shade evaluation means for calculating an evaluation value indicating a shade degree in the rectangular region, and evaluating the shade degree based on the calculated evaluation value and a predetermined threshold;
The image processing sensor, wherein the pass / fail judgment unit performs pass / fail judgment of an inspection object based on the character recognized by the character recognition unit and the shade degree evaluated by the shade evaluation unit.   The image processing sensor according to claim 1, wherein the density evaluation unit calculates an evaluation value by analyzing a luminance value histogram in the rectangular area. Image display means for displaying an image of the imaged inspection object;
The image processing sensor according to claim 1, wherein the image display unit includes a threshold setting unit configured to set the threshold used in the density evaluation unit.   4. The image processing sensor according to claim 1, further comprising a mode selection unit that selects a first mode in which the density evaluation unit is executed and a second mode in which the shade evaluation unit is not executed. 5.   4. The image processing according to claim 1, further comprising a mode selection receiving unit that receives a selection between a first mode in which the shade evaluation unit is executed and a second mode in which the shade evaluation unit is not executed. 5. Sensor. Inspection setting data generating means for generating inspection setting data including at least the predetermined threshold value, trigger condition, and image processing tool;
Inspection setting data storage means for storing the generated inspection setting data;
Mode switching means for switching between a setting mode for receiving input by a user and generating inspection setting data, and an inspection mode for inspecting an image repeatedly acquired based on a trigger condition;
When the setting mode is switched from the setting mode to the execution mode by the mode switching unit, the stored examination setting data is read, and the image processing tool is used for the image that is repeatedly acquired based on the trigger condition. The image processing sensor according to claim 1, wherein the image processing sensor executes means. An imaging means for imaging an inspection object;
Image processing means for performing image processing of the inspection object from the image of the inspection object imaged by the imaging means;
A pass / fail determination means for determining pass / fail of the inspection object based on the processing result of the image processing means;
In an image processing method that can be executed by an image processing sensor having output means for outputting the determination result of the pass / fail determination means to the outside,
The image processing sensor
Identifying a rectangular region including a character or character string to be recognized, which is included in the image of the imaged inspection object;
Recognizing characters in the specified rectangular area;
Calculating an evaluation value indicating a degree of shading in the rectangular area, and evaluating the degree of shading based on the calculated evaluation value and a predetermined threshold value, and
An image processing method comprising: determining whether or not an inspection target is acceptable based on a recognized character and an evaluated shade degree.   The image processing method according to claim 7, wherein the image processing sensor calculates an evaluation value by analyzing a luminance value histogram in the rectangular area.   The image processing method according to claim 7, wherein the image processing sensor includes a step of displaying a captured image of the inspection object and a step of setting the threshold value.   The image processing method according to claim 7, further comprising a step of selecting whether or not to evaluate the degree of shading. The image processing sensor
Generating inspection setting data including at least the predetermined threshold, a trigger condition, and an image processing tool;
Storing the generated examination setting data;
Switching between a setting mode for receiving input by a user and generating inspection setting data, and an inspection mode for inspecting an image repeatedly acquired based on a trigger condition,
When the setting mode is switched to the execution mode, the stored examination setting data is read out, and image processing is performed on the image repeatedly acquired based on the trigger condition using the image processing tool. The image processing method according to claim 7. An imaging means for imaging an inspection object;
Image processing means for performing image processing of the inspection object from the image of the inspection object imaged by the imaging means;
A pass / fail determination means for determining pass / fail of the inspection object based on the processing result of the image processing means;
In a computer program that can be executed by an image processing sensor having output means for outputting the determination result of the pass / fail determination means to the outside,
The image processing means;
An area specifying means for specifying a rectangular area including a character or a character string to be recognized, which is included in the image of the imaged inspection object;
Character recognition means for recognizing characters in the specified rectangular area, and an evaluation value that calculates an evaluation value indicating the degree of density in the rectangular area and evaluates the degree of density based on the calculated evaluation value and a predetermined threshold value Function as a means,
A computer program that causes the pass / fail judgment means to function as a means for judging pass / fail of an inspection object based on a character recognized by the character recognition means and a shade degree evaluated by the shade evaluation means.