JP7575664B2 - Image generating device, quality judgement device, image generating method, quality judgement method and program - Google Patents

Description

The present invention relates to an image generating device, a quality determination device, an image generating method, a quality determination method, and a program.

Patent Document 1 describes a defect inspection method that uses a trained classifier to determine the presence or absence of defects in a tire from an inspection image in which each pixel is associated with a pixel value that represents the height (position in the unevenness direction) at a position along the inner surface of the tire. This inspection image is generated using light-section technology based on an image captured by a camera of a slit of light irradiated from a light source.

Patent document 2 describes an information processing system that can extract features of the shape of an object using a trained neural network.

JP 2019-27826 A JP 2019-95827 A

When using a trained classifier to determine whether or not a tire has a defect, if it were possible to obtain a multi-channel input image in which each pixel is associated with not only height as described in Patent Document 1, but also other physical quantities such as temperature, it is expected that the accuracy of the determination can be improved.

However, it is difficult to obtain a multi-channel input image in which each pixel is associated with the desired multiple physical quantities from a single sensor such as a camera.

The above not only applies to determining whether or not a tire has a defect using a trained classifier, but also generally applies to situations where multi-channel images are input to a trained machine learning model, such as determining whether a given object is good or bad using a trained machine learning model.

The present invention has been made in consideration of the above-mentioned circumstances, and one of its objectives is to provide an image generation device, a quality determination device, an image generation method, a quality determination method, and a program that can easily generate a multi-channel input image in which a desired number of physical quantities are associated with each pixel, and which is input to a trained machine learning model.

In order to solve the above problem, the image generating device according to the present invention is an image generating device that generates multi-channel input images to be input to a trained machine learning model and is capable of communicating with a first line sensor that sequentially generates a first type of line image showing a first physical quantity at a measurement timing at a first time interval and a second line sensor that sequentially generates a second type of line image showing a second physical quantity at a measurement timing at a second time interval, the image generating device including a first type of line image acquisition means that acquires the first type of line image and a second type of line image acquisition means that acquires the second type of line image. a line image selection means for selecting at least one of the first type line images and at least one of the second type line images corresponding to a given pixel array according to a measurement timing difference determined based on the separation distance between the first line sensor and the second line sensor and the moving speed of the object; and an input image generation means for generating the multi-channel input image by associating, with each pixel included in the pixel array, the pixel value of a pixel included in the selected first type line image that corresponds to the pixel, and the pixel value of a pixel included in the selected second type line image that corresponds to the pixel.

In one aspect of the present invention, the system further includes a resizing means for performing resizing at least one of the selected first type line image or the selected second type line image at a given resizing rate, and the input image generating means generates the input image based on the first type line image or the second type line image on which the resizing has been performed.

In one aspect of the present invention, the first physical quantity and the second physical quantity are different from each other and are either temperature, height, amount of reflected light, unevenness, gloss, or color.

The quality determination device according to the present invention is capable of communicating with a first line sensor that sequentially generates a first type of line image indicating a first physical quantity at a measurement timing at a first time interval, and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being spaced apart from each other along the direction of movement of an object moving at a constant speed in one direction, and the second line sensor is configured to acquire the first type of line image, acquire the second type of line image, and acquire the second type of line image based on a distance between the first line sensor and the second line sensor and the moving speed of the object. The system includes a line image selection means for selecting at least one of the first type line images and at least one of the second type line images corresponding to a given pixel array according to a measurement timing difference, an input image generation means for generating a multi-channel input image by associating, with each pixel included in the pixel array, a pixel value of a pixel included in the selected first type line image that corresponds to the pixel, and a pixel value of a pixel included in the selected second type line image that corresponds to the pixel, and a quality determination means for determining the quality of the object based on an output from a trained machine learning model when the input image is input to the machine learning model.

The image generating method according to the present invention is an image generating method in which an image generating device capable of communicating with a first line sensor that sequentially generates a first type of line image showing a first physical quantity at a measurement timing at a first time interval and a second line sensor that sequentially generates a second type of line image showing a second physical quantity at a measurement timing at a second time interval, which are arranged at a distance from each other along the movement direction of an object moving at a constant speed in one direction, generates a multi-channel input image to be input to a trained machine learning model, and includes a step of acquiring the first type of line image, a step of acquiring the second type of line image, a step of selecting at least one of the first type of line image and at least one of the second type of line image corresponding to a given pixel array according to a measurement timing difference determined based on the separation distance between the first line sensor and the second line sensor and the movement speed of the object, and a step of generating the multi-channel input image by associating each pixel included in the pixel array with a pixel value of a pixel included in the selected first type of line image that corresponds to the pixel and a pixel value of a pixel included in the selected second type of line image that corresponds to the pixel.

The quality determination method according to the present invention is a quality determination method executed by a quality determination device capable of communicating with a first line sensor that sequentially generates a first type of line image indicating a first physical quantity at a measurement timing at a first time interval and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being spaced apart from each other along the moving direction of an object moving at a constant speed in one direction, the method including the steps of acquiring the first type of line image, acquiring the second type of line image, and determining a quality determination value based on the distance between the first line sensor and the second line sensor and the moving speed of the object. The method includes a step of selecting at least one of the first type line images and at least one of the second type line images corresponding to a given pixel array according to the measurement timing difference to be measured, a step of generating a multi-channel input image by associating, with each pixel included in the pixel array, the pixel value of a pixel included in the selected first type line image that corresponds to the pixel, and the pixel value of a pixel included in the selected second type line image that corresponds to the pixel, and a step of judging the quality of the object based on the output from a trained machine learning model when the input image is input to the machine learning model.

The program according to the present invention causes a computer that generates multi-channel input images to be input to a trained machine learning model and that can communicate with a first line sensor that sequentially generates a first type of line image showing a first physical quantity at a measurement timing at a first time interval and a second line sensor that sequentially generates a second type of line image showing a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being spaced apart from each other along the moving direction of an object moving at a constant speed in one direction, to execute the steps of acquiring the first type of line image, acquiring the second type of line image, selecting at least one of the first type of line image and at least one of the second type of line image corresponding to a given pixel array according to a measurement timing difference determined based on the separation distance between the first line sensor and the second line sensor and the moving speed of the object, and generating the multi-channel input image by associating each pixel included in the pixel array with a pixel value of a pixel included in the selected first type of line image that corresponds to the pixel and a pixel value of a pixel included in the selected second type of line image that corresponds to the pixel.

In addition, another program according to the present invention causes a computer capable of communicating with a first line sensor that sequentially generates a first type of line image showing a first physical quantity at a measurement timing of a first time interval and a second line sensor that sequentially generates a second type of line image showing a second physical quantity at a measurement timing of a second time interval, which are arranged at a distance from each other along the movement direction of an object moving at a constant speed in one direction, to execute the following steps: acquiring the first type of line image; acquiring the second type of line image; selecting at least one of the first type of line image and at least one of the second type of line image corresponding to a given pixel array according to a measurement timing difference determined based on the separation distance between the first line sensor and the second line sensor and the movement speed of the object; generating a multi-channel input image by associating each pixel included in the pixel array with the pixel value of a pixel included in the selected first type of line image that corresponds to the pixel and the pixel value of a pixel included in the selected second type of line image that corresponds to the pixel; and judging the quality of the object based on the output from a machine learning model that has been trained when the input image is input to the machine learning model.

1 is a diagram showing an example of a configuration of a quality determination system according to an embodiment of the present invention; FIG. 10 is a diagram illustrating an example of a state in which an object is transported. FIG. 10 is a diagram illustrating an example of a state in which an object is transported. FIG. 10 is a diagram illustrating an example of a state in which an object is transported. FIG. 10 is a diagram illustrating an example of a state in which an object is transported. 11A to 11C are diagrams showing examples of a temperature measurement result image, a height measurement result image, and a reflected light amount capture result image generated in the calibration process. 11A to 11C are diagrams showing examples of a temperature measurement result image, a height measurement result image, and a reflected light amount photographing result image generated in the pass/fail determination process. 11A to 11C are diagrams showing examples of a temperature measurement result image, a height measurement result image, and a reflected light amount photographing result image generated in the pass/fail determination process. 11A to 11C are diagrams showing examples of a temperature measurement result image, a height measurement result image, and a reflected light amount photographing result image generated in the pass/fail determination process. 11A to 11C are diagrams showing examples of a temperature measurement result image, a height measurement result image, and a reflected light amount photographing result image generated in the pass/fail determination process. FIG. 2 is a functional block diagram showing an example of functions of an information processing device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a data structure of training data. FIG. 1 is a diagram illustrating an example of learning of a machine learning model. FIG. 13 is a diagram illustrating an example of pass/fail determination using a trained machine learning model. 10 is a flow diagram showing an example of the flow of a learning process performed by an information processing device according to an embodiment of the present invention. FIG. 4 is a flow diagram showing an example of the flow of a quality determination process performed in a quality determination system according to an embodiment of the present invention. FIG.

One embodiment of the present invention will be described below with reference to the drawings.

Figure 1 is a diagram showing an example of the configuration of a quality determination system 1 according to one embodiment of the present invention. The quality determination system 1 according to this embodiment includes, for example, an information processing device 10, a conveying unit 12, an encoder 14, a temperature line sensor 16, and a light cutting line sensor 18.

The information processing device 10 is, for example, a computer such as a personal computer. As shown in FIG. 1, the information processing device 10 includes, for example, a processor 10a, a memory unit 10b, a communication unit 10c, a display unit 10d, and an operation unit 10e.

The processor 10a is, for example, a program-controlled device such as a CPU that operates according to a program installed in the information processing device 10.

The storage unit 10b is, for example, a storage element such as a ROM or RAM, or a hard disk drive. The storage unit 10b stores programs executed by the processor 10a, etc.

The communication unit 10c is a communication interface such as a network board. In this embodiment, the information processing device 10 is capable of communicating with the transport unit 12, the temperature line sensor 16, and the light cutting line sensor 18 via the communication unit 10c.

The display unit 10d is a display device such as a liquid crystal display, and displays various images according to instructions from the processor 10a.

The operation unit 10e is a user interface such as a keyboard or mouse, which accepts user input and outputs a signal indicating the content of the input to the processor 10a.

The information processing device 10 may also include an optical disk drive that reads optical disks such as DVD-ROMs and Blu-ray (registered trademark) disks, a USB (Universal Serial Bus) port, etc.

The transport unit 12 is, for example, a transport device such as a conveyor, and transports the object 20 that is the subject of quality determination according to this embodiment, as shown in Figures 2A to 2D. In this embodiment, the object 20 is transported at a constant speed in one specific direction. In other words, in this embodiment, the degree of freedom of the transport direction of the object 20 is 1. Therefore, the object 20 moves at a constant speed in one direction.

For example, in this embodiment, as shown in Figures 2A to 2D, the transport unit 12 transports the object 20 at a constant speed in a direction from left to right in Figures 2A to 2D in response to a signal received from the information processing device 10. In this embodiment, the object 20 is transported from the position shown in Figure 2A to the positions shown in Figures 2B and 2C, and then to the position shown in Figure 2D.

An example of the object 20 in this embodiment is a tire.

The temperature line sensor 16 in this embodiment is a line sensor, such as an infrared line sensor, that can measure temperatures at multiple positions on a line.

The light cutting line sensor 18 according to this embodiment is a sensor device capable of measuring heights at multiple positions on a line, for example. For example, the light cutting line sensor 18 includes a camera 18a and a laser light source 18b that emits linear slit laser light. The light cutting line sensor 18 is capable of measuring heights at multiple positions on the line using a light cutting method based on the position at which the laser light emitted by the laser light source 18b appears in the image captured by the camera 18a.

In this embodiment, multiple line sensors (here, for example, a temperature line sensor 16 and a light cutting line sensor 18) are arranged spaced apart from each other above the object 20 being transported by the transport unit 12 along the direction of movement of the object 20.

In this embodiment, the temperature line sensor 16 is installed so that the line direction in which the temperature sensors are arranged in a row in the temperature line sensor 16 is perpendicular to the movement direction of the object 20 (the scanning direction of the temperature line sensor 16). Also, in this embodiment, the light section line sensor 18 is installed so that the direction in which the light sources are arranged in a row in the light section line sensor 18 is perpendicular to the movement direction of the object 20 (the scanning direction of the light section line sensor 18).

Then, as shown in FIG. 2B, when the object 20 passes the sensing position of the temperature line sensor 16, the surface temperature of the object 20 is scanned by the temperature line sensor 16.

As shown in FIG. 2C, when the object 20 passes the sensing position of the light section line sensor 18, the light section line sensor 18 scans the height of the object 20 (the position of the object 20 in the vertical direction in FIGS. 2A to 2D). In this embodiment, the object 20, such as a tire, has unevenness on its surface, and the light section line sensor 18 measures this unevenness.

In addition to measuring the height of the object 20, the light-section line sensor 18 according to this embodiment also measures the amount of reflected light at the position where the irradiated laser light appears in the image captured by the camera 18a. By also measuring the amount of reflected light, it is possible to obtain measurement results that cannot be obtained by measuring the height alone.

In this embodiment, the distance traveled by the transport unit 12 to transport the object 20 is measured by an encoder 14 such as a linear encoder. The encoder 14 may be a rotary encoder, and the encoder 14 may measure the rotation angle of the drive motor that drives the transport unit 12.

In this embodiment, each time the transport unit 12 transports the object 20 a predetermined transport distance, a signal generated by the encoder 14 is input from the encoder 14 to the temperature line sensor 16 and the light section line sensor 18. Here, for example, the signal may be input from the encoder 14 to the temperature line sensor 16 and the light section line sensor 18 via a signal distributor (not shown).

The temperature line sensor 16 performs temperature measurements at the measurement timing of the first time interval, and sequentially generates temperature line images showing the temperatures resulting from the measurements.

Here, the temperature line image is an image in which pixels representing a single temperature measurement result are arranged in a line, generated based on the result of the single temperature measurement by the temperature line sensor 16. The pixel value of each pixel included in the temperature line image is associated with a temperature.

The light cutting line sensor 18 also measures the height and the amount of reflected light at the measurement timing at the second time interval, and sequentially generates a height line image showing the height resulting from the measurement, and a reflected light amount line image showing the amount of reflected light resulting from the measurement.

Here, the height line image is an image in which pixels representing a single height measurement result are arranged in a line, generated based on the result of the single height measurement by the light cutting line sensor 18. The pixel value of each pixel included in the height line image is associated with a height.

The reflected light amount line image is an image in which pixels representing a single measurement result of the reflected light amount by the light cutting line sensor 18 are arranged in a line, and is generated based on the result of the measurement. The pixel value of each pixel included in the reflected light amount line image is associated with the reflected light amount.

Here, for example, the temperature line sensor 16 may measure the temperature in response to a signal input from the encoder 14. Also, the light cutting line sensor 18 may measure the height and the amount of reflected light in response to a signal input from the encoder 14. Thus, in this embodiment, the timing of measurements by the multiple line sensors may be synchronized. Furthermore, measurements by the multiple line sensors may be performed at the same timing. Note that in the following description, measurements by the multiple line sensors are assumed to be performed at the same timing.

In this embodiment, a pass/fail judgment of the object 20 is performed based on the temperature line image, height line image, and reflected light amount line image that show the measurement results of the object 20.

In addition, in this embodiment, a calibration process is performed beforehand to determine whether the target object 20 is good or bad. The calibration process is described below.

In the calibration process, for example, an object for calibration (hereinafter referred to as a calibration object) moves in one direction at a constant speed, similar to the target object 20 described above. Then, multiple line images (here, for example, multiple temperature line images, multiple height line images, and multiple reflected light amount line images) showing the measurement results of the calibration object are generated.

In this embodiment, a measurement result image is generated based on the line image generated in this manner. Figure 3 shows a temperature measurement result image 30, a height measurement result image 32, and a reflected light amount measurement result image 34, which are examples of measurement result images for a calibration object.

In this embodiment, for example, a temperature measurement result image 30 is generated in which multiple temperature line images generated by the temperature line sensor 16 are arranged side by side. In the temperature measurement result image 30 shown in the example of FIG. 3, temperature line images in which pixels representing the results of one temperature measurement are arranged vertically are arranged from left to right in order of earliest measurement timing.

In addition, in this embodiment, for example, a height measurement result image 32 is generated in which multiple height line images generated by the light cutting line sensor 18 are arranged side by side. In the temperature measurement result image 30 shown in the example of Figure 3, height line images in which pixels representing one height measurement result are arranged vertically in a row are arranged from left to right in order of the earliest measurement timing.

In addition, in this embodiment, for example, a reflected light amount measurement result image 34 is generated in which a plurality of reflected light amount line images generated by the light cutting line sensor 18 are arranged side by side. In the reflected light amount measurement result image 34 shown in the example of FIG. 3, the reflected light amount line images in which pixels representing the measurement results of one reflected light amount are arranged vertically in a row are arranged from left to right in order of the earliest measurement timing.

In the following explanation, for each measurement result image, the upper left is the origin, the right direction is the positive X-axis direction, and the downward direction is the positive Y-axis direction.

Figure 3 also shows a calibration area, which is an area in the measurement result image that corresponds to the measurement result of the calibration object. For example, Figure 3 shows a temperature calibration area 36, which is an area in the temperature measurement result image 30 that corresponds to the measurement result of the surface temperature of the calibration object. In this embodiment, for example, a closed area in the temperature measurement result image 30 that includes pixels whose pixel values are within a predetermined range is identified as the temperature calibration area 36. Note that the temperature calibration area 36 in the temperature measurement result image 30 may be identified using a known image recognition technique.

FIG. 3 also shows a height calibration area 38, which is an area in the height measurement result image 32 that corresponds to the measurement result of the height of the calibration object. In this embodiment, for example, a closed area in the height measurement result image 32 that includes pixels whose pixel values are within a predetermined range is identified as the height calibration area 38. Note that the height calibration area 38 in the height measurement result image 32 may be identified using a known image recognition technique.

Figure 3 also shows a reflected light amount calibration area 40, which is an area in the reflected light amount measurement result image 34 that corresponds to the measurement result of the reflected light amount of the calibration object. In this embodiment, for example, a closed area in the reflected light amount measurement result image 34 that includes pixels whose pixel values are within a predetermined range is identified as the reflected light amount calibration area 40. Note that the reflected light amount calibration area 40 in the reflected light amount measurement result image 34 may be identified using a known image recognition technique.

In this embodiment, as described above, the temperature line sensor 16 and the light cutting line sensor 18 perform measurements synchronously each time the calibration object is transported a predetermined transport distance by the transport unit 12. Therefore, the number of horizontal pixels (width) of the temperature calibration area 36, the height calibration area 38, and the reflected light amount calibration area 40 are all the same. In the example of FIG. 3, the number of pixels in this width is represented by w.

Then, a short time after the calibration object passes under the temperature line sensor 16 as shown in FIG. 2B, the calibration object passes under the light cutting line sensor 18 as shown in FIG. 2C. Therefore, as shown in FIG. 3, the position of the temperature calibration area 36 in the temperature measurement result image 30 is to the left of the position of the height calibration area 38 in the height measurement result image 32. Also, the position of the temperature calibration area 36 in the temperature measurement result image 30 is to the left of the position of the reflected light amount calibration area 40 in the reflected light amount measurement result image 34.

In the calibration process, the number of pixels D of the left-right shift between the position of the temperature calibration area 36 and the position of the height calibration area 38 shown in the example of FIG. 3 is identified as the shift amount D. Note that in this embodiment, the number of pixels of the left-right shift between the position of the temperature calibration area 36 and the position of the reflected light amount calibration area 40 is also D.

In addition, due to the difference in resolution between the temperature line sensor 16 and the light section line sensor 18, the number of vertical pixels in the temperature calibration area 36 is different from the number of vertical pixels in the height calibration area 38 and the reflected light amount calibration area 40. In FIG. 3, the number of vertical pixels in the temperature calibration area 36 is represented by h, and the number of vertical pixels in the height calibration area 38 and the reflected light amount calibration area 40 is represented by H.

Then, in the calibration process, the resize ratio r is determined based on the number of pixels h and the number of pixels H. Here, for example, the value of h/H is determined as the resize ratio r.

In this embodiment, the calibration data indicating the shift amount D and the size change rate r determined by the calibration process as described above is stored in the storage unit 10b. This calibration data is then used to determine whether the object 20 is good or bad. This calibration data can be commonly used in determining whether each of the multiple objects 20 is good or bad. The quality determination process is described below.

In the process of determining whether the object 20 is good or bad, multiple line images (here, for example, multiple temperature line images, multiple height line images, and multiple reflected light amount line images) are generated that show the measurement results of the object 20 moving at a constant speed in one direction.

In this embodiment, a measurement result image is generated based on the line image generated in this manner. Figure 4A shows a temperature measurement result image 50, a height measurement result image 52, and a reflected light amount measurement result image 54, which are examples of measurement result images for the object 20.

Figure 4A also shows an object region that is an area corresponding to the measurement result of the object 20 in the measurement result image. For example, Figure 4A shows a temperature object region 56 that is an area corresponding to the measurement result of the surface temperature of the object 20 in the temperature measurement result image 50. Figure 4A also shows a height object region 58 that is an area corresponding to the measurement result of the height of the object 20 in the height measurement result image 52. Figure 4A also shows a reflected light amount object region 60 that is an area corresponding to the measurement result of the reflected light amount of the object 20 in the reflected light amount measurement result image 54.

As explained with reference to FIG. 3, the number of horizontal pixels (width) of the temperature object area 56, the height object area 58, and the reflected light amount object area 60 are all the same.

In this embodiment, the temperature measurement result image 50, the height measurement result image 52, and the reflected light quantity measurement result image 54 are aligned based on the calibration data. For example, the height measurement result image 52 and the reflected light quantity measurement result image 54 are aligned by shifting them left and right based on the shift amount D indicated by the calibration data. For example, the height measurement result image 52 and the reflected light quantity measurement result image 54 are aligned by deleting D line images from the left. FIG. 4B shows the height measurement result image 52 and the reflected light quantity measurement result image 54 after this deletion. Note that FIG. 4B also shows a temperature measurement result image 50 similar to FIG. 4A. This deletion causes the ranges of the X coordinate values of the temperature object region 56, the height object region 58, and the reflected light quantity object region 60 to be the same.

In this embodiment, for example, based on the resizing rate r indicated by the calibration data, the height measurement result image 32 and the reflected light quantity measurement result image 34 are resized (here, reduced) so that their vertical lengths become r times larger. The situation after this resizing is shown in FIG. 4C. As a result, the vertical lengths of the temperature object region 56, the height object region 58, and the reflected light quantity object region 60 become the same.

In this embodiment, for example, for each of the temperature measurement result image 50, the height measurement result image 52, and the reflected light quantity measurement result image 54 shown in FIG. 4C, the portions having a common range of X coordinate values are cut out. For example, the portions shown in FIG. 4C having X coordinate values of x1 or more and x2 or less are cut out. The temperature measurement result image 50, the height measurement result image 52, and the reflected light quantity measurement result image 54 cut out in this manner are shown in FIG. 4D.

Here, for example, for any one of the measurement result images, the object region within that measurement result image may be identified. For example, the temperature object region 56 within the temperature measurement result image 50 shown in FIG. 4C may be identified. Here, for example, a closed region within the temperature measurement result image 50 that includes pixels with pixel values within a predetermined range may be identified as the temperature object region 56. Alternatively, for example, the temperature object region 56 within the temperature measurement result image 50 may be identified using a known image recognition technique.

In this embodiment, for example, the X coordinate value x1 of the left side and the X coordinate value x2 of the right side of the temperature object region 56 are determined in this manner.

Then, portions of the temperature measurement result image 50, the height measurement result image 52, and the reflected light quantity measurement result image 54 whose X coordinate values are greater than or equal to x1 and less than or equal to x2 may be cut out.

In this embodiment, a multi-channel input image in which multiple pixel values are associated with each pixel is generated based on the temperature measurement result image 50, height measurement result image 52, and reflected light amount measurement result image 54 shown in FIG. 4D that are extracted in this manner. Here, for example, a multi-channel input image is generated in which each pixel is associated with a pixel value associated with temperature, a pixel value associated with height, and a pixel value associated with the reflected light amount. Here, for example, a multi-channel input image in which the number of vertical and horizontal pixels is a predetermined number may be generated. Note that there is no particular restriction on the implementation of the multi-channel input image, and for example, the input image may be a single image in which each pixel has multiple pixel values, or may be multiple images associated with each other in which the associated pixels each have one pixel value.

Here, for example, a height object region 58 may be identified in the height measurement result image 52 shown in FIG. 4D. Here, for example, a closed region including pixels with pixel values within a predetermined range in the height measurement result image 52 may be identified as the height object region 58. Alternatively, for example, the height object region 58 may be identified in the height measurement result image 52 using a known image recognition technique.

For example, the reflected light quantity object area 60 may be identified in the reflected light quantity measurement result image 54 shown in FIG. 4D. Here, for example, a closed area in the reflected light quantity measurement result image 54 that includes pixels with pixel values within a predetermined range is identified as the reflected light quantity object area 60. Alternatively, for example, the reflected light quantity object area 60 may be identified in the reflected light quantity measurement result image 54 using a known image recognition technique.

The temperature object area 56, the height object area 58, and the reflected light amount object area 60 identified in the above manner all have the same number of vertical and horizontal pixels.

A multi-channel input image may then be generated in which each pixel is associated with the pixel value of the pixel associated with that pixel in the temperature object region 56, the pixel value of the pixel associated with that pixel in the height object region 58, and the pixel value of the pixel associated with that pixel in the reflected light quantity object region 60.

For example, the three images shown in FIG. 4D may be shifted vertically by a predetermined shift amount, and rectangular areas with the same coordinate values may be cut out and pixels with the same coordinate values may be associated with each other to generate a multi-channel input image. Here, for example, the image of the cut-out rectangular area may be transformed or resized so that the number of vertical and horizontal pixels is the above-mentioned predetermined number of pixels, thereby generating a multi-channel input image.

According to this embodiment, by doing the above, it is possible to easily generate a multi-channel input image in which each pixel is associated with multiple desired physical quantities.

In this embodiment, the quality of the object 20 is determined based on the output when this multi-channel input image is input to a trained convolutional neural network (CNN).

The following provides further explanation of the functions implemented in the information processing device 10 and the processing performed by the quality determination system 1.

Figure 5 is a functional block diagram showing an example of functions implemented in the information processing device 10 according to this embodiment. Note that it is not necessary for all of the functions shown in Figure 5 to be implemented in the information processing device 10 according to this embodiment, and functions other than the functions shown in Figure 5 may also be implemented.

As shown in FIG. 5, the information processing device 10 functionally includes, for example, a machine learning model 70, a training data storage unit 72, a learning unit 74, a calibration data storage unit 76, a line image acquisition unit 78, a line image selection unit 80, an input image generation unit 82, and a pass/fail determination unit 84.

The machine learning model 70 is implemented primarily in the processor 10a and the memory unit 10b. The training data memory unit 72 and the calibration data memory unit 76 are implemented primarily in the memory unit 10b. The learning unit 74, the line image selection unit 80, the input image generation unit 82, and the pass/fail judgment unit 84 are implemented primarily in the processor 10a. The line image acquisition unit 78 is implemented primarily in the communication unit 10c.

The above functions may be implemented by executing a program including instructions corresponding to the above functions, which is installed in the information processing device 10, which is a computer, on the processor 10a. This program may be supplied to the information processing device 10 via a computer-readable information storage medium, such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet, for example.

In addition, in the information processing device 10 according to this embodiment, learning of the machine learning model 70 is performed. Then, the trained machine learning model 70 is used to determine whether the target object 20 is good or bad.

The following describes the functions of the information processing device 10 in learning the machine learning model 70.

The machine learning model 70 is a machine learning model such as CNN. Note that the machine learning model 70 does not have to be CNN.

In this embodiment, the training data storage unit 72 stores multiple pieces of training data, an example of which is shown in FIG. 6.

As shown in FIG. 6, the training data in this embodiment includes, for example, a training data ID, a learning input image, and teacher data.

In this embodiment, a multi-channel input image is generated for each of a plurality of positive example samples and a plurality of negative example samples in the same manner as described above. Hereinafter, the input image generated in this manner for learning the machine learning model 70 will be referred to as a learning input image. Hereinafter, for example, a learning input image is generated in which each pixel is associated with a pixel value corresponding to temperature, a pixel value corresponding to height, and a pixel value corresponding to the amount of reflected light based on the measurement results of temperature, height, and reflected light amount. Each of the learning input images according to this embodiment has the same number of vertical and horizontal pixels as the input image generated based on the measurement results of the object 20 to be judged as good or bad. Here, as described above, a learning input image having the above-mentioned predetermined number of vertical and horizontal pixels may be generated.

Here, a positive example sample refers to a sample of the object 20 that should be judged as good in a quality determination using the trained machine learning model 70. Also, a negative example sample refers to a sample of the object 20 that should be judged as bad in a quality determination using the trained machine learning model 70.

Then, training data is generated that includes a training data ID associated with the sample, a learning input image generated based on the sample, and teacher data corresponding to the sample. Here, for example, a value indicating a positive example (e.g., 1) is set as the value of the teacher data associated with the positive example sample. Also, a value indicating a negative example (e.g., 0) is set as the value of the teacher data associated with the negative example sample.

The training data storage unit 72 then stores training data associated with each of the multiple positive example samples generated in the above manner, and training data associated with each of the multiple negative example samples.

Note that the above-mentioned method is merely one example of a method for generating training data, and the training data storage unit 72 may store training data generated by a method other than the above-mentioned method.

In this embodiment, for example, the learning unit 74 performs learning of the machine learning model 70 using the training data stored in the training data storage unit 72.

As shown in FIG. 7, the learning unit 74 inputs, for example, a learning input image included in the training data to the machine learning model 70. The learning unit 74 then obtains output data indicating the output from the machine learning model 70 in response to the input. The learning unit 74 then identifies the difference between the value of the output data and the value of the teacher data included in the training data. The learning unit 74 then performs supervised learning to update the parameter values of the machine learning model 70 using the backpropagation method so that the value of the loss function associated with the identified difference is minimized.

In this way, a trained machine learning model 70 is generated. Then, as described above, this trained machine learning model 70 is used to determine whether the target object 20 is good or bad.

The following describes the function of the information processing device 10 in determining whether the object 20 is good or bad using the trained machine learning model 70.

In this embodiment, the calibration data storage unit 76 stores the above-mentioned calibration data that is generated by a preliminary calibration process, for example.

In this embodiment, the line image acquisition unit 78 acquires, for example, a first type of line image (a temperature line image in the above example) generated by a first line sensor (the temperature line sensor 16 in the above example).

In addition, in this embodiment, the line image acquisition unit 78 acquires a second type of line image (a height line image and a reflected light amount line image in the above example) generated by a second line sensor (the light cutting line sensor 18 in the above example), for example.

In this embodiment, the line image selection unit 80 selects, for example, at least one first type line image and at least one second type line image that correspond to a given pixel array that corresponds to the input image. Here, for example, a temperature line image, a height line image, and a reflected light amount line image whose X coordinate value range is from x1 to x2 as shown in FIG. 4C are selected.

As described above, the line image selection unit 80 may generate a measurement result image. Then, the line image selection unit 80 may select at least one line image that is part of the measurement result image.

In this embodiment, the selected line image corresponds to the measurement timing difference determined based on the separation distance between the first line sensor and the second line sensor and the moving speed of the target object 20. This measurement timing difference corresponds to, for example, the shift amount D indicated by the calibration data stored in the calibration data storage unit 76. In other words, the line image selection unit 80 may select a line image based on the shift amount D indicated by the calibration data stored in the calibration data storage unit 76.

As described above, the difference in measurement timing (shift amount D in the above example) may be determined based on the measurement result image generated in the calibration process. Also, the user may determine the shift amount D by calculation based on the separation distance between the temperature line sensor 16 and the light cutting line sensor 18 and the moving speed of the object 20 (the conveying speed of the object 20 by the conveying unit 12).

In this embodiment, the input image generating unit 82 generates a multi-channel input image by associating, for example, each pixel included in the pixel array corresponding to the input image with a pixel value included in the selected first type line image that corresponds to that pixel and a pixel value included in the selected second type line image that corresponds to that pixel. Here, for example, the input image is generated by associating, for example, each pixel included in the pixel array corresponding to the input image with a pixel value included in the selected temperature line image that corresponds to that pixel, a pixel value included in the selected height line image that corresponds to that pixel, and a pixel value included in the selected reflected light amount line image that corresponds to that pixel.

The physical quantities measured by the line sensor are not limited to the temperature, height, and amount of reflected light described above. For example, the line sensor may identify unevenness, gloss, or color. The physical quantity indicated by the pixel value of a pixel included in the input image may be unevenness, gloss, or color. In other words, the pixel value of the input image indicates a physical quantity such as temperature, height, amount of reflected light, unevenness, gloss, color, etc.

As described above, the line image selection unit 80 may also resize at least one line image at a given resize ratio (here, for example, resize ratio r indicated by the calibration data stored in the calibration data storage unit 76). In the above example, resize is performed at the resize ratio r on the vertical lengths of the height measurement result image 52 including multiple height line images and the reflected light amount measurement result image 54 including multiple reflected light amount line images.

Then, the input image generation unit 82 may generate an input image based on the line image that has been resized.

In this embodiment, the pass/fail judgment unit 84 judges the pass/fail quality of the object 20 based on the output from the trained machine learning model 70 when a multi-channel input image is input to the trained machine learning model 70. Here, for example, as shown in FIG. 8, a multi-channel input image generated based on the measurement results of the physical quantities of the object 20 is input to the trained machine learning model 70. Then, the pass/fail quality of the object 20 is judged based on the value of the output data output from the machine learning model 70 in response to the input. Here, for example, if the value of the output data is equal to or greater than a predetermined threshold (e.g., 0.5), the object 20 is judged to be pass/fail, and if not, the object 20 is judged to be fail/fail. The pass/fail judgment unit 84 may notify the user of the pass/fail judgment result.

Here, an example of the flow of the learning process performed by the information processing device 10 according to this embodiment will be described with reference to the flow diagram illustrated in FIG. 9. In this processing example, it is assumed that multiple pieces of training data are stored in advance in the training data storage unit 72.

First, the learning unit 74 selects one piece of training data from among the training data stored in the training data storage unit 72, for which the process shown in S102 has not been performed (S101).

Then, the learning unit 74 performs learning of the machine learning model 70 using the training data selected in the process shown in S101 (S102).

Then, the learning unit 74 checks whether the process shown in S102 has been executed for all the training data stored in the training data storage unit 72 (S103). If it is confirmed that the process has not been executed (S103: N), the process returns to the process shown in S101. On the other hand, if it is confirmed that the process has been executed (S103: Y), the process shown in this processing example is terminated.

Next, an example of the flow of the quality determination process of the target object 20 using the trained machine learning model 70, which is performed in the quality determination system 1 according to this embodiment, will be described with reference to the flow diagram illustrated in FIG. 10. Note that in this processing example, it is assumed that the above-mentioned calibration process has been performed in advance, and that calibration data indicating the shift amount D and the size change rate r has been stored in the calibration data storage unit 76.

When the transportation of the object 20 starts, the temperature line sensor 16 and the light cutting line sensor 18 wait for the measurement timing to arrive by waiting for the input of a signal from the encoder 14 (S201).

When the temperature line sensor 16 and the light cutting line sensor 18 detect the arrival of the measurement timing by receiving a signal from the encoder 14 (S201: Y), the temperature line sensor 16 and the light cutting line sensor 18 execute a measurement process (S202). In this processing example, a line image is generated by this measurement process. As described above, the timing of the measurement process is synchronized between the temperature line sensor 16 and the light cutting line sensor 18.

Then, the temperature line sensor 16 checks whether the number of times the measurement process shown in S202 has been performed has reached a predetermined number (S203).

If the measurement process has not been executed the predetermined number of times (S203: N), the process returns to S201.

When the number of times the measurement process has been performed reaches a predetermined number (S203: Y), the line image acquisition unit 78 acquires line images generated by the predetermined number of measurements up to that point from the temperature line sensor 16 and the light cutting line sensor 18 (S204). In the process shown in S204, for example, the temperature line sensor 16 may transmit multiple temperature line images to the information processing device 10. Then, the line image acquisition unit 78 of the information processing device 10 may receive the multiple transmitted temperature line images. Also, the light cutting line sensor 18 may transmit multiple height line images and multiple reflected light amount line images to the information processing device 10. Then, the line image acquisition unit 78 of the information processing device 10 may receive the multiple height line images and multiple reflected light amount line images. Note that the line images are associated with a timestamp indicating the order in which the line images were generated or the time when the line images were generated.

Then, the line image selection unit 80 generates a measurement result image based on the line image acquired in the process shown in S204 (S205). Here, for example, a temperature measurement result image 50 is generated based on multiple temperature line images, a height measurement result image 52 is generated based on multiple height line images, and a reflected light amount measurement result image 54 is generated based on multiple reflected light amount line images.

Then, the line image selection unit 80 aligns the height measurement result image 52 and the reflected light quantity measurement result image 54 based on the shift amount D indicated by the calibration data stored in the calibration data storage unit 76 (S206). As described above, the height measurement result image 52 and the reflected light quantity measurement result image 54 are aligned by deleting D line images from the left.

Then, the line image selection unit 80 resizes the height measurement result image 52 and the reflected light quantity measurement result image 54 based on the resize rate r indicated by the calibration data stored in the calibration data storage unit 76 (S207). Here, as described above, the vertical lengths of the height measurement result image 52 and the reflected light quantity measurement result image 54 are multiplied by r.

Then, the line image selection unit 80 selects a line image (S208). As described above, the line image selection is performed by cutting out the portions of the temperature measurement result image 50, the height measurement result image 52, and the reflected light quantity measurement result image 54 illustrated in FIG. 4C, each of which has an X coordinate value of at least x1 and at most x2.

Then, the input image generation unit 82 generates a multi-channel input image based on the line image selected in the process shown in S208 (S209).

Then, the pass/fail judgment unit 84 inputs the multi-channel input image generated by the process shown in S209 into the trained machine learning model 70 (S210).

Then, the quality determination unit 84 determines whether the transported object 20 is good or bad based on the output from the machine learning model 70 in response to the input in the process shown in S210 (S211). Here, for example, the quality of the object 20 is determined based on the value of the output data output from the machine learning model 70 in response to the input of an input image.

Then, the processing shown in this processing example ends.

The present invention is not limited to the above-described embodiments.

For example, the above-mentioned process may be performed for each of a number of objects 20 that are transported in succession, and the pass/fail determination process may be performed for the multiple objects 20 in succession.

In addition, multiple multi-channel input images may be generated based on a single measurement result image showing the results of successive measurements on multiple objects 20.

In addition, the measurement timing of the temperature line sensor 16 and the light cutting line sensor 18 does not need to be synchronized, and at least one first type line image and at least one second type line image whose measurement positions on the object 20 correspond to each other may be selected. A multi-channel input image may then be generated based on the first type line image and the second type line image selected in this manner.

In addition, in this embodiment, the number of line sensors that generate the line image is not limited to two, and a multi-channel input image may be generated based on line images generated by three or more line sensors that are arranged at a distance from each other.

The division of roles among the information processing device 10, the temperature line sensor 16, and the light cutting line sensor 18 is not limited to the above. For example, the temperature line sensor 16 may generate the temperature measurement result image 30 and the temperature measurement result image 50 instead of the information processing device 10. Also, the light cutting line sensor 18 may generate the height measurement result image 32, the reflected light quantity measurement result image 34, the height measurement result image 52, and the reflected light quantity measurement result image 54 instead of the information processing device 10.

Furthermore, the specific numerical values and character strings above, as well as the specific numerical values and character strings in the drawings, are examples and are not limited to these numerical values and character strings.

1. Quality determination system; 10. Information processing device; 10a. Processor; 10b. Memory unit; 10c. Communication unit; 10d. Display unit; 10e. Operation unit; 12. Conveying unit; 14. Encoder; 16. Temperature line sensor; 18. Light cutting line sensor; 18a. Camera; 18b. Laser light source; 20. Object; 30. Temperature measurement result image; 32. Height measurement result image; 34. Reflected light quantity measurement result image; 36. Temperature calibration area; 38. Height calibration area; 40. Reflected light quantity calibration area; 50. Temperature measurement result image; 52. Height measurement result image; 54. Reflected light quantity measurement result image; 56. Temperature object area; 58. Height object area; 60. Reflected light quantity object area; 70. Machine learning model; 72. Training data memory unit; 74. Learning unit; 76. Calibration data memory unit; 78. Line image acquisition unit; 80. Line image selection unit; 82. Input image generation unit, 84 pass/fail determination unit.

Claims (8)

An image generating device capable of communicating with a first line sensor that sequentially generates a first type of line image indicating a first physical quantity at a measurement timing at a first time interval, and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being spaced apart from each other along a moving direction of an object moving in one direction at a constant speed, the image generating device generating a multi-channel input image to be input to a trained machine learning model,
a first type line image acquisition means for acquiring the first type line image;
a second type line image acquisition means for acquiring the second type line image;
a means for generating a first type measurement result image in which the first type line images acquired in the order of measurement timing at the first time interval are arranged in the order of the measurement timing;
a means for generating a second type measurement result image in which the second type line images acquired in the order of measurement timing at the second time interval are arranged in the order of the measurement timing;
a means for identifying a first region of the object for calibration based on the first type measurement result image showing the object for calibration, and for identifying a second region of the object for calibration based on the second type measurement result image showing the object for calibration;
a means for specifying a measurement timing difference determined based on a separation distance between the first line sensor and the second line sensor and a moving speed of the object, based on a position of the first area in the first type measurement result image and a position of the second area in the second type measurement result image;
a line image selection means for selecting at least one of the first type line image and at least one of the second type line image, which indicate the object for pass/fail judgment and correspond to a given pixel array according to the measurement timing difference ;
an input image generating means for generating the multi-channel input image by associating, with each pixel included in the pixel array, a pixel value of a pixel included in the selected first type line image and associated with the pixel, and a pixel value of a pixel included in the selected second type line image and associated with the pixel;
13. An image generating device comprising: a resizing unit for performing resizing on at least one of the selected first type line image and the selected second type line image by a given resizing ratio;
the input image generating means generates the input image based on the first type line image or the second type line image on which the size change has been performed.
2. The image generating device according to claim 1. The first physical quantity and the second physical quantity are different from each other and are any one of temperature, height, amount of reflected light, unevenness, gloss, and color.
3. An image generating device according to claim 1 or 2. A quality determination device capable of communicating with a first line sensor that sequentially generates a first type of line image indicating a first physical quantity at a measurement timing at a first time interval, and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being spaced apart from each other along a moving direction of an object moving in one direction at a constant speed,
a first type line image acquisition means for acquiring the first type line image;
a second type line image acquisition means for acquiring the second type line image;
a means for generating a first type measurement result image in which the first type line images acquired in the order of measurement timing at the first time interval are arranged in the order of the measurement timing;
a means for generating a second type measurement result image in which the second type line images acquired in the order of measurement timing at the second time interval are arranged in the order of the measurement timing;
a means for identifying a first region of the object for calibration based on the first type measurement result image showing the object for calibration, and for identifying a second region of the object for calibration based on the second type measurement result image showing the object for calibration;
a means for specifying a measurement timing difference determined based on a separation distance between the first line sensor and the second line sensor and a moving speed of the object, based on a position of the first area in the first type measurement result image and a position of the second area in the second type measurement result image;
a line image selection means for selecting at least one of the first type line image and at least one of the second type line image, which indicate the object for pass/fail judgment and correspond to a given pixel array according to the measurement timing difference ;
an input image generating means for generating a multi-channel input image by associating, with each pixel included in the pixel array, a pixel value of a pixel included in the selected first type line image and associated with the pixel, and a pixel value of a pixel included in the selected second type line image and associated with the pixel;
a quality determination means for determining whether the object for quality determination is good or bad based on an output from a trained machine learning model when the input image is input to the trained machine learning model;
A quality determination device comprising: An image generation method in which an image generation device capable of communicating with a first line sensor that sequentially generates a first type of line image indicating a first physical quantity at a measurement timing at a first time interval and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being disposed apart from each other along a moving direction of an object moving in one direction at a constant speed, generate multi-channel input images to be input to a trained machine learning model, the method comprising:
acquiring the first type line image;
acquiring the second type line image;
generating a first type measurement result image in which the first type line images acquired in the order of measurement timing at the first time interval are arranged in the order of the measurement timing;
generating a second type measurement result image in which the second type line images acquired in the order of measurement timing at the second time interval are arranged in the order of the measurement timing;
identifying a first region of the object for calibration based on the first type measurement result image showing the object for calibration, and identifying a second region of the object for calibration based on the second type measurement result image showing the object for calibration;
specifying a measurement timing difference determined based on a separation distance between the first line sensor and the second line sensor and a moving speed of the object based on a position of the first area in the first type measurement result image and a position of the second area in the second type measurement result image;
selecting at least one of the first type line images and at least one of the second type line images, which correspond to a given pixel array according to the measurement timing difference and indicate the object for pass/fail determination ;
generating the multi-channel input image by associating, with each pixel included in the pixel array, a pixel value of a pixel included in the selected first type line image and associated with the pixel, and a pixel value of a pixel included in the selected second type line image and associated with the pixel;
13. An image generating method comprising: A quality determination method executed by a quality determination device capable of communicating with a first line sensor that is arranged apart from each other along a moving direction of an object moving in one direction at a constant speed, the first line sensor sequentially generating a first type of line image indicating a first physical quantity at a measurement timing at a first time interval, and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being disposed apart from each other along a moving direction of an object moving in one direction at a constant speed, the method comprising:
acquiring the first type line image;
acquiring the second type line image;
generating a first type measurement result image in which the first type line images acquired in the order of measurement timing at the first time interval are arranged in the order of the measurement timing;
generating a second type measurement result image in which the second type line images acquired in the order of measurement timing at the second time interval are arranged in the order of the measurement timing;
identifying a first region of the object for calibration based on the first type measurement result image showing the object for calibration, and identifying a second region of the object for calibration based on the second type measurement result image showing the object for calibration;
specifying a measurement timing difference determined based on a separation distance between the first line sensor and the second line sensor and a moving speed of the object based on a position of the first area in the first type measurement result image and a position of the second area in the second type measurement result image;
selecting at least one of the first type line images and at least one of the second type line images, which correspond to a given pixel array according to the measurement timing difference and indicate the object for pass/fail determination ;
generating a multi-channel input image by associating, with each pixel included in the pixel array, a pixel value of a pixel included in the selected first type line image and associated with the pixel, and a pixel value of a pixel included in the selected second type line image and associated with the pixel;
A step of determining whether the object for quality determination is good or bad based on an output from a trained machine learning model when the input image is input into the trained machine learning model;
A method for determining acceptability, comprising: a computer that generates a multi-channel input image to be input to a trained machine learning model and that is capable of communicating with a first line sensor that sequentially generates a first type of line image indicating a first physical quantity at a measurement timing at a first time interval, and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing at a second time interval, the first line sensor and the second line sensor being spaced apart from each other along a moving direction of an object moving in one direction at a constant speed;
obtaining the first type line image;
obtaining the second type line image;
a step of generating a first type measurement result image in which the first type line images acquired in the order of measurement timing at the first time interval are arranged in the order of the measurement timing;
a step of generating a second type measurement result image in which the second type line images acquired in the order of measurement timing at the second time interval are arranged in the order of the measurement timing;
specifying a first region of the object for calibration based on the first type measurement result image showing the object for calibration, and specifying a second region of the object for calibration based on the second type measurement result image showing the object for calibration;
specifying a measurement timing difference determined based on a separation distance between the first line sensor and the second line sensor and a moving speed of the object, based on a position of the first area in the first type measurement result image and a position of the second area in the second type measurement result image;
selecting at least one of the first type line images and at least one of the second type line images, which correspond to a given pixel array according to the measurement timing difference and show the object for quality determination ;
a step of generating the multi-channel input image by associating, with each pixel included in the pixel array, a pixel value of a pixel included in the selected first type line image and associated with the pixel, and a pixel value of a pixel included in the selected second type line image and associated with the pixel;
A program characterized by executing the above. a computer capable of communicating with a first line sensor that is arranged apart from each other along a moving direction of an object moving in one direction at a constant speed, the first line sensor sequentially generating a first type of line image indicating a first physical quantity at a measurement timing of a first time interval, and a second line sensor that sequentially generates a second type of line image indicating a second physical quantity at a measurement timing of a second time interval;
obtaining the first type line image;
obtaining the second type line image;
a step of generating a first type measurement result image in which the first type line images acquired in the order of measurement timing at the first time interval are arranged in the order of the measurement timing;
a step of generating a second type measurement result image in which the second type line images acquired in the order of measurement timing at the second time interval are arranged in the order of the measurement timing;
specifying a first region of the object for calibration based on the first type measurement result image showing the object for calibration, and specifying a second region of the object for calibration based on the second type measurement result image showing the object for calibration;
specifying a measurement timing difference determined based on a separation distance between the first line sensor and the second line sensor and a moving speed of the object, based on a position of the first area in the first type measurement result image and a position of the second area in the second type measurement result image;
selecting at least one of the first type line images and at least one of the second type line images, which correspond to a given pixel array according to the measurement timing difference and show the object for quality determination ;
a step of generating a multi-channel input image by associating, with each pixel included in the pixel array, a pixel value of a pixel included in the selected first type line image and associated with the pixel, and a pixel value of a pixel included in the selected second type line image and associated with the pixel;
A step of judging the quality of the object for quality judgment based on an output from a trained machine learning model when the input image is input into the trained machine learning model;
A program characterized by executing the above.

Images