JP7616968B2 - Information processing device, method for generating teacher data, method for generating trained model, and program - Google Patents

Description

The present invention relates to an information processing device that generates training data for use in machine learning.

Machine learning using training data has been practiced for some time. Machine learning generally requires a large amount of training data, but it can be difficult to obtain a large amount of training data. One known technology to solve this problem is to generate new training data based on existing training data to increase the total amount of training data.

For example, the following Patent Document 1 discloses a machine learning system that classifies training images into multiple patterns, identifies patterns with a small number of instances, and generates new training images that belong to the missing patterns by spatially inverting the training images or changing their color tones.

JP 2018-169672 A

In addition, in machine learning to generate a trained model for detecting a detection target appearing in an image, a teacher image in which the detection target appears is associated with a label indicating the position and range of the detection target, and this is used as the training data. Generally, the representative coordinates of a rectangular area surrounding the detection target (e.g., the coordinates of the center position of the rectangular area) and the width and height of the rectangular area are associated with the teacher image as labels. Note that width and height can also be interpreted as horizontal length and vertical length.

Figure 4 shows a conventional technique, and is a diagram showing an example of setting labels that indicate the position and range of a detection target in a teacher image. The teacher image IMG1 shown on the left side of Figure 4 shows a detection target OB1, and a rectangular region R1 that surrounds this detection target OB1 is set. The representative coordinates, width, and height of this rectangular region R1 are associated with the teacher image IMG1 as labels.

Here, as in the teacher image IMG1 in Figure 4, when the detection target OB1 is a rod-shaped object and is captured at an angle, the proportion of the background area (the upper right and lower left areas of the rectangular area R1) in which the detection target OB1 is not captured in the rectangular area R1 becomes high. Learning using such labels is not preferable because it increases the influence of the background area of the detection target OB1. For example, in the teacher image IMG1 in Figure 4, part of the plate-shaped object is captured in the rectangular area R1, which may affect the learning results.

As such, it is possible to set an inclined rectangular region R2, as in the teacher image IMG2 on the right side of Figure 4. This makes it possible to significantly reduce the proportion of the background region compared to the rectangular region R1. In addition to the representative coordinates, width, and height of the rectangular region R2, the inclination angle α can be associated with the teacher image IMG2 as a label.

However, when the rectangular region is tilted, and a transformation is performed to change the aspect ratio (ratio of height to width, equivalent to aspect ratio) of the original image in order to increase the variety of teacher images, there is a problem that the position and range of the detection target indicated by the label in the teacher image generated by the transformation may differ from the position and range of the detection target shown in the teacher image.

This will be explained with reference to Figure 5. Figure 5 is a diagram illustrating the problems with the conventional technology. In the example of Figure 5, like the teacher images IMG1 and IMG2 described above, a portion of the cut-out area indicated by the dashed dotted line is cut out from image IMG3, which contains detection target OB1. Then, that portion of the cut-out area is used as original image IMG3', and teacher image IMG4 is generated by adjusting the size of original image IMG3'.

More specifically, image IMG3 has a width and height of 608 pixels, and a rectangular region R3 inclined at an angle α is set on object OB1 depicted in image IMG3, similar to IMG2 in Figure 4. Furthermore, original image IMG3' cut out from image IMG3 has a width of 566 pixels and a height of 383 pixels. The bottom edge of original image IMG3' extends beyond image IMG3, but such a cutout method is also possible when generating a teacher image.

The original image IMG3' is then enlarged and reduced to create teacher image IMG4, which has a width and height of 544 pixels. Specifically, teacher image IMG4 is generated by multiplying original image IMG3' by (544/566) in the width direction and (544/383) in the height direction. Because the enlargement rates in the width and height directions are different, the aspect ratios of original image IMG3' and teacher image IMG4 are different. The change in aspect ratio itself is desirable from the perspective of increasing the variety of teacher images.

Furthermore, in IMG4, rectangular region R4 indicating the position and range of detection target OB1 is obtained by multiplying the width of rectangular region R2 by (544/566) and the height by (544/383). In this way, if rectangular region R4 is set by enlarging and reducing the width and height of rectangular region R3 in the same way as original image IMG3', part of detection target OB1 may extend beyond rectangular region R4, as shown in Figure 5.

This is because the inclination angle of the detection object OB1 has changed due to a change in the aspect ratio of the original image IMG3', whereas the inclination angle α of the rectangular region R4 is the same as the inclination angle of the rectangular region R3 of the original image IMG3'.

In this way, if the rectangular area set in the original image is tilted and a teacher image with a different aspect ratio from the original image is generated, the label associated with the teacher image will not correctly indicate the position and range of the detection target.

One aspect of the present invention aims to realize an information processing device or the like that is capable of generating training data including labels that correctly indicate the position and range of the detection target in the training image even in the above-mentioned cases.

In order to solve the above problem, an information processing device according to one aspect of the present invention includes a teacher image generation unit that generates a teacher image by performing a transformation on an original image in which a detection target is captured, enlarging or reducing the image in at least one of the height and width directions; a coordinate transformation unit that transforms the coordinates of each vertex of a rectangular area indicating the position and range of the detection target captured in the original image based on the magnification of the transformation into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target captured in the teacher image based on the coordinates after transformation by the coordinate transformation unit; and a teacher data generation unit that generates a label indicating the position and range of the detection target captured in the teacher image based on the coordinates after transformation by the coordinate transformation unit, and associates the label with the teacher image to generate teacher data.

In order to solve the above problems, a teacher data generation method according to one aspect of the present invention is a teacher data generation method executed by one or more information processing devices, and includes a teacher image generation step of generating a teacher image by performing a transformation on an original image in which a detection target is shown, enlarging or reducing the image in at least one of the height direction and the width direction, a coordinate transformation step of transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection target shown in the original image based on the magnification of the transformation into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target shown in the teacher image, and a teacher data generation step of generating a label indicating the position and range of the detection target shown in the teacher image based on the coordinates after transformation by the coordinate transformation step, and associating the label with the teacher image to generate teacher data.

In order to solve the above problem, a method for generating a trained model according to one aspect of the present invention is a method for generating a trained model executed by one or more information processing devices, and includes a trained image generation step of performing a transformation to enlarge or reduce an original image in which a detection target is shown in at least one of the height direction and the width direction to generate a trained image, a coordinate transformation step of transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection target shown in the original image into the coordinates of each vertex of a rectangular area indicating the position and range of the detection target shown in the trained image based on the magnification of the transformation, a trained data generation step of generating labels indicating the position and range of the detection target shown in the trained image based on the coordinates after transformation by the coordinate transformation step, and associating the labels with the trained image to use as trained data, and a learning step of generating a trained model for detecting the detection target from an image by machine learning using the trained data generated in the trained data generation step.

According to one aspect of the present invention, even if a teacher image with a different aspect ratio from an original image in which an inclined rectangular area is set is generated from the original image, teacher data can be generated that includes labels that correctly indicate the position and range of the detection target in the teacher image.

1 is a block diagram showing an example of a configuration of a main part of an information processing device according to an embodiment of the present invention; 3A to 3C are diagrams illustrating an example of generation of teacher images and labels by the information processing device. 4 is a flowchart illustrating an example of a process executed by the information processing device. FIG. 13 is a diagram illustrating a conventional technique, showing an example of setting labels indicating the position and range of a detection target in a teacher image. FIG. 1 is a diagram illustrating a problem with the conventional technology.

[Device configuration]
The configuration of an information processing device 1 according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of a main configuration of the information processing device 1. The information processing device 1 may be, for example, a personal computer, a server, or a workstation.

As shown in the figure, the information processing device 1 includes a control unit 10 that controls each unit of the information processing device 1, and a storage unit 11 that stores various data used by the information processing device 1. The information processing device 1 also includes a communication unit 12 that allows the information processing device 1 to communicate with other devices, an input unit 13 that accepts input of various data to the information processing device 1, and an output unit 14 that allows the information processing device 1 to output various data.

The control unit 10 also includes a teacher image generation unit 101, a coordinate conversion unit 102, a teacher data generation unit 103, a learning unit 104, and an inference unit 105. The memory unit 11 stores an image 111, teacher data 112, and a trained model 113.

Image 111 is an image in which a detection target appears. A label indicating the position and range of the detection target appearing in image 111 is associated with image 111. This label may indicate, for example, representative coordinates, width, height, and tilt angle of a rectangular area surrounding the detection target. Note that width and height can also be interpreted as horizontal length and vertical length. Furthermore, when generating a trained model that not only detects but also identifies the detection target, an identifier of the detection target can be added to the label in addition to the information described above.

The image 111 may be input via the input unit 13 or may be received via the communication unit 12. For example, the input unit 13 may be a USB (Universal Serial Bus) interface. In this case, the input unit 13 and a photographing device that photographs the detection target may be connected via the USB interface. The information processing device 1 may receive the image photographed by the photographing device via the input unit 13 and store it as the image 111. Alternatively, the image photographed by the photographing device may be transmitted via a LAN (Local-Area Network) or a wireless LAN, and the information processing device 1 may receive the image via the communication unit 12 and store it as the image 111.

The teacher image generating unit 101 generates a teacher image by performing a conversion on an original image cut out from an image 111 containing a detection target, enlarging or reducing the image in at least one of the height and width directions. Note that the teacher image may be generated by using the image 111 as the original image as it is. The method of generating the teacher image will be described in detail later with reference to FIG. 2.

The coordinate conversion unit 102 converts the coordinates of each vertex of a rectangular area indicating the position and range of the detection target in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target in the teacher image, based on the conversion magnification of the teacher image (magnification of enlargement/reduction in the height direction and width direction). The coordinate conversion method will be described in detail later with reference to FIG. 2.

The teacher data generation unit 103 generates labels indicating the position and range of the detection target appearing in the teacher image generated by the teacher image generation unit 101 based on the coordinates converted by the coordinate conversion unit 102, and associates the labels with the teacher image to generate teacher data. The teacher data generation unit 103 then stores the generated teacher data in the storage unit 11 as teacher data 112. The method of generating the labels will be described in detail below with reference to FIG. 2.

The learning unit 104 generates a trained model for detecting a detection target from an image by machine learning using the teacher data 112 generated by the teacher data generation unit 103. The learning unit 104 then stores the generated trained model in the storage unit 11 as the trained model 113. The model may be, for example, a neural network model (including a deep neural network model). Furthermore, learning may be performed by repeatedly changing the teacher data to calculate the error between the inference result obtained by inputting a teacher image included in the teacher data into the model being trained and the correct answer indicated in the teacher data, and updating the model by error backpropagation (error backpropagation method).

The inference unit 105 detects the detection target from the image using the trained model 113. More specifically, the inference unit 105 first reads the target image. The image may be one that has been stored in advance in the storage unit 11, or may be one that has been input from a photographing device or the like via the communication unit 12 or the input unit 13. The inference unit 105 then inputs the read image into the trained model 113, and causes the output unit 14 to output, from among the obtained detection results, those detection results whose accuracy is equal to or exceeds a threshold.

The trained model 113 is generated by learning using the teacher data 112 including labels that correctly indicate the position and range of the detection target, so that the inference unit 105 that performs detection using the trained model 113 can perform highly accurate object detection. The detection result indicates the position and range of the detection target that appears in the image. The detection result may be output as the representative coordinates, width, height, and tilt angle of the area surrounding the detection target, or may be output in the form of a rectangle that indicates the above area drawn on the target image.

As described above, the information processing device 1 includes a teacher image generation unit 101 that generates a teacher image by performing a transformation on an original image in which the detection target is captured, enlarging or reducing the image in at least one of the height and width directions; a coordinate transformation unit 102 that transforms the coordinates of each vertex of a rectangular area indicating the position and range of the detection target captured in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target captured in the teacher image based on the magnification of the transformation; and a teacher data generation unit 103 that generates a label indicating the position and range of the detection target captured in the teacher image based on the coordinates after transformation by the coordinate transformation unit 102, and associates the label with the teacher image to generate teacher data 112.

According to the above configuration, a teacher image is generated by enlarging or reducing the original image showing the detection target in at least one of the height and width directions. This makes it possible to generate a wide variety of teacher images, including those with a different aspect ratio (ratio of height to width, synonymous with aspect ratio) from the original image.

In addition, according to the above configuration, a label to be associated with the generated teacher image is generated based on the coordinates of the four vertices of a quadrangular area obtained by transforming the coordinates of the four vertices of the rectangular area of the original image based on the magnification of the transformation applied when generating the teacher image. Then, the label is associated with the teacher image to create teacher data.

The coordinates of the four vertices of the rectangular area are calculated based on the transformation magnification applied when generating the teacher image, and therefore correctly indicate the position and range of the detection target appearing in the teacher image. This makes it possible to generate teacher data 112 associated with labels that correctly indicate the position and range of the detection target appearing in the teacher image.

As described above, the above configuration has the effect of generating teacher data 112 that includes labels that correctly indicate the position and range of the detection target in the teacher image, even when the rectangular area set in the original image is tilted and a teacher image with a different aspect ratio from that of the original image is generated.

The information processing device 1 also includes a learning unit 104 that generates a trained model 113 for detecting a detection target from an image by machine learning using the teacher data 112 generated by the teacher data generation unit 103. With this configuration, the trained model 113 can be automatically generated.

[Example of generating training images and labels]
FIG. 2 is a diagram showing an example of teacher images and labels generated by the information processing device 1. In FIG. 2, a portion of the cutout area indicated by the dashed line is cut out from an image IMG5 having a width and height of 608 pixels, and an original image IMG5' is obtained by cutting out the cutout area, which is then enlarged and reduced to obtain a teacher image IMG6 having a width and height of 544 pixels. The width and height of the teacher image IMG6 may be determined by using a random number or the like. Also, a teacher image may be generated using an original image in which not the entire detection target OB1 but a part of it is cut out. By performing learning using such a teacher image, it becomes possible to detect the detection target OB1 from an image in which only a part of the detection target OB1 is captured.

In this way, the teacher image generating unit 101 performs a transformation on the original image IMG5' cut out from the image IMG5 containing the detection target, enlarging or reducing it in at least one of the height and width directions, to generate the teacher image IMG6. The teacher image generating unit 101 may determine the position and range of the cut-out area and the transformation magnification using, for example, random numbers.

When the teacher image IMG6 is generated, the coordinate conversion unit 102 determines the coordinates of each vertex P5a to P5d of the rectangular region R5 that indicates the position and range of the detection target OB1 appearing in the original image IMG5', based on the conversion magnification used when generating the teacher image IMG6. These coordinates are calculated from the representative coordinates, width, height, and tilt angle of the rectangular region R5, which are indicated in the label associated with the image IMG5.

Then, the coordinate conversion unit 102 converts the coordinates of the vertices P5a to P5d into the coordinates of each of the vertices P6a to P6d of the rectangular region R6 that indicates the position and range of the detection target OB1 that appears in the teacher image IMG6. For example, the coordinate conversion unit 102 performs the above conversion by multiplying the x coordinates of the vertices P5a to P5d by the conversion factor in the width direction when generating the teacher image IMG6, and multiplying the y coordinates of the vertices P5a to P5d by the conversion factor in the height direction when generating the teacher image IMG6.

The aspect ratios of the original image IMG5' and the teacher image IMG6 are different because the magnifications of the transformation in the height and width directions applied when generating the teacher image IMG6 are different. In this case, the balance between the height and width of the detection object OB1 shown in the teacher image IMG6 has changed and the shape is distorted. In this case, the rectangular area R6 corresponding to the tilted rectangular area R5 becomes a parallelogram area along the outer edge of the distorted detection object OB1. If the aspect ratio does not change between the original image and the teacher image, the detection object shown in the teacher image is enlarged or reduced without distortion. In this case, the rectangular area of the original image is enlarged or reduced without distortion, so the area after transformation is also rectangular.

Next, the teacher data generation unit 103 generates a label indicating the position and range of the detection target OB1 appearing in the teacher image IMG6 based on the coordinates after transformation by the coordinate transformation unit 102, i.e., the coordinates of the vertices P6a to P6d. Specifically, the teacher data generation unit 103 calculates the following four values from the coordinates of the vertices P6a to P6d.

(1) Representative coordinates of the rectangular region R6 (2) Length of the long side of the rectangular region R6 (3) Tilt angle of the long side of the rectangular region R6 (4) Distance between opposing long sides of the rectangular region R6 The above (1) is information indicating the position of the detection target OB1 in the teacher image IMG6. The representative coordinates may be any coordinates that can identify the position of the rectangular region R6. For example, the teacher data generation unit 103 may calculate the coordinates of the intersection P6e of the diagonals of the rectangular region R6 as the representative coordinates. Also, for example, the teacher data generation unit 103 may calculate the coordinates of the vertex at the top left corner of the rectangular region R6, i.e., the coordinates of the vertex P6a, as the representative coordinates.

The above (2) is information indicating the height H of the detection target shown in the teacher image IMG6. The teacher data generation unit 103 may calculate the distance between vertices P6a and P6d, or the distance between vertices P6b and P6c. Note that instead of the length of the long side of the quadrangular region R6, the length of the short side of the quadrangular region R6 may be calculated.

(3) above is information indicating the tilt angle of the detection target shown in the teacher image IMG6. The teacher data generation unit 103 need only calculate the angle β that the line connecting vertices P6a and P6d makes with respect to the width direction of the teacher image IMG6, or the angle that the line connecting vertices P6b and P6c makes with respect to the width direction of the teacher image IMG6. Note that in (2) above, if the length of the short side of the rectangular region R6 is calculated instead of the length of the long side of the rectangular region R6, the tilt angle of the short side can be calculated.

(4) above is information indicating the width W of the detection target shown in the teacher image IMG6. The teacher data generation unit 103 need only calculate the distance between the line connecting vertices P6a and P6d and the line connecting vertices P6b and P6c. In (2) above, if the length of the short side of the rectangular region R6 is calculated instead of the length of the long side of the rectangular region R6, the distance between the short sides can be calculated.

Note that the teacher data generating unit 103 may calculate the distance between vertices P6a and P6b, or the distance between vertices P6c and P6d, as the width of the rectangular region R6, although this will be slightly wider than the actual width of OB1.

Also, as shown in FIG. 2, the teacher data generating unit 103 may correct the vertices P6a to P6d to P6a' to P6d', identify a rectangle with P6a' to P6d' as its four vertices, and generate a label indicating the position and range of the rectangle. Specifically, the teacher data generating unit 103 corrects the positions of the vertices P6a to P6d so that the area is the same as that of a parallelogram with P6a to P6d as its four vertices, and the intersection of the diagonals is P6e. For example, the long side of the rectangle may be the same length as the long side of the parallelogram, and the short side of the rectangle may be the same length as the height of the parallelogram. This identifies a rectangle with P6a' to P6d' as its four vertices, as shown in FIG. 2. In this case, for example, the coordinates of P6a' to P6d' may be used as a label indicating the position and range of the rectangle. Of course, the short side of the rectangle can be the same length as the short side of the parallelogram, and the long side of the rectangle can be the same length as the height of the parallelogram.

Finally, the teacher data generation unit 103 associates labels indicating the above four calculated values with the teacher image IMG6 to generate teacher data. In this way, the teacher data generation unit 103 may, from the coordinates of each vertex obtained by the coordinate conversion unit 102, (1) calculate representative coordinates of the rectangular area as information indicating the position of the detection target, (2) calculate the length of one side of the rectangular area as information indicating the height of the detection target, (3) calculate the inclination angle of the one side as information indicating the inclination angle of the detection target, and (4) calculate the distance between the one side and the side opposite to the one side as information indicating the width of the detection target.

The above configuration makes it possible to generate training data that is associated with labels indicating tilt angles that are useful when learning about rod-shaped detection targets, in addition to representative coordinates, height, and width, which are common labels for object detection.

[Processing flow]
The flow of processing executed by the information processing device 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of processing executed by the information processing device 1. It is assumed that the reading of setting information necessary for learning has been completed before the start of the processing in Fig. 3. Examples of the setting information include a storage location for the image 111, a range of random numbers used when generating a teacher image, the number of images 111 to be read at one time, and a condition for ending learning.

In S1, the teacher image generating unit 101 acquires images 111 from the memory unit 11. The number of images 111 to be acquired may be set in the configuration information as described above. The number of images 111 to be acquired may be one or more. In addition, the teacher image generating unit 101 may determine which image 111 to acquire using a random number.

In S2, the teacher image generating unit 101 determines the cut-out range of the image 111 acquired in S1. For example, the teacher image generating unit 101 may determine the position, width, and height of the cut-out range using random numbers.

In S3 (teacher image generation step), the teacher image generation unit 101 cuts out the image 111 read out in S1 within the cut-out range determined in S2 to generate an original image that will be the basis for the teacher image. The teacher image generation unit 101 then performs a conversion on the original image to enlarge or reduce it in at least one of the height and width directions to generate a teacher image. The magnification of enlargement or reduction is determined by the height and width sizes of the teacher image to be generated and the height and width sizes of the original image. The teacher image generation unit 101 may determine the height and width sizes of the teacher image using random numbers.

In S4 (coordinate transformation step), first, the coordinate transformation unit 102 calculates the coordinates of the four vertices that indicate the range of the detection target in the original image. The coordinate transformation unit 102 then transforms the calculated coordinates based on the magnification of the transformation performed on the original image, to calculate the coordinates of the four vertices that indicate the range of the detection target in the teacher image generated in S3. The method of calculating the coordinates of each vertex is as described with reference to FIG. 2, so the description will not be repeated here.

In S5, the teacher data generation unit 103 calculates the representative coordinates, width, height, and tilt angle of the detection target as information indicating the position and range of the detection target appearing in the teacher image generated in S3 from the coordinates of each vertex calculated by the coordinate conversion unit 102 in S4. The method of calculating this information is as described with reference to FIG. 2, so the description will not be repeated here.

In S6 (teacher data generation step), the teacher data generation unit 103 generates labels from each piece of information calculated in S5, and associates them with the teacher image generated in S3 to generate teacher data. The teacher data generation unit 103 then stores the generated teacher data in the storage unit 11 as teacher data 112. The method of generating labels from the transformed coordinates is as described with reference to FIG. 2, and therefore will not be described again here.

In S7 (learning step), the learning unit 104 performs machine learning using the teacher data 112 generated in S6. For example, when learning a neural network model, the learning unit 104 inputs a teacher image included in the teacher data 112 to the model being learned, and calculates the error between the output value output from the model and the label value associated with the teacher image. Then, the learning unit 104 updates the weight values of the neural network model by the backpropagation method based on the calculated error.

In S8, the learning unit 104 determines whether or not to end the learning. If it is determined that the learning should be ended (YES in S8), the learning unit 104 stores the latest updated model in the storage unit 11 as the trained model 113, and the processing in FIG. 3 ends. On the other hand, if it is determined that the learning should not be ended (NO in S8), the processing returns to S1, and a new image 111 is read out. The end condition for S8 may be set in advance as setting information. For example, the end condition may be that the model has been updated a predetermined number of times or more, or that the error is equal to or less than a threshold value.

If multiple images 111 are read in S1, teacher data is generated from each of the multiple images 111 read in S1 through the processes of S2 to S6, and stored as teacher data 112. Then, in S7, learning is performed using the teacher data 112.

The processes S1 to S6 in FIG. 3 are a method for generating teacher data. Furthermore, S1 to S8, which are the method for generating teacher data plus the processes S7 to S8 for generating a trained model by machine learning, are a method for generating a trained model. Note that the processes S1 to S6 and S7 to S8 do not necessarily need to be performed consecutively. In S1, the number of images 111 required for learning may be acquired, and after the number of teacher data 112 required for learning is generated from these images 111 by the processes S2 to S6, the processes S7 to S8 may be performed.

In addition, when generating training data, the color and rotation angle of the training image may be changed from the original image to increase the variation of the training data. Various conventional methods can be used for color conversion and rotation conversion to increase the variation of the training data.

As described above, the method for generating teacher data executed by the information processing device 1 includes a teacher image generation step (S3) for generating a teacher image by performing a transformation on an original image in which the detection target is captured, enlarging or reducing the image in at least one of the height and width directions; a coordinate transformation step (S4) for transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection target captured in the original image based on the transformation magnification into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection target captured in the teacher image based on the coordinates after transformation in the coordinate transformation step; and a teacher data generation step (S6) for generating labels indicating the position and range of the detection target captured in the teacher image based on the coordinates after transformation in the coordinate transformation step, and associating the labels with the teacher image to generate teacher data. This makes it possible to generate teacher data including labels that correctly indicate the position and range of the detection target in the teacher image.

As described above, the method for generating a trained model executed by the information processing device 1 includes a trained image generation step (S3) for generating a trained image by performing a transformation to enlarge or reduce the original image in which the detection target appears in at least one of the height direction and the width direction, a coordinate transformation step (S4) for transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection target appearing in the original image into the coordinates of each vertex of a rectangular area indicating the position and range of the detection target appearing in the trained image based on the transformation magnification, a trained data generation step (S6) for generating labels indicating the position and range of the detection target appearing in the trained image based on the coordinates after transformation by the coordinate transformation step, and associating them with the trained image to use them as trained data, and a learning step (S7) for generating a trained model for detecting the detection target from an image by machine learning using the trained data generated in the trained data generation step. As a result, a trained model for detecting the detection target from an image can be automatically generated using trained data including labels that correctly indicate the position and range of the detection target.

[Modifications]
The execution subject of each process described in the above embodiment is arbitrary and is not limited to the above example. In other words, an information processing system having the same function as the information processing device 1 can be constructed by a plurality of information processing devices capable of communicating with each other. For example, the generation of the teacher image, the conversion of coordinates, the generation of the teacher data, learning, and inference may be respectively executed by different information processing devices. In addition, for example, an information processing system including an information processing device that executes the process from the generation of the teacher image to the generation of the teacher data, that is, the method of generating the teacher data, an information processing device that generates a trained model using the generated teacher data, and an information processing device that performs inference using the generated trained model can be constructed. In this way, the above-mentioned method of generating the teacher data can be realized by one or more information processing devices. The same applies to the above-mentioned method of generating the trained model. In addition, a photographing device that photographs the detection target may be included in the components of the information processing system.

[Software implementation example]
The functions of the information processing device 1 (hereinafter referred to as the "device") can be realized by a program for causing a computer to function as the device, and a program for causing a computer to function as each control block of the device (particularly each part included in the control unit 10).

In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing each of the functions described in each of the above embodiments.

The control device may be, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a combination of these. The storage device may include a high-speed storage unit capable of writing and reading data at high speed, and a large-capacity storage unit having a larger data storage capacity than the high-speed storage unit. The high-speed storage unit may be, for example, a high-speed access memory such as an SDRAM (Synchronous Dynamic Random-Access Memory). The large-capacity storage unit may be, for example, a hard disk drive (HDD), a solid-state drive (SSD), a secure digital (SD) card, or an embedded multi-media controller (eMMC).

The program may be recorded on one or more computer-readable recording media, not on a temporary basis. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.

In addition, some or all of the functions of each of the above control blocks can be realized by a logic circuit formed in an integrated circuit (IC chip) or the like. For example, an integrated circuit in which a logic circuit that functions as each of the above control blocks is formed is also included in the scope of the present invention. In addition, it is also possible to realize the functions of each of the above control blocks by, for example, a quantum computer.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. The technical scope of the present invention also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments.

1 Information processing device 101 Teacher image generation unit 102 Coordinate conversion unit 103 Teacher data generation unit 104 Learning unit

Claims (6)

a teacher image generating unit that performs a transformation to enlarge or reduce an original image including a detection target in at least one of a height direction and a width direction to generate a teacher image;
a coordinate conversion unit that converts, based on the conversion magnification, the coordinates of each vertex of a rectangular area indicating the position and range of the detection object in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection object in the teacher image;
a teacher data generation unit that generates labels indicating a position of the detection target shown in the teacher image , a range of the detection target shown in the teacher image , and an inclination angle of the detection target shown in the teacher image based on the coordinates converted by the coordinate conversion unit, and associates the labels with the teacher image to set as teacher data ;
The teacher data generation unit calculates an inclination angle of one side of the rectangular area as information indicating the inclination angle of the detection object in the label from the coordinates of each vertex of the rectangular area . The teacher data generation unit calculates, from the coordinates of each vertex of the quadrangular region,
(1) calculating representative coordinates of the rectangular area as information indicating the position of the detection target on the label ;
(2) The information processing device according to claim 1, further comprising: calculating a length of one side of the rectangular area and a distance between the one side and an opposite side to the one side as information indicating the range of the detection target on the label. The information processing device according to claim 1 or 2, further comprising a learning unit that generates a trained model for detecting the detection target from an image by machine learning using the training data generated by the training data generation unit. A method for generating teacher data executed by one or more information processing devices, comprising:
a teacher image generating step of generating a teacher image by performing a transformation such that an original image including a detection target is enlarged or reduced in at least one of a height direction and a width direction;
a coordinate transformation step of transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection object in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection object in the teacher image based on the magnification of the transformation;
a teacher data generating step of generating labels indicating a position of the detection target shown in the teacher image , a range of the detection target shown in the teacher image , and an inclination angle of the detection target shown in the teacher image based on the coordinates converted by the coordinate conversion step, and associating the labels with the teacher image to obtain teacher data;
A method for generating teacher data, in which the teacher data generating step calculates an inclination angle of one side of the rectangular area from the coordinates of each vertex of the rectangular area as information indicating the inclination angle of the detection object in the label . A method for generating a trained model executed by one or more information processing devices,
a teacher image generating step of generating a teacher image by performing a transformation such that an original image including a detection target is enlarged or reduced in at least one of a height direction and a width direction;
a coordinate transformation step of transforming the coordinates of each vertex of a rectangular area indicating the position and range of the detection object in the original image into the coordinates of each vertex of a quadrangular area indicating the position and range of the detection object in the teacher image based on the magnification of the transformation;
a teacher data generation step of generating labels indicating a position of the detection target shown in the teacher image , a range of the detection target shown in the teacher image , and an inclination angle of the detection target shown in the teacher image based on the coordinates converted by the coordinate conversion step, and associating the labels with the teacher image to set as teacher data;
A learning step of generating a trained model for detecting the detection target from an image by machine learning using the training data generated in the training data generation step ,
A method for generating a trained model , in which the teacher data generation step calculates an inclination angle of one side of the rectangular area from the coordinates of each vertex of the rectangular area as information indicating the inclination angle of the detection object in the label . A program for causing a computer to function as the information processing device according to claim 1, the program causing a computer to function as the teacher image generating unit, the coordinate conversion unit, and the teacher data generating unit.

Images