JP2021033576A - Information processing equipment, information processing method, analysis system, and program - Google Patents

Abstract

To estimate the surface shape of an object more accurately.SOLUTION: An information processing equipment acquires shape data of an object, estimates the surface shape at a specified part specified in displayed shape data, and outputs information indicating a scale having a plurality of reference points arranged at intervals corresponding to the estimated surface shape, which is a scale attached to the specified part of the object and arranged at the specified part of the object in order to estimate the orientation of the surface at the specified part.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing apparatus, an information processing method, an analysis system, and a program.

Conventionally, there is known a technique of acquiring information from an image by photographing a code for storing information with a photographing device. As a technique related to such a code, for example, a method of calculating the curvature and inclination of the curved surface of a scaled object from a point cloud arranged substantially parallel to the scale in a captured image is described in Patent Document 1. It is disclosed.

Japanese Unexamined Patent Publication No. 2014-229274

However, in the scale described in Patent Document 1, it is unclear whether the density of the point cloud on the scale is sufficient for calculating the shape of the curved surface with the scale, so that the surface of the object is surfaced. There is a problem that the accuracy of shape calculation is not guaranteed.

An object of the present invention is to more accurately estimate the shape of the surface of an object.

In order to achieve the object of the present invention, for example, the information processing apparatus according to one embodiment has the following configurations. That is, the first acquisition means for acquiring the shape data of the object, the first evaluation means for estimating the surface shape at the designated portion designated in the displayed shape data, and the orientation of the surface at the designated portion. An output means for outputting information indicating a scale arranged at the designated portion of the object for estimation and having a plurality of reference points arranged at intervals corresponding to the estimated surface shape. It is characterized by having.

The shape of the surface of the object can be estimated more accurately.

The figure which shows an example of the structure of the analysis system including the information processing apparatus which concerns on Embodiment 1. FIG. The figure which shows an example of the structure of the scale which concerns on Embodiment 1. FIG. The flowchart of the processing example in the analysis method which concerns on Embodiment 1. The figure which shows an example of the functional structure of the information processing apparatus which concerns on Embodiment 1. The figure which shows an example of the polygon of the shape data which concerns on Embodiment 1. FIG. The figure which shows an example of the UI of the display device in the information processing method which concerns on Embodiment 1. FIG. The flowchart of the processing example in the information processing method which concerns on Embodiment 1. The flowchart of the flatness evaluation example in the information processing method which concerns on Embodiment 1. The flowchart of the selection example in the information processing method which concerns on Embodiment 1. The figure which shows an example of the display device in the information processing method which concerns on Embodiment 1. FIG. The figure which shows an example of the functional structure of the analyzer which concerns on Embodiment 1. FIG. The figure which shows the acquisition example of the position of the light source in the analysis method which concerns on Embodiment 1. FIG. The figure which shows the calculation example of the reflection characteristic in the analysis method which concerns on Embodiment 1. FIG. The flowchart of the processing example in the analysis method which concerns on Embodiment 1. The figure which shows the shape calculation example in the analysis method which concerns on Embodiment 1. FIG. The figure which shows an example of the metadata in the analysis method which concerns on Embodiment 1. FIG. The flowchart of the shape calculation processing example in the analysis method which concerns on Embodiment 1. The figure which shows an example of the functional structure of the analysis part in the analysis method which concerns on Embodiment 1. FIG. The flowchart of the analysis example in the analysis method which concerns on Embodiment 1. The flowchart of the normal direction calculation example in the analysis method which concerns on Embodiment 1. The figure which shows the example of the light source direction calculation in the analysis method which concerns on Embodiment 1. FIG. The figure which shows an example of the analysis procedure in the analysis method which concerns on Embodiment 1. FIG. The figure which shows the example of the structure of the analysis system including the information processing apparatus which concerns on Embodiment 2. The figure which shows an example of the functional structure of the information processing apparatus which concerns on Embodiment 2. FIG. The figure which shows an example of the UI of the display device in the information processing method which concerns on Embodiment 2. FIG. The flowchart of the processing example in the information processing method which concerns on Embodiment 2. The figure which shows an example of the functional structure of the information processing apparatus which concerns on Embodiment 3. The figure which shows an example of the UI of the display device in the information processing method which concerns on Embodiment 3. The figure which shows an example of the evaluation of the measurement part in the information processing method which concerns on Embodiment 3. The flowchart of the evaluation example in the information processing method which concerns on Embodiment 3. The figure which shows the hardware configuration of the computer which concerns on Embodiment 4. FIG.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

[Embodiment 1]
[Information processing device]
FIG. 1 shows an example of the configuration of an analysis system having the information processing apparatus 101 according to the first embodiment. The analysis system includes an information processing device 101 and an analysis device 107, estimates the shape of the surface of the object 103, and analyzes the color of the surface of the object 103. The information processing device 101 outputs information indicating a scale (marker) 104 that assists in measuring the orientation of the surface of the object 103. The information processing device 101 may be connected to the display device 102 by wire as shown in FIG. 1, or may be connected to the display device 102 via wireless communication. Further, the information processing device 101 may be a device that is independent of the display device 102, or may be a device that is integrally configured in one device. Further, the information processing device 101 does not have to be connected to the display device 102. The display device 102 is a display capable of displaying images and characters. The display device 102 displays the result of the process output by the information processing device 101 on the display. The type of the display device 102 is not particularly limited, but for example, the display device 102 may be a tablet PC that is easy to carry and can check the processing result of the information processing device 101 regardless of the location.

The information processing device 101 performs information processing on the object 103. The object 103 is a three-dimensional object composed of a surface including a flat surface or a curved surface. In the present embodiment, for the sake of explanation, it is assumed that the object 103 is composed of a plurality of curved surfaces having different curvatures, that is, having different radii of curvature. In FIG. 1, the object 103 is configured with a cylinder having a large radius of curvature as a lower stage, a cylinder having a small radius of curvature as a middle stage, and a sphere having a curvature in a plurality of directions as an upper stage. However, the configuration of the object 103 is not particularly limited in this way, and may be, for example, a part of a mechanical product such as a door or a frame of an automobile.

Further, the information processing device 101 acquires three-dimensional shape data corresponding to the object 103 from the storage device. FIG. 5 is an example of this shape data. The shape data is composed of a set of polygons that are faces having three or four points as vertices in a three-dimensional space. The nth (n ≦ N) vertex of a polygon having N vertices holds Vn = (Xn, Yn, Zn) representing the coordinates of the vertices in the three-dimensional space. In the example of FIG. 5, the shape data is composed of polygons, but the format is not particularly limited thereto. The shape data may be composed of, for example, a wire frame which is data of only the points and ridges of the shape. Further, in the present embodiment, the 3D CAD data created at the time of designing the object 103 is used as the shape data, but the shape data is not particularly limited to the 3D CAD data. As the shape data, the information processing device 101 may use, for example, the three-dimensional shape data of the object 103 measured by a 3D scanner, a three-dimensional measuring device, or the like.

Further, the information processing apparatus 101 provides a scale 104 that assists in measuring the orientation of the designated portion on the surface of the object 103 corresponding to the measurement portion based on the surface shape of the measurement portion designated on the shape data of the object 103. Output the information shown. An example of the scale 104 in this embodiment is shown in FIG. By being attached to the surface of the object 103, the scale 104 can assist the estimation by the analyzer 107 of the orientation of the region of the surface. The scale 104 has a frame area 201, a reference point 202, a code area 203, and an opening 204. In the example of FIG. 2, the scale 104 is surrounded by a black frame region 201, and a plurality of reference points 202 can be arranged in this region. Each color of the frame area 201 and the reference point 202 is not particularly limited as long as a sufficient relative luminance ratio can be maintained in recognizing the reference point 202 in the recognition procedure of the reference point 202 described later. In this example, the color of the frame region 201 is black, and the color of the reference point 202 is white. For example, the scale 104 may be used by exchanging the color of the frame area 201 and the color of the reference point 202.

The reference point 202 is a plurality of regions configured so that when the scale 104 is attached to a designated portion on the surface of the object 103, the analyzer 107 can estimate the orientation of the designated portion based on the arrangement of the reference point 202. is there. That is, the reference point 202 may be arranged according to the degree of distortion of the designated portion. In the example of FIG. 2, the reference points 202 are circular pattern regions and are arranged in a grid pattern. In the following, for the sake of explanation, the scale 104 has rows having a plurality of circular reference points 202 at regular intervals arranged substantially in parallel over one or more columns at intervals of Δy. It is assumed that it has a grid-like arrangement of. In a scale 104 having such an array of reference points 202, the horizontal direction shall represent the row direction and the vertical direction shall represent the column direction. Further, the lateral interval of the reference point 202 is a constant interval Δx. However, the arrangement of the reference points 202 is not particularly limited as such, and may be arranged in an annular shape at regular intervals Δx, for example. The shape of the reference point is not particularly limited, and may be, for example, an ellipse or a quadrangle. For example, when recognizing or extracting the reference point 202 from the captured image, the reference point 202 may be set in consideration of the recognition or extraction accuracy when the outline of the reference point 202 becomes unclear due to the influence of the depth of field or the like. It may be circular as in the example of FIG. The value of Δx will be described later.

As described above, the arrangement of the reference points 202 is determined so as to assist in estimating the orientation of the designated portion of the object 103. Therefore, for the reference point 202, for example, when the scale 104 is attached on the surface of the shape data of the object 103, the arc on the surface of the shape data existing between the adjacent reference points 202 is the reference point 202. They may be arranged at intervals that can be approximated to a straight line connecting the above. In the present embodiment, the case where the curved surface constituting the three-dimensional object existing between a plurality of points on the three-dimensional object can be approximated to a straight line or a plane passing through those points based on the criteria described later is the case. Such a curved surface is called having high flatness. In the present embodiment, when the scale 104 is attached to the shape data of the object 103, the value of Δx is set so that the flatness of the arc of the surface connecting the two adjacent reference points 202 becomes high. .. Specifically, in the present embodiment, the difference between Δx, which is the length of the arc connecting the above-mentioned two adjacent reference points, and the linear distance between the two points is a known predetermined threshold Th1. When the following, the flatness becomes high. That is, when the value of the radius of curvature in the arc direction at the center point of the arc on the surface of the shape data of the object 103 is R, this is the case when Δx satisfies the following equation (1). The arc may be highly flat. The threshold Th1 may be preset as a desired value as appropriate for such applications. The threshold value Th1 may be, for example, a value of 1/40 of the radius of curvature R or a value of 1/50 of R. The distance Δx between the reference points 202 may take a value that satisfies the equation (1) when the radius of curvature R is given. That is, a scale 104 having such Δx can be used depending on the value of R. The value of R will be described later.
Δx-2R · sin (Δx / 2R) ≤ Th1 equation (1)

The code area 203 is an area in which the identification information of the scale 104 is encoded. The identification information is information for identifying the scale 104, and is generated according to the arrangement of the reference point 202. The identification information may be, for example, an ID assigned a different number for each candidate of the scale 104. Further, the identification information may include information indicating the number and positional relationship of the reference points 202. In the example of FIG. 2, the pattern of the code area 203 is divided into a total of 64 blocks of 8 × 8 pixels, and each block is represented by two values of white and black, so that 64-bit identification information is obtained. Can be expressed. The pattern of the code area 203 is not particularly limited in this way. The pattern of the code area 203 is, for example, a margin area which is an area not including a pattern and is composed of colors having a sufficient relative luminance ratio with respect to the color of the scale 104 in consideration of being recognized in the image. It may be surrounded by. Further, the code area 203 may have, for example, a cutout symbol for recognizing the position of the code area. Further, for example, a QR code (registered trademark) may be used in the code area 203. A different code area 203 is allocated to the scale 104 in which the reference points 202 are arranged differently according to the arrangement. The opening 204 may be an area where the surface of the object 103 can be directly photographed by the photographing apparatus from this area when the scale 104 is attached to the object 103. Further, the opening 204 may be generated by cutting out the region of the opening 204 when the scale 104 is used. In the present embodiment, for the sake of explanation, the region on the scale 104 where the opening 204 is formed by cutting is referred to as a cut region. Further, the area per reference point 202 and the area of the code area 203, or their area ratio may be known.

The scale 104 is printed on the base material by using a printing portion (not shown). The material of this base material is not particularly limited, but a foldable material may be used from the viewpoint of being attached to the surface of the object 103. Further, as the material of the scale 104, a material such as a magnet or an adsorption sheet may be used in consideration of attaching or detaching the scale 104 a plurality of times. Further, the type of recording material used for printing the scale 104 is not particularly limited, but ink or toner having less gloss may be used. According to such a structure of the base material and the recording material, it is possible to obtain a scale 104 that can be bent, can be attached and detached a plurality of times, and reduces the light source and the reflection of the surroundings at the time of photographing. ..

The information processing device 101 according to the present embodiment uses the scale 104 as the scale 104, which can assist in estimating the orientation of the designated portion of the object 103 from the information indicating the candidates of the scale 104 held by the display device 102. Selectively acquire the information to be shown. However, the method of acquiring the scale 104 is not limited to such a form, and the information processing apparatus 101 may generate information indicating the scale 104 that can assist in estimating the orientation of the designated portion of the object 103, for example. .. Such an example will be described in the second embodiment described later.

FIG. 4 is a block diagram showing an example of the functional configuration of the information processing apparatus 101 according to the present embodiment. The information processing device 101 includes a scale holding unit 401, a scale acquisition unit 402, a shape holding unit 403, a shape acquisition unit 404, a site designation unit 405, a flatness evaluation unit 406, and an output unit 407. There is. In this embodiment, the information processing device 101 is connected to the display device 102.

The scale holding unit 401 holds one or more pieces of information indicating candidates for the scale 104. The scale 104 is attached to a designated site of the object 103 to assist in estimating the orientation of the surface at the designated site. The scale 104 has a plurality of reference points arranged at intervals corresponding to the estimated surface shape. The information indicating the candidate of the scale 104 may be input by the user, but the method of inputting by the user to the scale holding unit 401 will be described later.

The scale acquisition unit 402 acquires the value of the interval Δx of the reference point 202 of the candidate group of the scale 104 and the code region 203 from the information indicating the candidate group of the scale 104 held by the scale holding unit 401, respectively. This information is used by the selection unit 407.

The shape holding unit 403 holds the shape data as described above corresponding to the object 103. The shape acquisition unit 404 acquires shape data corresponding to the object from the shape holding unit 403. An example of the shape data polygon held by the shape holding unit 403 is shown in FIG. The polygons that make up the shape data are made up of polygons, as shown in FIG.

The site designation unit 405 designates a measurement site, which is a site to be measured by the analyzer 107, on the shape data acquired by the shape acquisition unit 404. When the part designation unit 405 designates the measurement part, the designation may be made on the UI displayed on the display device 102. An example of the UI is shown in FIG. In this example, the reference points 202 are arranged in a grid pattern on the scale 104, and the lateral spacing is a predetermined spacing Δx.

On the UI of FIG. 6, information indicating a candidate for the scale 104 is input to the scale holding unit 401, and a measurement site is designated on the shape data. This UI has a processing window 601 shown in FIG. 6A and a designated window 614 shown in FIG. 6B. In response to the user operation via the processing window 601, information indicating a candidate for the scale 104 is input to the scale holding unit 401, and among such candidates, the scale 104 satisfying the condition based on the measurement site is indicated. Information is selected. Therefore, the processing window 601 has an input tab 602, an additional button 603, a shape input field 611, a setting button 612, and a selection button 613.

In the input tab 602, the information of the scale candidate that is a candidate for the scale 104 is input in the tab by the user and is held in the scale holding unit 401. The input tab 602 has a field for inputting an ID 604, a vertical reference point number 605, a vertical spacing 606, a horizontal reference point number 607, and a horizontal spacing 608, and a delete button 610. As the ID 604, a value corresponding to the identification information stored in the scale candidate code area 203 is input. As the number of vertical reference points 605 and the number of horizontal reference points 607, the number of horizontal rows of reference points and the number of reference points held in the row are input for each of the scale candidates. As the vertical spacing 606 and the horizontal spacing 608, the spacing between the horizontal row of the reference point and the adjacent row, and the spacing between the reference points 202 within the row (that is, Δx) are input in actual dimensions. Will be done. By switching the input tab 602, the user can input information on a plurality of different scale candidates in different tabs of the processing window 601. The add button 603 can acquire user input for adding a tab into the input tab 602. Further, the delete button 610 can acquire a user input for deleting an arbitrary tab from the input tab 602. The shape input field 611 acquires the shape data by the user inputting the address in which the shape data of the object 103 is recorded in this field. The address to be input to the shape input field 611 is known and is appropriately given according to the object 103. The shape data may be stored in a storage device (not shown). In addition, the setting button 612 can acquire the user input for displaying the designated window 614. The designation window 614 displays the shape data acquired through the operation on the shape input field 611.

The designation window 614 displays the shape data acquired for the object 103. The user can specify a measurement site, which is a site for measuring the surface of the shape data on the designation window 614. Therefore, the designation window 614 has a shape display field 615, a position display field 619, and a decision button 620. Further, the shape display field 615 displays the shape data 616, the pointer 617, and the measurement site 618. In the shape display field 615, the part designation unit 405 may acquire the user input via the pointer 617 that designates the measurement part 618 on the shape data 616. The user input acquired by the site designation unit 405 is performed via the pointer 617 in FIG. 6, but the type of such operation is not particularly limited. Such user input may be via, for example, a pointer 617 such as a mouse cursor as shown in FIG. 6, or may be via a touch operation by the user. The measurement site 618 may be displayed in any way as long as the designated point is clearly visible, and may be displayed in a shaded display, or may be covered with a dotted line, for example. At this time, the position display field 619 can display the three-dimensional coordinates of the point specified by the user. The site designation unit 405 can determine the measurement site 618 as the currently designated site and save the three-dimensional coordinates thereof as the measurement site P by pressing the decision button 620 by the user. Further, the flatness evaluation unit 406 acquires the radius of curvature at the measurement site 618 from the three-dimensional coordinates of the measurement site P. Next, when the user presses the selection button 613, the selection unit 407 inputs a scale 104 having Δx satisfying the equation (1) in which the smallest value among the radius of curvature of the acquired measurement site 618 is R. Select from the candidates entered in 602. The output unit 408 displays the information indicating the scale 104 selected here on the display device 102.

In the UI of FIG. 6, the method of operation including input to each field is not particularly limited. The user may perform the above-mentioned operation using, for example, a keyboard and a mouse, or may perform the above operation using a touch panel. Further, FIG. 6 is an example in the present embodiment, and the UI is not limited to such a form. For example, in order to acquire the value to be input in each field of the input tab 602, the input tab 602 acquires the image data of the scale candidate stored in the storage device (not shown), recognizes the reference point, and recognizes the reference point. And the value of each column may be obtained by measuring the value of each column on the image. The image data of the scale candidate may be stored in the storage device, or may be acquired by photographing the scale candidate attached to the plane with a photographing device having a known focal length. In such a case, it is assumed that the actual dimensions of the entire area of the scale candidate or each area are known. According to such a configuration, the measurement site can be specified using the shape data of the object 103, and an appropriate scale 104 can be selected from the scale candidates according to the shape data.

FIG. 7 is a flowchart showing an example of a processing procedure for performing processing by the information processing apparatus 101 according to the present embodiment. In the present embodiment, the information processing apparatus 101 selects an appropriate scale 104 that can assist in estimating the orientation of a designated portion on the surface of the object 103 based on the shape data of the object 103, and such a scale 104 Information indicating that can be output. In the following, a scale that can assist in estimating the orientation of a designated portion on the surface of the object 103 will be referred to as an appropriate scale.

In step S701, the scale acquisition unit 402 acquires the ID stored in each code area and the interval Δx between the reference points 202 as information indicating the scale candidate group from the scale holding unit 401. In step S702, the shape acquisition unit 404 acquires shape data corresponding to the object given in advance from the shape holding unit 403. Next, in step S703, the site designation unit 405 acquires the position of the measurement site designated on the shape data as the measurement site P. In the present embodiment, the site designation unit 405 acquires the three-dimensional coordinates of the designated point as the measurement site P via the designation window 614 in FIG. 6 described above.

In step S704, the flatness evaluation unit 406 calculates the condition of the scale 104 so that the orientation of the surface of the measurement site P on the shape data can be measured when the measurement site P is attached to the measurement site P. Although a detailed description here will be described later together with the flowchart of FIG. 8, the flatness evaluation unit 406 can calculate _{Δx max, which is the maximum Δx satisfying the equation (1).} In step S705, the selection unit 407 selects an appropriate scale 104 from the scale candidate group acquired by the scale acquisition unit 402 based on the conditions relating to the scale 104 calculated by the flatness evaluation unit 406. The processing here is described as an example in FIG. 9, and the details will be described later.

In step S706, the output unit 408 outputs information based on the selection result of the selection unit 407 on the display device 102. If the information indicating the appropriate scale 104 can be output in the process in step S705, the output unit 408 outputs all the information indicating the appropriate scale 104 on the display device 102. At this time, the method of outputting the information indicating the scale 104 is not particularly limited as long as the scales 104 can be discriminated from each other. For example, the output unit 408 may output only the ID of the scale 104, or may output the image data of the scale including the ID of the scale 104. As the image data of the scale 104, the image data corresponding to the numerical value input in the input tab 602 of the scale holding unit 401 may be stored in the storage device in advance, or the numerical value input in the input tab 602 may be used. A generation unit (not shown) may be generated accordingly. If the information indicating the appropriate scale 104 cannot be output, the output unit 408 can output a warning on the display device 102 notifying that the scale cannot be output because the scale satisfying the condition cannot be found.

FIG. 8 is a flowchart showing an example of a processing procedure for performing evaluation by the flatness evaluation unit 406 in step S704. In the present embodiment, the flatness evaluation unit 406 can calculate the condition of the possible value of the distance Δx between the reference points 202 on the appropriate scale 104 from the radius of curvature at the measurement site.

In step S801, the flatness evaluation unit 406 refers to the three-dimensional coordinates of the measurement portion P of the shape data and acquires the radius of curvature at the measurement portion P. The radius of curvature at the measurement site P may have a different value depending on the measurement direction. Here, the flatness evaluation unit 406 acquires the radius of curvature at the measurement site P that has the maximum value, and the reason will be described in detail in the second embodiment described later. The method of obtaining the radius of curvature is not particularly limited. For example, the flatness evaluation unit 406 can cut the polygon of the shape data through the measurement site P in the direction in which the radius of curvature is desired to be obtained. Next, the flatness evaluation unit 406 obtains the apex closest to the measurement site P and one apex on each side sandwiching the apex from the cut surface, and determines the radius of curvature as the radius of a circle passing through the three points. You may get it. The largest value among the radii of curvature calculated here is R _max . The direction for obtaining the radius of curvature of the measurement site P is not particularly limited. The flatness evaluation unit 406 first calculates the radius of curvature of the measurement site P in an arbitrary direction as the radius of curvature of the measurement site P, and then 10 degrees from that direction with the normal direction of the measurement site P as the axis. , The radius of curvature in the direction rotated up to 170 degrees may be obtained. In such a case, the normal direction of the measurement site P is calculated from the shape data.

In step S802, the flatness evaluation unit 406 determines whether the measurement position P has a _{curvature in substantially only one direction, based on the maximum radius of curvature R max at the measurement position P.} That is, the flatness evaluation unit 406 determines whether or not the measurement position P can be approximated to the side shape of the cylinder based on the criteria described later. This means that when the scale 104 is attached to the object 103, the flatness evaluation unit 406 determines whether or not the orientation of the surface can be measured with high accuracy without forming wrinkles on the scale 104. is there. _{Specifically, it is determined whether or not the absolute value of R max} , which is the value of the maximum radius of curvature, is equal to or greater than a predetermined threshold value Th2. If it is determined that the value of R _max satisfies the following equation (2), the process proceeds to step S803. If _{the absolute value of R max} is not equal to or greater than the threshold value, it is assumed that a usable scale 104 cannot be found, and the operation of the flatness evaluation unit 406 is terminated. Further, _{when it is determined that the value of R max} does not satisfy the equation (2), a warning may be displayed on the display device 102 notifying that the scale cannot be output because the flatness of the measurement site P is low.
｜ R _max ｜ ≧ Th2 equation (2)

In step S803, the flatness evaluation unit 406 acquires the value Rmin of the minimum radius of curvature at the measurement site P in the same manner as in step S801. Next, in step S804, the flatness evaluation unit 406 calculates _{Δx max,} which is the maximum value among Δx satisfying the equation (1) in which Rmin is substituted for the value of R. The flatness evaluation unit 406 evaluates that the arc between the adjacent reference points 202 has high flatness when the scale 104 in which the lateral distance Δx of the reference points is Δx _{max or less is attached to the measurement site P.} Such a reference point 202 can assist in estimating the orientation of the measurement site P. The method of estimating the orientation of the measurement site P will be described later.

FIG. 9 is a flowchart showing an example of a processing procedure for making a selection by the selection unit 407 in step S705. The selection unit 407 selects an appropriate scale by determining whether or not the conditions calculated by the flatness evaluation unit 406 are satisfied with respect to the information indicating all scale candidates held by the scale holding unit 401. In step S901, the selection unit 407 acquires _{Δx max calculated by the flatness evaluation unit 406.} The processing in the subsequent steps S902 and subsequent steps is performed for each of the scale candidates held in the scale holding unit 401, one for each of the scale candidates held in the scale holding unit 401.

_{In step S902, the selection unit 407 acquires the lateral distance Δx M} of the reference point 202 for each scale candidate acquired by the scale acquisition unit 402. In step S903, the selection unit 407 _{determines whether or not Δx M} is Δx _max or less, as in the following equation (3). That is, it is determined whether or not the flatness of the arc between the reference points 202 of the scale candidate attached to the measurement site P becomes high. If it is determined that the equation (3) is satisfied, the process proceeds to step S904. When it is determined that the equation (3) is not satisfied, the flatness of the arc between the reference points 202 does not increase in the scale candidates attached to the measurement site P, and the estimation of the direction of the measurement site P is sufficiently accurate. It is judged that it cannot be done with. In the case where the formula (3) are not satisfied with respect to [Delta] x _M of all scales candidate display device 102 may display a warning informing of being unable output scale due to missing satisfies scale ..
Δx _M ≤ Δx _max equation (3)

In step S904, the selection unit 407 records information indicating all the appropriate scales 104 determined in step S903 that the arc between the reference points 202 becomes more flat when attached to the measurement site P. According to such a configuration, an appropriate scale 104 can be selected from the held scale candidates.

When the output unit 408 can output information indicating an appropriate scale 104 on the display device 102, or the equation (2) is not satisfied in step S802 or the equation (3) in step S903. A warning can be output. FIG. 10 is a diagram showing an example in which the output from the output unit 408 is displayed on the display device 102. In FIG. 10, the shape data of the object 103 and the information or warning indicating the scale 104 are displayed on the output display field 1001 of the display device 102. In the output display field 1001, for example, the information indicating the scale 101 to be displayed may be switched and displayed according to the surface shape of the measurement portion designated on the shape data displayed in the designated window 614 of FIG. The type of the output display field 1001 is not particularly limited as long as the shape data of the object 103 and information or a warning indicating such a scale 104 can be output. The output display field 1001 may be, for example, an independent field existing on the display device 102, or the shape display field 615 may display information or a warning indicating the scale 104. Further, the output display field 1001 may display, for example, the shape data of the object 103 and the information or warning indicating the scale 104, or may display only the information or warning indicating the scale 104.

FIG. 10A shows an example in which two suitable scales 104 are found, and the output display column 1001 includes a measurement site 1002 and a selection display column 1003 for displaying the scale 104. In this example, the selection display field 1003 displays its ID and image data as information indicating the scale 104. The selection display field 1003 may display all the information indicating the acquired scale 104, or may selectively display one of the information indicating the scale 104. When one of the information indicating the scale 104 is selectively displayed, the standard is not particularly limited. For example, the selection display field 1003 may display information indicating the scale 104 having the largest value of Δx among the appropriate scales 104. FIG. 10B is an example in the case where the maximum radius of curvature R _{max at} the measurement site P does not satisfy the equation (2) in step S802, and for the display, the output display column 1001 is the measurement site. It has 1004 and warning 1005. In this example, when the scale 104 is attached to the measurement site 1004, wrinkles are generated on the scale 104, so that the orientation of this site cannot be estimated accurately. Therefore, the warning 1005 has a message to convey that the scale cannot be output because the flatness of the measurement site is low. FIG. 10C is _{an example in the case where Δx M} and Δx _max do not satisfy the equation (3) in step S903, and for the display thereof, the output display column 1001 has the measurement site 1006 and the warning 1007. have. In this example, as described above, there is no suitable scale 104 that can be displayed on the output display field 1001. Therefore, the warning 1007 has a message telling that there is no scale 104 that can be output because a scale satisfying the condition cannot be found.

According to such a configuration, based on the shape data of the object, a reference point that can assist in estimating the orientation of the designated part with sufficient accuracy when attached to the designated part of the object. It is possible to obtain an information processing apparatus that outputs information indicating a scale having an array of.

[Analysis equipment]
By measuring the orientation of the surface of the object 103, it becomes easy to analyze the color of the surface based on the direction of the position from the position of the surface to the light source and the photographing apparatus. From such a viewpoint, the analyzer 107 according to the present embodiment measures the color of the designated portion on the captured image. Further, the analyzer 107 provides information on the positions of the light source and the photographing device with respect to the designated portion, reflection information indicating the light reflection characteristics of the object for which the color is compared with the object, and information on the orientation of the surface at the designated portion based on the scale. To get. Further, the analyzer 107 calculates a reference color to be compared with the color of the designated portion based on each acquired information, compares the color of the designated portion with the reference color, and analyzes the color of the designated portion.

In FIG. 1, the light source 105 is used as the light source when the object 103 is photographed. The type of the light source 105 is not particularly limited. The light source 105 may be, for example, an artificial solar lamp or an LED light. Further, in this example, the number of light sources 105 is one, but the number is not particularly limited to that. As the light source 105, for example, a plurality of the same light source may be used, or a plurality of different light sources may be used in combination.

The photographing device 106 acquires a photographed image. The photographing device 106 is a digital camera capable of acquiring a photographed image as a two-dimensional distribution of color information indicating colors. In the present embodiment, the format of the color information is not particularly limited. The color information includes, for example, three components of R, G, and B (hereinafter referred to as RGB values), three components of X, Y, and Z (hereinafter referred to as XYZ values) of each pixel, or L ^* , a ^* , b ^*. 3 components (hereinafter referred to as L ^* a ^* b ^* value) may be contained. That is, for example, when the object 103 is photographed, the photographing device 106 may acquire data including RGB values of each pixel of the surface region of the object 103 in the photographed image as information indicating the color of the designated portion. it can.

The analyzer 107 analyzes the color information of the object 103. For example, the analyzer 107 analyzes the color information of the object 103 by comparing the colorimetric value indicating the color of the designated portion with the reference value indicating the reference color. In such a case, the analyzer 107 analyzes the color by comparing the colorimetric value and the reference value at the designated portion with the same color model and calculating the color difference which is the difference between those values. Can be done.

The color difference may be the Euclidean distance between the two colors in the same color space. ^{As the color difference, for example, ΔE *} _ab defined by the International Commission on Illumination (CIE) as the color difference can be used. ΔE ^* _ab is ^{calculated from the coordinates of two points in the L *} a ^* b ^* coordinates using the color difference formula defined by CIE. As the color difference formula, for example, CIE76, CIE94 and CIEDE2000 may be used. In the present embodiment, CIE94 is used as the color difference formula, but the present invention is not particularly limited thereto. As the color difference, for example, ΔL, Δa, Δb or ΔC may be calculated. ΔC is the difference in saturation in the LCh color space calculated from ^{the L *} a ^* b ^{* color space.} The calculation by the color difference formula will be described later.

The L ^* a ^* b ^* coordinates are the coordinates in the complementary color space having the dimension L * meaning lightness and the complementary color dimensions a * and b *. The L ^* a ^* b ^* coordinates are coordinates in which a change in the coordinates indicating a color is likely to be equal to a change in the perception of the color by a person who recognizes the color. That is, ^{the Euclidean distance between two points at the L *} a ^* b ^* coordinates can be approximated to the relative perceptual difference between the colors of the two points. From the viewpoint of numerically indicating the visual color difference, the analyzer 107 can perform quality control based on, for example, the color of the object by calculating the above-mentioned color difference based on the ^{L *} a ^* b ^{* value.} it can. The L ^* a ^* b ^* coordinates can be calculated from, for example, the XYZ coordinates. The calculation of the L ^* a ^* b ^* coordinates will be described later.

Hereinafter, the steps performed by the analysis system according to the present embodiment will be described in detail. FIG. 3 is a flowchart showing an example of a processing procedure for performing the analysis according to the present embodiment. In the present embodiment, the analyzer 107 and the information processing device 101 can perform the following operations on each region of the entire surface of the object that can be photographed.

In step S301, the information processing apparatus 101 selects the scale 104 as described above based on the shape data of the object 103. In step S302, the display device 102 displays the result of selection of the scale 104 by the information processing device 101. The display device 102 may display one or more scales 104, or may display a warning based on the failure to acquire the scales 104.

In step S303, the photographing apparatus 106 photographs the designated portion of the object 103 to which the scale 104 is attached. Before taking a picture of the photographing device 106, the user sets the scale 104 on the designated part of the object 103 corresponding to the measurement part of the shape data in the direction corresponding to the direction with respect to the measurement part when the appropriate scale 104 is selected. Paste it. When a plurality of information indicating the scale 104 displayed on the display device 102 exists, the user may select one of the scales 104 to be used. In such a case, the method of selecting the scale 104 is not particularly limited. For example, the user may select the scale 104 having the largest value of Δx from the information indicating the scale 104 as the scale 104 using the information having the smallest value of Δx. When acquiring the scale 104, the acquisition destination is not particularly limited. For example, the user may use the scale 104 printed on the base material in advance, or may print and use the scale 104 according to the selection result. Next, in step S304, although details will be described later, the analyzer 107 analyzes the color of the designated portion by calculating the difference between the value indicating the color of the designated portion and the value indicating the reference color.

FIG. 11 is a block diagram showing an example of the functional configuration of the analyzer according to the present embodiment. In this example, the analyzer 107 has a connection with the imaging device 106. The analyzer 107 analyzes the color of the designated portion of the object 103. Therefore, the analyzer 107 includes an image acquisition unit 1101, a shape calculation unit 1102, a light source acquisition unit 1103, a reflection characteristic holding unit 1104, and an analysis unit 1105.

The image acquisition unit 1101 acquires the digital image data captured by the photographing device 106. The shape calculation unit 1102 calculates the shape of the designated portion based on the scale 104 and the reference point 202 in the image. The processing by the shape calculation unit 1102 is shown in FIGS. 15, 16, 17, and 20, and a detailed description will be described later.

The light source acquisition unit 1103 acquires information on the position of the light source 105 that illuminates the object 103 from the captured image. As shown in FIG. 12, the photographing device 106 photographs the object 103 and the light source measuring unit 1201 in one image. As a result, the position of the light source 105 can be acquired from the captured image. FIG. 12 shows an example in which the light source acquisition unit 1103 acquires information on the position of the light source 105. The type of the light source measuring unit 1201 is a mirror-shaped hemisphere in FIG. 12, but is not particularly limited as long as the position of the light source 105 can be calculated from the captured image. The light source measuring unit 1201 may be, for example, a mirror-shaped sphere or a hollow transparent sphere. Further, the position where the light source measuring unit 1201 is installed is not particularly limited as long as the position of the light source 105 can be calculated from the captured image. The light source measuring unit 1201 may be installed, for example, on the wall surface of a space for photographing, may be installed on the floor of the same space, or may be suspended in the same space.

FIG. 21 shows an example of a method of calculating the direction of the light source from the captured image. In the example of FIG. 21, a mirror-shaped hemisphere is used as the light source measuring unit 1201 as in FIG. The light source calculation unit 1803, which will be described later, can calculate the direction of the light source from the image of the light source reflected on the mirror surface of the light source measurement unit 1201 in the captured image. In this example, the light source calculation unit 1803 obtains the direction from the center of the hemisphere to the light source by a three-dimensional vector. FIG. 21A shows an example of an image of the light source measuring unit 1201 when the photographing device 106 photographs the light source measuring unit 1201 under the light source 105. The image 2101 is an image of the light source 105 reflected on the mirror surface of the light source measuring unit 1201. Further, FIG. 21B shows an example of a method in which the light source acquisition unit 1103 calculates the direction of the light source from the image 2101 displayed on the light source measurement unit 1201 in the captured image. In FIG. 21B, above the semicircle 2103, the position of the light source 2101 reflected on the light source measuring unit 1201 and the height z indicating the position are shown, which are indicated by the light source measuring unit 1201 to the light source 2101. Corresponds to the direction to. Further, FIG. 21B shows a mirror image 2102 on the captured image, and each concentric circle corresponds to a point at an equal angle on the semicircle 2103. Therefore, the light source acquisition unit 1103 is directed from the light source measurement unit 1201 to the light source 2101 based on the positional relationship between the light source image 2101 and the concentric circles in the image 2102, or based on the xy coordinates of the light source image 2101. Can be determined.

The reflection characteristic holding unit 1104 has a reference value which is information indicating a reference color when the color of the object 103 is analyzed. That is, the reflection characteristic holding unit 1104 has information on the reflection characteristic given in advance for an object that serves as a color reference when analyzing the color of the object 103. The object that serves as a reference for the color of the object 103 is not particularly limited, and may be, for example, a flat plate made of the same material as the object 103, or a painted plate representing the color of the object 103. .. The reflection characteristic information is information indicating the color information of the object that serves as a color reference when the positional relationship between the light source 105, the object that serves as a color reference, and the photographing apparatus 106 changes in various ways. Information on such reflection characteristics is calculated in advance as, for example, RGB values. FIG. 13 shows an example of a method for calculating such reflection characteristics. In this example, the reference plate 1301, which is a flat plate made of the same material as the object 103, is photographed when the positional relationship between the imaging device 106 and the light source 105 changes in various ways. In FIG. 13A, by controlling the positions and orientations of the light source 105 and the imaging device 106 with respect to the reference plate 1301, the incident angle of the light from the light source 105 with respect to the reference plate 1301 and the direction in which imaging is performed changes. In the example of FIG. 13, the postures of the light source 105 and the photographing device 106 are simultaneously controlled so as to keep facing the point where the photographing is performed on the reference plate 1301.

An example of a method of controlling the positions and orientations of the light source 105 and the photographing apparatus 106 with respect to the reference plate 1301 is shown in FIGS. 13 (b) and 13 (c). In the example of FIG. 13B, it is shown that the light source 105 and the photographing apparatus 106 change the elevation angle with respect to the point where the photographing is performed on the reference plate 1301. That is, the positions and orientations of the light source 105 and the photographing device 106 are various at intervals of 10 degrees from the elevation angle of 0 to 90 degrees, with the plane parallel to the reference plate 1301 as the elevation angle of 0 degrees with respect to the point at which the photographing is performed. It can be controlled by various combinations. In the example of FIG. 13C, the direction (Yaw) angle is used instead of the elevation angle at the point of photographing, and the positions and orientations of the light source 105 and the photographing device 106 are set at intervals of 10 degrees from 0 to 350 degrees. , Each can be controlled in various combinations. By performing imaging while controlling FIGS. 13 (b) and 13 (c), the imaging device 106 is positioned when the positional relationship between the light source 105, the imaging device 106, and the reference plate 1301 changes in various ways. The color of the reference plate 1301 according to the relationship can be acquired. The reflection characteristic holding unit 1104 holds the color information acquired by the photographing apparatus 106 as the reflection characteristic information of the reference plate 1301 in association with the positional relationship at the time of acquisition. The reflection characteristic holding unit 1104 acquires the color of the reference plate 1301 as an RGB value, respectively. Further, in the examples of FIGS. 13 (b) and 13 (c), each angle is controlled by 10 degrees, but the control method is not so limited. For example, the elevation angle and the azimuth angle described above may be controlled once. Further, the interval for performing control may be adjusted according to the respective positional relationships. As such an example, for example, the above-mentioned elevation angle and azimuth angle are controlled at 1 degree intervals when the light source 105 and the photographing device 106 are in the specular reflection direction with respect to the point at which the photographing device 106 is photographed, and as the distance from the positional relationship increases It may be controlled so that the interval becomes large. In this way, the reflection characteristic holding unit 1104 holds the information on the reflection characteristic of the reference plate 1301, so that the analyzer 107 photographs at the designated portion based on the positional relationship between the light source 105, the imaging device 106, and the designated portion. Information on the reference color to be used, that is, the reference value can be calculated.

The analysis unit 1105 compares the information indicating the color of the measurement portion (color measurement value) acquired from the captured image acquired by the imaging device 106 with the information indicating the reference color of the reflection characteristic holding unit 1104 (reference value). Then, by calculating the difference between them, the color of the object is analyzed. Hereinafter, the analysis processing performed by the analysis unit 1105 according to the present embodiment will be described in detail.

FIG. 14 is a flowchart showing an example of a processing procedure for the analyzer 107 according to the present embodiment to perform analysis. In step S1401, the image acquisition unit 1101 acquires digital data of the captured image from the photographing device 106. The means by which the image acquisition unit 1101 acquires the data of the captured image is not particularly limited. For example, the image acquisition unit 1101 may acquire data via a USB cable, data may be acquired via an SD card, or data may be acquired via wireless communication. In step S1402, the shape calculation unit 1102 recognizes the reference point 202 and the code region 203 from the captured image, and measures the shape of the designated portion based on them. Details of this measurement method will be described with reference to FIGS. 15, 16 and 17. FIG. 15 shows the scale 104 attached to the surface of the object 103. As described above, the scale 104 has a reference point 202 and a code area 203.

FIG. 16 is a diagram for explaining the metadata possessed by each reference point on the scale 104. In this example, the scale 104 is a rectangular scale having reference points 202 arranged in a grid pattern (Nx in the horizontal direction and Ny in the vertical direction). Such a scale 104 has metadata based on an array of reference points. In FIG. 16, the scale 104 having a reference point 202 and a rectangular rectangular region having four adjacent reference points as vertices has metadata 1601, 1602, and 1603 shown in FIG. 16A as metadata. doing. In the following, the rectangular area refers to a rectangular rectangular area having four such adjacent reference points 202 as vertices.

The metadata 1601 is an area for recording the XYZ coordinate values (XYZ coordinate system) on the scale 104 for the center point of each reference point 202. The origin and slope of the XYZ coordinate values are not particularly limited. For example, as shown in FIG. 16, the metadata 1601 has the center point of the upper left reference point 202 as the origin, and the relative coordinate values for each center point are one line for each reference point 202. It may have a data structure recorded in. The metadata 1601 records the XYZ coordinate values for all the reference points 202 on the scale 104, that is, Nx × Ny. The metadata 1602 is an area for recording a two-dimensional (UV) coordinate value (UV coordinate system) on the scale 104 for the center point of each reference point 202. The origin and inclination of the UV coordinates are not particularly limited.

Considering the use of captured images in the analysis process in the present embodiment, for example, the metadata 1602 has a positive U-axis and a lower right direction along the edge of the scale 104 with the upper left edge of the scale 104 as the origin. The coordinate values for each pixel value may be indicated with the direction as the positive V-axis. Further, the metadata 1602 may have a data structure in which the coordinate values of the center points of one reference point 202 are recorded in one line. The metadata 1602 may have a data structure in which Nx × Ny UV coordinate values are recorded for all center points on the scale 104. In the metadata 1601 and 1602, the recording order of the information for each of the plurality of reference points 202 is not particularly limited, but both may be recorded in the same order. In this example, for the sake of explanation, in the metadata 1601 and 1602, first, each reference point 202 on the first line on the scale 104 is recorded in order from the left, and then every time one line ends, the next line is all recorded. It shall be recorded in order from the left.

The metadata 1603 may have a data structure in which each rectangular region, as shown in FIG. 16B, is recorded in association with its vertices. The method of recording the vertices of this rectangular area is not particularly limited. For example, in the metadata 1603, as shown in FIG. 16A, for each of the rectangular regions, the rectangular region is in the order in which the center point, which is each vertex, goes around the outer circumference of the rectangle counterclockwise from the upper left vertex. Each one may be recorded in one line. The metadata 1603 may represent the vertices of the rectangular region as, for example, a line number in the metadata 1601 / a line number in the metadata 1602. Further, although the metadata 1603 stores the data of the vertices of the rectangular area in this example, the area in which the metadata 1603 stores the data is not limited to the rectangular area. For example, the data structure of the metadata 1603 may record the vertices of a triangular region that can be obtained when the rectangular region is cut by one diagonal line. The scale 104 used in the present embodiment is selected as a scale that increases the flatness of the arc of the designated portion between the adjacent reference points 202 when the scale 104 is attached to the designated portion. These rectangular and triangular areas above can be approximated to a plane. That is, such a set of rectangular regions and triangular regions can be treated like polygons constituting a designated portion. Hereinafter, for the sake of explanation, the data structure of the metadata 1603 will be described as recording for the rectangular area, but even when such a data structure records for the triangular area, the analyzer 107 is the same. Can be processed. Further, for the sake of explanation, it is assumed that the data structure of the metadata 1603 first records the vertices of the leftmost region and then sequentially records the vertices of the rightward region. The order is not particularly limited in that way.

FIG. 17 is a flowchart showing an example of the calculation process performed by the shape calculation unit 1102 in step S1402. As shown below, in step S1402, the shape calculation unit 1102 recognizes the scale 104 and the reference point from the captured image, acquires the metadata of the reference point, and based on the metadata, the center point of each reference point. Calculate the two-dimensional coordinates.

In step S1701, the shape calculation unit 1102 acquires the image data acquired by the image acquisition unit 1101. In step S1702, the shape calculation unit 1102 acquires the identification number of the scale 104 by extracting the code region 203 on the scale 104 from the image data. In this example, it is assumed that the pattern of the code region 203 is a pattern that is divided into a total of 64 blocks of 8 × 8 pixels and has a margin region as shown in FIG. The extraction method of the code area 203 is not particularly limited. The shape calculation unit 1102 can, for example, binarize the pixel value of an image based on a predetermined threshold value. In such a case, the shape calculation unit 1102 performs edge extraction on the image after the binarization process by using the Canny method. Next, the shape calculation unit 1102 recognizes the position where the code region 203 exists from the image. In such a case, the shape calculation unit 1102 may recognize the position of the code area 203 in the image by recognizing the above-mentioned margin area of the code area 203. That is, the shape calculation unit 1102 extracts the contour in the image by regarding the edge of the pixel whose edge position is in the vicinity of 8 pixels as the contour and grouping the pixels having the same value. Then, the shape calculation unit 1102 can recognize the contour having the shape of the margin region, in this example, the contour of the quadrangle as the contour indicating the code region 203 from the plurality of contours extracted in this way. The shape calculation unit 1102 corrects the recognized contour so as to match the shape of the contour of the code region 203. Further, the shape calculation unit 1102 divides the inside of the corrected quadrangular contour into blocks of 8 × 8 pixels, and reads the identification number based on the pixel value pattern of each block. Since the Canny method is a known technique, detailed description thereof will be omitted. Further, when the shape calculation unit 1102 recognizes the contours of a plurality of quadrangles in the image, the following processing steps of the analyzer may be performed on all the recognized contours.

In step S1703, the shape calculation unit 1102 acquires the metadata related to the reference point 202 from the identification number acquired in step S1702. The metadata may be calculated from the information input to the input tab 602 of the scale holding unit 401, respectively, or may be acquired by user input, depending on the identification number.

In step S1704, the shape calculation unit 1102 extracts the center point of each reference point 202 on the scale 104 from the captured image, and calculates the two-dimensional coordinates of such a center point on the image. Therefore, the shape calculation unit 1102 extracts the contour in the image in the same manner as in step S1702. Next, the shape calculation unit 1102 extracts a plurality of contours of circles or ellipses from the plurality of extracted contours, and calculates the area of those circles or ellipses. The shape calculation unit 1102 calculates the area of the quadrangle corrected in step S1702, and calculates the scale of the scale 104 in the image from the area. Further, the shape calculation unit 1102 calculates the predicted area of the reference point 202 in the image from the scale, and calculates the absolute value of the difference between the predicted area and the area of the extracted circle or ellipse. And the ranking is performed from the one whose absolute value is close to 0. The shape calculation unit 1102 calculates the value of the number of rows possessed by the metadata 1601, and uses each of the circles or ellipses from the higher rank of the above-mentioned area to the number of values of the number of rows possessed by the metadata 1601 as a reference. Recognized as point 202. Further, the shape calculation unit 1102 calculates the two-dimensional coordinates on the image of the center point of the contour of the reference point 202 recognized as such. Further, the shape calculation unit 1102 associates and rearranges the center points so that the coordinates of the center points match the relative positional relationship with the coordinate group recorded in the metadata 1602. Therefore, the shape calculation unit 1102 performs matching with a known feature amount of the scale 104 by using, for example, template matching or feature point extraction, thereby associating and rearranging the recognized reference points. You may go. By such processing, the shape calculation unit 1102 can associate each of the reference points 202 in the image with those stored in the metadata. Further, in this example, when the shape calculation unit 1102 recognizes the reference point 202 from the contour of the circle or the ellipse, the contour of the rank up to the value of the number of rows possessed by the metadata 1601 is used. Not limited. From the viewpoint of preventing erroneous recognition based on low-ranked contours, the shape calculation unit 1102 extracts using the contours of the rank from the value of such number of lines to the value obtained by subtracting an appropriate value such as 2 or 4. The reference point 202 may be recognized from the circle or ellipse formed. Further, when the contour of the scale 104 can be recognized from the contour extracted in step S1702, the shape calculation unit 1102 can perform the above processing only for the circle or the ellipse on the contour of the scale 104. Since template matching and feature point extraction are known techniques, detailed description thereof will be omitted.

In this way, the shape calculation unit 1102 can recognize the designated portion as a set of rectangular regions stored in the metadata 1603 by recognizing the reference point 202 in the image in association with the metadata. In addition, the two-dimensional coordinates of the vertices of each region can be calculated. Further, since the internal parameters of the photographing device 106 are known as described later, the shape calculation unit 1102 calculates the two-dimensional coordinate values of each reference point 202 as the camera coordinate system (uv coordinate system) with the camera as the origin. And can be recorded.

In step S1403, the light source acquisition unit 1103 acquires the direction of the light source as described in the example of FIG. Next, in step S1404, the analysis unit 1105 analyzes the color of the designated portion by acquiring the colorimetric value, acquiring the reference value, and calculating the difference between those values at the designated portion. Next, in step S1405, the analyzer 107 displays the analysis result of the color of the designated portion. The place where the analysis result is displayed in step S1405 is not particularly limited. For example, the analyzer 107 may display the color analysis result on the display device 102, or may display the color analysis result on a second display device (not shown). In such a case, the analyzer 107 may be a device that exists separately from the display device 102 or the second display device, or may exist in the same device. FIG. 18 is a block diagram showing an example of the functional configuration of the analysis unit 1105 according to the present embodiment. The analysis unit 1105 includes a color measurement value calculation unit 1801, a normal calculation unit 1802, a light source calculation unit 1803, a reference value acquisition unit 1804, and an analysis value calculation unit 1805 in order to analyze the color of a designated portion.

The colorimetric value calculation unit 1801 acquires the color information in the captured image as the colorimetric value. For example, the colorimetric value calculation unit 1801 can acquire color information at a designated portion of the object 103 to which the scale 104 is attached as a colorimetric value based on the captured image. The colorimetric value may be, for example, an XYZ value. That is, for example, the color measurement value calculation unit 1801 may acquire the color measurement value of the color information by converting the RGB value included in the image data into the XYZ value. The colorimetric value calculation unit 1801 converts the RGB value to the XYZ value by using, for example, the color processing conditions generated by the method disclosed in paragraphs 17 to 47 of JP2009-260474A. It can be carried out.

The normal calculation unit 1802 calculates the direction of the surface of the object 103 at the designated portion, that is, the normal direction from the two- and three-dimensional coordinates of the reference point calculated by the shape calculation unit 1102. The normal direction may be, for example, a three-dimensional vector. The light source calculation unit 1803 calculates the direction from the light source 105 with respect to the designated portion based on the direction of the light source calculated by the light source measurement unit 1201. The reference value acquisition unit 1804 acquires a reference value for comparison with the colorimetric value. The reference value acquisition unit 1804 reflects the reference value corresponding to the near shooting conditions on the reference plate 1301 based on the normal direction at the designated portion, the direction from the light source 105 to the designated portion, and the direction from the imaging device 106 to the designated portion. Obtained from characteristic information. The analysis value calculation unit 1805 calculates the analysis value based on the colorimetric value and the reference value. The analysis value calculation unit 1805 may calculate the analysis value by calculating the difference between the colorimetric value and the reference value, that is, by obtaining the color difference between the observed color and the reference color. The analysis value is not particularly limited, but may be, for example, an XYZ value or an L ^* a ^* b ^* value. Subsequently, the normal calculation unit 1802, the light source calculation unit 1803, the reference value acquisition unit 1804, and the analysis value calculation unit 1805 will be described with reference to the drawings.

FIG. 20 is a flowchart showing an example of processing performed by the normal calculation unit 1802. In step S2001, the normal calculation unit 1802 acquires the information of the vertices of each rectangular region on the scale 104 by reading each line of the metadata 1603. Next, the normal calculation unit 1802 acquires the three-dimensional coordinates corresponding to the vertices of the rectangular region from the metadata 1601, respectively. As described in step S1704, the calculated two-dimensional coordinates of the center point are associated with the vertices of the region recorded in the metadata 1603. Therefore, in step S2002, the normal calculation unit 1802 reads each line of the metadata 1603 in the same manner as in step S2001, and acquires the two-dimensional coordinates corresponding to the vertices from the information calculated in step S1704. be able to.

In step S2003, the normal calculation unit 1802 describes the rotation matrix consisting of the coefficients of r11 to r33 and the translation vector consisting of the coefficients of t1 to t3 shown in the equation (4) for each region recorded in the metadata 1603. And, respectively, are estimated. fx and fx represent the focal length of the photographing device 106, and cx and cy represent the principal points of the photographing device 106, respectively. fx, fy, cx, and cy may be known, respectively. Further, the normal calculation unit 1802 may calculate the uv coordinate of the point from the XYZ coordinates of the XYZ coordinate system based on the equation (4). In the formula (4), (u, v) is the uv coordinate of the center point acquired in step S2002, and (X, Y, Z) is the XYZ coordinate of the same center point acquired in step S2001. The normal calculation unit 1802 may apply the equation (4) to all the center points for which the uv coordinates have been acquired in step S2002. Based on the known PnP (Perjective-n-Point) problem, the normal calculation unit 1802 uses three or more sets of the corresponding (u, v) and (X, Y, Z) from the equation (4). , Rotation matrix and translation vector can be estimated.

In step S2004, the normal calculation unit 1802 uses the XYZ coordinates of each center point recorded in the metadata 1601 to xyz of the corresponding center points in the xyz coordinate system (camera coordinate system) with the photographing device 106 as the origin. The coordinates can be calculated. Such a calculation is performed by the formula (5) shown below. R is a rotation matrix and t is a translation vector, which was estimated in step S2003. Next, the normal calculation unit 1802 calculates the normal vector of the rectangular region stored in the metadata 1603 from the coordinates of each center point. The normal calculation unit 1802 may calculate, for example, the normal vector at each vertex of the rectangle as the normal vector of the rectangular region from the viewpoint that the rectangular region can be approximated to a plane. That is, the normal calculation unit 1802 may use the normal vector at the apex of the rectangular vector as the normal vector in each pixel on the rectangular region. Specifically, the normal calculation unit 1802 calculates a vector from the xyz coordinate of a vertex to two other points that do not exist on the diagonal of the vertex in each rectangular region, and the outer product of the two vectors. The unit vector of can be calculated as the normal vector of the rectangular region. Further, for example, when the Z values in the XYZ coordinates of the vertices on the same rectangular region are equal, the normal calculation unit 1802 uses a unit vector having r13, r23, and r33 as components in the third column of the rotation matrix. , May be calculated as a normal vector of the rectangular area. The normal calculation unit 1802 can continuously perform a series of processes from step S2001 to step S2004 on the same vertex and rectangular region. Further, the normal calculation unit 1802 can perform such a series of processing on the vertices of all the rectangular regions observed in the captured image.

The light source calculation unit 1803 calculates the direction from the light source to the designated portion based on the direction of the light source calculated by the light source measurement unit 1201. For example, the light source calculation unit 1803 is based on the unit vector of the direction from the center of the light source measurement unit 1201 to the light source, the distance from the center of the light source measurement unit 1201 to the light source, and the coordinates of the center of the light source measurement unit 1201 in the xyz coordinate system. , The xyz coordinates of the light source can be calculated. Next, the light source calculation unit 1803 calculates a three-dimensional vector from the light source to the point on the designated portion based on the xyz coordinates of such a light source and the xyz coordinates of the point on the designated portion calculated in step S2004. can do. Further, for example, the coordinates of the light source in the xyz coordinate system may be known in advance. Even in such a case, the light source calculation unit 1803 moves from the light source to the point on the designated portion based on the xyz coordinates of such a light source and the xyz coordinates of the point on the designated portion calculated in step S2004. A three-dimensional vector can be calculated.

FIG. 22 is a diagram for explaining a method in which the reference value acquisition unit 1804 acquires a reference value based on the normal direction at the designated portion, the direction with respect to the light source, and the reflection characteristics of the reference plate 1301. For the sake of explanation, as shown in FIG. 22A, the normal direction at the designated portion is a three-dimensional vector (xn, yn, zn), the direction from the light source to the designated portion (xl, yl, zl), and the photographing apparatus Let the direction vector with respect to the designated portion be (xc, yc, zc). The reference value acquisition unit 1804 can calculate a rotation matrix such that the vector in the normal direction is (0, 0, (xn ² + yn ² + zn ² ) ^1/2). Next, the reference value acquisition unit 1804 rotates the three-dimensional vectors (xl, yl, zl) and (xc, yc, zc) using the same rotation matrix, and sets the value of each rotated vector to (xl'. , Yl', zl') and (xc', yc', zc'). An example of the result of such rotation is shown in FIG. 22 (b). Further, the reference value acquisition unit 1804 acquires a reference value based on the rotated vector acquired by the reference value acquisition unit 1804 from the information indicating the reflection characteristic of the reference plate 1301 held by the reflection characteristic holding unit 1104. To do. For example, the reference value acquisition unit 1804 has a positional relationship acquired by the reference value acquisition unit 1804 among the positional relationships of the reference plate, the light source, and the photographing device that hold the information held by the reflection characteristic holding unit 1104. The reference value on the reference plate 1301 in the closest positional relationship can be obtained. Further, for example, the reference value acquisition unit 1804 may acquire the reference value by acquiring a plurality of information indicating the reflection characteristics held by the reflection characteristic holding unit 1104 and calculating the average of them. Further, for example, the reference value acquisition unit 1804 may convert the reference value acquired in RGB values into an XYZ value. The reference value acquisition unit 1804 can perform the conversion from the RGB value to the XYZ value in the same manner as the process performed by the colorimetric value calculation unit 1801.

The analysis value calculation unit 1805 calculates the analysis value from the color measurement value calculated by the color measurement value calculation unit 1801 and the reference value acquired by the reference value acquisition unit 1804. The analysis value calculation unit 1805 may use the XYZ value as the color measurement value and the reference value. Further, the analysis value calculation unit 1805 may calculate the analysis value as an L ^* a ^* b ^* value.

Hereinafter, in order to calculate the analysis value by the L ^* a ^* b ^* value, a method of converting the colorimetric value and the reference value into the L ^* a ^* b ^{* value will be described.} Here, for the sake of explanation, the colorimetric value is an XYZ value (Xc, Yc, Zc), the reference value is an XYZ value (Xb, Yb, Zb), and the reference white is an XYZ value (Xw, Yw, Zw). The analysis value calculation unit 1805 uses, for example, the following equations (6), (7) and (8) to include (X, Y, Z) = (Xc, Yc, Zc) and (X, Y, Z). ) = (Xb, Yb, Zb) can be substituted to calculate (XRate, YRate, ZRate). (XRate, YRate, ZRate) may be a value capable of calculating ^{the L *} a ^* b ^* value of the analytical value according to the following equations (9) and (10). As for (XRate, Yrate, Zrate), those based on (Xc, Yc, Zc) are defined as (XcRate, YcRate, ZcRate), and those based on (Xb, Yb, Zb) are defined as (XbRate, YbRate, ZbRate).

^{The analysis value calculation unit 1805 may calculate the analysis value as an L *} a ^* b ^* value based on the (XRate, YRate, ZRate) calculated by the formulas (9) and (10). As the analysis value, the analysis value calculation unit 1805 can calculate the analysis value by, for example, the following formula (11). The analysis value calculation unit 1805 can calculate any one or more of ΔE, ΔL, Δa, Δb and ΔC represented by the equation (11) as the analysis value.

In the above-mentioned series of processes, the coordinates of the vertices of the rectangular region are used as the coordinates of the designated portion, but the coordinates used are not particularly limited to the vertices. The analyzer 107 according to the present embodiment may analyze color information in pixels on a rectangular region, for example. Therefore, the analyzer 107 can acquire the uv coordinates of the pixels on the rectangular region on the image. Next, the analyzer 107 can calculate the xyz coordinate of the pixel from the xyz coordinate of the rectangular region based on the uv coordinate of the pixel and the uv coordinate of each vertex of the rectangular region. Further, for example, since the analyzer 107 knows fx, fy, cx, and cy, that is, the internal parameters of the photographing apparatus 106 are known, even if the xyz coordinates are calculated from the above-mentioned uv coordinates. Good. As the normal vector on the rectangular region, the normal vector at the apex of the rectangle may be used as described above.

FIG. 19 is a flowchart for showing an example of the analysis process performed by the analysis unit 1105 in step S1404. In step S1901, the colorimetric value calculation unit 1801 calculates the color information of the designated portion as the colorimetric value from the captured image. In step S1902, the normal calculation unit 1802 calculates the normal vector in each of the rectangular regions represented by the reference point 202 based on the metadata 1601 to 1603 associated with the reference point 202 in the image in step S1402. .. In step S1903, the light source calculation unit 1803 calculates the direction from the light source to the designated portion from the captured image. In step S1904, the reference value acquisition unit 1804 obtains reference color information that the designated portion will indicate from the normal vector of the rectangular region, the direction from the light source to the designated portion, and the position in the direction from the photographing device to the designated portion. Obtained as a reference value. In this example, the reference value acquisition unit 1804 performs the calculation on the coordinates with the camera as the origin, but the calculation is not particularly limited to that. In step S1905, the analysis value calculation unit calculates the color analysis value of the designated portion from the colorimetric value calculated in step S1901 and the reference value acquired in step S1904.

According to such a configuration, the colorimetric value of the designated portion can be obtained from the captured image, and the orientation of the surface of the object at the designated portion can be calculated. Next, the reference value of the designated portion can be obtained from the orientation at such a designated portion and the position of the light source and the imaging device with respect to the designated portion. Further, from such a colorimetric value and a reference value, an analyzer that calculates an analysis value of a designated portion can be obtained.

[Embodiment 2]
[Scale generator and printing device]
FIG. 23 shows an example of the configuration of the analysis system having the information processing apparatus 2301 according to the second embodiment. The analysis system according to the second embodiment includes an information processing device 2301, a printing device 2302, and an analysis device 107. The information processing device 2301 generates scale image data based on the measurement site designated on the shape data of the object. The printing device 2302 prints the scale based on the image data of the generated scale. The analyzer 107 performs the same analysis process as in the first embodiment using the printed scale. In FIG. 23, the same reference numerals are given to the same configurations as those in the first embodiment, and duplicate description is omitted. The analysis system according to the second embodiment is the same as the analysis system according to the first embodiment, except that the information processing apparatus 2301 generates a scale and has a connection with the printing apparatus. Although the scale in the present embodiment is not shown in FIG. 23, the configuration having the scale is the same as that of the scale 104 in the first embodiment.

FIG. 24 is a block diagram showing an example of the functional configuration of the information processing apparatus 2301 according to the second embodiment. The information processing apparatus 2301 according to the present embodiment has a scale generation unit 2401 and a scale output unit 2402, and does not have a scale holding unit 401 and a scale acquisition unit 402. Other configurations are the same as those of the first embodiment shown in FIG. The information processing apparatus 2301 according to the present embodiment will not be described in duplicate with the first embodiment. The information processing device 2301 may be connected to the printing device 2302.

The scale generation unit 2401 generates an image of a scale that satisfies the conditions regarding the scale calculated by the flatness evaluation unit 406 as in the first embodiment. That is, the scale generation unit 2401 can generate a scale image having a reference point 202 indicating Δx of _{Δx max or less as the scale image.} In this case, the value of Δx is not particularly limited as long as it _{is Δx max or less.} When processing by the analyzer 107, by taking the value of Δx of the scale to be used as large as possible, the estimation error when calculating the normal vector of the rectangular region can be minimized, and the rectangular region can be processed. The calculation cost can be suppressed by reducing the number. From such a viewpoint, the scale generation unit 2401 generates a scale image in which the value of Δx is Δx _max , which is the maximum value that Δx can take. The scale generation unit 1501 outputs an image of the generated scale to the scale output unit 2402.

The scale output unit 2402 outputs the generated scale image to the display device 102. FIG. 25 is a diagram for explaining an example of a display device 102 that displays an image of the generated scale. The UI displayed on the display device 102 has a generation window 2501, and the generation window 2501 has a shape input field 611, a setting button 612, a scale display field 2502, and a print button 2503. The scale generation unit 2401 generates image data of an appropriate scale at the measurement site designated on the shape data according to the user's operation on the shape input field 611 and the setting button 612 as in FIG. The scale display field 2502 can display the image data of the generated scale. What is displayed in the scale display field 2502 is not limited to scale image data. The scale display field 2502 may display information related to the reference point 202, such as, for example, the number of vertical reference points 605, the vertical spacing 606, the number of horizontal reference points 607, and the horizontal spacing 608. The scale output unit 2402 outputs a scale image to the printing device 2302 when the user presses the print button 2503. The printing device 2302 prints the input image of the scale 104 on the base material. The base material and the recording material are not particularly limited, but as in the first embodiment, the base material can be attached and detached a plurality of times, and the light source and the surrounding reflection at the time of photographing can be reduced. May be good. Further, the scale generation unit 2401 can also acquire scale image data from the information indicating the scale candidate group acquired from the scale holding unit and the scale acquisition unit (not shown), as in the first embodiment.

FIG. 26 is a flowchart showing an example of a processing procedure for performing processing by the information processing apparatus 2301. In FIG. 26, the information processing apparatus 2301 has the same processing procedure as steps S702 to S704 of the first embodiment.

In step S2601, the scale generation unit 2401 generates scale image data from the shape data of the object 103 based on the input of the user. In step S2602, the scale output unit 2402 causes the display device 102 to display the image data of the scale. Next, the scale output unit 2402 outputs scale image data to the printing device 2302. After the printing device 2302 prints the scale, the analyzer 107 performs the same analysis process as in the first embodiment.

According to this configuration, it is possible to obtain an information processing device that generates information indicating the scale based on the shape data of the object. In particular, in the present embodiment, information indicating a scale having an array of reference points that can assist in estimating the orientation of the designated part with sufficient accuracy when attached to the designated part of the object is provided. Can be generated. Further, the printing device can print the information indicating the scale generated by the information processing device on the base material.

[Embodiment 3]
The information processing apparatus according to the third embodiment can specify two measurement sites on the shape data of the object and generate a scale suitable for estimating the orientation of the corresponding designated sites for the two measurement sites. it can. The configuration of the analysis system including the information processing apparatus 2301 according to the third embodiment may be the same as the configuration in the second embodiment shown in FIG.

In the third embodiment, the information processing apparatus 2301 generates one or more scale image data based on the two designated measurement sites. Next, the printing device 2302 prints the scales based on the image data of the scales. The analyzer 107 performs the same analysis processing as in the first embodiment for each of the printed scales.

FIG. 27 is a block diagram showing an example of the functional configuration of the information processing apparatus 2301 according to the third embodiment. The information processing apparatus 2301 according to the present embodiment has a site designation unit 2701, a position evaluation unit 2702, and a scale generation unit 2703, and does not have a site designation unit 405 and a scale generation unit 2401. Other configurations are the same as those of the second embodiment shown in FIG. 24. The information processing apparatus 2301 according to the present embodiment will not be described in duplicate with the first and second embodiments.

The site designation unit 2701 designates two measurement sites, which are the sites to be measured by the analyzer 107, on the shape data acquired by the shape acquisition unit 404. When the part designation unit 2701 designates the measurement part, the designation may be made on the UI displayed on the display device 102. An example of a UI for designating a measurement site is shown in FIG.

On the UI of FIG. 28, two measurement sites are designated on the shape data of the displayed object. This UI has a designated window 2801. The designation window 2801 displays the shape data of the object, and the user specifies two measurement sites on the shape data and determines the measurement site. Therefore, the designated window 2801 has a shape display field 615, a position display field 2804, and a decision button 620.

The shape display field 615 displays the shape data 616 and the pointer 617 as in the example of FIG. Further, the user can specify two measurement sites on the shape data 616 by operating the pointer 617. In this case, the user can designate the measurement sites 2802 and 2803 as the measurement sites by the same operation as in the first embodiment. The measurement sites 2802 and 2803 may be displayed in any manner as long as the designated points can be clearly identified, for example, they may be displayed in a shaded display or may be covered with a dotted line. Good. The measurement sites 2802 and 2803 may have different display forms so that they can be distinguished from each other. At this time, the position display field 2804 can acquire and display the three-dimensional coordinates 2805 and 2806 of the designated measurement sites 2802 and 2803, respectively. In the following, the measurement sites 2802 and 2803 will be referred to as P1 and P2, respectively. Next, the site designation unit 2701 can determine P1 and P2 as the currently designated site and save the three-dimensional coordinates thereof by the user pressing the decision button 620. According to this configuration, two measurement sites can be specified on the shape data 616 of the object.

The position evaluation unit 2702 evaluates the positional relationship of the three-dimensional coordinates of the two measurement sites P1 and P2 designated by the site designation unit 2701. That is, the position evaluation unit 2702 determines whether or not P1 and P2 can be analyzed on one scale when the analyzer 107 analyzes them.

FIG. 29 shows an example in which scales need to be prepared individually for P1 and P2. FIG. 29A is a diagram when the distance between P1 and P2 on the shape data 616 is larger than the value of the lateral length of the cut area of the scale. Region 2901 shows a part of the surface of the shape data 616, point 2902 shows P1 and point 2903 shows P2. In this case, the direction pointed to by the lateral direction follows the lateral direction in the arrangement of the reference points 202 in the first embodiment. Further, the scale 2904 in this figure may have the same configuration as the scale 104 in the first embodiment. When the distance between P1 and P2 exceeds the lateral length of the cut region as described above, the analyzer 107 estimates the orientation of the designated portion corresponding to P1 and P2 by one scale. Is impossible. Further, FIG. 29B is a diagram showing an example of a state in which the surface of the shape data 616 is divided by an edge, a step, or the like. In this example, the surface of the shape data 616 is divided into a region 2905 and a region 2906, where point 2907 indicates P1 and point 2908 indicates P2. Even in such a case, it is impossible for the analyzer 107 to estimate the orientation of the designated sites corresponding to P1 and P2 by one scale. That is, if the distance between P1 and P2 is less than or equal to the lateral length of the cut region and there is no edge between the two measurement sites, the position evaluation unit 2702 will perform one scale for P1 and P2. Determines that the analyzer 107 can perform the analysis.

The position evaluation unit 2702 determines whether the distance between P1 and P2 is equal to or less than the lateral length of the cut region, and whether the edge amount between P1 and P2 is equal to or less than a predetermined threshold value. Judge whether or not. The order in which these determinations are made is not particularly limited. The scale generation unit 2703 generates one or two images of scales corresponding to P1 and P2 based on the determination result. The scale generation unit 2703 can generate scale image data in the same manner as the scale generation unit 2401 of the second embodiment. FIG. 30 is a flowchart showing an example of a procedure for performing a determination by the position evaluation unit 2702 and a process by the scale generation unit 2703 following the determination.

In step S3001, the position evaluation unit 2702 calculates the distance between P1 and P2. This distance may be, for example, a linear distance between P1 and P2, or may be a distance along the shape data passing through P1 and P2. In this example, the position evaluation unit 2702 calculates the linear distance between P1 and P2 as the distance between P1 and P2. Further, in this example, for the sake of explanation, the three-dimensional coordinates 2805 of P1 are referred to as (X1, Y1, Z1), and the three-dimensional coordinates 2806 of P2 are referred to as (X2, Y2, Z2). The position evaluation unit 2702 calculates the distance L between P1 and P2 according to the following equation (12).

In step S3002, the position evaluation unit 2702 determines whether the distance between P1 and P2 is equal to or less than the lateral length of the cut region. Therefore, the position evaluation unit 2702 compares the distance L calculated by the formula (12) with the predetermined threshold value ThL according to the following formula (13). The value of the threshold value ThL is not particularly limited as long as it is not larger than the lateral length of the cut area. The threshold value ThL may be, for example, the same value as the lateral length of the cut area, or a value smaller than the lateral length of the cut area by a predetermined amount in consideration of allowing a margin for measurement. May be good. If the distance L satisfies the equation (13), the position evaluation unit 2702 proceeds to step S3003, and if not, proceeds to step S3006.

In step S3003, the position evaluation unit 2702 calculates the edge amount between P1 and P2. This edge amount may be calculated, for example, for the vertices between P1 and P2 on the polygon of the shape data, or for the side between the surface where P1 exists and the surface where P2 exists. You may. In this example, the position evaluation unit 2702 obtains the edge amount calculated for the apex between P1 and P2 on the polygon of the shape data as the edge amount. The position evaluation unit 2702 may calculate the edge amount for all the vertices between P1 and P2 where the scales can overlap when the measurement site is scaled. The position evaluation unit 2702 can calculate the edge amount G (Q) at a certain point Q (X, Y, Z) according to the following formula (14). In equation (14), ∂Q / ∂x represents the partial differential in the X direction at the point Q, and the same applies to the Y and Z directions.

In step S3004, the position evaluation unit 2702 determines whether or not the edge amount between P1 and P2 is equal to or less than a predetermined threshold value. Therefore, the position evaluation unit 2702 determines whether or not the edge amount G calculated by the equation (14) is equal to or less than a predetermined threshold value The. The threshold value The is a threshold value for determining that the edge amount at the point Q is sufficiently small, and may be predetermined as a desired value. If it is determined that the edge amount G is equal to or less than ThE at all the vertices for which the edge amount has been calculated, the position evaluation unit 2702 proceeds to step S3005, and if not, proceeds to step S3006.

In step S3005, which is a case where it is determined that the orientation of the designated portion corresponding to P1 and P2 can be estimated by one scale, the scale generator 2703 is an image of the scale commonly used for P1 and P2. Generate data. When the minimum radius of curvature in P1 and the minimum radius of curvature in P2 are different _{, the value of Δx max} according to the equation (1) also differs between P1 and P2. In such a case, the scale generator 2703 can generate an image of an appropriate scale at P1 and P2 by using _{Δx max} at the measurement site where the minimum radius of curvature is smaller. Further, the scale generation unit 2703 may acquire an image of an appropriate scale in P1 and P2 in the same manner as in steps S701 to S706 in the first embodiment.

In step S3006, which is a case where it is determined that the scales need to be generated individually, the scale generation unit 2703 generates image data of the scales for P1 and P2 separately. In the above-mentioned series of processes, the case where two measurement sites are designated on the shape data of the object has been described, but the designated measurement sites are not limited to two. For example, when three or more measurement sites are designated, the information processing apparatus 2301 generates an image of an appropriate scale by performing the processes of steps S3001 to S3006 for all combinations of those measurement sites. You may.

The analyzer 107 according to the third embodiment acquires a substrate on which an image of the generated scale is printed. The acquisition method is not particularly limited, and printing may be performed using the printing apparatus 2302, or a substrate on which a scale image is printed in advance may be acquired as in the first embodiment.

According to such a processing procedure, the information processing apparatus according to the third embodiment designates two measurement sites on the shape data of the object. The information processing apparatus can then switch between generating one or two appropriate scales by determining whether the two measurement sites can be measured simultaneously on one scale. it can. The analyzer 107 can perform analysis on such one or two scales in the same manner as in the method according to the first embodiment.

[Embodiment 4]
In the above-described embodiment, for example, each processing unit shown in FIG. 4 and the like is realized by dedicated hardware. A part or all of the processing units of the information processing device 101 and the display device 102 may be realized by a computer. In this embodiment, at least a part of the processing according to each of the above-described embodiments is executed by a computer.

FIG. 31 is a diagram showing a basic configuration of a computer. In FIG. 31, the processor 3101 is, for example, a CPU and controls the operation of the entire computer. The memory 3102 is, for example, a RAM, and temporarily stores programs, data, and the like. The computer-readable storage medium 3103 is, for example, a hard disk or a CD-ROM, and stores programs, data, and the like for a long period of time. In the present embodiment, the program that realizes the functions of each part stored in the storage medium 3103 is read into the memory 3102. Then, the processor 3101 operates according to the program on the memory 3102, so that the functions of each part are realized.

In FIG. 31, the input interface 3104 is an interface for acquiring information from an external device. Further, the output interface 3105 is an interface for outputting information to an external device. The bus 3106 connects the above-mentioned parts and enables data exchange.

(Other Examples)
In the above-described embodiment, the scale suitable for the designated part is presented based on the radius of curvature of the designated part specified by the user, but the designated part is suitable for the designated part based on whether the designated part is a flat surface or a curved surface. The scale may be presented. In this case, for example, identification information indicating whether the area is a flat area or a curved surface is added to each area on the displayed three-dimensional shape data in advance. The flatness evaluation unit 406 is based on a scale having a narrow reference point interval for a curved surface and a scale for a flat surface having a wider reference point interval than a curved surface scale based on the identification information of a designated part specified by the user. Select a suitable scale. This makes it possible to present a scale suitable for the designated portion without using the radius of curvature of the designated portion in the object. The identification information is not limited to two values of a flat surface and a curved surface, and the flatness may be identified by three or more values.
Further, although the scale in the above-described embodiment is directly attached to the object, it may not be attached as long as it is arranged so that the orientation of the surface of the object can be estimated. For example, an image of the scale projected on the object by the projector may be used as the scale in the above-described embodiment. The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

101: Information processing device, 404: Shape acquisition unit, 406: Flatness evaluation unit, 408: Output unit

Claims (19)

The first acquisition means for acquiring the shape data of the object,
A first evaluation means for estimating the surface shape at a designated site designated in the displayed shape data, and
A scale arranged at the designated part of the object in order to estimate the orientation of the surface at the designated part, and has a plurality of reference points arranged at intervals corresponding to the estimated surface shape. Output means to output the indicated information and
An information processing device characterized by being equipped with. The information processing apparatus according to claim 1, wherein the scale has a plurality of reference points arranged at intervals corresponding to the degree of distortion of the surface at the designated portion of the object. The first or second method is characterized in that a second acquisition means for acquiring information indicating the scale candidate is further provided, and the output means outputs information indicating the scale selected from the scale candidates. The information processing apparatus according to 2. The information processing apparatus according to claim 3, wherein the second acquisition means acquires a user input indicating information indicating a candidate for the scale. The output means will output a warning when there is no candidate for the scale having a plurality of reference points arranged at intervals corresponding to the estimated surface shape, according to claim 3 or 4. The information processing device described. The information processing apparatus according to any one of claims 1 to 5, wherein the output means generates information indicating the scale. The information processing apparatus according to any one of claims 1 to 6, further comprising a printing means for printing the scale based on the information indicating the scale. The first evaluation means evaluates the curvature of the surface in a plurality of directions at the designated site, and evaluates the curvature of the surface.
The output means
When the largest curvature among the curvatures of the surface in a plurality of directions is equal to or larger than the first threshold value, information indicating the scale is output.
The information according to any one of claims 1 to 7, wherein a warning is output when the largest curvature among the curvatures of the surface in a plurality of directions is smaller than the first threshold value. Processing equipment. The first evaluation means sets a second threshold value according to the curvature of the surface at the designated site.
The information processing apparatus according to any one of claims 1 to 8, wherein the output means outputs information indicating a scale in which the interval between the reference points is equal to or less than the second threshold value. .. The information processing apparatus according to claim 9, wherein the information processing means outputs a warning when the information indicating the scale in which the interval between the reference points is equal to or less than the second threshold value cannot be output. The first evaluation means evaluates the curvature of the surface in a plurality of directions at the designated portion, and sets the second threshold value according to the smallest curvature among the curvatures of the surface in the plurality of directions. The information processing apparatus according to claim 9 or 10. The scale according to any one of claims 1 and 2, wherein the scale is a sheet having the plurality of reference points and an opening or a cut area sandwiched between the plurality of reference points. Information processing equipment. The information indicating the scale is between the image of the scale, the identification information of the scale, the number of rows of the reference point that the scale has, the number of reference points included in the row of the reference point, and the rows of the reference point. The information processing apparatus according to any one of claims 1 to 12, wherein the information processing apparatus includes one or more of the distances or the distances between the reference points included in the line of the reference points. Any of claims 1 to 13, further comprising a third acquisition means for acquiring a user input indicating a designated portion of the object on an image of the object displayed on the display device. The information processing device according to paragraph 1. Further provided with a second evaluation means for evaluating the positional relationship of two or more of the designated parts,
The output means outputs information indicating two or more scales attached to each of the two or more designated parts, or is attached across the two or more designated parts, depending on the positional relationship. The information processing apparatus according to any one of claims 1 to 14, characterized in that information indicating one scale is output or switched. The first display means for displaying the three-dimensional shape data of the object,
Display of a scale arranged at the designated part of the object in order to estimate the orientation of the surface at the designated part according to the surface shape of the designated part designated by the user on the displayed three-dimensional shape data. With a second display means to switch between
The scale is an information processing apparatus having a plurality of reference points arranged at intervals corresponding to the surface shape. The information processing apparatus according to any one of claims 1 to 15.
Based on the photographed image of the object whose scale is attached to the designated portion by the photographing apparatus, the first color of the designated portion of the object and the orientation of the surface of the object at the designated portion. As a means of obtaining information indicating,
A means for acquiring information indicating a second color, which is a reference color of the object, according to the orientation of the surface of the object at the designated portion and the positions of the light source and the photographing apparatus with respect to the designated portion. ,
A means for analyzing the first color based on the second color, and
An analyzer equipped with
An analysis system characterized by being equipped with. It is an information processing method performed by an information processing device.
The process of acquiring the shape data of the object and
The process of estimating the surface shape at the designated part specified in the displayed shape data, and
A scale arranged at the designated part of the object in order to estimate the orientation of the surface at the designated part, and has a plurality of reference points arranged at intervals corresponding to the estimated surface shape. The process of outputting the information to be shown and
An information processing method characterized by being provided with. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 16.

Images