JP6305232B2 - Information processing apparatus, imaging apparatus, imaging system, information processing method, and program. - Google Patents

Description

  The present invention is a technique related to focusing of a plurality of imaging units.

  As a technique using images captured by a plurality of imaging units, there is a technique for estimating the shape of a subject by matching pixel values between a plurality of images obtained by capturing the same subject from different viewpoints with a plurality of cameras. . Thus, in order to estimate the subject shape with high accuracy from the image, it is necessary for each imaging unit to perform imaging while focusing on the subject whose shape is to be estimated. Further, when there are a plurality of subjects whose shapes are to be estimated, it is necessary to focus on all the subjects and perform imaging.

  A technique described in Patent Document 1 is known as a technique in which a plurality of imaging units focus on a plurality of subjects. In Patent Literature 1, the imaging conditions of each imaging unit are set so that each imaging device focuses on a different subject and the focus position of each imaging unit falls within the depth of field of each imaging unit. Have been described.

JP 2012-44540 A

  However, in the technique described in Patent Document 1, when there are a plurality of subjects, each imaging unit calculates a focus position on a separate subject, and each imaging is performed so that all the focus positions are within the depth of field. A large F value is set for the part. When the F value is large, it is necessary to increase the ISO sensitivity or reduce the shutter speed in order to make the exposure appropriate. However, if the ISO sensitivity is increased, noise increases, and if the shutter speed is decreased, subject blurring occurs.

  Therefore, an object of the present invention is to provide a technique for focusing and imaging all subjects even when the F value is small when a plurality of imaging devices capture a plurality of subjects from different directions.

  In order to solve the above problems, an information processing apparatus according to the present invention determines a focus parameter including information indicating a focus position of a plurality of imaging units that capture a scene including a plurality of subjects from a plurality of different viewpoint positions. The information processing apparatus, wherein the shapes and positions of the plurality of subjects are derived based on a plurality of images captured by the plurality of imaging units with an F value larger than an F value in imaging using the focus parameter. And a plurality of imaging units each of which corresponds to at least one of the plurality of subjects based on the shape information acquired by the acquisition unit. Based on the grouping means to be divided into groups, the shape information acquired by the acquiring means, and the grouping by the grouping means, the plurality of imaging Part is to focus on the corresponding object, respectively, and having a determining means for determining the focus parameter.

  When a plurality of imaging devices capture a plurality of subjects from different directions, all the subjects can be focused and imaged even when the F value is small.

The figure which shows the structural example of an imaging system. 1 is a diagram illustrating a configuration of an information processing apparatus according to a first embodiment. 1 is a diagram illustrating a functional configuration of an information processing apparatus according to a first embodiment. 3 is a flowchart illustrating a flow of processing performed by the information processing apparatus according to the first embodiment. The figure explaining the relationship between depth of field and subject depth. FIG. 5 is a diagram for explaining grouping processing according to the first embodiment. FIG. 6 is a diagram for explaining parameter determination processing according to the first embodiment. 9 is a flowchart showing a flow of grouping processing according to the second embodiment. The figure which shows the example of the visibility with respect to each to-be-photographed object of an imaging part. The figure explaining the grouping process of Example 2. FIG. FIG. 9 is a diagram illustrating a functional configuration of an information processing apparatus according to a third embodiment. 10 is a flowchart illustrating a flow of processing performed by the information processing apparatus according to the third embodiment.

<Example 1>
In this embodiment, when shooting a scene including a plurality of subjects from different directions using a plurality of imaging units, the imaging unit is divided into a plurality of groups, and shooting is performed by focusing on different subjects for each group. An example will be described. Thereby, imaging can be performed in a state where all the subjects are in focus. Note that the number of imaging units used in this embodiment is sufficiently larger than the number of subjects to be imaged.

  Hereinafter, the configuration of the imaging system of the present embodiment will be described. FIG. 1 is a diagram illustrating a configuration example of an imaging system according to the present embodiment. The imaging system of the present embodiment includes eight imaging units 101 to 108 and an information processing device 109, and the imaging units 101 to 108 are arranged at different viewpoint positions so as to surround two subjects. The imaging units 101 to 108 receive optical information of a subject with a sensor, and acquire digital data (captured image data) of an image captured by performing A / D conversion. The information processing device 109 calculates the imaging conditions of the imaging units 101 to 108 and controls the imaging units 101 to 108 according to the calculated imaging conditions. In this embodiment, the case where eight image pickup units 101 to 108 are used will be described as an example. However, the number of image pickup units is not limited to this, and an arbitrary number of image pickup units can be used.

  Next, the configuration of the information processing apparatus 109 will be described. FIG. 2 is a block diagram showing an internal configuration of the information processing apparatus 109. The information processing apparatus 109 includes a CPU 201, a RAM 202, a ROM 203, an HDD I / F 204, an HDD 205, an input I / F 206, an output I / F 207, and a system bus 208.

  The CPU 201 is a processor that controls each component of the information processing apparatus 109, and the RAM 202 and the ROM 203 are memories that store various data handled by the information processing apparatus 109. The CPU 201 executes a program stored in the ROM 203 using the RAM 202 as a work memory, and comprehensively controls each component of the information processing apparatus 109 via the system bus 208. Thereby, various processes described later are executed.

  The HDD I / F 204 is an interface such as serial ATA (SATA), for example, and connects the HDD 205 as a secondary storage device. The CPU 201 can read data from the HDD 205 and write data to the HDD 205 via the HDD I / F 204. Further, the CPU 201 can expand the data stored in the HDD 205 to the RAM 202, and can similarly store the data expanded in the RAM 202 in the HDD 205. The CPU 201 can regard the data expanded in the RAM 202 as a program and execute it. The secondary storage device may be a storage device such as an optical disk drive in addition to the HDD.

  The input I / F 206 is a serial bus interface such as USB or IEEE1394. The CPU 201 via the input I / F 206 from the imaging units 101 to 108, the operation unit 209 (for example, mouse and keyboard), the external memory 211 (for example, hard disk, memory card, CF card, SD card, USB memory) and the like. Get the data. What data is acquired will be described later.

  The output I / F 207 is a video output interface such as DVI or HDMI (registered trademark). The CPU 201 can display the captured image of the imaging units 101 to 108 or an image obtained by performing some processing on the captured image on the display unit 210 (various output devices such as a display) via the output I / F 207. it can. Note that a touch panel display may serve as the operation unit 209 and the display unit 210. Although there are other components of the information processing apparatus 109 than the above, they are not the main point of the present invention, and thus the description thereof is omitted.

  Next, the flow of focus position determination processing of the imaging units 101 to 108 in the information processing apparatus 109 according to the present embodiment will be described with reference to the block diagram shown in FIG. 3 and the flowchart shown in FIG. In the information processing apparatus 109 according to the present exemplary embodiment, the CPU 201 executes the program illustrated in the flowchart of FIG. 4 stored in the ROM 203, and performs the focus position determination process by serving as the function configuration unit illustrated in FIG. . Note that the CPU 201 does not have to perform all the processing, and some or all of the processing may be replaced with a processing circuit having a similar function. Further, it is assumed that the F value and the shutter speed have been set for the imaging units 101 to 108 by the user prior to the following processing and stored in the ROM 203. Here, the F value of each imaging unit is set to the minimum value (open value) that can be set in each imaging unit.

  First, in step S401, in the position acquisition unit 301, the three-dimensional positions and orientations of the imaging units 101 to 108 at the time of subject shape measurement (a three-dimensional vector representing the front of the imaging unit and a three-dimensional vector representing the upper side of the imaging unit). ) Is acquired. The three-dimensional position and orientation of the imaging unit are acquired from the imaging units 101 to 108 and the external memory 211 via the input I / F 206. Alternatively, a value input by the user via the operation unit 209 may be acquired. In addition, instead of acquiring externally input values as information indicating the position and orientation of the imaging unit, information indicating the position and orientation of the imaging unit is obtained from the captured image data acquired by the imaging unit using an existing method such as structure from motion. You may make it calculate. The position acquisition unit 301 outputs the information indicating the position and orientation of each imaging unit acquired here to the comparison unit 303 and the parameter determination unit 305, and the process proceeds to step S402.

  In step S402, the shape acquisition unit 302 acquires shape information indicating the approximate positions and shapes of a plurality of subjects to be imaged. The shape information includes information indicating a three-dimensional relative position between the subject and the imaging units 101 to 108 and information indicating a spatial extent of each subject. For example, the shape information includes a plurality of point groups plotted in a three-dimensional space. Expressed in coordinates. It is assumed that the subject shape information is measured in advance by some method and stored in the RAM 202 or ROM 203. Note that any method may be used for measuring the shape information of the subject, and measurement may be performed using an external sensor (not shown) such as a three-dimensional digitizer (3D scanner). The shape may be estimated using a multi-viewpoint image obtained as a preview. As a method for estimating the shape of the subject from the multi-viewpoint image, for example, Multi View Stereo, which is an existing method for measuring the distance based on the positional deviation of the pixels between the multi-viewpoint images, can be used. Since the method for acquiring the shape information of the subject is not the main point of the present invention, a detailed description thereof will be omitted. Whichever method is used to acquire the shape of the subject, it is desirable to sequentially acquire the shape of the subject and update the ROM 203 at the preview stage before shooting. In addition, when shape estimation is performed from a multi-viewpoint image obtained in the preview, if an image captured in a state where the F-number of each imaging unit is increased and the depth of field is deepened, a rough approximation of each subject is used. Can be obtained easily. The shape acquisition unit 302 outputs the acquired shape information to the comparison unit 303, and proceeds to step S403.

  Subsequently, in step S403, the comparison unit 303 acquires the depth of field of each imaging unit. Here, a method of calculating the depth of field from the F value set in advance before imaging and the distance from the imaging unit to the subject is shown. First, F values of the imaging units 101 to 108 are acquired from the ROM 203, respectively. Further, based on the shape information input from the shape acquisition unit 302, a subject distance that is a distance from each imaging unit to the subject is acquired. Here, the distance from each imaging unit to the position of the center of gravity of the shape of all the subjects is defined as the subject distance. The depth of field Dn in the n-th imaging unit can be calculated by the following equation where F value is Fn and subject distance is dn.

  Here, ε represents the diameter of the allowable circle of confusion, and fn represents the focal length of the lens. The diameter ε of the allowable circle of confusion is a value indicating how much blur is allowed. The diameter of the permissible circle of confusion varies depending on the sensor size of the imaging unit, the pixel pitch on the sensor, and the like. For example, in the case of an APS-C size sensor, a value of 0.019 mm is generally used. Any method other than the above may be used to calculate the depth of field. In addition, instead of calculating each time as described above, values calculated in advance may be stored in the ROM 203 as a table, and the subject depth may be acquired by referring to it. The comparison unit 303 acquires the depth of field Dn for all of the imaging units 101 to 108 by any one of the methods described above.

  Subsequently, in step S404, the comparison unit 303 compares the depth of field acquired in step S403 with the shape information input from the shape acquisition unit 302, and the imaging units 101 to 108 perform all the shootings once. It is determined whether the subject can be within the depth of field. The depth of the subject on the optical axis of each imaging unit is calculated using the shape information of the subject, and compared with the depth of field acquired in step S403. Here, the depth of the subject on the optical axis is from the point closest to the subject closest to the subject on the optical axis of the imaging unit to the point farthest from the subject farthest. Is the distance. When the depth of field is larger, all the objects are within the depth of field, while when the depth of field is smaller, it is determined that the subject is not within the depth of field. Here, the optical axis of the imaging unit is a line segment having a direction parallel to a three-dimensional vector representing the front of the imaging unit, starting from the three-dimensional position of the imaging unit.

  FIG. 5 shows a conceptual diagram of the processing. Note that FIG. 5 illustrates the subject, the imaging unit, and the subject depth in two dimensions for convenience of illustration, but in actuality, these are represented in three dimensions, and the following description can be simply expanded to three dimensions. FIG. 5A shows a case where the imaging unit 501 captures two subjects. A line segment 502 represents the optical axis of the imaging unit 501, and a range 503 represents the depth of the subject on the optical axis. A region 505 colored in gray represents a focus region included within the depth of field of the imaging unit 501. A subject outside this area is out of the focus range, and is blurred in the captured image. Therefore, in this case, it is not possible to fit all the subjects within the depth of field by one shooting. On the other hand, FIG. 5B is a diagram illustrating a case where the imaging unit 506 having a position different from that of the imaging unit 501 is used. In FIG. 5B, since the focus area 509 is larger than the subject depth 508 on the optical axis, it is possible to fit all subjects within the depth of field by one shooting. Here, the comparison unit 303 performs the above-described comparison for all the imaging units, and determines whether there is a subject that cannot be accommodated within the depth of field for each imaging unit. If any of the imaging units 101 to 108 cannot capture the entire subject within the depth of field at one time, the process proceeds to step S405. On the other hand, if all the imaging units can capture the entire subject within the depth of field, the process proceeds to step S406.

  Subsequently, in step S405, when there is an imaging unit in which the grouping unit 304 cannot fit all subjects within the depth of field at one time, the imaging units 101 to 108 are each assigned to at least one subject. Divide into corresponding pairs. Details of processing performed in the grouping unit 304 will be described later. The grouping unit 304 outputs the result of the grouping performed here to the parameter determination unit 305, and proceeds to step S406.

  Lastly, in step S406, the parameter determination unit 305 determines whether each of the imaging units 101 to 108 is based on the grouping result input from the grouping unit 304 and the positional relationship between each imaging unit and each subject. A focus parameter indicating a focus distance is determined. Here, the imaging units assigned to the same set determine the focus parameter so that the same subject is focused and different subjects are focused for each group. Here, the focus parameter may be a focus distance of each imaging unit, or a lens-sensor distance of each imaging unit. Details of processing performed by the parameter determination unit 305 will be described later. The parameter determination unit 305 outputs the focus parameter determined here to the imaging units 101 to 108 and ends the process. Hereinafter, details of the grouping process in step S405 and the parameter determination process in step S406 will be described.

<Grouping process>
First, details of the grouping process in step S405 described above will be described. This process is performed when there is an imaging unit that cannot fit all subjects within the depth of field at one time.

  In the grouping process of the present embodiment, the imaging units 101 to 108 are divided into a number of groups equal to the number of subjects. For example, when there are two subjects, the imaging units 101 to 108 are divided into a first set corresponding to the first subject and a second set corresponding to the second subject. An arbitrary method can be used as a method for grouping the image pickup units. For example, in this embodiment, as shown in FIG. 6A, adjacent image capturing units are alternately assigned to different groups. In the example of FIG. 6A, four imaging units 101, 103, 105, and 107 are included in the first set corresponding to the subject 1, and the imaging unit 102 is included in the second set corresponding to the subject 2. , 104, 106, and 108 are allocated.

  When there are three subjects, as shown in FIG. 6B, the imaging units 101, 104, and 107 are in the first group corresponding to the subject 1, and the imaging unit is in the second group corresponding to the subject 2. The imaging units 103 and 106 are allocated to the third group in which 102, 105, and 108 correspond to the subject 3. Thus, in the above example, each imaging unit is assigned to each set in order. With such a grouping method, the imaging units can be equally allocated to the respective subjects. Note that the grouping method that can be used here is not limited to the above-described method, and may be randomly assigned or assigned according to other rules. In addition, although an example in which an equal number of imaging units is assigned to each group has been shown, the number of imaging units in each group is not necessarily equal. Furthermore, the grouping does not need to be fixed for each image pickup, and may be newly set every time shooting is performed, or the grouping may be changed according to time.

<Parameter determination process>
Next, details of the focus parameter determination processing in step S406 described above will be described. In this process, the focus distances of the imaging units belonging to each group are set so that the subject corresponding to each group is in focus.

  As illustrated in FIG. 7A, the parameter determination unit 305 determines the focus distance so that the subject 1 is focused on the imaging units 101, 103, 105, and 107 belonging to the first set corresponding to the subject 1. To do. At this time, the parameter determination unit 305 obtains the gravity center position 701 of the subject 1 from the shape information of the subject, calculates the distances 702, 703, 704, and 705 from each imaging unit to the gravity center position, and focuses the corresponding imaging units. Distance.

  Similarly, as shown in FIG. 7B, the parameter determination unit 305 focuses the imaging units 102, 104, 106, and 108 belonging to the second group corresponding to the subject 2 so that the subject 2 is in focus. Determine the distance. If it is determined in step S404 that all the subjects can be accommodated within the depth of field by one shooting, the number of sets is treated as 1, and the gravity center positions of all subjects and the respective imaging The distance from the image capturing unit is defined as the focus distance of each image capturing unit.

  The parameter determination unit 305 determines a focus parameter to be output to each imaging unit based on the focus distance set here. As the focus parameter, the focus distance itself may be output, or as long as the information indicates the focus distance, the lens-sensor distance of each imaging unit may be output as the focus parameter. The type of focus parameter to be output may be determined according to the configuration of the imaging unit that is the output destination. For example, if the output destination imaging unit has a function of generating an optical system control signal in accordance with the input focus distance, the focus distance itself may be output as the focus parameter.

  The above is the processing performed in the imaging system of the present embodiment. According to the above processing, when a plurality of imaging units capture a plurality of subjects from different directions, all the subjects can be focused and imaged even when the F value is small. In the above description, the case where there are a plurality of subjects to be imaged has been described. However, the present invention can also be applied to a case where the size of the subject is large and the image does not fall within the depth of field in one shooting. In that case, it is possible to apply the above method by dividing one subject into a plurality of parts having a size that fits within the depth of field of each imaging unit. Further, the present invention can be similarly applied to a case where there are a large-size subject and a plurality of subjects that do not fall within the depth of field.

  In this embodiment, the shape acquisition unit 302 includes a plurality of subject shapes and imaging units derived based on a plurality of images captured with an F value larger than the F value in imaging using a focus parameter. It functions as an acquisition means for acquiring shape information indicating the positional relationship between the two. In addition, the grouping unit 304 divides the plurality of imaging units into a plurality of groups each corresponding to at least one of the plurality of subjects based on the shape information acquired by the acquisition unit. It functions as a dividing means. Further, the parameter determination unit 305 determines the focus parameter so that the plurality of imaging units focus on the corresponding subjects based on the shape information acquired by the acquisition unit and the grouping by the grouping unit. Functions as a means.

<Example 2>
In the first embodiment, an example in which the grouping process is performed by allocating the imaging unit to each subject using a predetermined pattern is shown. In this embodiment, when grouping the imaging units, the imaging units to be allocated to each subject are determined in consideration of whether the subject is visible from each imaging unit. Accordingly, it is possible to prevent the imaging unit from being allocated to a subject that is not visible from the imaging unit, and as a result, shooting is not wasted.

  The configuration of the information processing apparatus 109 of this embodiment is the same as that of the first embodiment. However, the processing performed in the grouping unit 304 is different from that in the first embodiment. Hereinafter, processing performed by the grouping unit 304 of the present embodiment will be described, and description of other elements that perform the same operations as those of the first embodiment will be omitted.

  FIG. 8 shows a flowchart of the grouping process in this embodiment. First, in step S801, the grouping unit 304 calculates visibility information regarding each subject for each imaging unit. Here, the visibility information is information indicating how much each subject is visible from each imaging unit.

  Visibility information can be calculated from the position / posture of each imaging unit acquired by the position acquisition unit 301 and the shape information of the subject acquired by the shape acquisition unit 302. For example, when the visibility of the subject 1 from the imaging unit 101 is calculated, it may be checked whether there is an obstructing object between the imaging unit 101 and the subject 1. When it can be confirmed that there are no obstructing objects between the imaging unit 101 at various points on the subject 1, the imaging unit 101 can determine that the subject 1 is “visible”. On the other hand, if there are objects that block between a plurality of points on the subject 1 and the imaging unit 101 at a ratio equal to or higher than a predetermined threshold, the imaging unit 101 can determine that the subject 1 is “invisible”. An arbitrary value can be used as a threshold for determining whether or not the subject is visible, but a value of 50% is used in this embodiment. Visibility means that when an image of each subject is projected on the imaging sensor of each imaging unit so that a subject close to each imaging unit is given priority, what percentage of the image that should originally be projected on the imaging sensor This can be calculated by calculating whether the subject image is replaced.

  As another method for calculating the visibility information, if the angle formed by connecting the subject for which the visibility information is calculated, the imaging unit, and the other subject is larger than a predetermined value, it is determined to be “visible”, and conversely, it is smaller than the predetermined value. There is also a method to determine that is “invisible”. Visibility can be calculated by any method other than the two methods described above.

  FIG. 9 shows the visibility in the situation shown in FIG. Here, the case where the subject is visible from each imaging unit is indicated by ◯, and the case where the subject is not visible is indicated by ×. For example, the imaging unit 101 indicates that both the subject 1 and the subject 2 are ◯, and both the subjects can be seen from the imaging unit 1. Further, the imaging unit 102 indicates that the subject 1 cannot be seen but the subject 2 is visible. When the grouping unit 304 calculates the visibility of each subject of each imaging unit and determines whether each subject is visible or not visible from each imaging unit, the process proceeds to step S802.

  In step S <b> 802, the grouping unit 304 groups each imaging unit based on the subject visible from each imaging unit. The imaging unit is assigned to each subject from an imaging unit with a small number of circles in the visibility information. First, an imaging unit having only one circle in the visibility information, that is, an imaging unit in which only one subject is visible is assigned to a subject with circle. Next, an imaging unit having two circles in the visibility information is assigned to any one of the two objects circled. At this time, which of the two subjects is allocated is determined at random. Subsequently, the imaging unit having three circles selects one subject at random from the three subjects circled and assigns them to the subject. Thereafter, the number of circles is increased, and all of the imaging units are allocated to any subject. In the above description, each imaging unit is randomly allocated to one subject. However, the imaging units may be allocated so that the number of imaging units allocated to each subject is uniform.

  FIG. 10 shows an example of the result of grouping the imaging units based on visibility in the situation of FIG. In FIG. 10A, among the imaging units in which the subject 1 is visible, the imaging unit 101, the imaging unit 104, the imaging unit 106, and the imaging unit 107 are allocated as the first set corresponding to the subject 1. In FIG. 10B, among the imaging units in which the subject 2 is visible, the imaging unit 102, the imaging unit 103, the imaging unit 105, and the imaging unit 108 are allocated as a second set corresponding to the subject 2.

  The above is the grouping process of each imaging unit of the present embodiment. According to the above processing, it is possible to prevent the imaging unit from being assigned to the invisible subject and the result of the imaging from being wasted. As a result, it becomes possible to capture a good multi-viewpoint image in which all the depths of field fall within the depth of field.

<Example 3>
In the present embodiment, an example will be described in which shape information of a subject is accurately calculated based on image data captured using a focus parameter determined by the information processing apparatus 109. Description of points common to the first and second embodiments is omitted, and only differences are described. As shown in FIG. 11, the configuration of the information processing apparatus 109 according to the present embodiment is obtained by adding a shape estimation unit 1101 to the configurations of the first and second embodiments. Hereinafter, processing performed by the information processing apparatus 109 according to the present exemplary embodiment will be described with reference to a flowchart illustrated in FIG.

  When the processes in steps S401 to S406 are completed, the shape estimation unit 1101 acquires captured image data newly captured by the imaging units 101 to 108 using the focus parameter determined in step S406 (step S1201). Next, the shape estimation unit 1101 estimates more detailed shape information of each subject based on the captured image data acquired in step S1201. For the estimation of the shape information of each subject, a known method such as Multiview Stereo can be used in the same manner as described in the first embodiment. At this time, the image data used for acquiring the distance information of each subject is the image data captured by the corresponding imaging unit. Since the image data picked up by the corresponding image pickup unit is image data with little noise and focused on the corresponding subject, the shape information of the subject can be obtained with high accuracy. At this time, each imaging unit may add and output metadata indicating the subject corresponding to each captured image data so that the imaging unit corresponding to each subject can be understood.

  Further, in order to further improve the estimation accuracy of the shape information of the subject, at the time of grouping each imaging unit, at least two adjacent imaging units are set as one small group, and each imaging unit set is divided into a plurality of small groups. You may make it comprise by the set of. By performing the grouping in this way, at least two imaging units that capture the same surface of the subject are secured, so that the possibility that the shape information of each subject can be obtained with high accuracy is increased. In order to obtain the same effect, each imaging unit may be replaced with a compound eye camera having at least two or more optical systems.

<Other embodiments>
Embodiments of the present invention are not limited to those described above, and can take other various forms. For example, in the above embodiment, the grouping unit 304 has grouped the imaging units so that each imaging unit corresponds to one subject, but a plurality of subjects are included in the depth of field. Alternatively, the grouping may be performed so that each imaging unit corresponds to a plurality of subjects.
In the above embodiment, the comparison unit 303 compares the depth of each subject and the depth of field of each imaging unit when the F value is the open value of each imaging unit. However, the comparison may be performed while changing the F value within a range not exceeding a predetermined threshold. When the F value used for imaging is changed according to the shape of the subject, the parameter determination unit 305 adds information indicating the depth of field used for imaging, such as the F value, to the focus parameter and causes the imaging units 101 to 108 to Add it.

In the second embodiment, an example has been described in which each imaging unit is randomly assigned to a subject whose visibility exceeds the threshold. However, when there are a plurality of subjects whose visibility exceeds the threshold, priority is given to a subject with greater visibility May be assigned automatically.
In the above embodiment, the imaging system in which the imaging units are arranged so as to surround the subject has been described. However, the arrangement example of the imaging units is not limited to this, and each imaging unit is arranged side by side on one plane. It can also be applied to multi-lens cameras.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101-108 Imaging unit 109 Information processing device 201 CPU
202 RAM
203 ROM
301 Position Acquisition Unit 302 Shape Acquisition Unit 303 Comparison Unit 304 Grouping Unit 305 Parameter Determination Unit

Claims (11)

An information processing apparatus for determining a focus parameter including information indicating a focus position of a plurality of imaging units that capture a scene including a plurality of subjects from a plurality of different viewpoint positions,
Obtaining shape information indicating the shape and position of the plurality of subjects derived based on a plurality of images captured by the plurality of image capturing units with an F value larger than an F value in imaging using the focus parameter Acquisition means to
Grouping means for dividing the plurality of imaging units into a plurality of groups each corresponding to at least one of the plurality of subjects based on the shape information acquired by the acquisition unit;
Determining means for determining the focus parameter based on the shape information acquired by the acquiring means and the grouping by the grouping means so that the plurality of imaging units respectively focus on corresponding subjects; An information processing apparatus comprising:   An estimation unit that acquires a plurality of images captured by the plurality of imaging units based on the focus parameter determined by the determination unit and estimates the shapes of the plurality of subjects based on the acquired plurality of images. The information processing apparatus according to claim 1, further comprising:   The grouping unit calculates visibility indicating the degree of appearance of each subject for each of the plurality of imaging units based on the shape information, and performs the grouping based on the calculated visibility. The information processing apparatus according to claim 1.   4. The apparatus according to claim 1, further comprising a determination unit that determines whether to perform the grouping based on the shape information and information indicating the depth of field of the plurality of imaging units. The information processing apparatus according to any one of claims.   5. The information processing apparatus according to claim 1, wherein the information indicating the focus position is information indicating a positional relationship between lenses of the plurality of imaging units and imaging sensors. 6.   The information processing apparatus according to claim 5, wherein the focus parameter further includes information indicating a depth of field of the plurality of imaging units.   The information processing apparatus according to claim 6, wherein the information indicating the depth of field of the plurality of imaging units is information indicating an F value of the plurality of imaging units. A plurality of imaging units for imaging a scene including a plurality of subjects from a plurality of different viewpoint positions;
Obtaining means for obtaining shape information indicating a shape of the plurality of subjects and a positional relationship with the plurality of imaging units, derived from a plurality of images captured by the plurality of imaging units;
Grouping means for dividing the plurality of imaging units into a plurality of groups each corresponding to at least one of the plurality of subjects based on the shape information acquired by the acquisition unit;
Based on the shape information acquired by the acquisition unit and the grouping by the grouping unit, the plurality of imaging units focus on corresponding subjects and are used for imaging the plurality of images. An image pickup apparatus comprising: a control unit that controls the plurality of image pickup units so as to have an F value smaller than a given F value. An imaging system including: a plurality of imaging units that capture a scene including a plurality of subjects from a plurality of different viewpoint positions; and an information processing apparatus that determines focus parameters including information indicating a focus position of the plurality of imaging units. There,
Positions of the shapes of the plurality of subjects and the plurality of imaging units derived based on a plurality of images captured by the plurality of imaging units with an F value larger than an F value in imaging using the focus parameter Acquisition means for acquiring shape information indicating the relationship;
Grouping means for dividing the plurality of imaging units into a plurality of groups each corresponding to at least one of the plurality of subjects based on the shape information acquired by the acquisition unit;
A determination unit that determines the focus parameter so that the plurality of imaging units focus on corresponding subjects, based on the shape information acquired by the acquisition unit and the grouping by the grouping unit;
An imaging system comprising: a control unit that controls the plurality of imaging units so that each of them focuses on a corresponding subject based on the focus parameter determined by the determination unit. An information processing method for determining a focus parameter including information indicating a focus position of a plurality of imaging units that capture a scene including a plurality of subjects from a plurality of different viewpoint positions,
Positions of the shapes of the plurality of subjects and the plurality of imaging units derived based on a plurality of images captured by the plurality of imaging units with an F value larger than an F value in imaging using the focus parameter An acquisition step of acquiring shape information indicating the relationship;
A grouping step of dividing the plurality of imaging units into a plurality of groups each corresponding to at least one of the plurality of subjects based on the shape information acquired in the acquisition step;
A determining step for determining the focus parameter so that the plurality of imaging units focus on corresponding subjects based on the shape information acquired in the acquiring step and the grouping in the grouping step; An information processing method comprising:   A program that causes a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 7.

Images