JP5760558B2 - Camera simulation apparatus, camera simulation method, and camera simulation program - Google Patents

Description

  The present invention relates to a camera simulation device, a camera simulation method, and a camera simulation program, and more particularly to a camera simulation device, a camera simulation method, and a camera simulation program for efficiently calibrating camera parameters.

  Conventionally, an in-vehicle distance measuring device using a stereo camera is required to be able to accurately measure a distance up to a long distance and to have a wide field of view. In order to realize this requirement, it is necessary to accurately calibrate the external parameters and internal parameters of the camera.

  Conventionally, research on camera calibration methods has been widely performed, and various methods are already known. In distance measurement using a stereo camera, camera internal parameters are identified by camera calibration, and a camera model simulated on a computer using the internal parameters is used. Note that the difference between the left and right camera models causes the accuracy of distance measurement to deteriorate. In addition, since actual cameras have manufacturing errors, internal parameters of each camera are identified by camera calibration, and correction is performed so that the camera models on the left and right camera computers are equal. Such correction is an essential process for distance measurement using a stereo camera. The smaller the manufacturing error of the camera, the smaller the difference between the corrected left and right camera models, which is desirable. However, reducing the manufacturing error causes an increase in cost due to a decrease in yield.

  Conventionally, a method based on Monte Carlo simulation is already known as a method for setting a tolerance between an optical member and a mechanism member of a camera. In this method, a large number of data of an optical system in which tolerances and manufacturing distributions are given to the design values of the optical system are created, and after checking the optical performance of each, the distribution of the optical performance of all data is obtained. If this result falls within the desired distribution range, it is determined that the tolerance set value is optimum.

  Conventionally, a method for analyzing how much the tolerance and manufacturing distribution set in the design parameters affect the optical performance is provided so that an optical unit suitable for manufacturing is provided. An optical unit setting support method for selecting a parameter for changing the tolerance and optimizing the tolerance of the design parameter is disclosed (for example, see Patent Document 1).

  The technique disclosed in Patent Document 1 is based on tolerances of optical design parameters and manufacturing distributions for an optical unit that includes a plurality of lens groups, and each lens group includes at least one optical member and at least one mechanism member. In addition, a large number of lens data with manufacturing errors is generated, and the optical performance distribution is calculated based on the large number of lens data, and the optical design against the fluctuation of the optical performance by the contribution analysis based on the distribution of the optical performance. An optical unit design support method for calculating a first contribution of parameters is shown.

  However, in the development of the calibration method for the conventional stereo camera described above, the accuracy of the distance measurement accuracy must be judged by measuring the error of the distance measurement accuracy after correcting the internal parameters. It has not been possible to consider whether it is caused by the size of the camera manufacturing error or the calibration method (particularly the algorithm).

  Conventionally, since it is necessary to perform an experiment for actually measuring an error in distance measurement accuracy outdoors, there is a problem that the distance measurement accuracy varies depending on the weather and the experiment is not reproducible.

  In addition, in the conventional tolerance setting method based on Monte Carlo simulation, the non-defective product rate is defined based on the optical performance, mainly based on MTF (Modulation Transfer Function), and the optimum setting value of the tolerance is determined. There was a problem that tolerance could not be set based on distance measurement accuracy.

  Further, the method disclosed in Patent Document 1 also has a problem that a calibration method cannot be developed on a simulation, and a tolerance cannot be set based on distance measurement accuracy.

  That is, in each of the conventional methods described above, there is no efficient camera parameter calibration method suitable for, for example, mass production.

  The present invention has been made in view of the above problems, and an object thereof is to provide a camera simulation apparatus, a camera simulation method, and a camera simulation program for efficiently calibrating camera parameters.

  In order to solve the above-described problems, the present invention employs means for solving the problems having the following characteristics.

The present invention simulates a simulated camera system simulating the lens system, a first image to obtain the simulated camera system image acquisition means when capturing a predetermined object in simulated the simulated camera system, the lens simulates a pinhole camera system using the focal length of the system design values, when the predetermined object captured from the same position taken the first image in simulated the pin-hole camera system a pinhole camera system image acquiring means for acquiring a second image of, and the obtained first image by simulating a camera system image acquiring unit, the second image obtained by the pinhole camera system image acquiring means wherein the deviation of the position of the predetermined object image is outputted as a difference value for the parallax triangulation using the pinhole camera, by using the difference value from A camera simulation apparatus characterized by having a parallax error output means for outputting a prediction value of the difference error.

The present invention simulates a simulated camera system simulating the lens system, the simulated camera system image acquisition step of acquiring a first image obtained by imaging a predetermined object in simulated the simulated camera system, wherein A pinhole camera system using a focal length at the design value of the lens system was simulated, and the predetermined object was captured from the same position as the position where the first image was captured by the simulated pinhole camera system . the second pin hole camera system image acquisition step of acquiring an image of a first image obtained by the simulated camera system image acquisition step, the second image obtained by the pinhole camera system image acquisition process when use the deviation of the position of the predetermined object image is outputted as a difference value for the parallax triangulation using the pinhole camera, the difference value from the A camera simulation method characterized by having a parallax error output step of outputting the predicted value of the parallax error Te.

  The present invention is also a camera simulation program that causes a computer to function as the above-described camera simulation apparatus.

  According to the present invention, it is possible to efficiently calibrate camera parameters.

It is a figure which shows the function structural example of the camera simulation apparatus in 1st Embodiment. It is a figure which shows an example of the hardware constitutions which can implement | achieve the camera simulation process in this embodiment It is a flowchart which shows an example of the camera simulation process sequence in 1st Embodiment. It is a figure which shows the function structural example of the camera simulation apparatus in 2nd Embodiment. It is a flowchart which shows an example of the camera simulation process sequence in 2nd Embodiment. It is a figure which shows the function structural example of the camera simulation apparatus in 3rd Embodiment. It is a figure which shows an example of distribution of the maximum value of the distance measurement error in 3rd Embodiment. It is a flowchart which shows an example of the camera simulation process sequence in 3rd Embodiment.

<About the present invention>
The present invention, for example, an image (first image) captured by a camera system (simulated camera system) simulating an actual lens system, and an image captured by a pinhole camera system using the focal length of the design value of the lens system A difference value from the (second image) is output, and a predicted parallax value adjusted using the above-described difference value between both images is output with respect to the parallax of triangulation using an ideal pinhole camera. The difference value is, for example, the difference (deviation) in the position on the image when the same object (for example, the subject) is photographed from the same position with the two camera systems described above and the two images are compared. Etc. Therefore, the difference value in the present embodiment indicates, for example, the difference in image coordinates of the lattice points of the chart in the simulated camera system and the pinhole camera system (difference in coordinates between the X-axis direction and the Y-axis direction of the image). In the present invention, the present invention is not limited to this, and may be a difference between other elements obtained from an image.

  Hereinafter, embodiments in which a camera simulation apparatus, a camera simulation method, and a camera simulation program according to the present invention are suitably implemented will be described with reference to the drawings.

<First Embodiment: Camera Simulation Device: Functional Configuration Example>
FIG. 1 is a diagram illustrating a functional configuration example of the camera simulation apparatus according to the first embodiment. In the first embodiment, a camera simulation apparatus for acquiring a parallax error with respect to a camera system simulating an actual lens system is shown.

  The camera simulation apparatus 10 shown in FIG. 1 includes an input unit 11, an output unit 12, a storage unit 13, a simulated camera system image acquisition unit 14, a pinhole camera system image acquisition unit 15, and a parallax error output unit 16. The transmission / reception means 17 and the control means 18 are provided.

  The input means 11 is necessary for input of a simulated camera system image acquisition instruction, a pinhole system image acquisition instruction, a parallax error output instruction, a transmission / reception instruction, and other instructions such as start / end from a user, etc. Accepts various data input. The input means 21 is composed of a pointing device such as a keyboard and a mouse if it is a general-purpose computer such as a PC (Personal Computer), and is composed of a touch panel or a group of operation buttons if it is a portable terminal or smartphone.

  The output unit 12 outputs the content input by the input unit 11 and the content executed based on the input content. Specifically, for example, the acquired image obtained by the simulated camera system image acquisition unit 14, the acquired image obtained by the pinhole camera system image, the parallax error result obtained by the parallax error output unit 16, etc. Performs screen display of results of processing, audio output, etc. The output unit 12 includes a display, a speaker, and the like.

  Furthermore, the output unit 12 may have a printing function such as a printer, and can print the above-described output contents on various printing media such as paper. In addition, the input unit 11 and the output unit 12 may have an input / output integrated configuration such as a touch panel.

  The accumulation unit 13 accumulates various information required in the present embodiment and various data at the time of execution of various processes or after execution. Specifically, the accumulating unit 13 includes an image obtained by the simulated camera system image obtaining unit 14, an image obtained by the pinhole camera system image obtaining unit 15, a parallax error output result obtained by the parallax error output unit 16, and the like. Accumulate various data. Further, the storage means 13 can read out various data stored as required.

  The simulated camera system image acquisition unit 14 simulates a camera system that simulates various lens systems such as telephoto and magnification that are actually used for shooting, and outputs an image captured by the camera system. As the lens system in the present embodiment, for example, a general lens system used in a camera system having a lens and a two-dimensional image sensor can be used.

  As described above, for a technique for acquiring an image virtually captured by a camera system simulating a lens system, a technique provided by commercially available software such as CAD (Computer Aided Design) software is used. Can be used. Specific examples of the software described above include CodeV (manufactured by Optical Research Associates) and ZEMAX (manufactured by ZEMAX Development Corporation), but are not limited to this.

  The camera system may be an imaging means such as a stereo camera, but is not limited to this in the present invention.

  The simulated camera system image acquisition unit 14 creates a camera system in which, for example, manufacturing errors are randomly added to the design value of the lens system, and an image (first image) captured by the camera system based on the created design value. 1 image).

  The pinhole camera system image acquisition means 15 simulates a pinhole camera system using the focal length of the design value of the lens system in the simulated camera system image acquisition means 14 described above, and an image captured by the pinhole camera system ( 2nd image) is acquired.

  Here, both images acquired by the simulated camera system image acquisition means 14 and the pinhole camera system image acquisition means 15 are images taken with respect to the same imaging object, and both images were acquired by simulation. It is an image.

  The parallax error output means 16 calculates the parallax of triangulation using an ideal pinhole camera composed of ideal values of a lens system similar to the lens system described above. Further, the parallax error output unit 16 extracts a difference value between both images acquired by the simulated camera system image acquisition unit 14 and the pinhole camera system image acquisition unit 15.

  Further, the parallax error output means 16 adjusts the parallax of the triangulation using the ideal pinhole camera using the difference value of both images, and outputs the adjusted predicted parallax value. Specifically, the parallax error output unit 16 outputs a predicted value of parallax by adding, as a parallax error, a difference value between both images to, for example, a triangulation parallax using an ideal pinhole camera. To do. The adjustment content in the present invention is not limited to this, and other adjustments can be made based on values obtained using both images.

  The transmission / reception means 17 is an interface for acquiring a desired external image, an execution program for realizing each processing in the present invention, and the like from an external device connectable via a communication network or the like. The transmission / reception means 17 can transmit various information obtained in the camera simulation device 10 to an external device or the like.

  The control means 18 controls the entire components of the camera simulation device 10. Specifically, the control unit 18 performs control in each process such as acquisition of various images, parallax error output, and the like based on, for example, an instruction from the input unit 11 by a user or the like.

  Here, in the camera simulation apparatus 10 described above, an execution program (camera simulation program) that can cause a computer to execute each function is generated, and the execution program is installed in, for example, a general-purpose PC or server. The camera simulation processing and the like in the present invention can be realized.

  Here, an example of a hardware configuration of a computer capable of realizing the camera simulation process in the present embodiment will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a hardware configuration capable of realizing the camera simulation process according to the present embodiment.

  2 includes an input device 21, an output device 22, a drive device 23, an auxiliary storage device 24, a memory device 25, a CPU (Central Processing Unit) 26 for performing various controls, and a network connection device. 27, and these are connected to each other by a system bus B.

  The input device 21 has a pointing device such as a keyboard and a mouse operated by a user or the like, and inputs various operation signals such as execution of a program from the user or the like. The input device 21 may include an image input unit that inputs an acquired image obtained by the simulated camera system image acquisition unit 14, an acquired image obtained by a pinhole camera system image, and the like.

  The output device 22 has a display for displaying various windows and data necessary for operating the computer main body for performing the processing in the present embodiment. Can be displayed.

  Here, the execution program installed in the computer main body in the present invention is provided by a portable recording medium 28 such as a CD-ROM or a USB (Universal Serial Bus) memory. The recording medium 28 on which the program is recorded can be set, for example, in the drive device 23 or the like, and the execution program included in the recording medium 28 is installed in the auxiliary storage device 24 from the recording medium 28 via the drive device 23.

  The auxiliary storage device 24 is a storage means such as a hard disk, and can store an execution program according to the present invention, a control program provided in a computer, and the like, and can perform input / output as necessary.

  The memory device 25 temporarily stores an execution program read from the auxiliary storage device 24 by the CPU 26 and data being processed. The memory device 25 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

  The CPU 26 controls processing of the entire computer, such as various operations and input / output of data with each hardware component, based on a control program such as an OS (Operating System) and an execution program stored in the memory device 25. Thus, each process in the present embodiment can be realized. Various information necessary during the execution of the program can be acquired from the auxiliary storage device 24, and an execution result or the like can be stored.

  The network connection device 27 acquires an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program or an execution in the present invention The program itself can be provided to other terminals.

  With the hardware configuration as described above, the camera simulation process in the present embodiment can be executed efficiently at low cost without requiring a special device configuration. Also, by installing the program, the camera simulation process according to the present invention can be easily realized on a general-purpose PC or the like.

  Next, a camera simulation processing procedure in the above-described camera simulation program or the like will be described using a flowchart.

<Camera Simulation Processing Procedure in First Embodiment>
FIG. 3 is a flowchart illustrating an example of a camera simulation processing procedure according to the first embodiment. In the process shown in FIG. 3, first, an image captured by a simulated camera system is acquired (S01). Next, an image captured by the ideal pinhole camera system is acquired (S02). Note that the images obtained by the processes of S01 and S02 are images obtained by simulation. Further, the processes of S01 and S02 may be performed in the reverse order or may be performed simultaneously.

  Next, the difference value between the two images obtained by the processing of S01 and S02 is output as a parallax error (S03). Further, the parallax error obtained in S03 is added to the parallax of the triangulation using the ideal pinhole camera, and the predicted value of the parallax is output (S04). When adding an error as described above, for example, processing such as adding an error value for each pixel of a parallax image can be performed. However, the present invention is not limited to this.

  According to the first embodiment described above, it is possible to efficiently simulate, for example, a distance measurement error after calibrating camera parameters for a camera with various manufacturing errors. As a result, it is possible to quickly develop a camera parameter calibration method suitable for mass production, and to determine a camera tolerance optimal for a stereo camera.

<Second Embodiment: Camera Simulation Device: Functional Configuration Example>
Next, a functional configuration example of the camera simulation apparatus according to the second embodiment will be described with reference to the drawings. FIG. 4 is a diagram illustrating a functional configuration example of the camera simulation apparatus according to the second embodiment. In the following description, the same reference numerals are assigned to configurations that perform substantially the same processing as in the first embodiment, and a specific description thereof is omitted here. Also, the hardware configuration can be performed by a general-purpose computer or the like as in the first embodiment described above, and thus description thereof is omitted here.

  The camera simulation apparatus 30 shown in FIG. 4 includes an input unit 11, an output unit 12, a storage unit 13, a simulated camera system image acquisition unit 14, a pinhole camera system image acquisition unit 15, and a parallax error output unit 16. The transmission / reception means 17, the control means 18, the internal parameter identification means 31, the image correction means 32, and the distance error output means 33 are configured.

  Here, when the first embodiment and the second embodiment described above are compared, the internal parameter identification unit 31, the image correction unit 32, and the distance error output unit 33 are newly added to the second embodiment. Is provided.

  In the second embodiment, the simulated camera system image acquisition unit 14 adds a camera system in which a manufacturing error is randomly added to a design value for a preset lens system in addition to the processing shown in the first embodiment. Create and acquire an image captured by the camera system. The manufacturing error is not only randomly given as described above, but a manufacturing error in a predetermined order may be added with reference to, for example, a preset error input table. The error input table described above is accumulated in, for example, the accumulation means 13 and can be read and used when necessary.

  As described above, the internal parameter identification unit 31 identifies the internal parameter by calibration with respect to the image provided with the manufacturing error and the like obtained from the simulated camera system image acquisition unit 14. Here, the internal parameters are parameters determined by the combination of the camera and the lens. For example, the focal length f, the lens distortion coefficients k1 and k2, the lens centers Cx and Cy, the aspect ratio α, the shear distortion coefficient β, and the image Of the angle coefficient and the like, at least one parameter is shown, but the present invention is not limited to this.

  In addition, camera calibration refers to taking an image of an object (for example, a subject) with a camera to be adjusted, and using the two-dimensional image, the camera's internal parameters (for example, focal length, lens center, lens distortion, etc.) The process etc. which estimate etc. are shown. Here, in this embodiment, camera calibration is mainly performed on internal parameters. However, the present invention is not limited to this. For example, in addition to internal parameters, external parameters (for example, posture, Position etc.) can also be used, and these can also be combined.

  Various methods can be used as the calibration method in the second embodiment. For example, a calibration method using a chart or the like (for example, Japanese Patent No. 4234059 or the non-patent document “Z Zhang, using “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22: 1330-1334, 2000, etc. Techniques (for example, “Rachimasa Sagawa, Yasushi Yagi,“ Stable calibration of internal camera parameters by observation of two parallel lights ”, Image recognition / Symposium (MIRU2007) Proceedings, pp.615-620 "or the like can be used for reference), and the like. In the second embodiment, among the plurality of calibration methods, processing can be performed by appropriately selecting based on processing accuracy (error), processing time, apparatus performance, and the like.

  The image correction unit 32 performs correction by image processing on the image acquired by the simulated camera system image acquisition unit 14 using the internal parameter identified by the calibration in the internal parameter identification unit 31, and the image after the correction (Third image) is output. As correction contents, for example, distortion, image magnification, and the like are corrected. In the present invention, the present invention is not limited to this. For example, lateral chromatic aberration may be corrected.

  That is, in the correction in the second embodiment, the image acquired by the simulated camera system image acquisition unit 14 is an image captured by a pinhole camera system (ideal pinhole camera system) using the focal length of the design value of the lens system. Make corrections so that

  In addition, the parallax error output means 16 in 2nd Embodiment is the 3rd image obtained by the image correction means 32, and the image (2nd image) obtained by the pinhole camera system image acquisition means 15 mentioned above. A difference value is output, and a predicted value of the parallax error is output by adding the difference value to the parallax of triangulation using a pinhole camera.

  The distance error output means 33 calculates the distance with respect to the predicted value of the parallax error obtained by the parallax error output means 16 and outputs the error of the distance.

  Here, in the second embodiment, for example, when calculating an error in the parallax error output unit 16 or the distance error output unit 33, a plurality of times (for a predetermined number of times) depending on a random value or a plurality of preset different conditions. It is possible to perform processing and output the average value, maximum error, minimum error, and the like. Note that the processing for a predetermined number of times can be realized by providing a plurality of the above-described simulated camera systems, for example, but the present invention is not limited to this.

  Further, the distance error output means 33 may create and output the distribution of the maximum value of the distance error when the distance error is acquired by a large number of camera systems, for example. At this time, the distance error output means 33 calculates the distance using the predicted value of the parallax, outputs the error, outputs the distribution of the maximum values of the distance measurement errors of many cameras, and then outputs the maximum value of the distance measurement error. It is determined whether or not the distribution of is within a desired range. Then, the distance error output means 33 performs control so that the maximum value distribution falls within a predetermined range when the maximum value distribution does not fall within the desired range.

<Camera Simulation Processing Procedure in Second Embodiment>
Next, a camera simulation processing procedure in the second embodiment will be described using a flowchart. FIG. 5 is a flowchart illustrating an example of a camera simulation processing procedure according to the second embodiment. In the process shown in FIG. 5, first, a simulated camera system (real camera system) in which manufacturing errors are randomly added to design values is created (S11). In the process of S11, as a simulated camera system, for example, each optical element and mechanism component is given a manufacturing distribution indicating tolerance width and ease of occurrence of manufacturing error, and manufactured to the design value of the optical system by Monte Carlo simulation or the like. A plurality (for example, 100 to 1000) of optical system data to which an error is added is created.

  Next, an image captured by the simulated camera system is acquired (S12). Here, in the process of S12, for example, an image obtained by capturing a reference image or the like used for parallax calculation with a simulated camera system is output. In addition, as the above-described reference image, for example, a lattice chart can be used. When a grid-like chart is used, imaging simulation can be performed only by geometric optical calculation, and the coordinates of the chart image on the sensor are output.

  Next, internal parameters are identified by calibration (S13). Specifically, for example, the focal length f, lens distortion coefficients k1, k2, lens centers Cx, Cy, aspect ratio α, shear distortion coefficient β, and the like are acquired as internal parameters. Next, correction is performed by image processing using the internal parameters identified by calibration, and the corrected image is output (S14). For example, as described above, the correction here is performed so as to be close to an image obtained by a pinhole camera system (ideal pinhole camera system) using the focal length of the design value of the lens system.

  Next, an image captured by a pinhole camera system (ideal pinhole camera system) using the focal length of the design value of the lens system is acquired (S15). Here, in the process of S15, for example, an image obtained by capturing the reference image used for parallax calculation with the ideal pinhole camera system is output. The reference image is the same as that used in S12 described above. This is because the difference value between the image captured by the real camera system and the image captured by the ideal pinhole camera system is compared.

  Next, the difference value between the two images is output as a parallax error (S16). In the process of S16, for example, the difference between the imaging position of the ideal pinhole camera system at a specific point on the reference image and the imaging position of the real camera system is determined in the horizontal direction (u-axis direction) on the sensor. And the vertical direction (v-axis direction), and the calculation result is output as a parallax error. In addition, as an example of the specific point on the reference | standard image mentioned above, there exists a lattice point of a grid | lattice-like chart, for example. In this case, the difference between the imaging position in the ideal pinhole camera system and the imaging position in the real camera system is output for each lattice point of the chart. In the second embodiment, the distribution of parallax errors in the entire sensor can be simulated by selecting a chart in which the image size is equal to the sensor size.

Next, the difference value between the two images obtained in the process of S16 is added to the parallax of the triangulation using the ideal pinhole camera, and the predicted value of the parallax is output (S17). Note that the parallax value d for triangulation with an ideal pinhole camera is calculated by, for example, the following equation (1).
d = B * f / Z (1)
Here, in the above-described equation (1), B represents the baseline length between the two cameras in the stereo camera, f represents the design value of the focal length of the lens system, and Z represents from the camera to an object (for example, a subject). Shows the distance.

Next, a distance is calculated using the predicted parallax value obtained in the process of S17, and an error is output (S18). In the process of S18, when the distance Z is calculated using the predicted value of parallax, for example, the following equation (2) can be used.
Z = B * f / (d + Δd) (2)
Here, in the above-described equation (2), B represents the baseline length between the two cameras in the stereo camera, f represents the design value of the focal length of the lens system, and d + Δd represents the parallax prediction with the parallax error added. The value is shown.

  When a grid chart is used as the reference image, the difference between the distance calculated using the predicted parallax value for each grid point of the chart and the true value of the predetermined distance is output for each grid point. .

  Here, it is determined whether or not processing for a preset number of times (a predetermined number of times) has been performed by a random value or the like (S19). If the predetermined number of times of processing has not been performed (NO in S19), the process returns to S11, and the manufacturing error added to the design value is randomly changed, and the processes after S11 are performed.

  Further, in the process of S19, when a predetermined number of processes have been performed (YES in S19), a distance error is output based on distance measurement errors obtained from a plurality of simulated camera systems (S20). In the process of S20, for example, an average value of distance measurement errors obtained from a plurality of simulated camera systems, a maximum distance measurement error, a minimum distance measurement error, or a standard deviation is output. Based on the distance measurement error can be output. In the present invention, the output method is not limited to the above-described output method. For example, a plurality of the output methods described above may be combined.

  As described above, according to the second embodiment, a camera parameter suitable for mass production is simulated by simulating a distance measurement error after calibrating the camera parameter for a camera with various manufacturing errors. It is possible to quickly develop a calibration method. Further, according to the second embodiment, it is possible to determine a camera tolerance that is optimal for a stereo camera by simulating a distance measurement error.

<Third Embodiment: Camera Simulation Device: Functional Configuration Example>
Next, a functional configuration example of the camera simulation apparatus according to the third embodiment will be described with reference to the drawings. FIG. 6 is a diagram illustrating a functional configuration example of the camera simulation apparatus according to the third embodiment. In the following description, the same reference numerals are assigned to configurations that perform substantially the same processing as in the first and second embodiments, and a specific description thereof is omitted here. Also, the hardware configuration can be performed by a general-purpose computer or the like as in the first embodiment described above, and thus description thereof is omitted here.

  The camera simulation apparatus 40 shown in FIG. 6 includes an input unit 11, an output unit 12, a storage unit 13, a simulated camera system image acquisition unit 14, a pinhole camera system image acquisition unit 15, and a parallax error output unit 16. The transmission / reception means 17, the control means 18, the internal parameter identification means 31, the image correction means 32, the distance error output means 33, and the calibration / tolerance setting means 41 are configured.

  Here, when the second embodiment and the third embodiment described above are compared, a calibration / tolerance setting means 41 is newly provided in the third embodiment.

  In the third embodiment, the calibration / tolerance setting unit 41 determines whether or not the distribution of the maximum value of the distance measurement error obtained by the distance error output unit 33 is within a preset range. The judgment result is output. In addition, the calibration / tolerance setting means 41, when the distribution of the maximum value of the distance measurement error is not within the preset range in the determination result, the tolerance value of the manufacturing error given to the design value again. Is corrected and a new set value is set. Note that the calibration / tolerance setting unit 41 not only resets the tolerance value, but also may change the calibration method, for example, or reset both the calibration method and the tolerance value. The calibration / tolerance setting unit 41 instructs the control unit 18 and the like to perform the camera simulation again with the new set value.

<Distribution example of maximum distance measurement error>
Here, an example of the result of outputting the maximum value distribution of distance measurement errors of a large number of cameras in the third embodiment will be described with reference to the drawings. FIG. 7 is a diagram illustrating an example of the distribution of the maximum value of distance measurement errors in the third embodiment. In FIG. 7, the horizontal axis represents an error (m) from 100 m, and the vertical axis represents the frequency (number of times).

  That is, in the example of FIG. 7, the maximum value on the image of the error when the object 100 meters ahead from the camera is calculated is taken on the horizontal axis, and the appearance when 1000 (pattern) lens systems are tried. A histogram is shown with the frequency on the vertical axis.

  The calibration / tolerance setting means 41 in the third embodiment determines whether or not the distribution of the maximum value of the distance measurement error is within a preset range, in other words, the desired distribution measurement error distribution is a desired value. The quality of the calibration method and / or tolerance setting is determined based on whether or not the ratio of lenses that fall within the range satisfies the reference value.

  Note that the value on the horizontal axis shown in FIG. 7 is not limited to an error from a predetermined distance, and for example, an average value, variance, standard deviation, or the like of a distance measurement error can be used. In the present invention, the number of lens systems for changing the setting and acquiring the distance measurement error is not limited to this, and may be a predetermined number of 100 or more, for example.

<Camera Simulation Processing Procedure in Third Embodiment>
Next, a camera simulation processing procedure in the third embodiment will be described using a flowchart. FIG. 8 is a flowchart illustrating an example of a camera simulation processing procedure according to the third embodiment.

  In the process shown in FIG. 8, the processes in S21 to S29 are the same as S11 to S19 in the camera simulation process in the second embodiment described above, and a description thereof will be omitted here.

  In the process of S29 in the third embodiment, when a predetermined number of processes have been performed (YES in S29), the distribution of the maximum value of distance measurement errors obtained from a plurality of simulated camera systems is output (S30). Note that the processing of S30 is, for example, a step of outputting a preset distribution of maximum values of distance measurement errors of a plurality of cameras. As an index used for determining the quality of the calibration method, for example, other distances may be used. An average value of measurement errors, a standard deviation, or the like can be used, or they can be combined.

  Next, after the process of S30 is completed, it is determined whether or not the maximum value distribution is within a predetermined range (S31). Here, in the process of S31, when the distribution of the maximum value is not within the predetermined range (NO in S31), the quality determination of the calibration method or the tolerance setting is determined as “No” (S32), and the design value is set. The tolerance value of the given manufacturing error is corrected (S33), and the process returns to the above-described process of S21 using the new set value, and the subsequent process is performed again. In the process of S33 described above, the calibration method may be changed, and the calibration method and the tolerance value may be changed (reset).

  In the process of S31, if the maximum distribution is within a predetermined range (YES in S31), the quality determination of the calibration method or tolerance setting is determined as “good” (S34), and the process is terminated.

  That is, in the above-described processing of S31 to S34, it is a step of determining whether the calibration method or tolerance setting is good or not based on whether the distance measurement error distribution is within a desired range, and the distance measurement error distribution is within the desired range. If not, change the calibration method or tolerance setting and repeat from the beginning.

  As described above, according to the third embodiment, a camera parameter suitable for mass production is simulated by simulating a distance measurement error after calibrating the camera parameter for a camera with various manufacturing errors. It is possible to quickly develop a calibration method. Further, according to the third embodiment, it is possible to determine a camera tolerance that is optimal for a stereo camera. The first to third embodiments described above can be applied by appropriately combining a part of each configuration.

  As described above, according to the present embodiment, camera parameter calibration can be performed efficiently. Specifically, according to the present invention, a distance measurement error after calibrating camera parameters for a camera with various manufacturing errors can be obtained by simulation without using an actual camera. . In other words, the simulation of a camera parameter that is suitable for mass production is achieved through a simulation that compares an image from a simulated camera system simulating an actual lens system with an image from an ideal camera system used for distance calculation by the stereo method. The method can be developed quickly.

  Furthermore, according to the present invention, for example, a certain tolerance value is input and a simulation is performed. If the output ranging error is outside the desired range, the tolerance value is reset and then the simulation is performed again. The simulation is repeated until the error is within a desired range. As a result, the present invention can set the tolerance with reference to the distance measurement accuracy, and can determine the optimum camera tolerance for the stereo camera.

  The camera simulation device, the camera simulation method, and the camera simulation program according to the present invention include, for example, development of a lens system for various cameras in a stereo camera, a distance measuring device using a stereo camera, a three-dimensional shape recognition device, and the like. The present invention can be applied to a wide range of fields, such as adopting an appropriate lens system based on the simulation result in the present invention, for example, in an in-vehicle stereo camera.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10, 30, 40 Camera simulation apparatus 11 Input means 12 Output means 13 Accumulation means 14 Simulated camera system image acquisition means 15 Pinhole camera system image acquisition means 16 Parallax error output means 17 Transmission / reception means 18 Control means 21 Input device 22 Output device 23 Drive device 24 Auxiliary storage device 25 Memory device 26 CPU (Central Processing Unit)
27 Network connection device 28 Recording medium 31 Internal parameter identification means 32 Image correction means 33 Distance error output means 41 Calibration / tolerance setting means

JP 2006-98558 A

Claims (15)

The lens system to simulate a simulated camera system that simulates a first image to obtain the simulated camera system image acquisition means when capturing a predetermined object in simulated the simulated camera system,
A pinhole camera system using a focal length at the design value of the lens system is simulated, and the predetermined object is picked up from the same position as the position where the first image was taken with the simulated pinhole camera system. Pinhole camera system image acquisition means for acquiring a second image when
Said first image obtained by the simulated camera system image acquiring means, the output deviation of the position on the image of the predetermined object from said obtained second image by the pinhole camera system image acquiring means as the difference value And a parallax error output means for outputting a predicted value of the parallax error using the difference value with respect to the parallax of the triangulation using the pinhole camera system . Internal parameter identification means for identifying internal parameters by calibration with respect to the simulated camera system;
Image correction means for correcting an image obtained from the simulated camera system image acquisition means using the internal parameters identified by the internal parameter identification means;
The parallax error output means outputs a difference value between the third image obtained by the image correction means and the second image, and for the parallax of triangulation using the pinhole camera system , the parallax error output means The camera simulation apparatus according to claim 1, wherein the prediction value of the parallax error is output using the difference value.   The camera simulation apparatus according to claim 1, further comprising a distance error output unit that calculates a distance using the predicted value obtained from the parallax error output unit and outputs an error. The simulated camera system image acquisition means includes:
The camera simulation apparatus according to claim 3 , wherein a plurality of simulated camera systems in which manufacturing errors set for the design values are randomly added are generated, and images captured by the generated camera systems are acquired. . The distance error output means includes
5. The camera simulation apparatus according to claim 4, wherein a distribution of maximum values of distance measurement errors of the simulated camera system is output using distance measurement errors obtained from the plurality of simulated camera systems. Calibration / tolerance setting for determining whether the simulation camera system is calibrated or tolerated for tolerance setting based on whether or not the distribution of maximum values of distance measurement errors of the plurality of simulated camera systems is included in a preset range The camera simulation apparatus according to claim 5 , further comprising: means. The calibration / tolerance setting means includes:
The camera simulation according to claim 6, wherein when it is determined that the distribution of the maximum value is not included in a preset range, a tolerance value of a manufacturing error given to the design value is reset. apparatus. Simulating a simulated camera system simulating the lens system, the simulated camera system image acquisition step of acquiring a first image obtained by imaging a predetermined object in simulated the simulated camera system,
A pinhole camera system using a focal length at the design value of the lens system is simulated, and the predetermined object is picked up from the same position as the position where the first image was taken with the simulated pinhole camera system. A pinhole camera system image acquisition step of acquiring a second image when
Said first image obtained by the simulated camera system image acquiring step, the output deviation of the position on the image of the predetermined object from said obtained second image by the pinhole camera system image acquisition step as the difference value And a parallax error output step of outputting a predicted value of the parallax error using the difference value with respect to the parallax of the triangulation using the pinhole camera system . An internal parameter identification step of identifying internal parameters by calibration with respect to the simulated camera system;
An image correction step of correcting the image obtained from the simulated camera system image acquisition step using the internal parameter identified by the internal parameter identification step;
The parallax error output step outputs a difference value between the third image obtained by the image correction step and the second image, and for the parallax of triangulation using the pinhole camera system , 9. The camera simulation method according to claim 8, wherein a predicted value of the parallax error is output using the difference value.   10. The camera simulation method according to claim 8, further comprising a distance error output step of calculating a distance using the predicted value obtained from the parallax error output step and outputting an error. The simulated camera system image acquisition step includes:
The camera simulation method according to claim 10 , wherein a plurality of simulated camera systems in which manufacturing errors set for the design values are randomly added are generated, and images captured by the generated camera systems are acquired. . The distance error output step includes
The camera simulation method according to claim 11, wherein a distribution of maximum values of distance measurement errors of the simulated camera system is output using distance measurement errors obtained from the plurality of simulated camera systems. Calibration / tolerance setting for determining whether the simulation camera system is calibrated or tolerated for tolerance setting based on whether or not the distribution of maximum values of distance measurement errors of the plurality of simulated camera systems is included in a preset range The camera simulation method according to claim 12 , further comprising a step. The calibration / tolerance setting process includes:
14. The camera simulation according to claim 13, wherein when it is determined that the maximum value distribution is not included in a preset range, a tolerance value of a manufacturing error given to the design value is reset. Method.   A camera simulation program for causing a computer to function as the camera simulation apparatus according to any one of claims 1 to 7.

Images