JP2012195875A - Image output device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shoot an object placed on a table surface from above, for example, since the image is taken in a state where the paper surface of the book is curved, the image is taken in a distorted state.
[Solution]
A predetermined subject can be placed on the surface. When the projection unit projects light of a predetermined pattern onto the surface from a position higher than the surface of the table, the camera unit performs photographing on the surface. If the subject is placed on the surface, the pattern is projected onto the three-dimensional surface, and the distortion correction parameter calculation unit is based on the image data obtained by photographing the projected pattern by the camera unit. To calculate a parameter for returning the surface shape of the subject placed on the table to a flat surface. Then, the image data distortion correction calculation unit converts the image data into planar image data from which distortion generated in the image data photographed by the camera unit is removed based on the calculated distortion correction parameter. Finally, a planar image from which distortion has been removed is output.
[Selection] Figure 2

Description

  The present invention relates to an image output apparatus, and more particularly to an image output apparatus that can output an image of a subject placed on a table.

2. Description of the Related Art Conventionally, an image forming apparatus capable of photographing a subject placed on a predetermined base surface is known.
In the devices disclosed in Patent Document 1 and Patent Document 2, the subject placed on the surface is photographed from above by the photographing element that is pulled out on the surface after the surface is pulled out.

JP 2010-68289 A JP 2010-68290 A

However, when photographing what is placed on the table surface from above, for example, since the photographing is performed in a state where the paper surface of the book is curved, the photographing is performed in a distorted state.
The present invention relates to an image output apparatus capable of converting an image which is originally a plane but is three-dimensionally curved and photographed into an image in an original plane state.

  In the present invention, it is possible to place a predetermined subject on the base surface. When the projection unit projects light of a predetermined pattern onto the surface from a position higher than the surface of the table, the camera unit performs photographing on the surface. If the subject is placed on the surface, the pattern is projected onto the three-dimensional surface, and the distortion correction parameter calculation unit is based on the image data obtained by photographing the projected pattern by the camera unit. To calculate a parameter for returning the surface shape of the subject placed on the table to a flat surface. Then, the image data distortion correction calculation unit converts the image data into planar image data from which distortion generated in the image data photographed by the camera unit is removed based on the calculated distortion correction parameter. Finally, a planar image from which distortion has been removed is output.

Here, the distortion correction parameter is not limited to a parameter that can completely eliminate the distortion, and includes a parameter that reduces the distortion. The projection unit may be configured to project the light of the pattern radially from the vicinity of the center of the subject. When projected radially, a shift occurs at each point in proportion to the height of the surface of the three-dimensional object.
Further, the projection unit can be configured to project a grid pattern as the light of the radial pattern. If the pattern is a checkerboard pattern, each grid point is determined, and the height information of each grid point can be obtained, making it easy to grasp a three-dimensional object.

Further, the distortion correction parameter calculation unit has a table showing a correspondence relationship between the shift amount and the height for each grid-like lattice point, and obtains the shift amount for each lattice point of the distortion to obtain height information. You may comprise so that a three-dimensional surface shape may be calculated | required by converting.
Since a fixed positional relationship between the table surface and the projection unit is determined, a table for conversion between the shift amount and the height can be prepared for each lattice point in advance, and the calculation becomes easy.

Further, the distortion correction parameter calculation unit can be configured to calculate a distortion correction parameter for returning a curved solid surface to a flat state when a three-dimensional shape is obtained. Since the curved three-dimensional surface can be grasped, it is possible to return to the unfolded state by simply connecting the flat surfaces horizontally.
For this reason, the said distortion correction calculating part is comprised so that the calculation which returns the curved solid surface to the state expand | deployed on the plane based on the said distortion correction parameter may be performed.

  According to the present invention, since the paper surface of a book can be converted to a flat state, it is possible to obtain an image free from distortion due to bending.

1 is a perspective view showing an appearance of an MFP according to an embodiment of the present invention. It is a perspective view which shows the external appearance at the time of subject imaging. It is a perspective view which shows the back side of a head part. It is a perspective view which shows an optical sensor. 2 is a block circuit diagram of the MFP. FIG. It is a flowchart showing the process of 3D external light measurement. It is a flowchart which shows the process of external light removal in 3D. It is a figure which shows the position of the to-be-photographed object and virtual light source which were mounted on the base surface. It is a flowchart showing the process of 2D external light measurement. It is a flowchart which shows the process of external light removal in 2D. It is a top view which shows the state mounted in the state of opening a thick book on the base surface. It is the perspective view shown so that it might be easy to understand the state. It is a front view which shows the positional relationship of the book and camera which were seen from the front. It is a flowchart along the procedure of imaging | photography in 3D. It is a figure which shows the state which projected the checkered pattern on the base surface. It is a figure which shows the state which mounted the thick book on the base surface in two pages, and projected the same pattern. It is a flowchart which shows the procedure of imaging | photography in 2D. It is a perspective view which shows the external appearance of the multifunctional device concerning a modification. It is a perspective view which shows the external appearance at the time of subject photography. It is a top view at the time of imaging | photography. It is a perspective view which expands and shows the camera for imaging | photography partially. It is a flowchart which shows the procedure of imaging | photography by 2D. It is a figure which shows as a figure the distortion correction parameter which shows a response | compatibility with the image when a base surface is image | photographed at the position of a camera, and an original two-dimensional planar image. It is a perspective view which shows a non-photographing state. It is a side view which shows a non-photographing state. It is a perspective view which shows an imaging | photography state. It is a side view which shows a photographing state. It is a rear view which shows an imaging | photography state. It is a figure which shows as a figure the distortion correction parameter which shows a response | compatibility with the image when a base surface is image | photographed at the position of a camera, and an original two-dimensional planar image. It is a flowchart which shows the process which normalizes so that original size may be understood. It is a figure which shows the problem which loses size.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Hereinafter, an embodiment in which the image output apparatus of the present invention is applied to a multifunction peripheral (MFP) having a facsimile function, a printing function, a scanner function, and the like will be described.
FIG. 1 is a perspective view showing the appearance of an MFP according to an embodiment of the present invention, and FIG. 2 is a perspective view showing the appearance when an object is imaged.

  In the figure, a multifunction peripheral (MFP) 10 has a long rectangular outer frame in a plan view, and is formed as a generally rectangular casing having a sufficiently thin shape in the height direction compared to the width direction and the depth direction. . On the front side, a discharge port 11 that discharges a printed material that is long in the width direction is formed. A front wall surface above the discharge port 11 includes a plurality of operating elements 12, a display panel 13, and the like. The upper surface is a base surface 15 on which the subject is placed.

  The MFP 10 is a separate member from the main body housing 14 and extends from the middle of the right side toward the rear end toward the rear end when viewed from the front side, and then extends toward the left side along the back side so as to be centered on the back side. It has a generally L-shaped arm 20 that reaches a certain extent. The arm 20 includes a connecting portion 21 located on the right side surface of the main body housing 14 and a head portion 22 located on the back side of the main body housing 14, and the center of the right side surface of the main body housing 14 is the center of rotation. As described above, the connecting portion 21 is pivotally connected to the main body housing 14. Then, as the connecting portion 21 pivots, the head portion 22 is movable between a housing position where the head portion 22 is located on the back side of the main body housing 14 and a photographing position where the head portion 22 is above the main body housing 14. It has become.

  FIG. 3 is a perspective view showing the back side of the head portion 22, and as shown in FIG. 3, a predetermined distance in the width direction of the main body housing 14 is provided on the surface of the head portion 22 facing the base surface 15 at the shooting position. Two cameras 31 and 32 arranged apart from each other, a projection unit 33 positioned therebetween, and an illumination unit 34 wide in the width direction with a predetermined distance between the front side and the back side of the head unit 22 so as to cover the projection unit 33 35 is arranged. As described above, since the two cameras 31 and 32 facing the base surface 15 and spaced apart from each other by a predetermined distance are provided, it is possible to take two images with different photographing positions with respect to the subject on the base surface 15. 3D shape information can be obtained. The illumination units 34 and 35 can change the illuminance, and can be used for processing to reduce the influence of external light.

As described above, the arm 20 can be moved from the housed state to the photographing state, so that the camera unit configured by the cameras 31 and 32 can be housed in the housed position below the base surface. It is supported so as to be movable from a lower storage position to a shooting position on the same surface.
In a state where the head unit 22 is at the photographing position, an optical sensor 36 and a plurality of operating elements 37 are provided on the upper surface of the head unit 22 and are composed of one unit oriented in five directions.

  FIG. 4 shows the optical sensor 36 in a perspective view. As shown in the figure, the optical sensor 36 has a three-dimensional shape that is trapezoidal when viewed from the front, both sides, and the back, and has a square outer shape in a planar state, and opens upward on the upper surface. One element is provided, and one optical sensor element that opens in the same oblique direction is provided on each obliquely formed side surface. As will be described later, the light sensor 36 detects the condition of external light on the upper surface of the head unit 22.

  FIG. 5 is a block circuit diagram of the MFP. The printing unit 41 includes a printing mechanism housed in the main body housing 14 and performs printing. The camera unit 42 includes the cameras 31 and 32. An operation unit 43 that accepts operations by the operators 12 and 37 and displays necessary information on the display panel 13, a communication unit 44 that communicates with an external device by fax transmission / reception via a telephone line, a network, etc., and a projection unit 33, illumination units 34 and 35, an I / O unit 45 to which the optical sensor 36 is connected, and a control unit 46 for comprehensively controlling them. The control unit 46 mainly performs control of each unit while executing the processing of the following flowchart.

Next, the operation of the printing apparatus of this embodiment having the above-described configuration will be described.
FIG. 6 is a flowchart showing 3D ambient light measurement processing, and FIG. 7 is a flowchart showing 3D ambient light removal processing.
As shown in the figure, the external light removal can employ two methods. The first is 3D processing. In step S100, the detected light intensity of the optical sensor 36 in five directions is measured. Specifically, the control unit 46 inputs the detection output of the light sensor 36 via the I / O unit 45.

FIG. 8 is a diagram showing the positions of the subject placed on the table 15 and the virtual light source.
As described above, the optical sensor 36 detects the light intensity in the five directions, and therefore, in step S102, the control unit 46 performs the weighting calculation according to the light intensity in the five directions and the XY direction with the arrangement position of the optical sensor 36 as the origin. In addition to the above, the virtual light source direction consisting of the angle of attack θ is obtained. In this way, by setting the virtual light source direction (lighting), a correction value opposite to that of lighting is applied to the 3D model data later to be used for removing the influence of external light.

As will be described later, in this MFP, 3D modeling is possible by using two cameras 31 and 32, and 3D model data can be acquired. Based on this assumption, the external light removal process in 3D will be described first.
In step S110, 3D model data is acquired. The 3D model data includes vector data representing the coordinates of the surface of the three-dimensional object and image data of the surface of the three-dimensional object specified by the vector data. As a result, a three-dimensional object surface image can be expressed excluding a surface that becomes a blind spot. In step S112, the virtual light source direction obtained in step S102 is acquired. Normally, the virtual light source direction is used in order to inherently apply the shading effect to the image data of the surface of the three-dimensional object, but in step S114, the influence of lighting is obtained and the opposite correction value is calculated. That is, a correction value that darkens if the surface is brightly displayed facing the virtual light source, and a correction value that brightens if the surface is darker on the side of the virtual light source. In step S116, this correction value is set for each surface of the 3D model data. In step S118, the luminance correction of each surface of the 3D model data is reflected and converted into flat image data, thereby removing the external light.

As described above, the optical sensor 36 oriented in a plurality of directions is used, information on the external light source (virtual light source direction) is detected based on the detection result of the optical sensor 36, and external light is detected based on the information. The influence of is removed.
In this example, the optical sensor 36 constitutes an external light detection unit that detects the state of external light on the table surface 15. After measuring the detected high intensity of the light sensor 36 in step S100, a virtual light source is set in step S102, a virtual light source direction is acquired in step S112, and a lighting reverse correction value is calculated in step S114. In step S116, the process of setting the reverse correction value for each surface calculates the adjustment value that cancels the influence of the camera unit on the imaging result on the base surface based on the detection result of the external light detection unit. Configure. In step S118, the process of reflecting the luminance correction of each surface of the 3D model data reflects the adjustment value in an image obtained by photographing the subject on the table with the camera unit, thereby reducing the influence of external light. The external light removal processing unit for performing the processing is configured.

In particular, the optical sensor 36 includes optical sensors oriented in a plurality of directions, and can detect information on an external light source based on detection results of the optical sensors.
Next, a 2D method will be described as a method for removing external light.
FIG. 9 is a flowchart showing 2D external light measurement processing, and FIG. 10 is a flowchart showing 2D external light removal processing.
In step S130, calibration light projection and table surface photography are performed. Specifically, the control unit 46 sequentially changes the intensities of the illumination units 34 and 35 from the weakest, the weakest, the middle, and the maximum, and causes the camera 31 or the camera 32 to photograph the surface of the table 15 at each stage. The obtained image data becomes four image data having different illumination intensities. Since this is taken in the sense of calibration, if necessary, photographing may be performed after placing a non-uniform calibration sheet provided in advance on the surface 15. In step S132, the captured image is divided into a predetermined number of sections (cells), and an approximate light / dark state is detected for each section.

  If a bright / dark situation is obtained, the brightness correction value for each section is calculated in step S134. In the present embodiment, a relative value with a section near the center as a reference value is used. Photographing is performed a plurality of times by changing the intensity of illumination using the illumination units 34 and 35, and after obtaining a relative value based on the respective photographing results, an average value of the correction values for each time is further taken. This makes it possible to effectively remove external light. The 2D external light measurement process is thus completed.

  External light removal in each image is performed as follows. The control unit 46 controls the camera unit 42 to photograph the subject on the table 15 with either the camera 31 or the camera 32. In this case, since it is a 2D process, it is only necessary to obtain image data taken by one of the cameras. If the image data is obtained, the luminance correction value is applied to each section of the 2D data in step S140. The degree of influence of external light on the table 15 is grasped as a relative value of brightness for each section, and the same correction value is obtained in step S134. Therefore, this correction value is applied to each section of 2D image data. In general brightness correction, a γ curve is applied to the luminance value of image data. Prepare a γ value corresponding to the relative contrast ratio as a separate table, read the γ value from the table and draw a γ curve based on the γ value, create a correspondence table of input values and output values, The brightness value of image data is converted.

As described above, under the plurality of environments in which the illuminance is changed by the illumination units 34 and 35, the camera 31 and 32 is used to photograph the surface 15 and external light based on the luminance distribution for each section (cell). Then, an adjustment value that cancels the influence on the photographing result on the table surface is calculated based on the distribution.
The printing unit in the present invention corresponds to the printing unit 41 in the present embodiment, the base corresponds to the base 15, the camera corresponds to the camera unit 42 and the cameras 31 and 32, and the external light detection unit includes the optical sensor 36 and The cameras 31 and 32 that photograph the surface 15 of the table 15 correspond. The external light calculation unit corresponds to the processing of steps S100 and S102 and the processing of steps S130 to S134 and the respective hardware, and the external light removal processing unit corresponds to the processing of steps S110 to S118 and step S140 and the respective hardware. It can be said that the control unit 46 corresponds to the print control unit.

  In particular, in step S130 and step S132, the captured image is divided into a predetermined number of sections (cells) based on four image data having different illumination intensities, and an approximate light / dark state is detected for each section. This corresponds to the process of photographing the surface of the table under a plurality of illumination environments by the camera unit and detecting the situation of the external light based on the luminance distribution, and the process of obtaining the correction value in step S134 is based on the distribution. This corresponds to a process of calculating an adjustment value that cancels the influence on the imaging result on the table surface.

Next, a description will be given along the shooting procedure.
FIG. 11 is a plan view showing a state where a thick book is placed on the base surface 15 in a spread state. FIG. 12 is a schematic perspective view for easy understanding of the state, and FIG. FIG. 2 schematically shows the positional relationship between the book and the camera. FIG. 14 is a flowchart along the 3D shooting procedure.

  In photographing, outside light measurement processing is performed in step S200. This process is as described above. In step S202, 3D shooting is performed by the two cameras 31 and 32. As shown in FIG. 13, when the book on the table 15 is photographed by the two cameras 31 and 32 spaced apart from each other, two images slightly shifted are obtained. In step S <b> 204, the outline of the three-dimensional object placed on the table 15 is obtained by replacing the shift between the two images with parallax. This is the 3D model data described above. Thereafter, in step S206, external light removal processing is performed. The processing from step S110 to step S116 is equivalent. Then, a surface image of a 3D three-dimensional object whose brightness is adjusted so as to remove the influence of external light is obtained.

In step S208, parameters for converting the 3D model to 2D are calculated. Here, instead of obtaining an original 3D stereoscopic image, image data of a flat surface that should be obtained if it is not curved is generated from a state where the surface of the book is curved.
Since the image data can be pasted on each minimum surface formed by connecting them based on the surface position of the three-dimensional object determined by the vector data, each plane is developed on the plane in the process of step S210. 2D image data is obtained. This process is obtained by calculation. However, a 3D model having a three-dimensional shape patterned in advance may be prepared in consideration of calculation complexity, and a 3D model having a close three-dimensional shape may be selected. Of course, each 3D model has a simple shape that facilitates deployment. In this way, it is possible to easily obtain planar image data based on parameters for obtaining a development operation when image data is pasted on the selected 3D model. This is because the basic shape is similar if the book is spread.

  The two cameras 31 and 32 are provided on the rear surface of the arm 20 with a predetermined distance therebetween, and can photograph the same surface 15 at a predetermined position on the surface 15. Therefore, the camera unit is configured by these configurations. Based on the image data obtained by photographing the subject on the table 15 with the two cameras 31 and 32 in step S202, the outline of the three-dimensional object placed on the table 15 is obtained by replacing the deviation of the two images with parallax in step S204. This processing constitutes a distortion correction parameter calculation unit for calculating a parameter for returning the surface shape of the subject placed on the table surface to a planar shape based on image data photographed at a plurality of positions. In other words, the two contours of the subject placed on the table surface are obtained by replacing two images shifted in image data taken at a plurality of positions with parallax.

  In step S208, parameters for converting the 3D model into 2D are calculated. In step S210, 2D image data in which each plane is developed into a flat shape is obtained. Therefore, an image data distortion correction calculation unit that converts the distortion generated in the image data into planar image data based on the distortion correction parameter calculated by these processes is configured. That is, image data is obtained by expanding each plane formed by connecting them based on the surface position of the three-dimensional object. Thereafter, the image data is stored in step S212.

  In addition, in order to simplify the calculation, a processing example in which a 3D model having a three-dimensional shape patterned in advance is prepared and a three-dimensional shape obtained by applying the three-dimensional shape to the model is prepared. A model is prepared, and an image data distortion correction calculation unit that performs distortion correction based on a parameter for obtaining a development calculation when image data is pasted on a 3D model having a similar three-dimensional outline is configured.

Further, since the control unit 46 causes the printing unit 41 to print an image based on the stored plane image data, the processing constitutes a printing control unit.
The above-described example is an example of 3D shooting, but an example of 2D shooting will be described below.
FIG. 15 shows a state in which a grid-like pattern is projected on the table surface 15, and FIG. 16 shows a state in which the same pattern is projected by placing a thick book on the table surface 15 in an open state. FIG. 17 is a flowchart along the procedure of 2D shooting.

  First, in step S230, external light measurement processing is performed as described above. Next, in step S232, a distortion detection pattern that is light of a predetermined pattern is projected. A projection unit 33 is provided on the side of the head unit 22 facing the base surface 15 and projects a grid-like pattern (distortion detection pattern) as shown in FIG. This projection unit is located almost above the center of the table 15. When nothing is placed on the table 15, an accurate grid is projected as shown in FIG. 15, but when a book is placed on the table 15, a three-dimensional shape is reflected as shown in FIG. 16. The image is distorted. In the case of a grid pattern projected radially from the projection unit 33, each grid point of the grid shifts in the radial direction corresponding to the height. Therefore, the shift amount for each lattice point is the height information at that lattice point.

  In step S234, 2D shooting is performed with one camera 31. Note that the position of the projection unit 33 and the position of the camera 31 do not match, but can be eliminated by a correction calculation. Based on the photographed image data, lattice point detection of the distortion detection pattern is performed in step S236. If the above-described deviation amount is obtained for each position of each lattice point, a three-dimensional shape is obtained based on the height information. Therefore, in step S238, the curved three-dimensional surface is made flat as described above. A distortion correction parameter is calculated to return to the expanded state. Thus, a parameter for returning distortion when a three-dimensional object is photographed as a 2D image is obtained.

  In step S240, the subject on the surface 15 is photographed by one camera 31. Of course, the distortion detection pattern is not projected at this point. In step S242, external light removal processing is performed. The external light removal process may employ a 3D external light removal method or a 2D external light removal method. In any case, after removing the influence of external light, in step S244, distortion of 2D image data is corrected using the parameters obtained in step S238. After the correction, the corrected image data is stored in step S246.

  In the present embodiment, the projection unit is configured by the process of step S232 for projecting the distortion detection pattern from the projection unit 33 onto the surface 51. Further, based on the image data obtained by capturing the picture projected by the camera 31 in 2D, the surface shape of the subject placed on the table surface 51 is planar from the distortion of the picture in the processes of steps S236 and S238. The processing for calculating the parameter to return to (1) constitutes a distortion correction parameter calculation unit. In step S240, a single camera 31 captures a subject on the surface 15, and in step S244, the planar image data from which the distortion generated in the image data is removed based on the calculated distortion correction parameter. The image data distortion correction calculation unit constitutes the image data distortion correction calculation unit.

Further, since the control unit 46 causes the printing unit 41 to print an image based on the stored plane image data, the processing constitutes a printing control unit.
FIG. 18 is a perspective view showing an external appearance of a multifunctional machine according to a modification, and FIG. 19 is a perspective view showing an external appearance at the time of photographing a subject. FIG. 20 is a plan view at the time of shooting, and FIG. 21 is a partially enlarged perspective view of the camera for shooting.

  The MFP 50 has a table surface 51, and a strip-shaped arm 52 extending in the width direction on the near side is embedded in the table surface 51. The arm 52 is rotatable so that the left end is lifted around the right end as viewed from the front. As shown in FIG. 19, the arm 52 is almost upright. A recess 53 is formed near the upper end of the arm 52 and on the side facing the base surface 51. A camera 54 is disposed in the recess 53, and the camera 54 can photograph a subject on the table surface 51. In this example, only one camera 54 is provided, but it is also possible to provide a plurality of cameras so that stereoscopic images can be obtained by changing the viewpoint. Since the position of the camera 54 with the arm 52 standing upright is constant, it can be said that the position of the camera 54 can be determined by standing upright.

FIG. 22 is a flowchart illustrating a procedure of 2D shooting.
In step S250, camera position detection is performed. As described above, the position of the camera 54 when the arm 52 is upright is constant. Therefore, the position of the camera 54 can be detected by raising the arm 52 upright. In this example, the operation of raising the arm 52 is applicable, but the position of the camera is arbitrary, and the camera position can be set from the camera side. This corresponds to setting a predetermined position.

  In step S252, camera position corresponding distortion correction parameter acquisition processing is performed. FIG. 23 shows, as a figure, distortion correction parameters indicating the correspondence between an image when the table surface 51 is photographed at the position of the camera 54 and the original two-dimensional planar image. When the camera 54 is used to photograph the top surface 51, a long rectangular shape is photographed in a distorted diamond shape as shown in the upper part of FIG. Since this correspondence pattern is fixed, by associating the coordinate position of the rhombus image with the coordinate position of the original long rectangular image, the image captured by the camera 54 is not distorted. The original plane image can be converted.

Accordingly, 2D shooting is performed with the camera 54 in step S254, and distortion is removed by applying distortion correction parameters corresponding to the position of the camera 54 in step S256.
In the embodiment shown in FIGS. 18 to 21 described above, the camera 54 can move from the state in which the arm 52 is laid down to the upright state. Take a photo of In the processing of step S250, the camera position is detected, and it is known that the base 51 photographed from this position is photographed with distortion as shown in the upper part of FIG. Therefore, the process of step S250 corresponds to the process of the camera position detection unit that detects the positional relationship between the shooting position of the camera unit and the table surface. Further, since the correspondence shown in FIG. 23 is known, in step S252, information for associating the coordinate position of the rhombus image with the coordinate position of the original long rectangular image is acquired. Corresponds to a camera position corresponding distortion correction parameter calculation unit. The term “calculation” also includes referring to a table associated in advance in a broad sense, and such a two-dimensional image can be easily configured as a two-dimensional image conversion table. If the camera 63 is positioned as shown in FIG. 28 as will be described later, the correspondence between the trapezoid and the rectangle shown in FIG. 29 corresponds to the two-dimensional image conversion table.

  The processing for removing distortion by applying the distortion correction parameter corresponding to the position of the camera 54 in step S256 to the image captured in step S254 occurs in the image data based on the calculated camera position corresponding distortion correction parameter. This corresponds to a distortion correction calculation unit for image data to be converted into image data from which distortion is removed. Since the control unit 46 causes the printing unit 41 to print an image based on the planar image data obtained in this way, the processing constitutes a printing control unit.

24 to 28 show other modified examples of the present invention. FIG. 24 is a perspective view showing the non-photographing state, and FIG. 25 is a side view showing the same state. 26 is a perspective view, FIG. 27 is a side view, and FIG. 28 is a rear view.
As shown in the figure, in the MFP 60, the front panel 61 can be slid upward with respect to the main body casing 67 so that the base surface 64 can be looked down at the time of photographing or printing, and when not photographing or not in use. Slide down. By sliding upward, a discharge port 62 for discharging the printing paper is opened on the front surface. Although not shown, the printing unit 41 is accommodated in the main body casing 67.

  Further, when the panel 61 slides upward, the camera 63 disposed near the upper end on the back side also rises and moves to a position overlooking the top surface 64 from above. The camera 63 is disposed at the center of the panel 61 in the width direction. Note that illumination units 65 and 66 are disposed on both sides of the panel 61 so as to sandwich the camera 63 therebetween. Since the illumination units 65 and 66 are provided on the back side of the panel 61, the illumination light is not directly projected to the operator side at the time of photographing, and is not dazzling.

In this way, when the surface of the base 64 is photographed from the upper front side, distortion appears in the trapezoidal shape as shown in the upper part of FIG. This is called trapezoidal distortion. The correspondence between the trapezoidal coordinate position and the rectangular rectangular coordinate position below it is a distortion correction parameter corresponding to the camera position.
Therefore, even in this example, distortion correction parameters corresponding to the camera position can be acquired and distortion can be removed.

  In this embodiment, the printing unit 41 is accommodated in a main body casing 67, and the main body casing 67 has a thin box shape and has a base surface 64 on which an object can be placed. The panel 61 is disposed on the side surface of the main body housing 67 and is configured to be slidable in the vertical direction along the side surface. The camera 63 is disposed on the rear surface side of the panel 61, and when the panel 61 slides upward on the side surface of the main body housing 67, the camera 63 takes a position higher than the table surface 64 and performs shooting on the table surface 64. It constitutes the camera part.

Then, since the control unit 46 causes the printing unit 41 to print an image based on the image data photographed by the camera 63, the processing constitutes a printing control unit.
By the way, in the case of the subject placed on the surface, there is a problem that the size is not known because the photographing position changes.
FIG. 30 shows a process of normalization so that the original size can be understood, and FIG. 31 illustrates this problem by a diagram.

As shown in FIG. 31, when a thin subject is placed on the base surface 72 and the screen is shot full, and when a tall subject is placed on the base surface 72 and the screen is shot full. Both image data are photographed to the same size.
In step S260, therefore, distance measurement (AF) is performed. Various methods can be employed for distance measurement, and as an example, the distance can be measured by obtaining the sharpness of the captured image while shifting the focal position. After measuring the distance, 2D shooting is performed in step S262. Thereafter, in step S264, the same distance is reflected and normalized to a standard size. That is, the image is enlarged or reduced according to the distance measured. In step S268, the normalized image data is stored.

By doing in this way, even if it is a three-dimensional thing, it can always be set as image data in which the original size was reflected.
In this way, the distance is measured in step S260 and converted to the standard size based on the distance measured in step S264, and this process measures the distance to the subject on the surface, A print control unit is configured to print the standardized image data obtained by enlarging or reducing the image corresponding to the measured distance.
Further, the photosensors 36 may be provided on the table surface 15, and the number of photosensors 36 is not limited to one, and a plurality of photosensors 36 may be provided. Since the subject is placed on the table 15, the influence of external light can be further eliminated by doing in this way.

Needless to say, the present invention is not limited to the above embodiments. It goes without saying that those skilled in the art may apply to an output device that outputs an image by a method such as display output or data transmission other than printing,
・ Applying mutually interchangeable members and configurations disclosed in the above embodiments by appropriately changing the combination thereof.− Although not disclosed in the above embodiments, it is a publicly known technique and the above embodiments. The members and configurations that can be mutually replaced with the members and configurations disclosed in the above are appropriately replaced, and the combination is changed and applied.

・ Although not disclosed in the above-described embodiments, those skilled in the art may appropriately substitute members and configurations that can be assumed as substitutes for the members and configurations disclosed in the above-described embodiments based on known techniques, Changing and applying the combination is disclosed as an embodiment of the present invention.

DESCRIPTION OF SYMBOLS 10 ... MFP, 11 ... Discharge port, 12 ... Control, 13 ... Display panel, 14 ... Main body housing, 15 ... Base surface, 20 ... Arm, 21 ... Connection part, 22 ... Head part, 31, 32 ... Camera, 33 ... Projection unit, 34, 35 ... illumination unit, 36 ... optical sensor, 37 ... operator, 41 ... printing unit, 42 ... camera unit, 43 ... operation unit, 44 ... communication unit, 45 ... I / O unit, 46 ... control , 50 ... MFP, 51 ... base, 52 ... arm, 53 ... recess, 54 ... camera, 60 ... MFP, 61 ... panel, 62 ... outlet, 63 ... camera, 64 ... base, 65, 66 ... lighting part, 67 ... Body body, 72 ... Base surface

Claims (6)

The surface,
A projection unit that projects light of a predetermined pattern onto the surface from a position higher than the surface;
A camera unit for shooting on the surface;
A distortion correction parameter calculation unit for calculating a parameter for returning the surface shape of the subject placed on the table surface to a planar shape from distortion of the pattern based on image data obtained by photographing the projected pattern by the camera unit; ,
A distortion correction calculation unit for image data to be converted into planar image data from which distortion generated in the image data is removed based on the calculated distortion correction parameter;
An image output apparatus comprising: an output unit that outputs a planar image from which distortion has been removed.   The image output apparatus according to claim 1, wherein the projection unit projects the light of the pattern radially from the vicinity of the center of the subject.   The image output apparatus according to claim 2, wherein the projection unit projects a grid-like pattern as the light of the radial pattern.   The distortion correction parameter calculation unit has a table indicating a correspondence relationship between the shift amount and the height for each grid-shaped grid point, and obtains the shift amount for each grid point of the distortion and converts it into height information. The image output apparatus according to claim 3, wherein the three-dimensional surface shape is obtained by the above-described process.   5. The distortion correction parameter calculation unit according to claim 4, wherein when the three-dimensional shape is obtained, the distortion correction parameter calculation unit calculates a distortion correction parameter for returning the curved three-dimensional surface to a flat state. Image output device.   6. The image output apparatus according to claim 5, wherein the distortion correction calculation unit performs a calculation to return the curved three-dimensional surface to a flat state based on the distortion correction parameter.

Images