JP2014081519A - Imaging device and display device - Google Patents

Abstract

A high-definition aerial image is obtained by integral photography.
An imaging apparatus includes a plurality of imaging elements arranged at least in one of a horizontal direction and a vertical direction with a gap between imaging surfaces, and an imaging surface of each of the plurality of imaging elements. Correspondingly, a plurality of lens arrays provided with gaps are provided.
[Selection] Figure 1

Description

  The present invention relates to an imaging device and a display device.

  Integral photography is known as one of the aerial image reproduction methods (see, for example, Patent Document 1). A photographing apparatus to which this integral photography is applied photographs a light beam from a subject incident through a lens array and generates an integral image. A display device to which integral photography is applied generates a light beam by displaying an integral image on a display unit, and generates a spatial image by passing the light beam through a lens array. That is, integral photography is a reproduction method in which a display device generates a light beam similar to a light beam coming from an actual subject.

JP 2001-228570 A

  The resolution of the aerial image by integral photography depends on the number of light beams emitted from the display device through the lens array. The number of light rays emitted from the display device, that is, the number of light rays recorded in the integral image depends on the resolution of the photographing device. Therefore, it is effective to increase the resolution of the photographing apparatus as one method for increasing the definition of the aerial image by integral photography.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a photographing apparatus and a display apparatus that obtain a high-definition aerial image by integral photography.

[1] In order to solve the above-described problem, an imaging device according to one embodiment of the present invention includes a plurality of at least one of a horizontal direction and a vertical direction with a gap between imaging surfaces. An imaging device, and a plurality of lens arrays provided with a gap so as to correspond to each of the imaging surfaces of the plurality of imaging devices.
[2] In the photographing apparatus according to [1], the size of the gap between the imaging surfaces is a positive integer multiple of the pixel pitch of the imaging device, and the size of the gap between the lens arrays is the lens. It is a positive integer multiple of the lens pitch of the array.
[3] In the photographing apparatus according to any one of [1] or [2], by combining the plurality of first element image group data generated by the plurality of image sensors in correspondence with the arrangement of the image sensors. And an image processing unit for generating second element image group data.
[4] In the photographing apparatus according to [3], the image processing unit may combine the plurality of first element image group data from a pixel corresponding to a gap between the imaging surfaces to the lens array. The pixel values of the pixels excluding the pixels corresponding to the gaps of the plurality of pixels are obtained by pixel interpolation processing, the element images corresponding to the gaps between the lens arrays are obtained by the interpolation processing of the element images, and the plurality of first element images The second element image group data is generated by combining group data, the pixel value, and the element image.
[5] In order to solve the above-described problem, an imaging device according to one embodiment of the present invention includes a plurality of at least one of a horizontal direction and a vertical direction with a gap between imaging surfaces. An imaging device, and a lens array provided corresponding to the imaging surfaces of the plurality of imaging devices.

[6] In order to solve the above-described problem, a display device which is one embodiment of the present invention includes a plurality of display devices having a gap between display surfaces and arranged in at least one of a horizontal direction and a vertical direction. And a plurality of lens arrays provided with a gap so as to correspond to each of the display surfaces of the plurality of display elements.
[7] In the display device according to [6] above, the size of the gap between the display surfaces is a positive integer multiple of the pixel pitch of the display element, and the size of the gap between the lens arrays is the lens. It is a positive integer multiple of the lens pitch of the array.
[8] In the display device according to any one of [6] and [7] above, a plurality of second element image group data is obtained by dividing the first element image group data according to the arrangement of the display elements. And an image processing unit that generates and supplies the plurality of second element image group data to the plurality of display elements.
[9] In order to solve the above-described problem, a display device which is one embodiment of the present invention includes a plurality of display devices having a gap between display surfaces and arranged in at least one of a horizontal direction and a vertical direction. A display element, and a lens array provided in correspondence with the display surface of the plurality of display elements.

  According to the present invention, a high-definition aerial image can be obtained by integral photography.

1 is a configuration diagram illustrating a schematic configuration of an imaging display system to which an imaging apparatus according to a first embodiment of the present invention is applied. It is a side view at the time of seeing the multi lens imaging array shown in FIG. 1 in the X-axis direction. FIG. 2 is a diagram (XY plan view) schematically showing a gap between imaging surfaces in a multi-lens imaging unit when the multi-lens imaging array shown in FIG. 1 is viewed in the Z-axis direction from the subject side. FIG. 3 is a diagram (XY plan view) schematically showing a gap between imaging lens arrays in a multi-lens imaging unit when the multi-lens imaging array shown in FIG. 1 is viewed from the subject side in the Z-axis direction. FIG. 5 is a diagram showing both the gap between the imaging surfaces shown in FIG. 3 and the gap between the imaging lens arrays shown in FIG. 4. It is a block diagram which shows the schematic structure of the imaging | photography display system to which the imaging device which is 2nd Embodiment of this invention is applied. It is a side view at the time of seeing the multi lens imaging array in a 3rd embodiment of the present invention in the direction of the X-axis. It is a block diagram which shows the schematic structure of the imaging | photography display system to which the display apparatus which is 4th Embodiment of this invention is applied. It is a side view at the time of seeing the multi-lens display array shown in FIG. 8 in the X-axis direction. FIG. 9 is a diagram (XY plan view) schematically showing a gap between display surfaces in a multi-lens display unit when the multi-lens display array shown in FIG. 8 is viewed from the aerial image side in the direction opposite to the Z axis. FIG. 9 is a diagram (XY plan view) schematically showing a gap between display lens arrays in a multi-lens display unit when the multi-lens display array shown in FIG. 8 is viewed from the aerial image side in the direction opposite to the Z axis. . It is the figure which showed both the gap | interval between the display surfaces shown in FIG. 10, and the gap | interval between the lens arrays for a display shown in FIG. It is a block diagram which shows the schematic structure of the imaging | photography display system to which the display apparatus which is 5th Embodiment of this invention is applied. It is a side view at the time of seeing the multi lens display array in a 6th embodiment of the present invention to the X-axis direction. It is a block diagram which shows the schematic structure of the imaging | photography display system to which the imaging device and display apparatus which are 7th Embodiment of this invention are applied. FIG. 16 is a side view when the multi-lens display array shown in FIG. 15 is viewed in the X-axis direction.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.
[First Embodiment]
FIG. 1 is a configuration diagram showing a schematic configuration of an imaging display system to which an imaging apparatus according to a first embodiment of the present invention is applied. As shown in the figure, the photographing display system 1 includes a multi-lens imaging array 10, an image processing device (image processing unit) 20, a projection device 60, a diffusion plate 70, and a display lens array 80. Composed. In the photographing display system 1, the photographing device includes a multi-lens imaging array 10 and an image processing device 20.

  FIG. 1 shows a subject S photographed by the multi-lens imaging array 10. In the same figure, three axes (X axis, Y axis, and Z axis) orthogonal to each other are shown. The X axis and the Y axis are axes parallel to the horizontal direction and the vertical direction, respectively, of the imaging surfaces of the plurality of imaging elements included in the multi-lens imaging array 10. The Z axis is an axis that intersects perpendicularly with the imaging surface of each imaging element of the multi-lens imaging array 10. That is, the Z axis is an axis parallel to the main optical axis of the photographing apparatus.

  The multi-lens imaging array 10 includes, for example, nine multi-lens imaging units 100. As will be described in detail later, each multi-lens imaging unit 100 includes an imaging lens array (lens array) and the imaging element. As shown in FIG. 1, the multi-lens imaging array 10 includes a multi-lens so that the imaging surfaces of the imaging elements in each of the nine multi-lens imaging units 100 are parallel to a plane including the X axis and the Y axis (XY plane). A predetermined gap is provided between the image pickup units 100, and nine multi-lens image pickup units 100 are arranged in three in each of the X-axis direction and the Y-axis direction. In the present embodiment, each multi-lens imaging unit 100 is distinguished by attaching a subscript (two-digit number) indicating a two-dimensional coordinate to each of the nine multi-lens imaging units 100 (hereinafter, the same description). May be.)

  The multi-lens imaging array 10 is not limited to nine and may include a plurality of multi-lens imaging units 100. The multi-lens imaging array 10 is provided with a predetermined gap between the multi-lens imaging units 100, and the plurality of multi-lens imaging units 100 are arranged in at least one of the X-axis direction and the Y-axis direction. It may be configured.

  The multi-lens imaging unit 100 passes the light flux coming from the subject S through the imaging lens array, and photoelectrically converts the light flux that has passed through the imaging lens array by the imaging element to obtain element image group data (first element image group data). ) Is generated. The element image is an image obtained from the light flux from the element lens. Therefore, the element image group is an image group obtained from the light flux from the imaging lens array. The multi-lens imaging array 10 operates nine multi-lens imaging units 100 simultaneously to generate nine element image group data, and these nine element image group data are converted into element image group data sets (a plurality of first image groups). Element image group data) to the image processing apparatus 20. This element image group data set is a data set of element images corresponding to one frame.

  The image processing apparatus 20 takes in the element image group data set supplied by the multi-lens imaging array 10. The image processing apparatus 20 synthesizes the nine element image group data included in the element image group data set in correspondence with the arrangement of nine (3 × 3) multi-lens imaging units 100, thereby obtaining one frame. Element image group data is generated. Specifically, when combining the nine element image group data, the image processing apparatus 20, for example, removes pixels (excluding pixels corresponding to the gap between the imaging lens arrays from pixels corresponding to the gap between the imaging surfaces). A pixel value (interpolated pixel value) of the interpolation pixel) is obtained by pixel interpolation processing. Further, the image processing apparatus 20 obtains element image (interpolation element image data) data (interpolation element image data) corresponding to the gap between the imaging lens arrays by element image interpolation processing. Then, the image processing device 20 combines the nine element image group data, the interpolation pixel value obtained by the pixel interpolation process, and the interpolation element image data obtained by the element image interpolation process, Element image group data for one frame is generated. The image processing apparatus 20 supplies the element image group data for one frame to the projection apparatus 60 as integral image data (second element image group data). Details of the process (synthesizing process) in which the image processing apparatus 20 combines the elemental image group data sets to obtain integral image data will be described later. The image processing device 20 is realized by an arithmetic processing device, for example, a computer device, on which a CPU (Central Processing Unit) is mounted.

  The photographing display system 1 includes a display device. This display device includes a projection device 60, a diffusion plate 70, and a display lens array 80. This display apparatus takes in integral image data generated by the photographing apparatus according to the present embodiment, and displays the integral image data by integral photography.

  The projection device 60 takes in integral image data supplied by the image processing device 20. The projection device 60 converts the integral image data into image light and projects the image light toward the diffusion plate 70. The projection device 60 is realized by, for example, a projector device.

  The diffusing plate 70 is a flat plate having both light transmission and light diffusing properties. The diffusion plate 70 transmits the image light projected by the projection device 60 while diffusing from one surface (the surface on the projection device 60 side) and reaches the other surface. Thereby, the integral image is exposed on the other surface of the diffusion plate 70.

  The display lens array 80 is an element lens group configured by two-dimensionally arranging a plurality of element lenses regularly (for example, in a square lattice shape) so that the optical axes are parallel to each other. Each element lens is, for example, a convex lens. The display lens array 80 is provided at a position where the focal plane of each element lens coincides with the other surface of the diffusion plate 70. The aerial image corresponding to the subject S is generated by passing the luminous flux from the integral image displayed on the other surface of the diffusion plate 70 through the display lens array 80.

  In the imaging display system 1, the positional relationship between the plurality of element lenses and pixels included in the nine imaging lens arrays 100 is equal to the positional relationship between the plurality of element lenses included in the display lens array 80 and the pixels.

FIG. 2 is a side view of the multi-lens imaging array 10 shown in FIG. 1 when viewed in the X-axis direction. FIG. 2 shows a state in which the multi-lens imaging units 100 ₁₁ , 100 ₁₂ , and 100 ₁₃ are fixed to the substrate unit 103 among the nine multi-lens imaging units 100 included in the multi-lens imaging array 10. Referring to FIGS. 1 and 2 together, nine multi-lens imaging units 100 ₁₁ , 100 ₁₂ , 100 ₁₃ , 100 ₂₁ , 100 ₂₂ , 100 ₂₃ , 100 ₃₁ , 100 ₃₂ , and 100 ₃₃ are planes of the substrate unit 103. It is fixed to.

In FIG. 2, the multi-lens imaging unit 100 ₁₁ includes an imaging lens array 101 ₁₁ and an imaging element 102 ₁₁ . Similarly, the multi-lens imaging unit 100 ₁₂ includes an imaging lens array 101 ₁₂ and an imaging element 102 ₁₂ . The multi-lens imaging unit 100 ₁₃ includes an imaging lens array 101 ₁₃ and an imaging element 102 ₁₃ . Further, although not shown in the figure, for the multi-lens imaging units 100 ₂₁ , 100 ₂₂ , 100 ₂₃ , 100 ₃₁ , 100 ₃₂ , and 100 ₃₃ , the imaging lens array and the imaging device are provided in the same manner as in the figure. Prepare.

  The imaging lens array 101 is an element lens group configured by regularly arranging a plurality of element lenses in a two-dimensional manner so that the optical axes are parallel to each other. When the imaging lens array 101 is viewed from the Z-axis direction, the plurality of element lenses are two-dimensionally arranged in a square lattice pattern. Each element lens of the imaging lens array 101 is realized by, for example, a gradient index lens (GRIN) lens.

  Specifically, a gradient index lens having the following specifications is used as an element lens of the imaging lens array 101. That is, a refractive index distribution type lens having a specification satisfying the following formula (1) is used as an element lens, where L is the length in the optical axis direction of the refractive index distribution type lens, P is the meandering period of the refractive index distribution type lens. . However, in Formula (1), n is a positive integer.

  By using a gradient index lens satisfying the formula (1) as an element lens, an erect real image of the subject S existing sufficiently far from one end face of the gradient index lens can be obtained on the other end face or It can appear outside this end face. In order to cause an erect real image of the subject S existing sufficiently far from one end surface of the gradient index lens to appear on the other end surface, a gradient index lens having a specification satisfying the following formula (2) is used. Use. However, in Formula (2), n is a positive integer.

  FIG. 2 schematically illustrates an example in which the multi-lens imaging unit 100 is configured using a gradient index lens whose n is 1 in Formula (2). By using a gradient index lens satisfying Expression (2) as the element lens, crosstalk between element images can be suppressed.

  The image sensor 102 photoelectrically converts a light beam obtained from each element lens of the imaging lens array 101 to generate element image group data. The image sensor 102 is realized by a solid-state image sensor such as a CCD (Charged Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, for example.

  In the multi-lens imaging unit 100, the imaging lens array 101 is provided so that the focal plane of each element lens coincides with the imaging surface of the imaging element 102. For example, when a gradient index lens satisfying the formula (2) is used as an element lens of the imaging lens array 101, the focal plane of the gradient index lens is the subject S side of the gradient index lens. Can be matched with the end face on the opposite side. Therefore, in this case, as shown in FIG. 2, the multi-lens imaging unit 100 can be configured by bringing the end surface opposite to the subject S side of the imaging lens array 101 into contact with the imaging surface of the imaging element 102. By using the multi-lens imaging unit 100 configured as described above, it is possible to realize an imaging apparatus that does not include a condenser lens or a diffusion plate. Therefore, according to the photographing apparatus, image quality deterioration can be suppressed by the amount not including the condenser lens and the diffusion plate, and downsizing can be realized.

FIG. 3 is a diagram (XY plan view) schematically showing a gap between imaging surfaces in the multi-lens imaging unit 100 when the multi-lens imaging array 10 shown in FIG. 1 is viewed in the Z-axis direction from the subject S side. It is. The figure shows four multi-lens imaging units 100 ₁₁ , 100 ₂₁ , 100 ₁₂ , 100 ₂₂ adjacent to each other in the multi-lens imaging array 10. In addition, FIG. 5 shows an imaging lens array and pixels on the imaging surface arranged in a square lattice pattern in the XY plane in each multi-lens imaging unit 100.

  In FIG. 3, the pixel density with respect to the element lens is low, that is, the pixel is large with respect to the element lens. However, this is for convenience of explanation, and in fact, the pixel density with respect to the element lens is higher. . In addition, the imaging lens array in the figure is configured by arranging element lenses without gaps in the X-axis direction and the Y-axis direction. In this case, the lens pitch in each of the X-axis direction and the Y-axis direction is equal to the diameter of the element lens. equal. The lens pitch is the size between the center points of adjacent element lenses. Further, the image pickup device in FIG. 2 is a pixel in which pixels are arranged in a grid without gaps, and in this case, the pixel pitch in each of the X axis direction and the Y axis direction is equal to the pixel width. The pixel pitch is a size between the center points of adjacent pixels.

As shown in FIG. 3, in the XY plane, between the imaging plane of the imaging surface and the multi-lens imaging unit _{100 21} of the multi-lens imaging unit _{100 11,} and the imaging surface of the multi-lens imaging unit _{100 12} and a multi-lens imaging unit 100 A gap in the X-axis direction is provided between the ₂₂ imaging surfaces. The size P _X of the gap of the X-axis direction is a positive integer multiple of the X-axis direction pixel pitch in the imaging plane. The figure shows that the gap size P _X in the X-axis direction is 8 times the X-axis direction pixel pitch in the imaging plane.

Similarly, in the XY plane, between the imaging surface of the multi-lens imaging unit _{100 11} and a multi-lens imaging unit _{100 12} imaging surface, and the imaging surface of the multi-lens imaging unit _{100 21} of the imaging surface and the multi-lens imaging unit _{100 22} Is provided with a gap in the Y-axis direction. The size P _Y of the gap in the Y-axis direction is a positive integer multiple of the pixel pitch in the Y-axis direction on the imaging surface. This figure shows that the size P _Y of the gap in the Y-axis direction is 8 times the pixel pitch in the Y-axis direction on the imaging surface.

Note that the magnitude P _X and Y-axis direction of the gap size P _Y of the X-axis direction gap may not be equal may be equal.

4 is a diagram (XY plane) schematically showing a gap between imaging lens arrays in the multi-lens imaging unit 100 when the multi-lens imaging array 10 shown in FIG. 1 is viewed from the subject S side in the Z-axis direction. Figure). As shown in FIG. 4, in the XY plane, between the imaging lens array of multi-lens imaging lens array and the multi-lens imaging unit 100 ₂₁ of the imaging unit 100 _11, and the multi-lens imaging unit 100 ₁₂ imaging lens array and between the imaging lens array of multi-lens imaging unit 100 _22, a gap in the X-axis direction is provided. The size L _X of the gap in the X-axis direction is a positive integer multiple of the lens pitch of the imaging lens array. This figure shows that the size L _X of the gap in the X-axis direction is twice the lens pitch of the imaging lens array.

Similarly, in the XY plane, the multi-lens between the imaging lens array of the imaging unit 100 ₁₁ imaging lens array and the multi-lens imaging unit 100 _12, and the multi-lens imaging unit 100 ₂₁ imaging lens array and the multi-lens imaging between the imaging lens array parts 100 _22, the gap in the Y-axis direction is provided. The size L _Y of the gap in the Y-axis direction is a positive integer multiple of the lens pitch of the imaging lens array. This figure shows that the size L _Y of the gap in the Y-axis direction is twice the lens pitch of the imaging lens array.

Note that the size L _Y in the X-axis direction of the gap size L _X and Y-axis direction gap may not be equal may be equal.

FIG. 5 is a diagram showing both the gap between the imaging surfaces shown in FIG. 3 and the gap between the imaging lens arrays shown in FIG. As shown in FIG. 5, in the four multi-lens imaging units 100 ₁₁ , 100 ₂₁ , 100 ₁₂ , 100 ₂₂ arranged adjacent to each other in the XY plane, gaps between imaging surfaces in the X-axis direction and imaging lenses The gap between arrays is not the same. Similarly, the gap between the imaging surfaces in the Y-axis direction and the gap between the imaging lens arrays are not the same. According to FIG. 2 and FIG. 5, the nine multi-lens imaging units 100 have a size of the gap between the imaging surfaces of the imaging element 102 between the multi-lens imaging units 100 as a positive integer multiple of the pixel pitch on the imaging surface, And it fixes to the plane of the board | substrate part 103 so that the magnitude | size of the clearance gap between the lens arrays 101 for imaging satisfy | fills the positive integer multiple of a lens pitch.

  Here, the composition processing of the element image group data set by the image processing apparatus 20 will be described. The image processing apparatus 20 takes in the element image group data set from the multi-lens imaging array 10, and the nine element image group data included in the element image group data set is a gap corresponding to a positive integer multiple of the pixel pitch shown in FIG. Is stored in the storage unit built in the device itself. The storage unit is a frame memory capable of storing image data for at least one frame imaged by the multi-lens imaging array 10, for example. That is, nine element image group data are stored in the storage unit corresponding to the arrangement of the imaging surface of the imaging element 102.

  Then, the image processing device 20 obtains interpolation element image data corresponding to a gap (twice the lens pitch) between the imaging lens arrays shown in FIG. 4 by interpolation processing of adjacent element images or peripheral element images. . Since the plurality of element lenses in the imaging lens array 101 are arranged on a plane (XY plane) perpendicular to the main optical axis of the imaging apparatus, the line-of-sight direction in each element image depends on the arrangement of the element lenses. Change gradually. That is, there is a high probability that the correlation between images is high between adjacent element images, or between the element image of interest and the surrounding element images. Using such statistical properties, the image processing apparatus 20 uses an interpolation element whose line-of-sight direction is changed from an element image adjacent to an element image area corresponding to the gap between the imaging lens arrays 101 or a peripheral element image. An image is generated, and interpolation element image data is stored in the element image area in the storage unit.

  Then, the image processing device 20 sets the pixel value of the interpolation pixel, which is a pixel excluding the pixel of the interpolation element image, from the pixel corresponding to the gap between the imaging surfaces (eight times the pixel pitch of the imaging surface) illustrated in FIG. This is obtained by interpolation processing of pixels to be performed or peripheral pixels. For example, the image processing apparatus 20 obtains an interpolation pixel by linearly interpolating surrounding pixels with respect to the pixel of interest. Specifically, for example, when obtaining an interpolation pixel from four pixels adjacent to the pixel of interest, the image processing device 20 obtains the pixel value of the interpolation pixel by calculating an average value of the pixel values of the four pixels.

  As described above, the imaging apparatus according to the first embodiment includes a plurality of imaging elements 102 that are arranged in at least one of the X-axis direction and the Y-axis direction with a gap between imaging surfaces. A plurality of imaging lens arrays 101 provided with gaps are provided corresponding to the imaging surfaces of the plurality of imaging elements 102.

  With this configuration, it is possible to realize an imaging apparatus having a resolution that is extremely higher than the above-described resolution using the multi-lens imaging unit 100 having a predetermined resolution.

Here, the size of the gap between the imaging surfaces is a positive integer multiple of the pixel pitch of the imaging element 102, and the size of the gap between the imaging lens arrays 101 is a positive integer of the lens pitch of the imaging lens array 101. Is double.
In addition, the imaging apparatus according to the first embodiment is an image processing apparatus that generates integral image data by combining element image group data sets generated by a plurality of imaging elements 102 in correspondence with the arrangement of the imaging elements 102. 20 is further provided.

  Here, when synthesizing the element image group data set, the image processing apparatus 20 interpolates pixels excluding pixels corresponding to the gap between the imaging lens arrays 101 from pixels corresponding to the gap between the imaging surfaces of the imaging element 102. Are obtained by pixel interpolation processing. Then, the image processing apparatus 20 obtains interpolation element image data corresponding to the gap between the imaging lens arrays 101 by element image interpolation processing. Then, the image processing apparatus 20 generates integral image data by synthesizing the element image group data set, the interpolation pixel value, and the interpolation element image data.

  In this way, by providing a gap that is a positive integer multiple of the pixel pitch between the imaging elements 102 and by providing a gap that is a positive integer multiple of the lens pitch between the imaging lens arrays 101, the image processing apparatus 20 can be The pixel data acquired in (1) can be applied as it is to the corresponding pixels of the integral image data for one frame. Then, the image processing apparatus 20 can obtain the interpolation element image data by the interpolation process of the surrounding element images, and can obtain the interpolation pixel value by the interpolation process of the surrounding pixels. That is, the image processing apparatus 20 can obtain the integral image data for one frame by synthesizing the element image group data sets relatively easily.

  As described above, the photographing apparatus according to the first embodiment of the present invention can generate integral image data for obtaining a high-definition aerial image by integral photography.

[Second Embodiment]
The image processing apparatus 20 according to the first embodiment takes in an elemental image group data set from the multi-lens imaging array 10 and combines a plurality of elemental image group data included in the elemental image group data set to integrate one frame worth of integral. This is an example of generating image data. In the second embodiment of the present invention, an image processing apparatus that captures an element image group data set from the multi-lens imaging array 10 combines a plurality of element image group data included in the element image group data set for each predetermined group. This is an example of generating a plurality of divided integral image data obtained by dividing a frame into regions.

  FIG. 6 is a configuration diagram showing a schematic configuration of an imaging display system to which an imaging apparatus according to the second embodiment of the present invention is applied. Note that the same reference numerals in the second embodiment denote the same components as those in the first embodiment, and a description thereof will be omitted. As shown in the figure, the imaging display system 1a includes a multi-lens imaging array 10, an image processing device (image processing unit) 20a, and three projection devices 60 (60-1, 60-2, 60-3). And a diffusing plate 70 and a display lens array 80. In the photographing display system 1a, the photographing device includes a multi-lens imaging array 10 and an image processing device 20a.

  The image processing device 20a takes in the element image group data set supplied by the multi-lens imaging array 10. The image processing apparatus 20a is obtained by dividing nine frames of element image group data included in the element image group data set in correspondence with the arrangement of the nine multi-lens imaging units 100, for example, by dividing one frame into three in the X-axis direction. By combining each divided region, three divided integral image data are generated.

Specifically, the image processing apparatus 20 a corresponds to the arrangement of the three lens image pickup units 100 ₁₁ , 100 ₁₂ , and 100 ₁₃ with the three element image group data supplied by the multi-lens image pickup units 100 ₁₁ , 100 ₁₂ , and 100 _13. Then, the first divided integral image data is generated by synthesizing them. The image processing device 20a, the three elements image group data supplied multi-lens imaging unit _{100 21,} 100 22, _{100 23,} corresponding to the arrangement of the multi-lens imaging unit _{100 21,} 100 22, _{100 23} By synthesizing, second divided integral image data is generated. The image processing device 20a, the three elements image group Data supplied multi-lens imaging unit _{100 31.} 100 32, _{100 33,} corresponding to the arrangement of the multi-lens imaging unit _{100 31.} 100 32, _{100 33} By synthesizing, third divided integral image data is generated. The interpolation processing executed when the image processing device 20a generates the three pieces of divided integral image data is the same as the interpolation processing in the first embodiment, and thus description thereof is omitted here.

  The image processing device 20a projects the first divided integral image data to the projection device 60-1, the second divided integral image data to the projection device 60-2, and the third divided integral image data to the projection device 60-3. To supply. The image processing device 20a is realized by an arithmetic processing device equipped with a CPU, for example, a computer device.

  Note that the image processing device 20a is not limited to three, and may generate a plurality of divided integral image data. Further, the direction in which one frame is divided is not limited to the X-axis direction, and may be the Y-axis direction. Further, the area of each divided region may or may not be the same.

  The photographing display system 1a includes a display device. The display device includes projection devices 60-1, 60-2, 60-3, a diffusion plate 70, and a display lens array 80. This display apparatus takes in the three divided integral image data generated by the photographing apparatus according to the present embodiment, and displays the three divided integral image data by integral photography.

  The projection devices 60-1, 60-2, and 60-3 respectively include first divided integral image data, second divided integral image data, and third divided integral image data supplied by the image processing device 20a. Capture each one. The projection devices 60-1, 60-2, and 60-3 convert the first divided integral image data, the second divided integral image data, and the third divided integral image data into the first image light and the second divided integral image data. The image light is converted into the third image light, and the image light is projected toward the diffusion plate 70. Each projection device 60 is arranged so that the image light projected on the diffusion plate 70 does not overlap. Each projection device 60 is realized by, for example, a projector device.

  The diffusion plate 70 transmits the first image light, the second image light, and the third image light projected by the projection devices 60-1, 60-2, and 60-3 on one surface (on the projection device 60 side). Surface) and diffused from the regions D1, D2, D3 to reach the other surface. As a result, an integral image for one frame is displayed on the other surface of the diffusion plate 70. The aerial image corresponding to the subject S is generated by passing the luminous flux from the integral image displayed on the other surface of the diffusion plate 70 through the display lens array 80.

  In the photographing display system 1a, the positional relationship between the plurality of element lenses and pixels included in the nine imaging lens arrays 100 is equal to the positional relationship between the plurality of element lenses included in the display lens array 80 and the pixels.

  With this configuration, according to the imaging display system 1a in the second embodiment, the imaging apparatus can generate integral image data for obtaining a high-definition spatial image by integral photography. In addition, the display device can obtain an aerial image having a resolution extremely higher than the above-described resolution by using the projection device 60 having a predetermined resolution.

[Third Embodiment]
The multi-lens imaging array 10 in the first embodiment and the second embodiment is provided with a predetermined gap between the multi-lens imaging units 100, for example, nine multi-lens imaging units 100 in the X-axis direction and the Y-axis direction, respectively. In this example, three pieces are arranged. The third embodiment of the present invention is an example in which one imaging lens array is used in place of the nine imaging lens arrays 101. That is, the present embodiment is an example in which a multi-lens imaging array configured to include one imaging lens array and nine imaging elements 102 is used in an imaging apparatus.

FIG. 7 is a side view of the multi-lens imaging array according to this embodiment when viewed in the X-axis direction. In the figure, among the nine image pickup elements 102 included in the multi-lens image pickup array 10 a, the image pickup elements 102 ₁₁ , 102 ₁₂ , 102 ₁₃ are fixed to the substrate unit 103, and the image pickup elements 102 ₁₁ , 102 ₁₂ , 102 ₁₃ A state in which the imaging lens array 104 is provided on the imaging surface is shown.

  The imaging lens array 104 is an element lens group configured by regularly arranging a plurality of element lenses in a two-dimensional manner so that their optical axes are parallel to each other. When the imaging lens array 104 is viewed from the Z-axis direction, the plurality of element lenses are two-dimensionally arranged in a square lattice pattern. Each element lens of the imaging lens array 104 is realized by, for example, a gradient index lens (GRIN) lens used in the first embodiment.

  The multi-lens imaging array 10 in the first embodiment and the second embodiment includes nine multi-lens imaging units 100 provided with a gap that is a positive integer multiple of the pixel pitch between imaging surfaces, and an imaging lens array. A gap having a size that is a positive integer multiple of the lens pitch is provided between 101. In contrast, the multi-lens imaging array 10a according to the present embodiment uses a single imaging lens array 104, and thus has a simple structure compared to the multi-lens imaging array 10. In general, it is easier to manufacture and increase the area of the imaging lens array than to increase the size of the imaging device, and at a lower cost. Therefore, the imaging device according to the present embodiment is simple in structure and realized at a lower cost than the imaging devices in the first embodiment and the second embodiment.

[Fourth Embodiment]
FIG. 8 is a configuration diagram showing a schematic configuration of an imaging display system to which the display device according to the fourth embodiment of the present invention is applied. As shown in the figure, the photographing display system 2 includes an imaging lens array 510, a diffusion plate 520, a photographing device 530, an image processing device (image processing unit) 30, and a multi-lens display array 40. Composed. In the photographic display system 2, the display device includes an image processing device 30 and a multi-lens display array 40.

  FIG. 8 shows the subject S photographed by the photographing apparatus 530. In the same figure, three axes (X axis, Y axis, and Z axis) orthogonal to each other are shown. The X axis and the Y axis are axes parallel to the horizontal direction and the vertical direction of the display surfaces of the plurality of display elements included in the multi-lens display array 40, respectively. The Z axis is an axis that intersects the display surface of each display element of the multi-lens display array 40 perpendicularly. That is, the Z axis is an axis parallel to the main optical axis of the display device.

  The imaging lens array 510 is an element lens group configured by two-dimensionally arranging a plurality of element lenses regularly (for example, in a square lattice pattern) so that the optical axes are parallel to each other. Each element lens is, for example, a convex lens.

  The diffuser plate 520 is a flat plate having both light transmittance and light diffusibility. The diffusion plate 520 is provided at a position where one surface thereof coincides with the focal plane of each element lens of the imaging lens array 510. The diffusion plate 520 transmits the light beam from the imaging lens array 510 while diffusing from one surface (the surface on the imaging lens array 510 side) and reaches the other surface. Thereby, an element image group for one frame is displayed on the other surface of the diffusion plate 520.

  The imaging device 530 captures one frame of the element image group exposed on the other surface of the diffusion plate 520 to generate integral image data (first element image group data). The image is supplied to the image processing apparatus 30.

  The imaging lens array 510, the diffusion plate 520, and the imaging device 530 constitute an imaging system using integral photography.

  The multi-lens display array 40 includes, for example, nine multi-lens display units 400. As will be described in detail later, each multi-lens display unit 400 includes a display element and a display lens array (lens array). As shown in FIG. 8, the multi-lens display array 40 includes a multi-lens display so that the display surfaces of the display elements in each of the nine multi-lens display units 400 are parallel to a plane including the X axis and the Y axis (XY plane). A predetermined gap is provided between the display units 400, and nine multi-lens display units 400 are arranged in three in each of the X-axis direction and the Y-axis direction.

  The multi-lens display array 40 is not limited to nine, and may include a plurality of multi-lens display units 400. The multi-lens display array 40 is provided with a predetermined gap between the multi-lens display units 400, and the plurality of multi-lens display units 400 are arranged in at least one of the X-axis direction and the Y-axis direction. It may be configured.

  The multi-lens display array 40 operates nine multi-lens display units 400 simultaneously. The multi-lens display array 40 takes in the element image group data set supplied from the image processing apparatus 30 and displays the nine element image group data included in the element image group data set on each of the nine display elements for display. The luminous fluxes from the nine element image groups thus formed are each radiated to space through nine display lens arrays.

  The image processing device 30 takes in the integral image data supplied from the photographing device 530. The image processing apparatus 30 divides the integral image data in accordance with the arrangement of nine (3 × 3) multi-lens display units 400, so that the area for one frame is divided into nine areas. Element image group data is generated. Specifically, the image processing device 30 divides the integral image data by, for example, deleting the pixel value of the pixel corresponding to the gap between the display surfaces from the integral image data. Then, the image processing apparatus 30 supplies the nine element image group data to the multi-lens display array 40 as an element image group data set for one frame. Details of processing (division processing) in which the image processing apparatus 30 divides the integral image data to obtain an element image group data set will be described later. The image processing device 30 is realized by an arithmetic processing device equipped with a CPU, for example, a computer device.

  In the photographic display system 2, the positional relationship between the plurality of element lenses and pixels included in the imaging lens array 510 is equal to the positional relationship between the plurality of element lenses and pixels included in the nine display lens arrays 402.

FIG. 9 is a side view of the multi-lens display array 40 shown in FIG. 8 when viewed in the X-axis direction. FIG. 9 shows a state where the multilens display units 400 ₁₁ , 400 ₁₂ , and 400 ₁₃ are fixed to the substrate unit 403 among the nine multilens display units 400 included in the multilens display array 40. 8 and 9 together, nine multi-lens display units 400 ₁₁ , 400 ₁₂ , 400 ₁₃ , 400 ₂₁ , 400 ₂₂ , 400 ₂₃ , 400 ₃₁ , 400 ₃₂ , 400 ₃₃ are planes of the substrate unit 403. It is fixed to.

In FIG. 9, the multi-lens display unit 400 ₁₁ includes a display element 401 ₁₁ and a display lens array 402 ₁₁ . Similarly, the multi-lens display unit 400 ₁₂ includes a display element 401 ₁₂ and a display lens array 402 ₁₂ . Further, the multi-lens display unit 400 ₁₃ includes a display element 401 ₁₃ and a display lens array 402 ₁₃ . Although not shown in the figure, the multi-lens display units 400 ₂₁ , 400 ₂₂ , 400 ₂₃ , 400 ₃₁ , 400 ₃₂ , and 400 ₃₃ also have display lens arrays and display elements in the same manner as in the figure. Prepare.

  The display element 401 displays the element image group on the display surface based on the captured element image group data. The display element 401 is realized by a liquid crystal display device, for example.

  The display lens array 402 is an element lens group configured by regularly arranging a plurality of element lenses in a two-dimensional manner so that the optical axes are parallel to each other. When the display lens array 402 is viewed from the Z-axis direction, the plurality of element lenses are two-dimensionally arranged in a square lattice shape. Each element lens of the display lens array 402 is realized by, for example, a gradient index lens (GRIN) lens used in the first embodiment.

  In the multi-lens display unit 400, the display lens array 402 is provided such that the focal plane of each element lens coincides with the display surface of the display element 401. For example, when a gradient index lens satisfying the above-described expression (2) is used as an element lens of the display lens array 402, the focal plane of the gradient index lens is defined as a space in the gradient index lens. It can be made to correspond to the end surface opposite to the image side. Therefore, in this case, as shown in FIG. 9, the multi-lens display unit 400 can be configured by bringing the end surface of the display lens array 402 opposite to the aerial image side into contact with the display surface of the display element 401. By using the multi-lens display unit 400 configured as described above, a display device that does not include a projection lens or a diffusion plate can be realized. Therefore, according to the display device, image quality deterioration can be suppressed by the amount not including the projection lens and the diffusing plate, and miniaturization can be realized.

FIG. 10 is a diagram schematically showing the gap between display surfaces in the multi-lens display unit 400 when the multi-lens display array 40 shown in FIG. 8 is viewed from the aerial image side in the direction opposite to the Z axis (XY plane). Figure). This figure shows four multilens display units 400 ₁₁ , 400 ₂₁ , 400 ₁₂ , and 400 ₂₂ that are adjacent to each other in the multilens display array 40. In addition, the drawing shows a display lens array arranged in a square lattice pattern in the XY plane and pixels on the display surface in each multi-lens display unit 400.

  In FIG. 10, the pixel density with respect to the element lens is low, that is, the pixel is large with respect to the element lens. However, this is for convenience of explanation, and in fact, the pixel density with respect to the element lens is higher. . In addition, the display lens array in the figure is configured by arranging element lenses without gaps in the X-axis direction and the Y-axis direction. In this case, the lens pitch in each of the X-axis direction and the Y-axis direction is equal to the diameter of the element lens. equal. Further, the display element in the figure has pixels arranged in a grid without gaps, and in this case, the pixel pitch in each of the X-axis direction and the Y-axis direction is equal to the pixel width.

Fig As shown in 10, in the XY plane, the multi-lens display unit _{400 11} display surface and between the display surface of the multi-lens display unit _{400 21,} and the multi-lens display unit _{400 12} display surface and a multi-lens display unit 400 of the A gap in the X-axis direction is provided between the ₂₂ display surfaces. The gap size Q _X in the X-axis direction is a positive integer multiple of the pixel pitch in the X-axis direction on the display surface. This figure shows that the gap size Q _X in the X-axis direction is 8 times the pixel pitch in the X-axis direction on the display surface.

Similarly, in the XY plane, the display surface of the multi-lens display unit ₄₀₀ between the display surface and the display surface of the multi-lens display unit _{400 12 11,} and the multi-lens display unit _{400 21} of the display surface and the multi-lens display unit _{400 22} Is provided with a gap in the Y-axis direction. The size Q _Y of the gap of the Y-axis direction is a positive integer multiple of the Y-axis direction of the pixel pitch in the display surface. The figure shows that the gap size Q _Y of the Y-axis direction is 8 times the Y-axis direction of the pixel pitch in the display surface.

Note that the magnitude Q _Y of the X-axis direction of the gap size Q _X and Y-axis direction of the gap, may not be equal may be equal.

FIG. 11 is a diagram schematically showing a gap between the display lens arrays in the multi-lens display unit 400 when the multi-lens display array 40 shown in FIG. 8 is viewed from the aerial image side in the direction opposite to the Z axis. XY plan view). As shown in FIG. 11, in the XY plane, between the display lens array of the display lens array and the multi-lens display unit _{400 21} multi-lens display unit _{400 11,} and the multi-lens display unit _{400 12} view lens array and between the display lens array of multi-lens display unit 400 _22, a gap in the X-axis direction is provided. The size M _X of the gap in the X-axis direction is a positive integer multiple of the lens pitch of the display lens array. This figure shows that the size M _X of the gap in the X-axis direction is twice the lens pitch of the display lens array.

Similarly, in the XY plane, between the display lens array of multi-lens display unit 400 ₁₁ view lens array and the multi-lens display unit 400 _12, and the multi-lens display unit 400 ₂₁ view lens array and the multi-lens display between the display lens array parts 400 _22, the gap in the Y-axis direction is provided. The size M _Y of the gap in the Y-axis direction is a positive integer multiple of the lens pitch of the display lens array. This figure shows that the size M _Y of the gap in the Y-axis direction is twice the lens pitch of the display lens array.

Note that the magnitude M _Y in the X-axis direction of the gap size M _X and Y-axis direction gap may not be equal may be equal.

12 is a diagram showing both the gap between the display surfaces shown in FIG. 10 and the gap between the display lens arrays shown in FIG. As shown in FIG. 12, in the four multi-lens display units 400 ₁₁ , 400 ₂₁ , 400 ₁₂ , and 400 ₂₂ arranged adjacent to each other in the XY plane, gaps between display surfaces in the X-axis direction and display lenses The gap between arrays is not the same. Similarly, the gap between the display surfaces in the Y-axis direction and the gap between the display lens arrays are not the same. According to FIG. 9 and FIG. 12, the nine multi-lens display units 400 are such that the size of the gap between the display surfaces of the display elements 401 between the multi-lens display units 400 is a positive integer multiple of the pixel pitch on the display surface. In addition, the gap between the display lens arrays 402 is fixed to the plane of the substrate portion 403 such that the size of the gap satisfies a positive integer multiple of the lens pitch.

  Here, integral image data division processing by the image processing apparatus 30 will be described. The image processing apparatus 30 takes in the integral image data from the photographing apparatus 530 and stores the integral image data in a storage unit built in the apparatus. This storage unit is a frame memory capable of storing image data for at least one frame. Then, the image processing apparatus 30 removes the pixels corresponding to the gap (eight times the pixel pitch) between the display surfaces of the multi-lens display unit 400 shown in FIG. To get.

  As described above, the display device according to the fourth embodiment includes a plurality of display elements 401 having a gap between display surfaces and arranged in at least one of the X-axis direction and the Y-axis direction, A plurality of display lens arrays 402 provided with gaps are provided corresponding to the display surfaces of the plurality of display elements 401, respectively.

  With this configuration, it is possible to realize a display device having an extremely higher resolution than the above-described resolution using the multi-lens display unit 400 having a predetermined resolution.

Here, the size of the gap between the display surfaces is a positive integer multiple of the pixel pitch of the display element 401, and the size of the gap between the display lens arrays 402 is a positive integer of the lens pitch of the display lens array 402. Is double.
The display device according to the fourth embodiment further includes an image processing device 30 that generates a plurality of elemental image group data by dividing integral image data for one frame in correspondence with the arrangement of the display elements 401. Prepare.

  As described above, the display device according to the fourth embodiment of the present invention can obtain a high-definition aerial image by integral photography.

[Fifth Embodiment]
The image processing apparatus 30 in the fourth embodiment is an example in which the integral image data is taken from the photographing apparatus 530 and the elemental image group data set is generated by dividing the integral image data. In the fifth embodiment of the present invention, an image processing apparatus that captures a plurality of divided integral image data from a plurality of imaging apparatuses having different imaging areas divides each of the plurality of divided integral image data, thereby generating an element image group. It is an example which produces | generates a data set.

  FIG. 13 is a configuration diagram showing a schematic configuration of an imaging display system to which the display device according to the fifth embodiment of the present invention is applied. As shown in the figure, the imaging display system 2a includes an imaging lens array 510, a diffusion plate 520, three imaging devices 530 (530-1, 530-2, 530-3), and an image processing device ( An image processing unit) 30a and a multi-lens display array 40 are included. In the photographic display system 2a, the display device includes an image processing device 30a and a multi-lens display array 40.

  The photographing devices 530-1, 530-2, and 530-3 photograph the divided regions E1, E2, and E3 obtained by dividing the integral image for one frame displayed on the diffusion plate 520 into three in the X-axis direction. The three divided integral image data are generated, and the three divided integral image data are supplied to the image processing device 30a.

  The imaging lens array 510, the diffusion plate 520, and the imaging devices 530-1, 530-2, and 530-3 constitute an imaging system using integral photography.

The image processing device 30a takes in the three divided integral image data supplied by the photographing devices 530-1, 530-2, and 530-3. The image processing device 30a divides the three divided integral image data according to the arrangement of nine (3 × 3) multi-lens display units 400, thereby dividing the region for one frame into nine. Nine element image group data are generated. Specifically, the image processing device 30a takes in the first divided integral image data supplied by the photographing device 530-1, and uses the first divided integral image data as multilens display units 400 ₁₁ , 400 ₁₂ ,. to correspond to 400 ₁₃ the arrangement of generating the three elements image group data by dividing it. Further, the image processing device 30a takes in the second divided integral image data supplied from the photographing device 530-2, and uses the second integral image data of the multi-lens display units 400 ₂₁ , 400 ₂₂ , 400 ₂₃ . Three element image group data are generated by dividing in accordance with the arrangement. In addition, the image processing device 30a takes in the third integral image data supplied from the imaging device 530-3, and uses the third divided integral image data of the multi-lens display units 400 ₃₁ , 400 ₃₂ , and 400 ₃₃ . Three element image group data are generated by dividing in accordance with the arrangement. Since the division processing executed when the image processing device 30a generates the three-system element image group data is the same as the division processing in the fourth embodiment, description thereof is omitted here.

  The image processing device 30a is realized by an arithmetic processing device equipped with a CPU, for example, a computer device.

  Note that the number of the imaging devices 530 is not limited to three and may be plural. Further, the direction in which one frame is divided is not limited to the X-axis direction, and may be the Y-axis direction. Further, the area of each divided region may or may not be the same. In accordance with this, the image processing device 30a is not limited to three, and may generate a plurality of divided integral image data to generate an element image group data set.

  In the photographic display system 2a, the positional relationship between the plurality of element lenses and pixels included in the imaging lens array 510 is equal to the positional relationship between the plurality of element lenses and pixels included in the nine display lens arrays 402.

[Sixth Embodiment]
The multi-lens display array 40 in the fourth embodiment and the fifth embodiment is provided with a predetermined gap between the multi-lens display units 400, for example, nine multi-lens display units 400 in the X-axis direction and the Y-axis direction, respectively. In this example, three pieces are arranged. The sixth embodiment of the present invention is an example in which one display lens array is used in place of the nine display lens arrays 402. That is, the present embodiment is an example in which a multi-lens display array configured to include one display lens array and nine display elements 401 is used for a display device.

FIG. 14 is a side view of the multi-lens display array according to this embodiment when viewed in the X-axis direction. In the figure, among the nine display elements 401 included in the multi-lens display array 40a, the display elements 401 ₁₁ , 401 ₁₂ , and 401 ₁₃ are fixed to the substrate unit 403, and the display elements 401 ₁₁ , 401 ₁₂ , and 401 ₁₃ are displayed. A state in which a display lens array 404 is provided on the display surface is shown.

  The display lens array 404 is an element lens group in which a plurality of element lenses are regularly arranged two-dimensionally so that their optical axes are parallel to each other. When the display lens array 404 is viewed from the Z-axis direction, the plurality of element lenses are two-dimensionally arranged in a square lattice shape. Each element lens of the display lens array 404 is realized by, for example, a gradient index lens (GRIN) lens used in the first embodiment.

  The multi-lens display array 40 in the fourth and fifth embodiments includes nine multi-lens display units 400 provided with a gap that is a positive integer multiple of the pixel pitch between display surfaces, and a display lens array. A gap having a size that is a positive integer multiple of the lens pitch is provided between 402. In contrast, the multi-lens imaging array 40a according to the present embodiment uses a single display lens array 404, and thus has a simple structure compared to the multi-lens display array 40. Usually, it is easier to manufacture and increase the area of the display lens array than to increase the size of the display element at a lower cost. Therefore, the display device according to the present embodiment has a simple structure and is realized at a lower cost than the display devices according to the fourth and fifth embodiments.

[Seventh Embodiment]
FIG. 15 is a configuration diagram showing a schematic configuration of an imaging display system to which an imaging apparatus and a display apparatus according to the seventh embodiment of the present invention are applied. As shown in the figure, the photographing display system 3 includes a multi-lens imaging array 10, an image processing device (image processing unit) 90, and a multi-lens display array 40b. In the imaging display system 3, the imaging device includes a multi-lens imaging array 10. The display device includes a multi-lens display array 40b.

  The multi-lens display array 40b includes, for example, nine multi-lens display units 410. As will be described in detail later, each multi-lens display unit 410 includes a display element and a display lens array (lens array). As shown in FIG. 15, the multi-lens display array 40b includes a multi-lens display so that the display surfaces of the display elements in each of the nine multi-lens display units 410 are parallel to a plane including the X axis and the Y axis (XY plane). A predetermined gap is provided between the display units 410, and nine multi-lens display units 410 are arranged in three in each of the X-axis direction and the Y-axis direction.

  The multi-lens display array 40b operates nine multi-lens display units 410 simultaneously. The multi-lens display array 40b takes in the element image group data set supplied from the image processing apparatus 90, displays the nine element image group data included in the element image group data set on each of the nine display elements, and displays them. The luminous fluxes from the nine element image groups thus formed are each radiated to space through nine display lens arrays.

The image processing device 90 takes in the element image group data set supplied by the multi-lens imaging array 10. The image processing apparatus 90 stores the nine element image group data included in the element image group data set in association with the arrangement of nine (3 × 3) multi-lens imaging units 100 and is incorporated in the apparatus itself. Remember me. The storage unit is a frame memory capable of storing image data for at least one frame imaged by the multi-lens imaging array 10, for example. Further, the image processing apparatus 90 reads out the nine element image group data stored in the storage unit, and arranges nine (3 × 3) multi-lens display units 400 for the nine element image group data. Are supplied to the multi-lens display array 40 in association with each other. Specifically, the image processing apparatus 90 uses the multi-lens image capturing units 100 ₁₁ , 100 ₂₁ , 100 ₃₁ , 100 ₁₂ , 100 ₂₂ , 100 ₂₂ , 100 ₃₂ , 100 ₁₃ , 100 ₂₃ , 100 ₃₃ as multi-element image group data. The lens display units 410 ₁₁ , 410 ₂₁ , 410 ₃₁ , 410 ₁₂ , 410 ₂₂ , 410 ₃₂ , 410 ₁₃ , 410 ₂₃ , 410 ₃₃ are supplied. The image processing device 90 is realized by an arithmetic processing device equipped with a CPU, for example, a computer device.

FIG. 16 is a side view of the multi-lens display array 40b shown in FIG. 15 when viewed in the X-axis direction. FIG. 16 shows a state in which the multi-lens display units 410 ₁₁ , 410 ₁₂ , and 410 ₁₃ are fixed to the substrate unit 403 among the nine multi-lens display units 410 included in the multi-lens display array 40b. Referring to FIGS. 15 and 16 together, the nine multi-lens display portions 410 ₁₁ , 410 ₁₂ , 410 ₁₃ , 410 ₂₁ , 410 ₂₂ , 410 ₂₃ , 410 ₃₁ , 410 ₃₂ , 410 ₃₃ are planes of the substrate portion 403. It is fixed to.

In FIG. 16, the multi-lens display unit 410 ₁₁ includes a display element 401 ₁₁ and a display lens array 412 ₁₁ . Similarly, the multi-lens display unit 410 ₁₂ includes a display element 401 ₁₂ and a display lens array 412 ₁₂ . The multi-lens display unit 410 ₁₃ includes a display element 401 ₁₃ and a display lens array 412 ₁₃ . Although not shown in the figure, the multi-lens display units 410 ₂₁ , 410 ₂₂ , 410 ₂₃ , 410 ₃₁ , 410 ₃₂ , and 410 ₃₃ are also provided with a display lens array and display elements in the same manner as in the figure. Prepare.

  The display lens array 412 is an element lens group configured by regularly arranging a plurality of element lenses in a two-dimensional manner so that the optical axes are parallel to each other. When the display lens array 402 is viewed from the Z-axis direction, the plurality of element lenses are two-dimensionally arranged in a square lattice shape. Each element lens of the display lens array 412 is realized by, for example, a convex lens. The display lens array 412 is provided so that the focal plane of each element lens coincides with the display surface of the display element 401.

  In the photographic display system 3, the positional relationship between the plurality of element lenses and pixels included in the nine imaging lens arrays 100 is equal to the positional relationship between the plurality of element lenses included in the nine display lens arrays 412. .

  By using the multi-lens display unit 410 configured as described above, a display device that does not include a projection lens or a diffusion plate can be realized. Therefore, according to the display device, image quality deterioration can be suppressed by the amount not including the projection lens and the diffusing plate, and miniaturization can be realized.

[Other embodiments]
In the first to seventh embodiments described above, the imaging lens arrays 101, 104, 510 and the display lens arrays 80, 402, 404, 412 are configured by arranging element lenses in a square lattice pattern. did. The arrangement pattern of the element lenses in these lens arrays is not limited to a square lattice arrangement, and may be a delta arrangement, for example. When the element lenses are arranged in a delta shape without gaps, assuming that the diameter of the element lens is D, for example, the lens pitch in the X-axis direction is D for each row, and D / 2 between adjacent rows. The lens pitch in the Y-axis direction is (√3) × D / 2. The lens array constituted by the delta arrangement has a higher density of element lenses than the lens array constituted by the square lattice arrangement. Therefore, an imaging display system using a lens array configured by a delta arrangement can obtain high-quality interpolated element images and interpolated pixels when performing interpolation processing.

  In the first to seventh embodiments, the combination of the element lens of the imaging lens array and the element lens of the display lens array may be as follows. That is, either the imaging side or the display side is a gradient index lens, and the other is a convex lens. Alternatively, either one is a convex lens and the other is a concave lens. Alternatively, both the imaging side and the display side may be convex lenses or gradient index lenses, and image processing for inverting the depth of each element image in the image processing apparatus may be executed.

  Moreover, you may make it implement | achieve a part of function of the image processing apparatuses 20, 20a, 30, 30a, 90 in each embodiment mentioned above with a computer. In this case, a program for realizing the function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed by the computer system. Good. This computer system includes an operating system (OS) and hardware of peripheral devices. The computer-readable recording medium is a portable recording medium such as a flexible disk, a magneto-optical disk, an optical disk, or a memory card, and a storage device such as a magnetic hard disk or a solid state drive provided in the computer system. Furthermore, a computer-readable recording medium dynamically holds a program for a short time, such as a computer network such as the Internet, and a communication line when transmitting a program via a telephone line or a cellular phone network. In addition, a server that holds a program for a certain period of time, such as a volatile memory inside a computer system serving as a server device or a client in that case, may be included. Further, the above program may be for realizing a part of the above-described functions, and further, may be realized by combining the above-described functions with a program already recorded in the computer system. Good.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to that embodiment, The design of the range which does not deviate from the summary of this invention, etc. are included.

1, 1a, 2, 2a, 3 Shooting display system 10, 10a Multi-lens imaging array 20, 20a, 30, 30a Image processing device (image processing unit)
40, 40a, 40b Multi-lens display array 60, 60-1, 60-2, 60-3 Projector 70 Diffuser 80 Display lens array 100 ₁₁ , 100 ₂₁ , 100 ₃₁ , 100 ₁₂ , 100 ₂₂ , 100 ₃₂ , 100 ₁₃ , 100 ₂₃ , 100 ₃₃ Multi-lens imaging unit 101 ₁₁ , 101 ₁₂ , 10 ₁₃ Imaging lens array 102 ₁₁ , 102 ₁₂ , 102 ₁₃ Imaging element 103 Substrate unit 104 Imaging lens array 400 ₁₁ , 400 ₂₁ , 400 ₃₁ , 400 ₁₂ , 400 ₂₂ , 400 ₃₂ , 400 ₁₃ , 400 ₂₃ , 400 ₃₃ Multi-lens display unit 401 ₁₁ , 401 ₁₂ , 401 ₁₃ Display element 402 ₁₁ , 402 ₁₂ , 402 ₁₃ Display lens array 403 Substrate unit 404 For display Len Array 410 _{11, 410 21,} 410 _{31, 410 12,} 410 _22, 410 32, _{410 13,} 410 23, _{410 33} multi-lens display unit 412 _{11, 412} 12, _{412 13} display lens array 510 imaging lens array 520 spread Plate 530, 530-1, 530-2, 530-3

Claims (9)

A plurality of image sensors having a gap between the imaging surfaces and arranged in at least one of the horizontal direction and the vertical direction;
A plurality of lens arrays provided with a gap in correspondence with each of the imaging surfaces of the plurality of imaging elements;
An imaging apparatus comprising: The size of the gap between the imaging surfaces is a positive integer multiple of the pixel pitch of the imaging element,
The size of the gap between the lens arrays is a positive integer multiple of the lens pitch of the lens array.
The photographing apparatus according to claim 1, wherein: An image processing unit that generates second element image group data by combining a plurality of first element image group data generated by the plurality of image sensors in correspondence with the arrangement of the image sensors;
The imaging device according to claim 1, further comprising: The image processing unit obtains pixel values of pixels excluding pixels corresponding to the gap between the lens arrays from pixels corresponding to the gap between the imaging surfaces when combining the plurality of first element image group data. Obtained by interpolating pixels, obtaining element images corresponding to the gaps between the lens arrays by interpolating element images, and combining the plurality of first element image group data, the pixel values, and the element images And generating the second element image group data,
The photographing apparatus according to claim 3. A plurality of image sensors having a gap between the imaging surfaces and arranged in at least one of the horizontal direction and the vertical direction;
A lens array provided corresponding to the imaging surface of the plurality of imaging elements;
An imaging apparatus comprising: A plurality of display elements disposed in at least one of the horizontal direction and the vertical direction with a gap between the display surfaces;
A plurality of lens arrays provided with a gap corresponding to each of the display surfaces of the plurality of display elements;
A display device comprising: The size of the gap between the display surfaces is a positive integer multiple of the pixel pitch of the display element,
The size of the gap between the lens arrays is a positive integer multiple of the lens pitch of the lens array.
The display device according to claim 6. A plurality of second element image group data is generated by dividing the first element image group data in correspondence with the arrangement of the display elements, and the plurality of second element image group data is converted into the plurality of display elements. An image processing unit to be supplied to,
The display device according to claim 6, further comprising: A plurality of display elements disposed in at least one of the horizontal direction and the vertical direction with a gap between the display surfaces;
A lens array provided corresponding to the display surface of the plurality of display elements;
A display device comprising:

Images