JP2019213127A - Ip stereoscopic image display device and program thereof - Google Patents

Abstract

An IP stereoscopic video display apparatus is provided that allows a plurality of persons to view high-quality IP stereoscopic video substantially simultaneously.
An IP stereoscopic image display apparatus 1 includes a projector 10, a collimating lens 20, a lens array 30, a spatial light modulator 40, a control device 50, and a depth camera 60. The control device 50 includes: Viewpoint detection means 52 for detecting the number of viewers A and the viewpoint position, display position calculation means 53 for calculating the display position of the element image group E so that the viewer A is included in the viewing area, and element image group E Time-division display means 54 for time-division display on the spatial light modulator 40, and light source control means 55 for controlling the characteristics of the emitted light so that the viewer A is included in the viewing area based on the viewpoint position of the viewer A Is provided.
[Selection] Figure 3

Description

  The present invention relates to an integral photography (IP) 3D image display apparatus and a program thereof.

  The IP stereoscopic video display device is a three-dimensional video display device that does not require special glasses and enables stereoscopic viewing with the naked eye of the viewer. As shown in FIG. 11, the IP stereoscopic video display device 9 includes a direct-view display 91 such as liquid crystal or organic EL (Electro Luminescence), and a lens array 93 placed on the front surface thereof. The lens array 93 is composed of element lenses 95 arranged in a two-dimensional manner. The direct view display 91 displays an element image e corresponding to the element lens 95. Then, the element image e is projected into the space by the element lens 95, and the viewer A can view the stereoscopic image Z.

  The IP system is a system that reproduces many viewpoint videos, and therefore requires a very large number of pixels (information amount). There are three parameters that determine the quality of IP stereoscopic video: “the number of stereoscopic video pixels (number of elemental images)”, “viewing zone”, and “spatial frequency characteristics (depth reproduction range)”. These three parameters have a trade-off relationship (Non-Patent Document 1). That is, in the IP method, other parameters (the number of stereoscopic video pixels and spatial frequency characteristics) can be improved by narrowing the viewing zone. As shown in FIG. 12A, when the focal length f of the element lens 95 is shortened, the viewing zone α is widened. Therefore, as shown in FIG. Can be made longer.

  Even when the viewing zone α is narrow, IP stereoscopic video can be viewed in a wide range by sensing the viewer A in real time using the viewpoint tracking technique and controlling the direction of the viewing zone α (Non-patent Document 2). . Here, the direction control of the viewing zone α can be realized by moving the IP stereoscopic video (element image group E) in parallel (can be enlarged or reduced) in real time. As shown in FIG. 13A, when the viewer A is located on the left side when viewed from the IP stereoscopic video display device 9, when the element image group E is translated to the right side, the viewing zone α (illustrated with dots) is set. Move to the left. On the other hand, as shown in FIG. 13B, when the viewer A is located on the right side when viewed from the IP stereoscopic image display device 9, the viewing zone α moves to the right when the element image group E is translated to the left side. To do. As described above, in the conventional technology, when limited to individual viewing (viewing by only one person), the performance of the number of stereoscopic video pixels and spatial frequency characteristics can be improved, and an IP stereoscopic video in a wide viewing area can be presented. .

H. Hoshino, F. Okano, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15 (8), 2059-2065 (1998) Zhao-Long Xiong, Qiong-Hua Wang, Shu-Li Li, Huan Deng, and Chao-Chao Ji, "Partially-overlapped viewing zone based integral imaging system with super wide viewing angle," Opt. Express 22, 22268-22277 ( 2014)

However, in the above-described conventional technology, there is a problem that the number of viewers is limited to one, and a plurality of viewers cannot be handled.
Therefore, an object of the present invention is to provide an IP stereoscopic video display apparatus and a program therefor, in which a plurality of persons can view high-quality IP stereoscopic video substantially simultaneously.

  In view of the above-described problems, the IP stereoscopic video display device according to the present invention forms a viewing area for each person, and displays a group of element images in the formed viewing area in a time-division manner, thereby allowing a plurality of persons to display element images. 3D image display apparatus capable of viewing a light source, and a parallel light optical system that forms a viewing zone including at least a person's eyes by converting the light emitted from the light source and the light emitted from the light source into parallel light And a viewpoint detection means, a display position calculation means, a time-division display means, a display device, and a light source control means.

According to such an IP stereoscopic video display device, the viewpoint detection means detects the number of persons and the viewpoint position for each person.
The display position calculation means calculates the display position of the element image group for each person detected by the viewpoint detection means based on the viewpoint position so that the person is included in the viewing zone.
The time division display means time-divides the element image group based on the number of persons detected by the viewpoint detection means.
The light source control means controls the characteristics of the emitted light for each person detected by the viewpoint detection means based on the viewpoint position so that the person is included in the viewing zone.
The display device displays the element image group time-divided by the time-division display means at the display position.

  Thus, since the IP stereoscopic video display device forms a narrow viewing zone for each person using the viewpoint tracking technology, crosstalk does not occur, and the performance of the number of stereoscopic video pixels and spatial frequency characteristics can be improved. it can. Furthermore, since the IP stereoscopic video display device displays the element image group in a time-sharing manner in a narrow viewing area for each person, it can cope with a plurality of persons.

In the present invention, since time-division display is performed, strictly speaking, each person does not view the group of element images at the same time.
In addition, although the term “viewing” is used, in the IP stereoscopic video display device, whether or not there is sound is irrelevant.

  The above-described IP stereoscopic video display device can also be realized by a program that causes a general computer to operate cooperatively as the above-described viewpoint detection means, display position calculation means, time-division display means, and light source control means.

According to the present invention, the following excellent effects can be obtained.
The IP stereoscopic video display apparatus according to the present invention allows a plurality of persons to view high-quality IP stereoscopic video substantially simultaneously by the viewpoint tracking technique and time-division display.

It is explanatory drawing explaining the crosstalk in a prior art. (A) And (b) is explanatory drawing explaining the direction control of the visual field in embodiment. It is a block diagram which shows the structure of the IP stereoscopic video display apparatus which concerns on embodiment. It is explanatory drawing explaining the position and light distribution angle of the point light source in embodiment. It is explanatory drawing explaining the pattern for point light source group formation in embodiment. In embodiment, (a)-(d) is explanatory drawing explaining the relationship between the pattern for point light source formation, the formation position and light distribution angle of a point light source, and the light distribution characteristic of a point light source. In embodiment, it is explanatory drawing explaining the control which narrows the viewing zone angle of a stereo image, (a) is a pattern for point light source group formation, (b) is an IP stereo image display apparatus. In embodiment, it is explanatory drawing explaining the control which widens the viewing zone angle of a stereo image, (a) is a pattern for point light source group formation, (b) is an IP stereo image display apparatus. In embodiment, it is explanatory drawing explaining the time division display of an element image group. 4 is a flowchart illustrating an operation of the IP stereoscopic video display device of FIG. 3. It is explanatory drawing explaining the conventional IP stereoscopic video display apparatus. It is explanatory drawing explaining the relationship between a viewing zone and a focal distance as a prior art, (a) represents that a viewing zone will become wide if focal length is shortened, (b) will narrow a viewing zone if focal length is lengthened. Represents that. (A) And (b) is explanatory drawing explaining the direction control of the visual field in a prior art.

(First embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In addition, the same code | symbol was attached | subjected to the same means and the same member, and description was abbreviate | omitted.

[Principle of IP stereoscopic video display device]
Hereinafter, the principle of the IP stereoscopic video display device according to the embodiment will be described.
First, crosstalk that occurs in a conventional IP stereoscopic video display device will be described. As shown in FIG. 1, the crosstalk is a phenomenon in which when the element image e ₁ is viewed through the lens array 93, the adjacent element image e ₂ is visible. This crosstalk appears as a sidelobe viewing zone β outside the central viewing zone α viewed by the viewer (person) A, and hinders the improvement of the quality of the IP stereoscopic video (element image group E).

  Therefore, the pitch and focal length of the element lens 95 are designed so that a cross-talk does not occur and a narrow viewing zone α that can include both eyes of one person can be formed. As described above, each parameter (view zone, number of stereoscopic video pixels, spatial frequency characteristics) of the IP system has a trade-off relationship. For this reason, the performance of the number of stereoscopic video pixels and the spatial frequency characteristics can be improved by narrowing the viewing zone α.

  Next, a viewpoint tracking technique for tracking the viewing zone α to the viewpoint position of the viewer A will be described. For example, a viewpoint camera (not shown) is used to detect the viewpoint position of the viewer A, the element image e is translated, and the direction of the viewing zone α is controlled so that the viewer A always enters the viewing zone α. You can do it.

  Therefore, as shown in FIG. 2, an IP stereoscopic video display apparatus 1 that can control the direction of the viewing zone α without causing crosstalk is considered. In the IP stereoscopic image display device 1, the projector 10 that projects white light (emitted light) is used as the backlight of the spatial light modulator 40. In the IP stereoscopic video display device 1, the collimator lens 20 is disposed on the front surface of the projector 10 to collimate the light emitted from the projector 10. In the IP stereoscopic image display device 1, the lens array 30 is disposed after the collimating lens 20, and the light collimated by the collimating lens 20 is condensed on the focal point (point light source p) of the element lens 31. The lens array 30 forms a point light source group P in which point light sources p are arranged. Further, in the IP stereoscopic image display device 1, the spatial light modulator 40 is disposed at the subsequent stage of the lens array 30. At this time, if the light spread from the condensing point has the same size as the element image e on the spatial light modulator 40, the IP stereoscopic video can be reproduced.

  Here, by adjusting the position of the light incident on the lens array 30 using the point light source group forming pattern 70 (FIG. 5), the direction of the light in which the IP stereoscopic image is formed (the direction of the viewing zone α). Can be controlled. The point light source group forming pattern 70 is a mask image for controlling the light distribution characteristics of the point light source p, and details thereof will be described later. As shown in FIG. 2A, when the viewer A is located on the left side when viewed from the IP stereoscopic video display device 1, light may be incident on the right side of the lens array 30. On the other hand, as shown in FIG. 2B, when the viewer A is positioned on the right side when viewed from the IP stereoscopic video display device 1, light may be incident on the left side of the lens array 30. Since the collimating lens 20 emits parallel light, no crosstalk occurs, and an IP stereoscopic image is not seen outside the viewing zone α. As described above, the IP stereoscopic video display device 1 in FIG. 2 can display the IP stereoscopic video only for the specific viewer A.

  Furthermore, it considers corresponding to several viewers A. When there are a plurality of viewers A, IP stereoscopic video is time-divisionally displayed by the number of viewers, and the direction of the viewing zone α may be controlled in units of frames. Thereby, the frame rate of the IP stereoscopic video viewed by each viewer A becomes “1 / number of people” of the frame rate of the spatial light modulator 40, and a plurality of viewers A can view the IP stereoscopic video almost simultaneously. At this time, it is preferable to use the spatial light modulator 40 having a high frame rate so that the viewer A can view smoother IP stereoscopic video.

[Configuration of IP stereoscopic video display device]
With reference to FIG. 3, the configuration of the IP stereoscopic video display device 1 will be described.
The IP stereoscopic video display device 1 forms a narrow viewing zone α (FIG. 1) for each viewer A, and displays the element image group E in the formed viewing zone α in a time-sharing manner, so that a plurality of viewers A can be substantially omitted. At the same time, the element image group E can be viewed. As shown in FIG. 3, the IP stereoscopic image display apparatus 1 includes a projector (light source) 10, a collimating lens (parallel light optical system) 20, a lens array (parallel light optical system) 30, and a spatial light modulator (display). Device) 40, a control device 50, and a depth camera (depth estimation camera) 60.

  The projector 10 is a projection device that emits outgoing light. For example, the projector 10 projects the point light source group forming pattern 70 in accordance with control from the control device 50 described later. In the present embodiment, it is assumed that the projector 10 is disposed at the focal length F of the collimating lens 20 and the position calibration is accurately performed. Note that the size of the point light source p is about one pixel of the projection image of the projector 10.

The collimating lens 20 converts light emitted from the projector 10 into parallel light, and forms a narrow viewing zone α including at least both eyes of the viewer A. Specifically, the collimating lens 20 emits the point light source group forming pattern 70 projected by the projector 10 as parallel light. For example, examples of the collimating lens 20 include a general biconvex lens and a Fresnel lens.
The distance between the collimating lens 20 and the lens array 30 is arbitrary, and is preferably shortened in order to reduce the size of the IP stereoscopic video display device 1.

  The lens array 30 forms the point light source group P by condensing the parallel light from the collimating lens 20 at the position of the focal length f of each element lens 31. In the lens array 30, a plurality of element lenses 31 are two-dimensionally arranged in a vertical and horizontal direction or in a barrel shape. For example, the element lens 31 may be a general convex lens or concave lens having a circular shape when viewed from the front.

  The spatial light modulator 40 displays the element image group E input from the control device 50 in a time-sharing manner. For example, the spatial light modulator 40 can be a liquid crystal panel. Here, the spatial light modulator 40 is preferably disposed at a position where the backlight of the point light source group P irradiates the entire spatial light modulator 40 in order to effectively use the pixels of the spatial light modulator 40. The element image group E includes element images e corresponding to the element lenses 31.

  The control device 50 controls the projector 10 and the spatial light modulator 40 so as to form a narrow viewing zone α for each viewer A and display the element image group E in a time-sharing manner in the formed viewing zone α. . Here, the control device 50 detects the number of viewers A and the viewpoint position using the depth camera 60, and controls the projector 10 and the spatial light modulator 40 based on the detected number of viewers A and the viewpoint position. Synchronous control. Details of the control device 50 will be described later.

  The depth camera 60 is a general stereo camera that captures a stereo image of the viewer A in order to detect the number of viewers A and the viewpoint position for each viewer A. In the present embodiment, the depth camera 60 is disposed above the spatial light modulator 40 so as to face the viewer A.

  Hereinafter, after explaining the formation position and light distribution angle of the point light source, the point light source group formation pattern 70, and the control of the viewing zone α, the configuration of the control device 50 will be explained.

<Point light source formation position and light distribution angle>
As shown in FIG. 4, the projection direction of a point light source formation pattern 71 (FIG. 5) to be described later is defined as θ. The projection direction θ is an angle formed by the optical axis (x axis) of the element lens 31 and the optical axis (illustrated by a broken line) of the point light source formation pattern 71. At this time, the formation position (p _x , p _y ) of the point light source p is represented by the following formula (1). Furthermore, the light distribution angle φ of the point light source p is expressed by the following equation (2).

y _a and y _b represent the upper and lower ends of the non-mask region 73 of the point light source forming pattern 71, respectively. On the y axis, the light beam passes between the upper end y _a and the lower end y _b of the non-mask region 73. Therefore, by changing the projection direction θ and the size of the non-masked area of the point light source forming pattern 71 (upper y _{a ~} lower y _b), the formation position of the point light source _p (p _x, p _y) and the light distribution The angle φ can be controlled.

<Pattern for point light source group formation>
The point light source group formation pattern 70 will be described with reference to FIG.
As shown in FIG. 5, the point light source group forming pattern 70 is an image in which point light source forming patterns 71 are arranged corresponding to the element lenses 31. That is, the point light source group forming patterns 70 are configured by the same number of point light source forming patterns 71 in the same arrangement as the element lenses 31.

  Here, the point light source formation pattern 71 has a white non-mask region 73 and a black mask region 75. In the point light source formation pattern 71, a rectangular non-mask area 73 is located at the center, and a mask area 75 is located around the non-mask area 73. Here, the non-mask area 73 is an area where the projector 10 projects light having the maximum luminance. The mask area 75 is an area where the projector 10 does not project light. In other words, when the point light source formation pattern 71 is projected, the light beam passes through the non-mask area 73.

In FIG. 5, only six point light source forming patterns 71 are shown and the rest are omitted to make the drawing easier to see.
5 is merely an example, and the positions and sizes of the non-mask area 73 and the mask area 75 are not particularly limited.

<Relationship between point light source formation pattern and point light source>
The relationship between the point light source formation pattern 71 and the point light source p will be described with reference to FIG.
In FIG. 6, the point light source formation pattern 71 is shown in the upper part, the formation position and the light distribution angle of the point light source p are shown in the middle part, and the light distribution characteristics of the point light source p are shown in the lower part.

  In FIG. 6A, the center of the point light source formation pattern 71 is a non-mask area 73 and the periphery is a mask area 75. The width and height of the non-mask region 73 is about half of the point light source formation pattern 71. The point light source formation pattern 71 is projected from the front of the element lens 31. In this case, the point light source p is formed on the optical axis of the element lens 31. Further, since the light beam from the projector 10 passes only through the center of the element lens 31, the light distribution angle φ of the point light source p becomes narrow.

  In FIG. 6B, the non-mask region 73 in FIG. The width and height of the non-mask region 73 is about 1/3 of the point light source formation pattern 71. In this case, since the light beam from the projector 10 passes only through a narrow range at the center of the element lens 31, the light distribution angle φ of the point light source p becomes narrower.

  In FIG. 6C, the upper side of the point light source forming pattern 71 is a non-mask area 73 and the lower side is a mask area 75. The point light source forming pattern 71 is projected from the front of the element lens 31. In this case, the point light source p is formed on the optical axis of the element lens 31. Further, the light beam from the projector 10 passes only above the element lens 31. For this reason, the light distribution characteristic of the point light source p spreads uniformly in the downward direction.

In FIG. 6D, the non-mask area 73 in FIG. 6C is set below the point light source formation pattern 71. In this case, the light beam from the projector 10 passes only below the element lens 31. For this reason, the light distribution characteristic of the point light source p spreads uniformly upward.
6C and 6D, the example in which the non-mask area 73 is provided above and below the point light source formation pattern 71 has been described. However, the non-mask area 73 is provided on the left and right sides of the point light source formation pattern 71. It is good (not shown).

<Viewing area control>
The control of the viewing zone α will be described in detail with reference to FIGS.
7 and 8, the depth camera 60, the element lens 31, the point light source formation pattern 71, and a part of the element image e are omitted for easy viewing of the drawings.

FIG. 7 is an example in which the viewing zone angle of the IP stereoscopic video is reduced to narrow the viewing zone α.
As shown in FIG. 7A, the control device 50 sets the projection interval of the point light source formation pattern 71 so that all the element lenses 31 form the point light source p (hereinafter, projection interval = '1). '). That is, the projection interval means the interval between the element lenses 31 that project the point light source formation pattern 71 including the non-mask region 73.

  The control device 50 sets the size of the non-mask area 73 based on the focal length f of the element lens 31, the installation position of the spatial light modulator 40, and the size of the element image e. As shown in FIG. 7B, the size (height) of the non-mask region 73 is a, the size (height) of the element image e is b, the diameter of the element lens 31 is D, the point light source p and the space. The distance from the optical modulator 40 is L. In this case, the size a of the non-mask region 73 is expressed by the following formula (3). Further, when the size b of the element image e is equal to the diameter D of the element lens 31, the size a of the non-mask region 73 is expressed by the following formula (4).

  Note that the size of the point light source formation pattern 71 is equal to or smaller than the diameter D of the element lens 31. Therefore, in the formulas (3) and (4), it is necessary to satisfy a ≦ D and b ≦ D · L / f.

  In FIG. 7, the control device 50 sets the size a of the non-mask area 73 to be D · f / L. Further, the control device 50 sets the position of the non-mask area 73 at the center of the point light source formation pattern 71.

  Then, the projector 10 projects the point light source formation pattern 71 of FIG. 7A onto all the element lenses 31. Therefore, in the IP stereoscopic image display device 1, as shown in FIG. 7B, the light distribution angle of the point light sources p is narrowed, and the same number of point light sources p as the element lenses 31 are formed. As a result, in the IP stereoscopic video display device 1, the viewing zone α is narrowed (in contrast, the resolution of the stereoscopic video is increased).

  The spatial light modulator 40 displays the element image e according to the viewing zone angle of the IP stereoscopic video. The display position of the element image e can be obtained from the light distribution angle of the point light source p and the distance L between the point light source p and the spatial light modulator 40.

FIG. 8 is an example in which the viewing zone angle of the IP stereoscopic video is increased to widen the viewing zone α.
As shown in FIG. 8A, the control device 50 sets the projection interval of the point light source formation pattern 71 so that every other element lens 31 forms the point light source p (hereinafter, projection interval = '2').

  Also, since the size b of the element image e is twice the diameter D of the element lens 31, a = 2 · D · f / L is established by substituting b = 2D into Equation (3). Therefore, the control device 50 sets the size a of the non-mask area 73 to be 2 · D · f / L.

Then, the projector 10 projects the point light source forming pattern 71 including the non-mask area 73 onto the element lens 31 at intervals of one. In other words, the projector 10 alternately projects the point light source forming pattern 71 _W including the non-mask area 73 and the point light source forming pattern 71 _B including the entire mask area 75 on each element lens 31. Therefore, in the IP stereoscopic image display device 1, as shown in FIG. 8B, the light distribution angle of the point light source p is widened, and about half of the point light sources p are formed of the element lens 31. Thereby, in the IP stereoscopic video display device 1, the viewing zone α is widened (while the resolution of the stereoscopic video is low).

  The projection interval is not limited to ‘2’, and can be set to an arbitrary length N. In this case, the size a of the non-mask region 73 is expressed by the following formula (5) (where N is an integer of 2 or more).

As described above, in the IP stereoscopic image display device 1, the position of the non-mask area 73, the size of the non-mask area 73, the projection interval of the point light source formation pattern 71 _W having the non-mask area 73, and the point light source formation By adjusting any one or more of the projection directions of the pattern 71 for use, the direction of the viewing zone α and the width of the viewing zone α can be arbitrarily controlled.

  Therefore, in the IP stereoscopic image display device 1, the point light source group formation pattern 70 is formed so that the viewing zone α is formed at the viewpoint position of the viewer A and the light spread from the point light source p has the same size as the element image e. Can be adjusted. In the IP stereoscopic video display device 1, the element image group E may be translated according to the viewpoint position of the viewer A. Thereby, one viewer A can view high-quality IP stereoscopic video.

Furthermore, as shown in FIG. 9, the IP stereoscopic video display device 1 can deal with a plurality of viewers A by displaying the IP stereoscopic video in a time-sharing manner. In FIG. 9, each frame constituting the IP stereoscopic video is time-divided into three. For example, IP stereoscopic image display apparatus 1 displays the frame # 1 of the IP stereoscopic image in the viewing area alpha ₁ in viewer _{A 1,} and displays the frame # 2 to the viewing zone alpha ₂ of the viewer _{A 2,} frame # 3 is displayed on the viewing area alpha ₃ of the viewer a ₃ (frame # 4 since also repeated in the same manner).
Note that “#” represents the number of each frame constituting the IP stereoscopic video.

[Configuration of control device]
Returning to FIG. 3, the configuration of the control device 50 will be described.
As shown in FIG. 3, the control device 50 includes an element image group storage unit 51, a viewpoint detection unit 52, a display position calculation unit 53, a time division display unit 54, a light source control unit 55, and a synchronization control unit 56. With.

  The element image group storage means 51 is a storage device such as a general HDD (Hard Disk Drive) that stores the element image group E displayed by the IP stereoscopic video display device 1. The element image group E stored in the element image group storage unit 51 is read out by the time division display unit 54. In the present embodiment, the IP stereoscopic video display device 1 displays the same elemental image group E for all viewers A. That is, all viewers A view the same IP stereoscopic video.

The viewpoint detection means 52 performs depth estimation processing on the stereo image from the depth camera 60 to detect the number of viewers A and the viewpoint position for each viewer A. For example, the viewpoint detection unit 52 detects the face area of the viewer A from the stereo image, and performs face area matching between the stereo images. Then, the viewpoint detection unit 52 determines the depth position of the viewer A from the parallax of the corresponding face area, with the number of corresponding face areas as the number of viewers A. At this time, the viewpoint detection unit 52 may detect the position of the pupil region and obtain the viewpoint position of the viewer A in order to improve accuracy.
The viewpoint detection unit 52 outputs the detected number of viewers A to the time division display unit 54, and outputs the detected viewpoint positions of each viewer A to the display position calculation unit 53 and the light source control unit 55.

For each viewer A detected by the viewpoint detection unit 52, the display position calculation unit 53 is based on the viewpoint position of the viewer A so that the viewer A is included in the viewing area α. The display position is calculated. A viewing zone α is formed on an extension line connecting each point light source p and the center position of the element image e. Therefore, the display position calculation means 53 determines the position at which the straight line connecting each point light source p and the viewpoint position of the viewer A intersects the liquid crystal panel surface of the spatial light modulator 40 as the center position of each element image e (element image group E Display position). For example, when the viewer A is located on the left side when viewed from the IP stereoscopic video display device 1, the display position calculation unit 53 sets the display position of the element image group E on the right side. On the other hand, the display position calculation unit 53 sets the display position of the element image group E to the left side when the viewer A is located on the right side when viewed from the IP stereoscopic video display device 1.
The display position calculation unit 53 outputs the obtained display position of the element image group E to the time division display unit 54.

The time division display unit 54 time-divides the element image group E based on the number of persons detected by the viewpoint detection unit 52, and the element image group E is displayed on the spatial light modulator 40 at the display position calculated by the display position calculation unit 53. Display in time division. In FIG. 9, the time division display means 54 receives “3” as the number of viewers A from the viewpoint detection means 52. In this case, the time division display means 54 reads the element image group E from the element image group storage means 51, and displays the element image group E in a time division manner at intervals of three frames. For example, viewer _{A 1} is the frame # 1, # 4, # 7, # 10, and view ... element image group E of the viewer _{A 2} frame # 2, # 5, # 8, # 11, ... of viewing element image group E, the viewer _{a 3} frames # 3, # 6, # 7, # 12, will view ... element images E of.

The light source control means 55 controls the characteristics of the emitted light for each viewer A detected by the viewpoint detection means 52 based on the viewpoint position of the viewer A so that the viewer A is included in the viewing zone α. To do. In the present embodiment, the light source control unit 55, as a characteristic of the emitted light, the position and size of the non-masked region 73, and the projection distance of the point light sources forming pattern 71 _W, the projection direction of the point light source forming pattern 71 Control one or more. For example, the light source control means 55 adjusts the point light source group forming pattern 70 so that light is incident on the right side of the lens array 30 when the viewer A is located on the left side when viewed from the IP stereoscopic video display device 1. On the other hand, the light source control means 55 adjusts the point light source group forming pattern 70 so that light is incident on the left side of the lens array 30 when the viewer A is positioned on the right side when viewed from the IP stereoscopic video display device 1.

Here, a supplementary description will be given of a method for adjusting the point light source group forming pattern 70.
Since the size b of the element image e, the focal length f of the element lens 31, and the distance L between the point light source p and the spatial light modulator 40 are known, the light source control means 55 uses the above-described equation (3) to The size a of the mask area 73 can be calculated.
Similarly to the display position calculation unit 53, the light source control unit 55 can calculate the position of the non-mask area 73 and the projection direction of the point light source formation pattern 71 because the viewpoint position of the viewer A is known.
Further, the light source control means 55 calculates the projection interval of the point light source formation pattern 71 as “1” because the viewing zone α is narrowed so that crosstalk does not occur.

  The synchronization control means 56 synchronizes the timing for outputting the point light source group forming pattern 70 to the projector 10 and the timing for outputting the element image group E to the spatial light modulator 40. That is, the synchronization control means 56 synchronizes the processing timings of the display position calculation means 53, the time division display means 54, and the light source control means 55 by known synchronization control.

[Operation of IP stereoscopic video display device]
The operation of the IP stereoscopic video display device 1 will be described with reference to FIG.
The IP stereoscopic video display device 1 captures a stereo image of the viewer A using the depth camera 60 (step S1).
The IP stereoscopic video display device 1 detects the number of viewers A and the viewpoint position of the viewers A using the stereo image taken in step S1. For example, the viewpoint detection unit 52 obtains the number of viewers A and the viewpoint position of the viewers A by face area detection processing and depth estimation processing (step S2).

  The IP stereoscopic video display device 1 dynamically changes the outputs of the spatial light modulator 40 and the projector 10 according to the viewpoint position of the viewer A. That is, the IP stereoscopic video display device 1 performs the movement of the element image group E according to the viewpoint position of the viewer A and the control of the viewing zone α. For example, the display position calculating unit 53 calculates the display position of the element image group E, and the light source control unit 55 adjusts the point light source group forming pattern 70 (step S3).

  The IP stereoscopic video display device 1 displays the element image group E in a time-sharing manner according to the number of viewers A. For example, the time-division display means 54 time-divides the element image group E based on the number of viewers A detected in step S1, and the element image group E is displayed on the spatial light modulator 40 at the display position obtained in step S3. Display (step S4).

The IP stereoscopic video display device 1 determines whether to end time-division display of IP stereoscopic video. For example, when displaying up to the last frame of the IP stereoscopic video, the control device 50 determines to end the time-division display of the IP stereoscopic video (step S5).
When the time-division display is not finished (No in step S5), the IP stereoscopic video display device 1 returns to the process of step S1.
When the time-division display ends (No in step S5), the IP stereoscopic video display device 1 ends the operation.

[Action / Effect]
As described above, the IP stereoscopic video display device 1 forms a narrow viewing zone α for each viewer A by using the viewpoint tracking technique, so that crosstalk does not occur and the number of stereoscopic video pixels and spatial frequency characteristics are reduced. Performance can be improved. Furthermore, since the IP stereoscopic video display device 1 displays the element image group E in a time-sharing manner in the narrow viewing zone α for each viewer A, it can cope with a plurality of viewers A. As described above, the IP stereoscopic video display device 1 enables a plurality of viewers A to view high-quality IP stereoscopic video substantially simultaneously by the viewpoint tracking technique and time-division display.
Furthermore, even when the number of viewers A changes, the IP stereoscopic video display device 1 can detect the change in the number of viewers A in real time and perform time-division display according to the number of viewers.

  As mentioned above, although embodiment of this invention was explained in full detail, this invention is not limited to above-described embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.

(Modification 1)
In the above-described embodiment, it has been described that the same elemental image group is displayed in a time-sharing manner for all viewers, but the present invention is not limited to this. That is, different element image groups may be displayed in a time-sharing manner for each viewer.

With reference to FIG.3 and FIG.9, the IP stereoscopic video display apparatus 1B which concerns on the modification 1 is demonstrated.
As shown in FIG. 3, the IP stereoscopic video display device 1B includes a projector 10, a collimator lens 20, a lens array 30, a spatial light modulator 40, a control device 50B, and a depth camera 60.
The control device 50B includes an element image group storage unit 51B, a viewpoint detection unit 52, a display position calculation unit 53, a time division display unit 54B, a light source control unit 55, and a synchronization control unit 56.

  The element image group storage means 51B stores an element image group E preset for each viewer A. For example, in the IP stereoscopic video display device 1B, the element image group E desired by each viewer A is stored in advance in the element image group storage means 51B.

The time division display means 54B reads the element image group E for each viewer A from the element image group storage means 51, and displays the element image group E on the spatial light modulator 40 in a time division manner. 9, the viewer _{A 1} is the frame # 1 of the element images _{E 1,} # 2, # 3, # 4, and view ... viewer _{A 2} frame # 1 of the element image group _{E 2,} # 2 , # 3, # 4, ... to view, viewer _{a 3} frame # 1 of the element image group _{E 3,} # 2, # 3, # 4, ... will view.

Thus, IP stereoscopic image display device 1B can display the viewer _A 1 different elements image group _E 1 to E ₃ on to A _3. That is, in the IP stereoscopic video display device 1B, it is possible to view the elemental image group E desired by each viewer A even though each viewer A is in the same viewing space.

(Other variations)
In the above-described embodiment, it has been described that the number of viewers is three. However, the number of viewers may be two or more.
In the above-described embodiment, the projector is the light source, and the collimating lens and the lens array are the parallel light optical system. However, the light source and the parallel light optical system are not limited to these. Further, the display device is not limited to the spatial light modulator.
In the embodiment described above, the viewpoint position of the viewer is detected by the depth camera, but the present invention is not limited to this. For example, the viewer's viewpoint position may be detected using an infrared camera or a monocular camera.

In the above-described embodiment, it has been described that the viewer moves left and right, but the viewing zone can be made to follow even when the viewer moves up and down. That is, the elemental image group may be translated in the direction opposite to the viewer moving direction, and light may be incident on the lens array on the side opposite to the viewer moving direction.
In the above-described embodiment, the element image group is translated according to the viewpoint position of the viewer. However, the present invention is not limited to this. For example, when the viewer's viewpoint position moves to the near side or the far side, the element image group may be enlarged or reduced.

  In the above-described embodiment, an example of the point light source group forming pattern has been described. However, the present invention is not limited to this. In other words, the position and size of the non-mask area, the projection interval of the point light source formation pattern including the non-mask area, and the projection direction of the point light source formation pattern are the same size as the element image. It is optional.

  In the above-described embodiment, the control device is described as independent hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program that causes hardware resources such as a CPU, a memory, and a hard disk included in a computer to operate cooperatively as the control device described above. These programs may be distributed via a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

1,1B IP stereoscopic image display device 10 Projector (light source)
20 Collimating lens (Parallel light optical system)
30 Lens array (Parallel optical system)
31 element lens 40 spatial light modulator (display device)
50, 50B Control device 51, 51B Element image group storage means 52 View point detection means 53 Display position calculation means 54, 54B Time division display means 55 Light source control means 56 Synchronization control means 60 Depth camera (depth estimation camera)
70 Point light source group forming pattern 71, 71 _B , 71 _W Point light source forming pattern 73 Non-mask area 75 Mask area 9 IP stereoscopic image display device 91 Direct view display 93 Lens array 95 Element lenses A, A _{1 to} A ₃ viewing E element image group e, e ₁ , e ₂ element image P point light source group p point light source Z stereoscopic image α, α _{1 to} α ₃ , β viewing zone

Claims (5)

An IP stereoscopic video display apparatus in which a plurality of persons can view the element image group by forming a viewing area for each person and displaying the element image group in the formed viewing area in a time-sharing manner,
A light source that emits outgoing light;
A parallel light optical system that converts light emitted from the light source into parallel light and forms a viewing zone including at least both eyes of the person;
Viewpoint detection means for detecting the number of persons and the viewpoint position for each person;
Display position calculating means for calculating the display position of the group of element images so that the person is included in the viewing zone based on the viewpoint position for each person detected by the viewpoint detecting means;
Time-division display means for time-dividing the element image group based on the number of people detected by the viewpoint detection means;
For each person detected by the viewpoint detection means, based on the viewpoint position, a light source control means for controlling the characteristics of the emitted light so that the person is included in the viewing zone;
A display device for displaying the element image group time-divided by the time-division display means at the display position;
An IP stereoscopic video display device comprising: Element image group storage means for storing the element image group set in advance for the person,
2. The IP stereoscopic video display device according to claim 1, wherein the time-division display unit time-divisions the element image group read from the element image group storage unit. The light source is a projection device that projects a point light source group forming pattern in which point light source forming patterns, which are mask images representing the light distribution characteristics of the point light source, are arranged corresponding to the element lenses,
The display device is a spatial light modulator that displays the group of element images,
The light source control means includes, as the characteristics of the emitted light, the position and size of a non-mask area included in the point light source formation pattern, the projection interval of the point light source formation pattern including the non-mask area, and the point. Control one or more of the projection direction of the light source formation pattern,
The parallel light optical system includes:
A collimating lens that emits, as parallel light, the point light source group forming pattern projected by the projection device;
A lens array in which the element lenses are arranged in a two-dimensional manner, and condensing parallel light from the collimating lens at the focal point of each element lens to form a point light source group;
The IP stereoscopic video display device according to claim 1, comprising:   The viewpoint detection unit performs depth estimation processing on a stereo image obtained by photographing the person by a depth estimation camera, and detects the number of persons and the viewpoint position for each person. The IP stereoscopic video display device according to claim 3.   The program for functioning a computer as an IP stereoscopic video display apparatus as described in any one of Claims 1-4.

Images