JP7329795B2 - Image supply device, image supply method, display system and program - Google Patents

Description

The present invention relates to an image supply device, an image supply method, a display system and a program.

By projecting images from multiple viewpoints with horizontal parallax using multiple projectors and viewing images according to the viewer's viewpoint position (both eye position), 3D images with motion parallax can be viewed with the naked eye. A possible technique is proposed.

Non-Patent Document 1 discloses a technique of projecting a 3D image with sparse projector intervals (small number of projectors) by utilizing a visual mechanism of perception called linear blending.

M. Makiguchi, T. Kawakami, M. Sasai, and H. Takada, “Smooth Motion Parallax Glassless 3D Screen System Using Linear Blending of Viewing Zones and Spatially Imaged Iris Plane”, SID, Vol. 48, Issue 1, pp. 903 -906

In linear blending, there is a constraint that the parallax between adjacent viewpoint images must be less than the fusion limit. When the parallax interval between objects in adjacent viewpoint images exceeds the fusion limit, the objects are separated and a double image is perceived, degrading the quality of the stereoscopic image.

When projecting a 3D image using linear blending, if the viewer gets closer to the screen than the expected distance, the parallax interval between the objects in the image observed by the left and right eyes widens, and the parallax interval exceeds the fusion limit. There is a problem that a double image is perceived for the object.

The present invention has been made in view of the above, and an object of the present invention is to suppress deterioration in the quality of stereoscopic video.

A display system according to one embodiment of the present invention is a display system that displays stereoscopic images using linear blending, comprising a screen arranged with a reflective surface facing upward; a plurality of projectors for projecting images having parallax in a virtual space; a projection unit that photographs an object in a virtual space from different directions with a plurality of virtual cameras and supplies the images having the parallax to each of the plurality of projectors; a viewpoint calculation unit for calculating a viewpoint position and a viewing distance of a person; specifying one or more images having the parallax visually recognized by the observer based on the viewpoint position; and having the parallax based on the viewing distance. a control unit that determines whether or not a parallax interval of an object in an image exceeds a fusion limit, and controls the projection unit so that the parallax interval of the object does not exceed the fusion limit if the fusion limit is exceeded. .

According to the present invention, it is possible to suppress deterioration in the quality of stereoscopic video.

FIG. 1 is a diagram showing a configuration example of a display system according to this embodiment. FIG. 2 is a diagram for explaining linear blending. FIG. 3 is a diagram showing a luminance distribution according to viewpoint positions. FIG. 4 is a diagram for explaining linear blending. FIG. 5 is a diagram showing azimuth angles. FIG. 6 is a diagram showing viewing distances. FIG. 7 is a diagram for explaining the relationship between the distance to the screen, the depth distance, the viewpoint interval, and the parallax interval. FIG. 8 is a diagram for explaining the depth distance and projection distance of the fusion limit. FIG. 9 is a flowchart showing the processing flow of the display system of this embodiment. FIG. 10 is a diagram illustrating an example of a hardware configuration of a display system;

An embodiment of the present invention will be described below with reference to the drawings.

The display system of this embodiment shown in FIG. 1 projects images having parallax from a plurality of projectors 20 onto a circular table-like screen 30 to display a stereoscopic image. A video with parallax is a video of an object shot from different directions (viewpoints). In this embodiment, stereoscopic images are displayed using an optical linear blending technique that takes advantage of the visual effect that occurs when spatially coupled iris planes overlap each other.

The plurality of projectors 20 are arranged in a circle so as to surround the screen 30 . For example, 72 projectors 20 are suspended from the ceiling and arranged in a circle at intervals of 5 degrees.

A projection unit 13 is connected to each of the projectors 20 . The number of projection units 13 and the number of projectors 20 are the same, and one projection unit 13 is connected to one projector 20 . A plurality of projection units 13 supply a plurality of images having parallax to each of the projectors 20 . Each of the projectors 20 projects the image supplied from the projection unit 13 onto the screen 30 . Images from each of the projectors 20 are projected on the entire surface of the screen 30 . Each of the projection units 13 renders images of an object in the virtual space photographed by a virtual camera from different directions, and supplies the rendered images to the projector 20 .

The screen 30 is a table-shaped reflective screen with a reflective surface (projection surface) directed toward the ceiling. The observer 100 can observe the image projected on the screen 30 as if he were looking at an object on the table from 360 degrees around the screen 30 . Since the viewpoint of the observed image changes according to the position of the observer 100, the observer 100 can observe the image projected on the screen 30 from 360-degree directions.

The screen 30 reflects the projected image to form a spatially combining iris plane at a predetermined distance from the screen 30 . A spatial imaging iris plane is a combined plane with adjusted brightness. In the spatial imaging iris plane, the brightness gradually decreases in a nearly linear manner as the distance from the center of the spatial coupling iris plane increases in the horizontal direction. The screen 30 can use the spatial imaging iris surface type screen described in Non-Patent Document 1. FIG.

Observer 100 can see images projected by projectors 20 arranged on the opposite side of screen 30 . The spatially combined iris planes formed by the images from adjacent projectors 20 partially overlap. At viewpoint positions where the spatially coupled iris planes overlap, each of the projected images can be viewed. The luminance ratio of the video to be mixed changes according to the viewpoint position, and the position of the object in the video changes according to the luminance ratio. As a result, the observer 100 experiences motion parallax and can perceive an object in the depth direction or front direction of the screen 30 . In addition, even if the observer 100 is stationary, the images visually recognized by the left and right eyes are images mixed with different luminance ratios, and the left and right eyes perceive objects in the images at different positions. Therefore, the observer 100 can perceive an object in the depth direction or front direction of the screen 30 even with binocular parallax.

When the observer 100 gets closer to the screen 30 than the expected viewing distance, the objects in the image may be perceived as separated. In this embodiment, a viewpoint calculation unit 11 and a control unit 12 are provided. The viewpoint calculation unit 11 detects the viewpoint position of the observer 100, and the control unit 12 optimizes the position of the virtual camera according to the viewpoint position, Quality degradation of stereoscopic video is suppressed by performing image processing. Details of the viewpoint calculation unit 11, the control unit 12, and the projection unit 13 will be described later.

Linear blending will be described with reference to FIG. In FIG. 2, three projectors 20-1, 20-2, 20-3 and a screen 30 are shown.

The projectors 20-1, 20-2, and 20-3 are arranged at positions separated from the screen 30 by a distance a. A central projector 20-2 projects an object 50-2 onto the screen 30. FIG. The image projected by the projector 20-2 is reflected by the screen 30 to form a spatial imaging iris plane 40-2 at a distance d from the screen 30. FIG. The distance d at which the spatial coupling iris surface is formed satisfies the relationship of (1/a)+(1/d)=1/f, where f is the focal length of the screen 30 .

The projector 20-1 arranged on the right side of the projector 20-2 in the figure has a spatial imaging iris surface 40-1 that partially overlaps with the spatial imaging iris surface 40-2 at a position separated by a distance d from the screen 30. to form

The projector 20-3 arranged on the left side of the projector 20-2 has a spatial imaging iris surface 40-3 that partially overlaps with the spatial imaging iris surface 40-2 at a distance d from the screen 30. to form

FIG. 3 shows the luminance distribution of images from the projectors 20-1, 20-2, and 20-3 according to viewpoint positions. The luminance distribution shown in the figure is the luminance distribution on the spatially coupled iris plane when an image with uniform luminance (for example, a pure white image) is projected onto the screen 30 from each of the projectors 20-1, 20-2, and 20-3. be. Positions P1, P2, and P3 are directly in front of projectors 20-1, 20-2, and 20-3, respectively. At position P2, the luminance of the image from projector 20-2 is the highest. At the position P2, the images from the projectors 20-1 and 20-3 on both sides do not have brightness that can be discerned by the observer. Therefore, at the position P2, the observer sees only the image from the projector 20-2. As the viewpoint moves from the position P2 to the position P1, the brightness of the image projected from the projector 20-2 gradually decreases, and the brightness of the image projected from the projector 20-1 gradually increases. The sum of the luminance of both images is constant. Between the position P1 and the position P2, an image in which the images from the projectors 20-1 and 20-2 are mixed at a luminance ratio corresponding to the position can be observed.

3, the projector 20-2 projects an object 50-2 onto the screen 30, and the projector 20-1 projects an object 50-1 to the left of the object 50-2, as shown in FIG. Then, the observer perceives the object 50 as if it were at a depth S from the screen 30 due to motion parallax.

Also, as shown in FIG. 4, when the projector 20-2 projects an object 50-2 onto the screen 30 and the projector 20-1 projects an object 50-1 on the right side of the object 50-2, the observer sees the object It is perceived that 50 is positioned S′ in front of screen 30 .

In linear blending, it is necessary to make the parallax interval v of images visually recognized at the same time equal to or less than the fusion limit. When the parallax interval v exceeds the fusion limit, the objects 50-1 and 50-2 projected from the projectors 20-1 and 20-2 are separated and a double image is perceived. It is not perceived as being in position S'. When the observer gets closer than the assumed viewing distance d, the parallax interval v between the objects 50-1 and 50-2 exceeds the fusion limit, so the objects 50-1 and 50-2 are perceived separately.

Therefore, in this embodiment, the viewpoint position of the observer 100 is detected, and when the viewpoint position is closer to the screen 30 than the assumed visual distance d, the position of the virtual camera is adjusted so that the parallax interval of the object does not exceed the fusion limit. , or perform image processing to remove components of image quality degradation that are perceived separately from the rendered image.

Returning to FIG. 1, the viewpoint calculation unit 11, the control unit 12, and the projection unit 13 will be described.

The viewpoint calculator 11 calculates the viewpoint position of the observer 100 . For example, as shown in FIGS. 5 and 6, the viewpoint calculator 11 obtains an azimuth angle based on a certain azimuth on the screen 30 and a visual distance from the viewpoint of the observer 100 to the center of the screen 30 . The viewpoint calculation unit 11 may be provided with a stereo camera to calculate the viewpoint position, or may be provided with a camera and a depth sensor to calculate the viewpoint position.

The control unit 12 identifies a virtual camera (projection unit 13) that renders an image presented to the observer 100 based on the azimuth angle. As shown in FIG. 5 , one or more projectors 20 that exist on the opposite side of the observer 100 across the screen 30 and provide images to the observer 100 are identified. A virtual camera controlled by the projection unit 13 connected to the specified projector 20 is set as a camera to be controlled. Among the cameras to be controlled, the virtual camera controlled by the projection unit 13 that provides an image to the projector 20 near the front of the observer 100 is the main camera. For example, in FIG. 3 above, when the observer 100 is between the positions P1 and P2, the virtual camera of the projection unit 13 connected to the projectors 20-1 and 20-2 is the camera to be controlled. If the observer 100 is close to the position P1, the virtual camera of the projection unit 13 connected to the projector 20-1 becomes the main camera.

The control unit 12 determines an object that exceeds the parallax range based on the viewing distance. In linear blending, an object whose parallax interval exceeds the fusion limit is not perceived in the depth direction of the screen 30 and is perceived separately.

A case where the parallax interval exceeds the fusion limit will be described with reference to FIG. A parallax interval v of the fusion limit at a viewpoint position distant by a distance d from the screen 30 shown in FIG. 7 is expressed by the following equation.

where A is the fusion limit angle. The fusion limit angle A is an angle of 6 to 8 (min). As can be seen from FIG. 7 and equation (1), when the viewpoint position approaches the screen 30, the parallax interval v exceeds the fusion limit.

The distance in the depth direction perceived by the observer will be described. A viewpoint interval t at a viewpoint position d is expressed by the following equation.

where P is the virtual camera interval (deg).

The relationship between the depth distance S perceived by the observer, the distance d to the screen, the viewpoint interval t, and the parallax interval v is expressed by the following equation.

The relationship between the projection distance S' perceived by the observer, the distance d to the screen, the viewpoint interval t, and the parallax interval v is expressed by the following equation.

Objects beyond the depth distance S or projection distance S' expressed by equations (3) and (4) are perceived as separated.

With reference to FIG. 8, the relationship shown in FIG. 7 is applied to the display system of this embodiment. FIG. 8 is a side view of the screen 30 of the display system.

An object having a parallax interval v of the fusion limit, that is, an object perceived at a position of depth S is perceived at a position a depth distance h away from the screen 30 surface when viewed from a viewpoint position of visual distance d. In other words, an object located at a position lower than the depth distance h is perceived as being separated. The fusion limit depth distance h is expressed by the following equation.

Here, H is the vertical distance from the screen 30 to the viewpoint position.

Also, the projection distance h' of the fusion limit of the object perceived in front of the screen 30 is expressed by the following equation.

The control unit 12 determines whether or not there is an object displayed at a position lower than the depth distance h and at a position higher than the pop-out distance h', thereby merging the parallax intervals in the virtual space captured by the camera to be controlled. It can be determined whether there are any objects exceeding the limit. Since the height in the virtual space corresponding to the depth distance h and the pop-out distance h' is known, the control unit 12 determines whether or not there is an object with a height exceeding the depth distance h and the pop-out distance h'. judge.

If the parallax interval of the object exceeds the fusion limit between images that can be visually recognized by the observer 100, the control unit 12 controls the projection unit so that the parallax interval of the object does not exceed the fusion limit. Specifically, the control unit 12 instructs the projection unit 13 to change the arrangement of the camera to be controlled (virtual camera), or instructs the projection unit 13 to perform image processing for removing image quality degradation components for the object. give instructions.

When the distance between the cameras to be controlled is narrowed, the depth distance and the popping distance perceivable by the observer 100 are increased. The control unit 12 sets the interval t between the cameras to be controlled, that is, the interval P (deg) between the control cameras, so as to satisfy the following expression for the maximum depth distance h at which the object is perceived in the depth direction.

In addition, the control unit 12 sets the distance t between the cameras to be controlled so as to satisfy the following expression with respect to the maximum distance h' at which the object is perceived in the front direction.

When changing the interval between the controlled cameras, the control unit 12 brings the other controlled cameras closer to the main camera, for example.

Alternatively, without changing the interval P (deg) between the controlled cameras, the controlled camera may be moved farther away from the object to reduce the scale of rendering by the projection unit 13 .

Image processing that removes image quality deterioration components includes, for example, processing to hide objects that exceed the fusion limit, processing to crop and render the portion of the object that exceeds the fusion limit, and processing to fade out the object according to depth information. Alternatively, it is a process of blurring an object according to depth information. If the object is hidden or cropped, there will be no part of the image that exceeds the fusion limit. When the object is faded out or blurred, the outline of the object becomes unclear and the double image becomes less perceptible.

The projection unit 13 controls the virtual camera based on an instruction from the control unit 12, and renders an image captured by the virtual camera in the virtual space. Each of the projection units 13 renders images captured from different angles in the same virtual space. Since the projectors 20 are arranged in a circle around the screen 30, each virtual camera is arranged in a circle around the subject in the virtual space.

The image rendered by the projection unit 13 is supplied to the projector 20 . Note that one projection unit 13 may control a plurality of virtual cameras and supply images to each of the plurality of projectors 20 .

The image projected on the screen 30 by the projector 20 is reflected by the screen 30 and provided to the observer 100 on the opposite side of the projector 20 with the screen 30 interposed therebetween.

With reference to FIG. 9, the flow of processing for suppressing deterioration in image quality by the display system of the present embodiment will be described. Each of the projection units 13 renders images of an object in the virtual space photographed by a virtual camera from different directions, and supplies the rendered images to the projector 20 . Each of the projectors 20 projects an image rendered by the projection unit 13 onto the screen 30 .

In step S<b>11 , the viewpoint calculation unit 11 detects the viewpoint position of the observer 100 and obtains the azimuth angle and visual distance of the observer 100 .

In step S12, the control unit 12 identifies a camera to be controlled that provides an image to the observer 100 from the azimuth angle.

In step S13, the control unit 12 determines whether or not the object between the images falls within the fusion limit based on the viewing distance and the parallax between the cameras to be controlled.

If it does not fit within the fusion limit, in step S14, the control unit 12 transmits to the corresponding projection unit 13 an instruction to change the arrangement of the camera to be controlled or an instruction to perform image processing for deleting the image quality deterioration component.

In accordance with instructions from the control unit 12, the projection unit 13 changes the placement of the virtual camera, hides objects that exceed the fusion limit, crops portions of the object that exceed the fusion limit, and displays objects according to depth information. , or blur the object according to the depth information.

In this embodiment, a display system that projects images from 360 degrees onto a table-shaped screen 30 has been described. The present invention can also be applied to blending display systems.

Further, in this embodiment, an example using the reflective screen 30 that achieves linear blending has been described. good.

The viewpoint calculation unit 11, the control unit 12, and the projection unit 13 described above include, for example, a central processing unit (CPU) 901, a memory 902, a storage 903, and a communication device 904 as shown in FIG. , an input device 905, and an output device 906 can be used. In this computer system, the viewpoint calculation unit 11, the control unit 12, and the projection unit 13 are realized by the CPU 901 executing a predetermined program loaded on the memory 902. FIG. This program can be recorded on a computer-readable recording medium such as a magnetic disk, optical disk, or semiconductor memory, or distributed via a network.

As described above, the display system of this embodiment includes a table-shaped reflective screen 30, a plurality of projectors 20 arranged in a circle so as to surround the screen 30, and a plurality of projectors 20 connected to each other. It has a projection unit 13 and displays a stereoscopic image using linear blending. The viewpoint calculation unit 11 calculates the viewpoint position of the observer 100, the control unit 12 specifies the virtual camera (controlled camera) that captures the image visually recognized by the observer 100, and the parallax interval of the object in the image is the fusion limit. It is determined whether or not the When the fusion limit is exceeded, the control unit 12 instructs the projection unit 13 to change the arrangement of the cameras to be controlled to narrow the interval, or performs image processing to remove the image quality deterioration component for the object exceeding the fusion limit. The projection unit 13 is instructed as follows. As a result, a double image of the object is not perceived, and deterioration in the quality of the stereoscopic video can be suppressed.

DESCRIPTION OF SYMBOLS 11... Viewpoint calculation part 12... Control part 13... Projection part 20... Projector 30... Screen

Claims (6)

An image supply device for supplying an image having parallax to a display system that displays a stereoscopic image using linear blending,
a projection unit that captures images of an object in a virtual space from different directions with a plurality of virtual cameras and supplies images having the parallax to the display system;
a viewpoint calculation unit that calculates a viewpoint position and a viewing distance of an observer;
One or more images having the parallax visually recognized by the observer are specified based on the viewpoint position, and whether or not the parallax interval of the object in the image having the parallax exceeds the fusion limit based on the viewing distance. and a control unit that controls the projection unit so that the parallax interval of the object does not exceed the fusion limit when the fusion limit is exceeded. The image supply device according to claim 1,
The image supply device, wherein the control unit changes the arrangement of the virtual camera that captures the image having the parallax when the parallax interval of the object exceeds a fusion limit. The image supply device according to claim 1,
When the parallax interval of the object exceeds the fusion limit, the control unit hides the object so that the parallax interval of the object does not exceed the fusion limit when the projection unit photographs the object. cropping and rendering a portion of the subject that exceeds the fusion limit; fading out the subject according to depth information; or blurring the subject according to depth information. to the projection unit. An image supply method by an image supply device for supplying an image having parallax to a display system that displays a stereoscopic image using linear blending,
a step of photographing an object in a virtual space from different directions with a plurality of virtual cameras and supplying the images having the parallax to the display system;
calculating an observer's viewpoint position and viewing distance;
identifying one or more images having the parallax viewed by the observer based on the viewpoint position;
determining whether a parallax interval of an object in an image having the parallax exceeds a fusion limit based on the viewing distance;
and controlling, if the fusion limit is exceeded, the parallax interval of the subject not to exceed the fusion limit. A display system that displays a stereoscopic image using linear blending,
a screen arranged with the reflective surface facing upward;
a plurality of projectors arranged around the screen and projecting images having parallax on the screen;
a projection unit that captures images of an object in a virtual space from different directions with a plurality of virtual cameras and supplies images having the parallax to each of the plurality of projectors;
a viewpoint calculation unit that calculates a viewpoint position and a viewing distance of an observer;
One or more images having the parallax visually recognized by the observer are specified based on the viewpoint position, and whether or not the parallax interval of the object in the image having the parallax exceeds the fusion limit based on the viewing distance. a control unit that determines, and, if the fusion limit is exceeded, controls the projection unit so that the parallax interval of the subject does not exceed the fusion limit. A program for causing a computer to operate as each part of the image supply device according to any one of claims 1 to 3.

Images