JP2002027505A - Interleaver, stereoscopic video display device, and stereoscopic video generation method - Google Patents

Abstract

(57) [Summary] An object of the present invention is to further improve the reproducibility of the brightness of the entire original image with respect to the brightness of the entire stereoscopic video image. SOLUTION: The synthetic image sub-pixel included in the lenticular lens L1 multiplies the luminance of the sub-pixel of the original image pixel corresponding to the synthetic image pixel P0 by 3/4,
A value obtained by multiplying the luminance of the sub-pixel of the original image pixel corresponding to the composite image pixel P1 by 1/4 and adding the same is set. For the composite image sub-pixel included in the lenticular lens L2, a value obtained by multiplying the luminance of the sub-pixels of the original image pixels corresponding to the composite image pixels P1 and P2 by 2/4, respectively, is set. The luminance of the sub-pixel of the original image pixel corresponding to the composite image pixel P2 is reduced to １ ／ by the composite image sub-pixel included in the lenticular lens L3.
A value obtained by multiplying the luminance of the sub-pixel of the original image pixel corresponding to the composite image pixel P3 by 3/4 is set.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-lens lenticular interleaver, a stereoscopic video display apparatus, and a stereoscopic video generation method.

[0002]

2. Description of the Related Art Conventionally, in a multi-view lenticular stereoscopic image display apparatus, an image (hereinafter, referred to as an original image) 2 viewed from a plurality of different viewpoints as shown in FIG. A subpixel interleaver (hereinafter, referred to as an interleaver) that generates an image for stereoscopic vision (an image displayed as a stereoscopic video through a lenticular plate) (hereinafter, referred to as a composite image) by assigning the image to a subpixel interleaver is known. I have. Generally, images viewed from a plurality of different viewpoints are respectively different images. However, in FIG. 2, for simplification of description, the images viewed from all viewpoints are the same. FIG. 2 shows an original image 2 in the case of a four-lens lenticular system (hereinafter, referred to as a four-eye system). As the original image 2, four original images (far-left original image 2-
0, left original image 2-1, right original image 2-2, far right original image 2-
3) is included. The interleaver generates a composite image from these four original images. In the case of a four-eye system,
By interleaving every four pixels and repeating the process, an entire composite image is generated.
Only 4 pixels are shown as the original image 2 for simplicity. In the following description, only four pixels will be described for simplicity.

Conventionally, as a method of generating a composite image from a plurality of original images, a sub-pixel of each original image corresponding to each sub-pixel of the composite image (hereinafter, referred to as a composite image sub-pixel) as shown in FIG. There has been a method (hereinafter, referred to as direct sampling) of selecting and setting the luminance of the selected sub-pixel as the luminance of the composite image sub-pixel.
FIG. 8 shows a case where the far right original image 2-3, the right original image 2-2, the left original image 2-1, and the far left original image 2-0 are directly sampled.

[0004] More specifically, the direct sampling will be described in detail with reference to four synthesized image pixels P0 and P0.
Each of the composite image sub-pixels r ₀ , g ₀ , b ₀ ,..., R ₃ , g ₃ , and b ₃ included in 1, P2, and P3 are, in order, the far right original image 2-3 and the right original image 2-2. , Left original image 2-1 and far left original image 2-0. That is, the luminance of the composite image sub-pixel r ₀ is the far right original image 2
−3, and the luminance of the composite image sub-pixel g ₀ is
Determined from the right original image 2-2, the composite image sub-pixel b ₀
The luminance determined from the left original image 2-1, the luminance of the composite image subpixels r ₁ is determined from the far left original image 2-0. Then, the luminance of the composite image subpixel g ₁ is determined from the far right original image 2-3 again.

[0005] The composite image subpixel r₀Is the leftmost composite image
A sub-pixel of the image pixel (composite image pixel P0)
Therefore, this composite image sub-pixel r₀Against, far
Pixels of the right original image 2-3 (hereinafter, far right original image pixels
That. ), The sub-pixel of the leftmost pixel P30
r₃₀Is selected. Similarly, the composite image subpixel g ₀
To the pixels of the right original image 2-2 (hereinafter referred to as right original image pixels).
It is called Kussel. ) Of the leftmost pixel P20.
Kussel g₂₀Is selected and the composite image sub-pixel b₀To
Then, the pixels of the left original image 2-l (hereinafter, the left original image pic
It is called a cell. ), The sub-pixel of the leftmost pixel P10
Cell b_TenIs selected. Also, the composite image sub-pixel r
₁Is the second composite image pixel from the left (the composite image pixel
Since this is a sub-pixel of the cell P1),
Pixel r₁For the pixel of the far-left original image 2-0
(Hereinafter referred to as far-left original image pixels).
Subpixel r of the pixel P01₀₁Is selected
You.

Then, the luminance of each composite image sub-pixel
Is the luminance of each selected subpixel of the original image
Through the lenticular plate L, as a stereoscopic image
A composite image to be displayed was generated. This direct server
Each original image 2-0, 2 as shown in FIG.
-1, 2-2, 2-3 sampling and interleaving
The synthesized image that has been turned into an image as shown in FIG. Immediately
The video 3-3 viewed from the far right viewpoint is the far right original image 2
Composite image sub-pixel in which sub-pixel 3 is selected
Cell r₀, G₁, B_TwoIs caused by each lens of the lenticular plate L
Lens width (for four subpixels of the composite image)
Is displayed. Composite image subpixel r₀Brightness is far
Right original image sub-pixel r₃₀Of the composite image sub
Pixel g₁Is the far-right original image sub-pixel g₃₁of
Brightness, and the composite image sub-pixel b _TwoBrightness is far right
Original image sub-pixel b₃₂Brightness. Also, lens width
The sub-pixel enlarged to the size is called an apparent display cell.

Similarly, a stereoscopic image 3 viewed from the right viewpoint
-2 is a stereoscopic image 3 viewed from the left viewpoint, in which the composite image sub-pixels g ₀ , b ₁ , and r ₃ in which the sub-pixels of the right original image 2-2 are selected are each enlarged to the lens width and displayed. _{-1, b 0, r 2, g} 3 of subpixels left original image 2-1 is selected is enlarged and displayed on the lens width, respectively. Then, the stereoscopic video 3-0 viewed from the far-left viewpoint is displayed such that r ₁ , g ₂ , and b ₃ in which the sub-pixels of the far-left original image 2-0 are selected are each enlarged to the lens width. .

Further, as shown in FIG. 10, a method of selecting a sub-pixel from an original image pixel corresponding to a synthesized image pixel where a sample point is located by setting a center of an apparent display cell as a sample point (hereinafter referred to as point sampling). .) Was also used.

FIG. 10 is a diagram for explaining point sampling. As shown in FIG. 10, the center of each lenticular lens L1, L2, L3 of the lenticular plate L, that is,
The center positions of the apparent display cells are represented by sample points S1 and S, respectively.
2, and S3. Then, a sub-pixel is selected from the original image pixels corresponding to the composite image pixel where the sample point of each apparent display cell is located.

For example, the center of the lenticular lens L1,
That is, the sample point S1 is located at the composite image pixel P0. Therefore, the lenticular lens L1 included in the leftmost apparent display cell (when displayed as a stereoscopic video image)
Composite image sub-pixels r ₀ , g ₀ , b
₀ for r _1, selects the sub-pixel from the corresponding original image pixels in the composite image pixel P0. That is, since the composite image pixel P0 is the leftmost composite image pixel, the leftmost original image pixels P00, P1 of each original image
The sub-pixels 0, P20, and P30 are selected.

Accordingly, display far right original image subpixels r ₃₀ to the sub-pixels r ₀ is selected, the right original image sub-pixel g ₂₀ is selected for the composite image subpixels g _0, the synthesized image subpixel b ₀ left original image sub-pixel b ₁₀ is selected for the far left original image subpixels r ₀₀ are selected for the composite image subpixels r _1. Then, the luminance of the original image subpixel selected for each composite image subpixel is set as the luminance of each display subpixel.

Since the sample point S2, which is the center of the lenticular lens L2, is located between the composite image pixel P1 and the composite image pixel P2, the composite image pixel is not determined by the position of the sample point S2. Therefore, for the composite image sub-pixels g ₁ , b ₁ , r ₂ , and g ₂ enlarged by the lenticular lens L2, the composite image pixel P
1 and the original image pixel corresponding to the composite image pixel P2. That is, two original image sub-pixels are selected for one composite image sub-pixel. Then, the average value of the luminances of the two selected original image subpixels is set as the luminance of the composite image subpixel. For example, with respect to the synthetic image subpixels g _1, far right original of the sub-pixels of the image 2-3, g ₃₁ and g ₃₂ are selected, the composite image subpixel average value of the luminance of the two subpixels the brightness of g _1.

For the composite image sub-pixels b ₂ , r ₃ , g ₃ , b ₃ enlarged by the lenticular lens L ₃ , since the sample point S 3 is located at the rightmost composite image pixel P 3, Right original image pixel P03, P1
3. Select the original image sub-pixels of P23 and P33. Then, the luminance of the selected original image subpixel is set as the luminance of each composite image subpixel.

[0014]

However, as shown in FIG. 2, a composite image is generated by sampling and interleaving by direct sampling from an original image 2 in which the original image for all viewpoints has the same black and white pattern repeated. In this case, the stereoscopic images viewed from each viewpoint are stereoscopic images (far-right stereoscopic image 3-3, right stereoscopic image 3-2, left stereoscopic image 3-1, and stereoscopic image 3-1, as shown in FIG. 9).
Far-left stereoscopic video 3-0). As shown in FIG. 2, the far right original image 2-3 has a dark center and bright ends, but the far right stereoscopic image 3-3 shown in FIG. 9 has only the left end bright and the rest dark, and the far right The outline information is different from the original image 2-3. Similarly, the far-left stereoscopic image 3-0 is the far-left original image 2
−0 and the outline information is different. Thus, there has been a problem that the contour information of the original image is not reproduced in the stereoscopic video. Here, the outline information means the outline (shape) of the display object indicated by the luminance value (brightness / darkness) of each subpixel. In the far-right original image 2-3 and the far-right stereoscopic video 3-3, the outline information is not reproduced because the shapes of the bright portion and the far portion are different from the original image.

Further, there is a problem that the brightness of the entire stereoscopic image when displaying a composite image generated by direct sampling according to the same theory is different from the brightness of the entire original image. For example, in each original image shown in FIG. 2, the luminance of half the sub-pixels of the entire original image is “1”, and the luminance of the other half sub-pixels is “0”. That is, the brightness is half of the case where all the luminance values of the sub-pixels of the original image are “1” (the brightest case). However, in the stereoscopic video shown in FIG.
In the far-left stereoscopic video image 3-0 and the far-right stereoscopic video image 3-3, the brightness becomes 1/3 of the brightness of all the sub-pixels of the entire stereoscopic video image as "1". In the case of 3-1 and the right stereoscopic video 3-2, 2 / when the brightness of all the sub-pixels of the stereoscopic video is “1”.
3, which is different from the brightness of the entire original image.

When a composite image is generated by sampling and interleaving by point sampling from the original image as shown in FIG. 2, a stereoscopic video as shown in FIG. 11 is displayed. That is, the information of the original image pixels at both ends (white portions in the original image) is displayed by being enlarged to the width of the lenticular lens, and the information of the two central pixels (black portions in the original image) is represented by one lens width. Will be displayed in a reduced form. For example, as shown in FIG. 2, in the case of an original image in which two pixels at the center are black (dark) and pixels at both ends are white (bright), the black (dark) portion is reduced in the stereoscopic video image. Therefore, there is a problem that the entire stereoscopic video image becomes brighter than the entire original image.

For example, in the original image as shown in FIG. 2, the black and white pattern moves in the horizontal direction (the ratio between white and black does not change, only the position changes. That is, the brightness of the entire original image changes. Does not change.) When displaying a moving image as a stereoscopic video, the brightness of the entire original image does not change.
The brightness of the entire stereoscopic video image generated and displayed by point sampling changes. (Each time the position of white and black changes, the white part is reduced or the black part is reduced. (The brightness of the entire image becomes brighter or darker than the brightness of the original image.) Therefore, there is a problem that the image flickers, giving eyestrain and discomfort to the observer. Further, such a problem becomes more remarkable as the number of viewpoints increases.

An object of the present invention is to further improve the reproducibility of the brightness of the entire original image with respect to the brightness of the entire stereoscopic image.

[0019]

Means for Solving the Problems To solve the above problems,
According to the first aspect of the present invention, a plurality of original images having different viewpoints are provided.
(For example, an original image 2 shown in FIG. 2), a stereoscopic video image
(For example, the composite image sub-pixel shown in FIG. 3).
Xell r₀, G₀, B₀, ..., r_Three, G_Three, B_ThreeCorresponding to
Sub-pixel (for example, the original image sub-pixel shown in FIG. 2)
Le r₀₀, G₀₀, B ₀₀, ..., r₃₃, G₃₃, B₃₃Inside
Or select the sub-pixel
Stereoscopic video generated by moving
Lenticular system stereoscopic video
Output to a display device (for example, the display unit 30 shown in FIG. 1)
Interleaver (for example, interleaver 1 shown in FIG. 1)
0) wherein the display cell is a unit and the display cell
Based on the ratio of the pixel area of the stereoscopic image included in
Setting the brightness of each sub-pixel of the stereoscopic video
Brightness setting means for performing
You.

According to a fifth aspect of the present invention, there is provided a stereoscopic video image generating method for generating a stereoscopic video image to be displayed on a multi-view lenticular type stereoscopic video image display device, wherein the method comprises the steps of: When selecting a sub-pixel corresponding to each sub-pixel of the stereoscopic video, and when interleaving the selected sub-pixels, the brightness of each of the sub-pixels, in units of display cells, and included in the display cells, Based on the ratio of the pixel area of the stereoscopic image,
It is characterized by the decision.

Here, the display cell refers to a sub-pixel (apparent display cell) of a stereoscopic image enlarged by one lens of the lenticular plate. That is, the width of the display cell is
It is equal to the width of the lenticular lens.

According to the first or fifth aspect of the present invention, the brightness of each sub-pixel of the stereoscopic video is set based on the ratio of the pixel area of the stereoscopic video included in the display cell. Since the ratio of each pixel of the original image corresponding to the display cell can be reflected on each subpixel of the stereoscopic video, the reproducibility of the brightness of the entire original image in the brightness of the stereoscopic video can be improved.

According to a second aspect of the present invention, a stereoscopic image generated by selecting a subpixel corresponding to each subpixel of a stereoscopic video image from a plurality of original images having different viewpoints and interleaving the selected subpixels. An interleaver for outputting a visual image to a multi-view lenticular-type stereoscopic image display device, wherein the luminance of each subpixel of the stereoscopic image is set as an average luminance value of the subpixels of the original image. It is characterized by comprising a brightness setting means.

According to a sixth aspect of the present invention, there is provided a stereoscopic image generating method for generating a stereoscopic image to be displayed on a multi-view lenticular stereoscopic image display device, wherein the method comprises the steps of: Selecting a sub-pixel corresponding to each sub-pixel of the stereoscopic video, and, when interleaving the selected sub-pixels, determining the luminance of each of the sub-pixels as a luminance average value of the sub-pixels of each of the original images. It is characterized by.

According to the second or sixth aspect of the present invention, the luminance of each sub-pixel of the stereoscopic image is set as the luminance average value of each sub-pixel of the original image.
The brightness of all pixels of the original image can be reflected on the brightness of the sub-pixels of the stereoscopic video image. Therefore, the brightness of the entire stereoscopic video can be made equal to the brightness of the entire original image.

According to a third aspect of the present invention, there is provided an interleaver for generating a stereoscopic video to be displayed on a multi-view lenticular stereoscopic video display device based on a plurality of original images having different viewpoints. In order to determine a selection and a mixing ratio of sub-pixels corresponding to each sub-pixel of the stereoscopic video in the plurality of original images, based on a sampling coefficient (for example, a matrix K shown in FIG. Is provided.

According to a seventh aspect of the present invention, there is provided a stereoscopic image generating method for generating a stereoscopic image to be displayed on a multi-view lenticular stereoscopic image display apparatus based on a plurality of original images having different viewpoints. In the original image, a sub-pixel corresponding to each sub-pixel of the stereoscopic image is selected based on an arbitrarily settable sampling coefficient, and when the selected sub-pixels are interleaved, each of the sub-pixels is selected. Is determined according to a mixture ratio based on the sampling coefficient.

According to the third or seventh aspect of the present invention, the sampling method can be easily changed by changing the sampling coefficient, so that the sampling method can be properly used and the adjustment of the image quality becomes easier. . For example, in the case of an image where brightness reproducibility is important, a stereoscopic video is generated from the original image by a sampling method with higher brightness reproducibility, and in the case of an image in which contour information is important, A stereoscopic video image based on the original image can be generated by a sampling method with high reproducibility of the outline information. Further, for example, when the processing speed is important, it is possible to generate a stereoscopic video by a sampling method capable of higher-speed processing. Here, the sampling method refers to a method including sampling (selection of sub-pixels corresponding to each sub-pixel of the stereoscopic video) and interleaving.

According to a fourth aspect of the present invention, there is provided a stereoscopic image display apparatus including the interleaver according to any one of the first to third aspects.

According to the fourth aspect of the present invention, it is possible to generate a stereoscopic image in which the reproducibility of the brightness of the entire original image is improved.
A stereoscopic video display device for displaying can be provided. as a result,
It can improve the flicker of images.

[0031]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a stereoscopic video display apparatus 100 including an interleaver to which the present invention is applied.
It is a figure which shows schematic structure of. In the figure, a stereoscopic video display device 100 includes an interleaver 10 and a display unit 30.
Consists of The interleaver 10 is a device for generating a stereoscopic video image that samples and interleaves the original image 2 based on a matrix K input from the outside and displays it on the display unit 30. In addition, the R which stores the image generation program
It comprises an AM, a CPU for generating a stereoscopic video by executing the program, and the like. Further, the interleaver 10 may be configured as a circuit using a dedicated image generation IC.

The display unit 30 is a stereoscopic display including a liquid crystal display device and provided with a lenticular plate. The display unit 30 displays a composite image interleaved by the interleaver 10 to display a stereoscopic image through a lenticular plate.

FIG. 1 shows a four-eye type stereoscopic image display apparatus 100, and an original image 2 is a far left original image 2-
0, left original image 2-1, right original image 2-2, far right original image 2-
3 are sampled and interleaved by the interleaver 10. In FIG. 1, the original image 2
Is input from outside the stereoscopic video display device 100, but may be stored in advance in the stereoscopic video display device 100 or generated in the stereoscopic video display device 100. Specifically, for example, a case is considered in which the stereoscopic video display device 100 is a game device and also generates and displays a game image.

As will be described in detail later, the matrix K is a coefficient for determining a sampling method when the interleaver 10 performs sampling and interleaving. This matrix K is input from the outside of the stereoscopic video display device 100 similarly to the original image 2 in FIG. 1, but may be stored in the stereoscopic video display device 100.

FIG. 2 is a diagram showing an example of the original image 2.
In the figure, four original images 2-0 (at four viewpoints) as original image 2 are all the same image, and four pixels are white (from left). r,
It is assumed that the image is such that the values (luminance) of g and b are all “1”, black (the values of r, g, and b are all “0”), black, and white. Throughout this specification, the brightness of each sub-pixel (r, g, b) is “1” when it is brightest and “0” when it is darkest.

FIG. 3 is a diagram showing an example of sampling and interleaving by the interleaver 10 in the present embodiment. Hereinafter, the sampling and interleaving method shown in FIG. 3 is referred to as area sampling.
FIG. 3A is a diagram illustrating a ratio of a synthesized image pixel included in the region, where the synthesized image is divided into regions according to the width of each lenticular lens of the lenticular plate L. For example, the composite image sub-pixels (composite image sub-pixels r ₀ , g ₀ , b ₀ ,
r ₁ ) is that three belong to the composite image pixel P0 (the composite image sub-pixel r ₀ , g ₀ , b ₀ ) and one that belongs to the composite image pixel P 1 (the composite image sub-pixel r ₁ ). is there.

For this reason, as shown in FIG. 3B, for the composite image sub-pixels r ₀ , g ₀ , b ₀ , and r _{1 in the} area divided by the lenticular lens L1,
The luminance of the sub-pixels of the pixels P30, P20, P10, P00 of the original image corresponding to the position of the composite image pixel P0 is multiplied by 3/4, and the original image pixels P31, P21, P11, corresponding to the composite image pixel P1. A value obtained by adding a value obtained by multiplying the luminance of the sub-pixel P01 by 1/4 is set.

For example, the luminance of the synthesized image sub-pixel r ₀ is obtained by multiplying the luminance of the far right original image sub-pixel r ₃₀ by 3/4 and the luminance of the far right original image sub-pixel r ₃₁ by 1/4. It is set to the sum of

As shown in FIG. 3A, the composite image sub-pixels (composite image sub-pixels g ₁ , b ₁ , r ₂ , g ₂ ) of the area delimited by the lenticular lens L2.
Are two belonging to the synthesized image pixel P1 (synthesized image sub-pixels g ₁ and b ₁ ) and two belong to the synthesized image pixel P2 (synthesized image sub-pixels r ₂ and g ₂ ).

For this reason, as shown in FIG. 3B, for the composite image sub-pixels g ₁ , b ₁ , r ₂ , and g _{2 in the} area divided by the lenticular lens L2,
The luminance of the sub-pixels of the original image pixels P31, P21, P11, and P01 corresponding to the position of the composite image pixel P1 multiplied by 2/4 (= 1/2), and the original image pixel P32 corresponding to the composite image pixel P2 , P22, P1
2. The luminance of the sub-pixel P02 is 2/4 (= 1/2)
Set the value obtained by adding the multiplied value.

For example, the luminance of the composite image sub-pixel g ₁ is 輝 度 the luminance of the far right original image sub-pixel g ₃₁ and 輝 度 the luminance of the far right original image sub-pixel g ₃₂ . It is set to the sum of

Similarly, for the composite image sub-pixel in the area delimited by the lenticular lens L3, the original image pixel P3 corresponding to the position of the composite image pixel P2
2, the brightness of the sub-pixels of P22, P12 and P02 is 1
The value is set to a value obtained by adding the value obtained by multiplying the luminance of the sub-pixels of the original image pixels P33, P23, P13, and P03 corresponding to the position of the composite image pixel P3 by 3/4.

FIG. 4 shows an original image as shown in FIG.
FIG. 4 is a diagram illustrating a stereoscopic video image of each viewpoint generated by area sampling by the interleaver 10 when the original image at all viewpoints is the same in white, black, black, and white patterns. As shown in FIG. 2, the brightness of the entire image of the original image is half the brightness when the entire image is white (the brightest). In addition, the brightness of the entire stereoscopic video generated by the area sampling shown in FIG. 4 is half the brightness of the case where the entire stereoscopic video is white.

That is, when a stereoscopic image is generated by area sampling, if the luminance of the sub-pixels (r, g, b) in each pixel is the same as in the original image shown in FIG. The brightness of the entire image of the image can be reproduced as the brightness of the entire stereoscopic video image by the composite image generated by the interleaver 10.

Next, the relationship between the value (luminance) of the subpixel of each original image and the value of each subpixel after interleaving will be described. Here, the case where the number of viewpoints is n (n is a natural number other than a multiple of 3) is described, not limited to the case of the four-eye system (that is, the number of viewpoints is 4). Formula (1)
Is an expression showing the relationship between the sub-pixel values sI _j after subpixel values interleaved with each original image.

[0046]

(Equation 1)

Here, (sI ₀ , sI ₁ , sI ₂ ,..., S
I _3n-1 ) = (r ₀ , g ₀ , b ₀ ,..., R _n-1 , g _n-1 , b
_n-1 ). Also, v = (n−1− (j mod n), c
= J mod 3, j = 0, 1, 2,..., 3n-1.
Here, mod is an operator indicating a remainder.

V indicates a number corresponding to the original image of each viewpoint (for example, in the case of a four-eye system, v = 3, 2, 1,.
0). C indicates r, g, and b of the original image. (For example, c = 0 for r, c = 1 for g, c = 2 for b
Becomes ).

J indicates the number of the composite image sub-pixel. For example, in the case of the n-eye type, the number of pixels is n (sampling and interleaving are performed every n pixels). Since one pixel includes three sub-pixels, the number of composite image sub-pixels is 3n. Therefore, the value of j changes from "0" to "3n-1".

Also, i indicates a pixel number (for example, in the case of an n-eye system, the value of i takes "0" to "n-1" in order to perform sampling and interleaving every n pixels). .

That is, s _{v, 3i + c} indicates each subpixel of each original image before sampling. k _{j, i} indicates the ratio of mixing each sub-pixel value, and all j (= 0, 1, 2, 2,
, 3n-1), Equation (2) holds.

[0052]

(Equation 2)

The vector k _j = (k _{j, 0} , k _{j, 1} ,...
·, K _{j, n-1} ), vector s _j = (s _{v, c} , s _{v, 3 + c} , ...,
s _{v, 3 (n−1) + c} ), Equation (1) is represented as Equation (3).

SI _j = k _j · s _j (3) (j = 0, 1,..., 3n−1)

When the vector k _j for each j value is represented by a matrix K, the matrix K is represented by the following equation (4).

[0056]

(Equation 3)

For example, in the case of performing the area sampling in the four-lens system, the matrix K is represented by the following equation (5).

[0058]

(Equation 4)

As described above, if sampling and interleaving are performed in accordance with equation (1) _{, the} sampling method can be changed by changing the matrix K, that is, k _{j, i} in equation (1).

For example, equation (6) is a four-eye equation:
It is a mathematical expression showing a matrix K when performing direct sampling.

[0061]

(Equation 5)

Equation (7) is an equation showing a matrix K when performing point sampling in the four-eye equation.

[0063]

(Equation 6)

Note that sampling and interleaving may be performed by using, for example, a matrix K as shown in Expression (8) in addition to the three samplings described above.

[0065]

(Equation 7)

FIG. 5 is a diagram for explaining sampling in the four-lens system (hereinafter, referred to as “all-averaged sampling”) when the matrix K shown in Expression (8) is used. As shown in the figure, for each composite image subpixel, a corresponding original image (for example, when the composite image pixel is r ₀ ,
If the corresponding sub-pixels of all the pixels of the far-right original image 2-3 (for example, the composite image sub-pixel is r ₀ ,
R sub-pixels (r ₃₀ , r ₃₁ , r) of the far-right original image 2-3
_32, sets the average value of r _33)).

For example, when the original image shown in FIG. 2 is subjected to full-average sampling and interleaved, a three-dimensional image as shown in FIG. 6 is obtained. That is, although the contour information of the stereoscopic video generated by the interleaver 10 is different from the original image, the brightness of the entire image of the original image is reproduced as the brightness of the entire stereoscopic video by the composite image generated by the interleaver 10. can do.

As described above, the matrix K is changed (for example, in the case of a four-lens system, as shown by equations (5) to (8)).
That is, the sampling method can be changed by changing k _{j, i} in Expression (1).

For example, when the contour information is more important for the image, point sampling is performed. When the color information and the brightness information of the whole image are more important, the whole averaged sampling is performed, the contour information and the brightness of the whole image are obtained. When both pieces of information are important, it is possible to adjust the image quality by selectively using sampling such as area sampling. That is, the generated stereoscopic video can have higher image quality.

Further, for example, when priority is given to speeding up processing over image quality, it is possible to selectively use direct sampling.

Next, operations relating to sampling and interleaving in the present embodiment will be described with reference to the flowchart shown in FIG. Note that the operation illustrated in FIG. 7 illustrates an operation related to a synthetic image generation process of a still image (one frame image in a moving image).

First, the interleaver 10 determines a sampling method, that is, determines a matrix K (step S1). Next, n (n = 4 in the case of a four-lens system) pixels are obtained from the original image of each viewpoint (step S2), and sampling and sampling are performed according to the equation (1) based on the matrix K determined in step S1. Interleave is performed (step S3).

Then, it is determined whether or not all the pixels of the original image have been processed (step S4). If not, the process returns to step S2 to acquire the next n pixels and to execute steps S3 to S3. The processing of S4 is repeated.
Then, when the interleaving has been completed for all the pixels, the combined image is displayed according to the luminance of the combined image pixel after the interleaving (step S5), and the process ends.

By repeating the processing of steps S2 to S5 and sequentially generating and displaying a composite image of each frame, it is possible to generate and display a stereoscopic video image for a moving image.

As described above, according to the present invention, the reproducibility of the brightness of the entire original image in a stereoscopic video can be further improved by area sampling. Therefore, for example, flickering of the screen (change in brightness of the screen) that occurs when a moving image in which the image of the black and white pattern moves in the horizontal direction as shown in FIG. 2 is displayed as a stereoscopic video is reduced. Can be.

Further, since the sampling can be changed by changing the matrix K, the sampling can be easily changed according to the image, and a higher quality stereoscopic video image can be displayed.

The present invention is not limited to those described in the above embodiments, and various modifications can be made. For example, in the above-described embodiment, as shown in FIG. 2, the original image has the same original image for all viewpoints in white, black, black, and white patterns. Images may be different, and images for each viewpoint may be different from each other.

For example, in each pixel of the original image,
When the values of r, g, and b are different, if the synthesized image is generated by area sampling, the brightness of the entire original image may be different from the brightness of the entire stereoscopic video. However, since the luminance of the composite image sub-pixel is set based on the ratio of the composite image pixel included in the area divided by each lenticular lens, the composite image pixel included in the area can be reflected. Therefore,
The brightness of the original image can be reflected on the brightness of the stereoscopic video image as compared with the conventional point sampling or direct sampling.

[0079]

According to the present invention, the luminance of each sub-pixel of the stereoscopic video is set based on the ratio of the pixel area of the stereoscopic video included in the display cell. The ratio of each pixel of the image can be reflected on each sub-pixel of the stereoscopic video. Therefore, the reproducibility of the brightness of the entire original image in the brightness of the stereoscopic video can be improved.

Further, since the luminance of each subpixel of the stereoscopic video is set as the average luminance value of each subpixel of the original image, the luminance of all pixels of the original image is reflected on the luminance of the subpixel of the stereoscopic video. be able to. Therefore, the brightness of the entire stereoscopic video can be made equal to the brightness of the entire original image.

Further, since the sampling method can be easily changed only by changing the sampling matrix, the sampling method according to the image can be more easily set, and a higher quality stereoscopic video image can be displayed. .

[Brief description of the drawings]

FIG. 1 is a diagram showing a schematic configuration of a stereoscopic video display device according to the present embodiment.

FIG. 2 is a diagram illustrating an example of an original image.

FIG. 3 is a diagram illustrating the principle of area sampling.

FIG. 4 is a diagram showing an example of a stereoscopic video image displayed when images are synthesized by area sampling.

FIG. 5 is a diagram illustrating the principle of full averaging sampling.

FIG. 6 is a diagram illustrating an example of a stereoscopic video image displayed when images are combined by full averaging sampling.

FIG. 7 is a flowchart illustrating an example of an operation related to sampling and interleaving in the present embodiment.

FIG. 8 is a diagram illustrating the principle of direct sampling.

FIG. 9 is a diagram illustrating an example of a stereoscopic video image displayed when images are combined by direct sampling.

FIG. 10 is a diagram illustrating the principle of point sampling.

FIG. 11 is a diagram showing an example of a stereoscopic video image displayed when images are synthesized by point sampling.

[Explanation of symbols]

 2 Original image 100 Stereoscopic image display device 10 Interleaver 11 Display unit

Claims (7)

[Claims] 1. A stereoscopic video image generated by selecting a subpixel corresponding to each subpixel of a stereoscopic video image from a plurality of original images having different viewpoints and interleaving the selected subpixels to obtain a multiview lenticular image. An interleaver for outputting to a stereoscopic image display device of a system, wherein each of the stereoscopic images is based on a ratio of a pixel area of the stereoscopic image included in the display cell and a display cell. An interleaver comprising a luminance setting means for setting luminance of a sub-pixel. 2. A multi-lens lenticular which generates a stereoscopic video generated by selecting a subpixel corresponding to each subpixel of the stereoscopic video from a plurality of original images having different viewpoints and interleaving the selected subpixels. An interleaver for outputting to a stereoscopic image display device of a system, comprising: a luminance setting unit for setting the luminance of each subpixel of the stereoscopic image as an average luminance value of the subpixels of each of the original images. Interleaver characterized by the following. 3. An interleaver for generating a stereoscopic image to be displayed on a multi-view lenticular stereoscopic image display device based on a plurality of original images having different viewpoints, and an arbitrarily settable sampling coefficient. An interleaver comprising: a sampling method determining unit for determining a selection and a mixing ratio of sub-pixels corresponding to each sub-pixel of the stereoscopic video image in the plurality of original images based on the above. 4. A multi-view lenticular stereoscopic video display device comprising the interleaver according to claim 1. 5. A stereoscopic video image generating method for generating a stereoscopic video image to be displayed on a multi-view lenticular type stereoscopic video image display device, wherein each of the stereoscopic video images is selected from a plurality of original images having different viewpoints. When a sub-pixel corresponding to the sub-pixel is selected, and when the selected sub-pixel is interleaved, the luminance of each of the sub-pixels is determined in units of display cells,
The stereoscopic video generation method is characterized in that the stereoscopic video generation is determined based on a ratio of a pixel area of the stereoscopic video included in the display cell. 6. A stereoscopic video image generating method for generating a stereoscopic video image to be displayed on a multi-view lenticular stereoscopic video display device, comprising the steps of: Selecting a sub-pixel corresponding to the sub-pixel and, when interleaving the selected sub-pixel, determining the luminance of each of the sub-pixels as a luminance average value of the sub-pixels of each of the original images. Video generation method. 7. A stereoscopic image generation method for generating a stereoscopic image to be displayed on a multi-view lenticular stereoscopic image display device based on a plurality of original images having different viewpoints, wherein: From among them, a sub-pixel corresponding to each sub-pixel of the stereoscopic image is selected based on an arbitrarily configurable sampling coefficient, and when the selected sub-pixels are interleaved, the luminance of each of the sub-pixels is used as the sampling coefficient. A stereoscopic video generation method characterized in that the stereoscopic video generation method is determined in accordance with a mixing ratio based on the mixing ratio.