JP2846836B2 - How to convert 2D video to 3D video - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for converting a two-dimensional video output from a VTR, a video camera, or the like, or transmitted by a CATV broadcast, a TV broadcast, or the like, into a three-dimensional video.

[0002]

2. Description of the Related Art Most of the three-dimensional video software used in a three-dimensional video display system, which has recently become a hot topic, is three-dimensional video software.
It has been specially created for 3D video display systems. Generally, such three-dimensional video software is obtained by capturing and recording a left-eye video and a right-eye video using two cameras. The left and right images recorded in the three-dimensional image software are superimposed and displayed almost simultaneously on the display device. Then, the left-eye image and the right-eye image, which are superimposed and displayed, are separately incident on the left and right eyes of the observer, whereby the observer recognizes the three-dimensional image.

[0003] At present, there are many two-dimensional video softwares. Therefore, if a three-dimensional video can be generated from such two-dimensional video software, the trouble of recreating three-dimensional video software having the same contents as existing two-dimensional video software from the beginning can be omitted.

[0004] Under such circumstances, a method for converting a two-dimensional image into a three-dimensional image has already been proposed. Conventional methods for converting a two-dimensional image into a three-dimensional image include the following. That is, in the case of a two-dimensional video in which an object moving from left to right is shown, the original two-dimensional video is used as the left-eye video, and the video several frames before the left-eye video is used as the right-eye video. Is the way. In this case, a parallax is generated between the left-eye image and the right-eye image. By displaying these images almost simultaneously on the screen, the moving object is raised forward with respect to the background.

An image several frames before the left-eye image can be obtained by storing the original two-dimensional image in a field memory and reading it with a delay of a predetermined number of fields. The conventional method described above is referred to as a field delay method.

[0006]

In the above conventional method, when the delay amount of one of the left-eye image and the right-eye image is constant, the parallax increases as the moving object moves faster. The three-dimensional effect changes, making it difficult to see a three-dimensional image.

According to the present invention, a stable three-dimensional effect can be obtained, and the movement of a moving object can be made smooth.
It is an object of the present invention to provide a method of converting a two-dimensional video into a three-dimensional video in which the amount of delay can be adjusted according to the type of monitor used, conditions for viewing the monitor, and the like.

[0008]

A first method for converting a two-dimensional image into a three-dimensional image according to the present invention is a method for converting a two-dimensional image signal into a main image signal and a sub-image delayed with respect to the main image signal. In the method of converting a two-dimensional image into a three-dimensional video by generating a signal, a plurality of types of tables indicating the relationship between the speed of the motion of the image and the amount of delay are stored in advance, and the plurality of types of tables are stored. Input means for designating a predetermined table from among them is provided, and the speed of movement of the main video signal is calculated for each field, and the table specified by the input means and the motion of the current field of the main video signal are calculated. And a delay amount of the sub-video signal with respect to the main video signal in the next field is determined based on the obtained delay amount.

According to a second method for converting a two-dimensional video to a three-dimensional video according to the present invention, a main video signal and a sub-video signal delayed with respect to the main video signal are generated from the two-dimensional video signal. In a method of converting a two-dimensional image into a three-dimensional image by performing a phase shift in the horizontal direction on the obtained main image and sub-image signals, a plurality of signals indicating the relationship between the speed of image movement and the amount of delay are provided. The type table and the phase shift amount predetermined for each table are stored in advance, and a simple table for designating a predetermined table from among a plurality of types of tables and simultaneously specifying a corresponding phase shift amount is stored. One input means is provided, and the speed of movement of the main video signal is calculated for each field, and based on the table specified by the input means and the speed of motion of the current field of the main video signal. The delay amount of the sub video signal with respect to the main video signal in the next field is determined based on the determined delay amount. In the next field, the delay amount determined from the main video signal and the main video signal is determined. It is characterized in that a phase shift is performed on the sub-picture signal delayed by only a time according to the phase shift amount specified by the input means.

According to a third method of converting a two-dimensional video to a three-dimensional video according to the present invention, a main video signal and a sub-video signal delayed with respect to the main video signal are generated from the two-dimensional video signal. In the method of converting a two-dimensional image into a three-dimensional image, a relational expression for calculating the delay amount is created in advance based on the three-dimensional effect adjustment data and the speed of image movement, and the three-dimensional effect adjustment data is input. Input means for calculating the speed of movement of the main video signal for each field, adjusting data input by the input means,
A delay amount is determined based on the speed of movement of the current field of the main video signal and a relational expression created in advance, and the delay amount of the sub-video signal with respect to the main video signal in the next field is determined based on the determined delay amount. Is determined.

According to a fourth method of converting a two-dimensional video to a three-dimensional video according to the present invention, a main video signal and a sub-video signal delayed with respect to the main video signal are generated from the two-dimensional video signal. Thus, in the method of converting a two-dimensional image into a three-dimensional video, a table indicating the relationship between the speed of movement of the image and the delay amount is stored in advance, and input means for inputting delay amount magnification data is provided. Every, the speed of the movement of the main video signal is calculated for each field, the delay amount is obtained based on the speed of the movement of the current field of the main video signal and the table, and the delay amount magnification data input by the input means is calculated. The delay amount is integrated with the obtained delay amount, and the delay amount of the sub-video signal with respect to the main video signal in the next field is determined based on the integrated value.

[0012]

In the first method for converting a two-dimensional video into a three-dimensional video according to the present invention, a plurality of types of tables indicating the relationship between the speed of image movement and the amount of delay are stored in advance. Further, an input means for designating a predetermined table from among a plurality of types of tables is provided. The speed of movement of the main video signal is calculated for each field. The delay amount is obtained based on the table specified by the input means and the speed of movement of the current field of the main video signal. Then, the delay amount of the sub-video signal with respect to the main video signal in the next field is determined based on the obtained delay amount.

According to a second method of converting a two-dimensional video into a three-dimensional video according to the present invention, a plurality of types of tables indicating the relationship between the speed of image movement and the amount of delay, and a predetermined phase for each table The shift amount is stored in advance.
In addition, a single input means is provided for designating a predetermined table from among a plurality of types of tables and simultaneously designating a corresponding phase shift amount.

The speed of movement of the main video signal is calculated for each field. The delay amount is obtained based on the table specified by the input means and the speed of movement of the current field of the main video signal. Based on the amount of delay determined,
The amount of delay of the sub video signal with respect to the main video signal in the next field is determined. In the next field, the main video signal and the sub video signal delayed from the main video signal by the determined delay amount are phase-shifted according to the phase shift amount specified by the input means.

In the third method for converting a two-dimensional video into a three-dimensional video according to the present invention, a relational expression for obtaining the delay amount is created in advance based on the stereoscopic effect adjustment data and the speed of image movement. . Further, input means for inputting data for adjusting a three-dimensional effect is provided. The speed of movement of the main video signal is calculated for each field. Based on the adjustment data input by the input means, the speed of movement of the current field of the main video signal, and a relational expression created in advance,
The amount of delay is determined. Then, the delay amount of the sub-video signal with respect to the main video signal in the next field is determined based on the obtained delay amount.

In the fourth method for converting a two-dimensional video into a three-dimensional video according to the present invention, a table indicating the relationship between the speed of image movement and the amount of delay is stored in advance. Further, input means for inputting delay amount magnification data is provided. The speed of movement of the main video signal is calculated for each field. The delay amount is obtained based on the speed of movement of the current field of the main video signal and the table. The delay amount magnification data input by the input means is added to the obtained delay amount. Then, the delay amount of the sub-video signal with respect to the main video signal in the next field is determined based on the integrated value.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the configuration of a 2D / 3D conversion device for converting a two-dimensional image into a three-dimensional image.

This 2D / 3D converter generates a parallax by generating a left-eye image and a right-eye image by a field delay method, and shifts the phase to one or both of the generated left-eye image and right-eye image. To change the positional relationship between the subject and the reference screen surface.

The input terminal 1 receives a two-dimensional video signal a. The two-dimensional video signal a is sent to the motion vector detection circuit 16, the plurality of field memories 11, and the video switching circuit 13, respectively.

As is well known, the motion vector detection circuit 16 generates data for detecting a motion vector based on a representative point matching method.
The data generated by the motion vector detection circuit 16 is sent to the CPU 20.

The representative point matching method will be briefly described. A plurality of motion vector detection areas are set in the video area of each field, and each motion vector detection area is divided into a plurality of small areas. A plurality of sampling points and one representative point are set in each of the small areas.

The difference (correlation value at each sampling point) between the video signal level of the sampling point in each small area in the current field and the video signal level of the representative point in the corresponding small area in the previous field is determined by each motion vector detection. Required for each area. Then, correlation values between sampling points having the same deviation with respect to the representative point are cumulatively added between small areas in each motion vector detection area. Then, in each motion vector detection area, the deviation of the point where the cumulative correlation value becomes minimum, that is, the deviation of the point having the highest correlation is extracted as the motion vector (movement of the subject) in the motion vector detection area. You.

The field memory 11 is provided for delaying and outputting the two-dimensional video signal a in units of fields. The delay amount is set to 0 by the memory control circuit 24.
To a maximum of 60 fields (1 second in the NTSC system).

An output b (a delayed two-dimensional video signal) of the field memory 11 is sent to a video switching circuit 13 and an interpolation circuit 12, respectively. The interpolation circuit 12 generates a vertical interpolation signal for the input signal b. The output c of the interpolation circuit 12 (the vertical interpolation signal of the delayed two-dimensional video signal) is sent to the video switching circuit 13.

Therefore, the input two-dimensional video signal a and the delayed two-dimensional video signal b
And a vertical interpolation signal c of the delayed two-dimensional video signal b is input. The video switching circuit 13 sends a signal b to the left image phase control circuit 14 and the right image phase control circuit 15.
And one of the signals c (sub-video signal) and the signal a (main video signal) are switched and output according to the moving direction of the subject. However, when the delay amount is 0, the signal a is sent to both the left image phase control circuit 14 and the right image phase control circuit 15.

One of the signals b and c is selected based on whether the two-dimensional video signal a is an odd field or an even field. That is, a signal corresponding to the field type (odd field or even field) of the two-dimensional video signal a is selected from the signals b and c. The image switching by the image switching circuit 13 is performed by the CP
Controlled by U20.

Each of the phase control circuits 14 and 15 is provided to shift the display position of the input image in the horizontal direction by shifting the phase of the input image signal. The phase shift amount and the shift direction are controlled by the memory control circuit 24. Left image phase control circuit 14
Is sent to the left image output terminal 2. The output of the right image phase control circuit 15 is sent to the right image output terminal 3.

The CPU 20 controls the memory control circuit 24 and the video switching circuit 13. The CPU 20 includes a ROM 21 for storing the program and the like and a RAM 22 for storing necessary data. Data necessary for detecting a motion vector is sent from the motion vector detection circuit 16 to the CPU 20. The CPU 20 is connected to an operation / display unit 23 including various input means and a display.

The CPU 20 calculates the motion vector based on
The number of delay fields (delay amount) by the field memory 11 is calculated. That is, in principle, the delay amount is determined so that the delay amount is reduced when the motion vector is large, and is increased when the motion vector is small.

The CPU 20 controls the video switching circuit 13 based on the direction of the motion vector. That is, when the direction of the motion vector is from left to right, the input 2
The two-dimensional video signal a or the delayed two-dimensional video signal b or c is supplied to the phase control circuit 15 for the right eye.
Send to When the direction of the motion vector is from right to left, the input two-dimensional video signal a is converted to the right-eye phase control circuit 14.
Then, the delayed two-dimensional video signal b or c is sent to the left-eye phase control circuit 15.

In this 2D / 3D conversion apparatus, a parallax is generated by generating a left-eye image and a right-eye image by a field delay method, and the phase is shifted to both or one of the generated left-eye image and right-eye image. Is performed to change the positional relationship between the subject and the reference screen surface.

FIG. 2 shows a 2D / 3D conversion processing procedure by the CPU.

The 2D / 3D conversion process by the CPU is performed every time the field switching timing of the input video signal a comes.

(1) In step 1, of the plurality of field memories 11, a memory (write memory) for writing the two-dimensional video signal for the current field and a memory (read-out) for reading the already stored two-dimensional video signal ) Is output to the memory control circuit 24. Further, each phase control circuit 14, 1
The data indicating the phase shift amount and the direction due to 5 is output to the memory control circuit 24. Further, a video switching control signal is output to the video switching circuit 13.

The read memory is determined based on the delay amount determined in the previous 2D / 3D conversion processing.
Further, the phase shift amount and the direction by each of the phase control circuits 14 and 15 are determined based on the data that has already been captured and stored in step 2 of the 2D / 3D conversion processing.

The selection of one of the delayed two-dimensional video signals b and c depends on the field type of the two-dimensional video signal b to be read from the field memory 11 and
It is determined based on the field type of the two-dimensional video signal a. Further, the switching between the selected signal b or c and the signal a is determined based on the direction of the horizontal motion vector obtained in the previous 2D / 3D conversion processing.
The switching direction between the selected signal b or c and the signal a is represented by the polarity of the delay amount.

(2) In step 2, the operation / display unit 23
Are input and stored. For various input signals, a signal for setting the phase shift amount and direction, an automatic / manual mode setting signal indicating whether the delay amount is automatically calculated (automatic mode) or manual setting (manual mode), and an automatic mode are set. A delay amount setting signal performed when the manual mode is set, and a delay amount setting signal performed when the manual mode is set.

(3) In step 3, the previous 2D / 3D
Only reliable motion vectors are extracted based on the reliability results for the motion vectors for each motion vector detection area obtained in step 10 of the conversion process.

(4) In step 4, only the reliable motion vectors extracted in step 3 whose vertical component is smaller than a predetermined value are extracted.

(5) In step 5, the average value of the horizontal component (effective horizontal motion vector) of the reliable motion vector extracted in step 4 is calculated.

(6) In step 6, a delay amount calculation process is performed based on the average value of the effective horizontal motion vectors calculated in step 5. The details of the delay amount calculation processing will be described later.

(7) In step 7, it is determined whether the mode is the automatic mode or the manual mode based on the data fetched and stored in step 2.

(8) If it is determined in step 7 that the manual mode is set, the delay amount is fixed to the set value fetched in step 2 (step 8).

(9) If it is determined in step 7 that the automatic mode is set, the history data used in the delay amount calculation processing in step 6 is updated (step 9).

(10) In step 10, data necessary for motion vector detection is fetched from the motion vector detection circuit 16, and a motion vector for each motion vector detection area is calculated. Further, the reliability of the motion vector is determined for each motion vector detection area based on the average value and the minimum value of the correlation accumulated value for each motion vector detection area. Then, the calculated motion vector and the reliability determination result are stored in the RAM 22.

(11) In step 11, cinema mode detection / control processing is performed. The details of the cinema mode detection / control processing will be described later.

(12) In step 12, scene change detection / control processing is performed. Scene change detection
Details of the control processing will be described later.

FIG. 3 shows a detailed procedure of the delay amount calculation processing in step 6 of FIG.

First, a first delay amount d1 is obtained (step 21). There are a plurality of methods for determining the first delay amount d1, as described below.

(1) Description of First Method for Determining First Delay Amount d1 A description will be given of a first method for determining the first delay amount d1.
In the first determination method, the delay amount magnification set value set and stored in the step 2 and the step 5
The first delay amount d1 is calculated based on the average value v (hereinafter, referred to as a motion vector average value) of the effective horizontal motion vectors obtained in (1).

FIG. 4 shows the relationship between the average motion vector value and the delay amount. The relationship as shown in FIG. 4 is stored in the ROM 21 as a delay amount table. Then, first, the delay amount corresponding to the motion vector average value is obtained from the delay amount table.

By the way, even for the same three-dimensional video signal, the parallax varies depending on the condition of the stereoscopic display device (monitor), that is, the type of monitor and the condition for viewing the monitor. Therefore, regardless of the monitor conditions, the delay amount magnification set value set and stored in the above step 2 is set in the delay amount table in order to obtain the same three-dimensional effect or to suit the observer's preference. The first delay amount d1 is obtained by integrating the delay amount obtained from the above.

(2) Description of Second Method for Determining First Delay Amount d In order to obtain a similar three-dimensional effect regardless of the monitor conditions, a plurality of types of delay amount tables are stored and operated. An instruction for selecting a delay amount table according to monitor conditions or observer's preference is input from the display unit 23. And the delay amount corresponding to the motion vector average value is
It is obtained from the selected delay amount table. The obtained delay amount is defined as a first delay amount d1.

(3) Description of Third Method for Determining First Delay Amount d1 The first delay amount d1 may be obtained based on a predetermined relational expression instead of the delay amount table. . How to obtain the relational expression in this case will be described with reference to FIG.

A suitable distance between the monitor surface S and the eyes 31 and 32 of the observer is defined as an appropriate viewing distance A [mm]. The distance between the right image R and the left image L of the gazing object on the monitor surface S is defined as a parallax B [mm]. The distance between eyes is C [mm]. The suitable viewing distance A is determined according to monitor conditions.
The parallax B of the gazing object differs depending on the monitor conditions, even if the three-dimensional video signal is the same.

The stereoscopic image position P of the gazing object is determined by the suitable viewing distance A, the parallax B, and the interocular distance C. That is, the amount D [mm] of the gazing object protruding from the monitor surface S is determined by the suitable viewing distance A, the parallax B, and the interocular distance C.

Regardless of the monitor conditions, the parallax B for making the amount of projection of the gazing object from the monitor surface S a constant amount D is expressed by the following equation (1).

[0059]

## EQU1 ## B = DC / (AD)

Assuming that the horizontal length of the monitor is H [mm], the number of pixels in the horizontal direction of the monitor is h [pixel], the average value of the motion vector is v [pixel / field], and the first delay amount is d1 [field], The following relationship holds:

[0061]

## EQU2 ## d1.v = (h / H) .B

Here, the pixel conversion amount of the parallax B (= (h /
Assuming that H) and B) are adjustment amounts X (data relating to monitor conditions or data according to the observer's preference) set by the operation / display unit 23, the first delay amount d1 is obtained by the following relational expression. Can be

[0063]

## EQU3 ## d1 = X / v

(4) Description of Fourth Determination Method for First Delay Amount d1 When the fourth determination method is used, the operation / display unit 23
Is provided with a three-dimensional effect adjuster for adjusting the three-dimensional effect. In addition, a delay amount table of a type corresponding to a plurality of levels of volumes that can be set by the stereoscopic effect adjuster is stored in the ROM 21. Further, a horizontal phase shift amount of a type corresponding to a plurality of levels of volume that can be set by the stereoscopic effect adjuster is stored in the ROM.
21.

FIG. 35 shows an example of the relationship between the parallax amount and the phase shift amount for a plurality of levels of volumes that can be set by the stereoscopic effect adjuster. In this example, it is possible to make adjustments in four steps in the direction in which the three-dimensional effect becomes stronger, and in four directions in the direction in which the three-dimensional effect becomes weaker. That is, it is possible to adjust the three-dimensional effect in nine steps.

The amount of parallax (unit: pixel) is a measure of how much the amount of parallax is set in each volume.
The delay amount table for each volume is, in principle,
The parallax amount is created so as to be the parallax amount shown in FIG. The disparity amount is represented by V × D, where V is the motion vector and D is the delay amount.

FIG. 36 shows the relationship between the motion vector V corresponding to the volume “0” (standard) and the delay amount D. A curve S in FIG. 36 is a graph (D = D) determined by the parallax amount “12” with respect to the standard shown in FIG.
12 / V). In FIG. 36, the polygonal line indicates the relationship between the actually used motion vector V and the delay amount D. The delay amount table for volume 0 (standard) is created based on the relationship between the motion vector V and the delay amount D represented by the broken line in FIG.
It is stored in OM21.

FIG. 37 shows the relationship between the motion vector V corresponding to the volume "-4" (weakest) and the delay amount D. The curve S in FIG. 37 represents a graph (D = 4 / V) determined by the parallax amount value “4” for the volume “−4” shown in FIG. The delay amount table for the volume “−4” is created based on the relationship between the motion vector V and the delay amount D represented by the broken line in FIG.

FIG. 38 shows the relationship between the motion vector V corresponding to volume "4" (strongest) and the delay amount D. The curve S in FIG. 38 represents a graph (D = 20 / V) determined by the parallax amount value “20” for the volume “4” shown in FIG. The delay amount table for the volume “4” is created based on the relationship between the motion vector V and the delay amount D represented by the polygonal line in FIG.

Other volumes "-3", "-2", "-"
The delay amount tables for 1 "," 1 "," 2 ", and" 3 "are similarly created and stored in the ROM 21.

Each phase shift amount (unit: pixel) shown in FIG. 35 indicates the sum of the phase shift amounts used in each of the phase control circuits 14 and 15 in FIG. In this example, the phase shift direction is set to the left direction for the left-eye image, and is set to the right direction for the right-eye image. That is, in the phase control by the phase control circuit 14, the shift direction is always the left direction. In the phase control by the phase control circuit 15, the shift direction is always the right direction.

When the phase shift amount shown in FIG. 35 is an even number, the phase shift amount used in each of the phase control circuits 14 and 15 is determined to be の of the phase shift amount in FIG.
When the phase shift amount shown in FIG. 35 is an odd number, 2
Therefore, the phase shift amount used in one of the phase control circuits 14 and 15 is made larger by one pixel than the phase shift amount used in the other phase control circuit. Since the phase control can be performed by half of one pixel by interpolation or the like, even if the phase shift amount shown in FIG.
The phase shift amount used in 4 and 15 may be determined to be １ ／ of the phase shift amount in FIG.

When the fourth determination method is adopted, the volume set by the stereoscopic effect adjuster is taken in step 2 of FIG. Then, in step 6 of FIG. 2, more specifically, in step 21 of FIG. 3, the first delay amount d1 corresponding to the motion vector average value v is obtained from the delay amount table corresponding to the volume set by the stereoscopic effect controller. Can be

In step 1 of FIG. 2, the phase shift amount used in each of the phase control circuits 14 and 15 is determined based on the phase shift amount corresponding to the volume set by the stereoscopic effect adjuster. Then, the determined phase shift amount is determined by the memory control circuit 2 together with the phase shift direction described above.
4 to the respective phase control circuits 14 and 15.

That is, when the fourth determination method is adopted, one of the plurality of delay amount tables is selected by the stereoscopic effect adjuster and used by each of the phase control circuits 14 and 15. The phase shift amount is determined.

FIG. 39 shows a case where the volume is set to "-4" for an image having the same motion vector (FIG. 3).
9 (a)), when set to “0” (FIG. 39)
(B)) and “4” (FIG. 39)
(C) shows a change in the stereoscopic image position of the watched object due to only the field delay. In FIG. 39, 31 is the observer's left eye, 32 is the observer's right eye, Ra is the right image of the watched object on the monitor surface S, La is the left image of the watched object on the monitor surface S, Pa is the stereoscopic image position of the watched object,
Pb (= Rb = Lb) indicates a still image as a background. The distance between the right image R and the left image L of the gazing object is the amount of parallax. The background surface coincides with the monitor surface S. As can be seen from this figure, the larger the volume,
The amount of parallax increases, and the background surface of the gazing object (monitor surface S)
It can be seen that the three-dimensional effect increases because the amount of protrusion from the object increases.

FIG. 40 shows a case where the volume is set to "-4" for an image having the same motion vector (FIG. 4).
0 (a)), when set to “0” (FIG. 40
(B)) and the case where “4” is set (FIG. 40)
(C)) due to field delay and phase shift
The change of the stereoscopic image position of a gazing object and a stationary object as a background is shown. That is, the figure shows changes in the stereoscopic image positions of the gazing object and the background still object when the fourth determination method is adopted. In FIG. 40, the same reference numerals as in FIG. 39 indicate changes in the stereoscopic image position of the watched object and the background still object due to only the field delay.

In FIG. 40, 31 is the left eye of the observer,
32 is the observer's right eye, Ra 'is the right image of the watched object on the monitor surface S, La' is the left image of the watched object on the monitor surface S, and Pa 'is the stereoscopic image position of the watched object. Each is shown. Also, Rb 'is a right image of a stationary object (background) on the monitor surface S, Lb' is a left image of the stationary object (background) on the monitor surface S, and Pb 'is a stereoscopic image position of the stationary object. Each is shown.

When comparing FIG. 39 with FIG. 40, the stereoscopic position Pb ′ of the background (stationary object) and the stereoscopic position Pa ′ of the gazing object in FIG. It can be seen that the camera is moving backward with respect to the monitor surface S as compared with the stereoscopic position Pa of the gazing object. That is, the stereoscopic position P of the background (stationary object)
b ′ moves rearward from the monitor surface S, and the stereoscopic position Pa ′ of the watched object is closer to the monitor surface S than the stereoscopic position Pa of the watched object in FIG. As described above, the stereoscopic position Pb ′ of the background (stationary object) and the stereoscopic position P of the watched object
The stereoscopic effect is amplified by moving a ′ backward with respect to the monitor surface S.

In the above example, the phase is shifted to the right for the right-eye image and to the left for the left-eye image. However, the phase may be shifted in the opposite direction.

In any of the above methods, when the first delay amount d1 is obtained, based on the delay amount history data,
The average value of the delay amount for the 10 fields from this time to the past 9 times, the average value of the delay amount for the 10 fields from the previous time to the past 9 times, and the delay amount for the 10 fields from the previous time to the past 9 times Are calculated (step 22).

The delay amount history data used in step 22 is the first delay amount d1 obtained in step 21 in the past.

Next, if two or more of the three average values are the same value, the value (multiple values) is selected as the second delay amount d2. If all the values are different, the intermediate value is selected. Is selected as the second delay amount d2 (step 23).

Next, the second delay amount d2 selected in step 23 and 1 (for example, 1
One of the second delay amounts d2 (five fields before) and 30
The second delay amount d2 before the field is compared (step 24). The delay amount history data used in step 24 is the second delay amount d2 obtained in step 23 in the past.

If all the second delay amounts d2 match (YES in step 25), the target delay amount Pd is changed to the second delay amount selected in step 23 (Pd = d
2) (Step 26), and proceed to Step 30. Therefore, as shown in FIG. 6, when three second delay amounts d2 (represented by d2-1, d2-2, and d2-3 in order from the past) change, and all the second delay amounts d2 match. , The target delay amount Pd is changed to the second delay amount (d2-3).

When all the second delay amounts d2 do not match (NO in step 25), all the second delay amounts d2 are larger than the current target delay amount Pd or all the second delay amounts d2.
It is determined whether 2 is smaller than the current target delay amount Pd or does not satisfy those conditions (step 27).

When all the second delay amounts d2 are larger than the current target delay amount Pd, the target delay amount Pd is incremented by 1 (Pd = Pd + 1) (step 28), and then step 30
Proceed to. For example, as shown in FIG. 7, three second delay amounts d2 (d2-1, d2-2, d2
When the second delay amount d2 is greater than the current target delay amount Pd, the target delay amount Pd is incremented by one.

When all the second delay amounts d2 are smaller than the current target delay amount Pd, the target delay amount Pd is decremented by one (Pd = Pd-1) (step 29), and then step 30
Proceed to. All the second delay amounts d2 are equal to the current target delay amount Pd.
If it is not larger and all the second delay amounts d2 are not smaller than the current target delay amount d, the process proceeds to step 30.

In step 30, it is determined whether or not the target delay amount Pd and the currently set delay amount (set delay amount d3) match. If the target delay amount Pd does not match the set delay amount d3, it is determined whether the current set delay amount d3 has already continued for four fields (step 31). If the current set delay amount d3 has already continued for 4 fields, the set delay amount d3 is changed by 1 in the direction approaching the target delay amount Pd (d3 = d
3 ± 1) (step 32). Then, step 7 in FIG.
Move to

If it is determined in step 30 that the target delay amount matches the current set delay amount, or if the current set delay amount has not already continued for four fields in step 31, the delay amount is set. The process proceeds to step 7 in FIG. 2 without any change.

That is, in this example, the set delay amount d3 is 4
The control is performed so as to approach the target delay amount Pd in units of a field cycle and one field at a time.

After the power is turned on, step 21 is executed.
In the above, when the first delay amount d1 is calculated for the first time, the second delay amount d2, the target delay amount Pd, and the set delay amount d3 become equal to d1.

In the process of FIG. 3, in step 22, only the average value of the delay amounts for the ten fields from the current time to the past nine times is calculated, and this is set as the target delay amount, and steps 23, 24, 25, 26, The processes of 27, 28, and 29 may be omitted.

In step 22, it is also possible to calculate only the average value of the delay amounts of the ten fields in the past nine times from the current time, use this as the second delay amount, and omit the processing of step 23.

Further, the second delay amount obtained in step 23 is set as a target delay, and steps 24, 25, 26, 27,
Steps 28 and 29 may be omitted.

Further, the processing of steps 22 and 23 may be omitted. In this case, the first delay amount d1 obtained in step 21 is used as the second delay amount used in step 24.

FIGS. 8 to 14 show details of the cinema mode detection / control processing in step 11 of FIG.

Before describing these processing procedures, an outline of telecine conversion will be described.

The difference between the number of images per second (number of frames) of a movie film and the number of frames per second of a television is adjusted by a telecine system. The number of frames per second of 35 mm and 16 mm standard motion picture films is 24 frames. The number of frames per second of an 8 mm movie film is 16 or 18 frames.
On the other hand, the number of frames per second in NTSC is 30 frames (60 fields).

As shown in Table 1, a general intermittent projector as a telecine system feeds a film intermittently.
Irradiation is performed on the image aperture during the period when the film is stopped, and this is received by a film camera and converted into a video signal.

[0101]

[Table 1]

When the image obtained by the telecine system is viewed frame by frame by one field, fields without motion appear at regular intervals. Therefore, as for the motion vector detected for each field, a vector of motion 0 is detected at regular intervals if disturbance such as noise is ignored.

FIG. 15 shows an example of a change of a motion vector with respect to an image obtained from a movie film of 24 frames per second. In this example, the motion vector is α1 → 0 → β1 → 0 → 0 → α2 → 0 → β2 →
A pattern such as 0 → 0 is repeated.

Video signal a input to input terminal 1 in FIG.
An important parameter that determines whether or not telecine conversion has been performed or has not been performed is a motion vector. That is, by knowing the regularity of the motion vector to be 0, it is possible to grasp the presence / absence of telecine conversion and the conversion rule.

However, since the average of the motion vectors obtained by the plurality of motion vector detection areas is used as the final motion vector, a problem may occur. For example, in an image having two opposing movements from the left and right of the video area toward the left and right center directions, when the average of the motion vectors obtained by the respective motion vector detection areas is averaged, the left and right movements are offset. It is difficult to accurately detect a motion vector. In addition, since the detected motion vector has ambiguity, erroneous determination may occur when the presence / absence of telecine conversion is determined using only the motion vector.

Therefore, in the cinema mode detection / control processing to be described later, an attempt is made to prevent occurrence of erroneous determination as much as possible.

Next, the concept of the delay amount conversion processing performed when it is detected that the mode is the cinema mode (telecine conversion has been performed) will be described.

The concept of the delay amount conversion processing when it is detected that the telecine conversion has been performed by the 2-3 pull-down method will be described below.

The delay conversion rule in this case is as follows. (A) Conversion is performed so that the delay amount of the frames of both eyes is as constant as possible. (B) In order to provide smooth images with only one eye,
The number of fields assigned to one frame after conversion is two or three. (C) The number of delay fields after conversion (the number of delay fields of the next field) is such that the number of delay fields at normal time (the set delay amount d3 determined in step 6 in FIG. 2) is 1 to 3 (-).
When the number of delay fields is 4 to 6 (-4 to -6), one frame delay is assigned when 1 to -3) and two frames are delayed when normal delay field number is 4 to 6 (-4 to -6). (D) When the number of delayed frames changes in a direction to increase,
Three fields are assigned to one frame. Conversely, when the number of delay frames changes in a direction to decrease, two fields are assigned to one frame. (E) In order to select, as much as possible, one of the delayed video signals b and c of FIG. 1 that is not the output c of the interpolation circuit 12,
Make the number of delay fields after conversion as even as possible.

A mode that allows the delay field output in the next field to be selected from those older than the delay field output in the current field (field return) is called a compromise delay mode. The prohibited mode is referred to as a complete delay mode.

A video signal subjected to telecine conversion in the 2-3 pull-down mode is represented as shown in FIG. Here, symbols A, B, C... Indicate the types of frames, and subscripts 1, 2, 3,.
Indicates a field number.

FIGS. 17 to 20 show that the input video signal a is
6 shows a delayed image actually output by the cinema mode detection / control process when the complete delay mode is set.

FIG. 17 shows that the number of delay fields at normal time is one.
4 shows delayed images actually output at the time of -1 to -3 (-1 to -3). In this case, one frame is delayed. In this figure, the image corresponding to the number 0 in the upper column represents the image of the current field output as the video signal for one eye. Also, the images corresponding to the numbers 1 to 6 in the upper column indicate images corresponding to the number of delay fields 1 to 6. The image enclosed in parentheses indicates the actually output delayed image.

FIG. 18 shows that the number of delay fields at normal time is four.
6 shows delayed images actually output when the number is from -6 to -4 (-4 to -6). In this case, there is a two-frame delay.

FIG. 19 shows a delay image actually output when shifting from one-frame delay to two-frame delay.

FIG. 20 shows a delay image actually output when shifting from two-frame delay to one-frame delay.

FIGS. 21 to 24 show that the input video signal a is
6 shows a delayed image actually output when the compromise delay mode is set.

FIG. 21 shows that the number of delay fields at normal time is one.
4 shows delayed images actually output at the time of -1 to -3 (-1 to -3). In this case, one frame is delayed.

FIG. 22 shows that the number of delay fields at normal time is four.
6 shows delayed images actually output when the number is from -6 to -4 (-4 to -6). In this case, there is a two-frame delay.

FIG. 23 shows a delay image actually output when shifting from one-frame delay to two-frame delay.

FIG. 24 shows a delay image actually output when shifting from two-frame delay to one-frame delay.

Next, the concept of the delay amount conversion processing when it is detected that the telecine conversion has been performed by the 2-2 pull-down method will be described.

The delay amount conversion rule in this case is as follows. (A) Conversion is performed so that the delay amount of the frames of both eyes is as constant as possible. (B) In order to provide smooth images with only one eye,
The number of fields assigned to one frame after conversion is one to three fields. (C) The number of delay fields after conversion (the number of delay fields of the next field) is such that the number of delay fields at normal time is 1-2.
In the case of (-1 to -2), the delay is one frame. When the number of delay fields in the normal state is 3 to 4 (-3 to -4), the delay is two frames, and the number of delay fields in the normal state is 5 to 6 (-5 to -5). -6)
In the case of, the assignment is made so as to delay three frames. (D) When the number of delayed frames changes in a direction to increase,
Three fields are assigned to one frame. Conversely, when the number of delayed frames changes in the direction of decreasing, one field is allocated to one frame. (E) In order to select, as much as possible, one of the delayed video signals b and c of FIG. 1 that is not the output c of the interpolation circuit 12,
Make the number of delay fields after conversion as even as possible.

A video signal subjected to telecine conversion in the 2-2 pull-down mode is represented as shown in FIG. Here, symbols A, B, C... Indicate the types of frames, and subscripts 1, 2, 3,.
Indicates a field number.

FIGS. 26 to 31 show that the input video signal is
5 shows a delayed image actually output by the cinema mode detection and control process when the video signal is as shown in FIG.

FIG. 26 shows that the number of delay fields at normal time is one.
2 shows delayed images actually output at the time of -1 to -1 (-1 to -2). In this case, one frame is delayed. In this case, whether the delay mode is the complete delay mode or the compromise delay mode, the actually output delayed image is the same. In this figure, the image corresponding to the number 0 in the upper column represents the image of the current field output as the video signal for one eye. Also, the images corresponding to the numbers 1 to 6 in the upper column indicate images corresponding to the number of delay fields 1 to 6. The image enclosed in parentheses indicates the actually output delayed image.

FIG. 27 shows that the number of delay fields at normal time is three.
4 shows delayed images actually output at times of (−3 to −4). In this case, there is a two-frame delay. In this case, whether the delay mode is the complete delay mode or the compromise delay mode, the actually output delayed image is the same.

FIG. 28 shows that the number of delay fields at normal time is 5
6 shows a delayed image actually output at times -6 (-5 to -6). In this case, there is a three-frame delay. In this case, whether the delay mode is the complete delay mode or the compromise delay mode, the actually output delayed image is the same.

FIG. 29 shows a delayed image actually output when the number of delayed frames changes to 0-1-2-3 when the complete delay mode is set.

FIG. 30 shows a delay image actually output when the number of delay frames changes to 0-1-2-3 when the compromise delay mode is set.

FIG. 31 shows that the number of delay frames is 3-2-1-0.
Shows the delayed image actually output when the image changes to. In this case, whether the delay mode is the complete delay mode or the compromise delay mode, the actually output delayed image is the same.

Table 2 shows an allocation table of periodic patterns based on peak histories of motion vectors of an image subjected to telecine conversion by the 2-3 pull-down method. Such an allocation table is stored in the ROM 2 as a periodic pattern allocation table.
1 is stored.

[0133]

[Table 2]

In Table 2, the delay field number 0 indicates the current field, and the delay field numbers 1 to 6 indicate fields which are delayed by 1 to 6 with respect to the current field, respectively.

Table 3 is used in a delay amount conversion process for an image telecine-converted by the 2-3 pull-down method.
The figure shows the optimal number of delay fields with respect to the periodic pattern and the number of delay fields at normal time (normal number of delay fields). Such a table is stored in the ROM 21 as an optimal delay field number table.

[0136]

[Table 3]

In Table 3, the numbers in parentheses indicate the optimum number of delay fields in the compromise delay mode.

Table 4 is used in a delay amount conversion process for an image telecine-converted by the 2-3 pull-down method.
The relationship between the optimum number of delay fields and the number of delay frames is shown. Such a table is referred to as RO frame number table RO
It is stored in M21.

[0139]

[Table 4]

Table 5 shows an allocation table of periodic patterns based on peak histories of motion vectors of an image telecine-converted by the 2-2 pull-down method. Such an allocation table is stored in the ROM 2 as a periodic pattern allocation table.
1 is stored.

[0141]

[Table 5]

In Table 5, the delay field number 0 indicates the current field, and the delay field numbers 1 to 6 indicate fields which are delayed from the current field by 1 to 6 respectively.

Table 6 is used in a delay amount conversion process for an image subjected to telecine conversion by the 2-2 pull-down method.
The figure shows the optimum number of delay fields for the periodic pattern and the number of normal delay fields. Such a table is stored in the ROM 21 as an optimal delay field number table.

[0144]

[Table 6]

Table 7 is used in a delay amount conversion process for an image subjected to telecine conversion by the 2-2 pull-down method.
The relationship between the optimum number of delay fields and the number of delay frames is shown. Such a table is referred to as RO frame number table RO
It is stored in M21.

[0146]

[Table 7]

The cinema mode detection / control processing will be described below with reference to FIGS. Here, 2-
The video signal subjected to the telecine conversion by the 3 pull-down method and the video signal subjected to the telecine conversion by the 2-2 pull-down method can be detected, and if these are detected, the delay amount d3 determined in step 6 of FIG. Here is an example in which is converted.

FIGS. 8 to 10 show the overall processing procedure of the cinema mode detection / control processing.

First, the cycle pointer indicating the cycle pattern is updated (step 41).

Next, the number of peak values “1” of the motion vector in the past predetermined times (for example, the past seven times) is calculated from the peak history array of the motion vector (step 42).

Next, determination of the peak value of the current field,
The peak history is updated (step 43). In other words, the correlation accumulated value of the current field has a peak value of “1”.
And the correlation accumulation value of the latest field having a peak value of “0” (hereinafter referred to as the minimum value of the correlation accumulation value). If it is larger than 1/2 of the sum, the peak value of the motion vector of the current field is set to "1". Conversely, if the correlation accumulated value of the current field is equal to or less than の of the sum of the maximum value of the accumulated correlation value and the minimum value of the accumulated correlation value, the peak value of the motion vector of the current field is set to “0”. You.

However, all the peak histories of the past predetermined times are "
When the value is "0", the peak value of the motion vector of the current field is set to "1", and all the peak histories of the past predetermined times are "
When it is "1", the peak value of the motion vector of the current field is set to "0".

When the peak value of the motion vector of the current field is determined, the peak history of the past predetermined times is updated to include the peak value of the motion vector of the current field.

Next, the period pattern (peak pattern) indicated by the current period pointer (see Table 2 or Table 5) and the pattern of the past past peak history except for the peak value for the current field are completely completed. It is determined whether they match (step 44). If they completely match, it is determined whether the peak values of the current field match periodically (step 45).

If the peak values of the current field do not match periodically, the peak value of the current field is determined again and the peak history is updated again (step 46). That is, when the peak value of the current field is determined to be “0” in step 43,
If the correlation accumulated value of the current field is larger than a value obtained by adding a prescribed value to the minimum value of the past accumulated correlation values, the peak value of the current field is changed to “1”. If the peak value of the current field is determined to be "1" in step 43, if the accumulated cumulative value of the current field is smaller than a value obtained by subtracting a predetermined value from the maximum value of the past cumulative accumulated values of correlations. , Change the peak value of the current field to “0”.
When the peak value of the current field is changed, the peak history is updated again.

If it is determined in step 44 that they do not completely match, then it is determined whether or not a quasi-matching state exists (step 47). Here, the quasi-coincidence state refers to a periodic pattern (peak pattern) indicated by the current periodic pointer (see Table 2 or Table 5), and a pattern of a past predetermined peak history excluding the peak value for the current field. However, it means that only one bit does not match.

If it is determined in step 47 that the state is a quasi-coincidence state, the flow shifts to step 48. Also in step 45, when it is determined that the peak value of the current field matches periodically, and when the process of step 46 is performed, step 48 is also performed.
Move to

In step 48, the number of mismatch bits between the periodic pattern and the past predetermined number of history patterns including the current field is calculated. If the number of unmatched bits is 0, it is determined that a completely matched state exists (step 5).
1) If the number of mismatched bits is 1, it is determined to be in a quasi-matched state (step 52), and if the number of mismatched bits is 2 or more, it is determined to be in a mismatched state (step 50).

When it is determined that there is a mismatch (step 5
0) Or, if it is determined in step 47 that the quasi-coincidence state is not established, the process proceeds to step 53, where the cycle pointer is updated and it is searched whether or not there is a cycle pattern that completely matches the history pattern. . At this time, whether the input video signal is a telecine-converted video by the 2-3 pull-down scheme (2-3 pull-down mode),
It is also determined whether the input video signal is a video subjected to telecine conversion by the 2-2 pull-down method (the 2-2 pull-down mode), and the determination result is stored.

After steps 51, 52, and 53, the flow shifts to step 54, where it is determined whether or not there is a mismatch. Step 51 or Step 52 to Step 5
When the process proceeds to No. 4, the result is N0 in step 54, and the process proceeds to step 55. In step 53, when a periodic pattern completely matching the history pattern is found,
If NO in step 54, the process proceeds to step 55, and if a periodic pattern that completely matches the history pattern is not searched, YES is determined in step 54 and step 5
Proceed to 6.

In step 55, the number of zeros of the motion vector is corrected. That is, the number of motion vectors of a predetermined level or less among the values of the motion vectors of the past predetermined times is calculated. If the number of motion vectors equal to or lower than the predetermined level is within the predetermined range, it is finally determined to be a completely matched state, otherwise it is determined to be a mismatched state. Step 5
Steps 4 and 55 may be omitted. In particular, in a system in which the detection accuracy of the motion vector is not so high, step 5
It is preferable to omit the processing of steps 4 and 55.

At step 56, it is determined whether or not the peak value of the current field is "1". If the peak value of the current field is "1", the maximum value of the accumulated correlation value is updated (step 57). On the other hand, if the peak value of the current field is "0", the minimum value of the accumulated correlation value is updated (step 58).

Next, it is determined whether or not the state is a perfect match state based on the result of the state determination (step 59).
If it is in the completely matched state, it is determined whether or not it is the forward protection period (step 63). If it is the forward protection period, a delay amount conversion process in the cinema mode is performed (step 65). If it is not the forward protection period, it is determined whether or not it is the backward protection period (step 6).
4). If it is not the backward protection period, a delay amount conversion process in the cinema mode is performed (step 65).

If it is not the backward protection period, the delay amount determined by the normal delay amount determination processing (the delay amount d3 calculated in step 6 of FIG. 2) is directly used as the delay amount (step 61).

The front protection period and the rear protection period will be described. In this embodiment, the delay amount conversion process in the cinema mode is not performed unless the perfect match state continues for a predetermined N times. The period corresponding to (N-1) times from the transition from the non-perfect match state to the perfect match state is the backward protection period.

In the case where the process once shifts to the delay amount conversion process in the cinema mode, a normal delay amount determination process (calculated in step 6 of FIG. 2) is required unless the state that is not the perfect match state continues for a predetermined number of M times. (Delay amount) is not used. After temporarily shifting to the delay amount conversion processing in the cinema mode, a period corresponding to (M-1) times from the first change from the completely matched state to the state not being the completely matched state is the forward protection period.

In the forward protection period, when it is determined that the state is completely coincident, the processing shifts to the delay amount conversion processing in the cinema mode. When it is determined that the state is not the perfect match state during the forward protection period,
As described later, fine adjustment of the number of delay fields is performed.

If it is not determined in step 59 that the state is a perfect match, it is determined whether or not the current time is during the forward protection period (step 60). If it is not the forward protection period, the delay amount determined by the normal delay amount determination processing (the delay amount d3 calculated in step 6 of FIG. 2) is directly used as the delay amount (step 61).

If it is the front protection period, fine adjustment of the number of delay fields is performed (step 62). That is, the current delay field number is set so as to approach the normal delay amount (the delay amount calculated in step 6 in FIG. 2) by a predetermined field number cycle unit, for example, one field in four field cycle units. The quantity d3 is determined.

FIGS. 11 and 12 show details of the delay amount conversion processing in the cinema mode in step 65 of FIG.

First, the periodic pattern of the next field, the number of normal delay fields calculated in the current step 6 (FIG. 2), the optimum delay field number table (see Table 3 or 6), and the delay frame number table (Table 4) Or Table 7
), The optimum delay field number X0 and the delay frame number Y0 of the next field are determined (step 71).

Next, the number of delay frames Z0 (hereinafter, referred to as the delay frame) in the next field of the delay frame output in the current field.
The next field conversion delay frame number of the current field is calculated (step 72).

Next, it is determined whether or not the number of delay frames Z0 is smaller than the number of delay frames Y0 (step 73). When the number of delay frames Z0 is smaller than the number of delay frames Y0, that is, the frame corresponding to the number of delay frames of the next field calculated in step 71 corresponds to the number of delay frames converted to the next field of the current field calculated in step 72. If the old frame is older than the current frame, in other words, if the number of delayed frames changes in the direction of increasing, it is determined whether or not the delayed frame output in the current field is already displayed three times (step 7).
4).

If the delayed frame of the current field has already been displayed three times, a search process for a new frame is performed (step 75). If the delayed frame output in the current field is not already displayed three times, the same frame is searched for in order to display the same frame three times (step 76). The details of the new frame search process and the same frame search process will be described later.

In step 73, the number of delay frames Z
If 0 is equal to or greater than the number of delay frames Y0, that is, the frame corresponding to the number of delay frames of the next field calculated in step 71 is replaced by the frame corresponding to the number of delay frames converted to the next field of the current field calculated in step 72. Is the same as or newer, in other words, when the number of delayed frames changes in the direction of decreasing, the difference between the number of delayed frames Z0 and the number of delayed frames Y0 (Z
It is determined whether (0-Y0) is greater than 1 (step 77).

If the difference (Z0−Y0) between the number of delayed frames Z0 and the number of delayed frames Y0 is greater than 1, the delayed frame output in the next field is the delayed frame output in the current field. Since the frames are further separated by two or more frames (frame skipping occurs), a search process for a new frame is performed to prevent this (step 78).

If the difference (Z0-Y0) between the number of delayed frames Z0 and the number of delayed frames Y0 is 1 or less (0 or 1), that is, if no frame skipping occurs, the delay mode is set to the complete delay mode. It is determined whether the mode is the compromise delay mode (step 79). If the delay mode is the complete delay mode (step 79), the number of delay fields of the current field is set to 1 in order to determine whether or not a field return occurs.
Is determined whether or not the value obtained by adding is smaller than the number of delayed fields X0 of the next field calculated in step 72 (step 80).

If the value obtained by adding 1 to the number of delay fields of the current field is smaller than the number of delay fields X0 of the next field, a field return occurs as it is. To prevent this, search processing of the same frame must be performed. (Step 81).

On the other hand, the value obtained by adding 1 to the number of delay fields of the current field is the number of delay fields X of the next field.
When the value is 0 or more, no field return occurs.
Move to step 82.

In step 77, the number of delay frames Z
If the difference (Z0-Y0) between 0 and the number of delay frames Y0 is 1 or less and the delay mode is the compromise delay mode (step 79), it is not necessary to be involved in the presence / absence of the field return. Then, the process proceeds to step 82 without performing the process.

In addition, the above steps 75, 76 or 78
Even after the processing of the above is completed, the processing shifts to step 82. When the process proceeds from step 75, 76, 78 or 81 to step 82, the delay frame number Y1 of the next field is calculated based on the delay field number X1 of the next field obtained by the search processing of these steps. You. To calculate the number of delay frames Y1 in the next field,
The current periodic pattern and delay frame number table (see Table 4 or Table 7) are used.

On the other hand, when the process proceeds from step 79 or step 80 to step 82, the optimum delay field number X of the next field obtained in step 71 is calculated.
0, the number of delayed frames Y1 (Y1 =
Y0) is calculated.

Next, it is determined whether or not the delay frame number Z0 is equal to the delay frame number Y1 (Z0 = Y1), that is, the delay frame output in the current field is the same as the delay frame output in the next field. It is determined whether or not it is (step 83). When the number of delay frames Z0 is equal to the number of delay frames Y1, the number of delay frames output in the current field is displayed three times so that the number of delay frames output in the current field is not displayed more than three times. It is determined whether or not the operation has been performed (step 84).

When the delay frame output in the current field is displayed three times, a new frame search process is performed so that the delay frame output in the current field is not displayed more than three times. Is performed (step 85). Then, a polarity is added to the number of delay fields (step 86). Then, the process proceeds to step 12 in FIG.

In step 83, the number of delayed frames Z
If 0 is not equal to the number of delay frames Y1, it is determined whether the mode is the 2-3 pull-down mode or the 2-2 pull-down mode (step 87).

In the 2-3 pull-down mode, in order to display the same frame at least twice, it is determined whether or not the delay frame output in the current field has already been displayed twice (step 88). If the currently output delayed frame is not already displayed twice, the same frame is searched for at least twice so that the same frame is displayed (step 89).

If it is determined in step 88 that the currently output delay frame has already been displayed twice, the flow advances to step 86 to add a polarity to the number of delay fields. Then, the process proceeds to step 12 in FIG.

In step 87, when it is determined that the mode is not the 2-3 pull-down mode, or because there is no restriction that the same frame is displayed at least twice, the process in steps 88 and 89 is not performed and the process proceeds to step 86. Then, the polarity is added to the number of delay fields. Then, the process proceeds to step 12 in FIG.

FIG. 13 shows details of the search processing of the same frame in steps 76, 81 and 89 in FIGS. 11 and 12.

It is determined whether or not the number of delay fields of the current field is the maximum value "6" (step 91).
If the number of delay fields of the current field is the maximum value "6", the number of delay fields of the next field (set delay amount d3) is set to 6 (step 92).

If the number of delay fields of the current field is not the maximum value "6", that is, 5 or less, a value obtained by adding 1 to the number of delay fields of the current field, that is, the number of delay fields converted to the next field Is an even number (step 93).

If the value obtained by adding 1 to the number of delay fields of the current field is an even number, the video signal b not interpolated by the interpolation circuit 12 is selected as the delayed video signal output in the next field. As a result, the number of delay fields of the next field (set delay amount d3) is set to a value obtained by adding 1 to the number of delay fields of the current field (step 94).

If the value obtained by adding 1 to the number of delay fields of the current field is an odd number, a delay frame (frame [current delay) corresponding to the number of delay fields obtained by adding 1 to the number of delay fields of the current field. Number of fields +1])
Is the same as the frame output in the current field (frame [current delay field number]) (step 95).

If both delay frames are the same, the number of delay fields of the next field (set delay amount d3) is set to the number of delay fields of the current field (step 96). As a result, in the next field, a video signal not interpolated by the interpolation circuit 12 is selected, and the same frame as that output in the current field is output.

In step 95, if the two frames are different, to obtain the same frame, the number of delay fields of the next field (set delay amount d3) is set to a value obtained by adding 1 to the number of delay fields of the current field. (Step 9
4). In this case, in the next field, the video signal c interpolated by the interpolation circuit 12 is selected.

FIG. 14 shows the details of the new frame search process in steps 75, 78 and 85 in FIGS. 11 and 12.

The target number of delay frames in the next field is set to be one less than the number of delay frames in the current field (step 101). That is, a frame that is one newer than the delayed frame output in the current field is set as a target frame.

Next, the absolute value of the number of delay fields in the current field is set to i (step 12).

Next, it is determined whether or not i = 0 (step 103). If i = 0, that is, if the number of delay fields in the current field is 0, the number of delay fields of the next field (set delay amount d3) is set to 0 (step 104).

[0200] If i is not equal to 0, in the next field, the delayed frame whose number of delayed fields is i is set to frame [i] (step 105).

Next, it is determined whether or not the target frame set in step 101 is the same as frame [i] (step 106).

If the target frame is not the same as frame [i], i is changed to i-1 (step 10).
7). Then, the process returns to step 103. I after change is 0
If so, the process proceeds to step 104, where the number of delay fields of the next field (set delay amount d3) is set to 0.

If i after the change is 0, step 105
In the next field, the number of delay fields is i
Is a frame [i]. Then, the target frame set in step 101 is frame [i]
It is determined whether it is the same as (step 106).

That is, steps 103, 105, and 106
And 107, the same field as the target frame is searched for in the field between the delay field output in the current field and the field of the main image currently being output (the field of the input video signal). I have.

If it is determined in step 106 that the target frame is the same as frame [i], it is determined whether i is an even number (step 108). If i is an even number, the number of delay fields of the next field is set to i
However, since the video signal b not interpolated by the interpolation circuit 12 is selected, the delay field number (set delay amount d3) of the next field is set to i (step 10).
9).

If it is determined in step 108 that i is an odd number, the delay frame when the number of delay fields is i-1 in the next field is frame [i-
1] (step 110). Then, it is determined whether or not the target frame is the same as the frame [i-1] (step 111).

If the target frame is the same as the frame [i-1], the number of delay fields in the next field is set to i-1 in order to output the video signal b not interpolated by the interpolation circuit 12 in the next field. However, since the target frame can be selected, the delay field number (set delay amount d3) of the next field is set to i-1 (step 112). If the target frame is not the same as the frame [i-1], the number of delay fields (set delay amount d3) of the next field is set to i (step 109). In this case, the video signal c interpolated by the interpolation circuit 12 in the next field is output.

FIG. 32 shows details of the scene change detection / control processing of FIG.

By the way, when a three-dimensional image is generated from a two-dimensional image by a frame delay method, when an image is changed from a currently displayed image scene to a completely different scene, a new scene is displayed in the left-eye image. On the other hand, in the right-eye image delayed by the frame, the image of the previous scene is displayed. For this reason, there is a problem that the observer feels strange at the transition of the scene.

The scene change detection / control processing is performed by detecting a scene change and, when a scene change is detected, setting all the delay data (d1, d2, d3, Pd) and the delay amount history data to 0. This is a process for solving the above problem.

First, a plurality of motion vector detection areas a,
Among the b, c,..., the magnitude (motion vector amount) of the horizontal direction motion vector in the current field in one predetermined area a is obtained (step 121). Next, the motion vector amount acquired in step 1 is added to the existing value (step 122). The initial value of the existing value is 0.

Next, returning to step 121, the magnitude (motion vector amount) of the horizontal direction motion vector in the current field in another motion vector detection area b is obtained (step 121). Then, the obtained motion vector amount is added to the existing value, that is, the motion vector amount in the motion vector detection area a (step 122).
In this way, the sum of the motion vector amounts in the current field of all the motion vector detection areas is calculated (steps 121, 122, 123).

Assuming that the current field is the t-th field (t is a natural number) and the sum of the motion vector amounts in the current field is MX (t), MX (t) is expressed by the following equation.

[0214]

(Equation 4)

When the sum of the motion vector amounts in the current field of all the motion vector detection areas is calculated (YES in step 123), the average value of the sum of the motion vector amounts up to two fields before is obtained (step 12).
4).

More specifically, the average value M of the sum MX (t-3) of the motion vectors of the video three fields before and the sum MX (t-3) of the motion vectors of the video two fields before is obtained.
Xave (t-2) is the average value MXave (t-2) of the sum of the motion vector amounts up to two fields before. MXave (t−2) is obtained based on the following equation.

[0219]

(Equation 5)

However, in this example, S = 2. In this embodiment, the average value MXave (t) of the total sum MX (t−3) of the motion vector amounts of the video three fields before and the total sum MX (t−3) of the motion vectors of the video two fields before.
-2) is the average value of the sum of the motion vector amounts up to two fields before, but more than that, for example, eight fields from two fields to nine fields before (S = 8)
May be the average of the sum of the motion vector amounts up to two fields before.

Next, the sum MX (t) of the motion vectors of the current field obtained in step 2 is secured in the save area of the RAM 22 (step 125). In the save area, a total sum of motion vector amounts for each field up to several fields before is secured.

Next, the sum MX (t−t) of the motion vector amounts of the field one field before, which is secured in the save area.
1) is the average value MXave (t
It is determined whether the value is greater than -1) by a predetermined value (for example, 40 pixels) (step 126). Sum MX (t-
If 1) is not larger than the average value MXave (t-1) by a predetermined value or more, this processing ends.

The sum MX (t-1) is equal to the average value MXave.
If it is greater than (t-1) by a predetermined value or more, the sum MX (t-1) of the motion vectors of the previous field secured in the save area is calculated as the motion vector of the current field secured in the save area. It is determined whether or not the sum of the amounts MX (t) is greater than a predetermined value (for example, 40 pixels) (step 127). The sum MX (t-1) is the sum MX (t)
If it is not larger than the predetermined value, this process ends.

When the sum MX (t-1) is larger than the sum MX (t) by a predetermined value or more, the image two fields before and the one
It is determined that there is a scene break between the video before the field and the process proceeds to step 128.

That is, MX (t-1) >> MXave
(T-1) and MX (t-1) >> MX (t)
When Expression 6 is satisfied that satisfies the condition of (t-2), it is determined that the motion vector amount has rapidly increased, and
It is determined that the scene has changed between the field and the (t-1) th field.

[0224]

(Equation 6)

In step 128, the delay amount (setting) is set so that both the left-eye video signal and the right-eye video signal output from the video switching circuit 13 become two-dimensional video signals input to the input terminal 1. The delay amount d3) is set to 0. Thereby, at the transition of the scene, the left and right video signals output from the output terminals 2 and 3 become the same video signal,
Since the left and right images projected on the monitor are the same image, the observer does not feel discomfort.

Thereafter, all the sums of the motion vector amounts of the respective fields secured in the save area are cleared (step 129).

FIG. 33 shows a modification of the processing after step 30 in FIG. 3, that is, steps 31 and 32.

At step 30, it is determined whether or not the target delay amount Pd and the currently set delay amount (set delay amount d3) match. Target delay amount Pd and set delay amount d
If 3 does not match, it is determined whether or not the set delay amount d3 has already continued for two fields (step 33).

If the set delay d3 has already continued for two fields, it is determined whether the number of continuous fields of the set delay d3 is an odd multiple of 2 or an even multiple of 2 (step 34).

If the number of continuous fields of the set delay amount d3 is an even multiple of 2, the following processing is performed (step 35), and the process proceeds to step 7 in FIG.
That is, the set delay amount d3 is changed by 1 in a direction approaching the target delay amount Pd (d3 = d3 ± 1).

The horizontal shift amount is calculated by the phase control circuits 14 and 15 such that the movement amount of the subject between the fields corresponding to the change in the set delay amount d3 is reduced to half. This horizontal shift amount is calculated based on the average value of the motion vector between the fields corresponding to the change of the set delay amount d3.
In other words, the horizontal shift amount is calculated such that the set delay amount d3 is changed in a direction approaching the target delay amount Pd by substantially 0.5 fields. The calculation of the horizontal shift amount is preferably performed only when the average value of the motion vector between the fields corresponding to the change of the set delay amount d3 is equal to or more than a predetermined value.

In step 34, when it is determined that the number of continuous fields of the set delay amount d3 is an odd multiple of 2, the horizontal movement amount is equal to the original value, that is, the operation / operation amount.
The value is returned to the set value set on the display unit 23 (step 36). Then, the process proceeds to step 7 in FIG.

If the target delay amount and the current delay amount d3 match in step 30 or if the current delay amount d3 has not already continued for two fields in step 33, the delay amount is The process proceeds to step 7 in FIG. 2 without any change.

That is, also in this modification, the set delay amount d3
Is controlled so as to approach the target delay amount Pd in units of four fields and one field at a time. For example, FIG.
As shown in FIG. 4, the set delay amount d3 is 0, 0, 0, 0,
.., 1, 1, 1, 1, 2, 2, 2, 2,...
However, in this modification, the phase control circuits 14, 1
5 is adjusted so that the movement amount of the subject between the fields corresponding to the change in the set delay amount d3 is halved. Therefore, as shown in FIG. 34, the horizontal shift amount is substantially equal to the set delay amount d3. Are 0, 0, 0, 0, 0.5, 0.5, 1,
It is possible to obtain the same effect as in the case of changing as 1, 1.5, 1.5, 2, 2,.

Accordingly, in this modified example, the obtained 3
The pop-out amount of the three-dimensional image changes smoothly, and an easily viewable three-dimensional image is obtained.

[0236]

According to the present invention, a stable three-dimensional effect can be obtained and the movement of a moving object can be made smooth. A method of converting a two-dimensional image into a three-dimensional image, which can adjust the image quality, is realized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a 2D / 3D conversion device.

FIG. 2 is a flowchart showing an overall procedure of 2D / 3D conversion processing by a CPU.

FIG. 3 is a flowchart illustrating a detailed procedure of a delay amount calculation process in step 6 of FIG. 2;

FIG. 4 is a graph showing a relationship between a motion vector average value and a first delay amount.

FIG. 5 is a schematic diagram for explaining how to derive a relational expression for obtaining a first delay amount from a motion vector average value.

FIG. 6 is a time chart showing how the target delay amount is changed when all three second delay amounts match.

FIG. 7 is a time chart showing how the target delay amount is changed when all three second delay amounts become larger than the current target delay amount.

FIG. 8 is a flowchart showing a detailed procedure of a cinema mode detection / control process in step 11 of FIG. 2;

FIG. 9 is a flowchart showing a detailed procedure of a cinema mode detection / control process in step 11 of FIG. 2;

FIG. 10 is a flowchart showing a detailed procedure of a cinema mode detection / control process in step 11 of FIG. 2;

FIG. 11 is a flowchart illustrating a detailed procedure of a delay amount conversion process in step 65 of FIG. 10;

FIG. 12 is a flowchart showing a detailed procedure of a delay amount conversion process in step 65 of FIG. 10;

FIG. 13 shows steps 76 and 81 of FIGS. 11 and 12;
It is a flowchart which shows the detailed procedure of the search processing of the same frame of 89.

FIG. 14 shows steps 75 and 88 of FIGS. 11 and 12;
It is a flowchart which shows the detailed procedure of a search process of 85 new frames.

FIG. 15 is a graph showing a change in a motion vector with respect to a field of an image subjected to telecine conversion by a 2-3 pull-down method.

FIG. 16 is a schematic diagram showing, for each field, a video subjected to telecine conversion by a 2-3 pull-down method.

17 is an output signal when the input signal is the signal shown in FIG. 16 and the number of delay fields in a normal state is 1 to 3 (-1 to -3) when the complete delay mode is set; It is a schematic diagram which shows a delay image.

FIG. 18 shows a case where the input signal is the signal shown in FIG. 16 and the full delay mode is set, and is actually output when the number of delay fields in a normal state is 4 to 6 (−4 to −6). It is a schematic diagram which shows a delay image.

FIG. 19 is a diagram showing a case where the input signal is the signal of FIG. 16 and the full delay mode is set;
It is a schematic diagram which shows the delay image actually output when shifting to a frame delay.

20 is a diagram showing a case where the input signal is the signal shown in FIG. 16 and the full delay mode is set;
It is a schematic diagram which shows the delay image actually output when shifting to a frame delay.

21 is an output signal when the input signal is the signal of FIG. 16 and the number of delay fields in a normal state is 1 to 3 (-1 to -3) in a case where the compromise delay mode is set; It is a schematic diagram which shows a delay image.

FIG. 22 shows a case where the input signal is the signal shown in FIG. 16 and the compromise delay mode is set; when the number of delay fields in a normal state is 4 to 6 (−4 to −6), the signal is actually output. It is a schematic diagram which shows a delay image.

FIG. 23 shows a case where the input signal is the signal shown in FIG. 16 and the compromise delay mode is set;
It is a schematic diagram which shows the delay image actually output when shifting to a frame delay.

FIG. 24 shows a case where the input signal is the signal of FIG. 16 and the compromise delay mode is set;
It is a schematic diagram which shows the delay image actually output when shifting to a frame delay.

FIG. 25 is a schematic diagram showing, for each field, a video subjected to telecine conversion by the 2-2 pull-down method.

26 shows a case where the input signal is the signal shown in FIG. 25 and the number of delay fields in a normal state is 1-2 (-1 to -2).
FIG. 9 is a schematic diagram showing a delayed image actually output at the time of FIG.

FIG. 27 shows a case where the input signal is the signal shown in FIG. 25 and the full delay mode is set, and is actually output when the number of delay fields in the normal state is 3 to 4 (−3 to −4). It is a schematic diagram which shows a delay image.

28 is an actual output when the input signal is the signal shown in FIG. 25 and the number of delay fields in a normal state is 5 to 6 (−5 to −6) when the complete delay mode is set. It is a schematic diagram which shows a delay image.

29 shows a case where the number of delay frames is 0 when the input signal is the signal of FIG. 25 and the complete delay mode is set.
It is a schematic diagram which shows the delay image actually output when changing to -1-2-3.

30 shows a case where the number of delay frames is 0 when the input signal is the signal of FIG. 25 and the compromise delay mode is set.
It is a schematic diagram which shows the delay image actually output when changing to -1-2-3.

31 is a schematic diagram showing a delayed image actually output when the number of delay frames changes to 3-2-1-0 when the input signal is the signal of FIG. 25;

FIG. 32 shows the scene change detection / step S12 of FIG.
It is a flowchart which shows the detailed procedure of a control process.

FIG. 33 is a flowchart showing a modified example of the processing after step 30 in FIG. 3;

FIG. 34 is a time chart showing a substantial change in delay amount when the processing of the modification of FIG. 33 is performed.

FIG. 35 is a diagram illustrating a relationship between a delay amount and a phase shift amount with respect to a volume set by the stereoscopic effect adjuster.

FIG. 36 is a graph showing a relationship between a motion vector corresponding to volume “0” and a delay amount.

FIG. 37 is a graph showing a relationship between a motion vector corresponding to volume “−4” and a delay amount.

FIG. 38 is a graph showing a relationship between a motion vector corresponding to volume “4” and a delay amount.

FIG. 39 shows only field delays when the volume is set to “−4”, “0”, and “4” for an image having the same motion vector. FIG. 9 is a schematic diagram showing changes in the stereoscopic image positions of a gazing object and a stationary object serving as a background, according to the first embodiment.

FIG. 40 shows the field delay and the field delay when the volume is set to “−4”, “0”, and “4” for the same motion vector. It is a schematic diagram which shows the change of the three-dimensional image position of the gazing object and the stationary object which is a background by phase shift.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Field memory 12 Interpolation circuit 13 Video switching circuit 14, 15 Phase control circuit 20 CPU 21 ROM 22 RAM 23 Operation / display part 24 Memory control circuit

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Seiji Okada 2-5-5 Keihanhondori, Moriguchi-shi, Osaka Inside Sanyo Electric Co., Ltd. (72) Inventor Susumu Tanase 2-5-2 Keihanhondori, Moriguchi-shi, Osaka No. 5 Inside Sanyo Electric Co., Ltd. (72) Takashi Ikeda 2-5-5 Keihanhondori, Moriguchi-shi, Osaka Prefecture Inside of Sanyo Electric Co., Ltd. Minor Takahashi 2-5-2 Keihanhondori, Moriguchi-shi, Osaka Prefecture No. 5 Sanyo Electric Co., Ltd. (72) Inventor Shugo Yamashita 2-5-5 Keihanhondori, Moriguchi-shi, Osaka Sanyo Electric Co., Ltd. (58) Field surveyed (Int. Cl. ⁶ , DB name) H04N 13/00-15/00

Claims (4)

(57) [Claims] 1. A method of converting a two-dimensional image into a three-dimensional image by generating a main video signal and a sub-video signal delayed from the main video signal from the two-dimensional video signal. A plurality of tables indicating the relationship between the movement speed and the delay amount are stored in advance, and input means for designating a predetermined table from among the plurality of tables is provided. Is calculated for each field, and the delay amount is calculated based on the table specified by the input means and the speed of movement of the current field of the main video signal. Based on the calculated delay amount, the delay amount in the next field is calculated. A method of converting a two-dimensional video into a three-dimensional video, wherein a delay amount of a sub-video signal with respect to a main video signal is determined. 2. A main video signal and a sub video signal delayed with respect to the main video signal are generated from a two-dimensional video signal, and a phase in a horizontal direction is obtained with respect to the obtained main video and sub video signals. In the method of converting a two-dimensional image into a three-dimensional image by performing a shift, a plurality of types of tables indicating the relationship between the speed of image movement and the amount of delay and a phase shift amount predetermined for each table are compared. In addition to storing in advance, a single input means for designating a predetermined table from among a plurality of types of tables and at the same time designating a corresponding phase shift amount is provided, and the speed of movement of the main video signal is reduced. The delay amount is calculated for each field, and the delay amount is calculated based on the table specified by the input means and the moving speed of the current field of the main video signal. Determining the delay amount of the sub-picture signal to the main video signal at node, the next field, with respect to the sub-picture signal delayed by the delay amount determined above from the main video signal and the main video signal,
A method for converting a two-dimensional video into a three-dimensional video, wherein the two-dimensional video is subjected to a phase shift according to a phase shift amount designated by an input means. 3. A main video signal and a sub video signal delayed with respect to the main video signal are generated from the two-dimensional video signal.
In the method for converting a two-dimensional image into a three-dimensional image, the method comprises the steps of:
A relational expression for calculating the amount of delay is created in advance, and input means for inputting stereoscopic effect adjustment data is provided, and the speed of movement of the main video signal is calculated for each field, and is input by the input means. The amount of delay is determined based on the adjusted data, the speed of movement of the current field of the main video signal, and the relational expression created in advance, and based on the determined amount of delay, A method of converting a two-dimensional image into a three-dimensional image, characterized by determining a delay amount of a sub-image signal. 4. A method for converting a two-dimensional image into a three-dimensional video by generating a main video signal and a sub-video signal delayed from the main video signal from the two-dimensional video signal. A table indicating the relationship between the movement speed and the delay amount is stored in advance and input means for inputting delay amount magnification data is provided, and the movement speed of the main video signal is calculated for each field. The delay amount is calculated based on the speed of movement of the current field of the video signal and the table, and the delay amount magnification data input by the input means is integrated with the calculated delay amount. A method of converting a two-dimensional video into a three-dimensional video, wherein a delay amount of a sub-video signal with respect to a main video signal in a field is determined.