JP2001069502A - Video image transmission terminal and video image reception terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize scalability of coded data of an object video image, without causing deterioration of the quality for important part of a video. SOLUTION: A priority setting section 4 of the video transmission terminal 102 sets a priority area of an important part of area extraction data 110 extracted by an area extract section 108 from the object of a digital video image, obtained from a video source 104 and an object video image coding section 120 encodes the digital video image 106, with respect to the set priority area to realize scalability of coded data 122. The coded data 122 can be displayed without deteriorating the quality of the important part, when the coded data 122 are reproduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video transmitting terminal for realizing scalability without deteriorating the quality of an object video when generating an object video by an MPEG4 encoding method or the like, and transmitting from the video transmitting terminal. The present invention relates to a video receiving terminal that receives encoded object video data.

[0002]

2. Description of the Related Art Heretofore, an area has been extracted from an image captured by a camera using a videophone or the like or an image read from a file, and the contour of an object or an object of interest in the image has been extracted. A technique of cutting out a video within the contour described and encoding it as an object has become widespread. As such an object encoding technology, the MPEG4 object video encoding system has been standardized as a prominent technology.

FIG. 22 is a configuration diagram of a video transmission terminal in a conventional videophone system, and FIG. 23 is a configuration diagram of a video reception terminal.

In FIG. 22, a video transmitting terminal 102 generates a digital video 106 from a video including a subject obtained from a video source 104. The video source 104 is, for example, a video captured by a camera or a video read from a file. The digital video 106 is applied to the region extracting unit 108, and extracts a transgression for a region of a subject (for example, a human image) to generate region extraction data 110.

[0005] Further, an area extracting unit 112 obtains a digital image 116 from another image source 114, extracts another object (such as a background), and generates area extraction data 118.

On the other hand, the digital video 106 and the region extraction data 110 are applied to an object video encoding unit 120. The object video encoding unit 120 performs object encoding on the region of the digital video 106 specified by the region extraction data 110, and generates encoded data 122. At the same time, the object video encoding unit 124 also uses the digital video 116 and the region extraction data 118 to
The coded data 126 is generated.

The control unit 128 sends scene configuration information 130
Is output. The scene configuration information 130 is information indicating at which coordinates on the display screen each object is located on the time axis.

[0008] The coded data 122, the coded data 126, and the scene configuration information 130 are transmitted to the multiplexing unit 13.
2, and are output to the outside as serial data 134.

[0009] As shown in FIG. 23, the serial data 134 is transmitted via a transmission line 136 to the separating unit 1 of the video receiving terminal 138.
40 is applied. In the separating unit 140, the encoded data 14
2. The encoded data 144 and the scene configuration information 146 are separated.

[0010] The encoded data 142 is applied to an object video decoding unit 148, and the original object video 150 is reproduced by the object video decoding unit 148. The encoded data 144 is stored in the object video decoding unit 1.
52, and the object video decoding unit 152 reproduces the object video 154.

The control unit 156 has scene configuration information 146
Is input and decoded, and the object video 150 and the object video 154 applied to the synthesizing unit 158 are arranged at the designated coordinates every time. Then, the synthesizing unit 158
These are output as composite video data 160, and the composite video data 160 is applied to the monitor 162,
2, the object video 150 and the object video 154
Is displayed.

FIG. 24 is a display diagram of an object image displayed on a conventional monitor 162.

Referring to FIG. 24, a person image 164 is assigned as the object image 150, and a background 166 is assigned as the object image 154. Here, noise and the like may increase in the transmission path 136, and the transmission capability may decrease. When the transmission capacity decreases, the data capacity of the serial data 134 needs to be reduced. In the MPEG4 object encoding method, spatial scalability and temporal scalability are prepared as techniques for reducing encoded data.
In order to execute the scalability function, the scalability conversion unit 1 is included in the object video encoding unit 120 in FIG.
68, and the object video encoding unit 124 has a scalability conversion unit 170.

FIG. 25 is a diagram for explaining an example of conventional spatial scalability. First, the human image 172 is encoded by the object video encoding unit 120 as shown in FIG. Then, when it becomes necessary to reduce the data amount of the coded data 122, the scalability conversion unit 168 thins out the coded data of the human image 172 by halving it, and reduces the coded data as shown in FIG. As shown, a person image 174 is generated. Then, the coded data 122 of the human image 174 is sent to the video receiving terminal 138 and is decoded by the object video decoding unit 148. An interpolation process is performed on the reduced data to generate an object video 150 that becomes a human image 178 of the original size as shown in FIG. Similarly, if necessary, the background object is reduced by the scalability conversion unit 170, and the scalability reproduction unit 180 performs an interpolation process to reproduce the original object video 154.

FIG. 26 is a diagram for explaining an example of conventional time scalability. When scalability is not performed, the human image moves for each of the frames a, b, c, d, and e, and FIG. 26 (a), (b), (c), (d),
As shown in (e), the images are encoded as a person image 182, a person image 184, a person image 186, a person image 188, and a person image 190. However, the scalability conversion unit 168 encodes the human image 192, the human image 194, and the human image 196 by thinning out the frames and encoding the encoded data 122.
Decrease. Then, the scalability reproduction unit 176 repeatedly uses the human image 192 in the frame b and the human image 194 in the frame d to reproduce the object video 150 temporally.

[0016]

In the spatial scalability described with reference to FIG. 25, when the human image 172 is compressed into the human image 174, the amount of information is lost, and the image is enlarged to the original size like the human image 178. However, the person image 178 is the person image 172
The resolution is reduced compared to.

In the time scalability described with reference to FIG. 26, when the movement from the person image 182 to the person image 190 is large, the person image 184 in the frame b and the person image 1 in the frame d
88 is lost and the movement becomes awkward if replaced with the person image of the previous frame.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and is intended to realize the scalability of encoded data obtained by object video encoding without deteriorating the quality of an important part of a video. It is an object to provide a video transmission terminal and a video transmission terminal capable of performing the following.

[0019]

According to a first aspect of the present invention, there is provided a video transmitting terminal for extracting an area of a subject from a video including the subject, and the extracting means. Is characterized by comprising: setting means for setting a priority area for an area to be extracted; and coding means for performing object video coding from video for the priority area set by the setting means.

According to the first aspect of the invention, for an object such as a human image, an important portion such as a face portion is preferentially left, so that when the data amount of the encoded data is compressed, the face portion or the like is reduced. It does not lower the resolution of important parts. Therefore, the resolution does not decrease in the entire object image as in the case of spatial scalability. For example, a portion below a low-priority face is cut according to the required degree of compression, so that it can be changed according to the situation. Scalability is achieved. Further, unlike the temporal scalability, it is not necessary to delete the encoded data in units of frames, so that the reproduced object video does not have awkward movement.

According to a second aspect of the present invention, there is provided a video transmitting terminal for extracting an area of a subject from a video including the subject, and setting a priority area for the area extracted by the extracting means. Means for generating priority area information, means for performing object video encoding on the priority area extracted by the extraction means from the video to generate encoded data, and means for transmitting the encoded data and the priority area information It is characterized by the following.

According to a third aspect of the present invention, there is provided a video receiving terminal for receiving coded data transmitted by the video transmitting terminal and priority area information, and coded data received by the receiving means. Reproduction means for decoding the object video by decoding the video, display means for displaying the object video reproduced by the reproduction means, and a predetermined means designated by using the priority area information when the display means displays the object video. Video superimposing means for superimposing another video on the area.

According to the video transmitting terminal according to the second aspect of the present invention and the video receiving terminal according to the third aspect of the present invention, the video receiving terminal transmits the video in accordance with the priority. A low priority part can be hidden while leaving an important part, and another arbitrary image can be superimposed and displayed.

According to a fourth aspect of the present invention, there is provided a video transmitting terminal for extracting an area of a subject from a video including the subject, and setting a priority area for the area extracted by the extracting means. Means, selecting means for selecting a predetermined priority area from the priority areas set by the setting means, coding means for performing object video coding from video for the predetermined priority area selected by the selecting means, The transmitting means for transmitting the encoded data to be encoded by the means, and the control means for controlling the selecting means in order to control the rate of the encoded data transmitted by the transmitting means.

According to the fourth aspect of the present invention, in controlling the rate of the encoded data, complicated processing such as changing the quantization precision or the frame rate of the object video is not performed as in the related art. Only the simple processing of changing only the image can be performed, and the quality of the important part of the image is not deteriorated.

Furthermore, the video transmitting terminal according to the present invention is characterized in that an extracting means for extracting an area of a subject from a video including a subject, and a setting for setting a priority area for the area extracted by the extracting means. Means, selecting means for selecting a predetermined priority area from the priority areas set by the setting means, coding means for performing object video coding from video for the predetermined priority area selected by the selecting means, Error correction coding means for performing error correction coding on the coded data to be coded by the means, transmission means for transmitting the coded data to which the error correction coding means performs the error correction coding, and a video receiving terminal. Acquiring means for acquiring information on the line quality of the transmission path between the control means, and controlling the selecting means and the error correction coding means based on the information on the line quality acquired by the acquiring means. Characterized by comprising a control means for setting a combination of data amount of the coded data and error correction coded data amount of the data.

According to the fifth aspect of the present invention, the combination of the data amount of the encoded data and the data amount of the error correction encoded data can be changed according to the line quality of the transmission path.
Thus, even when the data amount of the encoded data is changed, the quality of the important portion of the video is not deteriorated.

[0028]

Embodiments of the present invention will be described below in detail with reference to the drawings. In the following drawings, the same reference numerals indicate the same parts or corresponding parts including the figures showing the conventional example.

(First Embodiment) FIG. 1 is a block diagram of a video transmitting terminal according to a first embodiment of the present invention, and FIG. 2 is a block diagram of the video receiving terminal.

Referring to FIG. 1, a video transmitting terminal 102 receives a digital video 106 obtained from a video source 104
The region extraction unit 108 extracts a region of the subject, and outputs region extraction data 110.

When the subject is a person image, the subject can be extracted by, for example, the technique disclosed in Japanese Patent Application Laid-Open No. Hei 7-29014. This known example is a method of detecting the movement of a subject from a digital video 106 obtained from a video source 104, detecting the cheek coordinates of the subject when the subject moves sufficiently, and extracting the feature information of the subject from the cheek coordinates. . FIG. 3 is a display diagram in which cheek coordinates of a subject are detected according to a conventionally known example. As shown in FIG.
, The size of the subject of the human image 2 can be statistically measured.

The area extraction data 110 is applied to the priority setting section 4. FIG. 4 is a diagram for explaining the processing of the priority setting unit 4 in the present embodiment. As shown in FIG. 4, with respect to the region extraction data 110, that is, the human image 6, the human image 6 is cut at lengths a, b, and c from the top of the head. That is, the region extraction data cut by the length a from the top of the head covers only the face portion. The region extraction data cut by the length b from the top of the head includes a part of the face and a little upper body.
The region extraction data cut by the length c from the top of the head includes the face portion and the majority of the upper body.

FIG. 5 is a view for explaining the region extraction data 110 in this embodiment, which is cut in order while leaving important portions. According to FIG. 5, the human image 8 is only a face part, and the encoding amount is small when encoded. That is, the minimum necessary face portion is coded at the conventional coding rate, and the portion other than the face is cut to reduce the coding amount. The human image 10 has a little upper body left, and the coding amount is larger than that of the human image 8. The person image 12 has the largest coding amount in a state where the region is extracted by the region extracting unit 108. That is, in the priority setting unit 4, a stepwise encoding amount is set based on the boundaries a, b, and c from the region extraction data 110 and becomes the priority region data 14.

Based on the selection information 18 input from the control unit 128 in the area dividing unit 16, the priority area data 1
From 4, a required person image is selected from the person image 8, the person image 10, and the person image 12, and applied to the object video encoding unit 120 as the processed area data 20.

On the other hand, the digital video 106 is applied to the object video encoding unit 120, and the digital video 106 in the area designated by the processed area data 20 is subjected to, for example, an object video encoding process according to the MPEG4 system. It is output as encoded data 122.

Similarly, other subjects (such as background 22 in FIG. 3)
, A digital video 116 is obtained from the video source 114. An area extraction unit 112 calculates area extraction data 118 from the digital video 116. From the region extraction data 118, the priority setting unit 24 determines the boundary of the region to be cut from the important region in the region extraction data 118 in the same manner as described above, and sets it as the priority region data 26.

The area dividing section 28 obtains the selection information 30 from the control section 128, selects a required video area from the priority area data 26, and applies it to the object video encoding section 124 as processed area data 32. .

The object video coding unit 124 generates the coded data 126 using the processed area data 32 and the digital video 116.

Encoded data 122, encoded data 12
6 and the scene configuration information 130 are multiplexed by the multiplexing unit 132 and output from the video transmission terminal 102 to the outside as serial data 134.

As shown in FIG. 2, the serial data 134 is transmitted via the transmission line 136 to the separating unit 14 of the video receiving terminal 138.
0 is applied. In the separating unit 140, the encoded data 14
2. The encoded data 144 and the scene configuration information 146 are separated.

The coded data 142 is applied to the object video decoding unit 148, and the object video decoding unit 1
At 48, an object video 150 is generated. The encoded data 144 is applied to the object video decoding unit 152, and the object video decoding unit 152 generates the object video 154. On the other hand, the scene configuration information 146 is applied to the control unit 156.

The object image 150 and the object image 154 are applied to the synthesizing unit 158 and the control unit 156
Arranges the object video 150 and the object video 154 at the coordinates specified by the scene configuration information 146, and outputs the composite video data 160. The monitor 162 inputs and displays the composite video data 160.

Here, the generation of the priority area data 14, 26, which has been automatically performed by the priority setting sections 4, 24, may be manually performed by the setting section 34.

FIGS. 6A, 6B, and 6C are display diagrams in which a human image and a background are displayed on the monitor 162 in this embodiment. FIG. 6A shows a person image 8 superimposed on a background 22. FIG. 6B shows the person image 10 superimposed on the background 22. FIG. 6C shows the person image 12 overlaid on the background 22. As can be seen from FIGS. 6A, 6B, and 6C, the coding amount varies depending on the size of the human image 8, the human image 10, and the human image 12, and the purpose of scalability becomes possible. Then, the encoding amount can be adjusted by cutting the image of the lower body part of low importance without cutting the most important face part in the person image.

In the above-described example, the coding amount is reduced by cutting as necessary the portions of low importance from the portions of high importance while leaving the portions of high importance. Here, the boundary to be cut by the priority setting unit 4 for the region extraction data 110 does not need to be limited to a straight line. FIG. 7 is a diagram showing boundaries of priorities for region extraction data of an arbitrary shape in the present embodiment. As shown in FIG. 7, the boundary that separates the priorities does not need to be a straight line as described above. Any curve may be used. According to FIG. 7, the area delimited by a is the area with the highest priority, and the priority becomes lower in the order of b and c. That is, depending on the required encoding amount, a,
What is necessary is to select the region extraction data based on the boundary between b and c.

FIG. 8 is a display diagram in which the lower side of the background 22 is divided by priority according to the present embodiment. That is, the boundary a has the highest priority, and b and c have the lowest priority. When displaying the background 22 on the video receiving terminal 138, if the video on the lower side is not so important, the video can be cut from the lower side as necessary to reduce the encoding amount.

FIG. 9 is a display diagram in which both ends of the background 22 are divided according to priority in the present embodiment. FIG. 10 is a display diagram in which the lower side and both ends of the background 22 are divided according to priority in the present embodiment. As described above, with respect to the background 22, an area which can be cut at the left, right, upper, and lower ends of the video can be cut to reduce the encoding amount.

FIG. 11 is a display diagram in which a low-priority region is cut out from the background image and the human image in the present embodiment and then arranged. As shown in FIG. 11, by cutting the lower side of the human image 36 and also cutting the lower side of the background 22, and arranging the human image 36 and the background 22, a surplus area 38 is generated at the lower side of the screen. Since the areas of the human image 36 and the background 22 are reduced, the transmission capacity has a margin, so that another object can be sent from the video transmission terminal 102 to the video reception terminal 138 and displayed in the surplus area 38.

Further, the video receiving terminal 138 stores another object video 42 from the video source 40 in the surplus area 38.
Is read, and the combining unit 158 reads the object video 15
0, may be arranged together with the object video 154 and displayed on the monitor 162 as the composite video data 160.
Further, instead of the object video 42, text or an arbitrary video may be displayed in the surplus area 38.

As described above, in the present embodiment, when the amount of code transmission on the transmission line 136 is reduced, the amount of code is reduced by cutting unimportant portions such as a human image or a background, and A surplus area 38 is generated in the display area of the monitor 162 at 138. Then, an arbitrary object video or text can be displayed on the surplus area 38 by the video receiving terminal 138. Also,
Even when the transmission amount of the transmission path 136 does not decrease, another object video to be displayed in the surplus area 38 can be sent from the video transmission terminal 102 and displayed in the surplus area 38.

(Second Embodiment) Next, a second embodiment of the present invention will be described.

In this embodiment, an object image such as a person image and a background is sent from a video transmitting terminal to a video receiving terminal without being cut, and priority area data such as boundaries a, b, and c of priority areas are also received. The feature is to send to the terminal.

FIG. 12 is a block diagram of the video transmitting terminal in the present embodiment, and FIG. 13 is a block diagram of the video receiving terminal.

FIG. 14 is a display diagram in which regions of the human image 48 are divided according to priority in the present embodiment. As for the human image 48, an area extracting unit 108 for the digital video 106 read from the video source 104.
Is performed to generate region extraction data 110.
The object image encoding unit 120 encodes the region extraction data 110 and the digital image 106 to generate encoded data 122.

The encoded data 122 is a person image having the largest area of the boundary c in the person image 48 according to FIG. The area extraction data 110 is stored in the priority setting unit 4 as shown in FIG.
The boundaries between a, b, and c are determined according to the priority.

The present embodiment is characterized in that the boundaries determined as a, b, and c are sent to the video receiving terminal 138 as the priority area data 14. In the present embodiment, the distance from the top of the person image 48, that is, a, b, and c may be used as the priority area data 14.

FIG. 15 is a display diagram in which the background 22 is divided into areas according to priority in the present embodiment. The background 22 as shown in FIG. 15 is read from the video source 114, and distances a, b, and c from the upper end of the display screen are set as priority area data 26.

The video receiving terminal 138 applies the priority area data 50 and the priority area data 52 to the control unit 156.

FIG. 16 is a display diagram in which the person image 54 and the background 22 in this embodiment are displayed according to the priority. As shown in FIG. 16, the control unit 156 superimposes and displays the human image 54 and the background 22 according to the priority according to the scene configuration information 146. That is, the boundaries a and b indicating the priority of the person image 54 and the boundaries a and b indicating the priority of the background 22 are in the vertical direction,
Place them so that they match. By arranging the human image 54 and the background 22 in this way, the surplus area 38 is classified into an area below the boundary a or an area below the boundary b.

Then, the object video 42 can be read from the video source 40 and displayed over the surplus area 38 (the area below the boundary a or the boundary b).

That is, in the region of low priority in the person image 54 and the background 22, that is, the region below the boundary a or the boundary b, an arbitrary image can be displayed on the image receiving terminal 138 so as to be overlaid thereon. By hiding the area under the image 54 and the background 22, another display area can be secured in the video receiving terminal 138.

(Third Embodiment) Next, a third embodiment of the present invention will be described.

In the present embodiment, the coding amount to be transmitted to the video receiving terminal as a whole is changed by changing the area of each object video while maintaining the priority for determining the important area of each object video for a plurality of object videos. The purpose is to realize rate control for controlling.

FIG. 17 is a configuration diagram of a video transmission terminal that performs rate control in the present embodiment. In the configuration of FIG.
Region extracting unit 108, region extracting unit 112, priority setting unit 4, priority setting unit 24, object video encoding unit 12
0 and the object video encoding unit 124 are the same as those described above.

The coded data 122 and coded data 126 coded by the object video coding unit 120 and the object video coding unit 124 are converted into the serial output 58 by the multiplexer 56 and
0 is applied.

The contents stored in the buffer memory 60 are transmitted at the transmission rate of the transmission line 136. Here, if the transmission rate is high and is higher than the rate input from the multiplexer 56, the buffer memory 60 becomes empty, causing an underflow. Conversely, if the transmission rate is slow and the rate input from the multiplexer 56 is high, the buffer memory 60 will soon fill up and overflow. Therefore, the capacity information 62 of the buffer memory 60 is detected and applied to the rate control unit 64.

The rate control section 64 receives the input capacity information 62
Then, the boundary lines to be divided by the region dividing unit 16 and the region dividing unit 28 are determined. That is, when the buffer memory 60 is about to overflow, the area dividing unit 1 is used to reduce the amount of the encoded data 122 and the encoded data 126.
In the priority area data 14 and the priority area data 26, a boundary is selected so that the area is reduced. Conversely, if the buffer memory 60 is about to overflow, the area dividing unit 16 and the area dividing unit 28 perform processing on the priority area data 14 and the priority area data 26 in order to increase the encoded data 122 and the encoded data 126. Select the boundaries so that the area is large.

That is, the priority area data 14 and the priority area data 14 in the area division sections 16 and 28 are controlled by the rate control section 64 so that the storage capacity of the buffer memory 60 and the data amount to be transmitted to the transmission path 136 are balanced. By selecting the boundary of the degree region data 26, the encoded data 1
22 and the data amount of the encoded data 126 are changed.

Here, the priority of the priority area data 14 is set higher than that of the priority area data 26, and the boundary in the priority area data 26 is changed as much as possible.
Changing the shape of the priority area data 14 may be reduced.

The rate control according to the present embodiment does not require complicated processing such as changing the quantization precision or the frame rate of the object video unlike the conventional rate control, but requires simple processing of changing only the shape. There are advantages that can be.

(Fourth Embodiment) Next, a fourth embodiment of the present invention will be described.

In the present embodiment, the data encoded by the object video encoding unit is subjected to error correction encoding, sent to the video receiving terminal, and subjected to error correction decoding by the video receiving terminal, so that the original data is When the object video is reproduced by decoding the object video after error correction and restoration, the ratio between the amount of encoded data and the data amount of the error correction code is changed according to the line quality of the transmission path. .

FIG. 18 is a block diagram of the video transmitting terminal in this embodiment, and FIG. 19 is a block diagram of the video receiving terminal.

In FIG. 18, the error correction code 68 is added to the coded data 122 generated by the object video coding unit 120 by the error correction
Is applied.

FIG. 20A is a configuration diagram of data obtained by adding an error correction code to encoded data in the present embodiment.
(B) is a diagram showing a person image 88 in that case, and
FIG. 21A is a configuration diagram of data when the ratio of the data amount of the coded data to the error correction code is changed, and FIG. 21B is a diagram illustrating a human image in that case.

The encoded data 122 as shown in FIG.
The error correction code 68 capable of correcting the error is added to the end by the error correction coding unit 66, and becomes encoded data with correction code 70, and is applied to the multiplexing unit 132.

The serial data 134 including the encoded data 70 with the correction code output from the multiplexing unit 132 is shown in FIG.
As shown by, the video receiving terminal 1 via the transmission path 136
38 is applied to the separation unit 140. The coded data 72 with the correction code separated by the separation unit 140 is applied to the error correction decoding unit 74, and the coded data 122 is subjected to a required error correction to reproduce the coded data 142.
The object video 150 is applied to the object video decoding unit 148, and is restored, and is displayed on the monitor 162.

Here, in order to check the line quality of the transmission line 136, the test pattern generating unit 76 is used in the video receiving terminal 138.
, The reference signal 78 is applied to the demultiplexer 140, passes through the transmission line 136, passes through the multiplexer 132 of the video transmission terminal 102, and is applied as the reference signal 80 to the BER measuring unit 82. Have been. BER measurement unit 8
In step 2, the quality of the transmission path 136 is checked using the reference signal 80, and the result is applied to the control unit 128.

The control unit 128 controls the area dividing unit 16 to process the processed area data 2 based on the quality data.
Select 0. On the other hand, the control unit 128 controls the error correction encoding unit 66 to select the error correction code 68 suitable for the post-processing area data 20 and add it to the end of the encoded data 122. Then, the control unit 128 outputs the encoding control information 8
4, the multiplexing unit 132, the transmission path 136, and the demultiplexing unit 14.
0, and sends the encoded control information 86 to the control unit 156 of the video receiving terminal 138. The control unit 156 selects the correction capability of the error correction decoding unit 74 according to the error correction encoding unit 66.

Here, as shown in FIGS. 20 (a) and 21 (a), it is assumed that the data amount a obtained by combining the coded data 122 and the data amount of the error correction code 68 is controlled to be constant. ing.

In FIG. 20A, since the line quality of the transmission line 136 is low, the data amount b1 of the coded data 122 is reduced, and the data amount c1 of the error correction code 68 is increased instead. Has been strengthened. In this case, in order to reduce the data amount b1 of the encoded data 122,
As shown in FIG. 20B, the post-processing area data 20 selects a human image 88 having a small area by the area dividing unit 16.

On the other hand, in FIG. 21A, since the line quality of the transmission line 136 is high, the data amount b2 of the encoded data 122 is increased, and the error correction code 68 data amount c2 is reduced instead. The correction ability is low. In this case, in order to increase the data amount b2 of the encoded data 122, the region dividing unit 16 selects a person image 90 having a large area as the post-processing region data 20 as shown in FIG.

As described above, in this embodiment, the line quality of the transmission line 136 is measured, the error correction code 68 is determined according to the line quality, and when the error correction code 68 is determined, the data of the coded data 122 is determined. The amount is determined, and the area of the priority area data 14 is appropriately cut in accordance with the amount of data. Even if the line quality of the transmission path 136 deteriorates, the important part of the object video 150 is displayed on the monitor 162. It is possible to display.

[0084]

As described above, according to the present invention, it is possible to realize the scalability of coded data obtained by object video coding without deteriorating the quality of important parts of a video.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a video transmission terminal according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a video receiving terminal according to the first embodiment of the present invention.

FIG. 3 is a diagram for explaining detection of a subject by cheek coordinate detection.

FIG. 4 is a view for explaining processing of a priority setting unit according to the first embodiment of the present invention.

FIG. 5 is a diagram illustrating a state in which a human image is sequentially cut while leaving important portions in the first embodiment of the present invention.

FIG. 6 is a display diagram in which a human image and a background are superimposed and displayed on a monitor in the first embodiment of the present invention.

FIG. 7 is a view showing boundaries of priorities for region extraction data of an arbitrary shape in the first embodiment of the present invention.

FIG. 8 is a display diagram in which a lower side of a background image is divided by priority according to the first embodiment of the present invention.

FIG. 9 is a display diagram in which both ends of a background image are divided by priority according to the first embodiment of the present invention.

FIG. 10 is a display diagram in which a lower side and both ends of a background image are divided by priority according to the first embodiment of the present invention.

FIG. 11 is a display diagram in which a low-priority region is cut out from a background video and a human image in the first embodiment of the present invention, and then cut out.

FIG. 12 is a block diagram illustrating a configuration of a video transmission terminal according to a second embodiment of the present invention.

FIG. 13 is a block diagram showing a configuration of a video receiving terminal according to a second embodiment of the present invention.

FIG. 14 is a display diagram in which a human image is divided into regions according to priority in the second embodiment of the present invention.

FIG. 15 is a display diagram in which a background is divided into areas according to priority in the second embodiment of the present invention.

FIG. 16 is a display diagram in which a human image and a background are displayed according to the priority in the second embodiment of the present invention.

FIG. 17 is a block diagram illustrating a configuration of a video transmission terminal according to a third embodiment of the present invention.

FIG. 18 is a block diagram illustrating a configuration of a video transmission terminal according to a fourth embodiment of the present invention.

FIG. 19 is a block diagram showing a configuration of a video receiving terminal according to a fourth embodiment of the present invention.

FIG. 20A is a configuration diagram of data obtained by adding an error correction code to encoded data according to the fourth embodiment of the present invention,
(B) is a diagram showing a person image in that case.

FIG. 21A is a configuration diagram of data when the ratio between the amount of encoded data and the amount of data of an error correction code according to the fourth embodiment of the present invention is changed, and FIG. 21B shows a human image in that case; FIG.

FIG. 22 is a block diagram showing a configuration of a video transmission terminal in a conventional videophone system.

FIG. 23 is a block diagram showing a configuration of a video receiving terminal in a conventional videophone system.

FIG. 24 is a display diagram of an object video displayed on a conventional monitor.

FIG. 25 is a view for explaining an example of conventional spatial scalability.

FIG. 26 is a diagram for explaining an example of conventional time scalability.

[Explanation of symbols]

2, 6, 8, 10, 12, 36, 48, 54, 88, 9
0 ... person image 4, 24 ... priority setting unit 14, 26, 50, 52 ... priority area data 16, 28 ... area dividing unit 18, 30 ... selection information 20, 32 ... processed area data 22, 166 ... background 34 setting section 38 surplus area 40, 104, 114 video source 42, 150, 154 object video 56 multiplexer 58 serial output 60 buffer memory 62 capacity information 64 rate control section 66 error correction coding Part 68: Error correction code 70, 72 ... Encoded data with correction code 74 ... Error correction decoding part 76 ... Test pattern generation part 78, 80 ... Reference signal 82 ... BER measurement part 84, 86 ... Coding control information 102 ... Video transmission terminals 106, 116 ... Digital video 108, 112 ... Area extraction unit 110, 118 ... Area extraction data 120, 12 ... object video coding units 122, 126, 142, 144 ... coded data 128, 156 ... control units 130, 146 ... scene configuration information 132 ... multiplexing units 134 ... serial data 136 ... transmission path 138 ... video receiving terminal 140 ... Separation units 148, 152 ... object video decoding unit 158 ... synthesis unit 160 ... synthesized video data 162 ... monitor

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 KK22 KK35 KK45 LB11 MA00 MA32 MA33 MA37 MB03 MB04 MB06 MB12 NN36 PP04 PP28 PP29 RB01 RB14 RC00 RE00 RF05 SS07 TA75 TA76 TB18 TC18 TC21 TC34 TC37 UA02 UA05

Claims (5)

[Claims] 1. An extracting means for extracting an area of a subject from a video including a subject, a setting means for setting a priority area for an area to be extracted by the extracting means, and a setting means for setting a priority area from the video for the priority area set by the setting means. A video transmission terminal, comprising: coding means for performing object video coding. 2. An extracting means for extracting an area of a subject from a video including a subject, a means for setting a priority area and generating priority area information for the area to be extracted by the extracting means, and the extracting means extracting the information. A video transmitting terminal comprising: means for performing object video coding on a priority area from the video to generate coded data; and means for transmitting the coded data and the priority area information. 3. Receiving means for receiving coded data and priority area information transmitted by a video transmitting terminal, reproducing means for decoding the coded data received by the receiving means to reproduce an object video, Display means for displaying the object video reproduced by the reproduction means; and video superimposing means for superimposing another video on a predetermined area designated using the priority area information when the display means displays the object video. A video receiving terminal comprising: 4. An extracting means for extracting an area of a subject from a video including a subject, a setting means for setting a priority area for the area to be extracted by the extracting means, and a predetermined priority from the priority area set by the setting means. Selecting means for selecting an area, coding means for performing object video coding from the video for a predetermined priority area selected by the selecting means, and transmitting means for transmitting coded data to be coded by the coding means. And a control unit for controlling the selection unit in order to control the rate of the encoded data transmitted by the transmission unit. 5. An extracting means for extracting an area of a subject from a video including a subject, a setting means for setting a priority area for the area to be extracted by the extracting means, and a predetermined priority from the priority area set by the setting means. Selecting means for selecting an area; coding means for performing object video coding from the video for a predetermined priority area selected by the selecting means; and error correction coding for the coded data to be coded by the coding means. Error correction coding means for applying, transmission means for transmitting coded data to which the error correction coding means performs error correction coding, and acquisition means for obtaining information on line quality of a transmission path between the video receiving terminal and the video reception terminal And controlling the selection means and the error correction coding means based on the information on the line quality acquired by the acquisition means, and the data amount of the encoded data and the error Video transmitting terminal, characterized by comprising a control means for setting the combination of the data amount of the positive coded data.