JP2007336277A - Moving image encoding and decoding method, encoding and decoding device and moving image encoding and decoding program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible to decode a moving image signal with equipment mounted with a processor of a low processing speed. <P>SOLUTION: A subframe 102 is set in a frame 101 of a moving image signal by area information. A motion detecting part obtains a motion vector in each block in the frame, and an encoding part performs compensation interframe predictive encoding of an input moving image signal on the basis of the motion vector. The motion detecting part sets a search range 104a obtained by excluding an area outside the subframe from a motion vector search range about a block 105 in the subframe. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a coding / decoding technique for a moving image signal.

  In recent years, with the improvement of compression technology, it has become possible to encode a moving image signal at a high compression rate. For example, the standard H.264 for videophones and the like. 261 and standards such as MPEG (Motion Picture Expert Group) for broadcasting and storage media have been established. A moving image signal encoding method compliant with these standards has a motion compensation interframe predictive encoding method and a discrete cosine transformation (DCT) as the core. For this reason, the moving image signal encoding process requires a large amount of processing.

  The processing amount required for the moving image signal decoding process is small as compared with the processing amount required for the encoding process. However, this decryption processing is a heavy burden for a portable terminal equipped with a low processing speed processor.

  With reference to FIG. 1 and FIG. 2, a conventional moving picture coding technique will be described. This technique is described in JP-A-9-037269 (Patent Document 1), JP-A-6-268996 (Patent Document 2), and the like.

  FIG. 1 is a block diagram illustrating a configuration example of a conventional moving image signal encoding device, and FIG. 2 is a block diagram illustrating a configuration example of a conventional moving image signal decoding device.

  As shown in FIG. 1, the conventional moving image signal encoding apparatus includes a motion detection unit 2 and an encoding unit 30. The encoding unit includes a subtractor 31, a conversion unit 32, an inverse conversion unit 33, an adder 34, and a motion compensation unit 35.

  As shown in FIG. 2, the conventional moving image signal decoding apparatus includes a decoding unit 40. The decoding unit includes an inverse conversion unit 41, an adder 42, and a motion compensation unit 43.

  Next, the operation of FIGS. 1 and 2 will be described.

  As is well known, the first frame of the moving image signal is directly supplied to the converting unit 32 as it is and is intra-coded, and the output of the inverse converting unit 33 is directly used as the local decoded signal 39. Below, only the process after the 2nd frame of a moving image signal is demonstrated.

In FIG. 1, an input moving image signal 1 is supplied to a motion detector 2 and a subtracter 31. The motion detector 2 detects a motion vector v _{x, y} for each block of the input video signal from the input video signal 1 and the local decoded signal 39 from the adder 34. Note that the configuration and operation of the motion detection unit 2 are described in Patent Document 1 and Patent Document 2 described above, and thus detailed description thereof is omitted.

  The input signal image signal 1 is supplied to the first input of the subtractor 31. A motion compensation inter-frame prediction signal is supplied from the motion compensation unit 35 to the second input of the subtractor 31. The subtractor 31 subtracts the motion compensation inter-frame prediction signal from the input moving image signal and supplies the difference signal to the conversion unit 32.

  The conversion unit 32 performs DCT conversion and quantization on the difference signal, and outputs a prediction error signal E.

  The inverse transform unit 33 performs inverse quantization and inverse DCT transform on the prediction error signal E, and supplies the inverse transform result to the adder 34. The motion compensation inter-frame prediction signal is supplied from the motion compensation unit 35 to the other input of the adder 34. The adder 34 adds the output of the inverse converter 33 and the motion compensation inter-frame prediction signal to generate a local decoded signal 39. This local decoded signal 39 is supplied to the subtractor 31 and the motion detector 2.

The local decoded signal 39 is also supplied to the motion compensation unit 35. The motion compensation unit 35 includes a memory having a capacity of at least one frame, and generates a motion compensation inter-frame prediction signal based on the local decoded signal 39 and the motion vector vx _{, y} .

The prediction error signal E and the motion vector v _{x, y} generated in this way are multiplexed and transmitted as encoded moving image data or stored in a storage medium.

  Next, the operation of the moving picture signal decoding apparatus in FIG. 2 will be described.

In FIG. 2, the prediction error signal E is supplied to the inverse transform unit 41, and the motion vector v _{x, y} is supplied to the motion compensation unit 43. The operation of the inverse transform unit 41 is the same as that of the inverse transform unit 33 in FIG. 1, and the output of the inverse transform unit 41 is supplied to the adder 42.

  The motion compensation inter-frame prediction signal is supplied from the motion compensation unit 43 to the other input of the adder 42. The adder 42 adds the output of the inverse transform unit 41 and the motion compensated inter-frame prediction signal, and outputs a decoded moving image signal 44.

The decoded moving image signal 44 is also supplied to the motion compensation unit 43. The configuration and function of the motion compensation unit 43 are the same as those of the motion compensation unit 35 in FIG. The motion compensation unit 43 generates a motion compensation inter-frame prediction signal based on the decoded moving image signal 44 and the motion vector v _{x, y} .

JP 9-037269 JP-A-6-268996 JP2001008040

  As described above, with the progress of encoding technology, the amount of processing required for encoding of moving image signal and decoding of the encoded moving image signal has increased. For this reason, an electronic device that performs these processes is required to have a high processing capacity.

  However, this requirement is not always satisfied. A case where a moving image signal is encoded by a device having a high processing capability and this encoded data is decoded by another device having only a low processing capability cannot be avoided.

  One example thereof is a case where encoded moving image data is downloaded from a site that stores the encoded moving image signal, and this moving image signal is viewed by a low processing speed device such as a mobile phone. In this example, the process of decoding the encoded moving image data cannot follow the supply speed of the encoded moving image data supplied from the communication line, and it becomes difficult to decode the encoded moving image data in real time. Sometimes.

  As another example, the following case can be considered. In the portable terminal, in order to prevent the battery from being depleted, the operating speed of the processor may be lowered when the remaining capacity of the battery decreases. In this case, if the remaining capacity of the battery is sufficient, even if it is encoded moving image data that can be normally decoded, if the remaining capacity of the battery is low, a normal decoding operation cannot be performed. . In addition, there is a device that prohibits decoding of encoded moving image data when the remaining capacity of the battery is not sufficient.

  In addition, even in a device having high processing capability, when decoding a plurality of encoded moving image data in parallel, the processing capability allocated to each encoded moving image data becomes small, and the same problem occurs. Can do.

  One approach to addressing these issues is to decode only the subframes that are part of all frames (for example, the central part of a moving image frame) without decoding all the encoded data in one frame. Decoding is conceivable. Although it depends on the subject, generally, the region closer to the center of the frame often contains information useful to the viewer. For example, Japanese Patent Laid-Open No. 2001-8040 (Patent Document 3) discloses a technique of extracting a part of encoded still image data and decoding only a part of the still image.

  However, in the moving picture encoding technique represented by MPEG, the technique described in Patent Document 3 cannot be used as it is because motion compensation interframe predictive encoding is the core. The reason for this will now be described.

  As already described, motion compensation inter-frame predictive encoding is the difference between the current frame and the past frame (reference frame) when encoding / decoding the encoding / decoding target frame (current frame) of moving image data. Is a technology that uses

  FIG. 3 is a diagram for explaining the concept of motion compensation interframe predictive coding.

  A frame 101 of a moving image signal is divided into a plurality of blocks, and a motion vector is obtained for each block. Here, attention is paid to the block 103 in the current frame. The motion vector of the block 103 is obtained as follows. The motion detection unit 2 in FIG. 1 obtains the similarity between the image data in the block 103 and the image data in the motion vector search range 104 in the reference frame for various candidate vectors. Then, the motion detection unit outputs the candidate vector that maximizes the similarity as the motion vector described above.

  Here, the subframe 102 is set in the frame 101 of the moving image signal, only the encoded data belonging to the subframe is extracted from the encoded moving image data, and only the moving image signal of the subframe 102 is decoded. Think about it. In this case, a motion compensated interframe encoded prediction signal for the block 103 in the subframe may be generated from the reference frame outside the subframe. As described above, in order to decode the encoded moving image data generated by the motion compensation interframe predictive encoding, the reproduced moving image signal in the search area in the reference frame is indispensable. However, in this case, the decoding device cannot always obtain a playback video signal outside the subframe 102. That is, there is no guarantee that the reproduced moving image signal in the reference frame is always obtained for all the blocks in the subframe of the current frame. Therefore, the technique described in Patent Document 3 cannot be directly applied to motion compensation interframe predictive coding.

  Of course, it is possible to decode all the encoded moving image data in the frame 101 to obtain a reproduced moving image signal and extract a subframe from the reproduced moving image signal. However, in such a form, the required calculation amount cannot be reduced even if the size of the moving image to be displayed is reduced.

  The present invention provides a motion compensated inter-frame predictive coding / decoding post-operation that can extract and decode a part of coded moving image data.

  In a first aspect of the present invention, a motion vector of an input video signal is detected, the input video signal is encoded based on the motion vector, and a prediction error signal and the motion vector are multiplexed to generate encoded data. This is a moving image signal encoding method to be output. This input moving image signal is an input moving image signal in which a subframe specified by region information is set in each frame, and a block of a current frame in this subframe. When the motion vector is detected, a motion picture signal encoding method is provided in which the search range of the motion vector is limited to a subframe in a reference frame.

  Moreover, the 2nd side surface of this invention provides the decoding method for decoding the encoding data produced | generated by this 1st side surface, for example. The decoding method provided by the second aspect includes an extraction step of extracting encoded data in the subframe from the encoded data based on the region information, and decoding the extracted encoded data And a step of performing.

  Further aspects of the present invention will become apparent from the description of the best mode for carrying out the invention.

  According to the present invention, it is possible to decode a moving image signal that has undergone motion compensation interframe predictive coding even by a device equipped with a processor having a low processing speed.

  In the present invention, for a block in a subframe, an area outside the subframe of the reference frame is excluded from the motion vector search range. For this reason, in order to decode the moving image signal in a sub-frame, the reproduction | regeneration moving image signal outside a sub-frame becomes unnecessary. Therefore, the decoding apparatus can reproduce the moving image signal in the subframe by decoding only the encoded data of the block in the subframe. For this reason, even a device equipped with a processor with a low processing speed can extract only the encoded data in the subframe from the video signal encoded by motion compensation interframe, and only the video signal in the subframe can be extracted. Decoding and playback can be performed.

  Next, the best mode for carrying out the present invention will be described with reference to the drawings.

  FIG. 4 is a block diagram illustrating a configuration example of a motion compensation interframe prediction encoding apparatus according to an embodiment of the present invention. FIG. 5 is a block diagram showing a configuration example of a motion compensation inter-frame decoding apparatus according to an embodiment of the present invention. FIG. 6 is a flowchart for explaining the operation of the apparatus of FIG. FIG. 7 is a diagram for explaining the operation of the motion detector 50 of FIG. FIG. 8 is a flowchart for explaining the operation of the motion compensation interframe decoding apparatus of FIG.

  The motion compensated inter-frame prediction encoding apparatus in FIG. 4 includes a motion detection unit 50 and an encoding unit 30. The encoding unit 30 has the same configuration as the encoding unit 30 in FIG.

  The motion compensated inter-frame decoding apparatus in FIG. 5 includes a filter unit 60 and a decoding unit 40. The decoding unit 40 has the same configuration as the decoding unit 40 in FIG.

  Next, the operation of the motion compensated interframe predictive coding apparatus in FIG. 4 will be described with reference to FIGS. However, since the operation of the encoding unit 30 has already been described with reference to FIG. 1, redundant description will be omitted, and the operation of the motion detection unit 50 will be described in detail.

The process shown in FIG. 6 is activated for each block of the current frame.
In step S1 of FIG. 6, the motion detection unit 50 internally sets a candidate vector list that is a list of a plurality of candidate vectors ux _{, y) that} are motion vector search targets. This candidate vector list can be created by copying a master list built in the motion detector 50. Further, many known techniques are known for creating this candidate vector group, but a candidate vector list may be created based on these known techniques. As an example, Patent Document 1 described above describes generating a candidate vector list based on the distribution of past motion vector detection results.

In this example, FIG.
| U _x | ≦ Uxmax, | u _y | ≦ Uymax
This indicates that a candidate vector list having candidate vectors that satisfies the above is generated. For example, for the block 105 in FIG. 7, the moving vector search range 106 corresponding to the candidate vector list that satisfies this condition is set in the reference frame.

In step S2, the motion detection unit 50 determines whether or not the current block is in a subframe. For example, the block 103 of the current frame in FIG. 7 is determined to be inside the subframe 102, and the block 105 is determined to be outside the subframe. This determination is based on the upper left coordinate value (x ₀ , y ₀ ), block size (horizontal M pixels, vertical N pixels), and region information ((X _min , Y _min ), (X _max , Y _max )).

  If the determination result in Step S2 is Yes, the process of the motion detection unit proceeds to Step S3, and if not, the process of the motion detection unit proceeds to Step S4.

In step S3, in the example illustrated in FIG. 7, the motion detection unit 50 selects candidate vectors (u _x , u _y ) that satisfy at least one of the following conditions (1) to (4) as a candidate vector list. Delete from. That is, candidate vectors whose moving vector search range goes out of the subframe are deleted from the candidate vector list.

(1) x ₀ -u _x <X _min
(2) x ₀ + M−u _x > X _max
(3) y ₀ -u _y <Y _min
(4) y ₀ + N−u _y > Y _max
By performing such deletion from the candidate vector list, the motion detection unit 50 does not search for a reference frame outside the subframe for a block within the subframe. For example, the search range of the book 105 in FIG. 7 is the range indicated by reference numeral 104a.

  Further, FIG. 7 shows that the block 105 outside the subframe is not deleted from the candidate vector list (see the motion vector search range 106).

In step S4, the motion detection unit 50 calculates the similarity between the image data of the block of the current frame and the reference frame image data for the candidate vectors remaining in the candidate vector list, and the motion vector v for the block is calculated. _{X and y} are calculated.

In step S5, the encoding unit 30 uses the motion vector obtained in step S4 to perform the motion compensation interframe predictive encoding on the input moving image signal, generates a prediction error signal E, and outputs it together with the motion vector.
In this example, the region information ((X _min , Y _min ), (X _max , Y _max )) is multiplexed with the prediction error signal E and the motion vectors v _{x, y,} and this multiplexed information is encoded. It may be transmitted or stored in storage media as converted video data. A configuration example in this case is shown in FIG. Since the operation of FIG. 9 is clear from the above description, the description is omitted.
Of course, when this region information is already known between the motion compensation inter-frame prediction encoding device and the motion compensation inter-frame decoding device, such multiplexing of region information is not necessary.

  Next, the operation of the video signal decoding apparatus in FIG. 5 will be described with reference to FIG.

  The motion compensated interframe decoding apparatus in FIG. 5 includes a filter unit 60 and a decoding unit 40. The decoding unit 40 has the same configuration as the decoding unit 40 in FIG. Therefore, hereinafter, the operation of the filter unit 60 will be mainly described.

  Here, the region information is already known between the motion compensation interframe coding apparatus and the motion compensation interframe decoding apparatus, and this region information is set in the motion compensation interframe decoding apparatus. Explain that it is. FIG. 8 is a flowchart for explaining the operation of the motion compensation interframe predictive coding apparatus. In the motion compensation inter-frame decoding apparatus in FIG. 5, the determination in step S22 in FIG. 8 is not performed. Steps S21, S23, and S24 are executed in this order. The case where the process of step S22 is necessary ”will be described later.

In step S21, the filter unit 60 receives the motion compensation inter-frame prediction error E and the motion vector vx _{, y} for each block.

Next, in step S23, the filter unit determines whether the input motion compensation inter-frame prediction error signal and the motion vector are the encoded moving image data of the block in the subframe based on the region information. judge. The filter unit 60 outputs the encoded moving image data only when the encoded moving image data belongs to the block in the subframe in step S23 (E ′ and v ′ _{x, y in} FIG. 5).

  In step S24, the decoding unit 40 decodes the encoded moving image data supplied from the filter unit. In this way, the moving image signal in the subframe is reproduced.

  As described above, the embodiment described with reference to FIGS. 4 and 5 extracts a part of the encoded moving image data generated by using the motion compensation interframe predictive encoding method, Decoding and reproduction are possible. Therefore, this embodiment makes it possible to use a moving image signal even in a portable terminal or the like equipped with a processor with a low calculation speed.

  Next, another example of the moving picture interframe decoding apparatus according to the present invention will be described. This will be described with reference to FIG.

  First, as described in the section “Problems to be Solved by the Invention”, a portable terminal or the like has a function of reducing the processing speed of the processor and reducing power consumption when the remaining capacity of the battery decreases. There are a lot of things to prepare. In such a device, even if a device capable of decoding full-size encoded moving image data is normally used, a full-size moving image signal cannot be decoded if the remaining battery capacity is low.

Further, in a device that can execute a plurality of tasks in parallel, the processing capacity of the processor is allocated to the plurality of tasks. Even in such a device, when other tasks are in operation, the processing capability of the processor cannot be sufficiently given to the decoding processing of the encoded moving image data. Must be terminated or whether to abandon execution of the decoding process of the encoded moving image data.
The motion compensation interframe decoding apparatus of FIG. 10 is useful in such a case. This will be described below.

  The motion compensation inter-frame decoding apparatus in FIG. 10 includes a filter unit 61 and a decoding unit 40. The decoding unit 40 in FIG. 10 has the same configuration as the decoding unit 40 in FIGS. 2 and 5. In this example, the region information may be set in advance in the decoding device, or may be supplied from a storage medium or the like together with the prediction error signal E and the motion vector.

  In this example, the process shown in the flowchart of FIG. 8 is executed as it is.

  In step S22, the filter unit 61 checks the selection information in FIG. 10 and determines whether or not to extract the encoded moving image data using the region information.

  This selection information is generated by an electronic device equipped with this motion compensation interframe decoding apparatus. That is, when the remaining battery capacity decreases, a device that reduces the processing speed of the processor generates this selection information and supplies it to the filter unit 61 when the remaining battery capacity decreases. Also, a device that assigns the processing capacity of the processor to each task according to the task operating status generates this selection information according to the operating status and supplies it to the filter unit 61.

  When the determination result in step S22 is “Yes”, the filter unit 61 supplies only the prediction error signal and the motion vector of the block in the subframe to the decoding unit 40 as in the filter unit 60 of FIG. Then, the decoding unit 40 reproduces the moving image signal based on the supplied motion compensation inter-frame prediction error signal and the motion vector.

  When the determination result in step S22 is “No”, the filter unit 61 supplies the received prediction error signal and motion vector to the decoding unit.

  As described above, the motion compensation inter-frame decoding apparatus of FIG. 10 is generated by motion compensation inter-frame prediction encoding according to the remaining capacity of the battery and the processing capacity of the processor allocated to the motion compensation inter-frame decoding processing. The reproduction of the moving image signal can be continued by changing the display size of the encoded moving image data.

  It will be apparent to those skilled in the art that the present invention can also be executed by a computer loaded with a program corresponding to the flowcharts illustrated in FIGS.

  The present invention is applicable to a moving image signal encoding technique.

It is a block diagram which shows the structural example of the conventional moving image signal encoding apparatus. It is a block diagram which shows the structural example of the conventional moving image signal decoding apparatus. It is a figure for demonstrating the subject of a prior art. It is a block diagram which shows one Embodiment of the moving image signal encoding apparatus of this invention. It is a block diagram which shows one Embodiment of the moving image signal decoding apparatus of this invention. 6 is a flowchart for explaining processing of the moving image signal encoding device in FIG. 4. It is a figure for demonstrating operation | movement of the motion detection part 50 of FIG. 6 is a flowchart for explaining processing of the video signal decoding device in FIG. 5. It is a block diagram which shows the other example of the moving image signal encoding device of this invention. It is a block diagram which shows the other example of the moving image signal decoding apparatus of this invention.

Explanation of symbols

2, 50 Motion detection unit 30 Encoding unit 31 Subtractor 32 Conversion unit 33, 41 Inverse conversion unit 34, 42 Adder 35, 43 Motion compensation unit 101 Frame 102 Subframe 103, 105 Block 104, 104a, 106 Motion vector search range

Claims (16)

A moving image signal encoding method that detects a motion vector of an input moving image signal, encodes the input moving image signal based on the motion vector, multiplexes a prediction error signal and the motion vector, and outputs encoded data. The input moving image signal is an input moving image signal in which a subframe specified by region information is set in each frame;
A moving picture signal encoding method characterized by limiting a search range of a motion vector to a subframe in a reference frame when detecting a motion vector of a block of a current frame in the subframe.   The moving image signal encoding method according to claim 1, wherein the area information is output as the encoded data together with the prediction error signal and a motion vector. A method for decoding encoded data generated by detecting a motion vector of an input moving image signal and encoding the input moving image signal based on the motion vector. When encoding an input video signal in which a subframe specified by region information is set, the motion vector search range of the current frame block in the subframe is limited to the subframe in the reference frame. A video signal decoding method for decoding encoded data generated by
A step of extracting the encoded data in the subframe from the encoded data based on the region information;
A moving picture signal decoding method, wherein the extracted encoded data is decoded.   4. The moving picture signal decoding method according to claim 3, wherein the region information is multiplexed and supplied in the encoded data.   When selection information indicating use / non-use of the area information is input, and the selection information indicates non-use, the extraction step uses the prediction error signal and a moving vector to generate the current frame. 5. The moving picture signal decoding method according to claim 3, wherein the whole moving picture signal is decoded. A motion detecting means for detecting a motion vector of the input moving picture signal; and an encoding means for encoding the input moving picture signal based on the motion vector, multiplexing the prediction error signal and the motion vector, and outputting the multiplexed signal. In the moving image signal encoding device,
The input moving image signal is an input moving image signal in which a subframe specified by region information is set in each frame,
The motion detection means, when detecting a motion vector of a block of a current frame in a subframe, restricts a search range of the motion vector to a subframe in a reference frame. Device.   7. The moving picture signal encoding apparatus according to claim 6, wherein the encoding unit multiplexes the region information together with the prediction error signal and a motion vector and outputs the multiplexed information. An apparatus for decoding encoded data generated by a method of detecting a motion vector of an input moving image signal and encoding the input moving image signal based on the motion vector. When encoding an input video signal in which a subframe specified by region information is set, the motion vector search range of the current frame block in the subframe is limited to the subframe in the reference frame. A video signal decoding device for decoding encoded data generated by
Means for extracting encoded data in the subframe from the encoded data based on the region information;
Decoding means for decoding the extracted encoded data;
A video signal decoding device comprising:   9. The moving picture signal decoding apparatus according to claim 8, wherein the area information is multiplexed and supplied in the encoded data.   When selection information indicating the use / non-use of the area information is input and the selection information indicates non-use, the extraction unit sends all the prediction error signal and motion vector to the decoding unit. 10. The moving picture signal decoding apparatus according to claim 8, wherein the moving picture signal decoding apparatus outputs the moving picture signal decoding apparatus. A moving image signal encoding program for causing a computer to execute a moving image signal encoding method, wherein the input moving image signal is an input moving image signal in which a subframe specified by region information is set in each frame And the program is
A motion detection step for detecting a motion vector of the input video signal;
The input video signal is encoded based on the motion vector, and includes an encoding step for multiplexing the prediction error signal and the motion vector and outputting encoded data,
In the motion detection step, when detecting a motion vector of a block of a current frame in the subframe, the motion vector search range is limited to a subframe in a reference frame. Encoding program.   The moving image signal encoding program according to claim 11, wherein the region information is output as the encoded data together with the prediction error signal and a motion vector. A program for detecting a motion vector of an input moving image signal and causing a computer to decode encoded data generated by encoding the input moving image signal based on the motion vector. When detecting an input moving image signal in which a subframe specified by region information is set in each frame and detecting a motion vector of a block of the current frame in the subframe, the search range of the motion vector is set as a reference frame. A moving image signal encoding program for causing a computer to decode encoded data generated by limiting within a subframe of
Extracting the encoded data in the subframe from the encoded data based on the region information; and
A decoding step of decoding the extracted encoded data;
A moving picture signal decoding program comprising:   14. The moving picture signal decoding program according to claim 13, wherein the area information is supplied after being multiplexed in the encoded data. A step of inputting selection information indicating use / non-use of the area information is provided;
In the extraction step, when the selection information indicates non-use, the extraction step supplies all the prediction error signal and the motion vector to the decoding step.
15. The moving picture signal decoding program according to claim 13 or 14, An electronic device equipped with the video signal decoding device according to any one of claims 13 to 15.

Images