JP4344184B2 - Moving picture encoding apparatus, moving picture encoding method, moving picture encoding program, moving picture decoding apparatus, moving picture decoding method, and moving picture decoding program - Google Patents

Description

ここで、式（１）において、ｆは予測残差画像、Ｒ_nｆは第ｎ反復後の残差成分であり、ｇ_knはＲ_nｆとの内積値を最大とする基底であり、Ｒ_mｆは第ｍ反復処理後の残差成分である。
【００３８】
ＭＰ法によれば、基底セットのうちから残差成分との内積値を最大とする基底が選択され、選択された基底とこの基底に乗じる為の係数である最大の内積値とに残差成分が分解される。
【００３９】
ここで、動き補償予測の予測残差である予測残差画像は偏った分布を持つ画像信号であることが知られている。ＭＰ法では、予め基底として予測残差画像に出現しやすいパターンを二次元基底として記憶しておくことによって、記憶された二次元基底の中からより残差成分が小さくなるような基底を用いて画像を順次分解することが可能となる。したがって、ＤＣＴなどといった直交変換に比べて少ない数の基底を用いて効果的に予測残差画像を表すことができる。
【００４０】
本実施形態では、基底記憶部６ｂが基底セットを記憶している。変換部６ａは、基底記憶部６ｂによって提供される基底セットを用い、残差成分との内積値を最大とする基底を特定するための基底情報Ｉ₁₂を求め、基底情報Ｉ₁₂を逆変換部６ｅに出力する。また、基底情報Ｉ₁₂は、残差補償情報Ｉ₅の一部として符号化部１２に出力される。変換部６ａは、上記の内積値を補償係数Ｉ₁₃として求め、補償係数Ｉ₁₃を量子化部６ｃに出力する。
【００４１】
量子化部６ｃは、変換部６ａによって出力された補償係数Ｉ₁₃に量子化操作を施すことによって、量子化補償係数Ｉ₁₄を生成する。量子化部６ｃは、量子化補償係数Ｉ₁₄を逆量子化部６ｄに出力すると共に、量子化補償係数Ｉ₁₄を残差補償情報Ｉ₅の一部として符号化部１２に出力する。
【００４２】
逆量子化部６ｄは、量子化補償係数Ｉ₁₄を逆量子化し、逆量子化補償係数Ｉ₁₅を生成する。逆量子化部６ｄは、逆量子化補償係数Ｉ₁₅を逆変換部６ｅに出力する。逆変換部６ｅは、逆量子化補償係数Ｉ₁₅、基底情報Ｉ₁₂、及び基底記憶部６ｂに記憶されている基底を用い、変換部６ａによって行われる上述の変換処理の逆変換処理を実行することによって、残差補償画像Ｉ₄を生成する。逆変換部６ｅは、残差補償画像Ｉ₄を減算部８及び加算部１６に出力する。
【００４３】
上述したように、符号化部１２は、動き補償予測部２によって出力された動き情報Ｉ₂を符号化してなる符号化データ、量子化変換部１０によって出力された量子化変換係数Ｉ₇を符号化してなる符号化データ、及び、残差補償部６よって出力された残差補償情報Ｉ₅を符号化してなる符号化データを生成する。符号化部１２は、これら符号化データを含む圧縮データＩ₁₆を生成する。この符号化処理には、例えば、ハフマン符号化や算術符号化といったエントロピー符号化処理を用いることができる。符号化部１２は、動き情報Ｉ₂および量子化変換係数Ｉ₇に関しては、例えばＭＰＥＧ−４符号化方式などといった従来の動画像符号化方式に採用されているエントロピー符号化と同様の方法によって符号化を行うことができる。
【００４４】
残差補償情報Ｉ₅のエントロピー符号化について説明する。上述したように、残差補償情報Ｉ₅は基底情報Ｉ₁₂および量子化補償係数Ｉ₁₄を含んでいる。基底情報Ｉ₁₂は基底セットの中から１つの基底を特定するための情報であるので、例えば２^c個の基底が基底記憶部６ｂに記憶されている場合、ｃビットの固定長符号によって基底情報Ｉ₁₂を符号化することができる。また、量子化補償係数Ｉ₁₄は基底に乗じるスカラ値を量子化したものであり、例えばレンジを１〜２^dの範囲とすると、ｄビットの固定長符号によって量子化補償係数Ｉ₁₄を符号化することができる。
【００４５】
次に、動画像符号化装置１の動作について説明する。併せて、本発明の第１実施形態にかかる動画像符号化方法について説明する。図３は第１実施形態にかかる動画像符号化方法のフローチャートである。また、図４は、第１実施形態の動画像符号化方法における残差補償に関する処理のフローチャートである。
【００４６】
図３に示すように、第１実施形態にかかる動画像符号化方法においては、まず、動き補償予測部２によって動き補償予測が行われる（ステップＳ０１）。動き補償予測においては、符号化対象画像Ｉ₀が所定のサイズのブロックに分割される。そして、各々のブロックについて動き補償予測部２によって参照画像に対する動きベクトルが求められ、符号化対象画像Ｉ₀に関する予測画像Ｉ₁が生成される。参照画像としては、参照画像記憶部２０に記憶された画像が用いられる。また、動き補償予測部２によって、動きベクトルといった情報を含む上述の動き情報_I2が符号化部１２に出力される。
【００４７】
次に、動き補償予測部２によって出力された予測画像Ｉ₁と符号化対象画像Ｉ₀との差演算が、動き残差生成部４によって実行されることによって予測残差画像Ｉ₃が生成される（ステップＳ０２）。
【００４８】
次に、残差補償部６によって残差補償が行われる（ステップＳ０３）。残差補償においては、図４に示すように、変換部６ａによって基底記憶部６ｂに記憶されている基底を用いて予測残差画像Ｉ₃が分解される。そして、変換部６ａによって、予測残差画像Ｉ₃の分解に用いられた基底を特定するための基底情報Ｉ₁₂と基底に乗ずる補償係数Ｉ₁₃が出力される（ステップＳ０４）。
【００４９】
次に、量子化部６ｃによって補償係数Ｉ₁₃が量子化され、量子化補償係数Ｉ₁₄とされる（ステップＳ０５）。基底情報Ｉ₁₂および量子化補償係数Ｉ₁₄は残差補償情報Ｉ₅として符号化部１２に出力される。
【００５０】
次に、逆量子化部６ｄによって量子化補償係数Ｉ₁₄が、逆量子化され、逆量子化補償係数Ｉ₁₅とされる（ステップＳ０６）。そして、逆変換部６ｅによって、逆量子化補償係数Ｉ₁₅、基底情報Ｉ₁₂、及び基底記憶部６ｂに記憶されている基底が用いられ変換部６ａによる変換処理の逆操作である逆変換処理が行われることによって、残差補償画像Ｉ₄が生成される。（ステップＳ０７）。
【００５１】
図３を参照すると、次に、残差補償部６によって出力された残差補償画像Ｉ₄と動き残差生成部４よって出力された予測残差画像Ｉ₃との差演算が減算部８によって実行されることによって、補償残差画像Ｉ₆が生成される（ステップＳ０８）。
【００５２】
次に、量子化変換部１０によって、補償残差画像Ｉ₆が上述したように係数の集合に分解された後、その集合に含まれる係数に量子化操作が施されることによって量子化変換係数Ｉ₇が出力される（ステップＳ０９）。
【００５３】
次に、逆量子化変換部１４によって量子化変換係数Ｉ₇に逆量子化操作が施されることによって、逆量子化変換係数が生成される。そして、逆量子化変換係数に、量子化変換部１０によって実行された変換処理の逆変換処理が施され、復号補償残差画像Ｉ₈が生成される（ステップＳ１０）。
【００５４】
次に、復号補償残差画像Ｉ₈と残差補償画像Ｉ₄が、加算部１６によって加算され、復号予測残差画像Ｉ₉が生成される（ステップＳ１１）。さらに、復号画像生成部１８によって復号予測残差画像Ｉ₉と予測画像Ｉ₁が加算されることによって、復号画像Ｉ₁₀が生成される（ステップＳ１２）。復号画像Ｉ₁₀は、上述したように参照画像記憶部２０によって参照画像Ｉ₁₁として記憶され（ステップＳ１３）、動き補償予測部２によって符号化対象画像の予測画像を生成する処理に用いられる。そして、動き情報Ｉ₂と残差補償情報Ｉ₅と、量子化変換係数Ｉ₇とが符号化部１２によって符号化されることによって、圧縮データＩ₁₆が生成される（ステップＳ１４）。
【００５５】
次に、コンピュータを動画像符号化装置１として機能させる動画像符号化プログラム３０について説明する。図５は、動画像符号化プログラム３０の構成を示す図である。図５に示すように動画像符号化プログラム３０は、処理を統括するメインモジュール３１と、動き補償予測モジュール３２と、動き残差生成モジュール３４と、残差補償モジュール３６と、減算モジュール３８と、量子化変換モジュール４０と、符号化モジュール４２と、逆量子化変換モジュール４４と、加算モジュール４６と、復号画像生成モジュール４８と、参照画像記憶モジュール５０とを備える。残差補償モジュール３６は、変換サブモジュール３６ａと、基底記憶サブモジュール３６ｂと、量子化サブモジュール３６ｃと、逆量子化サブモジュール３６ｄと、逆変換サブモジュール３６ｅとを備えている。
【００５６】
動き補償予測モジュール３２、動き残差生成モジュール３４、残差補償モジュール３６、減算モジュール３８、量子化変換モジュール４０、符号化モジュール４２、逆量子化変換モジュール４４、加算モジュール４６、復号画像生成モジュール４８、参照画像記憶モジュール５０、変換サブモジュール３６ａ、基底記憶サブモジュール３６ｂ、量子化サブモジュール３６ｃ、逆量子化サブモジュール３６ｄ、逆変換サブモジュール３６ｅがコンピュータに実現させる機能は、動き補償予測部２、動き残差生成部４、残差補償部６、減算部８、量子化変換部１０、符号化部１２、逆量子化変換部１４、加算部１６、復号画像生成部１８、参照画像記憶部２０、変換部６ａ、基底記憶部６ｂ、量子化部６ｃ、逆量子化部６ｄ、逆変換部６ｅとそれぞれ同様である。
【００５７】
以下、第１実施形態にかかる動画像符号化装置１の作用及び効果について説明する。動画像符号化装置１では、符号化対象画像に対して動き補償予測を施して生成される予測画像と上記符号化対象画像との差分である予測残差画像に対してＭＰ法を適用することによって、少ない基底数で効率よく予測残差画像に出現しやすい信号パターンを表すことができる。例えば、エッジ部分の画像のようにＤＣＴやウェーブレット変換のような直交変換ではすべての周波数帯域に係数が出現するため、効率的に符号化を行うことができない。しかし、二次元基底にエッジ形状のものを予め記憶することによって、少ない数の基底を用いて予測残差画像を表現することが可能となるので、より効率的な符号化が実現される。
【００５８】
また、残差補償画像と予測残差画像との差分である補償残差画像は、予測残差画像において出現しやすいパターンを取り除かれた画像であるため、ＤＣＴを用いて効率よく表すことができる。このように、本発明の第１実施形態にかかる動画像符号化装置１においては、予測残差画像と残差補償画像をそれぞれに適した方法で分解することができる。その結果、符号化対象画像をより効率よく符号化することができる。
【００５９】
なお、本実施形態における変換部６ａは、所定の第１の変換規則としてＭＰ法に従うように構成されていたが、ＭＰ法に限らず、特定のパターンをより効果的に分解できる変換規則に従う限り、本発明の思想による効果が得られる。例えば、所定の第１の変換規則には、ＭＰ法の他にベクトル量子化法（以下ＶＱ法とよぶ）を適用することができる。
【００６０】
ＶＱ法は、予測残差画像を初期の残差成分とし、下式（２）を用いて残差成分を基底セットを用いて分解する処理を反復するものである。
【数２】

ここで、式（２）において、ｆは予測残差画像であり、Ｒ_mｆは第ｍ反復処理後の残差成分である。Ｒ_mｆのエネルギーを最小とする（係数ａ_n，基底ｖ_kn）の組み合わせを選択し、選択された基底とこの基底に乗じる為の係数に残差成分が分解される。ＶＱ法では、予測残差画像に出現しやすいパターンを基底とする基底セットを用いることによって、基底セットの中から残差成分が小さくなるような基底を用いて画像を順次分解するので、少ない基底数でより効率的に予測残差画像を分解することができる。
【００６１】
次に、本発明の第２実施形態にかかる動画像符号化装置６０について説明する。動画像符号化装置６０は、参照復号予測残差画像Ｉ₁₇を記憶するための復号予測残差画像記憶部（復号予測残差画像記憶手段）６８を備え、残差補償における変換で用いられる基底として復号予測残差画像記憶部６８に記憶された参照復号予測残差画像Ｉ₁₇の部分画像を用いることを特徴とする。以下、動画像符号化装置６０について詳細に説明する。
【００６２】
動画像符号化装置６０の機能的な構成について説明する。図６は、動画像符号化装置６０の機能的な構成を示す図である。動画像符号化装置６０は、機能的には、動き補償予測部２と、動き残差生成部４と、残差補償部（残差補償手段）６２と、減算部８と、量子化変換部１０と、符号化部（符号化手段）６４と、逆量子化変換部１４と、加算部（加算手段）６６と、復号画像生成部１８と、参照画像記憶部２０と、復号予測残差画像記憶部（復号予測残差画像記憶手段）６８とを備える。以下、第１実施形態の動画像符号化装置１と異なる要素について説明する。
【００６３】
復号予測残差画像記憶部６８は、加算部６６によって出力された復号予測残差画像Ｉ₉を参照復号予測残差画像Ｉ₁₇として記憶し、記憶した参照復号予測残差画像Ｉ₁₇を残差補償部６２に提供する。
【００６４】
残差補償部６２は、動き残差生成部４によって出力された予測残差画像Ｉ₃に対する残差補償を行い残差補償画像Ｉ₄を生成する。また、残差補償部６２は、残差補償を再現するために必要な情報を残差補償情報Ｉ₅として符号化部６４に出力する。
【００６５】
以下、残差補償部６２について詳細に説明する。図７は、残差補償部６２の構成を示す図である。残差補償部６２は、変換部（変換手段）６２ａと、量子化部（量子化手段）６２ｂと、逆量子化部（逆量子化手段）６２ｃと、逆変換部（逆変換手段）６２ｄとを有している。
【００６６】
変換部６２ａは、第１実施形態の変換部６ａと同様に、予測残差画像Ｉ₃に所定の第１の変換規則を用いた変換処理を施す。変換部６２ａは、この所定の第１の変換規則として、ＭＰ法やＶＱ法を適用することができる。第１実施形態の変換部６ａは、ＭＰ法やＶＱ法に用いる基底として基底記憶部６ｂに記憶されている基底を用いていたが、変換部６２ａは、参照復号予測残差画像Ｉ₁₇の部分画像を基底として用いる。
【００６７】
図８は、変換部６２ａによる変換処理を説明するための図である。図８に示すように、変換部６２ａは、予測残差画像Ｉ₃における注目領域Ｒ₁の画像の変換処理に、参照復号予測残差画像Ｉ₁₇の対応する領域Ｒ₂の部分画像とその近傍の領域Ｒ₃における部分画像を基底として用いる。変換部６２ａは、これらの部分画像のうち、注目領域Ｒ₁の画像に対する残差エネルギーを最小とする部分画像を基底として選択することによって、注目領域Ｒ₁の画像を基底に分解する。変換部６２ａは、分解に用いた基底（部分画像）を特定するための情報を基底情報Ｉ₁₂として、逆変換部６２ｄに出力する。また、変換部６２ａは、基底情報Ｉ₁₂を、残差補償情報Ｉ₅の一部として符号化部６４に出力する。また、変換部６２ａは、基底に乗じる係数を補償係数Ｉ₁₃として、量子化部６２ｂに出力する。
【００６８】
量子化部６２ｂは、変換部６２ａによって出力された補償係数Ｉ₁₃に量子化操作を施すことによって、量子化補償係数Ｉ₁₄を生成する。量子化部６２ｂは、量子化補償係数Ｉ₁₄を逆量子化部６２ｃに出力すると共に、量子化補償係数Ｉ₁₄を残差補償情報Ｉ₅の一部として符号化部６４に出力する。
【００６９】
逆量子化部６２ｃは、量子化補償係数Ｉ₁₄を逆量子化し、逆量子化補償係数Ｉ₁₅を生成する。逆量子化部６２ｃは、逆量子化補償係数Ｉ₁₅を逆変換部６２ｄに出力する。逆変換部６２ｄは、逆量子化補償係数Ｉ₁₅、基底情報Ｉ₁₂、及び参照復号予測残差画像Ｉ₁₇を用い、変換部６２ａによって行われる上述の変換処理の逆変換処理を実行することによって、残差補償画像Ｉ₄を生成する。逆変換部６２ｄは、残差補償画像Ｉ₄を減算部８及び加算部６６に出力する。
【００７０】
符号化部６４は、動き補償予測部２によって出力された動き情報Ｉ₂と、量子化変換部１０によって出力された量子化変換係数Ｉ₇と、残差補償部６２によって出力された残差補償情報Ｉ₅をそれぞれ圧縮してなる符号化データを生成し、これら符号化データを含む圧縮データを出力する。なお、符号化部６４による圧縮処理は、第１実施形態の符号化部１２と同様の処理である。
【００７１】
以下、動画像符号化装置６０の動作を説明し、併せて本発明の第２実施形態にかかる動画像符号化方法について説明する。図９は、第２実施形態にかかる動画像符号化方法のフローチャートである。図１０は、第２実施形態の動画像符号化方法における残差補償に関する処理のフローチャートである。
【００７２】
図９に示すように、第２実施形態にかかる動画像符号化方法においては、まず、動き補償予測部２によって動き補償予測が行われる（ステップＳ２１）。ステップＳ２１における処理は、第１実施形態のステップＳ０１の処理と同様である。
【００７３】
次に、動き補償予測部２によって出力された予測画像Ｉ₁と符号化対象画像Ｉ₀との差演算が、動き残差生成部４によって実行されることによって予測残差画像Ｉ₃が生成される（ステップＳ２２）。
【００７４】
次に、残差補償部６２によって残差補償が行われる（ステップＳ２３）。残差補償においては、図１０に示すように、変換部６２ａによって予測残差画像Ｉ₃が分解される。ここで、変換部６２ａは、上述したように、復号予測残差画像記憶部６８によって提供される参照復号予測残差画像Ｉ₁₇の部分画像を基底として用い予測残差画像Ｉ₃の分解を行う。そして、変換部６２ａによって、予測残差画像Ｉ₃の分解に用いられた基底（部分画像）を特定するための基底情報Ｉ₁₂と基底に乗ずる補償係数Ｉ₁₃が出力される（ステップＳ２４）。
【００７５】
次に、量子化部６２ｂによって補償係数Ｉ₁₃が量子化され、量子化補償係数Ｉ₁₄とされる（ステップＳ２５）。基底情報Ｉ₁₂および量子化補償係数Ｉ₁₄は残差補償情報Ｉ₅として符号化部６４に出力される。
【００７６】
次に、逆量子化部６２ｃによって量子化補償係数Ｉ₁₄が、逆量子化され、逆量子化補償係数Ｉ₁₅とされる（ステップＳ２６）。そして、逆変換部６２ｄによって、かかる逆量子化補償係数Ｉ₁₅、基底情報Ｉ₁₂、及び参照復号予測残差画像Ｉ₁₇が用いられ、変換部６２ａによる変換処理の逆操作である逆変換処理が行われることによって、残差補償画像Ｉ₄が生成される。（ステップＳ２７）。
【００７７】
図９を参照すると、次に、残差補償部６によって出力された残差補償画像Ｉ₄と動き残差生成部４よって出力された予測残差画像Ｉ₃との差演算が減算部８によって実行されることによって、補償残差画像Ｉ₆が生成される（ステップＳ２８）。
【００７８】
次に、量子化変換部１０によって、補償残差画像Ｉ₆が上述したように係数の集合に分解された後、その集合に含まれる係数に量子化操作が施されることによって量子化変換係数Ｉ₇が出力される（ステップＳ２９）。
【００７９】
次に、逆量子化変換部１４によって量子化変換係数Ｉ₇に逆量子化操作が施されることによって、逆量子化変換係数が生成される。そして、逆量子化変換係数に、量子化変換部１０によって実行された変換処理の逆変換処理が施され、復号補償残差画像Ｉ₈が生成される（ステップＳ３０）。
【００８０】
次に、復号補償残差画像Ｉ₈と残差補償画像Ｉ₄が、加算部６６によって加算され、復号予測残差画像Ｉ₉が生成される（ステップＳ３１）。復号予測残差画像Ｉ９は、復号予測残差画像記憶部６８によって参照復号予測残差画像Ｉ₁₇として記憶され、残差補償部６２による処理に提供される（ステップＳ３２）。
【００８１】
次に、復号画像生成部１８によって復号予測残差画像Ｉ₉と予測画像Ｉ₁が加算されることによって、復号画像Ｉ₁₀が生成される（ステップＳ３３）。復号画像Ｉ₁₀は、上述したように参照画像記憶部２０によって参照画像Ｉ₁₁として記憶され（ステップＳ３４）、動き補償予測部２によって符号化対象画像の予測画像を生成する処理に用いられる。そして、動き情報Ｉ₂と残差補償情報Ｉ₅と、量子化変換係数Ｉ₇とが符号化部６４によって符号化されることによって、圧縮データＩ₁₆が生成される（ステップＳ３５）。
【００８２】
次に、コンピュータを動画像符号化装置６０として機能させる動画像符号化プログラム７０について説明する。図１１は、動画像符号化プログラム７０の構成を示す図である。図１１に示すように動画像符号化プログラム７０は、処理を統括するメインモジュール３１と、動き補償予測モジュール３２と、動き残差生成モジュール３４と、残差補償モジュール７２と、減算モジュール３８と、量子化変換モジュール４０と、符号化モジュール７４と、逆量子化変換モジュール４４と、加算モジュール７６と、復号予測残差画像記憶モジュール７８と、復号画像生成モジュール４８と、参照画像記憶モジュール５０とを備える。残差補償モジュール７２は、変換サブモジュール７２ａと、量子化サブモジュール７２ｂと、逆量子化サブモジュール７２ｃと、逆変換サブモジュール７２ｄとを備えている。
【００８３】
動き補償予測モジュール３２、動き残差生成モジュール３４、残差補償モジュール７２、減算モジュール３８、量子化変換モジュール４０、符号化モジュール７４、逆量子化変換モジュール４４、加算モジュール７６、復号予測残差画像記憶モジュール７８、復号画像生成モジュール４８、参照画像記憶モジュール５０、変換サブモジュール７２ａ、量子化サブモジュール７２ｂ、逆量子化サブモジュール７２ｃ、逆変換サブモジュール７２ｄがコンピュータに実現させる機能は、動き補償予測部２、動き残差生成部４、残差補償部６２、減算部８、量子化変換部１０、符号化部６４、逆量子化変換部１４、加算部６６、復号予測残差画像記憶部６８、復号画像生成部１８、参照画像記憶部２０、変換部６２ａ、量子化部６２ｂ、逆量子化部６２ｃ、逆変換部６２ｄとそれぞれ同様である。
【００８４】
以上のように、残差補償における変換処理に用いる基底として、復号予測残差画像記憶部６８に記憶された参照復号予測残差画像（復号予測残差画像）の部分画像を用いることによって、予測残差画像の類似性を利用した更に効果的な残差補償を行うことができるので、効率の良い符号化を実現することができる。
【００８５】
次に、本発明の第３実施形態にかかる動画像符号化装置８０について説明する。動画像符号化装置８０は、基底セットを記憶するための基底記憶部（基底記憶手段）８２ｂと、参照復号予測残差画像Ｉ₁₇を記憶するための復号予測残差画像記憶部（復号予測残差画像記憶手段）６８とを備え、残差補償における変換で用いられる基底として基底記憶部８２ｂに記憶された基底セット、又は、復号予測残差画像記憶部６８に記憶された参照復号予測残差画像Ｉ₁₇の部分画像を用いることを特徴とする。
【００８６】
動画像符号化装置８０の機能的な構成について説明する。図１２は、動画像符号化装置８０の機能的な構成を示す図である。動画像符号化装置８０は、機能的には、動き補償予測部２と、動き残差生成部４と、残差補償部（残差補償手段）８２と、減算部８と、量子化変換部１０と、符号化部（符号化手段）８４と、逆量子化変換部１４と、加算部６６と、復号画像生成部１８と、参照画像記憶部２０と、復号予測残差画像記憶部６８とを備える。以下、第１実施形態の動画像符号化装置１、又は、第２実施形態の動画像符号化装置６０と異なる要素について説明する。
【００８７】
残差補償部８２は、予測残差画像Ｉ₃に対する残差補償を行い残差補償画像Ｉ₄を生成する。残差補償部８２は、生成した残差補償画像Ｉ₄を減算部８及び加算部６６に出力する。また、残差補償部８２は、残差補償を再現するために必要な情報を残差補償情報Ｉ₅として符号化部８４に出力する。
【００８８】
以下、残差補償部８２について詳細に説明する。図１３は、残差補償部８２の構成を示す図である。図１３に示すように、残差補償部８２は、変換部（変換手段）８２ａと、基底記憶部（基底記憶手段）８２ｂと、量子化部（量子化手段）８２ｃと、逆量子化部（逆量子化手段）８２ｄと、逆変換部（逆変換手段）８２ｅとを有している。
【００８９】
変換部８２ａは、予測残差画像Ｉ₃に所定の第１の変換規則を用いた変換処理を施す。所定の第１の変換規則としては、第１実施形態と同様にＭＰ法、或いはＶＱ法を用いることができる。
【００９０】
本実施形態では、基底記憶部８２ｂが複数の二次元基底を含む基底セットを記憶している。変換部８２ａは、第１実施形態と同様に基底記憶部８２ｂによって提供される基底セットを用いて予測残差画像Ｉ₃を分解することができ、また、第２実施形態と同様に復号予測残差画像記憶部６８から提供される参照復号予測残差画像Ｉ₁₇中の部分画像を用いて予測残差画像Ｉ₃を分解することもできる。
【００９１】
変換部８２ａは、基底記憶部８２ｂに記憶されている基底セットまたは復号予測残差画像記憶部６８から提供される参照復号予測残差画像Ｉ₁₇中の部分画像のうち、エネルギー減少量が最大のものを基底として選択し、予測残差画像Ｉ₃の分解を行う。変換部８２ａは、この分解に用いられた基底を特定するための情報を基底情報Ｉ₁₂として逆変換部６２ｄに出力する。また、変換部８２ａは、基底情報Ｉ₁₂を、残差補償情報Ｉ₅の一部として符号化部８４に出力する。また、変換部６２ａは、基底に乗じる係数を補償係数Ｉ₁₃として、量子化部８２ｃに出力する。
【００９２】
量子化部８２ｃは、変換部８２ａによって出力された補償係数Ｉ₁₃に量子化操作を施すことによって、量子化補償係数Ｉ₁₄を生成する。量子化部８２ｃは、量子化補償係数Ｉ₁₄を逆量子化部８２ｄに出力すると共に、量子化補償係数Ｉ₁₄を残差補償情報Ｉ₅の一部として符号化部８４に出力する。
【００９３】
逆量子化部８２ｄは、量子化補償係数Ｉ₁₄を逆量子化し、逆量子化補償係数Ｉ₁₅を生成する。逆量子化部８２ｄは、逆量子化補償係数Ｉ₁₅を逆変換部８２ｅに出力する。逆変換部８２ｅは、逆量子化補償係数Ｉ₁₅、基底情報Ｉ₁₂、基底記憶部８２ｂに記憶された基底セット、及び参照復号予測残差画像Ｉ₁₇を用い、変換部８２ａによって行われる上述の変換処理の逆変換処理を実行することによって、残差補償画像Ｉ₄を生成する。逆変換部８２ｅは、残差補償画像Ｉ₄を減算部８及び加算部６６に出力する。
【００９４】
符号化部６４は、動き補償予測部２によって出力された動き情報Ｉ₂と、量子化変換部１０によって出力された量子化変換係数Ｉ₇と、残差補償部６２によって出力された残差補償情報Ｉ₅をそれぞれ圧縮してなる符号化データを生成し、これら符号化データを含む圧縮データを出力する。なお、符号化部８４による圧縮処理は、第１実施形態の符号化部１２と同様の処理である。
【００９５】
以下、動画像符号化装置８０の動作を説明し、併せて本発明の第３実施形態にかかる動画像符号化方法について説明する。図１４は、第３実施形態にかかる動画像符号化方法のフローチャートである。図１５は、第３実施形態の動画像符号化方法における残差補償に関する処理のフローチャートである。
【００９６】
図１４に示すように、第３実施形態にかかる動画像符号化方法においては、まず、動き補償予測部２によって動き補償予測が行われる（ステップＳ４１）。ステップＳ４１における処理は、第１実施形態のステップＳ０１の処理と同様である。
【００９７】
次に、動き補償予測部２によって出力された予測画像Ｉ₁と符号化対象画像Ｉ₀との差演算が、動き残差生成部４によって実行されることによって予測残差画像Ｉ₃が生成される（ステップＳ４２）。
【００９８】
次に、残差補償部８２によって残差補償が行われる（ステップＳ４３）。残差補償においては、図１５に示すように、変換部８２ａによって予測残差画像Ｉ₃が分解される。ここで、変換部８２ａは、上述したように、基底記憶部８２ｂに記憶された記憶セット及び復号予測残差画像記憶部６８によって提供される参照復号予測残差画像Ｉ₁₇の部分画像を基底として用い予測残差画像Ｉ₃の分解を行う。そして、変換部８２ａによって、予測残差画像Ｉ₃の分解に用いられた基底を特定するための基底情報Ｉ₁₂と基底に乗ずる補償係数Ｉ₁₃が出力される（ステップＳ４４）。
【００９９】
次に、量子化部８２ｃによって補償係数Ｉ₁₃が量子化され、量子化補償係数Ｉ₁₄とされる（ステップＳ４５）。基底情報Ｉ₁₂および量子化補償係数Ｉ₁₄は残差補償情報Ｉ₅として符号化部６４に出力される。
【０１００】
次に、逆量子化部８２ｄによって量子化補償係数Ｉ₁₄が、逆量子化され、逆量子化補償係数Ｉ₁₅とされる（ステップＳ４６）。そして、逆変換部８２ｅによって、かかる逆量子化補償係数Ｉ₁₅、基底情報Ｉ₁₂、基底記憶部８２ｂに記憶された基底セット、及び参照復号予測残差画像Ｉ₁₇が用いられ、変換部６２ａによる変換処理の逆操作である逆変換処理が行われることによって、残差補償画像Ｉ₄が生成される。（ステップＳ４７）。
【０１０１】
図１４を参照すると、次に、残差補償部８２によって出力された残差補償画像Ｉ₄と動き残差生成部４よって出力された予測残差画像Ｉ₃との差演算が減算部８によって実行されることによって、補償残差画像Ｉ₆が生成される（ステップＳ４８）。
【０１０２】
次に、量子化変換部１０によって、補償残差画像Ｉ₆が上述したように係数の集合に分解された後、その集合に含まれる係数に量子化操作が施されることによって量子化変換係数Ｉ₇が出力される（ステップＳ４９）。
【０１０３】
次に、逆量子化変換部１４によって量子化変換係数Ｉ₇に逆量子化操作が施されることによって、逆量子化変換係数が生成される。そして、逆量子化変換係数に、量子化変換部１０によって実行された変換処理の逆変換処理が施され、復号補償残差画像Ｉ₈が生成される（ステップＳ５０）。
【０１０４】
次に、復号補償残差画像Ｉ₈と残差補償画像Ｉ₄が、加算部６６によって加算され、復号予測残差画像Ｉ₉が生成される（ステップＳ５１）。復号予測残差画像Ｉ₉は、復号予測残差画像記憶部６８によって参照復号予測残差画像Ｉ₁₇として記憶され、残差補償部８２による処理に提供される（ステップＳ５２）。
【０１０５】
次に、復号画像生成部１８によって復号予測残差画像Ｉ₉と予測画像Ｉ₁が加算されることによって、復号画像Ｉ₁₀が生成される（ステップＳ５３）。復号画像Ｉ₁₀は、上述したように参照画像記憶部２０によって参照画像Ｉ₁₁として記憶され（ステップＳ５４）、動き補償予測部２によって符号化対象画像の予測画像を生成する処理に用いられる。そして、動き情報Ｉ₂と残差補償情報Ｉ₅と、量子化変換係数Ｉ₇とが符号化部８４によって符号化されることによって、圧縮データＩ₁₆が生成される（ステップＳ５５）。
【０１０６】
次に、コンピュータを動画像符号化装置８０として機能させる動画像符号化プログラム９０について説明する。図１６は、動画像符号化プログラム９０の構成を示す図である。図１６に示すように動画像符号化プログラム９０は、処理を統括するメインモジュール３１と、動き補償予測モジュール３２と、動き残差生成モジュール３４と、残差補償モジュール９２と、減算モジュール３８と、量子化変換モジュール４０と、符号化モジュール９４と、逆量子化変換モジュール４４と、加算モジュール７６と、復号予測残差画像記憶モジュール７８と、復号画像生成モジュール４８と、参照画像記憶モジュール５０とを備える。残差補償モジュール９２は、変換サブモジュール９２ａと、基底記憶サブモジュール９２ｂと、量子化サブモジュール９２ｃと、逆量子化サブモジュール９２ｄと、逆変換サブモジュール９２ｅとを備えている。
【０１０７】
動き補償予測モジュール３２、動き残差生成モジュール３４、残差補償モジュール９２、減算モジュール３８、量子化変換モジュール４０、符号化モジュール９４、逆量子化変換モジュール４４、加算モジュール７６、復号予測残差画像記憶モジュール７８、復号画像生成モジュール４８、参照画像記憶モジュール５０、変換サブモジュール９２ａ、基底記憶サブモジュール９２ｂ、量子化サブモジュール９２ｃ、逆量子化サブモジュール９２ｄ、逆変換サブモジュール９２ｅがコンピュータに実現させる機能は、動き補償予測部２、動き残差生成部４、残差補償部８２、減算部８、量子化変換部１０、符号化部８４、逆量子化変換部１４、加算部６６、復号予測残差画像記憶部６８、復号画像生成部１８、参照画像記憶部２０、変換部８２ａ、基底記憶部８２ｂ、量子化部８２ｃ、逆量子化部８２ｄ、逆変換部８２ｅとそれぞれ同様である。
【０１０８】
以上のように、動画像符号化装置８０によれば、残差補償における変換で用いられる基底として復号予測残差画像記憶部６８に記憶された参照復号予測残差画像Ｉ₁₇の部分画像と基底記憶部８２ｂに記憶された基底セットとが併用される。したがって、動画像符号化装置８０は、予測残差画像の類似性を利用して効果的に予測残差画像Ｉ₃の残差補償を行うことができ、また、フレーム間に急激な変化が生じている場合には基底記憶部８２ｂに記憶された基底セットを用いることができるので、効率のよい符号化を実現することができる。
【０１０９】
次に、本発明の第４実施形態にかかる動画像復号装置１００について説明する。動画像復号装置１００は、第１実施形態の動画像符号化装置１によって生成された圧縮データを復号して動画像を生成する装置である。動画像復号装置１００は、物理的には、ＣＰＵ（中央演算装置）、メモリといった記憶装置、ハードディスクといった記憶装置等を備えるコンピュータである。ここでの「コンピュータ」とは、パーソナルコンピュータ等の通常のコンピュータに加えて、移動通信端末といった情報携帯端末も含むものであり、本発明の思想は情報処理可能な機器に広く適用される。
【０１１０】
以下、動画像復号装置１００の機能的な構成について説明する。図１７は、動画像復号装置１００の機能的な構成を示す図である。動画像復号装置１００は、機能的には、復号部（復号手段）１０２と、逆量子化変換部（逆量子化変換手段）１０４と、残差復号部（残差復号手段）１０６と、加算部（加算手段）１０８と、動き補償予測部（動き補償予測手段）１１０と、復号画像生成部（復号画像生成手段）１１２と、参照画像記憶部（参照画像記憶手段）１１４と、を備える。
【０１１１】
復号部１０２は、動画像符号化装置１によって生成された圧縮データＩ₁₆を復号する。復号部１０２は、圧縮データＩ₁₆を復号することによって得られる量子化変換係数Ｉ₇を逆量子化変換部１０４に出力する。また、復号部１０２は、圧縮データＩ₁₆を復号することによって得られる残差補償情報Ｉ₅を残差復号部１０６に出力する。また、復号部１０２は、圧縮データＩ₁₆を復号することによって得られる動き情報Ｉ₂を動き補償予測部１１０に出力する。
【０１１２】
逆量子化変換部１０４は、上記の量子化変換係数に逆量子化操作を施してなる逆量子化変換係数を生成する。逆量子化変換部１０４は、上述した所定の第２の逆変換規則を用い、逆量子化変換係数を逆変換することによって復号補償残差画像Ｉ₈を生成する。この所定の第２の逆変換規則は、第１実施形態の動画像符号化装置１によって用いられる所定の第２の変換規則の逆変換操作である。
【０１１３】
残差復号部１０６は、復号部１０２によって出力される残差補償情報Ｉ₅を用いて残差復号を行うことによって残差補償画像Ｉ₄を生成する。残差復号部１０６による処理の詳細については後述する。
【０１１４】
加算部１０８は、逆量子化変換部１０４によって生成された復号補償残差画像Ｉ₈と残差復号部１０６によって生成された残差補償画像Ｉ₄を加算して復号予測残差画像Ｉ₉を生成する。
【０１１５】
動き補償予測部１１０は、復号部１０２によって出力された動き情報Ｉ₂を用いて、参照画像記憶部１１４に記憶された参照画像Ｉ₁₁に対する動き補償予測を行い、復号画像Ｉ₁₀の予測画像Ｉ₁を生成する。
【０１１６】
復号画像生成部１１２は、加算部１０８によって生成された復号予測残差画像Ｉ₉と動き補償予測部１１０によって生成された予測画像Ｉ₁を加算して復号画像Ｉ₁₀を生成する。復号画像生成部１１２は、復号画像Ｉ₁₀を参照画像記憶部１１４へ出力し、復号画像Ｉ₁₀を参照画像記憶部１１４に記憶させる。参照画像記憶部１１４は、記憶した復号画像Ｉ₁₀を、動き補償予測のために動き補償予測部１１０に参照画像Ｉ₁₁として提供する。
【０１１７】
次に、残差復号部１０６の詳細について説明する。残差復号部１０６は、残差補償情報Ｉ₅に含まれる量子化補償係数Ｉ₁₄を逆量子化してなる逆量子化補償係数Ｉ₁₅と、残差補償情報Ｉ₅に含まれる基底情報Ｉ₁₂を用いて所定の第１の逆変換規則を用いて逆変換を行い残差補償画像Ｉ₄を生成する。この所定の第１の逆変換規則は、第１実施形態の動画像符号化装置１によって用いられる所定の第１の変換規則の逆変換操作である。
【０１１８】
図１８は、残差復号部１０６の構成を示す図である。残差復号部１０６は、逆量子化部（逆量子化手段）１０６ａと、逆変換部（逆変換手段）１０６ｂと、基底記憶部（基底記憶手段）１０６ｃとを備えている。
【０１１９】
逆量子化部１０６ａは、量子化補償係数Ｉ₁₄を逆量子化し、逆量子化補償係数Ｉ₁₅を生成する。逆変換部１０６ｂは、基底記憶部１０６ｃに記憶されている基底セットのうちから基底情報Ｉ₁₂によって特定される基底を抽出する。逆変換部１０６ｂは、抽出した基底と逆量子化部１０６ａによって生成された逆量子化補償係数Ｉ₁₅を用いて、変換部６ａによる変換処理の逆操作である逆変換処理を行い、残差補償画像Ｉ₄を生成する。なお、基底記憶部１０６ｃには、基底記憶部６ｂに記憶されている基底と同様の基底を含む基底セットを記憶させておくことができる。
【０１２０】
以下、動画像復号装置１００の動作について説明し、併せて第４実施形態にかかる動画像復号方法について説明する。図１９は、第４実施形態の動画像復号方法のフローチャートである。また、図２０は、第４実施形態の動画像復号方法の残差復号に関する処理を示すフローチャートである。
【０１２１】
図１９に示すように、第４実施形態の動画像復号方法においては、まず、動画像符号化装置１によって生成された圧縮データＩ₁₆から、復号部１０２によって量子化変換係数Ｉ₇と残差補償情報Ｉ₅と動き情報Ｉ₂が復号される（ステップＳ６１）。
【０１２２】
次に、逆量子化変換部１０４が、量子化変換係数Ｉ₇に逆量子化操作を施すことによって逆量子化変換係数を生成する。逆量子化変換部１０４は、生成した逆量子化変換係数に、所定の第２の逆変換規則による逆変換を施すことによって、復号補償残差画像Ｉ₈を復元する（ステップＳ６２）。
【０１２３】
次に、残差復号部１０６が、復号部１０２によって復号された残差補償情報Ｉ₅を用いて、残差復号を行い、残差補償画像Ｉ₄を生成する（ステップＳ６３）。図２０に示すように残差復号部１０６では、逆量子化部１０６ａが量子化補償係数Ｉ₁₄を逆量子化することによって逆量子化補償係数Ｉ₁₅を生成する（ステップＳ６４）。
【０１２４】
次に、逆変換部１０６ｂが、逆量子化補償係数Ｉ₁₅、残差補償情報Ｉ₅に含まれる基底情報Ｉ₁₂、及び基底記憶部１０６ｃに記憶され基底情報Ｉ₁₂によって特定される基底を用いて、変換部６ａによる変換処理の逆操作である逆変換処理を行い、残差補償画像Ｉ₄を生成する（ステップＳ６５）。
【０１２５】
図１９を参照すると、次に、加算部１０８が復号補償残差画像Ｉ₈と残差補償画像Ｉ₄とを加算することによって復号予測残差画像Ｉ₉を生成する（ステップＳ６６）。
【０１２６】
次に、動き補償予測部１１０が、復号部１０２によって出力された動き情報Ｉ₂を用いて、参照画像記憶部１１４に記憶される参照画像Ｉ₁₁に対する動き補償予測を行うことによって復号画像Ｉ₁₀の予測画像Ｉ₁を生成する（ステップＳ６７）。
【０１２７】
次に、復号画像生成部１１２が、予測画像Ｉ₁と復号予測残差画像Ｉ₉とを加算することによって、復号画像Ｉ₁₀を生成する（ステップＳ６８）。復号画像生成部１１２は、復号画像Ｉ₁₀を参照画像記憶部１１４へ出力し、復号画像Ｉ₁₀を参照画像記憶部１１４に記憶させる（ステップＳ６９）。参照画像記憶部１１４は、記憶した復号画像Ｉ₁₀を、動き補償予測のために動き補償予測部１１０に参照画像Ｉ₁₁として提供する。
【０１２８】
以下、コンピュータを動画像復号装置１００として動作させる動画像復号プログラム１２０について説明する。図２１は、動画像復号プログラム１２０の構成を示す図である。動画像復号プログラム１２０は、処理を統括するメインモジュール１２１と、復号モジュール１２２と、逆量子化変換モジュール１２４と、残差復号モジュール１２６と、加算モジュール１２８と、動き補償予測モジュール１３０と、復号画像生成モジュール１３２と、参照画像記憶モジュール１３４とを備える。残差復号モジュール１２６は、逆量子化サブモジュール１２６ａと、逆変換サブモジュール１２６ｂと、基底記憶サブモジュール１２６ｃとを有している。
【０１２９】
復号モジュール１２２、逆量子化変換モジュール１２４、残差復号モジュール１２６、加算モジュール１２８、動き補償予測モジュール１３０、復号画像生成モジュール１３２、参照画像記憶モジュール１３４、逆量子化サブモジュール１２６ａ、逆変換サブモジュール１２６ｂ、基底記憶サブモジュール１２６ｃがコンピュータに実現させる機能は、復号部１０２、逆量子化変換部１０４、残差復号部１０６、加算部１０８、動き補償予測部１１０、復号画像生成部１１２、参照画像記憶部１１４、逆量子化部１０６ａ、逆変換部１０６ｂ、基底記憶部１０６ｃとそれぞれ同様である。
【０１３０】
以下、第４実施形態にかかる動画像復号装置１００の作用及び効果を説明する。動画像復号装置１００は、圧縮データＩ₁₆を復号することによって得られる情報に逆量子化変換処理、残差復号処理、及び動き補償予測処理を行う。すなわち、動画像復号装置１００によれば、動画像符号化装置１によって動画像から圧縮データが生成される処理の逆処理が行われるので、上記の動画像が忠実に復元される。
【０１３１】
次に、本発明の第５実施形態にかかる動画像復号装置１４０について説明する。動画像復号装置１４０は、第２実施形態の動画像符号化装置６０によって生成された圧縮データＩ₁₆から動画像を復元する装置である。動画像復号装置１４０では、参照復号予測残差画像Ｉ₁₇を記憶するための復号予測残差画像記憶部（復号予測残差画像記憶手段）１４８を備え、残差復号における逆変換で用いられる基底として復号予測残差画像記憶部１４８に記憶された参照復号予測残差画像Ｉ₁₇の部分画像を用いることを特徴とする。
【０１３２】
以下、動画像復号装置１４０の機能的な構成について説明する。図２２は、動画像復号装置１４０の機能的な構成を示す図である。動画像復号装置１４０は、機能的には、復号部（復号手段）１４２と、逆量子化変換部１０４と、残差復号部（残差復号部）１４４と、加算部（加算手段）１４６と、動き補償予測部（動き補償予測手段）１１０と、復号画像生成部１１２と、参照画像記憶部１１４と、復号予測残差画像記憶部（復号予測残差画像記憶手段）１４８とを備える。以下、第４実施形態の動画像復号装置１００と異なる要素について説明する。
【０１３３】
復号部１４２は、動画像符号化装置６０によって生成された圧縮データＩ₁₆を復号し、量子化変換係数Ｉ₇と、残差補償情報Ｉ₅と、動き情報Ｉ₂とを出力する。この残差補償情報Ｉ₅には、復号予測残差画像記憶部１４８に記憶される参照復号予測残差画像Ｉ₁₇の部分画像を用いて残差補償画像Ｉ₄を復元するための基底情報Ｉ₁₂と、量子化補償係数Ｉ₁₄が含まれている。
【０１３４】
残差復号部１４４は、残差補償情報Ｉ₅を用いて残差復号を行い残差補償画像Ｉ₄を生成する。以下、残差復号部１４４について詳細に説明する。図２３は、残差復号部１４４の構成を示す図である。図２３に示すように、残差復号部１４４は、逆量子化部（逆量子化手段）１４４ａと、逆変換部（逆変換手段）１４４ｂとを有する。
【０１３５】
逆量子化部１４４ａは、残差補償情報Ｉ₅に含まれている量子化補償係数Ｉ₁₄に逆量子化操作を施すことによって、逆量子化補償係数Ｉ₁₅を生成する。
【０１３６】
逆変換部１４４ｂは、復号予測残差画像記憶部１４８に記憶された参照復号予測残差画像Ｉ₁₇のうち、残差補償情報Ｉ₅に含まれている基底情報Ｉ₁₂から部分画像（基底）を特定する。逆変換部１４４ｂは、特定した部分画像と、逆量子化部１４４ａによって生成された逆量子化補償係数Ｉ₁₅とを用いて変換部６２ａによる変換処理の逆操作である逆変換処理を行い、残差補償画像Ｉ₄を生成する。
【０１３７】
図２２を参照すると、加算部１４６は、復号部１４２によって復号された量子化変換係数Ｉ₇が更に逆量子化変換部１０４によって逆量子化及び逆変換されてなる復号補償残差画像Ｉ₈と残差復号部１４４によって生成された残差補償画像Ｉ₄とを加算することによって、復号予測残差画像Ｉ₉を生成する。加算部１４６は、復号予測残差画像Ｉ₉を復号画像生成部１１２に出力すると共に、復号予測残差画像Ｉ₉を復号予測残差画像記憶部１４８に参照復号予測残差画像Ｉ₁₇として記憶させる。復号予測残差画像記憶部１４８は、記憶した参照復号予測残差画像Ｉ₁₇を残差復号のために残差復号部１４４に提供する。
【０１３８】
次に、動画像復号装置１４０の動作について説明し、併せて第５実施形態にかかる動画像復号方法について説明する。図２４は、第５実施形態の動画像復号方法のフローチャートである。また、図２５は、第５実施形態の動画像復号方法の残差復号に関する処理を示すフローチャートである。
【０１３９】
図２４に示すように、第５実施形態の動画像復号方法においては、まず、動画像符号化装置６０によって生成された圧縮データＩ₁₆から、復号部１４２によって量子化変換係数Ｉ₇と残差補償情報Ｉ₅と動き情報Ｉ₂が復号される（ステップＳ７１）。
【０１４０】
次に、逆量子化変換部１０４が、量子化変換係数Ｉ₇に逆量子化操作を施すことによって逆量子化変換係数を生成する。逆量子化変換部１０４は、生成した逆量子化変換係数に、所定の第２の逆変換規則による逆変換を施すことによって、復号補償残差画像Ｉ₈を復元する（ステップＳ７２）。
【０１４１】
次に、残差復号部１４４が、復号部１４２によって復号された残差補償情報Ｉ₅を用いて、残差復号を行い、残差補償画像Ｉ₄を生成する（ステップＳ７３）。図２５に示すように残差復号部１４４では、逆量子化部１４４ａが量子化補償係数Ｉ₁₄を逆量子化することによって逆量子化補償係数Ｉ₁₅を生成する（ステップＳ７４）。
【０１４２】
次に、逆変換部１４４ｂが、逆量子化補償係数Ｉ₁₅、残差補償情報Ｉ₅に含まれる基底情報Ｉ₁₂、及び復号予測残差画像記憶部１４８によって提供される参照復号予測残差画像Ｉ₁₇を用いて、変換部６２ａによる変換処理の逆操作である逆変換処理を行い、残差補償画像Ｉ₄を生成する（ステップＳ７５）。
【０１４３】
図２４を参照すると、次に、加算部１４６が復号補償残差画像Ｉ₈と残差補償画像Ｉ₄とを加算することによって復号予測残差画像Ｉ₉を生成する（ステップＳ７６）。
【０１４４】
次に、復号予測残差画像記憶部１４８が、加算部１４６によって生成された復号予測残差画像Ｉ₉を参照復号予測残差画像Ｉ₁₇として記憶する（ステップＳ７７）。復号予測残差画像記憶部１４８は、記憶した参照復号予測残差画像Ｉ₁₇を残差復号のために残差復号部１４４に提供する。
【０１４５】
次に、動き補償予測部１１０が、復号部１０２によって出力された動き情報Ｉ₂を用いて、参照画像記憶部１１４に記憶された参照画像Ｉ₁₁に対する動き補償予測を行うことによって復号画像Ｉ₁₀の予測画像Ｉ₁を生成する（ステップＳ７８）。
【０１４６】
次に、復号画像生成部１１２が、予測画像Ｉ₁と復号予測残差画像Ｉ₉とを加算することによって、復号画像Ｉ₁₀を生成する（ステップＳ７９）。復号画像生成部１１２は、復号画像Ｉ₁₀を参照画像記憶部１１４へ出力し、復号画像Ｉ₁₀を参照画像記憶部１１４に記憶させる（ステップＳ８０）。参照画像記憶部１１４は、記憶した復号画像Ｉ₁₀を、動き補償予測のために動き補償予測部１１０に参照画像Ｉ₁₁として提供する。
【０１４７】
以下、コンピュータを動画像復号装置１４０として動作させる動画像復号プログラム１５０について説明する。図２６は、動画像復号プログラム１５０の構成を示す図である。動画像復号プログラム１５０は、処理を統括するメインモジュール１５１と、復号モジュール１５２と、逆量子化変換モジュール１２４と、残差復号モジュール１５４と、加算モジュール１５６と、復号予測残差画像記憶モジュール１５８と、動き補償予測モジュール１３０と、復号画像生成モジュール１３２と、参照画像記憶モジュール１３４とを備える。残差復号モジュール１５４は、逆量子化サブモジュール１５４ａと、逆変換サブモジュール１５４ｂとを有している。
【０１４８】
復号モジュール１５２、逆量子化変換モジュール１２４、残差復号モジュール１５４、加算モジュール１５６、復号予測残差画像記憶モジュール１５８、動き補償予測モジュール１３０、復号画像生成モジュール１３２、参照画像記憶モジュール１３４、逆量子化サブモジュール１５４ａ、逆変換サブモジュール１５４ｂがコンピュータに実現させる機能は、復号部１４２、逆量子化変換部１０４、残差復号部１４４、加算部１４６、復号予測残差画像記憶部１４８、動き補償予測部１１０、復号画像生成部１１２、参照画像記憶部１１４、逆量子化部１４４ａ、逆変換部１４４ｂとそれぞれ同様である。
【０１４９】
以上のように、動画像復号装置１４０は、残差復号における逆変換で用いられる基底として復号予測残差画像記憶部１４８に記憶された参照復号予測残差画像Ｉ₁₇の部分画像を用いるよう構成されているので、動画像符号化装置６０が動画像から圧縮データを生成する処理の逆処理を忠実に行うことができる。したがって、動画像復号装置１４０は、動画像符号化装置６０によって生成された圧縮データから上記の動画像を復元することができる。
【０１５０】
本発明の第６実施形態にかかる動画像復号装置１７０について説明する。動画像復号装置１７０は、第３実施形態の動画像符号化装置８０によって生成される圧縮データＩ₁₆から動画像を復元する装置である。動画像復号装置１７０では、基底セットを記憶するための基底記憶部（基底記憶手段）１７４ｃと、参照復号予測残差画像Ｉ₁₇を記憶するための復号予測残差画像記憶部（復号予測残差画像記憶手段）１４８とを備え、残差補償における変換で用いられる基底として基底記憶部１７４ｃに記憶された基底セット、又は、復号予測残差画像記憶部１４８に記憶された参照復号予測残差画像Ｉ₁₇の部分画像を用いることを特徴とする。
【０１５１】
以下、動画像復号装置１７０の機能的な構成について説明する。図２７は、動画像復号装置１７０の機能的な構成を示す図である。動画像復号装置１７０は、機能的には、復号部（復号手段）１７２と、逆量子化変換部（逆量子化変換手段）１０４と、残差復号部（残差復号手段）１７４と、加算部（加算手段）１４６と、動き補償予測部（動き補償予測手段）１１０と、復号画像生成部（復号画像生成手段）１１２と、参照画像記憶部（参照画像記憶手段）１１４と、復号予測残差画像記憶部（復号予測残差画像記憶手段）１４８とを備える。以下、第４実施形態の動画像復号装置１００又は第５実施形態の動画像復号装置１４０と異なる要素について説明する。
【０１５２】
復号部１７２は、動画像符号化装置８０によって生成された圧縮データＩ₁₆を復号し、量子化変換係数Ｉ₇と、残差補償情報Ｉ₅と、動き情報Ｉ₂とを出力する。この残差補償情報Ｉ₅には、基底記憶部１７４ｃに記憶された基底セットを用いて残差補償画像Ｉ₄を復元するための基底情報Ｉ₁₂、又は、復号予測残差画像記憶部１４８に記憶される参照復号予測残差画像Ｉ₁₇の部分画像を用いて残差補償画像Ｉ₄を復元するための基底情報Ｉ₁₂と、量子化補償係数Ｉ₁₄が含まれている。
【０１５３】
残差復号部１７４は、残差補償情報Ｉ₅を用いて残差復号を行い残差補償画像Ｉ₄を生成する。以下、残差復号部１７４について詳細に説明する。図２８は、残差復号部１７４の構成を示す図である。図２８に示すように、残差復号部１７４は、逆量子化部（逆量子化手段）１７４ａと、逆変換部（逆変換手段）１７４ｂと、基底記憶部（基底記憶手段）１７４ｃとを有する。
【０１５４】
逆量子化部１７４ａは、残差補償情報Ｉ₅に含まれている量子化補償係数Ｉ₁₄に逆量子化操作を施すことによって、逆量子化補償係数Ｉ₁₅を生成する。
【０１５５】
逆変換部１７４ｂは、基底記憶部１７４ｃに記憶された基底セットのうち、残差補償情報Ｉ₅に含まれている基底情報Ｉ₁₂から基底を特定する。また、逆変換部１７４ｂは、復号予測残差画像記憶部１４８に記憶された参照復号予測残差画像Ｉ₁₇のうち、残差補償情報Ｉ₅に含まれている基底情報Ｉ₁₂から部分画像を基底として特定する。逆変換部１７４ｂは、特定した基底と、逆量子化部１４４ａによって生成された逆量子化補償係数Ｉ₁₅とを用いて変換部８２ａによる変換処理の逆操作である逆変換処理を行い、残差補償画像Ｉ₄を生成する。
【０１５６】
以下、動画像復号装置１７０の動作について説明し、併せて第６実施形態にかかる動画像復号方法について説明する。図２９は、第６実施形態の動画像復号方法のフローチャートである。また、図３０は、第６実施形態の動画像復号方法の残差復号に関する処理を示すフローチャートである。
【０１５７】
図２９に示すように、第６実施形態の動画像復号方法においては、まず、動画像符号化装置８０によって生成された圧縮データＩ₁₆から、復号部１７２によって量子化変換係数Ｉ₇と残差補償情報Ｉ₅と動き情報Ｉ₂が復号される（ステップＳ９１）。
【０１５８】
次に、逆量子化変換部１０４が、量子化変換係数Ｉ₇に逆量子化操作を施すことによって逆量子化変換係数を生成する。逆量子化変換部１０４は、生成した逆量子化変換係数に、所定の第２の逆変換規則による逆変換を施すことによって、復号補償残差画像Ｉ₈を復元する（ステップＳ９２）。
【０１５９】
次に、残差復号部１７４が、復号部１７２によって復号された残差補償情報Ｉ₅を用いて、残差復号を行い、残差補償画像Ｉ₄を生成する（ステップＳ７３）。図３０に示すように残差復号部１７４では、逆量子化部１７４ａが量子化補償係数Ｉ₁₄を逆量子化することによって逆量子化補償係数Ｉ₁₅を生成する（ステップＳ９４）。
【０１６０】
次に、逆変換部１７４ｂが、逆量子化補償係数Ｉ₁₅、残差補償情報Ｉ₅に含まれる基底情報Ｉ₁₂、基底記憶部１７４ｃに記憶された基底セット、及び復号予測残差画像記憶部１４８によって提供される参照復号予測残差画像Ｉ₁₇を用いて、変換部８２ａによる変換処理の逆操作である逆変換処理を行い、残差補償画像Ｉ₄を生成する（ステップＳ９５）。
【０１６１】
図２９を参照すると、次に、加算部１４６が復号補償残差画像Ｉ₈と残差補償画像Ｉ₄とを加算することによって復号予測残差画像Ｉ₉を生成する（ステップＳ９６）。
【０１６２】
次に、復号予測残差画像記憶部１４８が、加算部１４６によって生成された復号予測残差画像Ｉ₉を参照復号予測残差画像Ｉ₁₇として記憶する（ステップＳ９７）。復号予測残差画像記憶部１４８は、記憶した参照復号予測残差画像Ｉ₁₇を残差復号のために残差復号部１７４に提供する。
【０１６３】
次に、動き補償予測部１１０が、復号部１７２によって出力された動き情報Ｉ₂を用いて、参照画像記憶部１１４に記憶された参照画像Ｉ₁₁に対する動き補償予測を行うことによって復号画像Ｉ₁₀の予測画像Ｉ₁を生成する（ステップＳ９８）。
【０１６４】
次に、復号画像生成部１１２が、予測画像Ｉ₁と復号予測残差画像Ｉ₉とを加算することによって、復号画像Ｉ₁₀を生成する（ステップＳ９９）。復号画像生成部１１２は、復号画像Ｉ₁₀を参照画像記憶部１１４へ出力し、復号画像Ｉ₁₀を参照画像記憶部１１４に記憶させる（ステップＳ１００）。参照画像記憶部１１４は、記憶した復号画像Ｉ₁₀を、動き補償予測のために動き補償予測部１１０に参照画像Ｉ₁₁として提供する。
【０１６５】
以下、コンピュータを動画像復号装置１７０として動作させる動画像復号プログラム１８０について説明する。図３１は、動画像復号プログラム１８０の構成を示す図である。動画像復号プログラム１８０は、処理を統括するメインモジュール１８１と、復号モジュール１８２と、逆量子化変換モジュール１２４と、残差復号モジュール１８４と、加算モジュール１５６と、復号予測残差画像記憶モジュール１５８と、動き補償予測モジュール１３０と、復号画像生成モジュール１３２と、参照画像記憶モジュール１３４とを備える。残差復号モジュール１８４は、逆量子化サブモジュール１５４ａと、逆変換サブモジュール１５４ｂ、基底記憶サブモジュール１８４ｃとを有している。
【０１６６】
復号モジュール１８２、逆量子化変換モジュール１２４、残差復号モジュール１８４、加算モジュール１５６、復号予測残差画像記憶モジュール１５８、動き補償予測モジュール１３０、復号画像生成モジュール１３２、参照画像記憶モジュール１３４、逆量子化サブモジュール１８４ａ、逆変換サブモジュール１８４ｂ、基底記憶サブモジュール１８４ｃがコンピュータに実現させる機能は、復号部１７２、逆量子化変換部１０４、残差復号部１７４、加算部１４６、復号予測残差画像記憶部１４８、動き補償予測部１１０、復号画像生成部１１２、参照画像記憶部１１４、逆量子化部１７４ａ、逆変換部１７４ｂ、基底記憶部１７４ｃとそれぞれ同様である。
【０１６７】
以上のように、動画像復号装置１７０は、残差復号における逆変換で用いられる基底として、基底記憶部１７４ｃに記憶された基底セット、及び、復号予測残差画像記憶部１４８に記憶された参照復号予測残差画像Ｉ₁₇の部分画像を用いるよう構成されているので、動画像符号化装置８０が動画像から圧縮データを生成する処理の逆処理を忠実に行うことができる。したがって、動画像復号装置１７０は、動画像符号化装置８０によって生成された圧縮データから上記の動画像を復元することができる。
【０１６８】
【発明の効果】
以上説明したように本発明によれば、予測残差画像と補償残差画像とをそれぞれ効率よく分解することができるので、効率の良い符号化が実現可能な動画像符号化装置、動画像符号化方法、及び動画像符号化プログラムが提供される。また、かかる動画像符号化装置によって生成された圧縮データから動画像を忠実に復元可能な動画像復号装置、動画像復号方法、及び動画像復号プログラムが提供される。
【図面の簡単な説明】
【図１】図１は、本発明の第１実施形態にかかる動画像符号化装置の機能的な構成を示す図である。
【図２】図２は、第１実施形態の残差補償部の構成を示す図である。
【図３】図３は、本発明の第１実施形態にかかる動画像符号化方法のフローチャートである。
【図４】図４は、第１実施形態の動画像符号化方法における残差補償に関する処理のフローチャートである。
【図５】図５は、本発明の第１実施形態にかかる動画像符号化プログラムの構成を示す図である。
【図６】図６は、本発明の第２実施形態にかかる動画像符号化装置の機能的な構成を示す図である。
【図７】図７は、第２実施形態の残差補償部の構成を示す図である。
【図８】図８は、第２実施形態の変換部ａによる変換処理を説明するための図である。
【図９】図９は、本発明の第２実施形態にかかる動画像符号化方法のフローチャートである。
【図１０】図１０は、第２実施形態の動画像符号化方法における残差補償に関する処理のフローチャートである。
【図１１】図１１は、本発明の第２実施形態にかかる動画像符号化プログラムの構成を示す図である。
【図１２】図１２は、本発明の第３実施形態にかかる動画像符号化装置の機能的な構成を示す図である。
【図１３】図１３は、第３実施形態の残差補償部の構成を示す図である。
【図１４】図１４は、本発明の第３実施形態にかかる動画像符号化方法のフローチャートである。
【図１５】図１５は、第３実施形態の動画像符号化方法における残差補償に関する処理のフローチャートである。
【図１６】図１６は、本発明の第３実施形態にかかる動画像符号化プログラムの構成を示す図である。
【図１７】図１７は、本発明の第４実施形態にかかる動画像復号装置の機能的な構成を示す図である。
【図１８】図１８は、第４実施形態にかかる残差復号部の構成を示す図である。
【図１９】図１９は、本発明の第４実施形態の動画像復号方法のフローチャートである。
【図２０】図２０は、第４実施形態の動画像復号方法の残差復号に関する処理を示すフローチャートである。
【図２１】図２１は、本発明の第４実施形態にかかる動画像復号プログラムの構成を示す図である。
【図２２】図２２は、本発明の第５実施形態にかかる動画像復号装置の機能的な構成を示す図である。
【図２３】図２３は、第５実施形態にかかる残差復号部の構成を示す図である。
【図２４】図２４は、本発明の第５実施形態の動画像復号方法のフローチャートである。
【図２５】図２５は、第５実施形態の動画像復号方法の残差復号に関する処理を示すフローチャートである。
【図２６】図２６は、本発明の第５実施形態にかかる動画像復号プログラムの構成を示す図である。
【図２７】図２７は、本発明の第６実施形態にかかる動画像復号装置の機能的な構成を示す図である。
【図２８】図２８は、第６実施形態にかかる残差復号部の構成を示す図である。
【図２９】図２９は、本発明の第６実施形態の動画像復号方法のフローチャートである。
【図３０】図３０は、第６実施形態の動画像復号方法の残差復号に関する処理を示すフローチャートである。
【図３１】図３１は、本発明の第６実施形態にかかる動画像復号プログラムの構成を示す図である。
【符号の説明】
１，６０，８０…動画像符号化装置、２…動き補償予測部、４…動き残差生成部、６，６２，８２…残差補償部、６ａ，６２ａ，８２ａ…変換部、６ｂ，８２ｂ…基底記憶部、６ｃ，６２ｂ，８２ｃ…量子化部、６ｄ，６２ｃ，８２ｄ…逆量子化部、６ｅ，６２ｄ，８２ｅ…逆変換部、８…減算部、１０…量子化変換部、１２，６４，８４…符号化部、１４…逆量子化変換部、１６，６６…加算部、１８…復号画像生成部、２０…参照画像記憶部、６８…復号予測残差画像記憶部、１００，１４０，１７０…動画像復号装置、１０２，１４２，１７２…復号部、１０４…逆量子化変換部、１０６，１４４，１７４…残差復号部、１０６ａ，１４４ａ，１７４ａ…逆量子化部、１０６ｂ，１４４ｂ，１７４ｂ…逆変換部、１０６ｃ，１７４ｃ…基底記憶部、１０８，１４６…加算部、１１０…動き補償予測部、１１２…復号画像生成部、１１４…参照画像記憶部、１４８…復号予測残差画像記憶部。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a video encoding device, a video encoding method, a video encoding program, a video decoding device, a video decoding method, and a video decoding program.
[0002]
[Prior art]
In a moving image encoding apparatus, generally, an encoding target image is divided into a plurality of blocks of a predetermined size, and a motion vector is detected by performing motion compensation prediction between each block and a reference image in a predetermined region of a reference frame. Thus, a predicted image of the encoding target image is generated. In such a moving image encoding apparatus, the redundancy existing in the time direction is reduced by expressing the encoding target image by the motion vector from the reference image. In addition, a prediction residual image that is a difference between an encoding target image and a prediction image is subjected to DCT (Discrete Cosine Transform) conversion and expressed as a DCT coefficient, thereby reducing the redundancy existing in the spatial direction.
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 7-264592
[0004]
[Problems to be solved by the invention]
However, since the prediction residual image takes a difference after performing motion compensation prediction between a reference image that is an encoded image and an encoding target image that is an image before encoding, a biased distribution is generated. It is known that this is an image signal. The above-described conventional moving image encoding apparatus performs efficient encoding using DCT for an image signal having such a feature without using the feature. There was a problem that I could not.
[0005]
The present invention was made to solve the above problems, and provides a moving image encoding apparatus, a moving image encoding method, and a moving image encoding program capable of realizing efficient encoding of moving images. Another object of the present invention is to provide a moving picture decoding apparatus, a moving picture decoding method, and a moving picture decoding program for decoding a moving picture encoded by the moving picture encoding apparatus.
[0006]
[Means for Solving the Problems]
  In order to solve the above problems, a moving image encoding apparatus according to the present invention includes a motion compensation prediction unit that generates a prediction image and motion information by performing motion compensation prediction of an encoding target image based on a reference image, and the code A residual image generating means for generating a prediction residual image based on a difference calculation between the target image and the prediction image, and generating residual compensation information from the prediction residual image based on a predetermined first conversion rule And a residual compensation means for generating a residual compensation image based on the residual compensation information, and a subtraction means for generating a compensation residual image based on a difference calculation between the residual compensation image and the predicted residual image. Generating a transform coefficient obtained by transforming the compensation residual image based on a predetermined second transform rule, and quantizing the transform coefficient to generate a quantized transform coefficient; and Quantize transform coefficient inverse quantum By generating an inverse quantization transform coefficient and performing an inverse transform using the inverse quantization transform coefficient based on a predetermined second inverse transform rule that is an inverse transform of the predetermined second transform rule. An inverse quantization transform unit that generates a decoded compensation residual image; an adding unit that generates a decoded prediction residual image by adding the decoded compensation residual image and the residual compensation image; and the decoded prediction residual A decoded image generating means for generating a decoded image by adding an image and the predicted image; and the reference for providing the decoded image generated by the decoded image generating means for motion compensated prediction by the motion compensated predictor Reference image storage means for storing as an image, and an encoding method for generating compressed data including encoded data obtained by encoding the motion information, the residual compensation information, and the quantized transform coefficient The residual compensation means is based on the predetermined first conversion rule and uses a plurality of two-dimensional bases to specify the base information for specifying the two-dimensional base and the base information. The prediction residual image is converted into a compensation coefficient attached to a two-dimensional basis, a conversion means for including the base information in the residual compensation information, and a quantization compensation coefficient is generated by quantizing the compensation coefficient A quantization means for including the quantization compensation coefficient in the residual compensation information; an inverse quantization means for generating an inverse quantization compensation coefficient by inverse quantization of the quantization compensation coefficient; Inverse transform means for generating the residual compensation image by performing inverse transform using the inverse quantization compensation coefficient based on a predetermined first inverse transform rule that is an inverse transform of one transform rule. It is characterized by.Furthermore, the moving picture encoding apparatus of the present invention further includes a decoded prediction residual image storage unit that stores the decoded prediction residual image as a reference decoded prediction residual image, and the conversion unit includes the decoded prediction residual image. The prediction residual image is transformed using the partial image of the reference decoded prediction residual image stored in the storage means as the two-dimensional basis, and the inverse transformation means is the two-dimensional basis used by the transformation means. It is characterized by performing inverse transformation using.
[0007]
  In addition, in order to solve the above-described problem, the moving image encoding method of the present invention includes a motion compensation prediction unit that generates a predicted image and motion information by performing motion compensated prediction of an encoding target image based on a reference image. A compensation prediction step, a residual image generation means for generating a prediction residual image based on a difference calculation between the encoding target image and the prediction image, and a residual compensation means A residual compensation step for generating residual compensation information from the prediction residual image based on the conversion rule of 1 and generating a residual compensation image based on the residual compensation information; A subtraction step for generating a compensation residual image based on a difference calculation between the compensation image and the prediction residual image, and a quantization transform unit transforms the compensation residual image based on a predetermined second transformation rule. Generate a conversion factor A quantizing transform step for generating a quantized transform coefficient by quantizing the transform coefficient; and an inverse quantization transform means for generating an inverse quantized transform coefficient by dequantizing the quantized transform coefficient. Inverse quantization for generating a decoded compensated residual image by performing inverse transform using the inverse quantization transform coefficient based on a predetermined second inverse transform rule that is an inverse transform of the predetermined second transform rule A conversion step; an addition step in which the addition means generates a decoded prediction residual image by adding the decoded compensation residual image and the residual compensation image; and a decoded image generation means includes the decoded prediction residual image. And a decoded image generation step for generating a decoded image by adding the predicted image and the reference image storage means, and the reference image storage means converts the decoded image generated by the decoded image generation means into the motion compensated prediction. A reference image storage step for storing the reference image to be provided for motion compensation prediction by the means; and encoded data obtained by encoding the motion information, the residual compensation information, and the quantized transform coefficient. An encoding step for generating compressed data including the step, wherein the residual compensation step specifies the two-dimensional basis using a plurality of two-dimensional bases based on the predetermined first transformation rule. A transform step of transforming the prediction residual image into a base information for performing the processing and a compensation coefficient attached to a two-dimensional base specified by the base information, and including the base information in the residual compensation information; Means for quantizing the compensation coefficient to generate a quantization compensation coefficient, and including the quantization compensation coefficient in the residual compensation information; and an inverse quantization means comprising the quantization compensation coefficient An inverse quantization step for generating an inverse quantization compensation coefficient by inverse quantization, and an inverse transform means based on a predetermined first inverse transform rule that is an inverse transform of the predetermined first transform rule, And an inverse transform step of generating the residual compensation image by performing inverse transform using the inverse quantization compensation coefficient.Furthermore, in the moving picture encoding method of the present invention, the decoded prediction residual image storage unit further includes a step of storing the decoded prediction residual image as a reference decoded prediction residual image, and in the conversion step, the conversion unit Transforms the prediction residual image using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage means as the two-dimensional basis, and the inverse transformation step includes the inverse transformation. The means is characterized by performing inverse transformation using the two-dimensional basis used by the transformation means.
[0008]
  In order to solve the above-described problem, the moving image encoding program of the present invention is a motion compensated prediction that generates a predicted image and motion information by causing a computer to perform motion compensated prediction of an encoding target image based on a reference image. Means, a residual image generating means for generating a prediction residual image based on a difference calculation between the encoding target image and the prediction image, and a residual from the prediction residual image based on a predetermined first conversion rule. Generating compensation information and generating a residual compensation image based on the residual compensation information; and calculating a compensation residual image based on a difference calculation between the residual compensation image and the prediction residual image. Subtracting means for generating, and a quantization coefficient for generating a quantized transform coefficient by generating a transform coefficient obtained by transforming the compensation residual image based on a predetermined second transform rule and quantizing the transform coefficient Conversion means and An inverse quantized transform coefficient is generated by dequantizing the quantized transform coefficient, and the inverse quantized transform coefficient is based on a predetermined second inverse transform rule that is an inverse transform of the predetermined second transform rule. An inverse quantization transforming means for generating a decoding compensated residual image by performing inverse transform using, and a decoded prediction residual image is generated by adding the decoded compensation residual image and the residual compensated image Addition means, decoded image generation means for generating a decoded image by adding the decoded prediction residual image and the prediction image, and the decoded image generated by the decoded image generation means as the motion compensation prediction means Reference image storage means for storing as the reference image to be provided for motion compensated prediction, and encoded data obtained by encoding the motion information, the residual compensation information, and the quantized transform coefficient The residual compensation means for specifying the two-dimensional basis using a plurality of two-dimensional bases based on the predetermined first conversion rule. Converting the prediction residual image into base information and a compensation coefficient attached to a two-dimensional base specified by the base information, converting the prediction information into the residual compensation information, and converting the compensation coefficient into a quantum A quantization compensation coefficient by generating the quantization compensation coefficient, the quantization means including the quantization compensation coefficient in the residual compensation information, and an inverse quantization compensation coefficient by dequantizing the quantization compensation coefficient The residual compensation image is obtained by performing inverse transformation using the inverse quantization compensation coefficient based on inverse quantization means and a predetermined first inverse transformation rule that is an inverse transformation of the predetermined first transformation rule. Inverse conversion means to generate It is characterized by doing.Furthermore, the moving image encoding program of the present invention causes the computer to further function as a decoded prediction residual image storage unit that stores the decoded prediction residual image as a reference decoded prediction residual image. The prediction residual image is converted using the partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage unit as the two-dimensional basis, and the inverse conversion unit is used by the conversion unit. Further, it is characterized by performing inverse transformation using the above two-dimensional basis.
[0009]
  According to these inventions, the prediction residual image is decomposed into residual compensation information including a compensation coefficient and base information for specifying the two-dimensional base by a predetermined first transformation rule using a two-dimensional base. Therefore, it is possible to represent a prediction residual image by a smaller number of information compared to orthogonal transformation such as DCT. Further, the compensation residual image based on the difference between the prediction residual image and the residual compensation image restored from the residual compensation information is an image from which the above-described biased distribution in the image signal is removed. Therefore, it is efficiently decomposed by a predetermined second conversion rule such as DCT.In addition, since the prediction residual image is decomposed using the partial image of the decoded prediction residual image as the two-dimensional basis, more effective residual compensation using the similarity of the prediction residual image is realized.
[0010]
In the moving picture coding apparatus of the present invention, the residual compensation means further includes a base storage means for storing a two-dimensional base provided to the conversion means, and the conversion means is stored in the base storage means. The prediction residual image may be transformed using a stored two-dimensional basis, and the inverse transformation unit may perform the inverse transformation using a two-dimensional basis stored in the basis storage unit.
[0011]
According to this invention, since the partial image of the decoded prediction residual image and the two-dimensional basis stored in the basis storage means can be used as the two-dimensional basis, for example, a sudden change occurs between frames. Even in this case, the prediction residual image is efficiently decomposed.
[0013]
  In order to solve the above problem, the video decoding device of the present invention includes motion information for generating a predicted image for a decoded image, and a quantization compensation coefficient for restoring a residual compensation image for the predicted image. Including encoded data obtained by encoding residual compensation information including base information and quantized transform coefficients for restoring a compensated residual image based on a difference between the predicted image and the residual compensated image Decoding means for generating the motion information, the residual compensation information, and the quantized transform coefficient by decoding compressed data, and an inverse based on a predetermined first inverse transform rule using the residual compensation information Residual decoding means for generating the residual compensated image by performing transformation, and generating an inverse quantized transform coefficient by dequantizing the quantized transform coefficient, and using the inverse quantized transform coefficient Second reverse of Inverse quantization conversion means for generating a decoded compensation residual image by performing an inverse transformation based on a conversion rule, and generating a decoded prediction residual image based on the addition of the residual compensation image and the decoded compensation residual image An addition unit, a motion compensation prediction unit that generates the prediction image by performing motion compensation prediction using the reference image and the motion information, and a decoded image based on the addition of the decoded prediction residual image and the prediction image A decoded image generating means for generating, and a reference image storage means for storing the decoded image generated by the decoded image generating means as the reference image provided for the motion compensation prediction by the motion compensated prediction means, Residual decoding means includes inverse quantization means for generating an inverse quantization compensation coefficient by inversely quantizing the quantization compensation coefficient included in the residual compensation information; Using the two-dimensional basis specified by the basis information included in the residual compensation information and the inverse quantization compensation coefficient inverse transform generated by the inverse quantization means. Inverse conversion means for generating the residual compensation image by performing inverse conversion based on one inverse conversion rule.Furthermore, the moving picture decoding apparatus of the present invention further includes a decoded prediction residual image storage unit that stores the decoded prediction residual image as a reference decoded prediction residual image, and the inverse transform unit includes the decoded prediction residual image. The inverse transform is performed using a partial image of the reference decoded prediction residual image stored in the storage means as the two-dimensional basis.

[0014]
  In order to solve the above-described problem, the moving picture decoding method according to the present invention includes a decoding unit that includes motion information for generating a predicted image for a decoded image and a quantum for restoring a residual compensation image for the predicted image. A code obtained by encoding residual compensation information including a quantization compensation coefficient and base information, and a quantized transform coefficient for restoring a compensation residual image based on a difference between the predicted image and the residual compensation image A decoding step for generating the motion information, the residual compensation information, and the quantized transform coefficient by decoding the compressed data including the quantized data, and a residual decoding means using the residual compensation information A residual decoding step for generating the residual compensated image by performing an inverse transformation based on the first inverse transformation rule, and an inverse quantization transforming unit inversely quantizes the quantized transform coefficient to perform inverse quantization. Conversion An inverse quantization transform step of generating a decoded compensation residual image by generating a number and performing an inverse transform based on a predetermined second inverse transform rule using the inverse quantized transform coefficient; An addition step of generating a decoded prediction residual image based on the addition of the residual compensation image and the decoded compensation residual image, and motion compensation prediction means performing motion compensation prediction using the reference image and the motion information A motion compensation prediction step for generating the predicted image, a decoded image generating unit for generating a decoded image based on the addition of the decoded prediction residual image and the predicted image, and a reference image storage unit, A reference image storage step of storing the decoded image generated by the decoded image generation means as the reference image to be provided to the motion compensation prediction by the motion compensation prediction means; The residual decoding step includes: an inverse quantization step in which an inverse quantization unit generates an inverse quantization compensation coefficient by inversely quantizing the quantization compensation coefficient included in the residual compensation information; The inverse transform means includes a two-dimensional basis specified by the basis information included in the residual compensation information among a plurality of two-dimensional bases, and the inverse quantization compensation coefficient inverse transform generated by the inverse quantization means; And an inverse transformation step of generating the residual compensation image by performing an inverse transformation based on the predetermined first inverse transformation rule.Furthermore, the moving picture decoding method of the present invention further comprises a step in which the decoded prediction residual image storage means stores the decoded prediction residual image as a reference decoded prediction residual image, and in the reverse conversion step, the reverse conversion is performed. The means performs the inverse transform using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage means as the two-dimensional basis.
[0015]
  In order to solve the above problem, the moving picture decoding program according to the present invention is a computer that performs motion information for generating a predicted image for a decoded image and quantization for restoring a residual compensation image for the predicted image. Coding in which residual compensation information including a compensation coefficient and base information and a quantized transform coefficient for restoring a compensation residual image based on a difference between the predicted image and the residual compensation image are encoded Decoding means for generating the motion information, the residual compensation information and the quantized transform coefficient by decoding compressed data including data; and a predetermined first inverse transform rule using the residual compensation information A residual decoding means for generating the residual compensated image by performing an inverse transform based on the method, and generating an inverse quantized transform coefficient by dequantizing the quantized transform coefficient, wherein the inverse quantized transform coefficient An inverse quantization transforming means for generating a decoded compensation residual image by performing an inverse transformation based on a predetermined second inverse transformation rule, and based on the addition of the residual compensation image and the decoded compensation residual image Addition means for generating a decoded prediction residual image, motion compensated prediction means for generating the prediction image by performing motion compensation prediction using a reference image and the motion information, the decoded prediction residual image, and the prediction image A decoded image generating means for generating a decoded image based on the addition of the above and a reference for storing the decoded image generated by the decoded image generating means as the reference image provided to the motion compensated prediction by the motion compensated predictor And the residual decoding means performs inverse quantization compensation by inversely quantizing the quantization compensation coefficient included in the residual compensation information. An inverse quantization means for generating a number, a two-dimensional basis specified by basis information included in the residual compensation information among a plurality of two-dimensional bases, and the inverse quantization compensation generated by the inverse quantization means Inverse transform means for generating the residual compensation image by performing inverse transform based on the predetermined first inverse transform rule using coefficient inverse transform.Furthermore, the moving picture decoding program of the present invention causes the computer to further function as a decoded prediction residual image storage unit that stores the decoded prediction residual image as a reference decoded prediction residual image, The inverse transformation is performed using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage means as the two-dimensional basis.
[0016]
  According to these inventions, the residual compensation image is restored based on the residual compensation information decoded from the compressed data, the decoded compensation residual image is restored based on the quantized transform coefficient decoded from the compressed data, A decoded prediction residual image is generated by adding the residual compensation image and the decoded compensation residual image. Also, a predicted image is generated based on motion information decoded from the compressed data, and a decoded image is generated by adding the predicted image and the decoded prediction residual image. Therefore, the moving image is faithfully restored from the compressed data generated by the above-described moving image encoding device.In addition, since the partial image of the decoded prediction residual image is used as a two-dimensional basis, the moving image is faithfully restored from the compressed data generated by the above-described moving image encoding device.
[0017]
In the video decoding device of the present invention, the residual decoding unit further includes a base storage unit that stores the plurality of two-dimensional bases, and the inverse transform unit is stored in the base storage unit. The inverse transformation may be performed using the two-dimensional basis.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.
[0021]
First, the moving image encoding device 1 according to the first embodiment of the present invention will be described. The moving image encoding device 1 is physically a computer including a CPU (Central Processing Unit), a storage device such as a memory, a storage device such as a hard disk, and the like. Here, the “computer” includes an information portable terminal such as a mobile communication terminal in addition to a normal computer such as a personal computer, and the idea of the present invention is widely applied to devices capable of information processing.
[0022]
FIG. 1 is a diagram illustrating a functional configuration of the moving image encoding apparatus 1. Functionally, the moving image encoding device 1 functionally includes a motion compensation prediction unit (motion compensation prediction unit) 2, a motion residual generation unit (motion compensation residual generation unit) 4, and a residual compensation unit (residual compensation). Means) 6, a subtracting section (subtracting means) 8, a quantizing conversion section (quantizing converting means) 10, an encoding section (encoding means) 12, and an inverse quantization converting section (inverse quantization converting means). 14, an adding unit (adding unit) 16, a decoded image generating unit (decoded image generating unit) 18, and a reference image storage unit (reference image storage unit) 20.
[0023]
The motion compensation prediction unit 2 performs motion compensation prediction using the reference image stored in the reference image storage unit 20 and performs encoding target image I.₀Predicted image I₁And predictive image I₁Are output to the motion residual generation unit 4 and the decoded image generation unit 18. The motion compensation prediction unit 2 also provides motion information I necessary for reproducing the motion compensation prediction.₂Is output to the encoding unit 12.
[0024]
More specifically, the motion compensated prediction unit 2 performs the encoding target image I.₀Is divided into blocks of a predetermined size. The predetermined size is, for example, a plurality of types of blocks having different sizes and shapes such as 16 pixels × 16 lines, 16 pixels × 8 lines, 8 pixels × 16 lines, 8 pixels × 8 lines, and the like.₀Can be divided.
[0025]
The motion compensation prediction unit 2 performs block matching on a prediction reference region of a predetermined region among the reference images stored in the reference image storage unit 20 for each of a plurality of blocks obtained by dividing the encoding target image, and A motion vector for the reference image of each block is detected.
[0026]
The motion compensation prediction unit 2 uses the motion vector described above to generate an encoding target image I from a reference image.₀Predicted image I₁Is generated. The motion compensated prediction unit 2 generates the generated predicted image I₁Are output to the motion residual generation unit 4 and the decoded image generation unit 18. Also, the motion compensation prediction unit 2 uses the motion information I and information for specifying the detected motion vector and block shape as motion information I.₂To the encoding unit 12.
[0027]
Note that the image of the predicted reference area generated from the reference image is not limited to an image composed of only integer pixels, and is an image provided with interpolation pixels that interpolate ½ and ¼ positions between integer pixels. May be. By providing the interpolation pixels in this way, it is possible to detect the motion vector with high resolution. The motion compensation prediction unit 2 applies overlap motion compensation that obtains a pixel value by weighting a pixel value obtained from a motion vector of a block of interest and a pixel value obtained using a motion vector of an adjacent block. Prediction image I₁You may get.
[0028]
The motion residual generation unit 4 is a prediction image I generated by the motion compensation prediction unit 2.₁And the encoding target image I₀The difference between the predicted image I₁And the encoding target image I₀Prediction residual image I consisting of residuals with_ThreeIs generated. The motion residual generation unit 4 generates a prediction residual image I._ThreeIs output to the residual compensation unit 6 and the subtraction unit 8.
[0029]
The residual compensation unit 6 is a prediction residual image I output by the motion residual generation unit 4._ThreeResidual compensation image I_FourIs generated. The residual compensation unit 6 generates the generated residual compensation image I_FourIs output to the subtraction unit 8 and the addition unit 16. Further, the residual compensation unit 6 obtains information necessary for reproducing the residual compensation as residual compensation information I._FiveTo the encoding unit 12. Details of the residual compensation unit 6 will be described later.
[0030]
The subtracting unit 8 outputs a residual compensation image I output from the residual compensating unit 6._FourAnd the prediction residual image I output by the motion residual generation unit 4_ThreeAnd the residual compensation image I_FourAnd prediction residual image I_ThreeCompensation residual image I consisting of residuals with₆Is generated. The subtraction unit 8 generates the generated compensation residual image I₆Is output to the quantization conversion unit 10.
[0031]
The quantization transformation unit 10 outputs the compensation residual image I output from the subtraction unit 8.₆Is converted into a set of coefficients using a predetermined second conversion rule. As this predetermined conversion rule, for example, DCT (Discrete Cosine Transform) can be used. When DCT is used, the compensation residual image I₆Is converted to a set of DCT coefficients. The quantization transform unit 10 performs a quantization operation on the above-described coefficients to quantize the transform coefficient I.₇Is generated. The quantization conversion unit 10 generates the generated quantization conversion coefficient I₇Is output to the encoding unit 12 and the inverse quantization transform unit 14.
[0032]
The encoding unit 12 outputs the motion information I output by the motion compensation prediction unit 2.₂Coded data obtained by encoding is generated. The encoding unit 12 also outputs residual compensation information I output from the residual compensation unit 6._FiveCoded data obtained by encoding is generated. Further, the encoding unit 12 outputs the quantized transform coefficient I output by the quantizing transform unit 10.₇Coded data obtained by encoding is generated. The encoding unit 12 includes compressed data I including these encoded data.₁₆Is generated. For this encoding process, for example, an entropy encoding process such as Huffman encoding or arithmetic encoding can be used.
[0033]
The inverse quantization transform unit 14 is a quantized transform coefficient I generated by the quantization transform unit 10.₇Is inversely quantized to generate inverse quantization transform coefficients. The inverse quantization transform unit 14 performs the inverse transform process of the transform process used by the quantization transform unit 10 on the generated inverse quantization transform coefficient, thereby decoding the decoded compensation residual image I.₈Is generated. The inverse quantization transform unit 14 generates the generated decoded compensation residual image I₈Is output to the adder 16.
[0034]
The adding unit 16 outputs a residual compensated image I output from the residual compensating unit 6._FourAnd the decoded compensation residual image I output by the adder 16₈Is added to the decoded prediction residual image I.₉Is generated. The adding unit 16 generates the generated decoded prediction residual image I.₉Is output to the decoded image generation unit 18.
[0035]
The decoded image generation unit 18 outputs the predicted image I output by the motion compensation prediction unit 2.₁And the decoded prediction residual image I output by the adder 16₉Is added to the decoded image I_TenIs generated. The decoded image generation unit 18 generates a decoded image I_TenIs output to the reference image storage unit 20. The reference image storage unit 20 includes a decoded image I output by the decoded image generation unit 18._TenReference image I₁₁And the reference image I for processing by the motion compensation prediction unit 2 for the next frame.₁₁I will provide a. Note that the decoded image generation unit 18 generates a decoded image I._TenAlternatively, an image to which the deblocking filter is applied may be output to the reference image storage unit 20.
[0036]
Hereinafter, the residual compensation unit 6 will be described in detail. FIG. 2 is a diagram illustrating a configuration of the residual compensation unit 6. The residual compensation unit 6 includes a conversion unit (conversion unit) 6a, a base storage unit (base storage unit) 6b, a quantization unit (quantization unit) 6c, and an inverse quantization unit (inverse quantization unit) 6d. And an inverse conversion unit (inverse conversion means) 6e.
[0037]
The conversion unit 6a performs prediction residual image I_ThreeIs subjected to a conversion process using a predetermined first conversion rule. As the predetermined first conversion rule, a Matching Pursuit method (hereinafter referred to as “MP method”) can be used. The MP method uses the prediction residual image I_ThreeIs an initial residual component, and the process of decomposing the residual component using the basis set is repeated according to the following equation (1).
[Expression 1]

Here, in Expression (1), f is a prediction residual image, R_nf is a residual component after the nth iteration, and g_knIs R_na base that maximizes the inner product value with f, and R_mf is a residual component after the m-th iteration.
[0038]
According to the MP method, a base that maximizes an inner product value with a residual component is selected from the base set, and a residual component is selected between the selected base and the maximum inner product value that is a coefficient for multiplying the base. Is disassembled.
[0039]
Here, it is known that a prediction residual image that is a prediction residual of motion compensation prediction is an image signal having a biased distribution. In the MP method, a pattern that tends to appear in a prediction residual image as a base in advance is stored as a two-dimensional base, and a base that has a smaller residual component from the stored two-dimensional base is used. Images can be decomposed sequentially. Therefore, a prediction residual image can be effectively expressed using a small number of bases compared to orthogonal transformation such as DCT.
[0040]
In the present embodiment, the base storage unit 6b stores a base set. The conversion unit 6a uses the basis set provided by the basis storage unit 6b and uses the basis information I for specifying the basis that maximizes the inner product value with the residual component.₁₂For the basis information I₁₂Is output to the inverse transform unit 6e. In addition, the basis information I₁₂Is the residual compensation information I_FiveIs output to the encoding unit 12 as a part of. The conversion unit 6a converts the above inner product value into the compensation coefficient I.₁₃As compensation coefficient I₁₃Is output to the quantizing unit 6c.
[0041]
The quantization unit 6c outputs the compensation coefficient I output from the conversion unit 6a.₁₃By applying a quantization operation to the quantization compensation coefficient I₁₄Is generated. The quantization unit 6c is configured to output a quantization compensation coefficient I₁₄Is output to the inverse quantization unit 6d and the quantization compensation coefficient I₁₄Residual compensation information I_FiveIs output to the encoding unit 12 as a part of.
[0042]
The inverse quantization unit 6d performs quantization compensation coefficient I₁₄Is inversely quantized, and the inverse quantization compensation coefficient I₁₅Is generated. The inverse quantization unit 6d includes an inverse quantization compensation coefficient I₁₅Is output to the inverse transform unit 6e. The inverse transform unit 6e is configured to output an inverse quantization compensation coefficient I₁₅, Basis information I₁₂, And the basis stored in the basis storage unit 6b, and performing the inverse transformation process of the above-described transformation process performed by the transformation unit 6a, the residual compensation image I_FourIs generated. The inverse transform unit 6e performs the residual compensation image I_FourIs output to the subtraction unit 8 and the addition unit 16.
[0043]
As described above, the encoding unit 12 outputs the motion information I output by the motion compensation prediction unit 2.₂Encoded data obtained by encoding the quantization transform coefficient I output by the quantization transform unit 10₇And the residual compensation information I output by the residual compensation unit 6._FiveCoded data obtained by encoding is generated. The encoding unit 12 uses the compressed data I including these encoded data.₁₆Is generated. For this encoding process, for example, an entropy encoding process such as Huffman encoding or arithmetic encoding can be used. The encoding unit 12 performs motion information I₂And quantized transform coefficient I₇For example, encoding can be performed by a method similar to the entropy encoding employed in the conventional moving image encoding scheme such as the MPEG-4 encoding scheme.
[0044]
Residual compensation information I_FiveThe entropy coding will be described. As described above, residual compensation information I_FiveIs the basis information I₁₂And quantization compensation coefficient I₁₄Is included. Basic information I₁₂Is information for specifying one base from the base set, for example, 2^cWhen the number of bases is stored in the base storage unit 6b, the base information I is represented by a c-bit fixed length code.₁₂Can be encoded. Also, the quantization compensation coefficient I₁₄Is a quantized scalar value multiplied by the base, for example, the range is 1 to 2^dQuantization compensation coefficient I by a d-bit fixed length code₁₄Can be encoded.
[0045]
Next, the operation of the video encoding device 1 will be described. In addition, the moving picture coding method according to the first embodiment of the present invention will be described. FIG. 3 is a flowchart of the moving picture coding method according to the first embodiment. FIG. 4 is a flowchart of processing relating to residual compensation in the moving picture coding method according to the first embodiment.
[0046]
As shown in FIG. 3, in the moving picture coding method according to the first embodiment, first, motion compensation prediction is performed by the motion compensation prediction unit 2 (step S01). In motion compensated prediction, the encoding target image I₀Are divided into blocks of a predetermined size. Then, the motion compensation prediction unit 2 obtains a motion vector for the reference image for each block, and the encoding target image I₀Predicted image I₁Is generated. As the reference image, an image stored in the reference image storage unit 20 is used. The motion compensation prediction unit 2 also includes the motion information described above including information such as motion vectors._I2Is output to the encoding unit 12.
[0047]
Next, the predicted image I output by the motion compensation prediction unit 2₁And the encoding target image I₀Is calculated by the motion residual generation unit 4 so that the prediction residual image I_ThreeIs generated (step S02).
[0048]
Next, residual compensation is performed by the residual compensation unit 6 (step S03). In the residual compensation, as shown in FIG. 4, the prediction residual image I is used by using the base stored in the base storage unit 6b by the conversion unit 6a._ThreeIs disassembled. Then, the prediction residual image I is converted by the conversion unit 6a._ThreeInformation I for identifying the basis used for decomposition of₁₂And compensation coefficient I multiplied by the base₁₃Is output (step S04).
[0049]
Next, the compensation coefficient I is obtained by the quantizing unit 6c.₁₃Is quantized and the quantization compensation coefficient I₁₄(Step S05). Basic information I₁₂And quantization compensation coefficient I₁₄Is residual compensation information I_FiveIs output to the encoding unit 12.
[0050]
Next, a quantization compensation coefficient I is obtained by the inverse quantization unit 6d.₁₄Are inversely quantized and the inverse quantization compensation coefficient I₁₅(Step S06). Then, by the inverse transform unit 6e, the inverse quantization compensation coefficient I₁₅, Basis information I₁₂, And the base stored in the base storage unit 6b is used, and an inverse transform process, which is an inverse operation of the transform process by the transform unit 6a, is performed, whereby the residual compensation image I_FourIs generated. (Step S07).
[0051]
Referring to FIG. 3, next, the residual compensation image I output by the residual compensation unit 6._FourAnd the prediction residual image I output by the motion residual generation unit 4_ThreeIs calculated by the subtracting unit 8 so that the compensation residual image I₆Is generated (step S08).
[0052]
Next, the quantized transform unit 10 performs compensation residual image I.₆Is decomposed into a set of coefficients as described above, and a quantization transform coefficient I is obtained by performing a quantization operation on the coefficients included in the set.₇Is output (step S09).
[0053]
Next, the quantization transform coefficient I is obtained by the inverse quantization transform unit 14.₇By applying an inverse quantization operation to, an inverse quantization transform coefficient is generated. Then, the inverse quantized transform coefficient is subjected to an inverse transform process of the transform process executed by the quantizing transform unit 10, and the decoded compensation residual image I₈Is generated (step S10).
[0054]
Next, the decoded compensation residual image I₈And residual compensation image I_FourAre added by the adder 16 and the decoded prediction residual image I₉Is generated (step S11). Further, the decoded image generation unit 18 decodes the decoded prediction residual image I.₉And predicted image I₁Is added to the decoded image I_TenIs generated (step S12). Decoded image I_TenIs referred to by the reference image storage unit 20 as described above.₁₁(Step S13) and is used by the motion compensation prediction unit 2 to generate a prediction image of the encoding target image. And movement information I₂And residual compensation information I_FiveAnd the quantized transform coefficient I₇Are encoded by the encoding unit 12 so that the compressed data I₁₆Is generated (step S14).
[0055]
Next, a moving picture coding program 30 that causes a computer to function as the moving picture coding apparatus 1 will be described. FIG. 5 is a diagram showing a configuration of the moving image encoding program 30. As shown in FIG. 5, the moving image encoding program 30 includes a main module 31 that supervises processing, a motion compensation prediction module 32, a motion residual generation module 34, a residual compensation module 36, a subtraction module 38, A quantization conversion module 40, an encoding module 42, an inverse quantization conversion module 44, an addition module 46, a decoded image generation module 48, and a reference image storage module 50 are provided. The residual compensation module 36 includes a transform submodule 36a, a base storage submodule 36b, a quantization submodule 36c, an inverse quantization submodule 36d, and an inverse transform submodule 36e.
[0056]
Motion compensation prediction module 32, motion residual generation module 34, residual compensation module 36, subtraction module 38, quantization conversion module 40, encoding module 42, inverse quantization conversion module 44, addition module 46, and decoded image generation module 48 , The reference image storage module 50, the transform submodule 36a, the base storage submodule 36b, the quantization submodule 36c, the inverse quantization submodule 36d, and the inverse transform submodule 36e have the functions realized by the computer, Motion residual generation unit 4, residual compensation unit 6, subtraction unit 8, quantization conversion unit 10, encoding unit 12, inverse quantization conversion unit 14, addition unit 16, decoded image generation unit 18, reference image storage unit 20 , Transformation unit 6a, base storage unit 6b, quantization unit 6c, inverse quantization unit 6d, inverse transformation unit 6e and the like It is the same.
[0057]
Hereinafter, the operation and effect of the video encoding device 1 according to the first embodiment will be described. In the moving image encoding device 1, the MP method is applied to a prediction residual image that is a difference between a prediction image generated by performing motion compensation prediction on an encoding target image and the encoding target image. Thus, a signal pattern that easily appears in the prediction residual image can be efficiently expressed with a small number of bases. For example, since the coefficients appear in all frequency bands in orthogonal transform such as DCT or wavelet transform like the image of the edge portion, it cannot be efficiently encoded. However, by storing the edge shape in the two-dimensional basis in advance, it is possible to express the prediction residual image using a small number of bases, and thus more efficient encoding is realized.
[0058]
Also, the compensation residual image, which is the difference between the residual compensation image and the prediction residual image, is an image from which a pattern that tends to appear in the prediction residual image is removed, and therefore can be efficiently expressed using DCT. . As described above, in the video encoding device 1 according to the first embodiment of the present invention, the prediction residual image and the residual compensation image can be decomposed by methods suitable for each. As a result, the encoding target image can be encoded more efficiently.
[0059]
In addition, although the conversion part 6a in this embodiment was comprised so that it might follow MP method as a predetermined | prescribed 1st conversion rule, as long as it follows the conversion rule which can decompose | disassemble a specific pattern more effectively than MP method. The effect of the idea of the present invention can be obtained. For example, a vector quantization method (hereinafter referred to as VQ method) can be applied to the predetermined first conversion rule in addition to the MP method.
[0060]
The VQ method repeats the process of using a prediction residual image as an initial residual component and decomposing the residual component using a basis set using the following equation (2).
[Expression 2]

Here, in Expression (2), f is a prediction residual image, and R_mf is a residual component after the m-th iteration. R_mminimize the energy of f (coefficient a_n, Basis v_kn) Is selected, and the residual component is decomposed into the selected basis and a coefficient for multiplying the basis. In the VQ method, by using a base set based on a pattern that tends to appear in a predicted residual image, the image is sequentially decomposed using a base in which the residual component is small from the base set. The prediction residual image can be decomposed more efficiently by the number.
[0061]
Next, a video encoding device 60 according to the second embodiment of the present invention will be described. The video encoding device 60 performs reference decoding prediction residual image I.₁₇A decoded prediction residual image storage unit (decoded prediction residual image storage unit) 68 for storing the reference decoded prediction stored in the decoded prediction residual image storage unit 68 as a base used for conversion in residual compensation. Residual image I₁₇The partial image is used. Hereinafter, the moving image encoding device 60 will be described in detail.
[0062]
A functional configuration of the moving image encoding device 60 will be described. FIG. 6 is a diagram illustrating a functional configuration of the moving image encoding device 60. Functionally, the moving image encoding device 60 includes a motion compensation prediction unit 2, a motion residual generation unit 4, a residual compensation unit (residual compensation means) 62, a subtraction unit 8, and a quantization conversion unit. 10, an encoding unit (encoding unit) 64, an inverse quantization conversion unit 14, an adding unit (adding unit) 66, a decoded image generation unit 18, a reference image storage unit 20, and a decoded prediction residual image And a storage unit (decoded prediction residual image storage means) 68. Hereinafter, elements different from the moving picture encoding apparatus 1 of the first embodiment will be described.
[0063]
The decoded prediction residual image storage unit 68 includes a decoded prediction residual image I output from the addition unit 66.₉Reference decoded prediction residual image I₁₇And stored reference decoded prediction residual image I₁₇Is provided to the residual compensation unit 62.
[0064]
The residual compensation unit 62 is a prediction residual image I output by the motion residual generation unit 4._ThreeResidual compensation image I_FourIs generated. Also, the residual compensation unit 62 obtains information necessary for reproducing the residual compensation as residual compensation information I._FiveTo the encoding unit 64.
[0065]
Hereinafter, the residual compensation unit 62 will be described in detail. FIG. 7 is a diagram illustrating a configuration of the residual compensation unit 62. The residual compensation unit 62 includes a conversion unit (conversion unit) 62a, a quantization unit (quantization unit) 62b, an inverse quantization unit (inverse quantization unit) 62c, and an inverse conversion unit (inverse conversion unit) 62d. have.
[0066]
The conversion unit 62a is similar to the conversion unit 6a of the first embodiment, and the prediction residual image I_ThreeIs subjected to a conversion process using a predetermined first conversion rule. The conversion unit 62a can apply the MP method or the VQ method as the predetermined first conversion rule. The conversion unit 6a of the first embodiment uses the base stored in the base storage unit 6b as the base used in the MP method or the VQ method.₁₇Are used as the basis.
[0067]
FIG. 8 is a diagram for explaining the conversion processing by the conversion unit 62a. As illustrated in FIG. 8, the conversion unit 62a performs the prediction residual image I._ThreeAttention area R₁In the image conversion process, the reference decoded prediction residual image I₁₇Corresponding region R₂And a region R in the vicinity thereof_ThreeThe partial image at is used as the basis. Of these partial images, the conversion unit 62a selects the attention area R.₁By selecting a partial image that minimizes the residual energy for the image of₁Is decomposed into bases. The conversion unit 62a uses information for specifying the base (partial image) used for the decomposition as the base information I.₁₂Is output to the inverse conversion unit 62d. In addition, the conversion unit 62a receives the base information I.₁₂, Residual compensation information I_FiveIs output to the encoding unit 64 as a part of. In addition, the conversion unit 62a uses a coefficient to be multiplied by the base as a compensation coefficient I.₁₃Is output to the quantization unit 62b.
[0068]
The quantization unit 62b outputs the compensation coefficient I output from the conversion unit 62a.₁₃By applying a quantization operation to the quantization compensation coefficient I₁₄Is generated. The quantization unit 62b generates a quantization compensation coefficient I₁₄Is output to the inverse quantization unit 62c and the quantization compensation coefficient I₁₄Residual compensation information I_FiveIs output to the encoding unit 64 as a part of.
[0069]
The inverse quantization unit 62c performs quantization compensation coefficient I₁₄Is inversely quantized, and the inverse quantization compensation coefficient I₁₅Is generated. The inverse quantization unit 62c generates an inverse quantization compensation coefficient I₁₅Is output to the inverse conversion unit 62d. The inverse transform unit 62d is configured to generate an inverse quantization compensation coefficient I₁₅, Basis information I₁₂, And reference decoded prediction residual image I₁₇And performing the inverse transformation process of the above-described transformation process performed by the transformation unit 62a to obtain the residual compensation image I_FourIs generated. The inverse transform unit 62d performs the residual compensation image I_FourIs output to the subtracting unit 8 and the adding unit 66.
[0070]
The encoding unit 64 outputs the motion information I output by the motion compensation prediction unit 2.₂And the quantized transform coefficient I output by the quantized transform unit 10₇And residual compensation information I output by the residual compensation unit 62_FiveAre generated, and compressed data including the encoded data is output. Note that the compression processing by the encoding unit 64 is the same processing as the encoding unit 12 of the first embodiment.
[0071]
Hereinafter, the operation of the moving image encoding device 60 will be described, and the moving image encoding method according to the second embodiment of the present invention will also be described. FIG. 9 is a flowchart of the moving picture coding method according to the second embodiment. FIG. 10 is a flowchart of processing relating to residual compensation in the moving picture coding method according to the second embodiment.
[0072]
As shown in FIG. 9, in the moving picture coding method according to the second embodiment, first, motion compensation prediction is performed by the motion compensation prediction unit 2 (step S21). The process in step S21 is the same as the process in step S01 of the first embodiment.
[0073]
Next, the predicted image I output by the motion compensation prediction unit 2₁And the encoding target image I₀Is calculated by the motion residual generation unit 4 so that the prediction residual image I_ThreeIs generated (step S22).
[0074]
Next, residual compensation is performed by the residual compensation unit 62 (step S23). In the residual compensation, as shown in FIG. 10, the prediction residual image I is converted by the conversion unit 62a._ThreeIs disassembled. Here, as described above, the conversion unit 62a performs the reference decoded prediction residual image I provided by the decoded prediction residual image storage unit 68.₁₇Prediction residual image I using a partial image of_ThreeDecompose. Then, the prediction residual image I is converted by the conversion unit 62a._ThreeInformation I for specifying the base (partial image) used in the decomposition of₁₂And compensation coefficient I multiplied by the base₁₃Is output (step S24).
[0075]
Next, the compensation coefficient I is obtained by the quantizing unit 62b.₁₃Is quantized and the quantization compensation coefficient I₁₄(Step S25). Basic information I₁₂And quantization compensation coefficient I₁₄Is residual compensation information I_FiveIs output to the encoding unit 64.
[0076]
Next, a quantization compensation coefficient I is obtained by the inverse quantization unit 62c.₁₄Are inversely quantized and the inverse quantization compensation coefficient I₁₅(Step S26). Then, by the inverse transforming unit 62d, the inverse quantization compensation coefficient I₁₅, Basis information I₁₂, And reference decoded prediction residual image I₁₇Is used, and an inverse transformation process, which is an inverse operation of the transformation process by the transformation unit 62a, is performed, whereby the residual compensation image I_FourIs generated. (Step S27).
[0077]
Referring to FIG. 9, next, a residual compensation image I output by the residual compensation unit 6 is obtained._FourAnd the prediction residual image I output by the motion residual generation unit 4_ThreeIs calculated by the subtracting unit 8 so that the compensation residual image I₆Is generated (step S28).
[0078]
Next, the quantized transform unit 10 performs compensation residual image I.₆Is decomposed into a set of coefficients as described above, and a quantization transform coefficient I is obtained by performing a quantization operation on the coefficients included in the set.₇Is output (step S29).
[0079]
Next, the quantization transform coefficient I is obtained by the inverse quantization transform unit 14.₇By applying an inverse quantization operation to, an inverse quantization transform coefficient is generated. Then, the inverse quantized transform coefficient is subjected to an inverse transform process of the transform process executed by the quantizing transform unit 10, and the decoded compensation residual image I₈Is generated (step S30).
[0080]
Next, the decoded compensation residual image I₈And residual compensation image I_FourAre added by the adding unit 66 to obtain a decoded prediction residual image I.₉Is generated (step S31). The decoded prediction residual image I9 is stored in the reference decoded prediction residual image I by the decoded prediction residual image storage unit 68.₁₇And provided to the processing by the residual compensation unit 62 (step S32).
[0081]
Next, the decoded predicted image I is decoded by the decoded image generator 18.₉And predicted image I₁Is added to the decoded image I_TenIs generated (step S33). Decoded image I_TenIs referred to by the reference image storage unit 20 as described above.₁₁(Step S34) and used by the motion compensated prediction unit 2 for generating a predicted image of the encoding target image. And movement information I₂And residual compensation information I_FiveAnd the quantized transform coefficient I₇Are encoded by the encoding unit 64, so that the compressed data I₁₆Is generated (step S35).
[0082]
Next, a video encoding program 70 that causes a computer to function as the video encoding device 60 will be described. FIG. 11 is a diagram showing a configuration of the moving image encoding program 70. As shown in FIG. 11, the moving image encoding program 70 includes a main module 31 that supervises processing, a motion compensation prediction module 32, a motion residual generation module 34, a residual compensation module 72, a subtraction module 38, The quantization transform module 40, the encoding module 74, the inverse quantization transform module 44, the addition module 76, the decoded prediction residual image storage module 78, the decoded image generation module 48, and the reference image storage module 50 Prepare. The residual compensation module 72 includes a transform submodule 72a, a quantization submodule 72b, an inverse quantization submodule 72c, and an inverse transform submodule 72d.
[0083]
Motion compensation prediction module 32, motion residual generation module 34, residual compensation module 72, subtraction module 38, quantization transform module 40, encoding module 74, inverse quantization transform module 44, addition module 76, decoded prediction residual image The functions that the storage module 78, the decoded image generation module 48, the reference image storage module 50, the transform submodule 72a, the quantization submodule 72b, the inverse quantization submodule 72c, and the inverse transform submodule 72d realize in the computer are motion compensated prediction. 2, motion residual generation unit 4, residual compensation unit 62, subtraction unit 8, quantization conversion unit 10, encoding unit 64, inverse quantization conversion unit 14, addition unit 66, decoded prediction residual image storage unit 68 , Decoded image generation unit 18, reference image storage unit 20, conversion unit 62a, quantization unit 62b, inverse quantization unit 2c, respectively similar to the inverse transform unit 62d.
[0084]
As described above, prediction is performed by using a partial image of the reference decoded prediction residual image (decoded prediction residual image) stored in the decoded prediction residual image storage unit 68 as a base to be used for conversion processing in residual compensation. Since more effective residual compensation using the similarity of residual images can be performed, efficient coding can be realized.
[0085]
Next, a video encoding device 80 according to the third embodiment of the present invention will be described. The moving image encoding apparatus 80 includes a base storage unit (base storage means) 82b for storing a base set, and a reference decoded prediction residual image I.₁₇A decoded prediction residual image storage unit (decoded prediction residual image storage unit) 68, and a base set stored in the base storage unit 82b as a base used for conversion in residual compensation, or decoding Reference decoded prediction residual image I stored in the prediction residual image storage unit 68₁₇The partial image is used.
[0086]
A functional configuration of the moving image encoding device 80 will be described. FIG. 12 is a diagram illustrating a functional configuration of the moving image encoding device 80. Functionally, the moving image encoding device 80 is functionally composed of a motion compensation prediction unit 2, a motion residual generation unit 4, a residual compensation unit (residual compensation means) 82, a subtraction unit 8, and a quantization conversion unit. 10, an encoding unit (encoding means) 84, an inverse quantization conversion unit 14, an addition unit 66, a decoded image generation unit 18, a reference image storage unit 20, and a decoded prediction residual image storage unit 68. Is provided. Hereinafter, elements that are different from the moving image encoding device 1 of the first embodiment or the moving image encoding device 60 of the second embodiment will be described.
[0087]
Residual compensation unit 82 predicts residual image I_ThreeResidual compensation image I_FourIs generated. The residual compensation unit 82 generates the generated residual compensation image I_FourIs output to the subtracting unit 8 and the adding unit 66. The residual compensation unit 82 also obtains information necessary for reproducing the residual compensation as residual compensation information I._FiveTo the encoding unit 84.
[0088]
Hereinafter, the residual compensation unit 82 will be described in detail. FIG. 13 is a diagram illustrating a configuration of the residual compensation unit 82. As shown in FIG. 13, the residual compensation unit 82 includes a conversion unit (conversion unit) 82a, a base storage unit (base storage unit) 82b, a quantization unit (quantization unit) 82c, and an inverse quantization unit ( (Inverse quantization means) 82d and an inverse transform unit (inverse transform means) 82e.
[0089]
The conversion unit 82a performs prediction residual image I_ThreeIs subjected to a conversion process using a predetermined first conversion rule. As the predetermined first conversion rule, the MP method or the VQ method can be used as in the first embodiment.
[0090]
In the present embodiment, the base storage unit 82b stores a base set including a plurality of two-dimensional bases. Similarly to the first embodiment, the conversion unit 82a uses the basis set provided by the basis storage unit 82b to predict the residual image I._Three, And the reference decoded prediction residual image I provided from the decoded prediction residual image storage unit 68 as in the second embodiment.₁₇Prediction residual image I using partial image in_ThreeCan also be disassembled.
[0091]
The converting unit 82a is configured to store a reference set stored in the base storage unit 82b or a reference decoded prediction residual image I provided from the decoded prediction residual image storage unit 68.₁₇Among the partial images in the middle, the one having the largest energy reduction amount is selected as a basis, and the prediction residual image I_ThreeDecompose. The conversion unit 82a converts the information used to specify the base used for the decomposition into the base information I.₁₂Is output to the inverse conversion unit 62d. In addition, the conversion unit 82a generates the base information I₁₂, Residual compensation information I_FiveIs output to the encoding unit 84 as a part thereof. In addition, the conversion unit 62a uses a coefficient to be multiplied by the base as a compensation coefficient I.₁₃Is output to the quantizing unit 82c.
[0092]
The quantization unit 82c outputs the compensation coefficient I output from the conversion unit 82a.₁₃By applying a quantization operation to the quantization compensation coefficient I₁₄Is generated. The quantization unit 82c generates a quantization compensation coefficient I₁₄Is output to the inverse quantization unit 82d and the quantization compensation coefficient I₁₄Residual compensation information I_FiveIs output to the encoding unit 84 as a part thereof.
[0093]
The inverse quantization unit 82d generates a quantization compensation coefficient I₁₄Is inversely quantized, and the inverse quantization compensation coefficient I₁₅Is generated. The inverse quantization unit 82d generates an inverse quantization compensation coefficient I₁₅Is output to the inverse conversion unit 82e. The inverse transform unit 82e generates a dequantization compensation coefficient I₁₅, Basis information I₁₂, The base set stored in the base storage unit 82b, and the reference decoded prediction residual image I₁₇Is used to execute the inverse transformation process of the above-described transformation process performed by the transformation unit 82a._FourIs generated. The inverse transform unit 82e generates a residual compensation image I._FourIs output to the subtracting unit 8 and the adding unit 66.
[0094]
The encoding unit 64 outputs the motion information I output by the motion compensation prediction unit 2.₂And the quantized transform coefficient I output by the quantized transform unit 10₇And residual compensation information I output by the residual compensation unit 62_FiveAre generated, and compressed data including the encoded data is output. Note that the compression processing by the encoding unit 84 is the same processing as the encoding unit 12 of the first embodiment.
[0095]
Hereinafter, the operation of the moving image encoding apparatus 80 will be described, and a moving image encoding method according to the third embodiment of the present invention will be described. FIG. 14 is a flowchart of the moving picture coding method according to the third embodiment. FIG. 15 is a flowchart of processing relating to residual compensation in the moving picture coding method according to the third embodiment.
[0096]
As shown in FIG. 14, in the moving picture coding method according to the third embodiment, first, motion compensation prediction is performed by the motion compensation prediction unit 2 (step S41). The process in step S41 is the same as the process in step S01 of the first embodiment.
[0097]
Next, the predicted image I output by the motion compensation prediction unit 2₁And the encoding target image I₀Is calculated by the motion residual generation unit 4 so that the prediction residual image I_ThreeIs generated (step S42).
[0098]
Next, residual compensation is performed by the residual compensation unit 82 (step S43). In the residual compensation, as shown in FIG. 15, a prediction residual image I is converted by the conversion unit 82a._ThreeIs disassembled. Here, as described above, the conversion unit 82a stores the storage set stored in the base storage unit 82b and the reference decoded prediction residual image I provided by the decoded prediction residual image storage unit 68.₁₇Prediction residual image I using a partial image of_ThreeDecompose. Then, the prediction residual image I is converted by the conversion unit 82a._ThreeInformation I for identifying the basis used for decomposition of₁₂And compensation coefficient I multiplied by the base₁₃Is output (step S44).
[0099]
Next, the compensation coefficient I is quantized by the quantizing unit 82c.₁₃Is quantized and the quantization compensation coefficient I₁₄(Step S45). Basic information I₁₂And quantization compensation coefficient I₁₄Is residual compensation information I_FiveIs output to the encoding unit 64.
[0100]
Next, the quantization compensation coefficient I is obtained by the inverse quantization unit 82d.₁₄Are inversely quantized and the inverse quantization compensation coefficient I₁₅(Step S46). Then, by the inverse transform unit 82e, the inverse quantization compensation coefficient I₁₅, Basis information I₁₂, The base set stored in the base storage unit 82b, and the reference decoded prediction residual image I₁₇Is used, and an inverse transformation process, which is an inverse operation of the transformation process by the transformation unit 62a, is performed, whereby the residual compensation image I_FourIs generated. (Step S47).
[0101]
Referring to FIG. 14, next, the residual compensation image I output by the residual compensation unit 82._FourAnd the prediction residual image I output by the motion residual generation unit 4_ThreeIs calculated by the subtracting unit 8 so that the compensation residual image I₆Is generated (step S48).
[0102]
Next, the quantized transform unit 10 performs compensation residual image I.₆Is decomposed into a set of coefficients as described above, and a quantization transform coefficient I is obtained by performing a quantization operation on the coefficients included in the set.₇Is output (step S49).
[0103]
Next, the quantization transform coefficient I is obtained by the inverse quantization transform unit 14.₇By applying an inverse quantization operation to, an inverse quantization transform coefficient is generated. Then, the inverse quantized transform coefficient is subjected to an inverse transform process of the transform process executed by the quantizing transform unit 10, and the decoded compensation residual image I₈Is generated (step S50).
[0104]
Next, the decoded compensation residual image I₈And residual compensation image I_FourAre added by the adding unit 66 to obtain a decoded prediction residual image I.₉Is generated (step S51). Decoded prediction residual image I₉The reference prediction prediction residual image I is decoded by the decoding prediction residual image storage unit 68.₁₇And provided to the processing by the residual compensation unit 82 (step S52).
[0105]
Next, the decoded predicted image I is decoded by the decoded image generator 18.₉And predicted image I₁Is added to the decoded image I_TenIs generated (step S53). Decoded image I_TenIs referred to by the reference image storage unit 20 as described above.₁₁(Step S54) and is used by the motion compensated prediction unit 2 for generating a predicted image of the encoding target image. And movement information I₂And residual compensation information I_FiveAnd the quantized transform coefficient I₇Are encoded by the encoding unit 84, so that the compressed data I₁₆Is generated (step S55).
[0106]
Next, a video encoding program 90 that causes a computer to function as the video encoding device 80 will be described. FIG. 16 is a diagram showing a configuration of the moving image encoding program 90. As shown in FIG. 16, the moving image encoding program 90 includes a main module 31 that supervises processing, a motion compensation prediction module 32, a motion residual generation module 34, a residual compensation module 92, a subtraction module 38, The quantization transform module 40, the encoding module 94, the inverse quantization transform module 44, the addition module 76, the decoded prediction residual image storage module 78, the decoded image generation module 48, and the reference image storage module 50 Prepare. The residual compensation module 92 includes a transform submodule 92a, a base storage submodule 92b, a quantization submodule 92c, an inverse quantization submodule 92d, and an inverse transform submodule 92e.
[0107]
Motion compensation prediction module 32, motion residual generation module 34, residual compensation module 92, subtraction module 38, quantization transform module 40, encoding module 94, inverse quantization transform module 44, addition module 76, decoded prediction residual image The storage module 78, the decoded image generation module 48, the reference image storage module 50, the transform submodule 92a, the base storage submodule 92b, the quantization submodule 92c, the inverse quantization submodule 92d, and the inverse transform submodule 92e are realized by a computer. Functions include motion compensation prediction unit 2, motion residual generation unit 4, residual compensation unit 82, subtraction unit 8, quantization conversion unit 10, encoding unit 84, inverse quantization conversion unit 14, addition unit 66, and decoding prediction. Residual image storage unit 68, decoded image generation unit 18, reference image storage unit 20, conversion unit 82 , Base storage unit 82b, a quantization unit 82c, the inverse quantization unit 82d, respectively similar to the inverse transform unit 82e.
[0108]
As described above, according to the video encoding device 80, the reference decoded prediction residual image I stored in the decoded prediction residual image storage unit 68 as a base used in conversion in residual compensation.₁₇Are used together with the base set stored in the base storage unit 82b. Therefore, the moving picture coding apparatus 80 effectively uses the similarity of the prediction residual image to effectively predict the prediction residual image I._ThreeThe base set stored in the base storage unit 82b can be used when there is a sudden change between frames, thereby realizing efficient coding. be able to.
[0109]
Next, the video decoding device 100 according to the fourth embodiment of the present invention will be described. The moving image decoding device 100 is a device that generates a moving image by decoding the compressed data generated by the moving image encoding device 1 of the first embodiment. The video decoding device 100 is physically a computer including a CPU (Central Processing Unit), a storage device such as a memory, a storage device such as a hard disk, and the like. Here, the “computer” includes an information portable terminal such as a mobile communication terminal in addition to a normal computer such as a personal computer, and the idea of the present invention is widely applied to devices capable of information processing.
[0110]
Hereinafter, a functional configuration of the video decoding device 100 will be described. FIG. 17 is a diagram illustrating a functional configuration of the video decoding device 100. Functionally, the moving image decoding apparatus 100 includes a decoding unit (decoding unit) 102, an inverse quantization conversion unit (inverse quantization conversion unit) 104, a residual decoding unit (residual decoding unit) 106, an addition Unit (adding unit) 108, motion compensation prediction unit (motion compensation prediction unit) 110, decoded image generation unit (decoded image generation unit) 112, and reference image storage unit (reference image storage unit) 114.
[0111]
The decoding unit 102 uses the compressed data I generated by the video encoding device 1.₁₆Is decrypted. The decoding unit 102 uses the compressed data I₁₆Quantized transform coefficient I obtained by decoding₇Is output to the inverse quantization transform unit 104. In addition, the decoding unit 102 uses the compressed data I₁₆Residual compensation information I obtained by decoding_FiveAre output to the residual decoding unit 106. In addition, the decoding unit 102 uses the compressed data I₁₆Motion information I obtained by decoding₂Is output to the motion compensation prediction unit 110.
[0112]
The inverse quantization transform unit 104 generates an inverse quantization transform coefficient obtained by performing an inverse quantization operation on the quantized transform coefficient. The inverse quantization transform unit 104 uses the predetermined second inverse transform rule described above to inversely transform the inverse quantization transform coefficient, thereby decoding the decoded compensation residual image I.₈Is generated. This predetermined second inverse conversion rule is an inverse conversion operation of the predetermined second conversion rule used by the moving picture coding apparatus 1 of the first embodiment.
[0113]
Residual decoding unit 106 provides residual compensation information I output by decoding unit 102._FiveThe residual compensated image I is obtained by performing residual decoding using_FourIs generated. Details of the processing by the residual decoding unit 106 will be described later.
[0114]
The adding unit 108 is configured to generate a decoded compensation residual image I generated by the inverse quantization transform unit 104.₈And the residual compensation image I generated by the residual decoding unit 106_FourTo obtain a decoded prediction residual image I₉Is generated.
[0115]
The motion compensation prediction unit 110 outputs the motion information I output by the decoding unit 102.₂Is used to store the reference image I stored in the reference image storage unit 114.₁₁Motion compensated prediction for the decoded image I_TenPredicted image I₁Is generated.
[0116]
The decoded image generation unit 112 generates a decoded prediction residual image I generated by the addition unit 108.₉And the predicted image I generated by the motion compensated prediction unit 110₁The decoded image I_TenIs generated. The decoded image generation unit 112 generates a decoded image I_TenIs output to the reference image storage unit 114, and the decoded image I is output._TenIs stored in the reference image storage unit 114. The reference image storage unit 114 stores the stored decoded image I._TenTo the motion compensation prediction unit 110 for motion compensation prediction.₁₁As offered.
[0117]
Next, details of the residual decoding unit 106 will be described. Residual decoding unit 106 provides residual compensation information I._FiveQuantization compensation coefficient I included in₁₄Inverse quantization compensation coefficient I obtained by inverse quantization₁₅And residual compensation information I_FiveBase information I contained in₁₂Is used to perform the inverse transformation using a predetermined first inverse transformation rule, and the residual compensation image I_FourIs generated. This predetermined first inverse conversion rule is an inverse conversion operation of the predetermined first conversion rule used by the moving picture coding apparatus 1 of the first embodiment.
[0118]
FIG. 18 is a diagram illustrating a configuration of the residual decoding unit 106. The residual decoding unit 106 includes an inverse quantization unit (inverse quantization unit) 106a, an inverse transform unit (inverse transform unit) 106b, and a base storage unit (base storage unit) 106c.
[0119]
The inverse quantization unit 106a performs quantization compensation coefficient I₁₄Is inversely quantized, and the inverse quantization compensation coefficient I₁₅Is generated. The inverse transform unit 106b receives the basis information I from the basis set stored in the basis storage unit 106c.₁₂Extract the base specified by. The inverse transform unit 106b includes the extracted base and the inverse quantization compensation coefficient I generated by the inverse quantization unit 106a.₁₅Is used to perform an inverse transformation process, which is an inverse operation of the transformation process by the transformation unit 6a, and the residual compensation image I_FourIs generated. The base storage unit 106c can store a base set including a base similar to the base stored in the base storage unit 6b.
[0120]
Hereinafter, an operation of the video decoding device 100 will be described, and a video decoding method according to the fourth embodiment will be described together. FIG. 19 is a flowchart of the video decoding method according to the fourth embodiment. FIG. 20 is a flowchart illustrating processing relating to residual decoding of the moving picture decoding method according to the fourth embodiment.
[0121]
As shown in FIG. 19, in the moving image decoding method of the fourth embodiment, first, the compressed data I generated by the moving image encoding device 1 is used.₁₆To the quantized transform coefficient I by the decoding unit 102.₇And residual compensation information I_FiveAnd movement information I₂Is decoded (step S61).
[0122]
Next, the inverse quantization transform unit 104 performs quantization transform coefficient I₇An inverse quantization transform coefficient is generated by performing an inverse quantization operation on. The inverse quantization transform unit 104 performs inverse transform on the generated inverse quantization transform coefficient according to a predetermined second inverse transform rule, thereby obtaining a decoded compensation residual image I₈Is restored (step S62).
[0123]
Next, the residual decoding unit 106 performs residual compensation information I decoded by the decoding unit 102._FiveIs used to perform residual decoding, and a residual compensated image I_FourIs generated (step S63). As shown in FIG. 20, in the residual decoding unit 106, the inverse quantization unit 106a performs the quantization compensation coefficient I.₁₄By inverse quantization of the inverse quantization compensation coefficient I₁₅Is generated (step S64).
[0124]
Next, the inverse transform unit 106b performs the inverse quantization compensation coefficient I₁₅, Residual compensation information I_FiveBase information I contained in₁₂, And basis information I stored in the basis storage unit 106c₁₂Is used to perform an inverse transform process, which is an inverse operation of the transform process by the transform unit 6a, and the residual compensation image I_FourIs generated (step S65).
[0125]
Referring to FIG. 19, the adder 108 then decodes the decoded compensation residual image I.₈And residual compensation image I_FourAnd the decoded prediction residual image I₉Is generated (step S66).
[0126]
Next, the motion compensation prediction unit 110 outputs the motion information I output by the decoding unit 102.₂Is used to store the reference image I stored in the reference image storage unit 114.₁₁By performing motion compensation prediction on the decoded image I_TenPredicted image I₁Is generated (step S67).
[0127]
Next, the decoded image generation unit 112 performs prediction image I₁And decoded prediction residual image I₉And the decoded image I_TenIs generated (step S68). The decoded image generation unit 112 generates a decoded image I_TenIs output to the reference image storage unit 114, and the decoded image I is output._TenIs stored in the reference image storage unit 114 (step S69). The reference image storage unit 114 stores the stored decoded image I._TenTo the motion compensation prediction unit 110 for motion compensation prediction.₁₁As offered.
[0128]
Hereinafter, a moving picture decoding program 120 that causes a computer to operate as the moving picture decoding apparatus 100 will be described. FIG. 21 is a diagram showing a configuration of the moving picture decoding program 120. The moving image decoding program 120 includes a main module 121 that supervises processing, a decoding module 122, an inverse quantization transform module 124, a residual decoding module 126, an addition module 128, a motion compensation prediction module 130, and a decoded image. A generation module 132 and a reference image storage module 134 are provided. The residual decoding module 126 includes an inverse quantization submodule 126a, an inverse transform submodule 126b, and a base storage submodule 126c.
[0129]
Decoding module 122, inverse quantization transformation module 124, residual decoding module 126, addition module 128, motion compensation prediction module 130, decoded image generation module 132, reference image storage module 134, inverse quantization submodule 126a, inverse transformation submodule 126b and the function that the base storage submodule 126c makes the computer realize are the decoding unit 102, the inverse quantization conversion unit 104, the residual decoding unit 106, the addition unit 108, the motion compensation prediction unit 110, the decoded image generation unit 112, and the reference image The storage unit 114, the inverse quantization unit 106a, the inverse transformation unit 106b, and the base storage unit 106c are the same.
[0130]
Hereinafter, operations and effects of the video decoding device 100 according to the fourth embodiment will be described. The moving picture decoding apparatus 100 uses compressed data I₁₆Are subjected to inverse quantization transformation processing, residual decoding processing, and motion compensation prediction processing. That is, according to the moving picture decoding apparatus 100, the moving picture encoding apparatus 1 performs a reverse process of the process of generating compressed data from a moving picture, so that the moving picture is faithfully restored.
[0131]
Next, a video decoding device 140 according to the fifth embodiment of the present invention will be described. The video decoding device 140 is a compressed data I generated by the video encoding device 60 of the second embodiment.₁₆It is a device that restores a moving image from. In the video decoding device 140, the reference decoded prediction residual image I₁₇Reference decoding stored in the decoded prediction residual image storage unit 148 as a base used in the inverse transform in the residual decoding is provided with a decoded prediction residual image storage unit (decoded prediction residual image storage unit) 148. Prediction residual image I₁₇The partial image is used.
[0132]
Hereinafter, a functional configuration of the video decoding device 140 will be described. FIG. 22 is a diagram illustrating a functional configuration of the moving picture decoding apparatus 140. Functionally, the moving image decoding apparatus 140 includes a decoding unit (decoding unit) 142, an inverse quantization conversion unit 104, a residual decoding unit (residual decoding unit) 144, and an adding unit (adding unit) 146. A motion compensation prediction unit (motion compensation prediction unit) 110, a decoded image generation unit 112, a reference image storage unit 114, and a decoded prediction residual image storage unit (decoded prediction residual image storage unit) 148. Hereinafter, elements different from the moving picture decoding apparatus 100 according to the fourth embodiment will be described.
[0133]
The decoding unit 142 uses the compressed data I generated by the video encoding device 60.₁₆And quantized transform coefficient I₇And residual compensation information I_FiveAnd motion information I₂Is output. This residual compensation information I_FiveIncludes a reference decoded prediction residual image I stored in the decoded prediction residual image storage unit 148.₁₇Using the partial images of the residual compensation image I_FourInformation I for restoring₁₂And the quantization compensation coefficient I₁₄It is included.
[0134]
Residual decoding unit 144 provides residual compensation information I._FiveIs used to perform residual decoding and to obtain a residual compensated image I_FourIs generated. Hereinafter, the residual decoding unit 144 will be described in detail. FIG. 23 is a diagram illustrating a configuration of the residual decoding unit 144. As shown in FIG. 23, the residual decoding unit 144 includes an inverse quantization unit (inverse quantization unit) 144a and an inverse transform unit (inverse transform unit) 144b.
[0135]
The inverse quantization unit 144a performs residual compensation information I_FiveQuantization compensation coefficient I included in₁₄By applying an inverse quantization operation to the inverse quantization compensation coefficient I₁₅Is generated.
[0136]
The inverse transform unit 144b is configured to generate a reference decoded prediction residual image I stored in the decoded prediction residual image storage unit 148.₁₇Of these, residual compensation information I_FiveInformation I contained in₁₂A partial image (base) is specified from The inverse transform unit 144b includes the identified partial image and the inverse quantization compensation coefficient I generated by the inverse quantization unit 144a.₁₅Are used to perform an inverse transformation process that is an inverse operation of the transformation process by the transformation unit 62a, and the residual compensation image I_FourIs generated.
[0137]
Referring to FIG. 22, the adding unit 146 includes the quantized transform coefficient I decoded by the decoding unit 142.₇Is further subjected to inverse quantization and inverse transform by the inverse quantization transform unit 104.₈And the residual compensated image I generated by the residual decoding unit 144_FourAnd the decoded prediction residual image I₉Is generated. The adding unit 146 performs the decoded prediction residual image I₉Is output to the decoded image generation unit 112 and the decoded prediction residual image I₉The decoded prediction residual image I is stored in the decoded prediction residual image storage unit 148.₁₇Remember as. The decoded prediction residual image storage unit 148 stores the stored reference decoded prediction residual image I.₁₇Are provided to the residual decoding unit 144 for residual decoding.
[0138]
Next, an operation of the video decoding device 140 will be described, and a video decoding method according to the fifth embodiment will be described together. FIG. 24 is a flowchart of the video decoding method according to the fifth embodiment. FIG. 25 is a flowchart illustrating processing relating to residual decoding of the moving picture decoding method according to the fifth embodiment.
[0139]
As shown in FIG. 24, in the video decoding method of the fifth embodiment, first, the compressed data I generated by the video encoding device 60 is displayed.₁₆To the quantized transform coefficient I by the decoding unit 142.₇And residual compensation information I_FiveAnd movement information I₂Is decoded (step S71).
[0140]
Next, the inverse quantization transform unit 104 performs quantization transform coefficient I₇An inverse quantization transform coefficient is generated by performing an inverse quantization operation on. The inverse quantization transform unit 104 performs inverse transform on the generated inverse quantization transform coefficient according to a predetermined second inverse transform rule, thereby obtaining a decoded compensation residual image I₈Is restored (step S72).
[0141]
Next, the residual decoding unit 144 performs residual compensation information I decoded by the decoding unit 142._FiveIs used to perform residual decoding, and a residual compensated image I_FourIs generated (step S73). As shown in FIG. 25, in the residual decoding unit 144, the inverse quantization unit 144a performs the quantization compensation coefficient I.₁₄By inverse quantization of the inverse quantization compensation coefficient I₁₅Is generated (step S74).
[0142]
Next, the inverse transform unit 144b performs an inverse quantization compensation coefficient I₁₅, Residual compensation information I_FiveBase information I contained in₁₂, And a reference decoded prediction residual image I provided by the decoded prediction residual image storage unit 148.₁₇Is used to perform an inverse transformation process, which is an inverse operation of the transformation process by the transformation unit 62a, and the residual compensation image I_FourIs generated (step S75).
[0143]
Referring to FIG. 24, the adding unit 146 then decodes the decoded compensation residual image I.₈And residual compensation image I_FourAnd the decoded prediction residual image I₉Is generated (step S76).
[0144]
Next, the decoded prediction residual image storage unit 148 has the decoded prediction residual image I generated by the adding unit 146.₉Reference decoded prediction residual image I₁₇(Step S77). The decoded prediction residual image storage unit 148 stores the stored reference decoded prediction residual image I.₁₇Are provided to the residual decoding unit 144 for residual decoding.
[0145]
Next, the motion compensation prediction unit 110 outputs the motion information I output by the decoding unit 102.₂Is used to store the reference image I stored in the reference image storage unit 114.₁₁By performing motion compensation prediction on the decoded image I_TenPredicted image I₁Is generated (step S78).
[0146]
Next, the decoded image generation unit 112 performs prediction image I₁And decoded prediction residual image I₉And the decoded image I_TenIs generated (step S79). The decoded image generation unit 112 generates a decoded image I_TenIs output to the reference image storage unit 114, and the decoded image I is output._TenIs stored in the reference image storage unit 114 (step S80). The reference image storage unit 114 stores the stored decoded image I._TenTo the motion compensation prediction unit 110 for motion compensation prediction.₁₁As offered.
[0147]
Hereinafter, a video decoding program 150 that causes a computer to operate as the video decoding device 140 will be described. FIG. 26 is a diagram showing a configuration of the moving picture decoding program 150. The video decoding program 150 includes a main module 151 that supervises processing, a decoding module 152, an inverse quantization transform module 124, a residual decoding module 154, an addition module 156, and a decoded prediction residual image storage module 158. , A motion compensation prediction module 130, a decoded image generation module 132, and a reference image storage module 134. The residual decoding module 154 has an inverse quantization submodule 154a and an inverse transform submodule 154b.
[0148]
Decoding module 152, inverse quantization transform module 124, residual decoding module 154, addition module 156, decoded prediction residual image storage module 158, motion compensation prediction module 130, decoded image generation module 132, reference image storage module 134, inverse quantum The functions that the computerization submodule 154a and the inverse transform submodule 154b realize in the computer are a decoding unit 142, an inverse quantization conversion unit 104, a residual decoding unit 144, an adding unit 146, a decoded prediction residual image storage unit 148, motion compensation The prediction unit 110, the decoded image generation unit 112, the reference image storage unit 114, the inverse quantization unit 144a, and the inverse transformation unit 144b are the same.
[0149]
As described above, the video decoding device 140 performs the reference decoded prediction residual image I stored in the decoded prediction residual image storage unit 148 as a base used in inverse transform in residual decoding.₁₇Therefore, the moving image encoding device 60 can faithfully perform the reverse process of the process of generating the compressed data from the moving image. Therefore, the moving picture decoding apparatus 140 can restore the above moving picture from the compressed data generated by the moving picture encoding apparatus 60.
[0150]
A video decoding device 170 according to the sixth embodiment of the present invention will be described. The video decoding device 170 is a compressed data I generated by the video encoding device 80 of the third embodiment.₁₆It is a device that restores a moving image from. In the video decoding device 170, a base storage unit (base storage means) 174c for storing a base set, and a reference decoded prediction residual image I₁₇A decoded prediction residual image storage unit (decoded prediction residual image storage unit) 148, and a base set stored in the base storage unit 174c as a base used in conversion in residual compensation, or decoding Reference decoded prediction residual image I stored in the prediction residual image storage unit 148₁₇The partial image is used.
[0151]
Hereinafter, a functional configuration of the video decoding device 170 will be described. FIG. 27 is a diagram illustrating a functional configuration of the video decoding device 170. Functionally, the moving picture decoding apparatus 170 includes a decoding unit (decoding unit) 172, an inverse quantization conversion unit (inverse quantization conversion unit) 104, a residual decoding unit (residual decoding unit) 174, and an addition Unit (adding unit) 146, motion compensation prediction unit (motion compensation prediction unit) 110, decoded image generation unit (decoded image generation unit) 112, reference image storage unit (reference image storage unit) 114, decoded prediction residual A difference image storage unit (decoded prediction residual image storage means) 148. Hereinafter, elements different from the video decoding device 100 of the fourth embodiment or the video decoding device 140 of the fifth embodiment will be described.
[0152]
The decoding unit 172 includes the compressed data I generated by the video encoding device 80.₁₆And quantized transform coefficient I₇And residual compensation information I_FiveAnd motion information I₂Is output. This residual compensation information I_FiveIncludes a base set stored in the base storage unit 174c using the base set stored in the base storage unit 174c._FourInformation I for restoring₁₂Or the reference decoded prediction residual image I stored in the decoded prediction residual image storage unit 148.₁₇Using the partial images of the residual compensation image I_FourInformation I for restoring₁₂And the quantization compensation coefficient I₁₄It is included.
[0153]
Residual decoding unit 174 provides residual compensation information I._FiveIs used to perform residual decoding and to obtain a residual compensated image I_FourIs generated. Hereinafter, the residual decoding unit 174 will be described in detail. FIG. 28 is a diagram illustrating a configuration of the residual decoding unit 174. As shown in FIG. 28, the residual decoding unit 174 includes an inverse quantization unit (inverse quantization unit) 174a, an inverse transform unit (inverse transform unit) 174b, and a base storage unit (base storage unit) 174c. .
[0154]
The inverse quantization unit 174a performs residual compensation information I_FiveQuantization compensation coefficient I included in₁₄By applying an inverse quantization operation to the inverse quantization compensation coefficient I₁₅Is generated.
[0155]
The inverse transform unit 174b includes the residual compensation information I of the basis set stored in the basis storage unit 174c._FiveInformation I contained in₁₂The base is specified from Further, the inverse transform unit 174b generates a reference decoded prediction residual image I stored in the decoded prediction residual image storage unit 148.₁₇Of these, residual compensation information I_FiveInformation I contained in₁₂To specify a partial image as a base. The inverse transform unit 174b includes the identified base and the inverse quantization compensation coefficient I generated by the inverse quantization unit 144a.₁₅Are used to perform an inverse transformation process, which is an inverse operation of the transformation process by the transformation unit 82a, and the residual compensation image I_FourIs generated.
[0156]
Hereinafter, an operation of the video decoding device 170 will be described, and a video decoding method according to the sixth embodiment will be described together. FIG. 29 is a flowchart of the video decoding method according to the sixth embodiment. FIG. 30 is a flowchart showing processing relating to residual decoding of the moving picture decoding method according to the sixth embodiment.
[0157]
As shown in FIG. 29, in the moving image decoding method of the sixth embodiment, first, the compressed data I generated by the moving image encoding device 80 is used.₁₆From the quantized transform coefficient I by the decoding unit 172₇And residual compensation information I_FiveAnd movement information I₂Is decoded (step S91).
[0158]
Next, the inverse quantization transform unit 104 performs quantization transform coefficient I₇An inverse quantization transform coefficient is generated by performing an inverse quantization operation on. The inverse quantization transform unit 104 performs inverse transform on the generated inverse quantization transform coefficient according to a predetermined second inverse transform rule, thereby obtaining a decoded compensation residual image I₈Is restored (step S92).
[0159]
Next, residual decoding section 174 performs residual compensation information I decoded by decoding section 172._FiveIs used to perform residual decoding, and a residual compensated image I_FourIs generated (step S73). As shown in FIG. 30, in the residual decoding unit 174, the inverse quantization unit 174a performs quantization compensation coefficient I₁₄By inverse quantization of the inverse quantization compensation coefficient I₁₅Is generated (step S94).
[0160]
Next, the inverse transform unit 174b performs an inverse quantization compensation coefficient I₁₅, Residual compensation information I_FiveBase information I contained in₁₂, The base set stored in the base storage unit 174c, and the reference decoded prediction residual image I provided by the decoded prediction residual image storage unit 148.₁₇Is used to perform an inverse transformation process, which is an inverse operation of the transformation process by the transformation unit 82a, and the residual compensation image I_FourIs generated (step S95).
[0161]
Referring to FIG. 29, the adding unit 146 then decodes the residual image for decoding compensation I.₈And residual compensation image I_FourAnd the decoded prediction residual image I₉Is generated (step S96).
[0162]
Next, the decoded prediction residual image storage unit 148 has the decoded prediction residual image I generated by the adding unit 146.₉Reference decoded prediction residual image I₁₇(Step S97). The decoded prediction residual image storage unit 148 stores the stored reference decoded prediction residual image I.₁₇To the residual decoding unit 174 for residual decoding.
[0163]
Next, the motion compensation prediction unit 110 outputs the motion information I output by the decoding unit 172.₂Is used to store the reference image I stored in the reference image storage unit 114.₁₁By performing motion compensation prediction on the decoded image I_TenPredicted image I₁Is generated (step S98).
[0164]
Next, the decoded image generation unit 112 performs prediction image I₁And decoded prediction residual image I₉And the decoded image I_TenIs generated (step S99). The decoded image generation unit 112 generates a decoded image I_TenIs output to the reference image storage unit 114, and the decoded image I is output._TenIs stored in the reference image storage unit 114 (step S100). The reference image storage unit 114 stores the stored decoded image I._TenTo the motion compensation prediction unit 110 for motion compensation prediction.₁₁As offered.
[0165]
Hereinafter, a video decoding program 180 that causes a computer to operate as the video decoding device 170 will be described. FIG. 31 is a diagram showing a configuration of the moving picture decoding program 180. The moving picture decoding program 180 includes a main module 181 that supervises processing, a decoding module 182, an inverse quantization transform module 124, a residual decoding module 184, an addition module 156, and a decoded prediction residual image storage module 158. , A motion compensation prediction module 130, a decoded image generation module 132, and a reference image storage module 134. The residual decoding module 184 includes an inverse quantization submodule 154a, an inverse transform submodule 154b, and a base storage submodule 184c.
[0166]
Decoding module 182, inverse quantization transform module 124, residual decoding module 184, addition module 156, decoded prediction residual image storage module 158, motion compensated prediction module 130, decoded image generation module 132, reference image storage module 134, inverse quantum The sub-module 184a, the inverse transform sub-module 184b, and the base storage sub-module 184c make the computer realize the decoding unit 172, the inverse quantization transform unit 104, the residual decoding unit 174, the adding unit 146, and the decoded prediction residual image The storage unit 148, motion compensation prediction unit 110, decoded image generation unit 112, reference image storage unit 114, inverse quantization unit 174a, inverse transform unit 174b, and base storage unit 174c are the same.
[0167]
As described above, the video decoding device 170 uses the base set stored in the base storage unit 174c and the reference stored in the decoded prediction residual image storage unit 148 as the base used in the inverse transform in the residual decoding. Decoded prediction residual image I₁₇Therefore, the moving image encoding device 80 can faithfully perform the reverse process of the process of generating the compressed data from the moving image. Therefore, the moving picture decoding apparatus 170 can restore the above moving picture from the compressed data generated by the moving picture encoding apparatus 80.
[0168]
【The invention's effect】
As described above, according to the present invention, the prediction residual image and the compensation residual image can be decomposed efficiently, so that the moving picture coding apparatus and the moving picture code capable of realizing efficient coding can be realized. And a moving picture encoding program are provided. Also provided are a moving picture decoding apparatus, a moving picture decoding method, and a moving picture decoding program capable of faithfully restoring a moving picture from compressed data generated by the moving picture encoding apparatus.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a functional configuration of a moving image encoding apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration of a residual compensation unit according to the first embodiment.
FIG. 3 is a flowchart of a video encoding method according to the first embodiment of the present invention.
FIG. 4 is a flowchart of processing relating to residual compensation in the moving picture coding method according to the first embodiment;
FIG. 5 is a diagram showing a configuration of a moving image encoding program according to the first embodiment of the present invention.
FIG. 6 is a diagram illustrating a functional configuration of a moving image encoding apparatus according to a second embodiment of the present invention.
FIG. 7 is a diagram illustrating a configuration of a residual compensation unit according to a second embodiment.
FIG. 8 is a diagram for explaining conversion processing by a conversion unit a according to the second embodiment;
FIG. 9 is a flowchart of a video encoding method according to the second embodiment of the present invention.
FIG. 10 is a flowchart of processing relating to residual compensation in the moving picture coding method according to the second embodiment;
FIG. 11 is a diagram showing a configuration of a moving image encoding program according to the second embodiment of the present invention.
FIG. 12 is a diagram illustrating a functional configuration of a moving image encoding device according to a third embodiment of the present invention.
FIG. 13 is a diagram illustrating a configuration of a residual compensation unit according to a third embodiment.
FIG. 14 is a flowchart of a video encoding method according to the third embodiment of the present invention.
FIG. 15 is a flowchart of processing relating to residual compensation in the moving picture coding method according to the third embodiment;
FIG. 16 is a diagram showing a configuration of a moving image encoding program according to a third embodiment of the present invention.
FIG. 17 is a diagram illustrating a functional configuration of a video decoding device according to a fourth embodiment of the present invention.
FIG. 18 is a diagram illustrating a configuration of a residual decoding unit according to the fourth embodiment.
FIG. 19 is a flowchart of a video decoding method according to the fourth embodiment of the present invention.
FIG. 20 is a flowchart illustrating processing related to residual decoding of the moving picture decoding method according to the fourth embodiment.
FIG. 21 is a diagram showing a configuration of a moving picture decoding program according to the fourth embodiment of the present invention.
FIG. 22 is a diagram illustrating a functional configuration of a video decoding device according to a fifth embodiment of the present invention.
FIG. 23 is a diagram illustrating a configuration of a residual decoding unit according to the fifth embodiment.
FIG. 24 is a flowchart of a video decoding method according to the fifth embodiment of the present invention.
FIG. 25 is a flowchart illustrating processing relating to residual decoding of the moving picture decoding method according to the fifth embodiment;
FIG. 26 is a diagram showing a configuration of a moving picture decoding program according to the fifth embodiment of the present invention.
FIG. 27 is a diagram illustrating a functional configuration of a video decoding device according to a sixth embodiment of the present invention.
FIG. 28 is a diagram illustrating a configuration of a residual decoding unit according to the sixth embodiment.
FIG. 29 is a flowchart of a video decoding method according to the sixth embodiment of the present invention.
FIG. 30 is a flowchart illustrating processing related to residual decoding in the moving picture decoding method according to the sixth embodiment;
FIG. 31 is a diagram showing a configuration of a moving picture decoding program according to the sixth embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1,60,80 ... Moving picture coding apparatus, 2 ... Motion compensation prediction part, 4 ... Motion residual generation part, 6, 62, 82 ... Residual compensation part, 6a, 62a, 82a ... Conversion part, 6b, 82b ... base storage unit, 6c, 62b, 82c ... quantization unit, 6d, 62c, 82d ... inverse quantization unit, 6e, 62d, 82e ... inverse transformation unit, 8 ... subtraction unit, 10 ... quantization transformation unit, 12, 64, 84 ... coding unit, 14 ... inverse quantization transform unit, 16, 66 ... addition unit, 18 ... decoded image generation unit, 20 ... reference image storage unit, 68 ... decoded prediction residual image storage unit, 100, 140 , 170 ... moving picture decoding apparatus, 102, 142, 172 ... decoding part, 104 ... inverse quantization conversion part, 106, 144, 174 ... residual decoding part, 106a, 144a, 174a ... inverse quantization part, 106b, 144b , 174b ... Inverse conversion unit, 106c, 174c ... Bottom storage unit, 108,146 ... adding unit, 110 ... motion compensation prediction unit, 112 ... decoded image generating unit, 114 ... reference image storage unit, 148 ... decoding prediction residual image storage unit.

Claims (8)

Motion compensated prediction means for generating a predicted image and motion information by performing motion compensation prediction of an encoding target image based on a reference image;
Residual image generation means for generating a prediction residual image based on a difference calculation between the encoding target image and the prediction image;
Residual compensation means for generating residual compensation information from the prediction residual image based on a predetermined first conversion rule, and generating a residual compensation image based on the residual compensation information;
Subtracting means for generating a compensation residual image based on a difference calculation between the residual compensation image and the prediction residual image;
A quantization transform unit that generates a transform coefficient obtained by transforming the compensation residual image based on a predetermined second transform rule, and generates a quantized transform coefficient by quantizing the transform coefficient;
An inverse quantized transform coefficient is generated by inverse quantizing the quantized transform coefficient, and the inverse quantized transform coefficient is based on a predetermined second inverse transform rule that is an inverse transform of the predetermined second transform rule. An inverse quantization transform means for generating a decoded compensation residual image by performing an inverse transform using
Adding means for generating a decoded prediction residual image by adding the decoded compensation residual image and the residual compensation image;
Decoded image generating means for generating a decoded image by adding the decoded prediction residual image and the predicted image;
Reference image storage means for storing the decoded image generated by the decoded image generation means as the reference image provided for motion compensation prediction by the motion compensation prediction means;
Encoding means for generating compressed data including encoded data obtained by encoding the motion information, the residual compensation information, and the quantized transform coefficient ;
With
The residual compensation means includes
Based on the predetermined first conversion rule, using a plurality of two-dimensional bases, base information for specifying the two-dimensional base and a compensation coefficient attached to the two-dimensional base specified by the base information Conversion means for converting the prediction residual image and including the base information in the residual compensation information;
Quantization means for generating a quantization compensation coefficient by quantizing the compensation coefficient, and including the quantization compensation coefficient in the residual compensation information;
Inverse quantization means for generating an inverse quantization compensation coefficient by inverse quantization of the quantization compensation coefficient;
Inverse transform means for generating the residual compensation image by performing inverse transform using the inverse quantization compensation coefficient based on a predetermined first inverse transform rule that is an inverse transform of the predetermined first transform rule. It has a door,
A decoding prediction residual image storage unit for storing the decoding prediction residual image as a reference decoding prediction residual image;
The converting means converts the prediction residual image using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage means as the two-dimensional basis,
The inverse transform means performs inverse transform using the two-dimensional basis used by the transform means;
A moving picture coding apparatus characterized by the above. The residual compensation means includes
Further comprising basis storage means for storing a two-dimensional basis to be provided to the conversion means;
The converting means converts the prediction residual image using a two-dimensional basis stored in the basis storage means,
2. The moving picture encoding apparatus according to claim 1, wherein the inverse transform unit performs the inverse transform using a two-dimensional basis stored in the basis storage unit. A motion compensation prediction step in which the motion compensation prediction means generates a prediction image and motion information by performing motion compensation prediction of the encoding target image based on the reference image;
A residual image generating means for generating a predicted residual image based on a difference calculation between the encoding target image and the predicted image;
Residual compensation means for generating residual compensation information from the prediction residual image based on a predetermined first conversion rule, and generating a residual compensation image based on the residual compensation information When,
A subtracting step for generating a compensation residual image based on a difference calculation between the residual compensation image and the prediction residual image;
Quantization in which a quantization transform unit generates a transform coefficient obtained by transforming the compensation residual image based on a predetermined second transform rule, and generates a quantized transform coefficient by quantizing the transform coefficient A conversion step;
Inverse quantization conversion means generates an inverse quantized transform coefficient by inversely quantizing the quantized transform coefficient, and converts the quantized transform coefficient to a predetermined second inverse transform rule that is an inverse transform of the predetermined second transform rule. An inverse quantization transform step of generating a decoded compensated residual image by performing an inverse transform using the inverse quantized transform coefficient based on:
An adding step in which an adding unit generates a decoded prediction residual image by adding the decoded compensation residual image and the residual compensation image;
A decoded image generation step, wherein the decoded image generation means generates a decoded image by adding the decoded prediction residual image and the prediction image;
A reference image storage step in which a reference image storage unit stores the decoded image generated by the decoded image generation unit as the reference image to be provided for motion compensation prediction by the motion compensation prediction unit;
An encoding step for generating compressed data including encoded data obtained by encoding the motion information, the residual compensation information, and the quantized transform coefficient ;
With
The residual compensation step includes:
Based on the predetermined first conversion rule, the conversion means uses a plurality of two-dimensional bases to attach base information for specifying the two-dimensional base and a two-dimensional base specified by the base information. Converting the prediction residual image into a compensation coefficient and including the base information in the residual compensation information;
A quantization step for generating a quantization compensation coefficient by quantizing the compensation coefficient, and including the quantization compensation coefficient in the residual compensation information;
An inverse quantization step in which an inverse quantization means generates an inverse quantization compensation coefficient by inversely quantizing the quantization compensation coefficient;
Inverse transform means performs the inverse transform using the inverse quantization compensation coefficient based on a predetermined first inverse transform rule that is an inverse transform of the predetermined first transform rule, thereby obtaining the residual compensation image. An inverse transformation step to generate ;
Seen including,
The decoded prediction residual image storage means further comprises a step of storing the decoded prediction residual image as a reference decoded prediction residual image,
In the conversion step, the conversion unit converts the prediction residual image using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage unit as the two-dimensional basis,
In the inverse transformation step, the inverse transformation means performs inverse transformation using the two-dimensional basis used by the transformation means.
A video encoding method characterized by the above. Computer
Motion compensated prediction means for generating a predicted image and motion information by performing motion compensation prediction of an encoding target image based on a reference image;
Residual image generation means for generating a prediction residual image based on a difference calculation between the encoding target image and the prediction image;
Residual compensation means for generating residual compensation information from the prediction residual image based on a predetermined first conversion rule, and generating a residual compensation image based on the residual compensation information;
Subtracting means for generating a compensation residual image based on a difference calculation between the residual compensation image and the prediction residual image;
A quantization transform unit that generates a transform coefficient obtained by transforming the compensation residual image based on a predetermined second transform rule, and generates a quantized transform coefficient by quantizing the transform coefficient;
An inverse quantized transform coefficient is generated by inverse quantizing the quantized transform coefficient, and the inverse quantized transform coefficient is based on a predetermined second inverse transform rule that is an inverse transform of the predetermined second transform rule. An inverse quantization transform means for generating a decoded compensation residual image by performing an inverse transform using
Adding means for generating a decoded prediction residual image by adding the decoded compensation residual image and the residual compensation image;
Decoded image generating means for generating a decoded image by adding the decoded prediction residual image and the predicted image;
Reference image storage means for storing the decoded image generated by the decoded image generation means as the reference image provided for motion compensation prediction by the motion compensation prediction means;
Encoding means for generating compressed data including encoded data obtained by encoding the motion information, the residual compensation information, and the quantized transform coefficient;
Function as
The residual compensation means includes
Based on the predetermined first conversion rule, using a plurality of two-dimensional bases, base information for specifying the two-dimensional base and a compensation coefficient attached to the two-dimensional base specified by the base information Conversion means for converting the prediction residual image and including the base information in the residual compensation information;
Quantization means for generating a quantization compensation coefficient by quantizing the compensation coefficient, and including the quantization compensation coefficient in the residual compensation information;
Inverse quantization means for generating an inverse quantization compensation coefficient by inverse quantization of the quantization compensation coefficient;
Inverse transform means for generating the residual compensation image by performing inverse transform using the inverse quantization compensation coefficient based on a predetermined first inverse transform rule that is an inverse transform of the predetermined first transform rule. and,
I have a,
Further causing the computer to function as decoded prediction residual image storage means for storing the decoded prediction residual image as a reference decoded prediction residual image;
The converting means converts the prediction residual image using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage means as the two-dimensional basis,
The inverse transform means performs inverse transform using the two-dimensional basis used by the transform means;
A moving picture encoding program characterized by: Residual information including motion information for generating a predicted image for a decoded image, quantization compensation coefficients for restoring a residual compensated image for the predicted image, and base information; the predicted image and the residual By decoding compressed data including encoded data obtained by encoding a quantized transform coefficient for restoring a compensation residual image based on a difference from the compensation image, the motion information and the residual compensation information Decoding means for generating the quantized transform coefficients;
Residual decoding means for generating the residual compensation image by performing inverse transformation based on a predetermined first inverse transformation rule using the residual compensation information;
A decoding compensation residual is generated by generating an inverse quantized transform coefficient by inversely quantizing the quantized transform coefficient and performing an inverse transform based on a predetermined second inverse transform rule using the inverse quantized transform coefficient. An inverse quantization transformation means for generating an image;
Adding means for generating a decoded prediction residual image based on the addition of the residual compensation image and the decoded compensation residual image;
Motion compensated prediction means for generating the predicted image by performing motion compensated prediction using a reference image and the motion information;
Decoded image generating means for generating a decoded image based on the addition of the decoded prediction residual image and the predicted image;
Reference image storage means for storing the decoded image generated by the decoded image generation means as the reference image provided to the motion compensated prediction by the motion compensated prediction means ;
With
The residual decoding means includes
Inverse quantization means for generating an inverse quantization compensation coefficient by inverse quantization of the quantization compensation coefficient included in the residual compensation information;
Using the two-dimensional basis specified by the basis information included in the residual compensation information among a plurality of two-dimensional basis, and the inverse quantization compensation coefficient inverse transform generated by the inverse quantization means, the predetermined quantization Inverse transformation means for generating the residual compensation image by performing inverse transformation based on a first inverse transformation rule ;
I have a,
A decoding prediction residual image storage unit for storing the decoding prediction residual image as a reference decoding prediction residual image;
The inverse transform unit performs the inverse transform using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage unit as the two-dimensional basis.
A moving picture decoding apparatus characterized by the above. The residual decoding means includes
Further comprising basis storage means for storing the plurality of two-dimensional bases;
The moving image decoding apparatus according to claim 5 , wherein the inverse transforming unit performs the inverse transform using the two-dimensional basis stored in the basis storage unit. Decoding means, motion information for generating a predicted image for the decoded image, residual compensation information including quantization compensation coefficients and base information for restoring a residual compensated image for the predicted image, and the predicted image By decoding compressed data including encoded data obtained by encoding a quantized transform coefficient for restoring a compensated residual image based on a difference between the motion information and the residual compensated image. A decoding step of generating difference compensation information and the quantized transform coefficients;
A residual decoding step, wherein the residual decoding means generates the residual compensation image by performing an inverse transformation based on a predetermined first inverse transformation rule using the residual compensation information;
An inverse quantization transformer generates an inverse quantization transformation coefficient by inverse quantization of the quantization transformation coefficient, and performs an inverse transformation based on a predetermined second inverse transformation rule using the inverse quantization transformation coefficient. Performing an inverse quantization transform step to generate a decoded compensated residual image;
An adding means for generating a decoded prediction residual image based on the addition of the residual compensation image and the decoded compensation residual image;
A motion compensation prediction step, wherein the motion compensation prediction means generates the prediction image by performing motion compensation prediction using a reference image and the motion information;
A decoded image generating step for generating a decoded image based on the addition of the decoded prediction residual image and the predicted image;
A reference image storage step in which a reference image storage unit stores the decoded image generated by the decoded image generation unit as the reference image provided to the motion compensated prediction by the motion compensation prediction unit ;
With
The residual decoding step includes:
An inverse quantization step in which an inverse quantization means generates an inverse quantization compensation coefficient by inversely quantizing the quantization compensation coefficient included in the residual compensation information;
Inverse transform means includes a two-dimensional basis specified by basis information included in the residual compensation information among a plurality of two-dimensional bases, and the inverse quantization compensation coefficient inverse transform generated by the inverse quantization means. Using the inverse transformation step to generate the residual compensation image by performing an inverse transformation based on the predetermined first inverse transformation rule ;
Only including,
The decoded prediction residual image storage means further comprises a step of storing the decoded prediction residual image as a reference decoded prediction residual image,
In the inverse transformation step, the inverse transformation means performs the inverse transformation using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage means as the two-dimensional basis.
A moving picture decoding method characterized by the above. Computer
Residual information including motion information for generating a predicted image for a decoded image, quantization compensation coefficients for restoring a residual compensated image for the predicted image, and base information; the predicted image and the residual By decoding compressed data including encoded data obtained by encoding a quantized transform coefficient for restoring a compensation residual image based on a difference from the compensation image, the motion information and the residual compensation information Decoding means for generating the quantized transform coefficients;
Residual decoding means for generating the residual compensation image by performing inverse transformation based on a predetermined first inverse transformation rule using the residual compensation information;
A decoding compensation residual is generated by generating an inverse quantized transform coefficient by inversely quantizing the quantized transform coefficient and performing an inverse transform based on a predetermined second inverse transform rule using the inverse quantized transform coefficient. An inverse quantization transformation means for generating an image;
Adding means for generating a decoded prediction residual image based on the addition of the residual compensation image and the decoded compensation residual image;
Motion compensated prediction means for generating the predicted image by performing motion compensated prediction using a reference image and the motion information;
Decoded image generating means for generating a decoded image based on the addition of the decoded prediction residual image and the predicted image;
Reference image storage means for storing the decoded image generated by the decoded image generation means as the reference image provided to the motion compensated prediction by the motion compensated prediction means;
Function as
The residual decoding means includes
Inverse quantization means for generating an inverse quantization compensation coefficient by inverse quantization of the quantization compensation coefficient included in the residual compensation information;
Using the two-dimensional basis specified by the basis information included in the residual compensation information among a plurality of two-dimensional basis, and the inverse quantization compensation coefficient inverse transform generated by the inverse quantization means, the predetermined quantization Inverse transformation means for generating the residual compensation image by performing inverse transformation based on a first inverse transformation rule ;
I have a,
Further causing the computer to function as decoded prediction residual image storage means for storing the decoded prediction residual image as a reference decoded prediction residual image;
The inverse transform unit performs the inverse transform using a partial image of the reference decoded prediction residual image stored in the decoded prediction residual image storage unit as the two-dimensional basis.
A moving picture decoding program characterized by the above.

Images