JP3759538B2 - Image signal processing method and image signal transmission apparatus - Google Patents

Description

でなる観測方程式の形で表わすことができる。但し、（１）式において、ｎは１つの補間対象画素ｙ_m を線形一次結合式によつて表わす際の空間的及び時間的に周辺の伝送画素数を表し（実施例の場合、１つの補間対象画素ｙ_m を３８タツプの線形一次結合モデルによつて表わすため、ｎ＝３８である）、ｍは１フレーム内に存在する補間対象画素数を表わす。
【００４４】
ここで（１）式に基づき、１フレームに対して１個の係数組を求めようとすると、（１）式から１フレームの補間対象画素数分の連立方程式を作ることになる。基本的には、この連立方程式を解いて係数を求めればよい。実施例では最小二乗法を用いてこの連立方程式を解く。
すなわち先ず、（１）式を残差行列Ｅを用いて次式、
【数２】

のように残差方程式の形に表現し直す。
【００４５】
ここで（２）式から各係数値ｗ_i の最確値を求めるためには、ｅ₁ ²＋ｅ₂ ²＋……＋ｅ_i-1 ²＋ｅ_i ² を最小にする条件、すなわち次式、
【数３】

となるｎ個の条件を入れてこれを満足するｗ₁ 、ｗ₂ 、……、ｗ_n を見つければよい。
ここで（２）式より、次式、
【数４】

を得、（３）式の条件をｉ＝１、２、……、ｎについて立てればそれぞれ、次式
【数５】

が得られる。ここで（２）式及び（５）式から次式の正規方程式が得られる。
【数６】

【００４６】
ここで（６）式で表わされる正規方程式は未知数の数がｎ個の連立方程式であるから、これにより最確値である各係数ｗ_i を求めることができる。正確には（６）式でｗ_i に掛かる（Σｘ_jnｘ_jn）（但し、ｊ＝１……ｍ）のマトリクスが正則であれば解くことができる。実際には、Gauss-Jordanの消去法（掃き出し法）を用いて連立方程式を解く。
【００４７】
実施例の場合には、クラス分類回路４５によつて求めた各クラス毎に上述した最小二乗法を用いて係数値ｗ_i を求める。この結果各クラス毎に１フレームにつき１組の係数を伝送すればよいことになり、全ての補間対象画素についての係数値ｗ_i を求めて伝送する場合に比して、格段に伝送情報量及び演算量を低減し得る。実際に、係数の情報量はフレーム当りの画素情報量に比べて無視できるくらいのオーダーである。
【００４８】
具体的には、係数選定回路４４は、図６に示すように構成されている。すなわち係数選定回路４４は復号データＤ２０（ｘ_i ）及び入力画像データＤ１に含まれる補間対象画素データｙ_m を時系列変換メモリ９０に入力する。時系列変換メモリ９０は、図５について上述したように線形一次結合モデルを形成するための画素（ｘ₁ 〜ｘ_n 、ｙ_m ）を同時化して出力する。
時系列変換メモリ９０から出力されたデータは、正規化方程式生成回路９１に与えられ、当該正規化方程式生成回路９１によつて各クラス毎に（６）式で表わされるような正規化方程式が生成され、続くＣＰＵ演算回路９２によつて掃き出し法によつて各クラス毎の係数組が求められる。
【００４９】
正規化方程式生成回路９１は、先ず乗算器アレイ９３によつて各画素同士の乗算を行う。乗算器アレイ９３は、図７に示すように構成されており、四角で表わす各セル毎に画素同士の乗算を行い、これにより得た各乗算結果を続く加算器メモリ９４に与える。
【００５０】
加算器メモリ９４は、図８に示すように、乗算器アレイ９３と同様に配列されたセルでなる加算器アレイ９５とメモリ（またはレジスタ）アレイ９６Ａ₁ 、９６Ａ₂ 、……とにより構成されている。メモリアレイ９６Ａ₁ 、９６Ａ₂ 、……はクラス数分（実施例の場合、クラス分類回路４５により分類される２^n*k クラス分）設けられており、インデツクスデータＤ２１をデコードするインデツクスデコーダ９７の出力（クラス）に応答して一つのメモリアレイ９６Ａ₁ 、９６Ａ₂ 、……が選択され、選択されたメモリアレイ９６Ａ₁ 、９６Ａ₂ 、……の格納値が加算器アレイ９５に帰還される。このとき加算器アレイ９５によつて得られる加算結果が、再び対応するメモリアレイ９６Ａ₁ 、９６Ａ₂ 、……に格納される。
【００５１】
このようにして乗算器アレイ９３、加算器アレイ９５及びメモリアレイ９６Ａ₁ によつて積和演算が行われ、インデツクスによつて決定されるクラス毎にメモリアレイ９６Ａ₁ 、９６Ａ₂ 、……のいずれかが選択されて、積和演算の結果によつてメモリアレイ９６Ａ₁ 、９６Ａ₂ 、……の内容が更新される。
【００５２】
なお、各々のアレイの位置は、（６）式で表わされる正規化方程式のｗ_i にかかる（Σｘ_jnｘ_jn）（但し、ｊ＝１……ｍ）の位置に対応する。（６）式の正規化方程式を見れば明らかなように右上の項を反転すれば左下と同じものになるため、各アレイは三角形の形状をしている。
【００５３】
このようにして、１フレームの間にクラス毎に積和演算が行われて各クラス毎の正規化方程式が生成される。クラス毎の正規化方程式の各項の結果は、それぞれのクラスに対応するメモリアレイ９６Ａ₁ 、９６Ａ₂ 、……に記憶されており、次にそれらのクラス毎の正規化方程式の各項が掃き出し法演算を実現するＣＰＵ演算回路９２によつて計算される。この結果各クラス毎の係数ｗ_i の組が求められ、これが係数データＤ４として送出される。
【００５４】
補間データ作成回路６３は、図９に示すように、１フレーム毎に各クラスの係数組を記憶する係数メモリ１００を有し、この係数メモリ１００は各クラス毎の係数組ｗ₁ 〜ｗ_n を格納すると共に、インデツクスデコーダ１０１の出力（クラス）に応答してクラスに応じた係数組ｗ₁ 〜ｗ_n を出力する。この係数組ｗ₁ 〜ｗ_n がそれぞれレジスタ１０２Ａ₁ 〜１０２Ａ_n を介して乗算器１０３Ａ₁ 〜１０３Ａ_n に与えられる。また乗算器１０３Ａ₁ 〜１０３Ａ_n には、時系列変換回路１０４により選択された復号データｘ₁ 〜ｘ_n が与えられる。従つて乗算器１０３Ａ₁ 〜１０３Ａ_n の出力が加算回路１０５により加算されることにより、補間対象画素の画素値ｙ（＝ｘ₁ ｗ₁ ＋ｘ₂ ｗ₂ ＋……＋ｘ_n ｗ_n ）が得られる。
【００５５】
（４）実施例の動作
以上の構成において、画像符号化装置４０は入力画像データＤ１に対して間引き処理を施すことによりデータ量を削減した後、圧縮符号化データＤ２を生成する。また画像符号化装置４０は、クラス分類回路４５において間引かれた画素を注目画素とし、当該注目画素の周辺の画素の状態に応じて各間引かれた画素をクラス分類し、次に係数選定回路４４において各クラス毎の係数組ｗ₁ 〜ｗ_n を求め、これを係数データＤ４として出力する。
【００５６】
ここでクラス分類回路４５は、先ず間引かれた画素の画素値をその両隣の間引かれずに残つた画素の平均により求め、これにより得た平均補間値を含む画像に対してエツジ強調処理を施す。この結果平均補間により失われたエツジ情報やテクスチヤ情報が復元される。そしてクラス分類回路４５はこのようにエツジ情報やテクスチヤ情報が復元された画像に基づき、間引かれた画素（すなわち平均補間後エツジ強調処理がなされた補間対象画素）を含めたその周辺の画素の状態に応じて各間引かれた画素をクラス分類する。
この結果特にエツジ部分やテクスチヤ部分において、間引かれた画素を正確にクラス分類できる。
【００５７】
画像復号化装置６０では、圧縮符号化データＤ２を復号した復号データＤ３０及び係数データＤ４に基づいて復元画像データＤ３３を生成する。
このときクラス分類回路６２において、画像符号化装置４０のクラス分類回路４５でしたのと同様のクラス分類処理を施すことにより各間引かれた画素をクラス分類する。次に補間データ作成回路６３において、クラス分類結果Ｄ３１に応じて係数データＤ４から選択した係数組ｗ₁ 〜ｗ_n と復号データＤ３０とを線形一次結合させることにより補間データＤ３２を作成する。
【００５８】
かくして補間対象画素（間引かれた画素）をその周辺の画像の特徴に応じて的確にクラス分類したことにより、一段と真値に近い補間データＤ３２を得ることができ、間引きに基づく画質劣化の少ない復元画像を得ることができる。
【００５９】
（５）実施例の効果
以上の構成によれば、間引かれずに残つた画素を用いて補間対象画素に対応する補間画素値を生成し、この補間画素値と間引かれずに残つた画素とでなる画像に対してエツジ強調処理を施した後、補間対象画素と当該補間画素周辺に位置する伝送画素の状態に応じて当該補間画素をクラス分類するようにしたことにより、特にエツジやテクスチヤ部分において正確なクラス分けができる。かくして復元画像におけるエツジやテクスチヤ部分での画質を格段に向上させることができる。
【００６０】
（６）他の実施例
なお上述の実施例のクラス分類回路４５及び６２においては、平均補間値生成回路５１によつて間引かれた画素の画素値を水平ライン方向の平均値補間により求めた場合について述べたが、本発明はこれに限らず、例えば２次元の適応補間等により間引かれた画素の補間画素値を求めるようにしてもよく、エツジ強調前の間引かれた画素の補間方法としては種々のものを適用できる。
【００６１】
また上述の実施例のクラス分類回路４５及び６２においては、ＡＤＲＣエンコーダ５５を設け、エツジ強度後の画像をＡＤＲＣ符号化し、これにより得たビツト数が圧縮されたコードのパターンに基づいて間引かれた画素をクラス分類する場合について述べたが、ＡＤＲＣエンコーダ５５に代えて、例えばＤＣＴ（Discrete Cosine Transform)変換符号化、ＤＰＣＭ（差分量子化）、ベクトル量子化、サブバンド符号化、ウエーブレツト変換等の他の圧縮機能を有する回路を設けて、その出力コードのパターンに基づいて間引かれた画素をクラス分類するようにしてもよい。さらに圧縮したコードのパターンに基づいてクラス分類する場合に限らず、エツジ強調後の画像の相関度の強い方向に応じて間引かれた画素をクラス分類してもよく、すなわちエツジ強調後のクラス分類としては種々の手法を適用できる。
【００６２】
また上述の実施例においては、間引き後に残つた伝送画素を圧縮エンコーダ４２により圧縮符号化して伝送する場合について述べたが、本発明はこれに限らず、圧縮符号化せずに間引き後に残つた間引きデータをそのまま伝送する場合にも適用することができる。この場合には、画像符号化装置４０側で圧縮エンコーダ４２及びローカルデコーダ４３を省略すると共に、画像復号化装置６０側で圧縮デコーダ６１を省略すればよい。
【００６３】
また上述の実施例においては、圧縮エンコーダ４２としてＡＤＲＣ回路を用いた場合について述べたが、本発明はこれに限らず、圧縮エンコーダ４２として例えばＤＣＴ変換符号化、ＤＰＣＭ、ベクトル量子化、サブバンド符号化、ウエーブレツト変換等の圧縮手法を用いた場合にも上述の実施例と同様の効果を得ることができる。
【００６４】
また上述の実施例においては、サブサンプリング回路４１によつて入力画像データＤ１に対して１／２の間引き処理を行う場合について述べたが、本発明はこれに限らず、例えば１／４の間引き処理等、他の間引き処理を行つた場合でも実施例と同様の効果を得ることができる。
【００６５】
また上述の実施例においては、注目画素とこの注目画素の空間的及び又は時間的に周辺の伝送画素との相関関係を表すパラメータとして、線形一次結合モデルを最小二乗法により解いて得た係数データＤ４を伝送する場合について述べたが、本発明はこれに限らず、パラメータとしては例えば各クラス毎に求めた注目画素周辺の伝送画素値の平均値を代表値として伝送するようにしても良い。この場合平均演算に重心法を用いるようにすれば、クラス毎に誤差の少ない代表値を容易に求めることができる。またこれに限らず、要は伝送されない注目画素と伝送画素との相関関係を表すような種々のパラメータを用いることができる。
【００６６】
また上述の実施例においては、圧縮符号化データＤ２と共に係数データＤ４を伝送する画像符号化装置４０のクラス分類回路４５及びその画像復号化装置６０のクラス分類回路６２に本発明を適用した場合について述べたが、本発明はこれに限らず、例えば図１１に示すように、各クラス毎の予測係数Ｄ１３を発生するメモリ３６を有する画像復号化装置（受信器）３０のクラス分類手段（クラスタリング回路）３５にも本発明を適用できる。
【００６７】
この場合のメモリ３６には、予め学習によつて各クラス毎に獲得した予測係数を格納するようにすれば良い。一例としては、各クラス毎に立てた線形一次結合モデルの係数を、最小二乗法により求めるといつた処理を様々な画像に対して行うことにより予測係数を求める手法がある。
【００６８】
さらには画像符号化装置（送信器）２０側にも実施例のような係数データを発生する係数選定回路４４（図１）を設け、予め学習により獲得したメモリ３６の内容を、新たに画像符号化装置から伝送されてくる係数データによつて更新するようにした画像信号伝送装置のクラス分類手段にも本発明を適用できる。
【００６９】
さらに図１１に示すメモリ３６に予測係数に代えて各クラス毎の予測値を予め格納するようにした画像復号化装置（受信器）３０のクラス分類手段（クラスタリング回路）３５にも本発明を適用できる。
【００７０】
この場合のメモリ３６に格納する予測値を求める第１の方法としては、加重平均を用いた学習方法がある。詳述すれば、補間対象画素の周辺の伝送画素を用いて、補間対象画素に対してクラス分類を行い、クラス毎に積算した補間対象画素の画素値を補間対象画素の個数によつてインクリメントされた度数によつて割ると言つた処理を様々な画像に対して行うことにより予測値を求めることができる。
【００７１】
またメモリ３６に格納する予測値を求める第２の方法としては、正規化による学習方法がある。詳述すれば、補間対象画素を含む複数の画素からなるブロツクを形成し、このブロツク内のダイナミツクレンジによつて、補間対象画素の画素値からそのブロツクの基準値を減算した値を正規化し、この正規化された値の累積値を累積度数で除した値を予測値とする処理を様々な画像に対して行うことにより予測値を求めることができる。
【００７２】
このように本発明によるクラス分類手法は、クラス分類を施し各クラス毎に求めた予測係数又は予測値を使つて補間画素値を生成する場合に広く適用することができる。
【００７３】
【発明の効果】
上述のように本発明によれば、入力画像の所定画素をサンプリングすることにより画素数の低減した伝送画像を生成するサンプリング手段と、伝送画像における注目する注目位置周辺の複数の画素値を用いた補間処理により、当該注目位置の画素に対応する仮補間画素値を生成する仮補間画素値生成手段と、仮補間画素値及び伝送画像の画素値でなる画像に対してエツジ強調処理を施すエツジ強調手段と、エツジ強調後の画像について、注目位置の画素と当該注目位置の周辺に位置する複数の画素との状態に応じて各注目位置をクラス分類するクラス分類手段と、クラス分類手段により分類されたクラス毎に、注目位置の画素値と伝送画像において当該注目位置の空間的及び又は時間的に周辺に位置する複数の画素の画素値とにより線形一次結合モデル方程式を立て、当該線形一次結合モデル方程式を解いて得た係数をパラメータとして生成し、当該パラメータをメモリ手段に一時記憶させるパラメータ生成手段と、伝送画像の画素値及びメモリ手段に一時記憶しているパラメータを伝送することにより、当該画素値及び当該パラメータを受信した画像信号処理装置に対して、注目位置の画素と当該注目位置の周辺に位置する複数の画素との状態に応じて当該注目位置をクラス分類させた上で、分類されたクラスに応じたパラメータと上記伝送画像の画素値との線形一次結合式の演算により注目位置の本補間画素値を算出させ、入力画像における注目位置に当該本補間画素値を配列させて復元画像を生成させる伝送手段とを設けたことにより、注目位置の画素をクラス分類する場合に、エツジ情報やテクスチヤ情報を考慮した、より的確なクラス分類ができる。この結果、復元画像の画質を向上させることができる。
【図面の簡単な説明】
【図１】実施例の画像符号化装置の全体構成を示すブロツク図である。
【図２】実施例の画像復号化装置の全体構成を示すブロツク図である。
【図３】実施例によるクラス分類回路の構成を示すブロツク図である。
【図４】エツジ強調フイルタのフイルタ係数を示す略線図である。
【図５】サブサンプリング回路による間引き処理の説明、並びに線形一次結合モデルに用いる画素及び係数の説明に供する略線図である。
【図６】係数選定回路の構成を示すブロツク図である。
【図７】乗算器アレイの構成を示す略線図である。
【図８】加算器メモリの構成を示す略線図である。
【図９】補間データ作成回路の構成を示すブロツク図である。
【図１０】従来のデイジタルデータ変換装置の構成を示すブロツク図である。
【図１１】受信器側に予測係数又は予測値を格納したメモリを有するデイジタルデータ変換装置の構成を示すブロツク図である。
【符号の説明】
１、２０……送信器、１０、３０……受信器、３５……クラスタリング回路、３６……メモリ、３７……補間データ作成回路、４０……画像符号化装置、４１……サブサンプリング回路、４２……圧縮エンコーダ、４３……ローカルデコーダ、４５、６２……クラス分類回路、４４……係数選定回路、５１……平均補間値生成回路、５３……エツジ強調フイルタ、５５……ＡＤＲＣエンコーダ、６３……補間データ推定回路、６０……画像復号化装置、Ｄ１……入力画像データ、Ｄ２……圧縮符号化データ、Ｄ４……係数データ、Ｄ２１、Ｄ３１……インデツクスデータ、Ｄ３２……補間データ、Ｄ３３……復元画像データ、Ｗ₁ 〜Ｗ₃₈……係数。[0001]
【table of contents】
The present invention will be described in the following order.
Industrial application fields
Conventional technology (FIGS. 10 and 11)
Problems to be Solved by the Invention (FIGS. 10 and 11)
Means for Solving the Problems (FIGS. 1 to 3 and FIG. 11)
Action (Figs. 1-3)
Example
(1) Overall configuration (FIGS. 1 and 2)
(2) Class classification circuit (Fig. 3)
(3) Detailed configuration (FIGS. 4 to 9)
(4) Operation of the embodiment (FIGS. 1 to 3)
(5) Effects of the embodiment (FIG. 3)
(6) Other embodiment (FIG. 11)
The invention's effect
[0002]
[Industrial application fields]
The present invention relates to an image signal processing method and an image signal transmission apparatus, and is particularly suitable for application to an image signal transmission apparatus that reduces and transmits the amount of information of original image data by thinning processing.
[0003]
[Prior art]
Conventionally, in a so-called image signal transmission system that transmits an image signal to a remote place, such as a video conference system, or in an apparatus that digitizes an image signal and records and reproduces it on a video tape recorder or a video disk recorder, In order to efficiently use recording media, the amount of transmitted information and recorded information is reduced by efficiently encoding significant information using the correlation of digitized image signals, and the transmission efficiency and recording efficiency are increased. Has been made.
[0004]
Specifically, a technique for reducing the amount of data to be transmitted by performing high-efficiency compression coding on image data is widely used. In this high-efficiency coding, in general, the original image data is transmitted by performing sub-sampling in the space or space and reducing the number of pixels, and then performing compression coding processing. The amount of information has been further reduced. An example of spatiotemporal subsampling is the MUSE (Multiple Sampling Encode) method.
[0005]
On the other hand, on the receiving side, the resolution is increased by obtaining by interpolation the pixels that are not transmitted, that is, the thinned pixels. In general, interpolation is performed using a fixed tap and a fixed coefficient filter.
However, in such an interpolation method, the interpolated pixel value is obtained by averaging pixel values of a plurality of pixels adjacent to the interpolated pixel. Therefore, depending on the type of image, image blurring and spatio-temporal fluctuation (that is, jerkiness) , Edge business, etc.) occur, and as a result, there is a problem in that the restored image is deteriorated in image quality.
[0006]
As one method for solving this problem, a digital data conversion device as described in Japanese Patent Application No. 5-201913 has been proposed. As shown in FIG. 10A, the transmitter 1 in this digital data converter generates compressed encoded data D2 by passing the input image data D1 through the sub-sampling circuit 2 and the encoder 3, and outputs the compressed encoded data D2 to the output terminal 4 as shown in FIG. To give.
[0007]
The transmitter 1 decodes the compressed encoded data D2 by the local decoder 5 and then supplies the compressed encoded data D2 to the least squares operation circuit 6. The least squares method arithmetic circuit 6 receives the decoded data D3 and the input image data D1, and uses the pixels thinned out by sub-sampling (removed pixels) as the target pixel, and the target pixel included in the input image data D1. A linear linear combination model is established with the pixel values and the pixel values of the surrounding pixels included in the decoded data D3, and the coefficients of the linear linear combination model are obtained by performing an operation of the least square method. As a result, the least square method arithmetic circuit 6 outputs coefficient data D4 corresponding to the pixels thinned out by the sub-sampling circuit 2, and the coefficient data D4 is given to the output terminal 7.
[0008]
Thus, in the transmitter 1, together with the compression encoded data D2 obtained by compressing and encoding the image data after sub-sampling, coefficient data D4 representing the correlation between the thinned pixels and the surrounding transmission pixels is obtained. It is designed to transmit. .
[0009]
The receiver 10 of the digital data converter is configured as shown in FIG. 10B, and inputs the compression encoded data D2 to the decoder 12 through the input terminal 11. The decoded data D5 decoded by the decoder 12 is supplied to the time series conversion circuit 13 and the interpolation calculation circuit 14. The receiver 10 inputs the coefficient data D4 to the interpolation calculation circuit 14 via the input terminal 15.
[0010]
The interpolation calculation circuit 14 uses the decoded data D5 and the coefficient data D4 to obtain the interpolation data D6 from the linear linear combination formula, and sends this to the time series conversion circuit 13. The time series conversion circuit 13 arranges the decoded data D5 and the interpolation data D6 in the same manner as the original image (that is, the input image data D1), and obtains restored image data D7.
[0011]
Thus, in the digital data conversion apparatus constituted by the transmitter 1 and the receiver 10, the transmitter 1 side does not transmit only the compression-encoded data D2 obtained by high-efficiency encoding, but is thinned out. Coefficient data D4 representing the correlation between the pixel and its surrounding transmission pixels is transmitted together, and on the receiver 10 side, thinned pixels that are not actually transmitted are generated using the coefficient data D4. Thus, even when the amount of transmission information is reduced by thinning, image quality deterioration on the receiver 10 side can be reduced.
[0012]
In addition, in Japanese Patent Application No. 5-201913, in addition to this, the transmitter 1 side classifies the interpolation target pixel according to the distribution state of the transmission pixels around the interpolation target pixel (target pixel), A method has been proposed in which the coefficient data D4 described above is obtained by the least square method arithmetic circuit 6 every time. According to this method, the image quality of the restored image at the receiver 10 can be improved by improving the accuracy of the coefficient data D4. In this case, the same classification is performed on the receiver side as on the transmitter side, and coefficient data for the class is selected to obtain an interpolated pixel value.
[0013]
As another method, as disclosed in Japanese Patent Laid-Open No. 5-328185, a transmission that has been subjected to thinning processing is provided on the receiving side in which a memory for storing a prediction coefficient or a prediction value obtained in advance by learning is provided. By generating an interpolated pixel value based on a prediction coefficient read from the memory according to image data, or by using a predicted value read from the memory according to transmission image data as an interpolated pixel value, image quality degradation based on thinning is reduced. Digital data conversion devices that reduce the number have been proposed.
[0014]
As shown in FIG. 11, in the digital data conversion apparatus, in the transmitter 20, the compression encoded data D10 obtained through the sub-sampling circuit 21, the encoder 22 and the transmission processing circuit 23 is transferred to the transmission line 25 via the output terminal 24. Send it out.
The receiver 30 obtains the decoded data D11 by passing the compressed encoded data D10 inputted through the input terminal 31 through the reception processing circuit 32 and the decoder 33, and sends it to the synchronization circuit 34. The synchronization circuit 34 uses the thinned pixels as the target pixel, and synchronizes a plurality of peripheral transmission pixels for each target pixel and sends the same to the clustering circuit 35.
[0015]
The clustering circuit 35 classifies the input transmission image data according to the gradation and pattern, and sends the class classification result to the memory 36 as class information D12 representing the class of each pixel of interest.
The memory 36 outputs the prediction coefficient D13 corresponding to the input class information D12 among the prediction coefficients previously obtained and stored for each class by learning using the class information D12 as an address.
[0016]
The interpolation data creation circuit 37 creates the interpolation pixel data D14 by performing an operation by linear linear combination using the prediction coefficient D13 and the peripheral pixel value synchronized. The interpolated pixel data D14 and the decoded data D11 are combined by a subsequent combining circuit 38, whereby restored image data D15 is generated.
[0017]
Thus, according to the receiver 30, it is possible to restore high-frequency components that do not exist in the transmission image data D10, and as a result, it is possible to suppress deterioration in image quality based on the thinning-out process and obtain a high-resolution restored image. it can.
[0018]
[Problems to be solved by the invention]
By the way, as described above, the interpolation target pixel is classified into classes using transmission pixels around the interpolation target pixel, and a prediction coefficient or a prediction value is obtained for each class and transmitted (Japanese Patent Application No. 5-201913). ), Or the prediction coefficient or the prediction value is obtained in advance by learning and stored in the memory on the receiving side, and the receiving side classifies the interpolation target pixel and outputs the prediction coefficient or the prediction value corresponding to the class. Thus, in the device for obtaining the interpolation target pixel value (Japanese Patent Laid-Open No. 5-328185), the accuracy of the prediction coefficient or the prediction value can be improved as the number of classes is increased, thereby improving the image quality of the restored image.
[0019]
However, if the number of classes is increased, the amount of calculation for obtaining a prediction coefficient or predicted value increases, and the amount of data that must be stored in the memory 36 in the receiver 30 (FIG. 11) increases. It was practically impossible to take an infinite number, and it could not be said that there was sufficient classification.
As a result, there is a problem that image quality deterioration due to insufficient class classification occurs in a portion where sufficient resolution is required, such as an edge or texture, and sufficient resolution cannot be obtained.
[0020]
The present invention has been made in consideration of the above points, and an image signal processing method and an image that can improve the image quality of a restored image by performing a more accurate class classification process when classifying a pixel to be interpolated. A signal transmission device is proposed.
[0021]
[Means for Solving the Problems]
  In order to solve such a problem, in the present invention, a pixel at a target position of interest in an input image composed of a plurality of pixels is classified into classes according to the state of surrounding pixels, and processing is performed for each class. A provisional interpolation step for generating a provisional interpolation pixel value corresponding to a pixel at the target position by interpolation processing using a plurality of pixel values around the target position, An edge enhancement step for performing edge enhancement processing on an image composed of a provisional interpolation pixel value and a pixel value of an input image, and a plurality of pixels positioned around the target position in the target position pixel and the input image with respect to the image after edge enhancement. Classifying step for classifying each target position according to the state of the target pixel, and coefficients of a linear linear combination model using pixel values of a plurality of pixels located around the target position A value reading step for reading a value corresponding to the class from a memory means storing a parameter value corresponding to the class classified by the class classification step from a memory means stored as a parameter for each class, and a parameter corresponding to the class A main interpolation step for generating a main interpolation pixel value corresponding to the target position by processing using a calculation of a linear linear combination of the value and the pixel values of a plurality of pixels located around the target position. .
[0022]
  In the present invention, sampling means for generating a transmission image with a reduced number of pixels by sampling predetermined pixels of the input image, and interpolation processing using a plurality of pixel values around the target position of interest in the transmission image, Temporary interpolation pixel value generation means for generating a temporary interpolation pixel value corresponding to the pixel at the target position, edge enhancement means for performing edge enhancement processing on an image composed of the temporary interpolation pixel value and the pixel value of the transmission image, and an edge For the image after enhancement, class classification means for classifying each attention position according to the state of the pixel at the attention position and a plurality of pixels located around the attention position, and for each class classified by the class classification means A linear linear combination mode based on the pixel value of the target position and the pixel values of a plurality of pixels located in the periphery of the target position in terms of space and / or time in the transmission image. A coefficient equation obtained by solving the linear linear coupled model equation as a parameter, a parameter generation unit for temporarily storing the parameter in the memory unit, a pixel value of the transmission image, and a memory unit for temporarily storing the parameter The target position is transmitted to the image signal processing apparatus that has received the pixel value and the parameter according to the state of the target position pixel and a plurality of pixels located around the target position. Are classified into a class, and the interpolation pixel value of the target position is calculated by calculating a linear linear combination of the parameter corresponding to the classified class and the pixel value of the transmission image, and the target position in the input image is calculated. Transmission means for arranging the interpolated pixel values and generating a restored image is provided.
[0023]
  In the present invention, a temporary interpolation pixel value corresponding to the pixel at the target position is generated by interpolation using a plurality of pixel values around the target position of interest in the input image received from the predetermined image transmission source. Interpolation pixel value generation means, edge enhancement means for performing edge enhancement processing on an image made up of the provisional interpolation pixel value and the pixel value of the input image, and the pixel at the position of interest and the surroundings of the position of interest for the image after edge enhancement Class classification means for classifying each target position according to the state of a plurality of pixels located in the pixel, and a plurality of pixels located in the spatial and / or temporal vicinity of the target position in the pixel value of the target position and the input image Receives the parameters for each class obtained by solving the linear linearly coupled model equation for each class based on the pixel values of the input image together with the input image, and classifies the class by the class classification means. The interpolation pixel value generating means for generating the interpolation pixel value corresponding to the position of interest by applying a linear linear combination expression of the same parameter and the pixel value of the input image to the equation, and the attention of the input image A restored image generating means for generating a restored image based on the arrangement of the pixel value and the interpolation pixel value at the position is provided.
[0024]
  In the present invention, a temporary interpolation pixel that generates a temporary interpolation pixel value corresponding to a pixel at the target position by interpolation processing using a plurality of pixel values around the target position of interest in the input image received from the image transmission source. A value generating means, an edge emphasizing means for performing edge emphasis processing on an image composed of a provisional interpolation pixel value and a pixel value of an input image; Class classification means for classifying each target position according to the state of a plurality of pixels, and for each class, a pixel value of the target position and a spatial and / or temporal periphery of the target position in the input image Memory means for preliminarily storing, as parameters, coefficients of an equation obtained by solving a linear linear combination model established based on pixel values of a plurality of pixels, and a class classified by the class classification means. This interpolation pixel that generates a main interpolation pixel value corresponding to the position of interest by reading a parameter according to the memory from the memory means and applying a linear linear combination expression of the parameter and the pixel value of the input image to the equation Value generating means and restored image generating means for generating a restored image based on arranging pixel values and main interpolation pixel values at the target position of the input image are provided.
  Further, in the present invention, a temporary interpolation pixel that generates a temporary interpolation pixel value corresponding to a pixel at the target position by interpolation processing using a plurality of pixel values around the target position of interest in the input image received from the image transmission source. A value generating means, an edge emphasizing means for performing edge emphasis processing on an image composed of a provisional interpolation pixel value and a pixel value of an input image; Class classifying means for classifying each target position according to the state of a plurality of pixels, memory means for storing predicted values of pixel values corresponding to the target position for each class, and classes classified by the classifying means The prediction value corresponding to the value is read from the memory means, and the prediction value is set as the main interpolation pixel value corresponding to the target position, and the pixel value and the main interpolation pixel value are arranged at the target position of the input image. And the to be provided and the restored image generating means for generating a restoration image based.
[0025]
[Action]
  A temporary interpolation pixel value corresponding to the pixel at the target position is generated using the pixel of the input image, and edge enhancement processing is performed on the image composed of the temporary interpolation pixel value and the pixel value of the input image. The target position is classified into classes according to the state of the pixel and the pixels located around the target position. As a result, the edge classification process can be performed based on the image in which the edge information and the texture information are returned to a value close to the original image by the edge enhancement process, so that an accurate classification can be performed particularly in the edge and texture portions.
  Thus, the interpolated pixel value generated by the processing using the value obtained by calculating the linear linear combination formula based on the pixel value of the actually transmitted input image and the parameter corresponding to the class is used as a basis. By generating the restored image, the image quality at the edges and textures in the restored image can be significantly improved.
[0026]
【Example】
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0027]
(1) Overall configuration
In FIG. 1, reference numeral 40 denotes an image coding apparatus to which the present invention is applied as a whole. Input image data D1 is input to a sub-sampling circuit 41, and the amount of information is reduced by thinning out predetermined pixels of the input image data D1. After that, the data is sent to the subsequent compression encoder 42. The compression encoder 42 compresses and encodes the pixels remaining after the thinning to generate compression encoded data D2, and outputs this as transmission image data.
[0028]
The compressed encoded data D2 is decoded by the local decoder 43, and the decoded data D20 is given to the coefficient selection circuit 44 and the class classification circuit 45.
The class classification circuit 45 classifies each pixel of interest according to the distribution state of surrounding transmission pixels in terms of space and / or time of the pixel of interest, with the pixel to be interpolated (that is, the thinned pixel) as the pixel of interest. The classification result is sent to the coefficient selection circuit 44 as class information (hereinafter referred to as index data) D21 of the interpolation target pixel.
[0029]
Based on the input image data D1 and the decoded data D20 input through the delay circuit 46, the coefficient selection circuit 44 for each class, the pixel value of the target pixel included in the input image data D1 and the pixels of the surrounding decoded data D20. The correlation with the value is obtained by learning for each frame, and the learning result is output as coefficient data D4.
The delay circuit 46 delays the input image data D1 by the processing time required by the sub-sampling circuit 41, the compression encoder 42, and the local decoder 43.
[0030]
The transmission data sent from the image encoding device 40 is received and decoded by an image decoding device 60 having a configuration as shown in FIG. The image decoding device 60 decodes the compression encoded data D2 by the compression decoder 61, and gives the decoded data D30 obtained thereby to the class classification circuit 62.
The class classification circuit 62 has the same configuration as that of the class classification circuit 45 described above with reference to FIG. 1, and uses an interpolation target pixel that is not actually transmitted as a target pixel, and transmits the target pixel in terms of space and / or time. By classifying the target pixel according to the state of the pixel, the index data D31 of each target pixel is generated and sent to the interpolation data creation circuit 63.
[0031]
The interpolation data creation circuit 63 obtains interpolation data D32 based on the index data D31, decoded data D30, and coefficient data D4 of each interpolation target pixel. In practice, the interpolation data creation circuit 63 selects coefficient data D4 corresponding to the index data D31, and uses the selected coefficient data D4 and decoded data D30 around the pixel to be interpolated to establish a linear linear combination equation. Interpolation data D32 is obtained.
[0032]
The interpolation data D32 obtained in this way is sent to the time series conversion circuit 64. The time series conversion circuit 64 generates the restored image data D33 by arranging the decoded data D30 and the interpolation data D32 in the same manner as the original image (that is, the input image data D1).
Thus, the image decoding device 60 can obtain an interpolated pixel close to a true value, and thereby obtain a restored image with almost no image quality deterioration.
[0033]
(2) Classification circuit
The class classification circuits 45 and 62 are configured as shown in FIG. Here, since the class classification circuit 62 of the image decoding device 60 has the same configuration as the class classification circuit 45 of the image encoding device 40, the class classification circuit 45 will be described.
The class classification circuit 45 arranges the decoded data D20 in the line scan order by the time series conversion circuit 50, and sends it to the subsequent average interpolation value generation circuit 51.
[0034]
The average interpolation value generation circuit 51 obtains the image thinned out by the sub-sampling circuit 41 by average value interpolation in the horizontal line direction. That is, the average interpolation value generation circuit 51 supplies the input data to the addition circuit 51D via delay elements (D) 51A and 51B that respectively delay the input data by one pixel in the horizontal direction, and also supplies the input data to the addition circuit 51D as it is. As a result, the adder circuit 51D calculates pixel values of pixels thinned out from the sampling pixels indicated by “◯” in FIG. 5, for example.
[0035]
Incidentally, the addition circuit 51D has a multiplication function of multiplying the addition result by 1/2 in addition to the addition operation, and the pixels of the pixels thinned between them by the average operation of the two sampling pixels in the horizontal direction Find the value.
The output of the adding circuit 51D is given to the selector 51C through the delay element (D) 51E. The selector 51C alternately outputs sampling pixels and average interpolation pixels. The time series conversion circuit 52 rearranges the sampling pixels and the average interpolation pixels in the order suitable for the processing of the subsequent edge enhancement filter 53.
[0036]
The edge enhancement filter 53 enhances the edges in the vicinity of the average interpolation pixel by performing a two-dimensional edge enhancement process. As a result, the edge information and texture information lost by the average value interpolation are returned to values close to the original image. In the case of the embodiment, the filter coefficient of the edge emphasis filter 53 is selected as shown in FIG.
The output of the edge emphasis filter 53 is rearranged in a predetermined order by the time series conversion circuit 54 and then given to the ADRC encoder 55. As a result, assuming that the average interpolation pixel is the target pixel, the ADRC encoder 55 receives n peripheral pixels including the target pixel.
[0037]
The ADRC encoder 55 applies k-bit adaptive quantization to each of n input pixels, and arranges the codes obtained as a result in a predetermined order and stores them in the shift register 56. Thus, from the shift register 56, the index data D21 (or D31) (L₁~ L_m) Is output. Therefore, attention pixel is 2^{n * k}It is classified into one of the classes.
[0038]
As described above, in the class classification circuits 45 and 62, first, pixel values corresponding to the thinned pixels are obtained by average interpolation, and then edge emphasis processing is performed, and class classification processing is performed on the obtained pixels. By doing so, it is possible to classify the restored image in consideration of the edge portion or the texture portion near which the restored image is to be restored particularly clearly.
[0039]
(3) Detailed configuration
Next, detailed configurations of the sub-sampling circuit 41, the compression encoder 42, the coefficient selection circuit 44, and the interpolation data creation circuit 63 will be described.
[0040]
In this embodiment, as shown in FIG. 5, the sub-sampling circuit 41 performs a thinning process so that the number of transmission pixels is halved with respect to the frame image at each time point T0, T1, T2,. Apply. At this time, the sub-sampling circuit 41 selects a pixel position to be thinned out alternately so that pixels at the same position as the position sampled at the previous time point are not sampled at the next time point between successive time points. By performing spatial offset sub-sampling, the amount of transmitted information is reduced by half while leaving as much image features as possible.
5 represents the pixels remaining after thinning (that is, transmission pixels), and “+” represents the interpolation target pixels (thinned pixels). The lower w₁~ W₃₈Represents a coefficient of a linear linear combination model obtained by a coefficient selection circuit 44 described later.
[0041]
The compression encoder 42 is configured by an ADRC (Adaptive Dynamic Range Coding) circuit. For example, pixel data having an information amount of 8 bits per pixel is obtained by calculating a difference between the maximum pixel value and the minimum pixel value in a predetermined block. The amount of transmission information is effectively reduced by expressing the dynamic range information to be expressed, the minimum pixel information, and the difference between each pixel value and the minimum pixel value, for example, by quantization information obtained by performing 1-bit quantization.
[0042]
The coefficient selection circuit 44 represents an interpolation target pixel by a linear linear combination model of a transmission pixel and a coefficient around the interpolation target pixel, and the coefficient used at this time is calculated by a least square method for each class. Ask.
The principle of coefficient selection will be described. As shown in FIG. 5, the interpolation target pixel y_mIf the frame in which T is present is T1, the interpolation target pixel y is determined from the frame T1._mAnd a region around the same as the region cut out in the frame T1 from the frame T0 at the previous time point and the frame T2 at the next time point. Also, the interpolation target pixel y_mAre extracted from the input image data D1.
[0043]
Here, the coefficient w is first applied to each transmission pixel in the region cut out from the frames T0 to T2._iTo be interpolated pixel y_mIs represented by a linear linear combination of surrounding transmission pixels spatially and temporally. As a result, the interpolation target pixel y_mDeterminant Y and surrounding transmission pixels x_miThe determinant X is the coefficient w_iUsing the determinant W of
[Expression 1]

Can be expressed in the form of an observation equation. However, in the formula (1), n is one interpolation target pixel y_mRepresents the number of pixels to be transmitted spatially and temporally when represented by a linear linear combination formula (in the case of the embodiment, one interpolation target pixel y)._mIs represented by a 38-type linear linear combination model, so that n = 38), m represents the number of interpolation target pixels existing in one frame.
[0044]
Here, if one coefficient group is obtained for one frame based on the equation (1), simultaneous equations for the number of pixels to be interpolated in one frame are made from the equation (1). Basically, the coefficients can be obtained by solving the simultaneous equations. In the embodiment, the simultaneous equations are solved using a least square method.
That is, first, the equation (1) is expressed by the following equation using the residual matrix E:
[Expression 2]

Re-express it in the form of a residual equation like
[0045]
Here, each coefficient value w from equation (2)_iE to obtain the most probable value of₁ ²+ E₂ ²+ …… + e_i-1 ²+ E_i ²Which minimizes the following condition:
[Equation 3]

W satisfying this with n conditions₁, W₂, ..., w_nFind out.
Here, from the equation (2), the following equation:
[Expression 4]

If the conditions of equation (3) are set for i = 1, 2,..., N, respectively,
[Equation 5]

Is obtained. Here, the following normal equation is obtained from the equations (2) and (5).
[Formula 6]

[0046]
Here, since the normal equation represented by the equation (6) is a simultaneous equation with n unknowns, each coefficient w which is the most probable value is thereby obtained._iCan be requested. To be precise, w in equation (6)_i(Σx_jnx_jn) (Where j = 1... M) can be solved if the matrix is regular. Actually, simultaneous equations are solved using Gauss-Jordan elimination (sweeping method).
[0047]
In the case of the embodiment, the coefficient value w is calculated for each class obtained by the class classification circuit 45 using the least square method described above._iAsk for. As a result, it is only necessary to transmit one set of coefficients per frame for each class, and the coefficient values w for all interpolation target pixels._iThe amount of transmission information and the amount of computation can be significantly reduced as compared with the case where transmission is performed in response to the request. Actually, the information amount of the coefficient is on the order of negligible as compared with the pixel information amount per frame.
[0048]
Specifically, the coefficient selection circuit 44 is configured as shown in FIG. That is, the coefficient selection circuit 44 receives the decoded data D20 (x_i) And interpolation target pixel data y included in the input image data D1_mIs input to the time series conversion memory 90. As described above with reference to FIG. 5, the time series conversion memory 90 has pixels (x) for forming a linear linear combination model.₁~ X_n, Y_m) Are output at the same time.
The data output from the time series conversion memory 90 is given to the normalization equation generation circuit 91, and the normalization equation generation circuit 91 generates a normalization equation as expressed by equation (6) for each class. Subsequently, the CPU arithmetic circuit 92 obtains a coefficient set for each class by the sweep-out method.
[0049]
The normalization equation generation circuit 91 first multiplies each pixel by the multiplier array 93. The multiplier array 93 is configured as shown in FIG. 7, performs pixel multiplication for each cell represented by a square, and gives each multiplication result obtained thereby to the adder memory 94.
[0050]
As shown in FIG. 8, the adder memory 94 includes an adder array 95 and a memory (or register) array 96 </ b> A composed of cells arranged similarly to the multiplier array 93.₁96A₂, ... and. Memory array 96A₁96A₂,... Are the number of classes (in the case of the embodiment, 2 classified by the class classification circuit 45).^{n * k}One memory array 96A in response to the output (class) of the index decoder 97 which decodes the index data D21.₁96A₂,... Are selected and the selected memory array 96A₁96A₂,... Are fed back to the adder array 95. At this time, the addition result obtained by the adder array 95 becomes the corresponding memory array 96A again.₁96A₂, ... are stored.
[0051]
In this way, multiplier array 93, adder array 95, and memory array 96A.₁The product-sum operation is performed by the memory array 96A for each class determined by the index.₁96A₂,... Is selected and the memory array 96A is selected according to the result of the product-sum operation.₁96A₂, ... contents are updated.
[0052]
Note that the position of each array is represented by w in the normalization equation expressed by equation (6)._i(Σx_jnx_jn) (Where j = 1... M). As is clear from the normalization equation (6), if the upper right term is inverted, it becomes the same as the lower left, so each array has a triangular shape.
[0053]
In this way, a product-sum operation is performed for each class during one frame, and a normalization equation for each class is generated. The result of each term of the normalization equation for each class is the memory array 96A corresponding to each class.₁96A₂,..., And then each term of the normalization equation for each class is calculated by the CPU arithmetic circuit 92 that realizes the sweep-out method arithmetic. As a result, the coefficient w for each class_iAre obtained and transmitted as coefficient data D4.
[0054]
As shown in FIG. 9, the interpolation data creation circuit 63 has a coefficient memory 100 that stores coefficient sets of each class for each frame. The coefficient memory 100 is a coefficient set w for each class.₁~ W_nAnd a coefficient set w corresponding to the class in response to the output (class) of the index decoder 101.₁~ W_nIs output. This coefficient set w₁~ W_nAre respectively registers 102A.₁~ 102A_nThrough the multiplier 103A₁~ 103A_nGiven to. The multiplier 103A₁~ 103A_nIncludes the decoded data x selected by the time-series conversion circuit 104.₁~ X_nIs given. Therefore, the multiplier 103A₁~ 103A_nAre added by the adder circuit 105, so that the pixel value y (= x₁w₁+ X₂w₂+ …… + x_nw_n) Is obtained.
[0055]
(4) Operation of the embodiment
In the above configuration, the image encoding device 40 generates compressed encoded data D2 after reducing the amount of data by performing a thinning process on the input image data D1. Further, the image encoding device 40 sets the pixel thinned out in the class classification circuit 45 as a pixel of interest, classifies each thinned-out pixel according to the state of pixels around the pixel of interest, and then selects a coefficient. The coefficient set w for each class in the circuit 44₁~ W_nIs obtained and output as coefficient data D4.
[0056]
Here, the class classification circuit 45 first obtains the pixel value of the thinned pixel by the average of the pixels left without being thinned out on both sides thereof, and performs edge enhancement processing on the image including the average interpolation value obtained thereby. Apply. As a result, the edge information and texture information lost due to the average interpolation are restored. Based on the image in which the edge information and texture information are restored in this way, the class classification circuit 45 includes the pixels of the surrounding pixels including the thinned pixels (that is, the interpolation target pixels subjected to the edge enhancement process after the average interpolation). Each thinned pixel is classified into classes according to the state.
As a result, the thinned out pixels can be classified correctly in the edge portion and the texture portion.
[0057]
The image decoding device 60 generates restored image data D33 based on the decoded data D30 obtained by decoding the compressed encoded data D2 and the coefficient data D4.
At this time, the class classification circuit 62 performs class classification processing similar to that performed by the class classification circuit 45 of the image encoding device 40 to classify each thinned pixel. Next, in the interpolation data creation circuit 63, the coefficient set w selected from the coefficient data D4 according to the class classification result D31.₁~ W_nAnd the decoded data D30 are linearly linearly combined to create interpolation data D32.
[0058]
Thus, by accurately classifying the interpolation target pixels (thinned pixels) according to the characteristics of the surrounding images, it is possible to obtain interpolation data D32 that is closer to the true value, and has less image quality degradation due to thinning. A restored image can be obtained.
[0059]
(5) Effects of the embodiment
  According to the above configuration, an interpolation pixel value corresponding to a pixel to be interpolated is generated using pixels remaining without being thinned out, and an image is formed with respect to an image composed of the interpolation pixel values and pixels remaining without being thinned out. After performing the emphasis processing, the interpolation pixels are classified according to the states of the interpolation target pixels and the transmission pixels located around the interpolation pixels, so that accurate classification can be performed particularly in the edges and texture portions. . Thus, the image quality at the edges and textures in the restored image can be significantly improved.
[0060]
(6) Other embodiments
In the class classification circuits 45 and 62 of the above-described embodiment, the case where the pixel values of the pixels thinned out by the average interpolation value generation circuit 51 are obtained by average value interpolation in the horizontal line direction has been described. The invention is not limited to this. For example, an interpolated pixel value of a pixel thinned out by two-dimensional adaptive interpolation or the like may be obtained, and various methods for interpolating the thinned out pixel before edge enhancement may be used. Applicable.
[0061]
In the class classification circuits 45 and 62 of the above-described embodiment, the ADRC encoder 55 is provided, the image after the edge strength is ADRC encoded, and the number of bits obtained thereby is thinned out based on the compressed code pattern. However, instead of the ADRC encoder 55, DCT (Discrete Cosine Transform) transform coding, DPCM (differential quantization), vector quantization, subband coding, wavelet transform, etc. are described. A circuit having another compression function may be provided to classify the thinned out pixels based on the output code pattern. Furthermore, not only the classification based on the compressed code pattern, but also the pixels that are thinned out according to the direction of strong correlation of the image after edge enhancement may be classified, that is, the class after edge enhancement. Various methods can be applied as the classification.
[0062]
In the above-described embodiments, the case where the transmission pixels remaining after the thinning are compressed and transmitted by the compression encoder 42 has been described. However, the present invention is not limited to this, and the thinning remaining after the thinning without compression coding is performed. The present invention can also be applied when data is transmitted as it is. In this case, the compression encoder 42 and the local decoder 43 may be omitted on the image encoding device 40 side, and the compression decoder 61 may be omitted on the image decoding device 60 side.
[0063]
In the above-described embodiment, the case where the ADRC circuit is used as the compression encoder 42 has been described. However, the present invention is not limited to this, and the compression encoder 42 may include, for example, DCT transform coding, DPCM, vector quantization, and subband codes. Even when a compression method such as conversion or wavelet conversion is used, the same effect as in the above-described embodiment can be obtained.
[0064]
In the above-described embodiment, the case where the sub-sampling circuit 41 performs the half-thinning process on the input image data D1 has been described. However, the present invention is not limited to this, and for example, the quarter-thinning is performed. Even when other thinning-out processing such as processing is performed, the same effect as in the embodiment can be obtained.
[0065]
In the above-described embodiment, coefficient data obtained by solving a linear linear combination model by the least square method as a parameter representing the correlation between the target pixel and the spatially and / or temporally surrounding transmission pixel of the target pixel. Although the case where D4 is transmitted has been described, the present invention is not limited to this, and as a parameter, for example, an average value of transmission pixel values around the target pixel obtained for each class may be transmitted as a representative value. In this case, if the centroid method is used for the average calculation, a representative value with a small error can be easily obtained for each class. In addition, the present invention is not limited to this, and various parameters that indicate the correlation between the target pixel that is not transmitted and the transmission pixel can be used.
[0066]
In the above-described embodiment, the present invention is applied to the class classification circuit 45 of the image encoding device 40 and the class classification circuit 62 of the image decoding device 60 that transmit the coefficient data D4 together with the compression encoded data D2. As described above, the present invention is not limited to this. For example, as shown in FIG. 11, class classification means (clustering circuit) of an image decoding apparatus (receiver) 30 having a memory 36 that generates a prediction coefficient D13 for each class. ) 35 is also applicable to the present invention.
[0067]
In this case, the memory 36 may store the prediction coefficient acquired for each class by learning in advance. As an example, there is a method of obtaining a prediction coefficient by performing processing on various images when a coefficient of a linear linear combination model established for each class is obtained by a least square method.
[0068]
Further, a coefficient selection circuit 44 (FIG. 1) for generating coefficient data as in the embodiment is also provided on the image encoding device (transmitter) 20 side, and the contents of the memory 36 acquired by learning in advance are newly added to the image code. The present invention can also be applied to the class classification means of an image signal transmission apparatus that is updated with coefficient data transmitted from the conversion apparatus.
[0069]
Furthermore, the present invention is also applied to the class classification means (clustering circuit) 35 of the image decoding apparatus (receiver) 30 in which the prediction value for each class is stored in advance in the memory 36 shown in FIG. 11 instead of the prediction coefficient. it can.
[0070]
As a first method for obtaining the predicted value stored in the memory 36 in this case, there is a learning method using a weighted average. Specifically, classifying the interpolation target pixel using the transmission pixels around the interpolation target pixel, the pixel value of the interpolation target pixel integrated for each class is incremented by the number of interpolation target pixels. The predicted value can be obtained by performing the process of dividing by the frequency on various images.
[0071]
Further, as a second method for obtaining a predicted value stored in the memory 36, there is a learning method by normalization. More specifically, a block composed of a plurality of pixels including the interpolation target pixel is formed, and a value obtained by subtracting the reference value of the block from the pixel value of the interpolation target pixel is normalized by the dynamic range in the block. The predicted value can be obtained by performing a process on the various images using a value obtained by dividing the accumulated value of the normalized values by the accumulated frequency.
[0072]
As described above, the class classification method according to the present invention can be widely applied to the case where the interpolated pixel value is generated using the classification coefficient and the prediction coefficient or the prediction value obtained for each class.
[0073]
【The invention's effect】
  As described above, according to the present invention, sampling means for generating a transmission image with a reduced number of pixels by sampling predetermined pixels of the input image, and a plurality of pixel values around the target position of interest in the transmission image are used. Temporary interpolation pixel value generation means for generating a temporary interpolation pixel value corresponding to the pixel at the target position by interpolation processing, and edge enhancement processing for performing edge enhancement processing on an image composed of the temporary interpolation pixel value and the pixel value of the transmission image Classifying means for classifying each attention position according to the state of the pixel at the attention position and a plurality of pixels located around the attention position, and the class classification means. For each class, a linear primary is determined by the pixel value of the target position and the pixel values of a plurality of pixels located in the periphery of the target position in terms of space and / or time in the transmission image. Establish a combined model equation, generate a coefficient obtained by solving the linear linearly coupled model equation as a parameter, temporarily generate the parameter in the memory unit, and temporarily store the parameter in the transmission image pixel value and the memory unit Is transmitted to the image signal processing apparatus that has received the pixel value and the parameter according to the state of the pixel at the target position and a plurality of pixels located around the target position. After classifying the position, the interpolation pixel value of the target position is calculated by calculating a linear linear combination of the parameter corresponding to the classified class and the pixel value of the transmission image, and the target position in the input image is calculated. When classifying the pixel at the target position by providing a transmission means for generating a restored image by arranging the interpolated pixel values , In consideration of the edge information and texturized information, it is more accurate classification. As a result, the image quality of the restored image can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram showing the overall configuration of an image encoding apparatus according to an embodiment.
FIG. 2 is a block diagram showing an overall configuration of an image decoding apparatus according to an embodiment.
FIG. 3 is a block diagram showing a configuration of a class classification circuit according to an embodiment.
FIG. 4 is a schematic diagram illustrating a filter coefficient of an edge emphasis filter.
FIG. 5 is a schematic diagram for explaining a thinning process by a sub-sampling circuit and explaining pixels and coefficients used in a linear linear combination model.
FIG. 6 is a block diagram showing a configuration of a coefficient selection circuit.
FIG. 7 is a schematic diagram illustrating a configuration of a multiplier array.
FIG. 8 is a schematic diagram illustrating a configuration of an adder memory.
FIG. 9 is a block diagram showing a configuration of an interpolation data creation circuit.
FIG. 10 is a block diagram showing a configuration of a conventional digital data converter.
FIG. 11 is a block diagram showing the configuration of a digital data conversion apparatus having a memory storing prediction coefficients or prediction values on the receiver side.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1,20 ... Transmitter, 10, 30 ... Receiver, 35 ... Clustering circuit, 36 ... Memory, 37 ... Interpolation data creation circuit, 40 ... Image coding apparatus, 41 ... Sub-sampling circuit, 42... Compression encoder 43. Local decoder 45 and 62 Class classification circuit 44 Coefficient selection circuit 51 Average interpolation value generation circuit 53 Edge emphasis filter 55 55 ADRC encoder 63: Interpolation data estimation circuit, 60: Image decoding device, D1: Input image data, D2: Compression encoded data, D4: Coefficient data, D21, D31: Index data, D32: Interpolation Data, D33 ...... Restored image data, W₁~ W₃₈……coefficient.

Claims (13)

A pixel signal for obtaining a pixel value of the target position by classifying the pixel of the target position of interest of the input image composed of a plurality of pixels according to the state of the surrounding pixels and performing processing for each class A processing method,
A temporary interpolation step for generating a temporary interpolation pixel value corresponding to the pixel at the target position by an interpolation process using a plurality of pixel values around the target position;
An edge emphasizing step for performing edge emphasis processing on an image composed of the temporary interpolation pixel value and the pixel value of the input image;
A class classification step for classifying each target position according to the state of the pixel at the target position and a plurality of pixels positioned around the target position in the input image with respect to the image after edge enhancement;
Parameter values according to the class classified by the class classification step from the memory means in which the coefficients of the linear linear combination model using pixel values of a plurality of pixels located around the target position are stored as parameters for each class A value reading step of reading a value corresponding to the class from the memory means storing
A book that generates a main interpolation pixel value corresponding to the target position by processing using a linear primary combination expression of a parameter value corresponding to the class and pixel values of a plurality of pixels located around the target position. An image signal processing method comprising: an interpolation step. The above classification steps are:
ADRC (Adaptive Dynamic Range Coding) coding is performed on the pixel at the target position and a plurality of pixels located around the target position in the image after edge enhancement, and the target position is classified based on the bit code pattern obtained thereby. The image signal processing method according to claim 1, wherein classification is performed. Sampling means for generating a transmission image with a reduced number of pixels by sampling predetermined pixels of the input image;
Temporary interpolation pixel value generation means for generating a temporary interpolation pixel value corresponding to the pixel at the target position by interpolation processing using a plurality of pixel values around the target position of interest in the transmission image;
Edge emphasis means for performing edge emphasis processing on an image composed of the temporary interpolation pixel value and the pixel value of the transmission image;
Class classification means for classifying each target position according to the state of the pixel at the target position and a plurality of pixels located around the target position for the image after edge enhancement;
For each class classified by the class classifying means, a linear linear combination model based on the pixel value of the target position and the pixel values of a plurality of pixels located spatially and / or temporally around the target position in the transmission image. A parameter generating unit that sets an equation, generates a coefficient obtained by solving the linear linear coupled model equation as a parameter, and temporarily stores the parameter in a memory unit;
By transmitting the pixel value of the transmission image and the parameter temporarily stored in the memory means, the pixel of the target position and the pixel of the target position are transmitted to the image signal processing apparatus that has received the pixel value and the parameter. After classifying the target position according to the state of a plurality of pixels located in the vicinity, the target position is classified by a linear linear combination formula between the parameter corresponding to the classified class and the pixel value of the transmission image. An image signal transmission device comprising: transmission means for calculating a main interpolation pixel value at a target position and arranging the main interpolation pixel value at the target position in the input image to generate a restored image . The classification means is
ADRC encoding the pixel at the target position and a plurality of pixels located around the target position in the image after edge enhancement, and classifying each target position based on the bit code pattern obtained thereby. The image signal transmission apparatus according to claim 3. The parameter generation means includes
For each class classified by said classification means, linear combination model the linear and spatial and or pixel values of a plurality of pixels around temporally of the target position in the pixel value and the transmission image of the target position claims make a coefficient of linear combination model, the coefficients of the linear combination model obtained by obtained by calculation of the least squares method to produce as the parameter, the parameter, characterized in that is temporarily stored in the memory means 4. The image signal transmission device according to 3. Temporary interpolation pixel value generation means for generating a temporary interpolation pixel value corresponding to a pixel at the target position by interpolation processing using a plurality of pixel values around the target position of interest in the input image received from the image transmission source;
Edge emphasis means for performing edge emphasis processing on an image composed of the temporary interpolation pixel value and the pixel value of the input image;
Class classification means for classifying each target position according to the state of the pixel at the target position and a plurality of pixels located around the target position for the image after edge enhancement;
Solving a linear linear coupled model equation for each class based on the pixel value of the target position and the pixel values of a plurality of pixels located spatially and / or temporally around the target position in the input image The obtained parameters for each class are received together with the input image, and a linear linear combination formula between the parameters according to the class classified by the class classification unit and the pixel value of the input image is applied to the equation. The interpolation pixel value generating means for generating the interpolation pixel value corresponding to the target position by,
An image signal transmission apparatus comprising: restored image generation means for generating a restored image based on arranging a pixel value and the main interpolation pixel value at the target position of the input image. The classification means is
ADRC encoding the pixel at the target position and a plurality of pixels located around the target position in the image after edge enhancement, and classifying each target position based on the bit code pattern obtained thereby. The image signal transmission apparatus according to claim 6. The interpolation pixel value generating means is
The linear linear combination model of the pixel value of the target position and the pixel values of a plurality of pixels located in the vicinity of the target position spatially and / or temporally in the input image is minimized for each class. Receives the parameter consisting of the coefficient obtained by the calculation of the square method together with the input image, and calculates the linear linear combination expression of the parameter and the pixel value of the input image according to the class classified by the class classification means image signal transmission apparatus according to claim 6, characterized in that to produce the present interpolation pixel value by the applied to the linear combination model. Temporary interpolation pixel value generation means for generating a temporary interpolation pixel value corresponding to a pixel at the target position by interpolation processing using a plurality of pixel values around the target position of interest in the input image received from the image transmission source;
Edge emphasis means for performing edge emphasis processing on an image composed of the temporary interpolation pixel value and the pixel value of the input image;
Class classification means for classifying each target position according to the state of the pixel at the target position and a plurality of pixels located around the target position for the image after edge enhancement;
For each class , solve a linear linear combination model established based on the pixel value of the target position and the pixel values of a plurality of pixels located spatially and / or temporally around the target position in the input image. Memory means for preliminarily storing the coefficient of the obtained equation as a parameter;
The position of interest is obtained by reading the parameter according to the class classified by the class classification unit from the memory unit and calculating a linear linear combination expression of the parameter and the pixel value of the input image to the equation. Main interpolation pixel value generation means for generating the main interpolation pixel value corresponding to
An image signal transmission apparatus comprising: restored image generation means for generating a restored image based on arranging a pixel value and the main interpolation pixel value at the target position of the input image. The memory means includes
For each of the classes, linear combination model the linear first stood on the basis of the pixel values of a plurality of pixels positioned spatially and or near temporally of the target position in the pixel value and the input image of the target position order Store the coefficients obtained by solving the combined model by the least squares method as the above parameters,
The interpolation pixel value generating means is
The interpolation corresponding to the target position is performed by reading out the parameter according to the class classified by the class classification unit from the memory unit and applying the parameter and the pixel value of the input image to the linear linear combination model. The image signal transmission device according to claim 9, wherein a pixel value is generated. Temporary interpolation pixel value generation means for generating a temporary interpolation pixel value corresponding to a pixel at the target position by interpolation processing using a plurality of pixel values around the target position of interest in the input image received from the image transmission source;
Edge emphasis means for performing edge emphasis processing on an image composed of the temporary interpolation pixel value and the pixel value of the input image;
Class classification means for classifying each target position according to the state of the pixel at the target position and a plurality of pixels located around the target position for the image after edge enhancement;
Memory means for storing a predicted value of a pixel value corresponding to the target position for each class in advance;
The predicted value corresponding to the class classified by the class classifying unit is read from the memory unit, and the predicted value is set as a main interpolation pixel value corresponding to the target position. An image signal transmission device comprising: restored image generation means for generating a restored image based on arranging the interpolation pixel values. The memory means includes
The pixel value obtained for each class by learning using a weighted average calculation using the pixel value of the input image and the pixel value of the target position in advance is stored as the predicted value. Item 12. The image signal transmission device according to Item 11. The memory means includes
The pixel value obtained for each class by learning by normalization using the pixel value of the input image and the pixel value of the target position in advance is stored as the predicted value. The image signal transmission device according to 11.

Images