JP3615086B2 - Compressed data content determination method and compressed data content determination device - Google Patents

Description

【００２１】
【表２】

【００２２】
ＤＣ値、ＡＣ値、動きベクトル値検出器４で検出された動きベクトルは動き量検出器８に入力され、そのコンテンツの動き量を画素精度のベクトルの絶対値で検出する。動き量は、画面全体がパニングしている場合などは、たとえ静止している物体であっても動きを伴う結果になるので、画面全体が一定方向に動いていると判断された場合にはそのグローバルな動き量の分、差し引いて検出される。
【００２３】
このように検出された画像圧縮データの特徴量データは特徴量比較器９に入力され、画像特徴データベース２の様々なコンテンツの特徴データ（例えば表３）と比較される。そして、画像圧縮データの特徴量データと適合性の最も高いデータベース内の特徴量データが示すコンテンツが、その画像圧縮データの画像内容と判別され、画像判別器１０から画像内容の判別結果が出力される。
【００２４】
【表３】

【００２５】
図１における画像判別器１０に入力される付加情報は、後述する音声内容判別装置（図３）からの判断結果の情報である。これは、判断結果に曖昧な部分がある場合、外からの判断結果を参考情報として使用できるように考慮されたものである。もちろん、外からの付加情報は音声判断結果に限らず、他の結果情報を入力しても良い。また、なにも入力しなくてもかまわない。
【００２６】
このように、本実施例の画像内容判別装置は、画像圧縮データを完全に復号しなくても（ＶＬＣ復号器３による復号まででよく、その先の復号化工程である図１２に示す逆量子化器５２、逆ＤＣＴ器５３、加算器５４、画像メモリ５５、動き補償予測器５６による完全な復号化は不要）、圧縮データの復号途中に発生するコンテンツ内容を示す特徴的データと、予め統計的に求めておいた特徴データとを用いて特徴を検索することで、圧縮データの内容判別を可能としている。これにより、本画像内容判別装置は、専用の復号手段や、大きなメモリー、記録媒体が不要となり、低コスト化、小型化が図れる。
【００２７】
次に、音声圧縮データの内容判別を行う実施例（音声内容判別装置）を図２に示す。音声圧縮データはヘッダー検出器１１に送信される。音声圧縮データは例えばＭＰＥＧ方式のものとすると、サンプリング周波数、チャンネル数、ビットレート、コーディングモードなどが記述されている。ヘッダー検出器１１ではこれらヘッダーの値を検出する。この領域は固定長の構造をもっているので検出は所定のビットを単に読むだけで可能である。
【００２８】
このヘッダーの値は音声特徴データベース１２に送信される。音声特徴データベース１２では、これらのヘッダーの値で、多くのコンテンツの特徴を予め統計的にデーター化したものが使われる。
【００２９】
一方、ヘッダー検出器１１でヘッダーを検出された後、圧縮データはＶＬＣ復号器１３に送信される。ＶＬＣされていない場合、固定長のＦＬＣ復号する。ここでは、ＭＰＥＧなどの圧縮ストリームの可変長符号（ＶＬＣ）もしくは固定長符号（ＦＬＣ）を所定サンプル毎に復号して、そのデータをＤＣ値、ＡＣ値検出器１４に送信する。ＤＣ値、ＡＣ値検出器１４では復号したＶＬＣもしくはＦＬＣからＤＣ値、ＡＣ値を検出する。
【００３０】
ＤＣ値は音圧検出器１５に入力され、音の大きさが検出される。ＡＣ値は周波数特性検出器１６に入力され、固有の特性が検出される。音声の圧縮はサブバンド方式やＤＣＴをもちいるが、次数は画像のように少なくなく、３８４サンプルや１１５２サンプルと非常に多い。従って、細かい周波数特性が選られる。これらに対し数多くのコンテンツにおける典型的なパターンを決定できる。これらはのパターンを用いて、音声特徴データベースを作成しておく。
【００３１】
そして、特徴量比較器１７に音圧検出器１５と周波数特性検出器１６とから入力された音圧や周波数特性の特徴データは、音声特徴データベース１２から読み出されたデータベースと比較され、そのもっとも適合したコンテンツが判別され、その結果が音声判別器８から出力される。
【００３２】
図２における音声判別器１８に入力される付加情報は、後述する画像内容判別装置（図３）からの判断結果情報である。これは、音声圧縮データのみによる判断結果に曖昧な部分がある場合、外からの判断結果を参考情報として使用できるように考慮されたものである。もちろん、外からの付加情報は画像判断結果に限らず、他の結果情報を入力しても良い。また、なにも入力しなくてもかまわない。
【００３３】
次に、画像音声圧縮データの内容を判別する装置の実施例を図３に示す。入力される圧縮データは相関のある音声コンテンツと画像コンテンツとを圧縮して作成した音声圧縮データと画像圧縮データであり、一般に両者は多重化され、それぞれが再生される時間が同期するように、タイムスタンプが打たれている。これの例はＭＰＥＧにおける多重化方法、ＭＰＥＧシステムＩＳＯ−ＩＥＣ１１１７２−１やＩＴＵＨ．２２２．０／ＩＳＯ−ＩＥＣ１３８１８−１に詳細が記述されている。この画像音声圧縮データは画像音声分離器２１に入力され、分離される。分離された画像と音声の圧縮データは同期管理器２２において、それぞれのタイムスタンプで同期がとられ、画像データは前述した画像内容判別装置２３（図１に示したもの）に、音声データは音声内容判別装置２４（図２に示したもの）に、同期して入力される。
【００３４】
それぞれの判別装置では前述の方法で判別が行われるが、内容を判別する際に、画像圧縮データから検出する情報と、音声圧縮データから検出される情報の両方を用いて、より精度の高い判別を行えるように工夫されている。（図１における画像判別器１０に、図２における音声判別器１８から音声判別結果が入力され、その音声判別結果を参考にして画像判別をより精度よく行う。同様に、図２における音声判別器１８に、図１における画像判別器１０から画像判別結果が入力され、その画像判別結果を参考にして画像判別をより精度よく行う。）
双方の判別結果は最終判別器２５に出力される。即ち、「海」なのか「青い絨毯」なのか画像判別では曖昧なところを、音声判別で「海」の音に近いと判別された場合、内容最終判別結果は「海」と判別される。
【００３５】
最終判別器２５では、上述の工夫により、基本的には同じ判別結果が得られるが、双方の判断結果があまりにも違う場合、最終判別器２５では、どちらかを優先するように設定できる。これは入力される圧縮データの種類が、予め、どちらが判別しやすいかわかっているような場合に人為的に、精度を高めることが出来るからである。
【００３６】
また、圧縮データがプライベートな情報であり内容を判別されたくないデータである場合、図４に示すように圧縮データのヘッダーに判断拒否データを付加しておき、前述した内容判別装置では、この情報により判断を停止することが出来る。判断拒否データはヘッダーに記述していなくても、図５に示すように圧縮データに番号（ＩＤ）が付加されていて、図６に示すように圧縮データの番号（ＩＤ）とリンクした、判別拒否情報構造体が別データとして存在するようにしてもよい。判別拒否情報構造体に、例えば圧縮データの所定の単位ごとに、そのデータの内容判断を拒否する場合には１、拒否しない場合には０を記述しておく。
【００３７】
判断拒否データ情報を用いる場合の内容判別装置のブロック図を図７、図８に示す。図７では、まず、圧縮データを判別拒否データ検出器３１に入力する。判別拒否データ検出器３１では、上記判定拒否データを検出する。その結果、判断を拒否している場合には、スイッチ３２を「データを流さないモード」に切り替え、圧縮データが内容判別部３３に入力されない様にする。また、拒否していない場合には、スイッチ３２を「データを流すモード」に切り替え、圧縮データは内容判別部３３に入力される。内容判別部３３では圧縮データの内容が判断され、判断結果を出力する。内容判別部３３は図１に示すブロック１〜１０、図２に示すブロック１１〜１８、図３に示すブロック２１〜２５に相当する。
【００３８】
図８では、圧縮データはまず内容判別部３３に入力される。同時に、判断拒否データ検出器３１にも入力される。判別拒否データ検出器３１では、上記判定拒否データを検出する。その結果、判断を拒否している場合には、スイッチ３２を「データを流さないモード」に切り替え、内容判別部３３によって判断された結果を出力しないようにする。また、拒否していない場合には、スイッチ３２を「データを流すモード」に切り替え、内容判別部３３によって判断された結果を出力するようにする。
【００３９】
【発明の効果】
以上の通り、本発明によれば、圧縮データを完全に復号しなくても、圧縮データの復号化段階の途中で発生するコンテンツ内容を示す特徴的データと、予め統計的に求めておいた特徴データとを用いて特徴を検索することで、圧縮データの内容判別を可能としている。これにより、本発明は、専用の復号手段や、大きなメモリー、記録媒体を必要としない圧縮データ内容判別方法及び圧縮データ内容判別装置を提供できる。
【図面の簡単な説明】
【図１】実施例による画像内容判別装置のブロック図である。
【図２】実施例による音声内容判別装置のブロック図である。
【図３】実施例による画像音声内容判別装置のブロック図である。
【図４】判定拒否データの第１の記録例を示す説明図である。
【図５】判定拒否データの第２の記録例を示す説明図である。
【図６】判定拒否データの第２の記録例を示す説明図である。
【図７】判定拒否データを用いた実施例による内容判別装置のブロック図である。
【図８】判定拒否データを用いた実施例による内容判別装置のブロック図である。
【図９】ＡＣパターンの説明図である。
【図１０】ＡＣと輪郭パターンの関係を示した説明図である。
【図１１】従来例の符号化器のブロック図である。
【図１２】従来例の復号化器のブロック図である。
【符号の説明】
１ ヘッダー検出器
２ 画像特徴データベース
３ ＶＬＣ復号器
４ ＤＣ値、ＡＣ値、動きベクトル値検出器
５ 色検出器
６ 周波数帯域検出器
７ 輪郭検出器
８ 動き量検出器
９ 特徴量比較器
１０ 画像判別器[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method and apparatus for determining the contents of compressed data, and more particularly to a method and apparatus for determining the contents of compressed data without decoding all of the compressed data.
[0002]
[Prior art]
Conventionally, the contents of digital data are determined by detecting information such as the color, contour, frequency, and motion of the image from the image, and performing image recognition to infer and determine the contents of the image. Also, in speech, the frequency characteristics of speech data are analyzed using Fourier transform, and speech recognition is performed using the results.
As a conventional example of image recognition, there is JP-A-7-73317 and the like, and as a conventional example of speech recognition, there is JP-A-7-261778.
[0003]
Here, MPEG used for creating compressed data of image data will be described. MPEG is the name of an organization that examines video coding standards established in 1988 by ISO / IEC JTC1 / SC2 (International Organization for Standardization / International Electrotechnical Standards Meeting Technical Committee 1 / Technical Committee 2, current SC29) ( It is an abbreviation for “Moving Pictures Expert Group”. MPEG1 (MPEG Phase 1) is a standard for storage media of about 1.5 Mbps, and is intended for video compression for low transfer rates of JPEG and ISDN video conferences and videophones for the purpose of still image coding. H. It inherits the basic technology of H.261 (CCITT SGXV, standardized by the current ITU-T SG15), and introduces a new technology for storage media. These were established in August 1993 as ISO / IEC 11172.
MPEG2 (MPEG Phase 2) is a general-purpose standard designed to be compatible with various applications such as communication and broadcasting, in November 1994, ISO / IEC 13818, H.264. It is established as 262.
[0004]
MPEG is created by combining several technologies. FIG. 11 shows a block diagram of a conventional MPEG encoder.
The input image is subjected to a difference in the locally decoded image subjected to motion compensation prediction by the motion compensation predictor 41 and the difference unit 42, thereby reducing a time redundant portion. There are three modes of prediction from the past and the future. These can be switched and used for each 16 pixels × 16 pixels MB (macroblock). The prediction direction is determined by the picture type given to the input image. There are two modes in which a prediction from the past and two modes in which the MB is independently encoded without prediction are P pictures. In addition, the B picture has four modes of prediction from the future, prediction from the past, prediction from both, and independent encoding. It is the I picture that all MBs are encoded independently.
[0005]
In motion compensation, a motion vector is subjected to pattern matching for each MB, a motion vector is detected with half-pel accuracy, and prediction is performed after shifting by the amount of motion. The motion vector has a horizontal direction and a vertical direction, and is transmitted as additional information of the MB together with an MC (Motion Compensation) mode indicating where the prediction is from. A picture from an I picture to a picture before the next I picture is called a GOP (Group Of Picture), and about 15 pictures are generally used when used in a storage medium or the like.
[0006]
The difference image is subjected to orthogonal transformation in the DCT unit 43. DCT (Discrete Cosine Transform) is an orthogonal transform that discretely transforms an integral transformation with a cosine function as an integral kernel into a finite space. In MPEG, MB is divided into four and two-dimensional DCT is performed on 8 × 8 DCT blocks. In general, a video signal has many low-frequency components and few high-frequency components. Therefore, when DCT is performed, coefficients are concentrated in a low frequency.
[0007]
Image data (DCT coefficient) subjected to DCT is quantized by a quantizer 44. Quantization is a quantized value obtained by multiplying a value obtained by weighting the 8 × 8 two-dimensional frequency called the quantization matrix with a visual characteristic and a value called a quantization scale that is a scalar multiplication of the whole, and the DCT coefficient is quantized. Calculate by value. When inverse quantization is performed by the decoder, a value approximating the original DCT coefficient is obtained by multiplying by the quantized value.
[0008]
The quantized data is variable length encoded by the VLC unit 45. Of the quantized values, a direct current (DC) component uses DPCM (differential pulse code modulation) which is one of predictive coding. Also, the AC (AC) component is zigzag scanned from low to high, with zero run length and effective coefficient value as one event, and Huffman coding that assigns codes with a short code length to those with a high probability of appearance. Is done.
[0009]
The variable-length encoded data is stored in the temporary buffer 46 and output as encoded data at a predetermined transfer rate. The generated code amount for each macroblock of the output data is transmitted to a code amount controller (not shown), and the error code amount with respect to the generated code amount with respect to the target code amount is fed back to the quantizer 44. Thus, the amount of code is controlled by adjusting the quantization scale.
[0010]
The quantized image data is inversely quantized by an inverse quantizer 47, inversely DCTed by an inverse DCT unit 48, temporarily stored in an image memory 50 via an adder 49, and then a motion compensated predictor. In 41, it is used as a reference decoded image (local decoded image) for calculating a difference image.
The encoded data output from the buffer 46 is decoded by the reverse process of encoding. FIG. 12 shows a block diagram of a conventional MPEG decoder. A VLD unit (variable length code decoder) 51, an inverse quantizer 52, an inverse DCT unit 53, an adder 54, an image memory 55, and a motion compensated predictor 56 are included.
[0011]
[Problems to be solved by the invention]
In the case of determining the contents of the compressed data (encoded data), conventionally, the compressed data is completely decoded (a VLD unit (variable length code decoder) 51, an inverse quantizer 52 shown in FIG. 12, After decoding by the inverse DCT unit 53, the adder 54, the image memory 55, and the motion compensation predictor 56), the contents are determined by appropriate image recognition, voice recognition, and the like. Therefore, a dedicated decoding means is necessary for content determination. Also, since the decrypted data is decompressed, a considerably large memory or recording medium is required.
It is an object of the present invention to provide a compressed data content determination method and a compressed data content determination device that can determine the content of compressed data without completely decoding the compressed data.
[0012]
[Means for Solving the Problems]
Then, in order to solve the said subject, this invention provides the method and apparatus as described in following (1)-(6).
(1)A header detection step of detecting header information of the input image compressed data and extracting image size information and frame rate information from the detected header information;
For each predetermined block generated in the decoding process of the input image compressed data DC value, AC Image feature data detection step for detecting frequency band information and motion amount information, which is information that can specify the characteristics of the content of the input image content obtained based on at least one of the three information values and motion vectors When,
An image in which the frequency band information, the motion amount information, and the content content information, which are statistically obtained in advance for each combination of image size information and frame rate information parameters, are stored in a database as a set of feature amount data. From the database in the feature storage means, select feature amount data that can be determined to have the highest suitability using image size information, frame rate information, frequency band information, and motion amount information of the input image compressed data, and select the selected feature Image compressed data content determination comprising: an image compressed data content determination step for determining that the content content information of the quantity data is the content content obtained when the input image compressed data is decoded and reproduced Method.
(2)A header detection step of detecting header information of the input audio compression data and extracting sampling frequency information from the detected header information;
Every predetermined sample generated in the decoding process of the input voice compression data DC value, AC An audio feature data detection step for detecting frequency characteristic information, which is information that can specify the feature in the content of the content of the input audio obtained based on at least one of the two pieces of value information;
From the database in the audio feature storage means which stores the frequency characteristic information and the content content information statistically obtained in advance for each sampling frequency information as a set of feature data, the input audio compression data The feature amount data that can be determined to be the most suitable using the sampling frequency information and frequency characteristic information of the selected feature amount data is selected, and the content content information of the selected feature amount data is obtained when the input audio compression data is decoded and reproduced. An audio compression data content determination method, comprising: an audio compression data content determination step for determining content content.
(3) When determining the contents of the content from the compressed input image data and the compressed input audio data generated by compressing the content correlated with the image information and the audio information,
As described in (1) aboveimageOf the input image compressed data obtained by the compressed data content determination methodContent content informationAnd the above (2)voiceOf the input voice compression data obtained by the compressed data content determination methodContent content informationAnd twoContent content informationA method for determining the content of compressed data, wherein the content of the content is determined.
(4)Header detection means for detecting header information of the input image compressed data and extracting image size information and frame rate information from the detected header information;
For each predetermined block generated in the decoding process of the input image compressed data DC value, AC Image feature data detection means for detecting frequency band information and motion amount information, which is information that can specify the characteristics of the content of the content of the input image obtained based on at least one of the three pieces of information of value and motion vector When,
Image feature storage that stores frequency band information, motion amount information, and content content information statistically obtained beforehand for each combination of parameters of image size information and frame rate information as a set of feature amount data in a database Means,
Feature amount data that can be determined to have the highest suitability using the image size information, frame rate information, frequency band information, and motion amount information of the input image compressed data from the database stored in the image feature storage means And an image compressed data content determining means for determining that the content content information of the selected feature amount data is the content of the content obtained when the input image compressed data is decoded and reproduced. Image compression data content size Another device.
(5)Header detection means for detecting header information of the input audio compression data and extracting sampling frequency information from the detected header information;
Every predetermined sample generated in the decoding process of the input voice compression data DC value, AC Audio feature data detection means for detecting frequency characteristic information, which is information that can identify the characteristics of the content of the input audio content obtained based on at least one of the two pieces of value information;
Voice feature storage means for storing frequency characteristic information and content content information statistically obtained in advance for each sampling frequency information in a database as a set of feature amount data;
From the database stored in the voice feature storage means, select feature quantity data that can be determined to have the highest suitability using sampling frequency information and frequency characteristic information of the input voice compression data, and the selected feature quantity An audio compressed data content discriminating device comprising audio compressed data content discriminating means for discriminating that the content content information of the data is the content content obtained when the input audio compressed data is decoded and reproduced. .
(6) A compressed data content discriminating device for discriminating the content of content from compressed input image data and compressed audio data generated by compressing content correlated with image information and audio information,
As described in (4) aboveimageA compressed data content determination device;
As described in (5) abovevoiceA compressed data content determination device;
As described in (4) aboveimageOf the input image compressed data obtained by the compressed data content discriminating device.Content content informationAnd the above (5)voiceOf the input voice compression data obtained by the compressed data content discrimination deviceContent content informationAnd twoContent content informationA compressed data content discriminating apparatus, comprising: a content discriminating unit for discriminating the content of the content by using a computer.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
The present invention searches for features using characteristic data indicating the contents of content that occurs during decoding of compressed data and feature data that has been statistically obtained in advance, even if the compressed data is not completely decoded. Thus, it is possible to determine the contents of the compressed data. As a result, the present invention can provide a compressed data content determination method and a compressed data content determination device that do not require dedicated decoding means, a large memory, and a recording medium. In addition, by providing information that can be used to refuse content content determination, a compressed data content determination method and a compressed data content determination device that reject content content determination in accordance with the information can be provided. In this case, it is possible to give confidentiality to the compressed data.
[0014]
First, FIG. 1 shows an embodiment of the compressed data content discriminating apparatus of the present invention. This embodiment is an image content determination device that determines the content of compressed image data. The compressed image data is transmitted to the header detector 1. For example, if the compressed image data is of the MPEG format, there is an area called a sequence header at the beginning of the data, which describes the horizontal pixel size, vertical pixel size, aspect ratio, frame rate, bit rate, etc. Has been. The header detector 1 detects these header values. Since this area has a fixed length structure, detection can be performed simply by reading a predetermined bit.
[0015]
The image feature database 2 uses the values of headers of compressed image data in which many content features are statistically converted into data beforehand. That is, when the image size changes, the band of the image changes relatively. When the bit rate decreases, the compression rate increases correspondingly, and the DCT coefficient effective coefficient (non-zero value) is affected. Even if the bit rate is the same, if the frame rate changes, the amount of code given for each frame also changes, so the frame rate is a large parameter when creating a database. The image feature database is a combination of all these parameters, in which various contents are statistically organized into a database and recorded.
[0016]
The header information detected by the header detector 1 is transmitted to the image feature database 2. The image feature database 2 that has received the header information prepares data in parameters according to the header information. Although the description here is based on MPEG, even if sub-band encoding other than MPEG or fractal encoding is used, data in the middle of decoding specific to the encoding method is used. Any information can be created in the database as long as it represents the characteristics of the image. Therefore, the highest parameter of the database is the method type information.
[0017]
On the other hand, after the header is detected by the header detector 1, the compressed data is transmitted to the VLC decoder 3. Here, a variable length code (VLC) of a compressed stream such as MPEG is decoded for each block. (In the case of a fixed length code (FLC), FLC decoding is performed.)
[0018]
The decoded data is transmitted to the DC value, AC value, and motion vector value detector 4. The DC value, AC value, and motion vector value detector 4 detects the DC value, AC value, and motion vector from the decoded VLC.
[0019]
The DC value is transmitted to the color detector 5, and the values of Y, U, and V (Y is a luminance signal and U and V are color difference signals) are detected. The AC value is input to the frequency band detector 6, and as shown in FIG. 9, it is determined which type of the coefficient 0 to 7 is the characteristic of the absolute value of the effective coefficient for the eight DCT coefficients in MPEG. . Further, the AC value is also input to the contour detector 7, and when the edge of the contour exists in the block, the positional relationship on the screen of the block is detected using the feature 6 or 7 of the pattern. Then, the pattern closest to the pattern prepared in advance as shown in Table 1 is detected using the contour corresponding to what pattern the shape corresponds to as shown in FIG. For simplicity, five contents are described in Table 1, but it is realistic to have many databases for various contents.
Further, as shown in Table 2, not only the contour but also a pattern indicating which part of the screen the content is mainly located as an area is detected.
[0020]
[Table 1]

[0021]
[Table 2]

[0022]
The motion vector detected by the DC value, AC value, and motion vector value detector 4 is input to the motion amount detector 8, and the motion amount of the content is detected by the absolute value of the pixel precision vector. If the entire screen is panning, etc., even if it is a stationary object, it will result in motion, so if it is determined that the entire screen is moving in a certain direction, It is detected by subtracting the amount of global movement.
[0023]
The feature amount data of the compressed image data thus detected is input to the feature amount comparator 9 and compared with feature data (for example, Table 3) of various contents in the image feature database 2. Then, the content indicated by the feature amount data in the database having the highest compatibility with the feature amount data of the compressed image data is determined as the image content of the compressed image data, and the determination result of the image content is output from the image discriminator 10. The
[0024]
[Table 3]

[0025]
The additional information input to the image discriminator 10 in FIG. 1 is information of a determination result from an audio content discriminating device (FIG. 3) described later. This is so considered that the judgment result from the outside can be used as reference information when there is an ambiguous part in the judgment result. Of course, the additional information from the outside is not limited to the voice determination result, and other result information may be input. You do not have to enter anything.
[0026]
As described above, the image content determination apparatus according to the present embodiment does not have to completely decode the compressed image data (the decoding up to VLC decoder 3 is sufficient, and the inverse quantum shown in FIG. Complete decoding by the coder 52, inverse DCT unit 53, adder 54, image memory 55, and motion compensation predictor 56), characteristic data indicating content contents generated during the decoding of the compressed data, and statistics in advance It is possible to determine the content of the compressed data by searching for the feature using the feature data that has been obtained in advance. This eliminates the need for dedicated decoding means, a large memory, and a recording medium in the present image content determination apparatus, thereby reducing cost and size.
[0027]
Next, FIG. 2 shows an embodiment (speech content discriminating apparatus) for discriminating the content of audio compression data. The audio compression data is transmitted to the header detector 11. If the audio compression data is of, for example, the MPEG system, the sampling frequency, the number of channels, the bit rate, the coding mode, etc. are described. The header detector 11 detects these header values. Since this area has a fixed length structure, detection can be performed simply by reading a predetermined bit.
[0028]
The value of this header is transmitted to the voice feature database 12. In the audio feature database 12, these header values are obtained by statistically converting many content features in advance.
[0029]
On the other hand, after the header is detected by the header detector 11, the compressed data is transmitted to the VLC decoder 13. When not VLC, fixed-length FLC decoding is performed. Here, a variable length code (VLC) or fixed length code (FLC) of a compressed stream such as MPEG is decoded for each predetermined sample, and the data is transmitted to the DC value / AC value detector 14. The DC / AC value detector 14 detects the DC / AC value from the decoded VLC or FLC.
[0030]
The DC value is input to the sound pressure detector 15 to detect the loudness of the sound. The AC value is input to the frequency characteristic detector 16 and a unique characteristic is detected. Audio compression uses a subband method or DCT, but the order is not as small as an image, and is very large as 384 samples or 1152 samples. Therefore, a fine frequency characteristic is selected. For these, typical patterns in many contents can be determined. A voice feature database is created using these patterns.
[0031]
Then, the sound pressure and frequency characteristic feature data input from the sound pressure detector 15 and the frequency characteristic detector 16 to the feature amount comparator 17 are compared with the database read from the voice feature database 12, and The adapted content is discriminated, and the result is output from the audio discriminator 8.
[0032]
The additional information input to the sound discriminator 18 in FIG. 2 is judgment result information from an image content discrimination device (FIG. 3) described later. This is so considered that the judgment result from the outside can be used as reference information when there is an ambiguous part in the judgment result based only on the audio compression data. Of course, the additional information from the outside is not limited to the image determination result, and other result information may be input. You do not have to enter anything.
[0033]
Next, FIG. 3 shows an embodiment of an apparatus for discriminating the contents of video / audio compressed data. The compressed data that is input is compressed audio data and compressed image data created by compressing correlated audio content and image content. In general, both are multiplexed, and the playback time of each is synchronized. Time stamp has been struck. Examples of this are multiplexing methods in MPEG, MPEG systems ISO-IEC 11172-1, ITUH. Details are described in 222.0 / ISO-IEC13818-1. This compressed image / audio data is input to the image / audio separator 21 and separated. The separated compressed image and audio data are synchronized by the respective time stamps in the synchronization manager 22, the image data is sent to the image content discriminating device 23 (shown in FIG. 1), and the audio data is sent to the audio data. The contents are input to the content discrimination device 24 (shown in FIG. 2) in synchronization.
[0034]
In each discriminator, discrimination is performed by the above-described method. When discriminating the contents, more accurate discrimination is performed by using both information detected from the compressed image data and information detected from the audio compressed data. It is devised to be able to do. (The voice discrimination result is input to the image discriminator 10 in FIG. 1 from the voice discriminator 18 in FIG. 2, and the image discrimination is performed more accurately with reference to the voice discrimination result. Similarly, the voice discriminator in FIG. 18, the image discrimination result is input from the image discriminator 10 in FIG. 1, and the image discrimination is performed more accurately with reference to the image discrimination result.
Both discrimination results are output to the final discriminator 25. In other words, if it is determined that the sound is “sea” or “blue carpet” and the sound is determined to be close to the sound of “sea”, the final content determination result is determined to be “sea”.
[0035]
In the final discriminator 25, basically the same discrimination result can be obtained by the above-described device. However, if both the judgment results are too different, the final discriminator 25 can be set to give priority to either. This is because the accuracy can be artificially increased when the type of the compressed data to be input is known in advance which is easy to distinguish.
[0036]
If the compressed data is private information and the contents of which are not to be discriminated, determination rejection data is added to the header of the compressed data as shown in FIG. Judgment can be stopped. Even if the judgment rejection data is not described in the header, the number (ID) is added to the compressed data as shown in FIG. 5 and linked to the number (ID) of the compressed data as shown in FIG. The rejection information structure may exist as separate data. For example, for each predetermined unit of compressed data, 1 is written in the discrimination rejection information structure, and 0 is written if the data content judgment is rejected.
[0037]
7 and 8 show block diagrams of the content discriminating apparatus when using the judgment rejection data information. In FIG. 7, first, the compressed data is input to the discrimination rejection data detector 31. The discrimination rejection data detector 31 detects the determination rejection data. As a result, if the determination is rejected, the switch 32 is switched to “a mode in which no data flows” so that the compressed data is not input to the content determination unit 33. If not rejected, the switch 32 is switched to the “data flow mode” and the compressed data is input to the content determination unit 33. The content determination unit 33 determines the content of the compressed data and outputs a determination result. The content determination unit 33 corresponds to the blocks 1 to 10 shown in FIG. 1, the blocks 11 to 18 shown in FIG. 2, and the blocks 21 to 25 shown in FIG.
[0038]
In FIG. 8, the compressed data is first input to the content determination unit 33. At the same time, it is also input to the decision rejection data detector 31. The discrimination rejection data detector 31 detects the determination rejection data. As a result, when the determination is rejected, the switch 32 is switched to “a mode in which data is not passed”, and the result determined by the content determination unit 33 is not output. If not rejected, the switch 32 is switched to the “data flow mode” and the result determined by the content determination unit 33 is output.
[0039]
【The invention's effect】
As described above, according to the present invention, even if the compressed data is not completely decoded, the characteristic data indicating the content content generated in the middle of the decoding stage of the compressed data and the characteristic obtained statistically in advance. By searching for features using data, it is possible to determine the content of compressed data. Accordingly, the present invention provides a compressed data content determination method and a compressed data content determination device that do not require dedicated decoding means, a large memory, and a recording medium.it can.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image content determination apparatus according to an embodiment.
FIG. 2 is a block diagram of an audio content determination apparatus according to an embodiment.
FIG. 3 is a block diagram of an audio / video content determination apparatus according to an embodiment.
FIG. 4 is an explanatory diagram illustrating a first recording example of determination rejection data.
FIG. 5 is an explanatory diagram illustrating a second recording example of determination rejection data.
FIG. 6 is an explanatory diagram illustrating a second recording example of determination rejection data.
FIG. 7 is a block diagram of a content determination device according to an embodiment using determination rejection data.
FIG. 8 is a block diagram of a content determination apparatus according to an embodiment using determination rejection data.
FIG. 9 is an explanatory diagram of an AC pattern.
FIG. 10 is an explanatory diagram showing a relationship between AC and a contour pattern.
FIG. 11 is a block diagram of a conventional encoder.
FIG. 12 is a block diagram of a conventional decoder.
[Explanation of symbols]
1 Header detector
2 Image feature database
3 VLC decoder
4 DC value, AC value, motion vector value detector
5 color detector
6 Frequency band detector
7 Contour detector
8 Motion detector
9 Feature comparator
10 Image classifier

Claims (6)

A header detection step of detecting header information of the input image compressed data and extracting image size information and frame rate information from the detected header information;
For each predetermined block generated in the decoding process of the input image compressed data DC value, AC Image feature data detection step for detecting frequency band information and motion amount information, which is information that can specify the characteristics of the content of the input image content obtained based on at least one of the three information values and motion vectors When,
An image in which frequency band information, motion amount information, and content content information, which have been statistically obtained in advance for each combination of image size information and frame rate information parameters, are stored in a database as a set of feature amount data. From the database in the feature storage means, select feature amount data that can be determined to have the highest suitability using image size information, frame rate information, frequency band information, and motion amount information of the input image compressed data, and select the selected feature Image compressed data content determination comprising: an image compressed data content determination step for determining that the content content information of the quantity data is the content content obtained when the input image compressed data is decoded and reproduced Method. A header detection step of detecting header information of the input audio compression data and extracting sampling frequency information from the detected header information;
Every predetermined sample generated in the decoding process of the input voice compressed data DC value, AC An audio feature data detection step for detecting frequency characteristic information, which is information that can specify the feature in the content of the content of the input audio obtained based on at least one of the two pieces of value information;
From the database in the audio feature storage means which stores the frequency characteristic information and the content content information statistically obtained in advance for each sampling frequency information as a set of feature data, the input audio compression data The feature amount data that can be determined to be the most suitable using the sampling frequency information and frequency characteristic information of the selected feature amount data is selected, and the content content information of the selected feature amount data is obtained when the input audio compression data is decoded and reproduced. An audio compression data content determination method, comprising: an audio compression data content determination step for determining content content. When determining the content of the content from the compressed input image data and the compressed input audio data created by compressing the content correlated with the image information and the audio information,
A content detail information of the input compressed image data obtained by the image compression data content discriminating method according to claim 1, Contents information of the input audio compression data obtained by the audio compressed data content discriminating method according to claim 2, wherein And determining the content of the content using the two pieces of content content information . Header detection means for detecting header information of the input image compressed data and extracting image size information and frame rate information from the detected header information;
For each predetermined block generated in the decoding process of the input image compressed data DC value, AC Image feature data detection means for detecting frequency band information and motion amount information, which is information that can specify the characteristics of the content of the content of the input image obtained based on at least one of the three pieces of information of value and motion vector When,
Image feature storage that stores frequency band information, motion amount information, and content content information statistically obtained in advance for each combination of parameters of image size information and frame rate information in a database as a set of feature amount data Means,
Feature amount data that can be determined to have the highest suitability using the image size information, frame rate information, frequency band information, and motion amount information of the input image compressed data from the database stored in the image feature storage means The content content information of the selected feature amount data is determined to be the content content obtained when the input image compressed data is decoded and reproduced. Separately, an image compressed data content discriminating device comprising image compressed data content discriminating means. Header detection means for detecting header information of the input audio compression data and extracting sampling frequency information from the detected header information;
Every predetermined sample generated in the decoding process of the input voice compressed data DC value, AC Audio feature data detection means for detecting frequency characteristic information, which is information that can identify features in the content of the content of the input audio obtained based on at least one of the two pieces of value information;
Voice feature storage means for storing frequency characteristic information and content content information statistically obtained in advance for each sampling frequency information in a database as a set of feature amount data;
From the database stored in the voice feature storage means, select feature quantity data that can be determined to have the highest suitability using sampling frequency information and frequency characteristic information of the input voice compression data, and the selected feature quantity A voice compressed data content discriminating device comprising: voice compressed data content discriminating means for discriminating that the content content information of data is the content content obtained when the input voice compressed data is decoded and reproduced. . A compressed data content discriminating device that discriminates the content of the content from compressed input image data and compressed audio data generated by compressing content correlated with image information and audio information,
An apparatus for determining the content of compressed image data according to claim 4,
A voice compressed data content determination device according to claim 5;
A content detail information of the input compressed image data obtained by the image compression data content discriminating apparatus according to claim 4, Contents information of the input audio compression data obtained by the audio compressed data content discriminating apparatus according to claim 5, wherein A compressed data content discriminating apparatus comprising: content discriminating means for discriminating the content content using the two content content information .

Images