CN100469128C - Watermark Detection in Digital Signals - Google Patents

Abstract

Watermark detectors have a buffer in which a number of image tiles are folded and accumulated prior to computing the correlation between buffer contents and the watermark pattern being looked for. The intention of the folding and accumulation process is to average out the video content while accumulating the embedded watermark energy. This no longer appears to hold for strongly compressed video, such as DIVX, which exhibits a lot of artificial noise and undesired similarity (block patterns). As a result thereof, correlation peaks are often below the threshold. In a similar manner, the compression affects scale detection. According to this invention, only frames (or parts thereof) that are not so heavily compressed and therefore have a high probability of carrying enough watermark energy are folded and accumulated. To this end, a quality metric is calculated, the quality metric being indicative of the degree of compression of the data. The quality metric may be calculated based on the compressed data itself or derived from the decompressed base-band data. An advantageous example is the number of non-zero DCT coefficients of a (residue) frame. A determination is then made as to whether to exclude the frame (or part thereof) from the watermark decode process. The quality metric may also be used to select data for use in a scale detection process.

Description

Watermark Detection in Digital Signals

technical field

The present invention relates to the detection of watermark signals embedded in digital data, typically representing multimedia content. A typical format for this data is MPEG2, but other formats may be used with the present invention.

Background technique

In order to embed certain information, such as copyright, copy control, source or authorization data, into a digital signal, well-known watermarking techniques are generally used. It involves processing the digital data to "paste" a recognizable pattern onto the data to be watermarked. Different types of watermarks serve different purposes. A simple robust watermark involves a wide range of processing steps in the analog and digital domains, which can simply indicate that the watermarked data is copyrighted, and can provide further details such as owner and date. Fragile watermarks are often added in such a way that if the data is manipulated in any way, the watermark will be broken or destroyed. Thus, the absence of a fragile watermark in a data file, or stream, in which a watermark is expected can indicate that the data has been processed or tampered with. This can be used in medical or forensic science applications where authorization is important.

Various types of watermarking patterns inherently contain pseudo-noise signals plastered or woven into the signal itself. The watermark signal ideally should not perceptually degrade the quality of the source data, but should be detectable by an appropriate decoder.

A particular problem arises when the watermark data is compressed to a very low bit rate suitable for transmission over the Internet or other data transmission system. DIVX is a system that can produce very low bit rates and is widely used to reduce the amount of bandwidth required to transmit video images over the Internet.

Currently used watermarking systems such as JAWS (Ton Kalker, Geert Depovere, Jaap Haitsma, Maurice Maes, "A Video Watermarking System for Broadcast Monitoring", Proceedings of SPIE Electronic Imaging'99, Security and watermarking of Multimedia Contents, San Jose (CA), USA, January 1999) use a decoder that finds embedded watermarks by collecting a large amount of video data, then superimpose and accumulate the video data, and then associate the accumulated data with the expected watermark pattern. When the video data has been compressed to a very low bitrate, for example using DIVX, a frequently encountered result is that correlation peaks appear below this detection threshold. This means that detection of the embedded watermark may fail, which may cause inconvenience to system users who are authorized to view, but are prohibited from viewing the watermarked video in the absence of proper watermark detection.

Further problems arise when the watermarked video has been scaled or resized. In order to detect the embedded watermark, the original scale of the video signal is required, so that the accumulation buffer captured into video data can be scaled accordingly to the dimensions of the original video. The original scale must be determined from the scaled video data itself. In contrast to the watermark detection process which correlates the video data against known watermark data, the prior art scaling detection process works by correlating two noise accumulation buffers with each other to generate a scaling factor.

In the JAWS system, watermark detection and the watermark detection process and the scaled retrieval process repeatedly use the watermark pattern embedded in the source data. During the watermark embedding process, a 128*128 watermark pattern is "tiled" over the entire extent of the data frame.

To retrieve the horizontal scale information from the scaled data, the process begins with arbitrarily selecting two horizontally adjacent tiles A and B from a plurality of accumulated frames. These two slices are then related to each other according to the following steps:

Calculate the 128×128 Hanning window of A and B; Han(A), Han(B)

A Hanning window is a filter that "fades" the edges of the slice to which it is applied. Thus, the data in the center of the patch is preserved, but decays to zero closer to the edge. This removes this edge effect of strong artificial frequency components introduced in subsequent FFT calculations.

Calculate the 128×128 fast Fourier transform (FFT) of A and B;

Calculate the complex conjugate of Han(B); Con(Han(B));

Calculate the point-wise product of Han(A) and Con(Han(B));

Normalizes the product result. For each complex value (z) in the result, this is done by the following formula, whereby z can be replaced by:

$\frac{\mathrm{<span\; class="notranslate">\; z</span>}}{\sqrt{\mathrm{<span\; class="notranslate">\; re</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; im</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}$

Computes the inverse FFT of the previous steps.

The position of the highest value in the first row of the IFFT result is then used to calculate the horizontal scaling factor. If the first value is the highest, then the horizontal scaling factor is 1, ie no scaling occurs.

The vertical scaling factor is calculated in a similar manner, but instead two vertically adjacent tiles and the first column of the IFFT result are used.

The correlation peak is even lower for the scaling retrieval process than for the watermark detection process because the inherently more noisy buffer samples are used. (Watermark detection consists of a correlation between a known pattern and a noise accumulation buffer: scaling detection is a correlation between two noise accumulation buffers). To further complicate matters, frame alignment may not be used in this scaling detection process. This is because frame alignment can only be used when the scaling is known. If the scaling is not known, out-of-sync patterns are accumulated and the resulting accumulation buffer is useless. As a result, only accumulation can be used. This means that more frames have to be collected before the correlation can be performed, which of course takes more time.

Folding works by "magnifying" the watermark data, since it always has the same mark. The underlying video signal is effectively "random" and thus averaged. Sufficiently long superposition can obtain this original watermark pattern. However, this process does not work if the pattern (128x128 tiles) is not exactly aligned.

The prior art attempts to eliminate these problems by accumulating more frames per detection, hoping that the video data will be averaged and the watermark signal amplified, thereby increasing the signal (watermark) to noise (video) ratio.

In a typical scaling inspection, up to 300 frames are currently used. In the case of DIVX compressed video however, a lot of artifacts and the like due to block patterns are introduced which are not desirable. During this accumulation process, usually more noise than watermark energy is accumulated. Also, the undesired pattern is also amplified and it is usually stronger than the watermark signal. All of these issues make reliable scaling detection of DIVX video difficult and often impossible. Watermark detection is impossible without reliable scaling detection.

Contents of the invention

It is an object of embodiments of the present invention to at least eliminate the above-mentioned problems encountered with prior art detection systems and to provide a better watermark detection system for highly compressed video or other multimedia data.

A further object of the embodiments of the present invention is to perform scaling detection processing with more reliable performance before watermark detection.

According to the present invention, there is provided a method of selecting data for use in decoding an embedded watermark in compressed multimedia data, comprising the steps of:

calculating a quality metric for a given portion of the multimedia data based on the degree of compression of the multimedia data;

If the quality metric of the given part is above a certain threshold, it is included in the watermark decoding process;

If the quality metric for the given part is below the threshold, it is not included in the watermark decoding process.

Preferably, the method further comprises the step of using the same quality measure to select data for use in a scaling detection process performed prior to the watermark decoding process. It returns a scaling factor of 1 when no scaling has occurred. Otherwise, the scale detection process returns a value that allows the accumulation buffer to be resized appropriately before decoding the watermark.

Preferably, the quality metric is calculated from an analysis of the compressed data stream. This compressed data stream is provided through the DIVX system.

Suitably, where access to the compressed data stream is available, the quality measure may be determined from one of quantization factors, the number of variable length codewords (VLC) used to encode frames of data, motion vectors.

The quality metric can also be calculated from multiple parameters.

Preferably, the quality metric can be calculated from an analysis of baseband data.

Preferably, the quality measure is calculated from a measure of the energy of the frame.

The quality metric can also be calculated from multiple parameters.

Preferably, the given portion of the data is a frame. Alternatively, partial frames may also be used.

Preferably, an apparatus is provided for carrying out the method according to the invention.

Description of drawings

For a better understanding of the invention, and in order to understand how it may be practiced, it will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 shows a schematic representation of an embodiment of the invention.

Detailed ways

Figure 1 shows a schematic representation of this data flow in an embodiment of the invention. The data buffer 10 is configured to receive an incoming data stream 110 . In a particular embodiment, the data stream 110 is a DIVX encoded video data stream. The data buffer 10 is used to select all or part of the frames 120 of the incoming data stream, which are then analyzed in the quality metric calculator 20 . The quality metric calculator operates on the data frame (or a portion thereof) 120 to establish a quality metric 130 for the input data frame 120 . The quality metric is indicative of the likelihood of the particular frame comprising sufficient watermark energy to be used in the watermark decoding process. Briefly explain how this quality metric was calculated.

This quality metric 130 is compared to a predetermined level in the threshold detector 30 . A frame 120 may only be used in the watermark detection process 40 if the quality metric indicates that there is a high probability that the frame 120 includes an appropriate amount of watermark energy.

However, if the quality metric 130 is below the predetermined acceptable level, then the threshold detector discards 50 the data in frames 120 and it will no longer function in the watermark decoding process 40 .

In this way, only data with a high probability of comprising sufficient watermark energy to be able to successfully perform decoding of the watermark are transmitted to the watermark decoding process. The output of this watermark decoding process is watermark 140 . Alternatively, the output 140 may be a binary signal indicating correctly decoded or no watermark detected.

To determine a quality metric (Q), one or more characteristics of the data are evaluated or measured. The examples below highlight properties that can be used in certain situations. Those skilled in the art will be aware of other properties that may form the basis for quality metric calculations in other cases.

The quality metric (Q) effectively provides a measure of how much the subject data has been compressed. The more this data is compressed, the more difficult it is to extract the watermark from it.

If access to the compressed data stream is available, parameters can be derived from the stream itself, which can be used to determine the quality metric (Q). Some suitable parameters are:

quantization factor;

the number of variable length codewords (VLC) used to encode the frame;

Motion vector.

In systems where this compressed data is accessible, the quality metric can be derived by counting the number of VLCs used to encode the frame. In this case, only frames encoded with more than 5000 coefficients are superimposed and used in this watermark detection process.

However in many cases it is not possible to access the original compressed stream and for example only the baseband video signal. In this case, the aforementioned parameter is not accessible, so a different metric can be used to determine Q. One measure is:

energy measure. Such a metric can be obtained, for example, by 8x8 DCT transformed frame blocks, quantizing the coefficients using a coarse standard MPEG quantization matrix, and counting the number of non-zero coefficients. A block's non-zero coefficient indicates its energy content. If there are many higher coefficients around the DC frequency, this indicates that there are sharp edges in the block. Many nonzero coefficients indicate that the block has a complex structure. If there are no AC coefficients, this indicates that the block is flat. In general, the more non-zero coefficients, the more energy of this watermark there may be in the block.

Once an appropriate quality metric (Q) has been computed from one or more given properties of the signal, it is possible to establish a threshold for a specific value of Q such that for decoding embedded watermarks, images with Data frame (or part thereof) of Q values. This actual data frame (or part thereof) is of course preserved so that its inherent data content (eg video) can be decoded.

The establishment of the threshold depends on the particular properties of the data signal chosen to base the quality measure on, and can preferably be determined experimentally in a particular case.

As previously mentioned, further problems arise when the compressed video signal has been scaled. Before the watermark can be decoded from the compressed signal, the original scaling of the signal must be restored.

Embodiments of the present invention recover scaling information in a manner similar to that just described for recovering watermark information. To recover the scaling information, the two accumulation buffers are associated, and the resulting association gives a direct representation of the scaling factor.

In order to improve the results of the correlation process, the same quality metric (Q) calculated above can be used to identify candidate frames (or parts thereof) that are not compressed enough and thus have a higher Q. These candidate frames may be prioritized for the scale determination correlation process over frames that are highly compressed and thus have a lower Q.

Experiments have shown that the scaling detection process has been greatly improved by selecting the data samples used in the association process. In cases where the correlation peak would be below a predetermined detection threshold for scaling detection using prior art methods, it was found that embodiments of the present invention, by selectively discarding certain data samples that do not contribute to a successful correlation, A scaling factor can be determined.

In fact, the same technique can be used first to find a scaling factor for the compressed signal, which can then be used to scale the accumulation buffer appropriately, and then enable more reliable watermark decoding.

Embodiments of the invention may be implemented using suitably adjusted or programmed hardware. Such hardware may comprise specialized hardware, such as a custom ASIC, or more general purpose processors or DSPs operating in accordance with appropriate programming.

Those skilled in the art know that other parameters can be used as a basis for calculating the quality metrics and that the examples described here do not limit the scope of the invention, which is defined by the appended claims.

Claims (10)

1. method that the data of using in to watermarked decoding the in the compressing multimedia data are selected may further comprise the steps:

According to the degree of compression of this multi-medium data, calculate the quality metric of giving certain portions of these compressing multimedia data;

If should be higher than a certain threshold level to the quality metric of certain portions, it just is included in the watermark decode process so;

If should be lower than this certain threshold level to the quality metric of certain portions, it just is not included in this watermark decode process so.

2. as 1 method in the claim, wherein this method also comprises step: use identical quality metric to be chosen in the data of using in the scale-detection performed before this watermark decode process.

3. as 1 method in the claim, wherein according to this quality metric is calculated in the analysis of packed data stream.

4. as 3 method in the claim, wherein one of them calculates this quality metric according to the following parameters relevant with this packed data stream: quantizing factor, be used for number, the motion vector of variable length codeword (VLC) of coded frame data.

5. as 3 method in the claim, wherein calculate this quality metric according to a plurality of parameters relevant with packed data stream.

6. as 1 method in the claim, wherein according to this quality metric is calculated in the analysis of base band data.

7. as 6 method in the claim, wherein according to the tolerance of frame energy is calculated this quality metric.

8. as 7 method in the claim, wherein calculate this quality metric according to a plurality of parameters relevant with this base band data.

9. as 1 method in the claim, wherein these data is frame to certain portions.

10. select data so that the equipment that uses comprises for one kind in to watermarked decoding the in the compressing multimedia data:

The device of the compressing multimedia data that reception enters;

Be arranged to the degree of compression, calculate the quality metric calculator of the quality metric of giving certain portions of these compressing multimedia data according to this multi-medium data; And

Threshold dector if be arranged to and should be higher than a certain threshold level to the quality metric of certain portions, so just is included in it in watermark decode process; And if be arranged to and should be lower than this threshold value to the quality metric of certain portions, so just it is not included in this watermark decode process.