JP2012205168A - Device, method and program for video processing - Google Patents

Abstract

A video processing apparatus capable of embedding additional information in a video displayed on a display device with a small amount of computation and further embedding a digital watermark capable of confirming the additional information without using a dedicated detection means. I will provide a.
Luminance modulation processing is performed in which video data is input for each frame, and luminance addition processing and subtraction are alternately performed for each constituent pixel of the video data for each frame as time passes. To embed watermark information and output.
[Selection] Figure 1

Description

  The present invention relates to a video processing apparatus, a video processing method, and a video processing program for embedding a digital watermark in digital data.

  Conventionally, in order to embed copyright information or the like as additional information in video data for the purpose of detecting or preventing illegal copying of video data, various digital watermarking methods have been proposed. For example, when compressing video information using a compression technique such as MPEG, there is a method of embedding additional information without greatly degrading the quality of the video by a method such as operating a specific frequency component.

  Also, a copy protection pattern composed of symbols, random patterns, digital watermarks, and text messages is embedded as multiple pixels in each frame of a digital video, and the displayed pixel intensity is invisible to human observers. There is also known a method modulated at a temporal frequency using a modulation characteristic intentionally selected to generate unpleasant aliasing in a copy made using a video camera (see, for example, Patent Document 1). .

JP 2002-314938 A

  However, these methods are computationally intensive and require a large amount of computing resources to perform real-time processing on high-definition video, so that real-time video generation and display such as virtual reality is required. There is a problem that it is not suitable for use. In addition, detection of embedded information requires detection processing corresponding to the embedded method, such as when the video is processed or when the displayed video is re-captured with a video camera, etc. However, there is a problem that the embedded information cannot always be restored.

  The present invention has been made in view of such circumstances, and embeds additional information in a video displayed on a display device with a small amount of computation, and further confirms the additional information without using a dedicated detection means. An object of the present invention is to provide a video processing apparatus, a video processing method, and a video processing program capable of embedding a digital watermark that can be executed.

  The present invention provides a luminance modulation process in which video data is input for each frame, and luminance addition and subtraction is alternately performed for each component pixel of the video data for each frame as time passes. By embedding, watermark information is embedded and output.

  According to the present invention, the luminance modulation processing is such that the watermark information is divided into a plurality of processing unit regions, and the result of time averaging the spatial average luminance of the constituent pixels in each processing unit region subjected to the luminance modulation processing is as follows: It is equal to the luminance obtained by averaging the pixels in the processing unit area before the luminance modulation processing.

  The present invention is characterized in that the amplitude of luminance modulation with respect to a pixel is set so that a temporal or spatial change in luminance in the processing unit region is less than or close to a viewer's perceptual limit.

  The present invention is characterized in that the amplitude of the luminance modulation is controlled based on the luminance of the modulation target pixel.

  The present invention is characterized in that the amplitude of the luminance modulation is controlled based on the magnitude of the spatial luminance change in the vicinity of the modulation target pixel.

  The present invention is characterized in that the video data is generated by a CG technique, and the magnitude of the spatial luminance change in the vicinity of the modulation target pixel is derived using the CG model data.

  The present invention is characterized in that the video data is generated by a CG technique, and the amplitude of luminance modulation is controlled based on the attractiveness of an object including a modulation processing target pixel.

  The present invention is characterized in that the position of the processing unit area is controlled following the movement of the subject in the video data.

  According to the present invention, input means for inputting video data for each frame, and addition and subtraction of luminance are alternately performed for each frame of each constituent pixel of the video data as time elapses with respect to the input video data. A watermark information embedding unit that embeds watermark information by performing luminance modulation processing, and an output unit that outputs the video data in which the watermark information is embedded are provided.

  According to the present invention, a watermark information embedding process for the video data is performed on a computer on a video processing apparatus including input means for inputting video data for each frame and output means for outputting the video data in which watermark information is embedded. A video processing program to be executed by performing a luminance modulation process for alternately adding and subtracting luminance for each component pixel of the video data over time for the input video data over time A watermark information embedding process for embedding watermark information is performed by the computer.

  According to the present invention, it is possible to embed additional information in a video displayed on a display device with a small amount of computation, and further to embed an electronic watermark that can confirm the additional information without using a dedicated detection means. The effect is obtained.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a figure which shows an example of an original image | video. It is a figure which shows an example of watermark information. It is a figure which shows an example of the image which performed the luminance subtraction. It is a figure which shows an example of the image which performed luminance addition. It is a figure which shows an example of the image which made addition and subtraction coexist. It is a figure which shows an example of the image which performed the averaging on a time axis. It is a figure which shows an example of the image which performed the averaging on a time axis. It is a figure which shows the time change of a brightness | luminance. It is a figure which shows the relationship between a brightness | luminance and a modulation amplitude. It is a figure which shows the relationship between a brightness | luminance and a modulation amplitude. It is a figure which shows an example of a modulation amplitude coefficient. It is a figure which shows an example of the image of a CG scene. It is a figure which shows the object structure of the image shown in FIG. It is a figure which shows an example of a brightness | luminance change map. It is a figure which shows an example of a modulation amplitude coefficient. It is a figure which shows an example of an attractiveness map. It is a figure which shows an example of a modulation amplitude coefficient.

  A video processing apparatus according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 100 denotes a video generation device that generates a video to be embedded with a digital watermark, and generates and outputs a moving image using CG (computer graphics). Reference numeral 201 denotes digital video data for one frame generated by the video generation apparatus 100. Reference numeral 202 denotes a luminance change map configured from luminance change information for applying luminance modulation to video data. Reference numeral 203 denotes an attractiveness map in which attractiveness information of the target video is defined.

  Reference numeral 300 denotes a video processing device that performs processing for embedding a digital watermark into the video data 201 and outputs the video data 201, and is configured by hardware configured by a computer device or a dedicated logic circuit. Reference numeral 301 denotes a video input unit that inputs the video data 201, the luminance change map 202, and the attractiveness map 203. Reference numeral 302 denotes a watermark information setting unit that sets watermark information that is additional information to be embedded as a digital watermark. Reference numeral 303 denotes a luminance modulation unit that performs luminance modulation processing according to a rule determined so that the watermark information can be restored based on the watermark information. Reference numeral 304 denotes a luminance change detection unit that detects a luminance change in the video data. Reference numeral 305 denotes a video output unit that outputs video data in which a digital watermark is embedded. Reference numeral 400 denotes a video display device that displays video data output from the video output unit 305.

  Next, the processing operation of the video processing apparatus 300 shown in FIG. 1 will be described. First, the video input unit 301 inputs video data 201 which is an original video to be embedded with a digital watermark from the video generation device 100. Here, for the original video shown in FIG. 2, as shown in FIG. 3, it consists of a processing unit area of value 0 and two processing unit areas of value 1 associated with the pixels of the original video. A graphic pattern having the same size (the same number of pixels) as 201 is embedded as watermark information, a pixel in the processing unit area corresponding to watermark information value 1 is modulated, and processing corresponding to watermark information value 0 is performed. The rule is that no modulation is performed on the pixels in the unit area.

  The watermark information setting unit 302 stores watermark information shown in FIG. 3 in advance. The luminance modulation unit 303 performs luminance modulation on the video data 201 input by the video input unit 301 based on the watermark information output from the watermark information setting unit 302. The luminance modulation is performed by adding or subtracting the luminance change amount to the pixel value corresponding to the watermark information value 1 for each pixel of the video data 201. That is, the luminance change amount is added to or subtracted from the luminance value of the pixel of the video data 201 in the area of the watermark information value 1.

  The luminance modulation unit 303 outputs the video data 201 after the luminance modulation to the video output unit 305. The video output unit 305 converts the luminance-modulated video data 201 output from the luminance modulation unit 305 into display data that can be displayed on the video display device 400 and outputs the display data to the video display device 400. As a result, the video data 201 after luminance modulation is displayed on the video display device 400.

  4 and 5 show the video data 201 after the luminance modulation. FIG. 4 is an example in which watermark information is added, and FIG. 5 is an example in which watermark information is subtracted. In addition, as shown in FIG. 6, you may process so that the pixel by which the brightness | luminance was added and the pixel by which subtraction was made to coexist within a process unit area | region. Thereby, the spatial average luminance of the pixels after luminance modulation in the processing unit area can be kept equal to the original image before luminance modulation, and the invisibility of the watermark with the naked eye can be improved.

  Next, the luminance modulation unit 303 performs luminance addition on the frame subjected to luminance subtraction for the same pixel in order to average the luminance of the pixels in the processing unit area on the time axis. As shown in FIG. 7, luminance modulation of opposite sign is performed on each of the odd and even frames of the video signal so that the frames are alternately repeated. In the example illustrated in FIG. 7, the repetition period is two frames, and the time average of the luminance of two consecutive frames is equal to that of the original image, but the repetition period is not limited to two frames.

  For example, luminance subtraction for one frame and luminance addition for two frames may be repeated every three frames. In this case, the time average luminance can be made equal to that of the original image by changing the luminance change amount of the addition processing and the subtraction processing and setting the luminance addition amount to ½ of the luminance subtraction amount. Alternatively, the luminance change amount may be changed in fine steps for each frame. Also, as shown in FIG. 8, in the luminance modulation in which luminance addition and luminance subtraction in FIG. 6 coexist, the area to be added and the area to be subtracted may be switched for each frame.

  The luminance-modulated original video signal is output to the video display device 400 via the video output unit 305, and the video in which the watermark information is embedded is displayed on the video display device 400. The video processing device 300 repeatedly performs this processing operation for each frame on the video data 201 output from the video generation device 100.

  As described above, by making the spatial average luminance of the processing unit area or the temporal average of the spatial average luminance equal to that of the original video, the invisibility of the watermark can be improved. Depends heavily on Here, by setting the amount of change in luminance so that the temporal or spatial change in luminance of the pixels in the processing unit area is less than or close to the perceptual limit of the observer, a watermark with excellent invisibility can be obtained. You may make it perform embedding.

  FIG. 9 shows changes along the time axis of the luminance of the luminance-modulated pixel. In FIG. 9, T is a modulation period. When this pixel displayed on the video display device 400 is re-photographed with a video camera or the like in the same exposure time frame t as the modulation period, the time average luminance in the time frame t is equal to the original image, so the luminance of the captured pixel Is the same as when the original image was taken, and the watermark pattern does not appear. Similarly, the watermark pattern does not appear in the time frame t ′ in which the exposure time is an integral multiple of T.

  The time t2 during which the luminance subtraction is performed within the time frame t ″ where the exposure time is not an integral multiple of T is longer than the time t1 + t3 during which the luminance addition is performed, and the time average luminance is lower than that of the original image. As a result, a luminance boundary is generated between a region where luminance modulation is not performed or a region where reverse code modulation is performed, and a watermark pattern is reflected. In a home video camera or the like, it is difficult to control the exposure time so as to exactly match the modulation period T, and in many cases, a video with a watermark pattern is re-photographed.

  The appropriate amplitude of the luminance modulation varies depending on the processing unit area of the original image and the luminance of the calculation target pixel. For example, when the range of the luminance value is 0 to 255, the modulation cannot be performed under the condition that the luminance of the pixel becomes negative or exceeds 255 as a result of the luminance modulation. FIG. 10 shows, as an example, the relationship between the luminance of the target pixel and the amplitude that can actually be modulated when the amplitude of 20 is modulated by simple addition and subtraction in units of pixels. This relationship may be held as a function or LUT (look-up table) in the video processing apparatus 300 and used for modulation amplitude control to avoid modulation processing using an inappropriate amplitude.

  Further, the relationship between the luminance of the original video and the modulation amplitude may be flexibly changed by a similar method. For example, the presence of a watermark is conspicuous in a region with a flat luminance such as a blue sky. In general, since the brightness of the blue sky is high, as shown in FIG. 11, when the brightness is high, the modulation amplitude is controlled using a function that reduces the amplitude or the LUT, thereby increasing the invisibility of the watermark in the blue sky portion. Can do.

  In addition, the presence of a watermark is conspicuous in a region where the luminance is flat. Therefore, the luminance modulation unit 303 pays attention to a plurality of pixels in the vicinity of the modulation target pixel, obtains a variance that is a statistic of the luminance values, and if the variance is 0, all the target pixels have the same luminance value, that is, The luminance change does not exist in the area, but it is determined that the luminance change in the vicinity of the target pixel becomes severe as the variance increases. Further, as shown in FIG. 12, a coefficient by which the modulation amplitude is multiplied using a function or LUT that has been adjusted to a desired characteristic in advance is obtained, and a pixel having a small luminance change in the vicinity detected by the luminance change detecting unit 304 is obtained. For a pixel having a small modulation amplitude and a large luminance change in the vicinity, a modulation process that increases the modulation amplitude may be performed.

  In addition, when the video to be processed is generated by the CG technology, the video generation device 100 can estimate the distribution of luminance change in the video from the CG model data. As an example, FIG. 13 and FIG. 14 show the configuration of a CG scene and object areas therein. Each object area of the CG scene shown in FIG. 13 corresponds to specific CG model data, and the luminance change in each object area is determined from the structure of the CG model data and the characteristics such as the material and texture used. The amount can be estimated based on the subjective judgment of the producer or the statistical properties of the CG model. In each object area, as shown in Table 1, a luminance change index serving as an index indicating the magnitude of the luminance change amount in the object area is set in advance at the time of CD data production.

  Each object of the CG scene is drawn on the projection plane by projection conversion in the video generation apparatus 100. At that time, the area occupied by each object on the display plane is calculated, and the display is performed based on the luminance change index shown in Table 1. What is necessary is just to produce | generate the luminance change map 202 which shows distribution of the luminance change parameter | index on a surface (refer FIG. 15). The luminance change map 202 is input to the luminance modulation unit 303 via the video input unit 301. The luminance modulation unit 303 obtains a coefficient by which the modulation amplitude is multiplied using a function or LUT previously adjusted to a desired characteristic as shown in FIG. 16, and the modulation amplitude is obtained for a pixel in a region where the luminance change index is small. For a small pixel with a large luminance change index, the modulation amplitude is determined so as to increase the modulation amplitude.

  Also, in a highly attractive area that is often watched by viewers, it is desirable to adjust the modulation amplitude to be small because the presence of a watermark is easily noticed. The attractiveness of an object varies greatly depending on the contextual importance and the viewer's interest in the object. For example, in the scene shown in FIG. 13, it is estimated that the attractiveness of the airplane is the highest. In this way, an attractiveness index as shown in Table 1 may be set in advance for each object based on the subjective judgment of the producer at the time of CG data production. The video generation apparatus 100 may calculate the area occupied by each object on the display surface, and generate the attractiveness map 203 indicating the attractiveness index distribution on the drawing surface based on the object attributes shown in Table 1. FIG. 17). The attractiveness map 203 is input to the luminance modulation unit 303 via the video input unit 301.

  The luminance modulation unit 303 obtains a coefficient by which the modulation amplitude is multiplied using a function or LUT that has been adjusted to an appropriate characteristic as shown in FIG. 18, and the modulation amplitude is obtained for a pixel in a region having a small attractiveness index. For a large pixel with a large attractiveness index, the modulation amplitude may be determined so that the modulation amplitude is small.

  As described above, the video processing apparatus 300 inputs video data for each frame, and alternately adds and subtracts luminance for each component pixel of the video data for each frame as time passes. Since the watermark information is embedded and output by performing the luminance modulation processing, the additional information is embedded in the video displayed on the display device with a small amount of calculation, and the additional information is used without using a dedicated detection means. It is possible to embed a digital watermark that can be confirmed.

  Note that a program for realizing the function of the processing unit in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to embed a digital watermark. Processing may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  It can be applied to applications where it is essential to embed digital watermarks in digital data.

  DESCRIPTION OF SYMBOLS 100 ... Video production | generation apparatus, 201 ... Video data, 202 ... Luminance change map, 203 ... Attraction map, 301 ... Video input part, 302 ... Watermark information setting part, 303. ..Luminance modulation unit, 304 ... Luminance detection unit, 305 ... Video output unit, 400 ... Video display device

Claims (10)

  By inputting video data for each frame, and performing luminance modulation processing for alternately adding and subtracting luminance for each component pixel of the video data as time elapses with respect to the input video data An image processing method, wherein watermark information is embedded and output.   In the luminance modulation processing, the watermark information is divided into a plurality of processing unit areas, and the result of time averaging the spatial average luminance of the constituent pixels in each processing unit area subjected to the luminance modulation processing is the luminance modulation processing. The video processing method according to claim 1, wherein the image processing method is equal to a luminance obtained by averaging pixels in the previous processing unit area.   3. The amplitude of luminance modulation with respect to a pixel is set so that a temporal or spatial change in luminance in the processing unit region is less than or close to a viewer's perceptual limit. The video processing method described in 1.   4. The video processing method according to claim 1, wherein an amplitude of the luminance modulation is controlled based on luminance of a modulation processing target pixel.   4. The video processing method according to claim 1, wherein the amplitude of the luminance modulation is controlled based on a magnitude of a spatial luminance change in the vicinity of a modulation processing target pixel.   6. The video data according to claim 1, wherein the video data is generated by a CG technique, and a magnitude of a spatial luminance change in the vicinity of a modulation target pixel is derived using CG model data. The video processing method according to any one of the above.   The video data is generated by a CG technique, and the amplitude of luminance modulation is controlled based on the attractiveness of an object including a modulation target pixel. Video processing method.   8. The video processing method according to claim 1, wherein the position of the processing unit area is controlled following the movement of a subject in the video data. Input means for inputting video data for each frame;
Watermark information embedding means for embedding watermark information by performing luminance modulation processing for alternately adding and subtracting luminance for each component pixel of the video data as time elapses with respect to the input video data ,
An image processing apparatus comprising: output means for outputting the image data in which the watermark information is embedded. Video processing for causing a computer on a video processing apparatus comprising input means for inputting video data for each frame and output means for outputting the video data embedded with watermark information to perform watermark information embedding processing on the video data A program,
A watermark information embedding process for embedding watermark information by performing a luminance modulation process for alternately adding and subtracting luminance for each component pixel of the video data as time elapses with respect to the input video data. A video processing program that is executed by the computer.