JP3703258B2 - Image processing apparatus and method, and storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method for adding predetermined information to an input image, and a storage medium.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, as disclosed in, for example, Japanese Patent Laid-Open No. 5-244389, a technique for adding a serial number of a copying machine that processes an input image to a reproduced image is known. Such a technique has been effective in tracking the origin of an image from a reproduced image.
[0003]
On the other hand, in recent years, various storage formats have been proposed as formats for storing image information.
[0004]
[Problems to be solved by the invention]
However, in the above-described storage format, a method for adding predetermined information so as not to be easily seen by human eyes has not been sufficiently studied. Therefore, for example, when an input image is divided into a plurality of areas and image information is stored in units of the areas, information accompanying the image information cannot be added by an appropriate method.
[0005]
The present invention has been made in view of such circumstances, and when image information is stored using a predetermined image storage format, the predetermined information is added to the input image by a method suitable for the storage format. With the goal.
[0006]
[Means for Solving the Problems]
In order to solve the above-described problems, according to the image processing apparatus of the present invention, an input unit that inputs image data, a storage mode selection unit that selects a storage mode according to the attribute of the image data, and the image data Conversion means for converting the data into the selected storage form; insertion method selection means for selecting a method for inserting watermark information according to the storage form; and It has an insertion means for inserting watermark information.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Preferred embodiments of the present invention will be described below with reference to the drawings.
[0009]
1. File structure
(1) The storage format of this embodiment is composed of two types of objects: directories (shaded portions) and files (shaded portions).
[0010]
In the example of FIG. 1, directory 1 is the root directory and is the directory at the top of the storage format. Directory 1 includes two directories (directory 2 and 3) and one file (file 1). Further, the directory 2 includes an empty directory 4, and the directory 3 includes two files (files 2 and 3).
[0011]
Image information and image attribute information are structured and stored by the above two objects.
[0012]
(2) Image file structure
Each image file (20) includes the following elements as shown in FIG.
(1) First attribute information (21)
Format ID, title, producer, keyword, comment, last saver, revision number (number of times the object was saved), total editing time, last print date, original production date, last save date, thumbnail attributes, creation application This is summary information of the target image such as name.
(2) Second attribute information (22)
General information about the storage target such as header, user type and clipboard format.
(3) Third attribute information (23)
This is information related to the output image, such as a list of locked properties, a title of the converted image, a last modified person, an output image index, a maximum image index, a maximum conversion item index, and a maximum operation index.
(4) Source image object (24)
It consists of image information of the source image and its attribute information. Details will be described later.
(5) Image object after conversion process (25)
It consists of image information obtained by applying a predetermined conversion process to the source image and its attribute information. The configuration is the same as the source image object.
(6) Fourth attribute information (26)
Image ID related to source image object or post-conversion image object, list of locked properties, title, last modified person, revision number, creation date, last modification date, creation application name, status, number of conversion processes, number of conversion processes This is additional information of the original image or the post-conversion image such as the height / width of the list / included image.
(7) Fifth attribute information (27)
Conversion item ID, operation class ID, list of locked properties, conversion title, last modified person, revision number, creation date, last modification date, creation application name input data list, output data list, operation number, aspect of processing result This is attribute information regarding conversion of image information of the source image, such as ratio, target rectangular area, filtering, geometric calculation, color conversion matrix, and contrast adjustment.
(8) Sixth attribute information (28)
This is information for identifying software for executing conversion.
[0013]
(3) Configuration of image object (30)
The source image object (24) and the post-conversion image object (25) include the following elements as shown in FIG.
(1) Seventh attribute information (31)
Format ID, title, producer, keyword, comment, last saver, revision number (number of times the object was saved), total editing time, last print date, original production date, last save date, thumbnail image, thumbnail image It is summary information of each object such as attributes and created application name.
(2) Eighth attribute information (32)
General information about the object to be stored, such as header, user type, clipboard format.
(3) Ninth attribute information (33)
Number of target image resolutions ・ Maximum resolution image width and height ・ Default width and height of image display ・ Image display width and height unit ・ Subimage width, height, color, format ・Reduction method (multi-resolution image creation method), reduction pre-filter width, sub-image color processing profile, compression parameters (for example, in JPEG compression, quantization table data, Huffman table data SOI, DQT, DHT, Attribute information relating to the storage method of image information, such as EOI.
(4) Tenth attribute information (34)
File source (image source device, scene type, image creation history, creation software name, user-defined ID, sharpness value), intellectual property rights (copyright indication, original image right holder, digital image right holder) , Producer, remarks), content description information (proofing data in image, description about image group, description about title or purpose of image, description about person in image, description about object in image, date of original image, Description about the event in the image, description about the place in the image, remarks), information about the camera used for shooting (camera manufacturer name, camera model name, camera serial number), camera setting information at the time of shooting (shooting date, Shutter speed, aperture value, exposure program, brightness (BV value), exposure correction, shooting distance, metering mode, light source, focal length , Aperture value, flash, guide number, flash confirmation, backlight, subject position, number of shots, special effect filter, remarks, digital camera characteristics (type of image sensor, focal plane X resolution, focal plane Y resolution) , Focal plane resolution unit, spatial frequency characteristics, color filter array pattern, spectrum sensitivity, ISO speed ratings, OECF), film information (film brand, film category, film size, film roll number, film frame number), original or book Original document scan description information (original scan image size, original document size, original media, document type) and scan device information (scanner manufacturer, scanner model name, serial number, scanner software, scanner software version) John number, scan service company name, scan the person in charge ID, scan date, last calibration date and time, scanner pixel size) is the information associated with the image information, such as.
[0014]
For example, it is information that can be used when using the image, such as how the image is captured and how it can be used.
(5) Image information
The same image is composed of image data (sub-image data) 36, 39, 42 of a plurality of resolutions (35, 38, 41) and headers 37, 40, 43 of sub-image data of the respective resolutions.
[0015]
It is also possible to adopt a file configuration that does not have an image of a plurality of resolutions but includes image data of the maximum resolution and its header.
[0016]
The headers 37, 40, and 43 include the length of the header stream, the width and height of the image, the number of tiles to be described later, the tile width and height, the number of channels, the offset of the tile header table, and the length of the tile header entry. A tile header table (consisting of tile offset, tile size, compression type, and compression subtype).
[0017]
Here, the tile offset indicates the position of the tile, the tile size indicates the amount of data in the tile, and the compression type indicates whether the tile data is uncompressed image data, single color, or JPEG compressed image data. Or a tile that has no valid image data, and the compression subtype represents the color if it is a single color, and the interleave type, chroma subsample method, and internal color conversion for JPEG compression. Presence / absence, presence / absence of JPEG table change.
[0018]
FIG. 4 shows an example of an image file composed of a plurality of images having different resolutions. In FIG. 4, an image 40 having the maximum resolution is composed of columns × rows of X0 × Y0, and an image 41 having the next highest resolution is X0 / 2 × Y0 / 2. Each row is reduced by half, and the processing is repeated until both the columns and rows are equal to or smaller than M × N pixels, and the image having the minimum resolution is stored.
[0019]
As a result of such hierarchization, header information and image information are required as image attribute information for “the number of hierarchies in one image file” and images in each hierarchy. Information about the number of layers in one image file, the width and height of the image with the maximum resolution, or the width, height, color configuration, compression method, etc. of each resolution is the first attribute information 21 described above. Described.
[0020]
Further, each resolution layer image is divided into M × N pixel tiles as shown in FIG. If the image is divided into M × N pixel tiles sequentially from the upper left of the image, a blank may be generated in a part of the tile at the right end and / or the lower end depending on the image (shaded portion).
[0021]
In this case, an M × N pixel tile is configured by repeatedly inserting the rightmost pixel or the lowermost pixel.
[0022]
Each tile image is stored in a data format of JPEG compression, single color, or non-compression. JPEG compression is an image compression method internationally standardized by ISO / IEC JTC1 / SC29. The single color is a data form in which the color of the tile is expressed by one color without storing the value of each pixel when one tile is substantially composed of a single color in the previous period. It is. This form is particularly effective for images generated by computer graphics.
[0023]
As described above, the tiled image data is stored in the sub image data file, and the total number of tiles, the size of each tile, the start position of the data, the data form, etc. are all stored in the sub image header. The
[0024]
FIG. 6 shows a distribution example of the data form of each tile of the images 50, 51, 52 after the tile division of the images 40, 41, 42 in FIG. In FIG. 6, attribute 1 represents uncompressed, attribute 2 represents JPEG compression, and attribute 3 represents single color.
[0025]
If the attribute of each tile of the image with resolution 0 is determined, the attribute of each tile of the image with resolution 1 to resolution n can be easily created according to a predetermined rule.
[0026]
For example, if all four tiles with resolution 0 corresponding to one tile with resolution 1 are attribute 1 (uncompressed), the tile is attribute 1; if all tiles are attribute 3 (single color), the tile is If any one tile is attribute 2 (JPEG), the tile can be attribute 2, and if the tile is attribute 1 and attribute 3, the tile can be attribute 1.
[0027]
2. Hardware configuration
An example of the hardware configuration in this embodiment is shown in FIG.
[0028]
Reference numeral 70 denotes a CPU, which is used to execute a processing procedure described later. Reference numeral 71 denotes a RAM, which is used as a CPU work area or the like. A ROM 72 stores a program executed by the CPU. Reference numeral 73 denotes an operation unit including a display, which is used as a user interface. Reference numeral 74 denotes a hard disk which stores the above-described file format data and the like. Reference numeral 75 denotes a removable disk such as a CD-R, DVD-RAM, floppy disk or the like, which stores the above-mentioned file format data and the like. An input interface 76 is a bidirectional interface for inputting image information and other information from an input device 77 such as an image scanner, a digital camera, or a digital video camera. Reference numeral 78 denotes an output interface, which is a bidirectional interface for outputting information to an external device via communication means such as the network 79.
[0029]
3. Creating an image file
An example of the above-described image file creation procedure will be described.
[0030]
FIG. 8 shows a basic procedure executed by the CPU of FIG.
[0031]
In step S1, image information from an input device 77 such as an image scanner, a digital camera, or a digital video camera is input via the input interface 76.
[0032]
In step S2, information serving as a basis for the first to tenth attribute information associated with the image information is input. This input is performed manually via the input interface 76 or by the operation unit 73.
[0033]
In step S3, the image information input in step S1 is formatted into the image file format having the above-described structure.
[0034]
In step S4, the attribute information input in step S2 is formatted into the image file format having the above structure.
[0035]
In step S5, the formatted image information and attribute information are stored in the hard disk 74 or the removable disk 75.
[0036]
Next, the procedure for formatting image information in step S3 in FIG. 8 will be described with reference to FIG.
[0037]
In step S10, the input image information is developed into pixel data for each pixel. For example, when image information is input in a compressed format such as JPEG, decompression processing is performed, and when it is input as a PDL code, expansion processing is performed to create image data for each pixel. If it is necessary to convert the size of the image information input at this time, the size is converted by trimming or scaling. If these processes are not necessary, the process of this step is skipped.
[0038]
If it is necessary to convert the color space of the pixel data in step S20, the color space is converted to one of the predetermined color spaces such as RGB (additional primary colors) and YCC (luminance and chromaticity). I do.
[0039]
In step S30, a formatting process is performed on an image with resolution 0, which is the highest resolution. Details will be described later. When the image information is formatted with only the highest resolution, the subsequent processing is not performed.
[0040]
In step S40, the resolution is reduced to 1/2. The reduction method is formatted as the ninth attribute information described above.
[0041]
In step S50, the created resolution 1 image is formatted.
[0042]
In step S60, the same reduction process as in step S40 is performed.
[0043]
Thereafter, the reduction process and the formatting process are repeated, and in step S70, the formatting process is performed on the image of resolution n, and the process ends.
[0044]
Next, the procedure of the processing for developing pixel data in step S10 of FIG. 9 will be described with reference to FIG.
[0045]
In step S110, it is determined whether or not the input image information is non-compressed pixel data. If the input image information is non-compressed pixel data and no particular expansion processing is required, the process proceeds to step S170.
[0046]
In step S120, it is determined whether or not the input image information is encoded data. If it is encoded data, decoding is performed in step S140.
[0047]
In step S130, it is determined whether the input image information is command data such as PDL. If it is command data, command expansion processing is performed in step S150.
[0048]
If any processing is necessary for data in other formats, the processing is performed in step S160.
[0049]
In step S170, the number of bits per pixel and the image size are adjusted.
[0050]
Next, the procedure of color space conversion processing in step S20 in FIG. 9 will be described with reference to FIG.
[0051]
In step S210, a color space is set in advance manually by the operation unit 73.
[0052]
In step S220, it is determined whether or not the color space of the input image information matches the color space set in step S210. If the color space does not match, color space conversion processing is performed in step S230.
[0053]
Next, the processing procedure for the image of resolution 0 in step S20 in FIG. 9 will be described with reference to FIG.
[0054]
In step S310, the image is divided into tiles having a size of M × N pixels.
[0055]
In step S320, the attribute of the image of the divided tile is detected. For example, if a character or thin line part is included, it is a character thin line tile, and if relatively important information such as a human face part is included, it is an important tile, and the image data for each pixel in the tile is When all the data is the same (or when only a few pixels have different values), the tile is a single color, and the other tiles are halftone tiles.
[0056]
In step S330, the data form of the tile is determined according to the detected attribute. For example, the above-described character thin line tile and important tile are uncompressed with attribute 1, halftone tiles are JPEG compressed with attribute 2, and single color tiles are single color with attribute 3.
[0057]
In step S340, the image data of each tile is processed so as to have the determined data form.
[0058]
The above processing can basically be performed in the same manner for the formatting processing for the image of resolution 1 to the image of resolution n.
[0059]
Next, the procedure of the reduction process in step S40 in FIG. 9 will be described with reference to FIG.
[0060]
In step S410, a reduction method is manually set in advance by the operation unit 73.
[0061]
In step S420, reduction processing is performed on the image data of the original resolution by the set reduction method.
[0062]
4). About watermark insertion
A method for adding a watermark to an image in this embodiment will be described.
[0063]
Here, the watermark refers to information related to the image information of the above-described image file and added to the image information so that it is invisible to human eyes or difficult to see. The above-mentioned first to tenth attribute information can be inserted as a watermark here.
[0064]
An example of a watermark insertion method will be described.
[0065]
(1) The procedure of watermark insertion method I is shown in FIG. This method inserts watermark information into the target image in the real image space.
[0066]
In step S510, the above-mentioned file format attribute information stored in the hard disk 74, a removable disk 75 such as a CD-R, DVD-RAM, floppy disk or the like is selectively read out.
[0067]
In step S520, the extracted watermark information is encrypted. As the encryption method, for example, a known encryption method such as conversion using a random number table can be used.
[0068]
In step S530, the encrypted watermark information is converted into pattern data for each pixel.
[0069]
In step S540, using the obtained image data of the watermark information, a mutual operation is performed with the image data of the target image in the real image space, and the watermark information is embedded in the target image.
[0070]
At this time, addition or multiplication can be considered as the mutual operation. In any calculation, the watermark information is added so as to be difficult for human eyes to see by sufficiently reducing the amplitude of the watermark information with respect to the amplitude (dynamic range) of the target image.
[0071]
In addition, the color components to which watermark information is added are less likely to be perceived by the human eye, such as the B component of RGB (additive primary color) data and the chromaticity component of YCC (brightness and chromaticity) data. Let it be a color component.
[0072]
(2) FIG. 15 shows the procedure of watermark insertion method II. This method inserts watermark information into a target image in a frequency space.
[0073]
In step S610, the attribute information of the above-described file format stored in the hard disk 74, the removable disk 75 such as a CD-R, DVD-RAM, floppy disk or the like is selectively read out.
[0074]
In step S620, the extracted watermark information is encrypted. As the encryption method, for example, a known encryption method such as conversion using a random number table can be used.
[0075]
In step S630, the encrypted watermark information is converted into pattern data for each pixel.
[0076]
In step S640, the obtained watermark information image data is converted into frequency components using, for example, orthogonal transformation.
[0077]
In step S650, the image data of the target image is converted into a frequency component using, for example, orthogonal transformation.
[0078]
In step S660, the image data of the target image and the encrypted watermark information are synthesized in the frequency space, and the watermark information is embedded in the target image.
[0079]
In step S670, the synthesized image is inversely converted into actual image data.
[0080]
(3) A procedure for inserting the above-described watermark will be described with reference to FIG.
[0081]
In the present embodiment, the processing for the image data of each tile in step S340 in FIG. 12 is performed as follows for watermark insertion.
[0082]
In step S1000, it is determined whether the data form is attribute 1. If it is attribute 1, watermark A is inserted by method I described above in step S1010.
[0083]
In step S1020, it is determined whether the data form is attribute 2. If it is attribute 2, watermark B is inserted by method II described above in step S1030, and block coding is performed in step S1040. At this time, if the block coding is, for example, JPEG coding using orthogonal transformation, the inverse transformation processing in step S670 in FIG. 15 can be omitted, and the frequency component can be quantized and Huffman coded as it is.
[0084]
As described above, by selecting a watermark insertion method according to the storage form of image data, it is possible to insert a watermark by a method suitable for each form.
[0085]
If it is determined in step S1020 that the data form is not attribute 2, the representative value of the tile is extracted as attribute 3. At this time, a watermark is not inserted into the tile having this attribute.
[0086]
The above processing is repeated for all tiles.
[0087]
The contents of watermark A and watermark B can be the same, for example, the name of the original image producer and the date of production.
[0088]
In addition, a watermark can be inserted into each resolution image by the same contents and method as described above.
[0089]
In this embodiment, since watermark information is added in units of tiles of M × N pixels, the watermark information is encrypted so that the pattern when the pattern is developed fits in the size of M × N pixels.
[0090]
(Modification 1)
In the above-described embodiment, the contents of the watermark A and the watermark B are common, but the contents may be different from each other. That is, since it is considered that the data format of the image is 1 and 2 in a normal image, the amount of watermark information inserted as an entire image can be increased by inserting different information into both. .
[0091]
For example, the name and date of the creator of the original image can be inserted as the watermark A, and the last corrector and the date of the last correction can be inserted as the watermark B.
[0092]
(Modification 2)
In the above-described embodiment, different watermark insertion methods are used for the watermark A and the watermark B, but both may be the same watermark insertion method.
[0093]
That is, both can be inserted by either of the watermark insertion methods I or II. In this case, by using a common watermark insertion method, software and hardware for watermark insertion can be simplified.
[0094]
At this time, the watermark insertion strength can be made different between the watermark A and the watermark B by using the same insertion method.
[0095]
That is, for example, since the watermark A is inserted into the uncompressed image data, the value of the pattern data of the watermark information is relatively small in the interoperation with the image data of the target image in order to prevent deterioration of the image quality. On the other hand, since the watermark B is inserted into the image data to be compressed, the image data of the target image can be correlated with the image data so that the watermark information is not lost at the time of compression (especially when irreversible compression is used). In the calculation, the pattern data value of the watermark information is made relatively large.
[0096]
In this way, by selecting a watermark insertion method according to the storage form of the image data, the watermark can be inserted with a strength suitable for the storage form of each image data.
[0097]
(Modification 3)
In the above-described embodiment, watermark insertion according to the storage mode of image data is performed for all tiles. However, tiles for watermark insertion are selected periodically or randomly, and a part of the target image is selected. You may make it insert a watermark into a tile.
[0098]
In this case, it is possible to suppress deterioration in image quality due to watermark insertion.
[0099]
Even if the present invention is applied as a part of a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus composed of a single device (for example, a copier, a facsimile machine). You may apply to one part of.
[0100]
In addition, the present invention is not limited only to the apparatus and method for realizing the above-described embodiment, and software for realizing the above-described embodiment on a computer (CPU or MPU) in the system or apparatus. A case in which the above-described embodiment is realized by supplying a software program code and causing the system or apparatus computer to operate the various devices according to the program code is also included in the scope of the present invention.
[0101]
In this case, the program code of the software itself realizes the function of the above embodiment, and the program code itself and means for supplying the program code to the computer, specifically, the program code Is included in the scope of the present invention.
[0102]
As a storage medium for storing such a program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
[0103]
Further, the computer controls various devices according to only the supplied program code, so that not only the functions of the above embodiments are realized, but also the OS (operating system) on which the program code is running on the computer. Such program code is also included in the scope of the present invention when the above embodiment is realized in cooperation with a system) or other application software.
[0104]
Further, after the supplied program code is stored in the memory of the function expansion board of the computer or the function expansion unit connected to the computer, the program code is stored in the function expansion board or function storage unit based on the instruction of the program code. The case where the CPU or the like provided performs part or all of the actual processing and the above-described embodiment is realized by the processing is also included in the scope of the present invention.
[0105]
【The invention's effect】
As described above, according to the present invention, when image information is stored using a predetermined image storage format, the predetermined information can be added to the input image by a method suitable for the storage format.
[Brief description of the drawings]
FIG. 1 is a diagram showing the configuration of a storage format according to an embodiment of the present invention.
FIG. 2 is a diagram showing the configuration of an image file in a storage format according to an embodiment of the present application.
FIG. 3 shows the structure of a storage format image file according to an embodiment of the present invention.
FIG. 4 is a diagram showing an example of an image file composed of a plurality of images having different resolutions.
FIG. 5 is a diagram showing an example of tile division
FIG. 6 is a diagram showing a distribution example of the data form of each tile of an image after tile division
FIG. 7 is a diagram illustrating an example of a hardware configuration in an embodiment of the present application;
FIG. 8 shows a procedure for creating a file.
FIG. 9 is a flowchart showing a procedure for formatting image information.
FIG. 10 is a flowchart showing a procedure for development into pixel data;
FIG. 11 is a flowchart showing a procedure for color space conversion;
FIG. 12 is a flowchart showing a procedure for processing with resolution 0;
FIG. 13 is a flowchart showing the procedure of reduction processing;
FIG. 14 is a flowchart showing a procedure of a watermark insertion method.
FIG. 15 is a flowchart showing a procedure of a watermark insertion method;
FIG. 16 is a flowchart showing a processing procedure for each tile;
[Explanation of symbols]
70 CPU
74 hard disk
75 Removable disk

Claims (7)

Input means for inputting image data;
Storage form selection means for selecting a storage form according to the attribute of the image data;
Conversion means for converting the image data into data of the selected storage form;
An insertion method selection means for selecting a method of inserting watermark information according to the storage form;
An image processing apparatus comprising insertion means for inserting watermark information into the image data by the selected insertion method.   The image data is partial image data obtained by dividing image information, and the storage form selection unit, the conversion unit, the insertion method selection unit, and the insertion unit perform processing for each partial image data. The image processing apparatus according to claim 1, wherein:   The storage form includes a storage form as uncompressed data and JPEG compressed data, and the insertion method means selects a method for inserting watermark information in a real image space in the case of a storage form of uncompressed data, and JPEG 3. The image processing apparatus according to claim 1, wherein, in the compressed data storage form, a method for inserting watermark information in a frequency space is selected. An input process for inputting image data;
A storage form selection step of selecting a storage form according to the attribute of the image data;
Converting the image data into data of the selected storage form;
An insertion method selection step of selecting a method of inserting watermark information according to the storage form;
An image processing method comprising an insertion step of inserting watermark information into the image data by the selected insertion method.   The image data is partial image data obtained by dividing image information, and each of the storage form selection step, the conversion step, the insertion method selection step, and the insertion step performs processing for each partial image data. The image processing method according to claim 4, wherein:   The storage form includes a storage form as uncompressed data and JPEG compressed data, and the insertion method step selects a method for inserting watermark information in a real image space in the case of a storage form of uncompressed data, and JPEG 6. The image processing method according to claim 4, wherein, in the case of storing compressed data, a method for inserting watermark information in a frequency space is selected.   A computer-readable storage medium storing a computer program for executing the image processing method according to any one of claims 4 to 6.

Images