JP6664179B2 - Information processing apparatus, information processing method and program - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program for multiplexing additional information.

  It is known to multiplex other information related to an image with image information. In the digital watermarking technique, for example, in image information such as a photograph or a picture, additional information such as a creator's name and permission / prohibition of use is multiplexed so as to be hard to visually distinguish. In addition, as an application field, in order to prevent counterfeiting of bills, stamps, securities, etc., with the high image quality of image output devices such as copiers and printers, an output device and its machine number are obtained from an image output on paper. There is known a technique for embedding additional information in an image in order to specify the information.

  Japanese Patent Application Laid-Open No. H11-163873 discloses a method for checking whether or not embedding is possible based on a region on an image to be embedded on the embedding processing side in order to guarantee that information can be extracted from an image or printed matter in which information is embedded. It is described what to do.

JP 2006-345017 A

  There are various types of additional information to be embedded in an image, and when performing multiplexing, effective multiplexing cannot be performed unless preprocessing according to the additional information is applied. For example, in a case where the additional information is audio information, compression may be performed by cutting a high frequency region using human auditory characteristics in order to obtain a capacity that can be embedded in an image. However, when the additional information is character information and the same compression processing is performed by regarding the character information as voice information, the information is lost and the original character information cannot be restored. In order to prevent such a situation, it is necessary to perform preprocessing according to the content of the additional information.

  An object of the present invention is to solve such a conventional problem. In view of the above, an object of the present invention is to provide an information processing apparatus, an information processing method, and a program that perform multiplexing by performing preprocessing according to additional information.

To solve the above problems, an information processing apparatus according to the present invention, a second acquisition for acquiring a first acquisition means for acquiring images data, the additional information of the first data format to be multiplexed into the image data Means, an analyzing means for analyzing the type of the additional information acquired by the second acquiring means, and a means for analyzing the result of the analysis by the analyzing means before multiplexing the additional information into the image data. a generating means for generating additional information having different second data format from the first data format by executing the processing to the additional information, the additional information of the second data format generated by the generation unit, the multiplexing means for multiplexing the image data acquired by the first acquisition unit, comprises a, additional information of the second data format is retrieved from the image data multiplexing is performed by the multiplexing means It is the, characterized in that.

  According to the present invention, it is possible to perform multiplexing by performing preprocessing according to additional information.

FIG. 1 is a block diagram illustrating a configuration of an image processing system. It is a block diagram which shows the structure of an additional information multiplexing part. It is a flowchart which shows the process of an additional information preprocessing apparatus. It is a flowchart which shows the process of an additional information preprocessing apparatus. FIG. 11 is a block diagram showing a configuration for performing embedding included in an additional information multiplexing unit. 5 is a flowchart illustrating processing of a quantization condition control unit. FIG. 3 is a diagram for explaining a multiplexing area. FIG. 3 is a diagram for explaining quantization conditions A and B. It is a block diagram which shows the structure of an additional information separation part. It is a figure showing a picked-up image. It is a figure showing a spatial filter. It is a figure showing a two-dimensional frequency domain. 9 is a flowchart illustrating a process from a thinning unit to a determination unit. It is a figure for explaining a thinning method. FIG. 2 is a diagram illustrating a block configuration of an image processing apparatus and a mobile terminal with a camera.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the present invention according to the claims, and all combinations of features described in the present embodiments are not necessarily essential to the solution of the present invention. . The same components are denoted by the same reference numerals, and description thereof will be omitted.

  FIG. 1 is a block diagram illustrating a configuration of the image processing system. The image processing system according to the present embodiment includes an image processing device 115 and the mobile terminal with camera 104. The image processing device 115 is a printing device having a printing function, and is, for example, an MFP (multifunctional peripheral device). The camera-equipped mobile terminal 104 is, for example, a camera-equipped mobile phone, a camera-equipped smartphone, or a tablet PC.

  In the present embodiment, the image processing device 115 prints the printed matter 112 on which the additional information is multiplexed. Then, the mobile terminal with camera 104 captures an image of the printed matter 112 by the image sensor 105 (camera). Therefore, in the present embodiment, the image processing apparatus 115 and the camera-equipped mobile terminal 104 may be communicable with each other via a wired network or a wireless network, or are communicably connected via a network or the like. You don't have to.

  The image processing device 115 includes an additional information multiplexing unit 102 that embeds additional information in a printed matter 112, and the camera-equipped mobile terminal 104 includes an additional information separating unit 106 that reads multiplexed additional information from a printed matter. The additional information multiplexing unit 102 is realized, for example, as printer driver software for creating image information to be output to the printing unit 103 (printer engine) or application software. Further, the present invention may be realized in a form incorporated in hardware such as a copier, a facsimile, a printer body, or the like as software. Further, the additional information separating unit 106 may be realized by, for example, internal application software or hardware for separating additional information from an image captured by a digital still camera. In the present embodiment, the additional information multiplexing unit 102 and the printing unit 103 are described as being included in the image processing device 115. However, the additional information multiplexing unit 102 and the printing unit 103 are divided into separate devices. Is also good. For example, an information processing apparatus such as a PC or a smartphone may include the additional information multiplexing unit 102, and an image processing apparatus different from the above information processing apparatus may include the print unit 103.

  In the present embodiment, the additional information multiplexing unit 102 analyzes an input additional information and performs preprocessing for multiplexing, as described later in FIG. 2, and a multiplexing processing unit 201. And an additional information multiplexing control unit 210 for controlling them.

[Configuration for embedding additional information]
Multi-tone image information is input from the input terminal 100, and additional information to be embedded in the image information is input from the input terminal 101. This additional information is information other than the image information, such as audio information and video information, text information, copyright related to the image, shooting date and time, shooting location, various information of the photographer, or completely different image information. is there. Further, the additional information may be included as a part of the image information configured as an image file.

  The additional information multiplexing unit 102 embeds (multiplexes) the additional information in the image information so that it is difficult to visually discriminate. The additional information multiplexing unit 102 performs multiplexing of the additional information, quantization of the input multi-tone image information, and preprocessing of multiplexing described later with reference to FIG.

  The printing unit 103 performs a printing process based on the information created by the additional information multiplexing unit 102. The printing unit 103 is a printer such as an ink-jet printer or a laser printer that realizes a gradation expression on the printed matter 112 by using pseudo gradation processing.

  FIG. 15A is a diagram illustrating a block configuration of the image processing device 115. The CPU 211 is a processor that totally controls the inside of the image processing apparatus 115. The additional information multiplexing unit 102 in FIG. 1 is realized by, for example, the CPU 211. The ROM 212 and the HDD 1501 store a basic program and a control program of the image processing apparatus 115, various applications, data, and the like. For example, the CPU 211 implements the operation of each embodiment by reading the program stored in the ROM 212 into the RAM 213 and executing the program. The RAM 213 is also used as a working memory of the CPU 211.

  The network interface (NW I / F) 1502 has a configuration according to the form of the network, such as a wired or wireless network. It is also possible to support a plurality of types of wireless networks having different communication ranges. For example, it is possible to communicate with the camera-equipped mobile terminal 104 by near field communication (NFC: Near Field Communication) having a communication distance of several cm. It is.

  The display 114 displays a setting screen, a preview screen, and the like to the user. The operation unit 1503 has, for example, a keyboard, a touch panel, and the like, and can receive an operation instruction from a user.

  A device I / F 1504 connects a printing unit 1505 (print engine) or scanning unit 1506 to the system bus 1508. The print unit 1505 corresponds to the print unit 103 in FIG. In FIG. 1, the print unit 1505 and the scan unit 1506 are shown, but other blocks such as a facsimile may be connected to the device I / F 1504 according to the functions executable by the image processing apparatus 115.

  The image processing unit 1507 performs image processing according to the use on data acquired from the outside, the scanning unit 1506, or the like. For example, the image processing unit 1507 executes processing such as color space conversion and binarization processing, and enlargement / reduction / rotation of an image according to the recording method of the printing unit 1505.

  Each block shown in FIG. 1 is communicably connected to each other via a system bus 1508. A configuration other than the configuration shown in FIG. 1 may be used. For example, the system bus 1508 and the image data bus are connected via a bus bridge. In that case, for example, the device I / F 1504 is connected to the image data bus.

  The input terminal 100 in FIG. 1 indicates a configuration input from the network I / F 1502, for example, and the input terminal 101 indicates a configuration input from the network I / F 1502 or the operation unit 1503, for example.

[Configuration for reading additional information]
The reading application executed on the camera-equipped mobile terminal 104 in FIG. 1 reads information on the printed matter 112 using the imaging sensor 105 and separates the additional information embedded in the printed matter 112 by the additional information separating unit 106. , To the output terminal 107. The output terminal 107 is an interface that outputs the acquired additional information. For example, if the information is audio information, the information is output to the speaker 108 of the camera-equipped mobile terminal 104, and if the information is image information or text information, the information is output to the display 109. . Alternatively, the data may be output to an interface that outputs data to an external device. When the camera-equipped mobile terminal 104 has a plurality of image sensors, the imaging of the printed matter 112 may be performed by the second image sensor 111.

  FIG. 15B is a diagram illustrating a block configuration of the camera-equipped mobile terminal 104. The mobile terminal with camera 104 includes a configuration of a general-purpose information processing device such as a CPU, a ROM, and a RAM. The CPU 1510 is a processor that totally controls the inside of the camera-equipped mobile terminal 104. The additional information separating unit 106 in FIG. 1 is realized by, for example, the CPU 1510. The ROM 1511 stores a basic program and a control program of the mobile terminal with camera 104, various applications, data, and the like. For example, the CPU 1510 reads the program stored in the ROM 1511 into the RAM 1512 and executes the program to implement the operation of each embodiment. The RAM 1512 is also used as a working memory of the CPU 1510.

  The network communication unit 113 has a configuration according to the form of the network, for example, wired or wireless. Also, it is possible to support a plurality of types of wireless networks having different communication ranges. For example, it is possible to communicate with the image processing apparatus 115 by near field communication (NFC: Near Field Communication) having a communication distance of several cm. is there.

  The display 109 displays each setting screen, preview screen, and the like to the user. The operation unit 1513 has, for example, hard keys and the like, and can receive an operation instruction from a user. The acceleration sensor 110 detects the attitude of the mobile terminal with camera 104. Each block shown in FIG. 15B is communicably connected to each other via a system bus 1514.

  FIG. 2 is a block diagram showing a configuration of the additional information multiplexing unit 102. The additional information preprocessing unit 200 performs a predetermined preprocessing described later on the additional information input to the input terminal 101 before multiplexing by the multiplexing processing unit 201. The multiplexing processing unit 201 embeds the additional information on which the preprocessing has been performed by the additional information preprocessing unit 200 into the image information input to the input terminal 100 and outputs it to the printing unit 103.

  2 includes a CPU 211, a ROM 212, and a RAM 213. If the processing results of the additional information preprocessing unit 200 and the multiplexing processing unit 201 result in an error, the additional information multiplexing control unit 210 stops the processing of the additional information multiplexing unit 102, and informs the operator of this. To the display 114 or the like. As described above, the CPU 211 is a processor that totally controls the inside of the image processing apparatus 115.

  The user interface unit 220 illustrated in FIG. 2 includes a display 221, a speaker 222, a keyboard 223, and the like. The display 221 corresponds to the display 114 in FIG. 15A, and is used not only for displaying a notification to the user, but also for previewing additional information such as character information and moving image information. The speaker 222 outputs audio of the additional information, ie, audio information or moving image information. The keyboard 223 corresponds to the operation unit 1503 in FIG. 15A, and is used for user instructions and selections, and for user input of character information as additional information.

  Next, the operation of the additional information preprocessing unit 200 will be described with reference to the flowchart of FIG. The additional information multiplexing unit 102 according to the present embodiment discriminately embeds character information, audio data, moving image data, and the other four types of information into image data as additional information.

  In S300, the CPU 211 determines the data type of the additional information input to the input terminal 101. The determination may be made, for example, by referring to the header information of the additional information, or may be made by receiving a designation from the user. In the present embodiment, four data types of character data, audio data, moving image data, and other types of data are determined. If the entity of the additional information cannot be specified, the processing in FIG. 3 is terminated as an error.

  FIG. 3A is a flowchart illustrating the overall processing of the additional information preprocessing unit 200. S301 to S303 show preprocessing when the data types are character data, audio data, and moving image data, respectively. If it is determined in step S300 that the data type of the additional information is character data, the process advances to step S301 to perform character preprocessing. If it is determined in S300 that the data type of the additional information is audio data, the process proceeds to S302, where audio preprocessing is performed. If it is determined in S300 that the data type of the additional information is moving image data, the process proceeds to S303, and a moving image pre-process is performed. If it is determined in S300 that the data type of the additional information is data of another type, the process proceeds to S304, where predetermined general preprocessing is performed.

  In step S305, the CPU 211 performs a preview of the additional information after the preprocessing in steps S301 to S303, or performs a preview when the additional information is audio data. This preview or preview is provided to the operator (user) as equivalent content to the additional information separated from the image by the additional information separating unit 106 described later. Therefore, the user can select whether or not to shift the processing from the additional information preprocessing unit 200 to the multiplexing processing unit 201, that is, whether or not to stop the multiplexing processing, based on the preview or the preview. it can. Note that if it is determined No in S411 of FIG. 4 described later, the process of S305 may be skipped.

  If the processing in each of steps S301 to S304 ends normally, the processing shifts to the subsequent multiplexing processing unit 201. However, if the processing is an error, the processing result of the additional information preprocessing unit 200 is an error. The processing of the additional information multiplexing unit 102 is stopped.

  FIG. 3B is a flowchart illustrating the processing of S301 when the additional information is character data. In S310, the CPU 211 analyzes information included in the additional information, and determines in S311 whether the information included in the additional information is only character data. Here, if it is determined that there is only character data, the process proceeds to S312, and if it is determined that information other than character data is included, the process of S301 is ended as an error.

  In S312, CPU 211 determines whether or not the character data includes personal information. Here, when it is determined that the personal information is included, the process proceeds to S313, and when it is determined that the personal information is not included, the process of FIG. 3B ends. In that case, the process is terminated normally and the process proceeds to S305. In S312, the CPU 211 may determine whether the information is personal information by general pattern matching, or may determine whether the information is personal information by explicit designation by the user.

  In S313, the CPU 211 encrypts the character data. The CPU 211 encrypts character data by, for example, a general encryption method. Thereafter, the processing of FIG. 3B ends. In that case, the process is terminated normally and the process proceeds to S305.

  FIG. 4A is a flowchart illustrating the process of S302 when the additional information is audio data. In S410, the CPU 211 analyzes the change amount of the audio data. The processing of S410 is intended to detect data that is not valid as additional information, such as silence or noise only. In the present embodiment, if the output value is simply a level that maintains a predetermined range along the time series, it is determined that the change amount is zero. However, in the case of audio data, the determination may be made by a method of obtaining a change in output intensity for each frequency from a power spectrum on which Fourier transform has been performed.

  In S411, the CPU 211 determines whether the data change amount analyzed in S410 is equal to or larger than a threshold. Here, if it is determined that the value is equal to or larger than the threshold, the additional information is determined to be valid data, and the process proceeds to S412. On the other hand, if it is determined that the data is less than the threshold value, it is determined that the data is not valid data, and the processing of FIG. In the present embodiment, for example, a threshold value of 10 db is used to determine whether there is a change of 10 db or more. The threshold value is not limited to this, and may be another value as long as it is possible to determine whether the additional information is valid data.

  In S412, the CPU 211 compresses the additional information with the setting for multiplexing. In the present embodiment, for example, compression is performed with the quality setting fixed at 1 (VBR) using Speex defined in RFC 5574 as the compression algorithm. In S413, the CPU 211 calculates the data capacity of the audio data compressed in S412.

  In S414, the CPU 211 determines whether the data capacity calculated in S413 is equal to or larger than a threshold. In the present embodiment, for example, it is determined whether the data capacity after compression is 4 kbytes or more. This is because the area required for multiplexing processing by the multiplexing processing unit 201 described later increases as the data capacity increases. As a result of the determination, if the data capacity is not more than 4 kbytes, the additional information can be embedded in the image. Thus, the process of S302 is completed, and the compressed additional information is sent to the multiplex processing unit 201 as preprocessed additional information. On the other hand, if it is 4 kbytes or more, the process proceeds to S415.

  In the present embodiment, when it is determined that the additional information is 4 kbytes or more, it is determined that the entity of the additional information cannot be embedded in the image, and the entity of the additional information is uploaded to a predetermined cloud server on the network. I do. At this time, link information is embedded in the image information as storage destination information instead of the additional information.

  In S415, the CPU 211 performs a compression process for uploading to the cloud server. In the present embodiment, compression is performed at 128 kbps of AAC defined by ISO / IEC.

  In S416, the CPU 211 uploads the additional information compressed in S415 to a predetermined server. The uploading server may be set in advance, or may be designated by the operator at this timing. When the upload is completed, the CPU 211 acquires the link information, ends the process of S302, and sends the link information to the multiplexing processing unit 201 as preprocessed additional information. In the present embodiment, Speex and AAC are used for audio compression, but an algorithm of another compression method may be used.

  After the process in S416, the process in FIG. 4A ends. In that case, the process is terminated normally and the process proceeds to S305.

  FIG. 4B is a flowchart illustrating the processing of S303 or S304 when the additional information is moving image data or general-purpose data. In the present embodiment, the processes of S303 and S304 differ in the choices of the upload destination described later, but the other operations are the same, and therefore will be described together.

  In S420, the CPU 211 calculates the data capacity of the additional information. In S421, it is determined whether the data capacity calculated in S420 is equal to or larger than a threshold. Here, the determination threshold is, for example, 4 kbytes. When it is determined that the data capacity is not more than 4 kbytes, it is determined that the additional information can be embedded in the image, the processing of S303 or S304 is terminated, and the multiplex processing unit 201 uses the additional information as preprocessed additional information. Send to On the other hand, if it is determined that it is 4 kbytes or more, the process proceeds to S422.

  In S422, similar to S416, the additional information is uploaded to a predetermined server, and link information is acquired as storage destination information. At this time, in the case of the process of S303, a shared server dedicated to a moving image can also be selected as a server option. When the upload is completed, the CPU 211 obtains the link information, ends the processing of S303 or S304, and sends the link information to the multiplexing processing unit 201 as preprocessed additional information.

  After the process of S422, the process of FIG. 4B ends. In this case, the process ends normally, and if the process in FIG. 4B corresponds to S303, the process proceeds to S305. When the processing in FIG. 4B corresponds to S304, the processing in FIG. 3 ends, and the processing shifts to the multiplex processing unit 201.

  FIG. 5 is a block diagram showing a configuration for performing embedding included in additional information multiplexing section 102 in FIG. The error diffusion processing unit 500 converts the image information input to the input terminal 100 into a quantization level smaller than the number of input gradations by performing pseudo gradation processing using a general error diffusion method. The gradation is expressed in terms of area by the quantized values of a plurality of pixels.

  The blocking unit 501 divides the image information input to the input terminal 100 into units of a predetermined area (block). The blocking performed by the blocking unit 501 may be partitioned into rectangles or may be partitioned into regions other than rectangles.

  Based on the additional information pre-processed by the additional information pre-processing unit 200 input at the input terminal 503, the quantization condition control unit 502 determines a quantization condition for each area blocked by the blocking unit 501. Control to change. Further, the quantization condition control unit 502 changes the quantization condition in block units based on the additional information input to the input terminal 101.

  Next, the entire process including the quantization condition control unit 502 will be described with reference to the flowchart of FIG. An example in which the quantization value is binary will be described. 6 is realized, for example, by the CPU 211 of the image processing device 115 reading out a program stored in the ROM 212 into the RAM 213 and executing the program.

  In S601, the CPU 211 initializes the variable i secured in the RAM 213. Here, the variable i is a variable for counting addresses in the vertical direction. In S602, the CPU 211 initializes the variable j secured in the RAM 213. Here, the variable j is a variable for counting addresses in the horizontal direction. Subsequently, in S603, the CPU 211 determines whether or not the coordinates (i, j) that are the current processing address belong to the area where the multiplexing process is to be executed.

  The multiplex area will be described with reference to FIG. FIG. 7 shows one image in which the number of horizontal pixels is WIDTH and the number of vertical pixels is HEIGHT. In the present embodiment, it is assumed that additional information is multiplexed in the image. Blocking is performed with N pixels horizontally and M pixels vertically with the upper left of the image as the origin. In the present embodiment, the block is formed using the origin as the reference point, but a point far from the origin may be set as the reference point. In order to multiplex the maximum information in this image, N × M blocks are arranged from the reference point. That is, assuming that the number of blocks that can be arranged in the horizontal direction is W and the number of blocks that can be arranged in the vertical direction is H, W and H are calculated from equations (1) and (2).

W = INT (WIDTH / N) (1)
H = INT (HEIGHT / M) (2)
Here, INT () indicates an integer part in ().

  The number of surplus pixels that cannot be divided in Equations (1) and (2) corresponds to the end when a plurality of N × M blocks are arranged, and in the present embodiment, that part is outside the code multiplexing area.

  In S603 of FIG. 6, if it is determined that the current pixel of interest is not within the multiplexing area, that is, outside the multiplexing area, the CPU 211 sets the quantization condition C in S604. On the other hand, if it is determined that the area is within the multiplexing area, in step S605, the CPU 211 reads additional information to be multiplexed. Here, for ease of explanation, it is assumed that the additional information is expressed one bit at a time using an array called code []. For example, assuming that the additional information is information for 48 bits, each bit from code [0] to code [47] is stored in the array code [].

  In step S <b> 605, the CPU 211 substitutes information in the array code [] into a variable bit secured in the RAM 213 as in Expression (3).

bit = code [INT (i / M) × W + INT (j / N)] (3)
In S606, the CPU 211 determines whether or not the assigned variable bit is “1”. As described above, since the information in the array code [] is stored one bit at a time, the value of the variable bit also indicates either “0” or “1”.

  Here, when it is determined to be “0”, the CPU 211 sets the quantization condition A in S607, and when it is determined to be “1”, in S608, the CPU 211 proceeds to step S608. Is set.

  Next, in S609, the CPU 211 performs a quantization process based on the quantization conditions set in S604, S607, and S608. This quantization process is performed by an error diffusion method.

  In S610, the CPU 211 counts up the horizontal variable j, and in S611, determines whether the counted variable j is less than WIDTH, which is the number of horizontal pixels of the image. Here, when it is determined that the variable j is less than WIDTH, the processing from S603 is repeated. On the other hand, when it is determined that the variable j is not less than WIDTH, that is, when the horizontal processing has been completed for the number of WIDTH pixels, the CPU 211 counts up the vertical variable i in S612. Then, in S613, the CPU 211 determines whether or not the counted variable i is less than HEIGHT, which is the number of vertical pixels of the image. Here, when it is determined that the variable i is less than HEIGHT, the processing from S602 is repeated. On the other hand, if it is determined that the variable i is not less than HEIGHT, that is, if the processing in the vertical direction has been completed for the number of HEIGHT pixels, the processing in FIG. 6 ends. Through the above processing, the quantization condition is changed in units of blocks each including N × M pixels.

  Next, examples of the quantization conditions A, B, and C will be described. There are various factors in the quantization condition in the error diffusion method. In the present embodiment, the factor of the quantization condition is a quantization threshold. Since the quantization condition C set in S604 is used outside the multiplexing area, any condition may be used as the quantization threshold. As described above, in the case where one pixel is represented by gradation of 8 bits and the quantization level is binary, the maximum value “255” and the minimum value “0” are the quantization representative values. However, the intermediate value “128” is often set as the quantization threshold. Therefore, in the present embodiment, the quantization condition C is a condition that sets the quantization threshold to a fixed value of “128”.

  Since the quantization condition A set in S607 and the quantization condition B set in S608 are used in blocks in the multiplexing area, it is necessary to cause a difference in image quality due to a difference in the quantization condition. However, it is necessary that the difference in image quality be expressed so as to be hard to visually discriminate, and that it can be easily identified from the paper.

  FIGS. 8A and 8B are diagrams for explaining the quantization conditions A and B. FIG. FIG. 8A is a diagram illustrating a cycle of a change in the quantization threshold under the quantization condition A. In the figure, one cell is assumed to be one pixel, a white cell is a fixed threshold, and a gray cell is a variable threshold. That is, in the example of FIG. 8A, a matrix of 8 pixels in the horizontal direction and 4 pixels in the vertical direction is formed, and a value protruding only for gray cells is set as a threshold value.

  FIG. 8B is a diagram similarly showing a cycle of a change in the quantization threshold under the quantization condition B. In the example of FIG. 8B, unlike FIG. 8A, a matrix of 4 pixels horizontally and 8 pixels vertically is set, and a value protruding only for gray cells is set as a threshold value.

  As described above, when one pixel has an 8-bit gradation value, for example, “128” is set as a fixed threshold, and “10” is set as a prominent threshold. When the quantization threshold value decreases, the quantization value of the target pixel tends to be “1” (quantization representative value “255”). That is, in both FIGS. 8A and 8B, the quantization value “1” is easily arranged in the arrangement of the gray cells in the figure. In other words, for each block of N × M pixels, a block in which dots occur in the arrangement of gray cells in FIG. 8A and a block in which dots occur in the arrangement of gray cells in FIG. It will be mixed.

  A slight change in the quantization threshold in the error diffusion method does not significantly affect the image quality. In the systematic dither method, the image quality of gradation expression largely depends on the dither pattern used. However, in the error diffusion method in which the quantization threshold value is regularly changed as described above, since the gradation expression for determining the image quality is the error diffusion method, the arrangement of the dots slightly changes, and A change in the occurrence hardly affects the image quality of the gradation expression. Even when the quantization threshold value changes, an error that is a difference between the signal value and the quantization value is diffused to surrounding pixels, so that the input signal value is stored macroscopically. That is, it can be said that the arrangement of dots and the occurrence of texture in the error diffusion method have extremely high redundancy.

  As described above, in the present embodiment, multiplexing is realized by superimposing a predetermined periodicity representing a code on the quantization threshold value of the error diffusion method. However, multiplexing may be realized by another superposition method. For example, multiplexing may be realized by a method in which periodicity is directly superimposed on RGB values (luminance information). Alternatively, multiplexing may be realized by a method of separating the RGB values into other color space information (for example, CIE L * a * b, YCrCb signals) such as luminance-color difference information and superimposing periodicity. Alternatively, multiplexing may be realized by a method of separating RGB values into ink colors (for example, CMYK signals) and superimposing periodicity.

[Reading additional information]
Next, the processing of the additional information separating unit 106 in the image processing system of FIG. 1 will be described. FIG. 9 is a block diagram illustrating a configuration of the additional information separation unit 106. For ease of explanation, similar to the case of the above-described additional information multiplexing unit 102, an example in which additional information is separated from a printed material 112 in which additional information of each bit is multiplexed in a divided block. explain. Naturally, the amount of additional information per block in the additional information multiplexing unit 102 and the amount of separated information per block in the additional information separating unit 106 are equal.

  Image information read by the mobile terminal with camera 104 is input to the input terminal 900. Here, the resolution (imaging resolution) of the imaging sensor 105 of the camera-equipped mobile terminal 104 is preferably equal to or higher than the printing resolution when creating the printed matter 112. Needless to say, in order to accurately read the dotted information of the dots of the printed matter 112, the imaging sensor 105 needs to have twice or more the resolution than the printer due to the sampling theorem. However, if they are equal or higher, it is possible to determine that dots are scattered to some extent, if not accurately. In the present embodiment, it is assumed that the printer resolution and the resolution of the image sensor 105 are the same for ease of explanation.

  The geometric shift detecting unit 901 detects a geometric shift of an image captured by the camera-equipped mobile terminal 104. Since the image information transmitted from the input terminal 900 has been output by the printing unit 103 and photographed by the camera-equipped mobile terminal 104, the image information may be geometrically shifted from the image information before the printer output. Therefore, the geometric deviation detecting unit 901 detects a boundary between the printed matter 112 and a part other than the printed matter 112 by edge detection.

  FIG. 10 is a diagram illustrating an example of a captured image. If the print resolution and the resolution of the image sensor 105 are the same, the rotation direction of the image (due to the skew of the print unit 103 when recording on paper, the shift when the camera-equipped mobile terminal 104 is held over the printed matter 112, and the like). Inclination) is a major factor to be corrected. Therefore, by detecting the boundary line of the printed matter 112, it is determined how much the rotational direction has shifted.

  The blocking unit 902 performs blocking in units of horizontal P pixels and vertical Q pixels. Here, each block is smaller than N × M pixels that are divided into blocks when the digital watermark is superimposed. That is, the relationship of Expression (4) holds.

P ≦ N and Q ≦ M (4)
Blocking in units of P × Q pixels is skipped at certain intervals. That is, the block is formed such that one block of P × Q pixels is included in an area assumed to be a block composed of N × M pixels at the time of multiplexing. The number of skip pixels is basically N horizontal pixels and M vertical pixels, but the shift amount detected by the geometric shift detection unit 901 is calculated by the number of blocks and the shift amount per block and the skip pixel. A correction is made based on the number.

  Spatial filters 903 and 904 indicate spatial filters A and B having different characteristics, respectively, and a filtering unit 905 indicates a digital filtering unit that calculates a sum of products with neighboring pixels. Each coefficient of this spatial filter is set in accordance with the cycle of the fluctuation threshold value of the quantization condition at the time of multiplexing. Here, it is assumed that the additional information is multiplexed by changing the quantization condition in the additional information multiplexing unit 102 using the two types of periodicity shown in FIGS. 8A and 8B. FIGS. 11A and 11B show examples of the spatial filter A 903 and the spatial filter B 904 used in the additional information separating unit 106 in that case. In FIGS. 11A and 11B, the central portion of 5 × 5 pixels is a target pixel, and the other 24 pixels are peripheral pixels. In FIGS. 11A and 11B, the pixels in the blank portion indicate that the filter coefficient is “0”. As is clear from FIG. 11, FIGS. 11A and 11B are edge enhancement filters. Moreover, the directionality of the edge to be emphasized and the directionality of the fluctuation threshold value when multiplexing are the same in FIGS. 11A and 11B and FIGS. 8A and 8B. That is, the spatial filter is created such that FIG. 11A matches FIG. 8A and FIG. 11B matches FIG. 8B.

  Each of the thinning units 906 and 907 thins out a signal after filtering (hereinafter, referred to as a conversion value) in a block including P × Q pixels based on a certain regularity. In the present embodiment, the thinning process is performed separately for each of the periodicity and the phase regularity. That is, the thinning units 906 and 907 have different periodicities of thinning, and each of them performs a plurality of thinning processes with different phases. The thinning method will be described later.

  The conversion value addition unit 908 adds the conversion values thinned out by the thinning units 906 and 907 for each phase. The thinning process and the process of adding the conversion values of the thinned pixels correspond to extracting power of a predetermined frequency vector emphasized by the spatial filter.

  The variance value calculation unit 909 calculates a variance value of a plurality of addition values added for each phase in each periodicity. The determining unit 910 determines the multiplexed code based on the variance value in each periodicity.

  FIG. 12 is a diagram illustrating a two-dimensional frequency domain. The horizontal axis indicates the frequency in the horizontal direction, and the vertical axis indicates the frequency in the vertical direction. The origin, which is the center, indicates a DC component, and becomes higher in frequency as the distance from the origin increases. The circle in FIG. 12 indicates a cutoff frequency due to error diffusion. The filter characteristics of the error diffusion method indicate the characteristics of an HPF (high-pass filter) in which a low-frequency region is cut off, and the cut-off frequency changes according to the density of the target image.

  In the present embodiment, the change in the quantization threshold changes the frequency characteristic generated after the quantization. However, the change in the quantization threshold according to FIG. 8A results in a large power spectrum on the frequency vector A in FIG. In addition, when the quantization threshold is changed according to FIG. 8B, a large power spectrum is generated on the frequency vector B in FIG. At the time of additional information separation, a multiplexed signal is determined based on detecting a frequency vector at which this large power spectrum occurs. In the present embodiment, each frequency vector is individually emphasized and extracted.

  FIGS. 11A and 11B correspond to an HPF having a directionality of a specific frequency vector. That is, in the spatial filter of FIG. 11A, it is possible to emphasize the frequency vector on the straight line A of FIG. 12, and in the spatial filter of FIG. It becomes possible to emphasize the vector. For example, it is assumed that a large power spectrum is generated on the frequency vector of the straight line A in FIG. 12 due to the change of the quantization condition as shown in FIG. At this time, the spatial filter of FIG. 11A amplifies the amount of change in the power spectrum, but the spatial filter of FIG. 11B hardly amplifies it. That is, when a plurality of spatial filters are filtered in parallel, amplification is performed only at the time of the spatial filter whose frequency vector matches, and when filtering is performed by other filters, there is almost no amplification. Therefore, it is possible to easily determine on which frequency vector a large power spectrum is generated.

  FIG. 13 is a flowchart showing processing of the thinning units 906 and 907, the conversion value adding unit 908, the variance value calculating unit 909, and the determining unit 910 in FIG. The process of FIG. 13 is realized, for example, by the CPU 1510 of the camera-equipped mobile terminal 104 reading a program stored in the ROM 1511 into the RAM 1512 and executing the program.

  In FIG. 13, S1301 and S1302 indicate the initialization of the variables, and the CPU 1510 initializes the values of the variables i and j secured in the RAM 1512 to 0.

  In step S1303, the CPU 1510 determines factors of the regularity of the thinning performed by the thinning units 906 and 907, that is, two factors of “periodicity” and “phase”. In this flowchart, a variable relating to the periodicity is represented by i, and a variable relating to the phase is represented by j. The conditions of the periodicity and the phase are managed by numbers (numbers). Here, the periodicity number (hereinafter abbreviated as No.) is i, and the phase number. Set the factor of the thinning method where is j.

  In S1304, the CPU 1510 adds the conversion values thinned out in the block, and stores the added value as a variable array TOTAL [i] [j].

  In S1305, the CPU 1510 counts up the variable j, and compares the variable j with the fixed value J in S1306. In J, the number of times of performing the thinning process by changing the phase is stored. Here, if the variable j is less than J, the process returns to S1303, and the new phase No. based on j after the count-up. Accordingly, the thinning process and the process of adding the conversion values of the thinned pixels are repeated.

  When the set number of times of the decimation process with the phase shifted and the addition process of the conversion values of the decimation pixels are completed, in S1307, the CPU 1510 calculates the variance value of the addition result TOTAL [i] [j]. That is, it is evaluated how much each addition result varies due to the phase difference. Here, i is fixed, and the variance of J TOTAL [i] [j] is obtained. Here, the variance value is B [i].

  In S1308, the CPU 1510 counts up the variable i, and compares the variable i with the fixed value I in S1309. In I, the number of times of performing the thinning process by changing the periodicity is stored. Here, if the variable i is smaller than I, the process returns to S1302, and the new periodicity No. based on i after the count-up. Using the condition (1), the thinning process and the process of adding the conversion values of the thinned pixels are repeated.

  In S1309, if the CPU 1510 determines that i has completed the set number of times, it means that I variance values B [i] have been calculated. In S1310, the maximum value of the variance value is detected from the set of I variance values, and the value of i at that time is substituted for a variable imax. In S1311, the CPU 1510 determines the periodicity No. Is determined to be a multiplexed code. Thereafter, the processing in FIG. 13 ends.

  Hereinafter, an example in which I = 2 and J = 4 will be described. FIG. 14 is a diagram for explaining a thinning-out method when the block size is P = Q = 16, and is shown in a table format. In FIG. 14, one cell in the block represents one pixel. In FIG. 14, the block shape is a square of P = Q, but the block shape is not limited to a square, and may be other than a rectangle.

  FIG. FIG. 14B shows a thinning method (corresponding to the thinning unit A906 in FIG. 9) in the case of = 0. 10 shows a thinning method (corresponding to the thinning unit B907 in FIG. 9). In the figure, the value shown for each pixel in the block is the phase No. J indicates a thinned pixel. For example, a pixel displayed as “0” corresponds to a thinned pixel when j = 0. That is, in each of FIGS. 14A and 14B, there are four types of phases. This corresponds to a thinning method when j is 0 to 3.

  The periodicity in FIG. 14A matches the periodicity in FIG. 8A, and the periodicity in FIG. 14B matches the periodicity in FIG. 8B. As described above, in both FIGS. 8A and 8B, the quantization values “1” (in the case of binary values of “0” and “1”) are easily arranged in the arrangement of gray cells in the figure. Become. Therefore, for example, in the case of a block having the quantization condition A at the time of multiplexing, the quantization value “1” is easily arranged with the periodicity of FIG. When filtering is performed by applying a suitable spatial filter, the frequency component is further amplified, and the thinning process of the converted value and the adding process of the converted value of the thinned pixel are performed with the periodicity of FIG. Then, the variance value of the addition result increases.

  On the other hand, the block having the quantization condition A is filtered by applying an incompatible spatial filter, and the thinning process and the addition process of the conversion value of the thinned pixel are performed by the periodicity of FIG. Then, the variance value of the result of adding the converted values becomes smaller. This is because, since the periodicity of the quantized value is different from the periodicity of the thinning, the added value of the converted value due to the difference in the phase of the thinning is averaged, and the variation is reduced. Similarly, when the conversion value thinning process and the addition process of the conversion value of the thinned pixel are performed with respect to the block having the quantization condition B at the time of the multiplexing with the periodicity of FIG. Becomes smaller. On the other hand, in this case, if the conversion value thinning process and the addition process of the conversion value of the thinned pixel are performed with the periodicity of FIG. 14B, the variance value of the addition result increases.

  As described with reference to FIG. 6, bit = 0 is set as the quantization condition A, and bit = 1 is set as the quantization condition B. Therefore, the periodicity No. When the variance of “= 0” is large, it can be determined that bit = 0, and the periodicity No. When the variance value of = 1 is large, it can be determined that bit = 1.

  As described above, multiplexing and demultiplexing can be easily realized by associating the quantization condition, the spatial filter characteristic, and the periodicity of the thinning condition. In the present embodiment, the periodicity No. Are two types, 0 and 1, and the multiplexing code in the block is 1 bit. However, the multiplexing code may be more than one bit. Naturally, the type of the quantization condition, the type of the spatial filter, and the periodicity No. of the thinning condition. (The value of I).

  According to the present embodiment, the multiplexed code can be easily separated without comparing the power of the frequency corresponding to the regularity of the quantization condition by the orthogonal transform. Moreover, since the processing is performed in the real space area, the separation processing can be realized at an extremely high speed.

  Although the present embodiment has been described above, the quantization conditions A and B, the spatial filters A and B, and the thinning units A and B are merely examples, and the present invention is not limited thereto. It may be performed for other periodicity, and the number of taps of the spatial filter, the block size of the thinning, and the like may be larger or smaller than the example in the present embodiment.

  In the processing of FIG. , And the phase i. Has been described. However, it may be realized by repetitive processing using pixel addresses in a block composed of P × Q pixels. That is, as shown in FIGS. 14A and 14B, the periodicity No. is assigned to each pixel address in the block. And phase No. Are stored in advance as a table, and the corresponding periodicity No. is stored. And phase No. Is a method of adding the conversion value to each of the variables. In this method, only the processing for P × Q pixels is performed, and the periodicity No. And phase No. Can be calculated in parallel.

  Further, in the process of FIG. 13, the variance value of the addition result of the conversion values of the thinned pixels after filtering by the spatial filter is calculated, and the sign is determined by comparing the variance values, but the present invention is not limited to this. . The determination may be made by comparing the evaluation functions without using the variance value. The bias of the addition result of the decimated conversion values tends to protrude only at one phase when the phase is shifted, so that it is only necessary to evaluate the “degree of variation”.

  For example, to evaluate the degree of variation, for example, the following evaluation function may be used in addition to the variance value.

  1. The difference between the maximum value and the minimum value of the added values obtained by adding the thinned conversion values.

  2. Either the difference between the maximum value of the added value obtained by adding the thinned conversion values and the second largest value, or the difference between the minimum value and the second smallest value.

  3. The maximum value of the difference in the front and rear order when creating a histogram based on the added value obtained by adding the thinned conversion values.

  Although the above three evaluation functions are absolute difference values, a relative ratio between these difference values and a converted value or a sum of pixel values or converted values may be used as the evaluation function. Also, the quantization value has been described by taking binarization as an example, but is not limited to this.

  As described above, according to the present embodiment, by analyzing the contents of the additional information to be embedded in the image and switching the contents of the preprocessing according to the analysis result, it is possible to execute the preprocessing suitable for the additional information. it can.

[Other Examples]
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

  102 additional information multiplexing unit: 104 portable terminal with camera: 106 additional information separation unit: 115 image processing device

Claims (17)

First acquisition means for acquiring image data;
Second acquisition means for acquiring additional information in a first data format to be multiplexed with the image data;
Analysis means for analyzing the type of the additional information acquired by the second acquisition means,
Before multiplexing the additional information into the image data, a process according to a result analyzed by the analysis unit is performed on the additional information, so that the additional information in a second data format different from the first data format. Generating means for generating
Multiplexing means for multiplexing the additional information in the second data format generated by the generating means with the image data obtained by the first obtaining means,
Obtaining additional information in the second data format from the image data that has been multiplexed by the multiplexing unit;
An information processing apparatus characterized by the above-mentioned. The additional information of the first data format to be multiplexed with the image data is character data,
2. The information processing apparatus according to claim 1, wherein the additional information in the second data format obtained from the image data that has been multiplexed by the multiplexing means is encrypted character data. The additional information in the first data format multiplexed with the image data is audio data,
2. The information according to claim 1, wherein the additional information in the second data format obtained from the image data multiplexed by the multiplexing unit is link information indicating a storage location of the audio data. Processing equipment. The additional information of the first data format to be multiplexed with the image data is moving image data,
2. The information according to claim 1, wherein the additional information in the second data format acquired from the image data that has been multiplexed by the multiplexing unit is link information indicating a storage location of the moving image data. Processing equipment. The analyzing means determines a type of the additional information in the first data format,
The generating means generates additional information in the second data format different from the first data format by performing a process corresponding to the determined type on the additional information.
The information processing apparatus according to claim 1, wherein:   4. The information processing apparatus according to claim 3, wherein when the capacity of the audio data is equal to or larger than a threshold, the generation unit generates link information indicating a location where the audio data is stored. 5.   The information processing apparatus according to claim 6, wherein when the volume of the audio data is equal to or larger than the threshold, the audio data is stored in the storage destination.   The information processing apparatus according to claim 4, wherein when the capacity of the moving image data is equal to or larger than a threshold, the generating unit generates link information indicating a location where the moving image data is stored.   The information processing apparatus according to claim 8, wherein when the capacity of the moving image data is equal to or larger than the threshold, the moving image data is stored in the storage destination.   The information processing apparatus according to claim 3, wherein the storage destination is a device on a network.   The information processing apparatus according to claim 10, wherein the generation unit compresses the additional information in the first data format, and stores the compressed additional information in the storage destination. When the type of the additional information is determined to be audio data by the analysis means,
4. The information processing apparatus according to claim 3, further comprising: a determination unit configured to determine whether or not the audio data is to be multiplexed with the image data based on a change amount of the audio data. 5. apparatus. When the type of the additional information is determined to be character data by the analysis means,
The information processing apparatus according to claim 2, wherein the generating unit generates encrypted character data from the character data corresponding to personal information.   2. The information processing apparatus according to claim 1, further comprising a preview unit that previews the additional information in the second data format.   15. The apparatus according to claim 1, further comprising a control unit configured to cause a printing unit to print image data in which the additional information in the second data format is multiplexed by the multiplexing unit. Information processing device.   The information processing apparatus according to any one of claims 1 to 15, wherein the information processing apparatus is a printing apparatus. An information processing method executed in the information processing apparatus,
A first acquisition step of acquiring image data;
A second acquisition step of acquiring additional information in a first data format to be multiplexed with the image data;
An analyzing step of analyzing a type of the additional information acquired in the second acquiring step;
Before multiplexing the additional information with the image data, a process according to a result analyzed in the analysis step is performed on the additional information, so that the additional information in a second data format different from the first data format. A generating step of generating
A multiplexing step of multiplexing the additional information in the second data format generated in the generating step with the image data obtained in the first obtaining step,
The additional information in the second data format is obtained from the multiplexed image data.
An information processing method, comprising:

Images