JP2009182823A - Image processor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor, etc. by which object information embedded in an electronic document as macro and script is associated with identification information and positional information about a medium, and complicated processing by the macro and the script is performed even after printing. <P>SOLUTION: This image processor 100 is constituted, characterized by providing: a document acquisition means for acquiring the electronic document which should be printed to the medium; an identification information acquisition means for acquiring the identification information about the medium; a printing instruction acceptance part 11 as a positional information acquisition means for acquiring the positional information in the medium; a PDL-for-output generation part 16 as an instruction means for instructing a printing mechanism to print superimposed images between an image of the electronic document and images of the identification information and positional information; an object information acquisition part 14 as an object information acquisition means for acquiring the object information regarding a predetermined macro or script from the electronic document which should be printed to the medium; and a registration part 15 as a registration means for registering management information for specifying the object information using the identification information and the positional information. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a program.

  In recent years, characters and pictures are drawn on special paper on which fine dots are printed, and the data written on the paper by the user is transferred to a personal computer or mobile phone. The technology that makes it possible is drawing attention.

Here, as a prior art described in the publication, a paper ID for identifying a paper document, a document paper ID related means for associating a paper ID and a paper ID assigned to an electronic document whose form is determined in advance, and paper A printing unit that obtains an ID and prints the encoded paper ID created by the encoding unit on a paper document, a paper ID management unit that manages the paper ID, and the handwritten writing information on which the encoded paper ID is printed. Information acquisition means for acquiring writing information and paper ID decoded by the decoding means from the paper document possessed, and processing for associating the processing ID indicating the processing performed on the writing information acquired by the information acquisition means with the paper ID A document processing system having a paper ID related unit and a processing ID management unit for managing a processing ID is disclosed. (For example, refer to Patent Document 1).
Further, as a known technique, there is a system that enables direct instruction electronically by mounting a directly instructable device such as a touch panel on a display.

JP 2005-122682 A

  Here, in a system using a paper document as an interface, it is required to perform not only simple processing but also complicated processing. In particular, after printing, it is generally impossible to execute a macro or script embedded in the electronic document, and it is desired to execute complicated processing even after the electronic document is printed.

  The invention according to claim 1 is an object information acquisition unit that acquires object information relating to a predetermined macro or script from an electronic document to be printed on a medium, an identification information acquisition unit that acquires identification information of the medium, and the medium And a registration means for registering management information for specifying the object information using the identification information and the position information. It is a processing device.

According to a second aspect of the present invention, the printing mechanism is instructed to print a document acquisition unit that acquires the electronic document, an image of the electronic document, an image of the identification information, and an image of the position information. The image processing apparatus according to claim 1, further comprising an instruction unit.
The invention according to claim 3 is characterized in that the registration unit registers correspondence information among the identification information of the medium, position information in the medium, and the object information as the management information. The image processing apparatus described in the above.
The invention according to claim 4 is the image processing apparatus according to claim 3, wherein the registration unit further registers the identification information of the electronic document as the management information.
The invention according to claim 5 is the image processing apparatus according to claim 1, wherein the object information is identification information related to the predetermined macro or script.

  The invention according to claim 6 is acquired by the acquisition unit that acquires the image read from the medium on which the image representing the identification information of the medium and the coordinate information indicating the position on the medium is printed, and the acquisition unit. Information detecting means for detecting the identification information of the medium and coordinate information indicating the position on the medium from the image, and the identification information of the medium detected by the information detecting means and the coordinate information indicating the position on the medium And an object information detecting means for detecting object information relating to a predetermined macro or script.

According to a seventh aspect of the present invention, the object information detecting means detects the object information based on management information including identification information of the medium, coordinate information indicating a position on the medium, and the object information. The image processing apparatus according to claim 6, wherein the image processing apparatus is characterized in that:
The invention according to claim 8 is the image processing apparatus according to claim 6, further comprising transmission means for transmitting the object information to an application program corresponding to the object information.

  According to a ninth aspect of the present invention, there is provided a computer having a function of acquiring object information relating to a predetermined macro or script from an electronic document to be printed on a medium, a function of acquiring identification information of the medium, and a position in the medium A program for realizing a function of acquiring information and a function of registering management information for specifying the object information using the identification information and the position information.

  According to a tenth aspect of the present invention, there is provided a computer having a function of acquiring the image read from the medium on which the image representing the identification information of the medium and the coordinate information indicating the position on the medium is printed; A function of detecting medium identification information and coordinate information indicating the position on the medium, and a function of detecting object information related to a predetermined macro or script from the medium identification information and the coordinate information indicating the position on the medium; It is a program for realizing.

According to the first aspect of the present invention, it becomes possible to associate object information embedded as a macro or script in an electronic document with the identification information and position information of the medium. Can be processed.
According to the second aspect of the present invention, when printing an electronic document on a medium, object information embedded as a macro or script in the electronic document can be associated with identification information and position information of the medium.
According to invention of Claim 3, registration of management information can be simplified more.
According to the invention of claim 4, the position of the object information in the original document created by the application program for executing the macro or script can be traced more reliably than in the case where the present configuration is not provided. it can.
According to the fifth aspect of the present invention, object information can be more easily registered as management information than when the present configuration is not provided.
According to the invention of claim 6, it is possible to execute a complicated process embedded as a macro or a script in an electronic document even after the electronic document is printed.
According to the invention of claim 7, the object information can be detected more easily.
According to the eighth aspect of the present invention, the function of the application program can be used by further including the transmission means for transmitting the object information in the application program corresponding to the object information.
According to the ninth aspect of the present invention, it is possible to associate the object information embedded as a macro or script in the electronic document with the identification information and the position information of the medium as compared with the case where this configuration is not provided. For example, even after printing, complicated processing using macros or scripts can be performed.
According to the invention of claim 10, compared with the case where the present configuration is not provided, a complicated process embedded as a macro or script in the electronic document is executed even after the electronic document is printed. It becomes possible.

Embodiments of the present invention will be described below with reference to the accompanying drawings.
FIG. 1 shows an example of the configuration of a system to which the present embodiment is applied. In this system, at least an image processing apparatus 100 that instructs to print an electronic document, a document repository 200 that stores the electronic document, and an image forming apparatus 400 that prints a superimposed image of an image of the electronic document and a code image are networked. It is comprised by being connected to 900. This system also includes a printed material 500 output from the image forming apparatus 400 and a pen device 600 as an example of an image processing apparatus that records characters or figures on the printed material 500 and reads the trajectory of the characters or figures. . Further, a terminal device 700 that displays the locus received from the pen device 600 and the electronic document acquired from the document repository 200 in an overlapping manner is also connected to the network 900.

The outline of the operation of this system will be described below.
First, the image processing apparatus 100 acquires an electronic document to be printed from the document repository 200 (A). Then, it instructs the image forming apparatus 400 to print this electronic document (B). At this time, the image processing apparatus 100 designates a print attribute that is a parameter related to printing. The print attributes include paper size, orientation, double-sided printing, and the like as in normal printing. In addition, regarding the code image, designation of an area in which the code image is to be printed may be included.
When receiving the instruction to print the electronic document, the image forming apparatus 400 prints an image obtained by superimposing the code image on the image of the electronic document on a medium such as paper, and outputs a printed matter 500 (C). In this case, the code image is an image of the identification code corresponding to the identification information and the position code corresponding to the position information. Alternatively, it may be an image including additional information which is other information. Note that the process of superimposing the image of the electronic document and the code image may be performed by the image processing apparatus 100 or the image forming apparatus 400.

Here, as the identification information, information for uniquely identifying each medium can be adopted. For example, it may be information obtained by combining the identification number of the image forming apparatus 400 and the serial number of printing of the medium in the image forming apparatus 400 or the date and time of printing. It may be managed information. Alternatively, information that uniquely identifies an electronic document printed on a medium may be adopted as identification information instead of information that uniquely identifies each medium.
The position information is information for specifying a coordinate position (X coordinate, Y coordinate) on each medium. For example, it is conceivable to express coordinates in a coordinate system set by taking the upper left point of the medium as the origin, taking the X axis in the right direction of the medium and the Y axis in the lower direction.
Further, the additional information includes the identification information of the user who issued the print instruction, information about whether copying is prohibited, or the like.

  Further, the image forming apparatus 400 forms the code image as an invisible image using invisible toner whose infrared light absorption rate is equal to or higher than a certain reference. On the other hand, it is preferable to form a document image of an electronic document as a visible image using visible toner having an infrared light absorption rate equal to or less than a certain standard. The difference in the absorption rate of infrared light between the toner used for forming the code image and the toner used for forming the document image is that the reading accuracy when reading the code image by irradiating the infrared light is increased. This is to ensure. In this specification, the description is based on the premise that the code image is read by infrared light irradiation, but the code image may be read by ultraviolet light.

Thereafter, it is assumed that the user has written characters or figures on the printed matter 500 using the pen device 600 (D). Thereby, the pen device 600 inputs a code image by irradiating the printed matter 500 with infrared light and detecting the reflected light. Then, information is acquired or generated from the code image, and the information is transmitted to the terminal device 700 via wired communication or wireless communication (E). The information transmitted here includes, for example, identification information of the printed matter 500 and position information of characters or figures written on the printed matter 500. Alternatively, the position information may be transmitted as trajectory information obtained by connecting the position information of characters or figures at a certain time.
Thereafter, based on the identification information received from the pen device 600, the terminal device 700 acquires an electronic document that is a source of the document image printed on the printed material 500 from the document repository 200 (F). Then, the electronic document acquired from the document repository 200 and the information acquired from the pen device 600 are superimposed and displayed.

By the way, when the identification information received from the pen device 600 is information that uniquely identifies each medium, in order to be able to acquire an electronic document based on this identification information, the identification information and the electronic document It is necessary to manage the correspondence. In FIG. 1, it is not clearly shown where to manage this correspondence, but it may be managed where it can be accessed from the terminal device 700. For example, the document repository 200 or the image forming apparatus 400 may be used.
Further, when the trajectory information is received from the pen device 600, the trajectory information is displayed superimposed on the position on the electronic document corresponding to the writing position on the printed matter 500. This is because the code image read by the pen device 600 includes writing position information, and the corresponding position in the display image of the electronic document can be specified from the information.

Although the system to which the present embodiment is applied has been described above, such a configuration is merely an example. For example, the process of superimposing the image of the electronic document and the code image may be performed by a server computer that relays a print instruction from the image processing apparatus 100 to the image forming apparatus 400. Further, the document repository 200 may be in the image processing apparatus 100. Furthermore, the image processing apparatus 100 and the terminal apparatus 700 may be the same apparatus.
In this specification, the term “electronic document” is used, but this does not mean only data obtained by digitizing a “document” including text. For example, “electronic document” includes image data such as pictures, photographs, figures, etc. (regardless of raster data or vector data) and other printable electronic data.

However, in the conventional system, only a simple routine process created in advance by a dedicated form editor and associated with a location on a physical medium or a link embedded in the location is required. Cannot be realized.
On the other hand, a system using a touch panel or the like has a problem that, when the amount of information to be displayed increases, it takes time to display a portion where a desired process is embedded because the listability is poor.

Therefore, in this embodiment, the correspondence between the object information in the electronic document and the identification information and position information of the medium on which the object information is printed is held. Here, the object information means information related to a macro or script for performing a predetermined process.
In this way, by maintaining the correspondence between the identification information, the position information, and the object information of the medium, the pen device 600 designates a specific position on the medium and traces the object information corresponding to the position. This makes it possible to execute a desired macro or script. That is, in FIG. 1, the electronic document and the locus of the pen device 600 are superimposed on the display of the terminal device 700, but if the object information is managed as in the present embodiment, the object information is traced. It is also possible to execute macros and scripts and display processing results on the display of the terminal device 700.

Here, a specific example will be given and described below.
When there is a document describing catalog information such as a list of part numbers, clicking a part number on the screen on which the application program runs will start a macro or script embedded in that part, corresponding to that part number. Suppose that there is an application program that acquires detailed information of parts from a database and displays the information in a separate window.
In this case, the correspondence between the object information related to the macro or script that calls detailed information from the database and the document identification information and the position information is held. Then, when the printed document is read by the pen device 600 and the part where the document identification information and position information are printed, a predetermined application program executes a macro or script according to the document identification information and position information, and corresponds to the part number. Detailed information of the parts to be acquired can be acquired from the database and displayed in a separate window.

If you want to do the same thing on the touch panel, you cannot get detailed information unless you display the part number you want. On the other hand, in the case of paper, since all the information is flipped through and the necessary part information page is found, the part number only needs to be acquired by the pen device 600, so the search efficiency is very high. When you want to handle a large amount of information like a catalog, you can enjoy both the good listability of paper and the good searchability of electronic information.
In addition, the dedicated form editor eliminates the need to associate the operation on the electronic document with the positional information on the medium.

In order to realize the above-described function, in the present embodiment, the image processing apparatus 100 instructs to print a superimposed image of a document image of an electronic document and a code image indicating medium identification information and position information in the medium. At this time, a correspondence relationship between the object information, the medium identification information, and the position information is created.
Hereinafter, the configuration and operation of the image processing apparatus 100 will be described.
FIG. 2 is a diagram illustrating a functional configuration of the image processing apparatus 100.
As illustrated, the image processing apparatus 100 includes a print instruction receiving unit 11, a document PDL generation unit 12, a code image generation unit 13, an object information acquisition unit 14, a registration unit 15, and an output PDL generation unit 16. And.

The print instruction reception unit 11 receives a print instruction from the user, acquires an electronic document read into the memory of the image processing apparatus 100 from the document repository 200 (see FIG. 1), and transfers the electronic document to the document PDL generation unit 12. The medium identification information (hereinafter referred to as “medium ID”) is acquired, and the position information in the medium is acquired. At the same time, a code image generation instruction for instructing generation of a code image is output to the code image generation unit 13, and an object information acquisition instruction for instructing acquisition of object information is output to the object information acquisition unit 14. The print instruction receiving unit 11 can be understood as including a document acquisition unit that acquires an electronic document, an identification information acquisition unit that acquires identification information, and a position information acquisition unit that acquires position information. .
The document PDL generation unit 12 converts the electronic document received from the print instruction reception unit 11 into a document PDL file that is a PDL (Page Description Language) file including a command sequence for the image forming apparatus 400 to perform image formation. .
When there is a code image generation instruction from the print instruction reception unit 11, the code image generation unit 13 generates a code image in a format that will be described in detail later, specifies a color, and outputs the code image to the output PDL generation unit 16.

When there is an object information acquisition instruction from the print instruction receiving unit 11, the object information acquisition unit 14 analyzes the electronic document or its additional information delivered at the same time, and acquires object information. At that time, the object information acquisition unit 14 also acquires position information of a region where the object information exists in the electronic document or a region to which the object information is added. Here, since such a region is a normal rectangle, it is represented by the coordinates of the upper left point and the lower right point of the rectangular region, which will be referred to as rectangle information. Here, the object information acquisition unit 14 can be grasped as an object information acquisition unit.
The registration unit 15 registers the object information acquired by the object information acquisition unit 14 in a storage device (not shown) as management information associated with the medium ID and the rectangular information. Here, the storage device may be provided anywhere, but is preferably provided in a computer that can be accessed in common from each device connected to the network, such as the document repository 200. Here, the registration unit 15 can be grasped as registration means for registering management information for enabling object information to be obtained based on identification information and position information.

  The output PDL generation unit 16 sets the code image and color designation information passed from the code image generation unit 13 in the document PDL file passed from the document PDL generation unit 12 as a PDL command. A command to superimpose a code image on a document) is generated and output to the image forming apparatus 400. Note that the output PDL generation unit 16 can also be understood as an instruction unit from the viewpoint of instructing the image forming apparatus 400 to print an image in which an image of an electronic document and a code image are superimposed.

  These functions are realized by cooperation between software and hardware resources. That is, the CPU (not shown) of the image processing apparatus 100 performs the functions of the print instruction receiving unit 11, the document PDL generation unit 12, the code image generation unit 13, the object information acquisition unit 14, the registration unit 15, and the output PDL generation unit 16. A program to be realized is read from an external storage device such as a hard disk into the main memory, and each of these functions is realized in the image processing apparatus 100.

Next, the operation of the image processing apparatus 100 will be described.
First, FIG. 3 shows a display screen 190 of an application program for creating, editing, printing and the like of an electronic document.
When “File” is selected from the menu bar 191 on the display screen 190, a pull-down menu 192 as shown is displayed. Of the items displayed in the pull-down menu 192, “Print (P)” is a selection item for performing normal printing without embedding object information. On the other hand, “print with object information (R)” is a selection item for embedding object information and performing printing.
Therefore, in this embodiment, an operation when “print with object information (R)” is selected will be described. That is, hereinafter, “print instruction” refers to a print instruction by selecting “print with object information (R)”. Here, the electronic document is printed as it is on the entire surface of the medium, and the image of the portion represented by the specific position information of the electronic document is printed at the position represented by the same position information in the medium. Shall be.

Next, the operation of the print instruction receiving unit 11 will be described.
FIG. 4 is a flowchart showing a flow of operation of the print instruction receiving unit 11.
First, the print instruction receiving unit 11 monitors whether there is a print instruction (step 111). That is, if there is no print instruction, step 111 is repeated, and if there is a print instruction, the print attribute is acquired (step 112). Print attributes include paper size, orientation, duplex printing, and the like. The print instruction also includes information for specifying the electronic document to be printed.
Next, the print instruction receiving unit 11 acquires a medium ID that uniquely identifies the medium on which the electronic document is printed (step 113). The medium ID can be obtained by issuing it from a predetermined server computer.

  Further, position information in the medium is generated based on the paper size and orientation information, and the medium ID, the position information, and the color designation are transmitted to the code image generation unit 13 to instruct generation of a code image (step 114). ). Here, the color designation is information for designating which color the code image is printed. If the image forming apparatus 400 has five developing units of yellow (Y), magenta (M), cyan (C), black (K), and invisible (I), the code image is printed invisible (I). This is information that should be done. On the other hand, the image forming apparatus 400 has only four developing units of yellow (Y), magenta (M), cyan (C), and black (K), and any of these colors is invisible (I). If it has been replaced, it is the designation of the replaced color. As a result, the code image is printed invisible (I).

Thereafter, the print instruction receiving unit 11 takes out the electronic document from the memory (step 115), passes the electronic document to the object information acquisition unit 14, and instructs acquisition of the object information (step 116).
Finally, the electronic document is delivered to the document PDL generation unit 12 to instruct conversion to the document PDL (step 117).

Next, the operation of the code image generation unit 13 will be described.
FIG. 5 is a flowchart showing a flow of operations of the code image generation unit 13.
First, the code image generation unit 13 monitors whether there is a code image generation instruction from the print instruction reception unit 11 (step 131). That is, if there is no code image generation instruction, step 131 is repeated, and if there is a code image generation instruction, an identification code is generated (step 132).
Specifically, identification information of a medium such as paper on which an electronic document is printed is encoded by a predetermined encoding method. For this encoding, for example, an RS (Reed Solomon) code or a BCH code which is a known error correction code can be used. Also, CRC (Cyclic Redundancy Check) or checksum value of position information can be calculated as an error detection code and added as a redundant bit.
Then, the encoded identification information is converted into a format embedded as code information. For example, the arrangement of each bit in the encoded identification information can be replaced or encrypted with a pseudo-random number so that it is difficult for a third party to decipher. When the identification code is two-dimensionally arranged, the bit value is two-dimensionally arranged in the same manner as the code arrangement.

Next, the code image generation unit 13 generates a position code (step 133).
Specifically, position information corresponding to the size of the medium on which the electronic document is printed is encoded by a predetermined encoding method. For this encoding, a method similar to that used for encoding the identification information can be used. The position information can also be expressed using an M sequence that is a kind of pseudo noise sequence. Here, the M sequence is a sequence of the maximum period that can be generated by a k-stage linear shift register, and the period is 2 ^k −1. Further, there is a property that a continuous k-bit partial sequence extracted from an arbitrary position in the M sequence appears only once in the M sequence. Due to this property, if the positional information is associated with each successive k-bit partial series, the positional information can be specified by detecting the partial series.
Then, the encoded position information is converted into a format embedded as code information. For example, the arrangement of each bit in the encoded position information can be replaced or encrypted with a pseudo-random number or the like so that it is difficult for a third party to decipher. When the position code is two-dimensionally arranged, the bit value is two-dimensionally arranged in the same manner as the code arrangement.

Thereafter, the code image generation unit 13 arranges the identification code generated in step 132 and the position code generated in step 133 in two dimensions (step 134). At this time, the same code is arranged as the identification code regardless of the position, but different codes are arranged as the position code depending on the position.
Also, a code image is generated by assigning a pattern image to be described later to each bit of the identification code and the position code (step 135).

Incidentally, the code image is a binary image at this point, but the code image generation unit 13 performs color designation for the code image (step 136). That is, since the code image generation instruction includes color designation information, color designation is performed using this information.
Finally, the code image generation unit 13 outputs the code image for which the color is designated to the output PDL generation unit 16 (step 137).

Next, a code image generated and printed by the code image generation unit 13 will be described.
FIGS. 6A to 6C are diagrams for explaining the above-described code image. FIG. 6A schematically shows a code pattern array that forms a code image. FIG. 6B is an enlarged view of a code block which is a basic unit of the code pattern array in FIG. Further, FIG. 6C is a diagram for explaining a specific pattern of unit code patterns constituting the code block.

  In the present embodiment, the code images shown in FIGS. 6A to 6C have a maximum absorption rate of, for example, 7% or less in the visible light region (400 nm to 700 nm), and the near infrared region (800 nm to 1000 nm). For example, an invisible toner having an absorptance of 30% or more. The invisible toner has an average dispersion diameter in the range of 100 nm to 600 nm in order to enhance the near infrared light absorption capability necessary for machine reading of an image. Here, “visible” and “invisible” are not related to whether they can be recognized visually. “Visible” and “invisible” are distinguished depending on whether or not an image formed on a printed medium can be recognized by the presence or absence of color development due to absorption of a specific wavelength in the visible light region. Further, “invisible” includes those that have some color developability due to absorption of a specific wavelength in the visible light region but are difficult to recognize with human eyes.

  In addition, the code image is formed as an invisible image that can be machine-readed and decoded by infrared light irradiation stably over a long period of time and can record information at high density. Furthermore, it is preferable that the image is an invisible image that can be provided in an arbitrary area regardless of the area where the visible image on the medium surface that outputs the image is provided. Furthermore, for example, an invisible image that can be recognized by a difference in gloss when visually observed is more preferable. For example, an invisible image is formed on the entire surface of the medium (paper surface) according to the size of the medium to be printed. However, “entire surface” does not mean to include all four corners of the sheet. In an apparatus such as an electrophotographic system, the area around the paper surface is usually a non-printable range, and therefore it is not necessary to print an invisible image in such a range.

In FIG. 6B, the code block is composed of a unit code pattern of 25 blocks in total of 5 blocks × 5 blocks. Each unit code pattern is formed by an area in which a total of 9 dots of 3 vertical dots and 3 horizontal dots shown in FIG. 6C can be arranged, and the code in the so-called 9C2 system in which 2 dots are arranged in the dot arrangeable area. It is a pattern. Here, the black area and the shaded area are areas where dots can be arranged, and the white area between them is an area where dots cannot be arranged. Of the areas where dots can be arranged, dots are arranged in black areas and dots are not arranged in hatched areas.
More specifically, 2 of the 9 dots that can be arranged are used, and each unit code pattern represents one piece of information. In this case, 36 (= ₉ C ₂ ) kinds of information can be expressed. By using such a method, it is possible to provide a code pattern in which noise given to a visible image is extremely small and a large amount of information can be digitized and embedded with high density.

In the present embodiment, among these unit code patterns, a specific unit code pattern is used as an information pattern, and the rest is used as a synchronization pattern. Here, the information pattern is a pattern representing information to be embedded in the medium. The synchronization pattern is a pattern used for taking out an information pattern printed on a medium. For example, it is used for specifying the position of the information pattern or detecting the rotation of the image.
In the example shown in FIG. 6B, the unit code pattern of one block on the upper left represented by the black area in FIG. 6B is used as the synchronization pattern. Then, it is necessary to prepare four types of unit code patterns as synchronization patterns in order to detect image rotation. That is, an upright synchronization pattern, a synchronization pattern rotated 90 degrees to the right, a synchronization pattern rotated 180 degrees to the right, and a synchronization pattern rotated 270 degrees to the right are prepared.
The remaining 24 blocks are used as information patterns. Then, an information pattern representing X coordinate information that specifies coordinates in the X direction on the paper surface is arranged in the four blocks to the right of the synchronization pattern, and Y coordinates that specify the Y direction coordinates on the paper surface are arranged in the four blocks below the synchronization pattern. An information pattern representing coordinate information is arranged. Further, an information pattern representing the identification information of the document printed on the paper surface or the paper surface is arranged in 16 blocks surrounded by the information pattern representing the coordinate information. In the information pattern area representing the X coordinate information and the information pattern representing the Y coordinate information, different position information is stored depending on the place where the information is arranged. On the other hand, the same identification information is stored in the area representing the identification pattern regardless of the place where it is arranged.

Next, the operation of the object information acquisition unit 14 will be described.
FIG. 7 is a flowchart showing a flow of operations of the object information acquisition unit 14.
First, the object information acquisition unit 14 monitors whether there is an object information acquisition instruction from the print instruction reception unit 11 (step 141). That is, if there is no object information acquisition instruction, step 141 is repeated, and if there is an object information acquisition instruction, the following processing is performed on each page of the electronic document delivered from the print instruction reception unit 11. The object information acquisition instruction includes the medium ID acquired by the print instruction receiving unit 11.

That is, the object information acquisition unit 14 reads the page of interest (step 142), searches the page for object information (step 143), and determines whether there is object information (step 144). As a result, if there is object information, rectangular information is generated for the area where the object information exists or added (step 145), and the correspondence between the medium ID, the rectangular information and the object information is stored in the memory (step 146). ). At this time, the object information acquisition unit 14 generates rectangular information using the coordinate system used when the print instruction reception unit 11 acquires the position information.
As described above, when the object information is searched and the object information is found, the process of storing the medium ID, the rectangle information, and the object information in association with each other is continued until the object information disappears on the page, and the object information is lost. If there is another page, it is determined (step 147).
Here, if there is another page, the processing of steps 142 to 146 is repeated, but if there is no other page, the registration unit 15 is made to correspond to the medium ID, rectangular information, and object information stored in the memory. Output (step 148).

In response to this, the registration unit 15 registers an object information table in which the medium ID, the rectangular information, and the object information are associated with each other in a storage device (not shown).
Thereafter, the registration unit 15 informs the output PDL generation unit 16 that the registration of the object information has been completed.

As a result, the output PDL generation unit 16 uses the code image and color designation information sent from the code image generation unit 13 for the code image superimposition on the document PDL sent from the document PDL generation unit 12. An output PDL is generated by inserting a PDL command.
Then, by transmitting this output PDL to the image forming apparatus 400, the image forming apparatus 400 prints a code image with invisible toner.

FIG. 8 is a diagram showing an example of the contents of the object information table registered here.
In the example shown in FIG. 8, one line of the object information table is extracted and the contents are illustrated. From the left side in FIG. 8, medium ID, intermediate document ID, rectangle information, object ID, contents of the original document, and correspondence information of the object ID in the original document are registered as management information. An original document is a document created by an application program that executes a macro or script.
Here, the intermediate document ID is identification information for identifying an electronic document to be printed. The intermediate document mediates between the original document and the printed medium, and providing this makes it easy to follow the original document. That is, since the intermediate document is created from the original document, the relationship between the original document and the intermediate document is clear. Further, since the medium to which the medium ID is assigned is printed from the intermediate document, this relationship is also clear. Therefore, by providing the intermediate document ID, it becomes easy to uniquely determine the position in the original document corresponding to the object information. When the original document has no page concept, it may be difficult to determine which position of the original document corresponds from the medium ID and the coordinate information, which is particularly effective in such a case. In the case where a concept such as a page exists in the original document and the original document can be easily traced from the medium ID, the information of the intermediate document ID is not always necessary.

FIG. 9 is a diagram illustrating the relationship among the medium to be printed, the intermediate document, and the original document.
First, medium identification information and coordinate information are printed as a code image in a portion indicated by “×”, which is a predetermined position of the medium, and this information is acquired by the pen device 600 (see FIG. 1). In this case, since the intermediate document is an electronic document printed on a medium, the coordinates where the object information exists in the intermediate document are uniquely determined. Here, the portion indicated by “x” in the intermediate document corresponds to this. Since the intermediate document is created from the original document, it can be understood to which part of the original document the portion indicated by “x” in the intermediate document corresponds. Here, the part illustrated by “x” in the original document corresponds.
Then, the application program that has obtained the information generates a mouse event using a function such as an OS (Operating System) on a screen for operating the application program, and a macro or script existing in this part is executed. Can do. Here, it is assumed that a mouse event has occurred in the portion indicated by “x” in the display on the screen of the application program. This screen may be a real screen displayed on a display or the like, but may be a virtual screen such as a frame buffer that is not displayed on the display or the like and is processed inside the application program, and is necessarily shown to the user. There is no need.

The rectangular information is preferably represented by the coordinates of the upper left point and the lower right point of the rectangular area as described above. For example, if the location of the object information is a rectangular area where the coordinates of the upper left point are (x11, y11) and the lower right point is (x12, y12), the information of this coordinate may be described.
The object ID in the medium is an ID attached to the object information in the medium to be printed, and the object ID in the original document is an ID attached to the object information in the original document, and the object information is recorded as identification information. . The object ID on the medium is preferably recorded as management information.
The contents of the original document may be the electronic information as it is, but may be reference information. When electronic information is described as it is, for example, when this content is displayed, it can be displayed reliably. However, a large storage capacity is required for the object information table. In the case of the reference information, the reliability is lowered, for example, the information cannot be acquired when the reference source is changed, but the storage capacity used for the object information table can be reduced.

  In the above-described example, the object information, the medium identification information, and the position information are displayed when instructing the printing of the superimposed image of the document image of the electronic document, the medium identification information, and the code image indicating the position information in the medium. However, the present invention is not limited to this. For example, object information, medium identification information and position information are acquired in advance, and these correspondences are created and managed. It may be registered as information. In this case, this management information may be referred to when a printing instruction is given.

Next, an image processing apparatus exemplified by the pen device 600 that reads and processes a code image formed on a paper surface will be described.
FIG. 10 is a block diagram illustrating a configuration example of the image processing apparatus.
As illustrated, the image processing apparatus includes an image reading unit 21, a dot array generation unit 22, a block detection unit 23, a synchronization code detection unit 24, a rotation determination unit 25, and a code array rotation unit 26. . In addition, an identification code detection unit 30, an identification code decoding unit 32, an identification code error detection unit 33, and an identification code error correction unit 34 are provided. Furthermore, an X coordinate code detection unit 40, an X coordinate code decoding unit 42, an X coordinate code error detection unit 43, an X coordinate code error correction unit 44, a Y coordinate code detection unit 45, and a Y coordinate code decoding unit 47. A Y coordinate code error detection unit 48, a Y coordinate code error correction unit 49, an object information detection unit 50, and a transmission unit 52.

The image reading unit 21 reads a code image printed on a paper surface using an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS).
The dot array generation unit 22 detects dots from the read code image, and generates a dot array by referring to the dot positions. Note that, as preprocessing for dot detection from the code image, processing for removing noise included in the read image is also performed. Here, the noise includes, for example, variations in image sensor sensitivity and noise generated by an electronic circuit. The type of noise removal processing should match the characteristics of the imaging system, but sharpening processing such as blurring processing and unsharp masking may be applied. In addition, dot detection is performed as follows. That is, first, a dot image portion and other background image portions are separated by binarization processing, and a dot position is detected from each binarized image position. At that time, since there are cases where a lot of noise components are included in the binarized image, it is necessary to combine filter processing for determining dots based on the area and shape of the binarized image. Thereafter, the dot array is generated by replacing the dot detected as an image with digital data on the two-dimensional array, for example, “1” for the position where the dot is and “0” for the position where there is no dot. Do. In the present embodiment, a dot array generation unit 22 is provided as an example of an acquisition unit that acquires a read image.

  The block detection unit 23 detects a block corresponding to the unit code pattern in the code block on the dot array. In other words, a rectangular block delimiter having the same size as the unit code pattern is appropriately moved on the dot array, the position where the number of dots in the block is equal is the correct block delimiter position, and the code that stores the pattern value in each block Create an array.

The synchronization code detector 24 refers to the type of each unit code pattern detected from the dot array and detects the synchronization code. The synchronization code is a code corresponding to the synchronization pattern.
The rotation determination unit 25 determines image rotation based on the detected synchronization code. For example, when a square unit code pattern is used, there is a possibility of rotating in units of 90 degrees. Therefore, the direction is detected depending on which of the four types of synchronization patterns the detected synchronization code corresponds to. Further, when a rectangular unit code pattern is used, there is a possibility that the unit code pattern is rotated by 180 degrees. Therefore, the direction is detected depending on which of the two types of synchronization patterns the detected synchronization code corresponds to.
The code array rotation unit 26 rotates the code array by the rotation angle detected by the rotation determination unit 25 and sets the code array in the correct direction.

The identification code detection unit 30 detects the identification code from the code array whose angle is corrected with reference to the position of the synchronization code. At this time, a process of rearranging the identification codes in the correct code order is also performed.
The identification code decoding unit 32 decodes the identification code using the same parameters (such as the number of blocks) used in the RS code encoding process described with reference to FIG. 10, and outputs identification information. The identification code is a code corresponding to an information pattern representing identification information.
The identification code error detection unit 33 detects an error of the decoded identification code, and the identification code error correction unit 34 corrects the error when the detected error is a correctable error.

The X coordinate code detection unit 40 detects the X coordinate code from the code array whose angle is corrected with reference to the position of the synchronization code. At this time, a process of removing the synchronization code from the X coordinate code is also performed. The X coordinate code is a code corresponding to an information pattern representing X coordinate information.
The X coordinate code decoding unit 42 extracts the M series partial sequence from the detected X coordinate code, refers to the position of this partial sequence in the M sequence used for image generation, and corrects this position with the shift amount of the code block The obtained value is output as X coordinate information.
The X coordinate code error detection unit 43 detects an error in the decoded X coordinate code, and the X coordinate code error correction unit 44 corrects the error when the detected error is a correctable error.

The Y coordinate code detection unit 45 detects the Y coordinate code from the code array whose angle is corrected with reference to the position of the synchronization code. At this time, a process of removing the synchronization code from the Y coordinate code is also performed. The Y coordinate code is a code corresponding to an information pattern representing Y coordinate information.
The Y coordinate code decoding unit 47 extracts the M series partial sequence from the detected Y coordinate code, refers to the position of this partial sequence in the M sequence used for image generation, and corrects this position with the shift amount of the code block. The obtained value is output as Y coordinate information.
The Y coordinate code error detection unit 48 detects an error of the decoded Y coordinate code, and the Y coordinate code error correction unit 49 corrects the error when the detected error is a correctable error.

The object information detection unit 50 refers to the object information table and detects object information from the identification information, the X coordinate information, and the Y coordinate information.
Then, the transmission unit 52 transmits the object information to the application program corresponding to the object information.
In this embodiment, an identification code detection unit 30, an X coordinate code detection unit 40, and a Y coordinate code detection unit 45 are provided as an example of information detection means for detecting coordinate information and identification information. Further, an object information detection unit 50 is provided as an example of the object information detection unit, and a transmission unit 52 is provided as an example of the transmission unit.
Note that the transmission unit 52 that transmits the object information to the application program is not necessarily provided in the above-described image processing apparatus, and may be performed by, for example, the terminal device 700 described with reference to FIG.

  These functions are realized by cooperation between software and hardware resources. Specifically, the CPU 91 (see FIG. 13) of the image processing apparatus performs a dot array generation unit 22, a block detection unit 23, a synchronization code detection unit 24, a rotation determination unit 25, a code array rotation unit 26, and an identification code detection unit 30. , Identification code decoding unit 32, identification code error detection unit 33, identification code error correction unit 34, X coordinate code detection unit 40, X coordinate code decoding unit 42, X coordinate code error detection unit 43, X coordinate code error correction unit 44 , A Y coordinate code detecting unit 45, a Y coordinate code decoding unit 47, a Y coordinate code error detecting unit 48, a Y coordinate code error correcting unit 49, an object information detecting unit 50, and a transmitting unit 52, for example, a magnetic disk This is realized by reading from the device 93 (see FIG. 13) into the main memory 92 (see FIG. 13) and executing it. Further, the program and data stored in the magnetic disk device 93 (see FIG. 13) may be loaded from a recording medium such as a CD, or may be downloaded via communication means such as the Internet.

Next, an outline of the operation of the image processing apparatus will be described.
First, the image reading unit 21 reads a code image of an area having a predetermined size from a medium on which the code image is printed.
Next, the dot array generation unit 22 generates a dot array in which “1” is set at a position where a dot is detected and “0” is set at a position where a dot is not detected.
Thereafter, the block detection unit 23 detects block boundaries by superimposing block delimiters on this dot array. Here, as described with reference to FIG. 6, the block is a minimum unit necessary for decoding the embedded information. In the present embodiment, a code block having a size of 5 blocks × 5 blocks is assumed. Therefore, a block delimiter having a size of 5 blocks × 5 blocks is used.

FIG. 11 is a diagram specifically showing processing when a block break is moved to detect a block. Here, it is known that encoding has been performed by the 9C2 system, and a case of decoding by the 9C2 system is shown.
First, the block detection unit 23 acquires a dot array from the dot array generation unit 22. The size of the dot array acquired here is set in advance and is (number of blocks necessary for decoding × number of dots on one side of block + number of dots on one side of block−1) ² . However, since this dot arrangement corresponds to an arbitrarily selected area of the image, the position of the block delimiter is unknown. Therefore, first, block division is performed with reference to the end of the dot array. In this example, since m = 9, block delimiters composed of blocks each having a size of 3 dots × 3 dots are overlapped. Next, the dots in each block are counted. In this example, since n = 2, the position of the block delimiter when there are two dots in each block is the correct delimiter position, but it can be seen that the number of dots varies at this position and is not correct. Therefore, the number of dots in the block is counted by shifting the block delimiter. That is, the same operation is performed for the start position in the right direction, the position moved by one dot, and the position moved by two dots. In addition, the same operation is performed for each of these positions with respect to the start position in the downward direction, the position moved by 1 dot, and the position moved by 2 dots. As a result, the number of dots in all the blocks is “2” at a position where the dot has moved one dot to the right and moved two dots downward. Therefore, this position is set as a correct separation position.

  Thereafter, the synchronization code detection unit 24, the identification code detection unit 30, the X coordinate code detection unit 40, the Y coordinate code detection unit 45, and the like refer to the dot arrangement in each block, so that the synchronization code, the identification code, the X coordinate The code and the Y coordinate code are detected.

Next, specific hardware configurations of the image processing apparatus 100 that instructs printing and the pen device 600 that realizes the image processing apparatus that reads a code image according to the present embodiment will be described.
First, the pen device 600 will be described.
FIG. 12 is a diagram showing the mechanism of the pen device 600.
As illustrated, the pen device 600 includes a control circuit 601 that controls the operation of the entire pen. The control circuit 601 includes an image processing unit 601a that processes a code image detected from an input image, and a data processing unit 601b that extracts identification information and coordinate information from the processing result there.
The control circuit 601 is connected to a pressure sensor 602 that detects a writing operation by the pen device 600 by a pressure applied to the pen tip 609. In addition, an infrared LED 603 for irradiating infrared light on the medium and an infrared CMOS 604 for inputting an image are also connected. Further, an information memory 605 for storing identification information and coordinate information, a communication circuit 606 for communicating with an external device, a battery 607 for driving a pen, and pen identification information (pen ID) are stored. A pen ID memory 608 is also connected.

  Note that the image reading unit 21 shown in FIG. 10 is realized by, for example, the infrared CMOS 604 of FIG. Further, the dot array generation unit 22 is realized by, for example, the image processing unit 601a in FIG. Further, the block detection unit 23, the synchronization code detection unit 24, the rotation determination unit 25, the code arrangement rotation unit 26, the identification code detection unit 30, the identification code decoding unit 32, the identification code error detection unit 33, the identification code illustrated in FIG. Error correction unit 34, X coordinate code detection unit 40, X coordinate code decoding unit 42, X coordinate code error detection unit 43, X coordinate code error correction unit 44, Y coordinate code detection unit 45, Y coordinate code decoding unit 47, Y The coordinate code error detection unit 48, the Y coordinate code error correction unit 49, the object information detection unit 50, and the transmission unit 52 are realized by, for example, the data processing unit 601b in FIG.

Further, the processing realized by the image processing apparatus 100 and the processing realized by the image processing unit 601a or the data processing unit 601b in FIG. 12 may be realized by a general-purpose computer, for example. Accordingly, assuming that such processing is realized by the computer 90, the hardware configuration of the computer 90 will be described.
FIG. 13 is a diagram illustrating a hardware configuration of the computer 90.
As shown in the figure, the computer 90 includes a CPU (Central Processing Unit) 91 as a calculation means, a main memory 92 as a storage means, and a magnetic disk device (HDD: Hard Disk Drive) 93. Here, the CPU 91 executes various types of software such as an OS (Operating System) and application programs to realize the above-described functions. The main memory 92 is a storage area for storing various software and data used for execution thereof, and the magnetic disk device 93 is a storage area for storing input data for various software, output data from various software, and the like. .
Further, the computer 90 includes a communication I / F 94 for performing communication with the outside, a display mechanism 95 including a video memory and a display, and an input device 96 such as a keyboard and a mouse.

  The program for realizing the present embodiment can be provided not only by communication means but also by storing it in a recording medium such as a CD-ROM.

1 shows an example of the configuration of a system to which the present embodiment is applied. It is the figure which showed the function structure of the image processing apparatus. It is the figure which showed the example of the display screen of the electronic document in this Embodiment. 5 is a flowchart showing a flow of operations of a print instruction receiving unit. It is the flowchart which showed the flow of operation | movement of the code image generation part. It is a figure for demonstrating a code image. It is the flowchart which showed the flow of operation | movement of an object information acquisition part. It is a figure which shows an example of the content of an object information table. It is a figure explaining the relationship between the medium to be printed, an intermediate document, and an original document. It is the block diagram which showed the structural example of the image processing apparatus. It is the figure which showed concretely the process at the time of moving a block break and detecting a block. It is the figure which showed the mechanism of the pen device. It is the figure which showed the hardware constitutions of the computer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Print instruction reception part, 12 ... Document PDL generation part, 13 ... Code image generation part, 14 ... Object information acquisition part, 15 ... Registration part, 16 ... Output PDL generation part, 21 ... Image reading part, 22 ... Dot Sequence generator 23, block detector 24, synchronous code detector 25, rotation determination unit 26, code array rotator 30, identification code detector 32, identification code decoder 33, identification code error detection 34: Identification code error correction unit, 40 ... X coordinate code detection unit, 42 ... X coordinate code decoding unit, 43 ... X coordinate code error detection unit, 44 ... X coordinate code error correction unit, 45 ... Y coordinate code detection , 47... Y coordinate code decoding unit, 48... Y coordinate code error detection unit, 49... Y coordinate code error correction unit, 50... Object information detection unit, 52. , 400 Image forming apparatus, 500 ... printed material, 600 ... pen device, 700 ... terminal

Claims (10)

Object information acquisition means for acquiring object information relating to a predetermined macro or script from an electronic document to be printed on a medium;
Identification information acquisition means for acquiring identification information of the medium;
Position information acquisition means for acquiring position information in the medium;
Registration means for registering management information for specifying the object information using the identification information and the position information;
An image processing apparatus comprising: Document acquisition means for acquiring the electronic document;
The image processing apparatus according to claim 1, further comprising an instruction unit that instructs a printing mechanism to print a superimposed image of the image of the electronic document, the image of the identification information, and the image of the position information. .   The image processing apparatus according to claim 1, wherein the registration unit registers correspondence information among the identification information of the medium, position information in the medium, and the object information as the management information.   The image processing apparatus according to claim 3, wherein the registration unit further registers identification information of the electronic document as the management information.   The image processing apparatus according to claim 1, wherein the object information is identification information related to the predetermined macro or script. An acquisition means for acquiring the image read from the medium on which the image representing the identification information of the medium and the coordinate information indicating the position on the medium is printed;
Information detecting means for detecting identification information of the medium and coordinate information indicating a position on the medium from the image acquired by the acquiring means;
Object information detection means for detecting object information relating to a predetermined macro or script from the identification information of the medium detected by the information detection means and coordinate information indicating the position on the medium;
An image processing apparatus comprising:   The object information detecting means detects the object information from management information including identification information of the medium, coordinate information indicating a position on the medium, and the object information. Image processing device.   The image processing apparatus according to claim 6, further comprising a transmission unit configured to transmit the object information to an application program corresponding to the object information. On the computer,
A function of acquiring object information relating to a predetermined macro or script from an electronic document to be printed on a medium;
A function of acquiring identification information of the medium;
A function of acquiring position information in the medium;
A function of registering management information for specifying the object information using the identification information and the position information;
A program to realize On the computer,
A function of acquiring the image read from the medium on which the image representing the identification information of the medium and the coordinate information indicating the position on the medium is printed;
A function of detecting identification information of the medium and coordinate information indicating a position on the medium from the image;
A function of detecting object information related to a predetermined macro or script from the identification information of the medium and the coordinate information indicating the position on the medium;
A program to realize

Images