JP2023140646A - Information processing device, information processing method, information processing program, and information processing system - Google Patents

Abstract

To provide an information processing apparatus, an information processing method, an information processing program and an information processing system in which confirmation of falsification existence/non-existence of a document such as an order sheet is realized in a low cost and high accuracy way compared with a conventional way.SOLUTION: In an information processing system 1 in which an information processing server 5 and a MFP 9 are connected to a communication network 100 and can communicate with each other, the information processing server 5 working as an information processing apparatus comprises: a first storage unit (order sheet data storage unit) that stores image data of an estimation document; a second storage unit (positional information storage unit) that stores positional information which corresponds to a desired object domain designated on the estimation document by associating with the image data; and an estimation unit that extracts first object data corresponding to the object domain from the image data stored in the first storage unit on the basis of the positional information stored in the second storage unit and performs estimation processing of existence/non-existence of a falsification trace of the estimation document.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing device, an information processing method, an information processing program, and an information processing system.

Order forms are generally used in commercial transactions between companies. Documents may be falsified during the creation of this order form. "Document falsification" refers to falsification (physical falsification) in which a part of a paper document is modified by gluing a part of another paper, etc. when the document is paper; data), there is falsification (electronic falsification) in which a part of the image data document is processed by electronically copying (pasting) a part of other image data. In any of the above cases, confirmation of whether or not the order form document has been tampered with is performed manually by visual inspection by a person in charge. Therefore, a large amount of cost (time cost and human cost) is required, and confirmation accuracy also varies.

It would be beneficial if it were possible to check whether or not an order form document has been tampered with at a lower cost and with higher accuracy than in the past.

The present invention has been made in view of the above circumstances, and provides an information processing device, an information processing method, and an information processing program that can check whether or not documents such as order forms have been tampered with at lower cost and with higher accuracy than before. and information processing system.

In order to solve the above-mentioned problems and achieve the purpose, an information processing device according to an embodiment of the present invention includes a first storage unit that stores image data of an estimation document, and a first storage unit that stores image data of an estimation document; a second storage unit that stores positional information corresponding to the desired target area that has been scanned in association with the image data; The estimation unit includes an estimating unit that extracts first target data corresponding to the target area from the stored image data and performs a process of estimating whether or not there is a trace of tampering in the estimation document.

According to the present invention, it is possible to realize an information processing device, an information processing method, an information processing program, and an information processing system that can check whether a document such as an order form has been tampered with at lower cost and with higher accuracy than before. can.

FIG. 1 is a diagram illustrating the configuration of an example of an information processing system according to an embodiment. FIG. 2 is a hardware configuration diagram of the information processing server according to the embodiment. FIG. 3 is a hardware configuration diagram of the MFP according to the embodiment. FIG. 4 is a block diagram regarding the functional configuration of the information processing server and MFP according to the embodiment. FIG. 5 is a diagram showing an example of the surface of an order form used for the target area setting process. FIG. 6 is a diagram showing an example of the back side of an order form used for the target area setting process. FIG. 7 is a diagram showing an example of a target area set to include a signature field by marking on an order form. FIG. 8 is a diagram showing an example of electronic traces of falsification of the signature field of an order form. FIG. 9 is a diagram showing another example of electronic traces of falsification of the signature field of the order form shown in FIG. 5. FIG. 10 is a flowchart showing the flow of target area setting processing executed in the MFP. FIG. 11 is a flowchart showing the flow of trace estimation processing executed in the information processing server.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an information processing system according to the present invention will be described in detail below with reference to the accompanying drawings. FIG. 1 shows the configuration of an example of an information processing system 1 according to an embodiment of the present invention. The information processing system 1 includes an information processing server 5 and an MFP (Multifunction Peripheral/Product/Printer) 9. Information processing server 5 and MFP 9 are connected to communication network 100 and can communicate with each other.

The information processing system 1 executes document tampering estimation processing to estimate whether or not a document has been tampered with. Here, "falsification of a document" refers to falsification (physical falsification) in which a part of a paper document is modified by gluing a part of another paper, etc. when the document is a paper document, data (image data), this includes any falsification (electronic falsification) in which part of the image data is electronically copied (pasted) or otherwise processed into a part of the document of the image data. shall be held.

Further, in the following description, in order to make the explanation more concrete, an example will be described in which the document for estimation is an order form, and the target for estimation of whether or not falsification has been made is the signature field of the order form.

The document tampering estimation process executed by the information processing system 1 includes a trace estimation process executed by the information processing server 5, and a target area setting process executed by the MFP 9 as preprocessing for the trace estimation process. Note that the MFP 9 is an example of an image processing device. Each process will be explained in detail later.

The information processing server 5 is, for example, a general computer. FIG. 2 is a hardware configuration diagram of the information processing server 5. As shown in FIG. The hardware configuration of the information processing server 5 will be explained below.

As shown in FIG. 2, the information processing server 5 is constructed by a computer, and includes a CPU 501, ROM 502, RAM 503, HD 504, HDD (Hard Disk Drive) controller 505, It includes a display 506, an external device connection I/F (Interface) 508, a network I/F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I/F 516. .

Among these, the CPU 501 controls the operation of the information processing server 5 as a whole.

The ROM 502 stores programs used to drive the CPU 501 such as IPL. RAM 503 is used as a work area for CPU 501. The HD 504 stores various data such as programs.

The HDD controller 505 controls reading and writing of various data to the HD 504 under the control of the CPU 501.

The display 506 displays various information such as a cursor, menu, window, characters, or images. External device connection I/F 508 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like.

The network I/F 509 is an interface for data communication using the communication network 100.

The bus line 510 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501 shown in FIG. 2.

Further, the keyboard 511 is a type of input means that includes a plurality of keys for inputting characters, numerical values, various instructions, and the like.

The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like.

The DVD-RW drive 514 controls reading and writing of various data on a DVD-RW 513, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may be DVD-R or the like.

The media I/F 516 controls reading or writing (storage) of data to a recording medium 515 such as a flash memory.

The MFP 9 is a digital multifunction device that has multiple functions such as a scanner, a printer, and a FAX.

FIG. 3 is a hardware configuration diagram of the MFP 9. As shown in FIG. 3, the MFP (Multifunction Peripheral/Product/Printer) 9 includes a controller 910, a short-range communication circuit 920, an engine control section 930, an operation panel 940, and a network I/F 950.

Of these, the controller 910 includes a CPU 901, which is the main part of the computer, a system memory (MEM-P) 902, a north bridge (NB) 903, a south bridge (SB) 904, an ASIC (Application Specific Integrated Circuit) 906, and a storage section. The NB 903 and the ASIC 906 are connected by an AGP (Accelerated Graphics Port) bus 921.

Among these, the CPU 901 is a control unit that performs overall control of the MFP 9.

NB903 is a bridge for connecting CPU901, MEM-P902, SB904, and AGP bus 921, and is a memory controller that controls reading and writing to MEM-P902, a PCI (Peripheral Component Interconnect) master, and AGP target. has.

The MEM-P 902 consists of a ROM 902a that is a memory for storing programs and data that realize each function of the controller 910, and a RAM 902b that is used as a memory for developing programs and data, and for drawing when printing the memory. Note that the program stored in the RAM 902b is configured to be provided as an installable or executable file recorded on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD. You may.

SB904 is a bridge for connecting NB903, PCI devices, and peripheral devices.

The ASIC 906 is an IC (Integrated Circuit) for image processing that includes hardware elements for image processing, and has the role of a bridge that connects the AGP bus 921, the PCI bus 922, the HDD 908, and the MEM-C 907, respectively. This ASIC906 includes a PCI target and an AGP master, an arbiter (ARB) that is the core of the ASIC906, a memory controller that controls the MEM-C907, and multiple DMACs (Direct Memory Access Controllers) that rotate image data using hardware logic, etc. , and a PCI unit that transfers data between the scanner section 931 and the printer section 932 via the PCI bus 922. Note that the ASIC 906 may be connected to a USB (Universal Serial Bus) interface or an IEEE 1394 (Institute of Electrical and Electronics Engineers 1394) interface.

MEM-C907 is a local memory used as a copy image buffer and code buffer.

The HD 909 is a storage for storing image data, font data used during printing, and forms. The HD 909 controls data reading or writing to the HD 909 under the control of the CPU 901 .

AGP bus 921 is a bus interface for graphics accelerator cards proposed to speed up graphics processing, and can speed up graphics accelerator cards by directly accessing MEM-P902 with high throughput. .

Further, the short-range communication circuit 920 includes a short-range communication circuit 920a. The short-range communication circuit 920 is a communication circuit such as NFC or Bluetooth.

Further, the engine control section 930 includes a scanner section 931 and a printer section 932.

The operation panel 940 also includes a panel display section 940a such as a touch panel that displays current setting values, a selection screen, etc., and accepts input from the operator, as well as displaying setting values for image forming conditions such as density setting conditions. It is provided with an operation section 940b consisting of a numeric keypad for accepting instructions, a start key for accepting copy start instructions, and the like.

The controller 910 controls the entire MFP 9, and controls, for example, drawing, communication, input from the operation panel 940, and the like. The scanner section 931 or the printer section 932 includes image processing sections such as error diffusion and gamma conversion.

Note that the MFP 9 can sequentially switch and select the document box function, copy function, printer function, and facsimile function using the application switching key on the operation panel 940. When the document box function is selected, the mode becomes document box mode, when the copy function is selected, the mode becomes copy mode, when the printer function is selected, the mode becomes printer mode, and when the facsimile mode is selected, the mode becomes facsimile mode.

Further, the network I/F 950 is an interface for performing data communication using the communication network 100. Near field communication circuit 920 and network I/F 950 are electrically connected to ASIC 906 via PCI bus 922.

FIG. 4 is a block diagram regarding the functional configurations of the information processing server 5 and the MFP 9. As shown in FIG. Note that FIG. 4 illustrates, for convenience, functions related to the embodiment of the present invention among the functions included in the functional configurations of the information processing server 5 and the MFP 9. As shown in the figure, the MFP 9 includes an image data acquisition section 9a, a position information acquisition section 9b, and an information transmission section 9c. The information processing server 5 includes a control section 5a, an information reception section 5b, an order form data storage section 5c, a position information storage section 5d, an estimation section 5e, and a display section 5f.

If the order form is a paper order form, the image data acquisition unit 9a uses the function of the scanner unit 931 of the MFP 9 to acquire image data of the order form marked with a signature field. Further, if it is an order form for image data, the image data acquisition unit 9a reads the image data of the order form from the HD 909 or the like.

The position information acquisition unit 9b sets an area (target area) to be subjected to trace estimation processing on the image data of the order form based on the area marked to include the signature field on the order form, and sets Obtain the location information of the target area. This position information acquisition section 9b is configured by, for example, a CPU 901, a system memory 902, and a scanner section 931.

FIG. 5 is a diagram showing an example of the surface of an order form used for the target area setting process. FIG. 6 is a diagram showing an example of the back side of an order form used for the target area setting process.

As shown in Figure 5, the front of the order form includes the name of the company you are ordering from, the order number, the order date, the details of the ordered product (product name, quantity, amount, etc.), and the transaction conditions (payment method, payment amount, etc.).・Payment date, etc.) are listed. The orderer's address, company name, position, name, and seal mark (marked in a circle) are written in the box surrounded by a square frame in the upper right corner of the front, and this area is generally called the signature field. This is within the range of Further, as shown in FIG. 6, important matters etc. are written on the back side of the order form.

FIG. 7 is a diagram showing an example of a target area set to include a signature field by marking M in an order form.

For example, in the case of a paper order form, it is assumed that a marking M is set on the paper order form using a marking pen as shown in FIG. In such a case, the position information acquisition unit 9b sets a target area R based on the marking M on the image data generated by scanning the paper order form, and acquires position information of the target area.

Further, for example, in the case of an order form for image data, it is assumed that a marking M is set on the order form for image data using drawing software or the like, as shown in FIG. In this case, the position information acquisition unit 9b sets a target area based on the marking M drawn on the order form of the image data, and acquires position information of the target area.

Here, various methods can be used to set the target area based on the marking M on the image data. For example, the outermost contour of the marking M can be extracted, and a closed region surrounded by pixels corresponding to the extracted contour can be set as the target region. Further, a predetermined closed area (for example, a circle, an ellipse, a polygon, etc.) circumscribing the marking M can be set, and the closed area can be used as the target area. Also, the uppermost point (pixel) and the lowermost point (pixel) of the marking M in the vertical direction of the document, and the rightmost point (pixel) of the marking M in the horizontal direction of the document. A desired closed area can be set as a target area using the leftmost point (pixel) as a reference. In either case, the positional information of the target area can be, for example, the positional information of each pixel located at the outermost side of the closed area.

Furthermore, in the example shown in FIG. 7, the signature field is surrounded by a rectangular frame. Therefore, in addition to marking with a wide line, for example, by setting multiple points outside the signature field frame, the user can use each point as a reference to mark a predetermined closed area including the signature field frame (for example, the signature An area obtained by enlarging the frame of a column) can also be set as the target area.

In addition, for example, with the upper left of the order form as the origin O (0, 0), input indicating the positional information of the upper left point A (xa, ya) and the lower right point B (xb, yb) of the signature field is accepted from the user. By executing the target area setting process, a predetermined closed area (for example, a rectangular area with points A and B as the vertices of the diagonal line) including the frame of the signature line based on points A and B is set as the target area. It can also be set.

The information transmitting unit 9c transmits the image data of the order form and the position information of the target area associated with the image data to the information processing server 5 via the communication network 100. This information transmitter 9c is configured by, for example, a CPU 901 and a network I/F 950.

The control unit 5a performs overall control over the operation of the information processing server 5. The control unit 5a controls the information receiving unit 5b, the order form data storage unit 5c, the position information storage unit 5d, the estimation unit 5e, and the display unit 5f in the trace estimation process. This control unit 5a is configured by, for example, a CPU 501, a ROM 502, a RAM 503, and the like.

The information receiving unit 5b receives the image data of the order form and the position information of the target area associated with the image data from the information transmitting unit 9c of the MFP 9 via the communication network 100. This information receiving section 5b is composed of a CPU 501 and a network I/F 509.

The order form data storage section 5c stores the image data of the order form (order form data) received by the information receiving section 5b. This order form data storage section 5c is composed of a ROM 502, an HD 504, and the like. Note that the order form data storage section 5c is an example of a first storage section.

The position information storage unit 5d associates the position information corresponding to the desired target area specified on the order form, which is received by the information reception unit 5b and performed by the MFP 9, with the image data of the corresponding order form. Remember. This position information storage section 5d is composed of a ROM 502, an HD 504, and the like. Note that the position information storage section 5d is an example of a second storage section.

The estimation unit 5e extracts image data (target data) corresponding to the target area from the image data stored in the order form data storage unit 5c based on the position information stored in the position information storage unit 5d. Furthermore, the estimation unit 5e receives the extracted target data as input, executes an estimation process for estimating whether there is any trace of tampering with the order form, and outputs the estimation result. The estimation unit 5e is typically an AI (Artificial Intelligence) such as a trained neural network, and uses a trained model (estimated model) such as a DNN (Deep Neural Network) or a CNN (Convolutional Neural Network). It is something to be prepared for. Note that the estimation result of estimating the presence or absence of traces of tampering in the order form output by the estimation unit 5e is an example of information regarding tampering of the estimation document.

Here, the training (learning method) of the estimation model of the estimation unit 5e will be explained. The estimation model of the estimator 5e is trained using teacher data generated based on the image feature amount of the second target data, which is image data corresponding to the target area set in the training document.

FIG. 8 is a diagram showing an example of electronic traces caused by physical falsification of the signature field of an order form. More specifically, FIG. 8 shows the paper after being tampered with by physically cutting out the signature field from another paper order form and pasting it on top of the signature field of the paper order form (physical tampering). This shows image data created by scanning an order form.

When the order form cut and pasted in this way is read by the scanner unit 931 of the MFP 9, image noise called a shadow appears as an electronic trace along the periphery of the cut and pasted part due to uneven illumination caused by the step on the document surface (see Figure 8). part of closed region C). That is, the image data of the order form after tampering includes electronic traces caused by physical tampering. Note that the electronic trace caused by physical tampering is an example of first trace data.

FIG. 9 is a diagram showing another example of electronic traces caused by electronic falsification of the signature field of an order form. That is, FIG. 9 shows an example of an electronic trace generated in the image data after cutting and pasting (electronic tampering) in the signature field of an order form as image data.

In the case of such electronic falsification, cutting and pasting of data may result in missing frame borders, missing seal marks, distorted seals, differences in the size of characters and lines, and differences in fonts in the order form of the image data after the tampering. Inconsistencies such as These inconsistencies do not exist in the image data of the order form that has not been tampered with, but exist as electronic traces caused by electronic tampering in the image data of the order form that has been tampered with. Note that the electronic trace caused by electronic tampering is an example of second trace data.

In the training of the estimation unit 5e, electronic traces caused by physical tampering and electronic traces caused by electronic tampering are used. In other words, it is determined whether the target data extracted from the image data of the training order form includes electronic traces caused by physical tampering or electronic traces caused by electronic tampering as image features, and the learning process is performed. Generate training data for. Note that electronic traces caused by physical tampering and electronic traces caused by electronic tampering are examples of image feature amounts.

Specifically, a plurality of order forms for image data in which the target area (in this case, the area including the signature field) has not been tampered with and a plurality of order forms for image data in which the target area has been tampered with are prepared. Correct answer data (teacher data) indicating whether or not the target data extracted from each image data includes electronic traces of tampering is created. Then, a group of training data is created as a training data group, with the target data as input data and the presence or absence of electronic traces of tampering in the target data as output data.

By performing supervised learning of the learning model using the training data group created in this way, it is possible to generate a trained model that implements the estimation unit 5e. Specifically, the DNN network parameters included in the learning model are initialized. The gradient of the loss function is calculated using the training data group and the current parameters of the learning model using, for example, error backpropagation. Update the network parameters of the current learning model based on the calculated slope of the loss function. A trained model can be generated by repeatedly performing similar gradient calculations using the updated network parameters and fixing the network parameters when the gradient converges.

In this embodiment, the estimation unit 5e is realized by the CPU 501 executing a dedicated program on the RAM 503. However, the present invention is not limited to this example, and part or all of the estimator 5e may be implemented using dedicated hardware designed to perform similar functions, such as a semiconductor integrated circuit such as an ASIC or an FPGA (Field Programmable Gate Array). It may be realized by a circuit, a conventional circuit module, or the like.

The display unit 5f displays the result of the trace estimation process performed by the estimation unit 5e. This display section 5f is composed of a CPU 501 and a display 506.

(Target area setting process)
FIG. 10 is a flowchart showing the flow of target area setting processing. The target area setting process will be described below with reference to FIG.

As shown in FIG. 10, first, the signature field of the order form is marked (step S1). For example, in the case of a paper order form, the signature field is marked with a marking pen, such as by artificially surrounding it. If it is an order form for image data, the signature field is marked by artificially surrounding it using drawing software or the like. Note that the marking in step S1 automatically marks the area including the signature field using character recognition processing using image data order forms (including those generated by scanning paper order forms) as input. You may also do so.

The area setting application is started on the MFP 9 (step S2). By starting this area setting application, the image data acquisition section 9a, position information acquisition section 9b, and information transmission section 9c of the MFP 9 are realized.

The image data acquisition unit 9a acquires image data of the order form on which the signature field was marked in step S1 (step S3). That is, if the order form is a paper order form, the image data acquisition unit 9a uses the scanning function of the MFP 9 to acquire image data of the order form with the signature field marked. Further, if the order form is for image data, the image data acquisition unit 9a acquires the image data processed in step S1 from a predetermined storage unit.

The location information acquisition unit 9b sets a target area for trace estimation processing on the image data of the order sheet acquired in step S3 based on the marking in the signature field in step S1, and acquires position information (currently) of the set target area. In this case, information for specifying the position of a closed area including a signature field on the image data is acquired (step S4). The acquired position information is stored in a predetermined storage unit in association with the image data of the corresponding order form.

The information transmitter 9c transmits the position information acquired in step S4, together with the image data of the corresponding order form, to the information processing server 5 via the communication network 100 (step S5). The information receiving unit 5b of the information processing server 5 receives the position information and order form image data transmitted from the MFP 9 via the communication network 100. The received position information and order form image data are stored in the position information storage section 5d and the order form data storage section 5c, respectively.

(Trace estimation processing)
FIG. 11 is a flowchart showing the flow of trace estimation processing executed in the information processing server 5. The trace estimation process will be described below with reference to FIG.

As shown in FIG. 11, first, a detection application is launched in the information processing server 5 (step S11). By starting this detection application, for example, the estimation unit 5e is realized as a function of the detection application. Note that the configuration in which the estimation unit 5e is implemented by a detection application is an example, and the estimation unit 5e does not necessarily need to be implemented as a function of an application, and may be implemented in another software format.

The estimating unit 5e reads the image data of the selected order form from the order form data storage unit 5c and the position information associated with the image data of the selected order form from the position information storage unit 5d (step S12). .

The estimation unit 5e extracts target data from the image data of the selected order form based on the read position information (step S13), executes estimation processing using the target data as input, and determines the estimation result (presence or absence of traces). (estimation result) is output (step S14). The output result of the estimation process is displayed on the display section 5f.

If it is estimated in the estimation process that there are "trace marks" (step S14), the selected order form is visually inspected to see if it has been tampered with (step S15). On the other hand, if it is estimated in the estimation process that there is "no trace" (no trace in step S14), it is determined that the selected order form has not been tampered with and the process ends.

As described above, according to the information processing server 5 as the information processing device according to the embodiment of the present invention, the estimation unit 5e uses image data corresponding to the target area set in the order form (estimation document). Acquire some first target data. The estimation unit 5e inputs the first target data and outputs the presence or absence of a trace of tampering as information regarding falsification of the order form.

Therefore, by setting a target area for the image data of the order form and inputting the target data into the estimation unit 5e as an estimation model, the user can automatically determine whether or not the order form has been tampered with. can be grasped. Further, compared to the case where it is determined whether or not an order form has been tampered with solely by human judgment, it is possible to suppress variations in the determination system for whether or not an order sheet has been tampered with. As a result, it is possible to check whether or not a document such as an order form has been tampered with at lower cost and with higher accuracy than in the past.

Furthermore, the estimation unit 5e generates training data generated based on the image feature amount (electronic trace data) in second target data, which is image data corresponding to a target area set in an order form as a training document. trained using Teacher data for training the estimation model as the estimation unit 5e can be easily created. Therefore, the user can relatively easily train the estimation model corresponding to the estimation unit 5e.

Further, the electronic trace data as an image feature amount is the first trace data included in the second target data due to the physical trace in the training document (paper order form) as the physical medium document, and second trace data included in the second target data due to the electronic trace of the training document (image data order form) as an electronic medium document. Therefore, either paper or image data order forms can be subjected to estimation processing.

(Modification 1)
In the trace estimation process, the estimation unit 5e of the information processing server 5 according to the embodiment described above outputs the presence or absence of a trace of tampering as information regarding the falsification of the order form. In response, the estimating unit 5e determines the types of tampering traces (i.e., shadows caused by physical cutting and pasting, missing frame ruled lines, missing seal marks, distorted seals, characters and lines) as information regarding the tampering of the order form. (differences in size, differences in font, etc.) may be output.

The training data group for the estimation model as the estimation unit 5e includes, for example, the target data extracted from the image data of each order form, including shadows caused by physical cutting and pasting in the target area, missing border lines, missing seal marks, etc. Correct data is created that indicates whether or not individual electronic traces exist, such as distortions in stamps, differences in the size of letters and lines, and differences in fonts. Then, a group of training data can be created in which the target data is input data and the type of electronic trace of tampering in the target data is output data.

(Modification 2)
In the embodiment described above, the case where AI such as a trained neural network is used as the estimation unit 5e of the information processing server 5 has been exemplified. On the other hand, it is also possible to use an estimator 5e that performs rule-based trace estimation using image feature amounts. In such a case, the estimation unit 5e calculates the similarity of image feature amounts between the target data and a plurality of prepared electronic trace patterns, and compares the obtained similarity with a threshold value. The presence or absence of electronic traces in the target area can be estimated.

(Modification 3)
In the above embodiment, the case where the estimation document is an order form has been described as an example. The tampering determination process according to the embodiment is not limited to the format of the order form. That is, as long as the target area can be set in the order form, the order form may have any format. Further, the target area is not limited to the area including the signature field, but can be set as any area on the image data.

(Modification 4)
In the embodiment described above, the case where the information processing server 5 includes the estimator 5e has been described as an example. On the other hand, a configuration may be adopted in which the MFP 9 includes the estimation unit 5e and executes the trace estimation process. Alternatively, the estimation unit 5e may be provided in a computer on the cloud, and the result of the trace estimation process may be obtained by communicating with the computer. Furthermore, the information processing server 5 may be configured to have the functions of the MFP 9.

Further, the MFP 9 described in the above embodiment is an example of an image processing device. That is, the image processing device having the functions of the image data acquisition unit 9a, location information acquisition unit 9b, information transmission unit 9c, etc. described in the above embodiment is not limited to the MFP, and may be, for example, a stand-alone scanner, a smartphone with a camera, or a camera-equipped smartphone. This can also be realized using a tablet type terminal or the like.

Note that the program executed by the information processing server 5 of this embodiment is provided as being incorporated in a ROM or the like in advance.

The programs executed by the information processing server 5 of this embodiment are files in an installable or executable format and are stored on a computer such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk). It may also be configured to be recorded on a readable recording medium and provided as a computer program product.

Furthermore, the program executed by the information processing server 5 of this embodiment may be stored on a computer connected to a network such as the Internet, and may be provided by being downloaded via the network. Further, the program executed by the information processing server 5 of this embodiment may be provided or distributed via a network such as the Internet.

The program executed by the information processing server 5 of this embodiment has a module configuration including the above-mentioned parts, and the actual hardware is such that the CPU (processor) reads the program from the ROM and executes it. Each part is loaded onto the main memory, and each part is created on the main memory.

(Other application examples)
Although the best mode for carrying out the present invention has been described above using examples, the present invention is not limited to these examples in any way, and various modifications can be made without departing from the gist of the present invention. and substitutions can be added.

For example, in the configuration example shown in FIG. 4, the information processing server 5 and the MFP 9 (an example of an image processing device) are divided according to their main functions in order to facilitate understanding of the processing. The present invention is not limited by the method of dividing the processing units or the names thereof. The processing of the information processing server 5 and the MFP 9 (an example of an image processing device) can also be divided into more processing units depending on the processing content. Furthermore, one processing unit can be divided to include more processing.

Each function of the embodiments described above can be realized by one or more processing circuits. Here, the term "processing circuit" as used herein refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function explained above. It includes devices such as ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), and conventional circuit modules.

Additionally, the devices described above are merely indicative of one of multiple computing environments for implementing the embodiments disclosed herein. In some embodiments, information processing server 5 includes multiple computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network, shared memory, etc., to perform the processes disclosed herein.

1 Information processing system 5 Information processing server 5a Control unit 5b Information receiving unit 5c Order form data storage unit 5d Position information storage unit 5e Estimation unit 5f Display unit 9 MFP
9a Image data acquisition unit 9b Location information acquisition unit 9c Information transmission unit 501 CPU
502 ROM
503 RAM
504 HD
505 HDD controller 506 Display 507 External device connection I/F
509 Network I/F
511 Keyboard 512 Pointing device 513 DVD
514 DVD-RW drive 515 Media 516 Media I/F
901 CPU
902 MEM-P
902a ROM
902b RAM
903NB
904SB
906 ASIC
907 MEM-C
908 HDD controller909 HD
920 Near field communication circuit 921 Scanner section 922 Printer section 930 Engine control section 931 Scanner section 932 Printer section 940 Operation panel 940a Panel display section 940b Operation section 950 Network I/F

Japanese Patent Application Publication No. 11-41450

Claims (8)

a first storage unit that stores image data of the estimation document;
a second storage unit that stores position information corresponding to a desired target area specified on the estimation document in association with the image data;
Based on the position information stored in the second storage unit, first target data corresponding to the target area is extracted from the image data stored in the first storage unit, and the estimation document is extracted from the image data stored in the first storage unit. an estimation unit that performs a process of estimating the presence or absence of tampering traces;
An information processing device equipped with The estimation unit outputs an estimation result of the estimation process.
The information processing device according to claim 1. The estimation unit is trained using training data generated based on image features in second target data corresponding to the target area set in the training document when executing the estimation process. Using a trained model,
The information processing device according to claim 1 or 2. The image feature amount is calculated based on electronic trace data included in the second target data.
The information processing device according to claim 3. The electronic trace data includes first trace data included in the second target data due to a physical trace in the training document as a physical medium document, and first trace data included in the training document as an electronic medium document. at least one of second trace data included in the second target data due to an electronic trace of the document;
The information processing device according to claim 4. Extracting target data corresponding to the target area from the image data of the estimation document based on position information corresponding to a desired target area specified on the estimation document,
performing a process of estimating whether or not there are any traces of tampering in the estimation document;
An information processing method with to the computer,
Extracting target data corresponding to the target area from the image data of the estimation document based on position information corresponding to a desired target area specified on the estimation document,
performing a process of estimating whether or not there are any traces of tampering in the estimation document;
An information processing program that executes. a first device that extracts target data corresponding to the target area from image data of the estimation document based on position information corresponding to a desired target area specified on the estimation document;
a second device that performs a process of estimating whether there is a trace of tampering in the estimating document;
An information processing system equipped with

Images