JP7512798B2 - Information processing device and computer program - Google Patents

Description

The present invention relates to an information processing device and a computer program.

Various methods have been proposed to assist in creating layouts for magazines, books, newspapers, etc. Patent Document 1 discloses an information processing device that extracts multiple pieces of content from a document, determines the position of each piece of content on the document based on the degree of semantic relevance between the extracted pieces of content, and generates a new document in which the content is placed in the determined position.

JP 2009-169536 A

The information processing device of Patent Document 1 calculates the degree of similarity between contents in a document according to the degree of matching of the text written in the contents, and arranges the contents based on the calculated degree of similarity. However, since the content in a document is not limited to text, but also includes other content such as figures, the information processing device of Patent Document 1 may not be able to arrange related content with a high degree of accuracy.

The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device and computer program that can accurately determine the relevance of content within a document.

An information processing device according to an embodiment of the present invention includes an acquisition unit that acquires document data including a plurality of pieces of content, an identification unit that identifies categories of the plurality of pieces of content included in the acquired document data, a generation unit that generates a set of content for the identified category, a first map generation unit that generates a set of first maps in which an area corresponding to the content for each category and other areas are imaged with different pixel values, a second map generation unit that generates a set of second maps for each set of content in which an area corresponding to the content for each category and other areas are imaged with different pixel values, and a determination unit that inputs the set of first maps and the set of second maps for each set of content into a trained model that has trained the relevance between the contents, and determines whether or not there is a relevance between the contents for each set of content.

A computer program according to an embodiment of the present invention causes a computer to execute a process of acquiring document data including a plurality of pieces of content, identifying categories of the plurality of pieces of content included in the acquired document data, generating a set of content for the identified category, generating a set of first maps for each category in which an area corresponding to the content and other areas are imaged with different pixel values, generating a set of second maps for each set of content in which an area corresponding to the content and other areas are imaged with different pixel values for each category, inputting the set of first maps and the set of second maps for each set of content into a trained model that has trained the relevance between content, and determining whether or not there is a relevance between the content for each set of content.

The present invention makes it possible to accurately determine the relevance of content within a document.

1 is a block diagram showing an example of a configuration of an information processing device according to an embodiment of the present invention; FIG. 13 is a schematic diagram showing an example of a category identification method. FIG. 2 is a schematic diagram showing an example of an association graph showing associations between contents in a required category. FIG. 11 is a schematic diagram showing a flow of a series of processes for determining the relevance of content. FIG. 2 is a schematic diagram showing an example of a set of contents. FIG. 2 is a schematic diagram showing an example of a content group map. FIG. 2 is a schematic diagram showing an example of an entire content map. FIG. 13 is a schematic diagram showing an example of a coordinate map. FIG. 2 is a schematic diagram illustrating an example of a configuration of a feature map. FIG. 13 is a schematic diagram showing another example of the content group map. FIG. 13 is a schematic diagram showing an example of a learning method of the relevance determination unit. 10 is a schematic diagram showing an example of a content relevance determination result by an information processing device; FIG. FIG. 13 is a schematic diagram showing an example of an operation on cluster content. 11 is a flowchart illustrating an example of a processing procedure for determining relevance of content by an information processing device.

The following describes an embodiment of the present invention with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of an information processing device 50 according to this embodiment. The information processing device 50 can be connected to a server 10 via a communication network 1. The server 10 can be, for example, a data server that stores document data. The information processing device 50 can acquire document data from the server 10 via the communication network 1. A scanner 20 can also be connected to the information processing device 50. The information processing device 50 can acquire document data obtained by reading with the scanner 20. The document data is page data for magazines, books, newspapers, etc., and includes multiple contents. The contents are the elements laid out in the document.

The information processing device 50 includes a control unit 51 that controls the entire device, a communication unit 52, a memory unit 53, a category identification unit 54, a content group generation unit 55, a map generation unit 56, a relevance determination unit 57, a display panel 58, a display processing unit 59, and an operation unit 60. The information processing device 50 may be configured, for example, as a personal computer, a tablet, a smartphone, etc. The control unit 51 may be configured as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc.

The communication unit 52 has a function of communicating with the server 10 via the communication network 1, and can transmit and receive required information. More specifically, the communication unit 52 can acquire document data from the server 10. The communication unit 52 also has an interface function with the scanner 20, and can acquire document data from the scanner 20.

The storage unit 53 is composed of a semiconductor memory or a hard disk, and can store document data acquired via the communication unit 52. The storage unit 53 can also store required data such as the processing results in the information processing device 50.

The display panel 58 can be composed of a liquid crystal display or an organic EL (Electro Luminescence) display.

The display processing unit 59 performs processing to display the required information on the display panel 58.

The operation unit 60 can be configured with, for example, a keyboard, a mouse, etc. The operation unit 60 may also be configured with a touch panel or the like, and may be used to input characters on the display panel 58, and to perform operations on icons, images, characters, etc. displayed on the display panel 58.

The category identification unit 54 has a function as an identification unit, and identifies the categories of multiple contents contained in the acquired document data via the communication unit 52. The categories include, for example, the title, main text, figures (images), and captions (explanation of figures). The category can be identified using various image processing techniques, such as a machine-learned classifier, image analysis that extracts features that represent the category, image recognition that identifies the type of category, and pattern matching.

Figure 2 is a schematic diagram showing an example of a category identification method. A document has multiple contents laid out, and each content is classified into one of the categories of title, text, figure, caption, etc. The category identification unit 54 can identify the category of each content laid out in the document. In the example of Figure 2, figures and captions are identified as the required category from among the categories of title, text, figure, caption, etc. In the diagram on the right, the category of content surrounded by a dashed line is either figure or caption. It should be noted that which category to identify from the categories of title, text, figure, caption, etc. can be set in advance depending on which category of content relationships is the target. In the following description, figures and captions are used as examples of the required categories.

Figure 3 is a schematic diagram showing an example of an association graph showing the association between contents of a required category. The required categories are figures and captions. The left diagram shows the contents of the specified category, specifically, a diagram in which the contents figure object1, figure object2, and figure object3, which are in the category of figures, and the contents caption object1, caption object2, and caption object3, which are in the category of captions, are laid out. By determining whether or not there is an association between each piece of content, a correct answer graph (association graph) such as that shown in the right diagram can be obtained. As shown in the correct answer graph, there is an association between figure fig1 and caption cap1 and cap2, respectively, and there is an association between figure fig2 and caption cap3, and between figure fig3 and caption cap3. In the diagram, related contents are connected by lines.

Next, we will explain how to determine the relevance of content.

Figure 4 is a schematic diagram showing a series of processing steps for determining the relevance of content. The content group generation unit 55 functions as a generation unit and can generate a content group for the category identified by the category identification unit 54.

Figure 5 is a schematic diagram showing an example of a content pair. The specified categories are figures (fig) and captions (cap), and there are three figure contents (fig1, fig2, fig3) and three caption contents (cap1, cap2, cap3) in the document. Note that figure 1 corresponds to figure object 1 in Figure 3, and cap1 corresponds to caption object 1 in Figure 3. The same applies to other contents. The content pair generation unit 55 can generate all pairs (nine pairs) of contents in different categories, as shown by (1) to (9) in the figure. Note that, although all pairs of contents in different categories are generated here, pairs of contents in the same category (for example, fig2 and fig3) may also be generated.

The map generating unit 56 generates a feature map that visualizes the layout of the content of the identified category. Specifically, the map generating unit 56 can generate a content group map, an entire content map, and a coordinate map. The feature map is described in detail below.

Figure 6 is a schematic diagram showing an example of a content set map. The content set map is also called a semantic map of an element set. The content set map is an image of the layout within a document of each content set generated by the content set generation unit 55, and can be generated for each content set. In other words, the content set map is a map (second map) in which the area corresponding to the content and the other areas are imaged with different pixel values for each category for one set, and the map generation unit 56 can generate a set of content set maps. A channel can be assigned for each category, and in the example shown, figures are assigned to channel 1 and captions are assigned to channel 2.

If the content set is fig1-cap2, the content set map for the figure (channel 1 map) can be an image of the layout (placement) of fig1 within the document, and the content set map for the caption (channel 2 map) can be an image of the layout (placement) of cap2 within the document. In other words, feature maps for channels 1 and 2 can be generated corresponding to one content set.

Furthermore, if the content set is fig2-cap1, then the content set map for figures (channel 1 map) can be an image of the layout (placement) of fig2 within the document, and the content set map for captions (channel 2 map) can be an image of the layout (placement) of cap1 within the document. In the same manner, content set maps can be generated for all content sets. If there are nine content sets, nine content set maps each for figures and captions can be generated.

The pixel values of the content group map can be, for example, 1 for the content area and 0 for the background outside the content area, but are not limited to this.

Figure 7 is a schematic diagram showing an example of an entire content map. The entire content map is also called a semantic map of all elements. The entire content map is an image of the layout of the entire content of a specified category within a document. In other words, the entire content map is a map (first map) in which the area corresponding to the content for each category and the other areas are imaged with different pixel values. The map generation unit 56 generates a set of entire content maps for each set of content. A channel can be assigned to each category; in the example shown, the figure is assigned to channel 1 and the caption is assigned to channel 2.

An overall content map can be generated for each identified category. The overall content map for figures (channel 1 map) can be an image of the layout (placement) of fig1, fig2, and fig3 within the document. The overall content map for captions (channel 2 map) can be an image of the layout (placement) of cap1, cap2, and cap3 within the document.

The pixel values of the entire content map can be, for example, 1 for the content area and 0 for the background outside the content area, but are not limited to this.

FIG. 8 is a schematic diagram showing an example of a coordinate map. The coordinate map is an image of position information in a document. Assume that the image of the document is composed of (m×n) pixels (m: number of pixels in the y-axis direction, n: number of pixels in the x-axis direction). The coordinate map can be composed of an x-axis coordinate map to which channel 1 is assigned, and a y-axis coordinate map to which channel ₂ is assigned. In the x-axis coordinate map, pixel values represent positions in the x-axis direction, and pixel values x ₀ , x 1 , x 2 , ..., x _n are given to pixels 0, ₁ , 2, ..., n in the x-axis direction, respectively. In the example shown in the figure, pixel values x ₀ , x ₁ , x ₂ , ..., x _n are set to 0, 0.1, 0.2, .... In the y-axis coordinate map, pixel values represent positions in the y-axis direction, and pixel values y ₀ , y ₁ , y 2 , ..., _yn are given to pixels 0, 1, ₂ , ..., m in the y-axis direction, respectively. In the illustrated example, pixel values y ₀ , y ₁ , y ₂ , ..., _yn are set to 0, 0.1, 0.2, .... As the pixel value increases, the image becomes brighter.

Figure 9 is a schematic diagram showing an example of the configuration of a feature map. The feature map can be constructed by combining the maps of six channels, namely the content set maps for the figure (channel 1) and caption (channel 2), the overall content maps for the figure (channel 1) and caption (channel 2), and the coordinate map (x-axis and y-axis). Note that the coordinate map is not a required configuration, but by using the coordinate map, the accuracy of determining the relevance between contents can be improved.

The relevance determination unit 57 has a function as a determination unit, and by inputting a feature map, can determine whether or not there is a relevance between the contents for each group of contents. That is, the relevance determination unit 57 can input the set of content group maps and the set of entire content maps to a trained model that has trained the relevance between the contents, and can determine whether or not there is a relevance between the contents for each group of contents. The trained model can be configured, for example, as a convolutional neural network, but is not limited to this, and may be configured as another neural network. In addition, other machine-learned models such as SVM (Support Vector Machine) and Bayesian networks may also be used.

As described above, for each set of content, the presence or absence of a relationship between the contents is determined using a content set map obtained by imaging the layout of the content within the document. This allows a greater amount of information to be used to determine the presence or absence of a relationship compared to when the determination is simply based on the relative positions (relative coordinates) of the contents, making it possible to more accurately determine the presence or absence of a relationship between the contents.

In addition, by using the entire content map, it is possible to take into account not only the layout of the contents fig1 and cap1 (content group map) but also the relevance with other content in the document, for example, the relevance of categories other than the contents fig1 and cap1 with the caption content, and the relevance of categories other than the contents cap1 and fig1 with the figure content, making it possible to more accurately determine whether the content in the document is relevant.

Furthermore, by using a coordinate map (common to each set of content) in addition to the overall content map (common to each set of content) and the content set map (different for each set of content), relevance can be determined for each set of content by taking into account information that can grasp the positional relationship within the document of each piece of content in the overall content map and content set map, so that the relevance of content within a document can be determined with even greater accuracy.

Figure 10 is a schematic diagram showing another example of a content group map. The content group map shown in Figure 6 is generated by using two channels, with channel 1 assigned to the figure and channel 2 assigned to the caption. In the example of Figure 10, the map generating unit 56 can generate a content group map in which the pixel values of the content differ for each specified category. For example, as shown in Figure 10, the figure is represented by a rectangle with no pattern, and the caption is represented by a rectangle with a pattern (diagonal lines). The pixel value of the pixel in the figure area in the document can be a, the pixel value of the pixel in the caption area can be b, and the pixel value of the pixel in the area other than the figure and the caption can be c. The difference in pixel value may represent a difference in brightness or a difference in color. This makes it possible to generate a content group map of one channel corresponding to one content set.

Although not shown, the map generator 56 can generate an entire content map in which the pixel values of the content differ for each identified category. That is, the pixel value of the pixels in the area of each figure in the document can be a, the pixel value of the pixels in the area of each caption can be b, and the pixel value of the pixels in the area other than each figure and each caption can be c. This makes it possible to generate an entire content map for one channel.

Next, we will explain the learning method of the relevance determination unit 57.

Figure 11 is a schematic diagram showing an example of a learning method for the relevance determination unit 57. A large amount of document data is collected in advance, and all pairs of content in the category of figures and content in the category of captions in each document are generated, and related and unrelated labels are assigned to the generated content pairs. A feature map is generated based on the generated content pairs. The training data can include feature maps that have been assigned related labels and feature maps that have been assigned unrelated labels.

In the example of FIG. 11, since the content fig1 and the content cap2 are related to each other, the neural network can be trained using the feature map generated based on the content fig1 and the content cap2 and the related labels to generate the relatedness determination unit 57. It goes without saying that the training data includes feature maps and related labels for other related content pairs.

Similarly, since content fig2 and content cap1 are unrelated, a neural network can be trained using a feature map generated based on content fig2 and content cap1 and an unrelated label to generate a relevance determination unit 57. Needless to say, the training data includes feature maps and unrelated labels for other unrelated content pairs.

In addition, in training a neural network, the loss function can be appropriately determined as a squared error function, or binary cross entropy can be used. Cross entropy is expressed as a logarithmic function of the probability that the training data will be the same as the teacher data, so if the difference between the teacher data and the training results is large, the loss function will be large, and the loss function will decrease more with each training, allowing for faster learning.

Figure 12 is a schematic diagram showing an example of a content relevance determination result by the information processing device 50. In Figures 12A and 12B, three images and three captions are laid out in a document. Figure 12A is a comparative example, and Figure 12B shows a relevance determination result by the information processing device 50 of this embodiment. In the comparative example of Figure 12A, only the relative positions of the contents are taken into consideration. For this reason, there is a tendency to determine that contents close to each other in a document are related, and as shown by the dotted line, there are cases where related content cannot be detected, or unrelated content is erroneously detected as related, as shown by the dashed and dotted line. In contrast, as shown in Figure 12B, it can be seen that the information processing device 50 of this embodiment accurately detects all related content.

In Figures 12C and 12D, two images and two captions are laid out in a document. Figure 12C is a comparative example, and Figure 12D shows the result of a relevance determination made by the information processing device 50 of this embodiment. In the comparative example of Figure 12C, only the relative positions of the contents are taken into consideration. For this reason, as shown by the dashed dotted line, unrelated contents may be erroneously detected as related. In contrast, as shown in Figure 12D, it can be seen that the information processing device 50 of this embodiment accurately detects all related contents.

The display processing unit 59 has a function as an output unit, and can output identification information that identifies the relevance of content of categories in a document based on the determination result of the relevance determination unit 57. The identification information may be, for example, a line segment connecting related content in a document, as exemplified in FIG. 12, or a frame of the same color surrounding the related content. The identification information may be displayed at all times, but if it is not necessary to display it at all times, it may be displayed in a way that shows that the content has been associated by a specified operation. This allows the user to easily recognize related content.

Fig. 13 is a schematic diagram showing an example of an operation on cluster content. Cluster content is content that has been determined to be related by the relevance determination unit 57. As shown in Fig. 13, a document (e.g., one page, or two pages in a double-page spread) in which multiple contents are arranged is displayed on the display panel 58. In the example of Fig. 13, the following contents are displayed: title, main text A, main text B, image (figure) A (figure object A), caption A (caption object A), and caption B (caption object B). It is also assumed that image (figure) A (figure object A) and captions A (caption object A) and B (caption object B) are related to each other.

As shown in the left diagram of FIG. 13, when the icon 100 is brought close to image A (or the periphery of image A, or caption A or B) and a touch operation and drag operation are performed, the captions A and B can be moved in the same way along with image A, as shown in the right diagram. Image A, caption A, and caption B make up one cluster content 101.

In this way, when the display processing unit 59 receives an operation to select cluster content displayed on the display panel 58, it can display each of the contents associated by the cluster content in the selected display mode. For example, when an operation is performed to select one piece of content or the periphery of a piece of content in the cluster content displayed on the display panel 58 and then moved (dragged) on the display panel 58, all of the contents in the cluster content are displayed in the selected display mode, and the entire cluster content can be moved (dragged). This eliminates the need to repeat the same operation for related content, improving operability of the content in the document.

Figure 14 is a flowchart showing an example of a processing procedure for determining the relevance of content by the information processing device 50. For convenience, the processing will be described below with the control unit 51 as the subject. The control unit 51 acquires document data including multiple pieces of content (S11), and identifies the category of the content (S12). The categories include, for example, titles, main body (text), figures (images), captions (explanations of figures), etc. Here, figures and captions can be identified as the required category.

The control unit 51 generates a set of content for the identified category (S13). The set of content may be, for example, all of the sets illustrated in FIG. 5. The control unit 51 generates an overall content map (S14). The overall content map may be, for example, the map illustrated in FIG. 7.

The control unit 51 generates a coordinate map (S15). The coordinate map can be, for example, the map illustrated in FIG. 8. The control unit 51 generates a content group map (S16). Here, the content group map can be, for example, one of the maps illustrated in FIG. 6.

The control unit 51 combines the entire content map, the content group map, and the coordinate map to construct a feature map (S17). The feature map can be, for example, the map illustrated in FIG. 9. Note that the coordinate map does not have to be used. In this case, the process of step S15 is not necessary.

The control unit 51 inputs the feature map to the relevance determination unit 57 to determine whether the content sets are related (S18). The control unit 51 determines whether all content sets have been determined (S19), and if not all content sets have been determined (NO in S19), it selects an unprocessed content set (S20) and continues the processing from step S16 onwards.

When all the content pairs have been determined (YES in S19), the control unit 51 records the related content as cluster content (S21), outputs identification information that identifies the relationship between the contents (S22), and ends the process. The identification information can be, for example, a line segment that connects the contents as shown in FIG. 12.

In the above example, a content group map is generated in step S16 each time a content group is selected, but this is not limited to the above. For example, a content group map may be generated for all groups in step S16, and the process of step S17 may be repeated each time a content group is selected to generate a feature map.

The information processing device 50 can be realized, for example, by using a computer equipped with a CPU (e.g., a multi-processor having multiple processor cores), a GPU (Graphics Processing Units), a RAM, etc. The information processing device 50 can be realized on a computer by loading a computer program (which can be recorded on a recording medium) that defines the processing procedure as shown in FIG. 14 into the RAM of the computer and executing the computer program with the CPU (processor).

In the above example, the relevance between content whose category is figures and content whose category is captions is determined, but the categories are not limited to figures and captions, and the relevance with content of other categories can also be determined. The relevance between multiple pieces of content whose category is figures may also be determined. The user may be able to set which category of content to determine the relevance of.

The information processing device of this embodiment includes an acquisition unit that acquires document data including multiple contents, an identification unit that identifies categories of the multiple contents included in the acquired document data, a generation unit that generates a set of contents of the identified category, a first map generation unit that generates a set of first maps in which an area corresponding to the content and other areas are imaged with different pixel values for each category, a second map generation unit that generates a set of second maps for each set of contents in which an area corresponding to the content and other areas are imaged with different pixel values for each category, and a determination unit that inputs the set of first maps and the set of second maps for each set of contents into a trained model that has trained the relevance between contents and determines whether or not there is a relevance between the contents for each set of contents.

The computer program of this embodiment causes a computer to execute a process of acquiring document data including a plurality of pieces of content, identifying categories of the plurality of pieces of content included in the acquired document data, generating a set of content for the identified category, generating a set of first maps for each category in which an area corresponding to the content and other areas are imaged with different pixel values, generating a set of second maps for each set of content in which an area corresponding to the content and other areas are imaged with different pixel values for each category, inputting the set of first maps and the set of second maps for each set of content into a trained model that has learned the relevance between content, and determining whether or not there is a relevance between the content for each set of content.

The acquisition unit acquires document data including multiple contents. The contents are the elements laid out in the document. The document data is page data for magazines, books, newspapers, etc., and may be acquired from a data server that records document data, or may be read from a reading device such as a scanner.

The identification unit identifies the categories of multiple contents contained in the acquired document data. The categories include, for example, the title, main text, figures (images), and captions (explanation of figures). The identification unit can identify the category of each element laid out in the document.

The generation unit generates a set of content in the specified category. Let us say that the specified category is figure (fig) and caption (cap), and that there are three figures (fig1, fig2, fig3) and three captions (cap1, cap2, cap3) in the document. The generation unit can, for example, generate all sets of content in different categories (fig1-cap1 set, fig1-cap2 set, ..., etc.). In this case, nine sets can be generated.

The first map generating unit generates a set of first maps in which the area corresponding to the content and the other areas are imaged with different pixel values for each category. The first map can be generated for each specified category. For example, if the category is a figure, the first map for the figure can be an image of the layout (arrangement) of fig1, fig2, and fig3 in the document, and for example, the pixel values corresponding to fig1, fig2, and fig3 can be set to 1, and the pixel values corresponding to other than fig1, fig2, and fig3 in the document can be set to 0. Similarly, if the category is a caption, the first map for the caption can be an image of the layout (arrangement) of cap1, cap2, and cap3 in the document, and for example, the pixel values corresponding to cap1, cap2, and cap3 can be set to 1, and the pixel values corresponding to other than cap1, cap2, and cap3 in the document can be set to 0.

The second map generating unit generates a set of second maps for each content pair, in which the area corresponding to the content and the other areas are imaged with different pixel values for each category of the pair. If the content pair is fig1-cap1, the second map for the figure can be an image of the layout (placement) of fig1 in the document, and the second map for the caption can be an image of the layout (placement) of cap1 in the document. That is, two second maps can be generated for one content pair. If the content pair is fig1-cap2, the second map for the figure can be an image of the layout (placement) of fig1 in the document, and the second map for the caption can be an image of the layout (placement) of cap2 in the document. In the same manner, second maps can be generated for all content pairs. If there are nine content pairs, nine second maps for the figures and nine second maps for the captions can be generated.

The determination unit inputs the set of first maps and the set of second maps for each group of content into a trained model that has learned the relevance between the contents, and determines whether there is a relevance between the contents for each group of content.

For example, the presence or absence of a relationship between contents fig1 and cap1 of a content pair (fig1-cap1) is determined based on a second map obtained by imaging the layout of contents fig1 and cap1 in a document, so the amount of information used for the determination can be increased and a more accurate determination can be made compared to when the determination is simply based on the relative positions (relative coordinates) of contents fig1 and cap1.

In addition, by using the first map, it is possible to take into account not only the layout (second map) of the contents fig1 and cap1, but also the relevance with other content in the document, for example, the relevance of categories other than the contents fig1 and cap1 with the caption content, and the relevance of categories other than the contents cap1 and fig1 with the figure content, making it possible to more accurately determine the relevance of the content in the document.

In the information processing device of this embodiment, the first map generation unit generates a first map in which the pixel values of the content differ for each identified category, and the second map generation unit generates a second map in which the pixel values of the content differ for each identified category.

The first map generation unit generates a first map in which the pixel values of the content differ for each identified category. The second map generation unit generates a second map in which the pixel values of the content differ for each identified category. If the set of contents is fig1-cap1, the pixel value of a pixel corresponding to fig1 in the document can be a, the pixel value of a pixel corresponding to cap1 can be b, and the pixel value of a pixel in the document corresponding to parts other than fig1 and cap1 can be c. The difference in pixel value may represent a difference in brightness or a difference in color. This makes it possible to generate one first map corresponding to one set of contents and one second map corresponding to one set of contents.

The information processing device of this embodiment includes an output unit that outputs identification information that identifies the relevance of the content of the category within the document based on the judgment result of the judgment unit.

The output unit outputs identification information that identifies the relevance of the content of the category in the document based on the determination result of the determination unit. The identification information may be, for example, a line segment connecting related content in the document, or a frame of the same color surrounding related content, and may not be displayed all the time, but may be displayed in a manner that indicates that the content is associated through a specified operation. This allows the user to easily recognize related content.

The information processing device of this embodiment includes a third map generation unit that generates a third map that visualizes positional information within a document, and the determination unit determines whether or not there is a relationship between the contents for each set of the contents based on the third map.

The third map generating unit generates a third map that visualizes position information in the document. Assume that the image of the document is composed of (m×n) pixels (m: number of pixels in the y-axis direction, n: number of pixels in the x-axis direction). The third map can be composed of an x-axis coordinate map and a y-axis coordinate map. The x-axis coordinate map indicates the position of the pixel in the x-axis direction, and pixel values _x0 , x1, x2, ..., xn are given to pixels 0, ₁ , ₂ , ..., _n in the x-axis direction. The y-axis coordinate map indicates the position of the pixel in the y-axis direction, and pixel values y0, y1, y2, ..., yn are given to pixels ₀ , ₁ , ₂ , ..., _m in the y-axis direction.

The determination unit inputs the third map and determines whether or not there is a relationship between the contents for each group of contents. That is, by determining the relationship for each group of contents using the third map (common to each group of contents) in addition to the first map (common to each group of contents) and the second map (different for each group of contents), the relevance can be determined taking into account information for extracting the positional relationship within the document of each piece of content in the first map and the second map, and therefore the relevance of the contents within the document can be determined with even greater accuracy.

REFERENCE SIGNS LIST 1 Communication network 10 Server 20 Scanner 50 Information processing device 51 Control unit 52 Communication unit 53 Storage unit 54 Category identification unit 55 Content group generation unit 56 Map generation unit 57 Relevance determination unit 58 Display panel 59 Display processing unit 60 Operation unit

Claims (5)

An acquisition unit that acquires document data including a plurality of contents;
an identification unit that identifies categories of the plurality of contents included in the acquired document data;
A generation unit that generates a set of content for the identified category;
a first map generator that generates a set of first maps in which an area corresponding to the content and other areas are imaged with different pixel values for each category;
a second map generating unit that generates, for each set of contents, a set of second maps in which an area corresponding to the content and other areas are imaged with different pixel values for each category of the set;
a determination unit that inputs the set of first maps and the set of second maps for each set of content into a trained model that has trained the relevance between contents and determines whether or not there is a relevance between the contents for each set of content;
An information processing device comprising: The first map generating unit is
generating a first map in which pixel values of the content differ for each of the identified categories;
The second map generating unit is
generating a second map in which pixel values of the content differ for each of the identified categories;
The information processing device according to claim 1 . an output unit that outputs identification information that identifies the relevance of the content of the category in the document based on the determination result of the determination unit;
3. The information processing device according to claim 1 or 2. a third map generating unit that generates a third map by imaging position information within a document;
The determination unit is
determining whether or not there is a relationship between the contents of each of the sets of contents based on the third map;
The information processing device according to claim 1 . On the computer,
Acquire document data including multiple contents,
Identifying categories of the plurality of contents included in the acquired document data;
Generate a set of content for the identified categories;
generating a set of first maps in which an area corresponding to the content and other areas are imaged with different pixel values for each category;
generating a set of second maps for each set of content, in which an area corresponding to the content and other areas are imaged with different pixel values for each category of the set;
The set of first maps and the set of second maps for each of the sets of content are input to a trained model that has trained the relevance between contents, and a determination is made as to whether or not there is a relevance between the contents for each of the sets of content.
A computer program that executes a process.

Images