JP2003173280A - Database generation device, database generation method, and database generation program - Google Patents

Abstract

(57) [Summary] [Problem] A database that collects data from multiple servers that independently manage and operate store information and the like on a network, has no duplicate data, and can display the update status of data ( DB). SOLUTION: This database generation device 10 collects data from each server 20 on a network 40 and information 11 such as an identification ID of the data, an update date and time, and an attribute characterizing the data from the collected data. The value is extracted and a new D consisting of the attribute value, identification ID, update date
A means for generating B18, a means for identifying data whose attribute values in the new DB can be regarded as being identical, a means for associating data whose attribute values can be regarded as identical between the new DB and the previously generated old DB, Compare the identification ID and update date of the DB data with the corresponding old DB data, and
Means 15 for adding update information to the corresponding data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention collects data from a plurality of servers or the like which are distributedly arranged on a network such as the Internet and independently manage / operate guide information of stores and the like, and search / guide. The present invention relates to an apparatus and method for generating a database for data, and a program thereof.

[0002]

2. Description of the Related Art Data such as shop guide information is often created independently by some organizations and updated as needed. Since the data set owned by one organization does not cover the guide information of all stores, if these data sets created and updated independently are integrated, a more complete information search service will be provided. It can be performed. The data set possessed by each organization is stored in a computer connected to a network such as the Internet and provided for browsing. Hereinafter, such a computer will be referred to as an information server.

In the conventional technique of collecting data sets from a plurality of information servers and generating a database, a database is a simple merge of data sets collected from a plurality of information servers. Each data in the generated database is provided with link information to the original data in the information server in which the data exists, and is displayed on the terminal when the user retrieves the data from the database using the terminal. The original information of the data can be accessed by the link information attached to the created data. FIG. 11 shows an example of a conventional search result display screen when a store having a business type of “Chinese” and an address of “Kagurazaka, Shinjuku-ku” is searched from the database. When the user clicks the link information on the screen, the detailed screen of the linked store is displayed.

[0004]

In a data set created independently by some organizations, names and addresses are often registered in different formats and expressions even in the same store. Therefore, in the conventional technique of simply merging data sets collected from a plurality of information servers to generate a database,
It may not be possible to combine the same overlapping stores into one, and a plurality of the same stores may be mixedly displayed in the store group of search results. In such a case, the search results are redundantly increased, and the user is forced to perform a complicated work of examining the contents of unnecessary data and determining whether the data is the same as the data already seen. For example, in the search result display screen of FIG. 11, the first store and the fourth store are the same, and the third store and the sixth store are the same.

Also, what is particularly interesting to a user who searches for information on a store or the like is new information on the store or information on a newly opened store. Therefore, when performing collection of a data set and generation of a database on a regular basis, for a user who retrieves data from the generated database, which data among the displayed data is updated, Alternatively, if information indicating that it is new is attached, new information can be quickly acquired.
However, in the related art, such update information is not added to each data.

An object of the present invention is to collect data sets from a plurality of information servers or the like which are distributed over the network, integrate the data so that there is no redundancy, and add data update information. To provide a database generating apparatus and method capable of generating a stored database, and a database generating program therefor.

[0007]

A database generation apparatus of the present invention comprises a storage means for storing a database (old database) generated in the past, data containing attribute values such as names and addresses from a plurality of points, Identification ID of the data,
A data collection means for collecting information such as update date and time, and a database having a configuration in which attribute values are extracted from each of the collected data, and each data includes the extracted attribute value, identification ID, update date and time ( Between a new database), an attribute information extraction unit that generates a new database), a name identification unit that classifies data sets that can be regarded as having the same attribute value in the generated new database into the same group, and a new database and the old database, A combination unit that determines that the data whose attribute values can be regarded as the same is the same and associates each data between the two databases, information such as an identification ID and update date and time of the data in the new database, and the corresponding data. By comparing it with the information such as the identification ID and the update date and time of the data in the old database attached, the data in the new database is compared. And having an update information providing means for providing update information to the data.

In the name identification means, the duplicated data is unified in the generated new database. Therefore, when the data matching the user's request is retrieved from this database and displayed, a plurality of data of the same store are not displayed, and the retrieval result can be grasped more easily. In addition, the joining means is a new database created,
The data of the same store or the like is specified with respect to the old database generated last time, and the identification information I
Since the update information of the data is derived by comparing D (for example, name) and update date and time, the finally generated database can display the data after the update information of the data is added. It is possible.

Next, the database generation method of the present invention is
A data collection process of collecting data including attribute values such as names and addresses from a plurality of points, identification ID of the data, information such as update date and time, and extracting attribute values from the collected data. The data is data in which the attribute information extracting process for generating a database (new database) having a structure including the extracted attribute value, identification ID, update date and time, and the attribute value in the generated new database can be regarded as the same. The name identification process that classifies the set into the same group and the new database and the old database that was generated and retained in the past are judged to be the same data that has the same attribute value. A combining process for associating data, information such as identification ID and update date and time of data in the new database, and the old data associated with the data By comparing the information such as an identification ID and update time of the data in the over scan, and having an update information imparting step of imparting updates to corresponding data in the new database.

Next, the computer-executable program of the present invention is a data collection process for collecting data including attribute values such as name and address, identification ID of the data, update date and time from a plurality of points. And an attribute information extraction process for extracting a value of an attribute from each of the collected data, and generating a database (new database) having a structure in which each data includes the extracted attribute value, identification ID, update date and time, and the like. , Between the new database and the old database generated in the past, the name identification process of classifying the data sets in which the values of the attributes in the new database generated in the attribute information extraction process can be regarded as the same, into the same group. A combining process that determines that data that can be regarded as having the same value is the same and associates each data between both databases, and the new data By comparing the information such as the identification ID and the update date and time of the data in the database with the information such as the identification ID and the update date and time of the data in the old database associated with the data, the corresponding information in the new database can be obtained. And an update information adding process for adding update information to the data.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration example of a database generation device according to an embodiment of the present invention. The database generation device 10 is connected to a network 40 such as the Internet, and a plurality of information servers 20 that manages and operates a data set such as guide information of a store via the network 40 and a user It is connected to the user terminal 30 to be used. The URL of each information server 20 connected to the network 40
(Uniform resource locator). Each information server 20 has a data set therein, and manages and operates the data set independently of other information servers. Therefore, guide information for the same store often exists in a plurality of information servers 20. The database generation device 10 itself may also be in the form of managing and operating with a data set inside. As the user terminal 30, typically, a personal computer (personal computer) or a mobile terminal in which WWW software (WWW browser) is incorporated is used. Each user uses the user terminal 30 to search for information, etc., and if necessary, notifies the database generation device 10 of requests and the like.

The database generating device 10 comprises a data collecting unit 11, an attribute information extracting unit 12, a name collating unit 13, a combining unit 14, an update information adding unit 15, and a database storing unit 16. The database storage unit 16 stores a database (old database) 17 created in the past (here, the previous time) and a newly created database (new database) 18.
The database generation device 10 is realized by a so-called computer, the processing means 11 to 15 are handled by the CPU and its built-in memory (RAM, ROM, etc.), the database storage unit 16 is a hard disk, other external storage device, etc. Is responsible for.

The database generating apparatus 10 itself may receive the search request from the user terminal 30 to implement the information search service. In this case, although omitted in FIG. 1, the database generation device 10 also includes information retrieval means. Further, the information retrieval device is the database generation device 1
As a configuration different from 0, the database generated by the database generation device 10 may be used by another information search device.

FIG. 2 is a diagram showing a flowchart of a database generating method according to an embodiment of the present invention. Hereinafter, the operation of the database generation device 10 will be described in detail with reference to the flowchart of FIG.

In the database generating device 10, the data collecting means 11 allows each information server 2 to be operated at a fixed period or at a specific date and time (for example, one day, one week, and every Monday).
0 to collect data (data set) in each information server 20 (step 111). Here, it is assumed that each data is one file and is under the directory where all files are. The location of this directory is pre-arranged between the administrator of the database generation device 10 and the administrator of each information server 20, and the data collection unit 11 stores the files under the directory of each information server 20. It is downloaded and temporarily stored in, for example, a RAM or a hard disk. Here, together with the file, the file name that becomes the name of the data (this becomes the link information of the data) and the information of the update date and time of the file are acquired.

FIG. 3 shows an example of data downloaded from the information servers A and B. In this example, the data of the same store “Korantei” is also stored in the information server A and B
It is assumed that it is also registered in and the data is shown. As shown in FIG. 3, in the information servers A and B, names and addresses are registered in different formats and expressions even in the same store “Korantei”.

Next, the attribute information extracting means 12 extracts the value of the attribute characterizing the data such as the name and address from each data collected by the data collecting means 11 (step 112). Each data file is typically HTM
It is an L document or an XML document, and the user can browse the content of the file by accessing the URL of the file from the WWW software (WWW browser) using the user terminal 30. Format information such as what attributes each data file has and what format each attribute is described in is determined for each information server 20. Here, it is assumed that the attribute information extraction means 12 holds a data file format analysis routine corresponding to each information server. The attribute information extraction means 12 extracts attribute values such as names and addresses from the data file by a data file format analysis routine corresponding to each information server. Next, the attribute information extraction means 12 creates data (records) including attribute values such as names and addresses that have been extracted, the name of the information server where the data exists, link information of the data, and update date and time, The database storing unit 16 generates a database in which such data is accumulated.
To store. This newly created database is referred to as a new database 18. In addition, the old database 17 is a database that was previously collected (1 day ago, 1 week ago, etc.) and similarly generated by collecting data from each information server 20 and subjected to processes such as name identification, combination, and update information addition, which will be described later. .

FIG. 4 shows an example of the newly created database (new database) 18 (a).
Is data of the information server A, and (b) is data of the information server B. Here, the type of industry, name, and
I have an address. The business system is generally different for each information server 20. In addition, even in the data of the same store, if the information server is different, the notation of the name and the address may be unstable.

In addition, when the information server 20 has a file including a list of data in which only basic information such as name, address, telephone number, and link information of each store is described in addition to the data file of the store, data is stored. Collecting means 11
In, in place of the data file group, such a list file may be downloaded. In this case, the attribute information extracting means 12 extracts the name, address, link information, etc. from each data of the list file, accesses the information server 20 again based on the extracted link information, and updates the date and time of the data file. Get information. Then, similarly, a new database 18 as shown in FIG. 4 is generated.

Next, the name identification means 13 classifies the data (records) in the new database 18, which can be regarded as having the same attribute values such as name and address, into the same group (step 113). That is, the names are identified as the same store.

For example, the new database 18 shown in FIG.
The value of the attribute of the name and the value of the attribute of the arbitrary two data are compared, and the matched records are classified into the same group. For example, the following method can be considered as a method of collating the nominal character string and the address character string. One is to consider a match when it is an exact match (called an exact match), and a method to consider it to be a match when the ratio of the number of characters contained in both is more than a certain threshold (character match). There is). Another method is to divide a character string into words and to consider that a match occurs when the ratio of the number of words commonly included in both is greater than or equal to a certain threshold (called word-unit matching). Either way, you can convert Kanji numbers into arithmetic numbers,
It is possible to further improve the accuracy of the collation by performing a process in advance to eliminate the fluctuation of the notation such as unifying the letters to uppercase. As a result of the collation, in the example of FIG.
The fourth and fourth data (records) match, and the third and sixth data match. The matched records are classified into the same group. Here, each group will be referred to as name identification data in order to distinguish it from normal data.

In the name identification means 13, as the name or address attribute value of each name identification data, for example, only one name or address attribute value of the data included in the name identification data is selected and the value itself is used, or Convert to a normalized value. Further, the business type name of each data is converted into the corresponding business type name in the business type system unique to the database generation device 10.

With reference to FIG. 4, an example of the new database 18 updated in this way is shown in FIG. For example, in the business type system unique to the database generation device 10, there are “Japanese food”, “Chinese food”, and the like as business types, and the business type names of the data in FIG. 4 are all converted to “Chinese food”. In FIG. 5, since the industry names of the first and fourth data classified into the same group are both converted to “Chinese”, the industry name as the name identification data is also “Chinese”. The same applies to the third and sixth data. Moreover, as the name of the name identification data and the attribute value of the address, in the first and fourth data,
Select "Korantei" as the name and the address as "1-
2-3 ”is selected. Similarly, in the 3rd and 6th data, "Otaketei" is selected as the name and "3-8-6 Kagurazaka, Shinjuku-ku" is selected as the address. The column of "update information" of the new database 18 in FIG. 5 is rewritten by the update information adding means 15 described later, and is all empty here.

The matching rule such as which matching method is used for matching which attribute value may be described in the program that implements the name identification means 13, or is referred to by the program in the database generation device 10. The external table may be described so that the administrator of the database generation device 10 can freely change the external table.

FIG. 6 shows an example of the contents of such an external table. In FIG. 6A, the criteria for matching the data are described. In this example, the name and address are specified as the matching items. When the evaluation value of the collation result of the name is 90 points or more and the evaluation value of the collation result of the address is 80 points or more, or the evaluation value of the collation result of the name is 80 points or more and the evaluation value of the collation result of the address is 90 points or more In the case of, it is determined that the two data match. In FIG. 6B, the collation method of the name is described. Here, exact matching, character-based matching, and word-based matching are specified as matching methods, and matching is performed by each method. If there is a match in the complete matching process, the evaluation value is 10
The evaluation value is 0, and if they do not match, the evaluation value is 0. The evaluation value of the matching result of character unit matching is 100 in the ratio of the number of matching characters.
Multiplied by. Assume that the evaluation value of the matching result of word unit matching is 100 times the ratio of the number of matching words. The evaluation value of the matching method that returned the highest evaluation value is the nominal evaluation value. In FIG. 6C, the address matching method is also described. Here, exact matching and word-based matching are designated as matching methods. The evaluation value of the matching method that returned the highest evaluation value is used as the evaluation value of the address.

Next, in the combining means 14, the new database 18 after the name identification is stored in the database storage unit 16.
Then, the name identification data that can be regarded as having the same attribute values such as name and address is linked with the old database 17 which is generated by collecting the data from each information server 20 last time, and linking them. (Step 114).
For example, both data determined to be the same in the old and new databases 17 and 18 are linked and associated with each other by adding information indicating that they are the same data.

Here, it is assumed that, in the information server 20, the link information of the same data can change over time. In deriving the update information of each data, it is necessary to compare the update date and time of the new data and the old data, and for that purpose, it is necessary to judge which data is the same in the old and new databases. If the link information does not change, it can be judged whether the link information is the same, but under the assumption that the link information can change, it is necessary to judge whether the name and address attribute values of the data are the same. is there. Here, even with the same data, the name and the like may be slightly changed with the passage of time, so the collation is performed in consideration of the fluctuation of the notation. Specifically, for example, a matching method such as character-end-based matching or word-based matching other than exact matching is performed. Basically, it is similar to the case of name identification. Further, if the items to be collated are, for example, only the name, the old and new data will match even if the address of the same store changes. In this way, under what conditions are the old and new data identified
It can be adjusted by changing the matching rule. Figure 7
Shows an example of the data matching criterion in the matching rule described in the external table. Here, an example is shown in which only the name is specified as the matching item. The description of the name matching method may be the same as that shown in FIG. 6, for example.

FIG. 8 shows an example of the old database 17. For the sake of convenience, it is assumed in FIG. 8 that each data has not been updated since the last time. The combining unit 14 identifies the name identification data of the old database 17 that is the same as each name identification data of the new database 18 shown in FIG. 5 by collating only the nominal values or the attribute values of the nominal and address. As a result, the new database 18 of FIG.
The third name identification data is linked to the first, second, and third name identification data in the old database 17 of FIG. 8, respectively. The name identification data linked to the fourth name identification data of the new database 18 of FIG. 5 does not exist in the old database 17 of FIG. Data having the same corresponding information server in the linked name identification data are also linked as the same data. Hereinafter, each data in FIG. 5 and FIG. 8 will be expressed by the number from the top.

Next, in the update information adding means 15, the link information of the data of the new database 18 and the information of the update date and time, and the link information of the data in the old database 17 which is judged to be the same by the combining means 14 and The update information is set / appended to the corresponding data in the new database 18 by comparing it with the update date / time information (step 1).
15). That is, when there is data in the old database 17 linked to the data in the new database 18 and the link information or the update date / time has been changed, it is determined that the data has been updated, and both have been changed. If there is not, the data is determined not to be updated, and the update information of the corresponding data in the new database 18 is set to "update" or "no update". If there is no data in the old database 17 linked to the data in the new database 18, it is determined that the data has been newly created, and the update information of the corresponding data in the new database 18 is set as “new”. To do.

For example, the first data of the new database 18 of FIG. 5 has the same link information as the first data of the linked old database 17 of FIG. 8, and the update date and time has changed. The data is judged to have been updated.

The second data in the new database 18 of FIG. 5 is the second data of the linked old database 17 of FIG.
Since the link information and the update date and time are unchanged compared to the second data, it is determined that the data has not been updated. The same applies to the fourth data of the new database 18 of FIG.

The third data of the new database 18 of FIG. 5 is the third data of the linked old database 17 of FIG.
Compared with the second data, the update date and time does not change, but the link information has changed, so it is determined that the data has been updated.

The fifth data of the new database 18 of FIG. 5 is the name identification data as the old database 1 of FIG.
Although it is linked to the third name identification data of No. 7, but the data linked as data is not in the old database 17 of FIG. 8, it is determined to be newly created.

Since the data linked to the sixth data of the new database 18 of FIG. 5 does not exist in the old database 17 of FIG. 8, it is judged that the data is newly created.

In this way, the new database 1 of FIG.
8 and the old database 17 in FIG. 8, a new database 18 with update information as shown in FIG. 9 is finally generated. The update information adding means 15 overwrites the old database 17 with the finally created new database 18.

With the above, the generation of the database is completed. The user terminal 30 is added to the finally generated database.
When accessing from, and searching for and displaying data that matches the user's request, information on the business name, name, and address of the name identification data, and link information and update information to a file in the information server 20 in which the data exists are displayed. To be done. Figure 10
9 is a display example of a search result when a store whose business type is “Chinese” and whose address is “Kagurazaka, Shinjuku-ku” is searched by the generation database of FIG. 9. By clicking this link information on the screen, the user can access the detailed information of the store, which is the content of the linked file.

The typical embodiment of the present invention has been described above. The old database before name identification is retained,
Even if the linking of the combining means 14 is performed not between the old and new databases (FIGS. 5 and 8) after name identification, but between the old and new databases before the name identification (old databases equivalent to FIGS. 4 and 4). Good. For example, in this case, the correspondence information server collates the same data.

In the information server 20, even if the link information of the same data changes with the passage of time, if the data contains ID information that is permanently invariable for each data, the following processing is performed. You can also The attribute information extracting means 12 extracts the ID information and links the combining means 14 by linking each data in the new database with the data in the old database having the same ID information as the data. To do.

In the information server 20, if the link information of the same data is always unchanged, the linking means 14 need not be linked. In the update information adding means 15, if the update date and time of the data in the generated database (which may be before or after the name identification) is after the time of collecting the data set before, the data is the updated data. It turns out that this is new data. Further, if the data having the same link information as the corresponding information server of the data is in the previously generated database, it is determined that the data is the updated data, and if not, it is the new data.

Besides the above, the present invention can be variously modified and expanded within the scope of the claims. For example, a configuration in which only the update information of each data is given without the name identification means and the name identification process is conceivable.

It should be noted that a part or all of the processing functions of each unit in the apparatus shown in FIG. 1 can be configured by a computer program, and the program can be executed by the computer to implement the present invention, or ,
It goes without saying that the processing procedure shown in FIG. 2 can be configured by a computer program and the computer can be caused to execute the program. Also, a program for realizing the processing function on a computer, or
A program for causing a computer to execute the processing procedure, a computer-readable recording medium,
For example, FD, MO, ROM, memory card, CD,
The program can be recorded on a DVD, a removable disk, etc., and saved or provided, and the program can be distributed through a network such as the Internet.

[0042]

As described above, according to the present invention,
When the data that meets the user's request is searched from the generated database, it is possible to display the search result in a form in which there is no duplicate data and data update information is added.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a database generation device according to an embodiment of the present invention.

FIG. 2 is a flow chart diagram of a database generation method according to an embodiment of the present invention.

FIG. 3 is a diagram showing an example of data downloaded from an information server.

FIG. 4 is a diagram showing an example of a new database generated by an attribute information extraction unit.

FIG. 5 is a diagram showing an example of a new database updated by a name identification means.

FIG. 6 is a diagram showing an example of a matching rule applied by a name identification means.

FIG. 7 is a diagram showing an example of a matching rule applied by a combining unit.

FIG. 8 is a diagram showing an example of an old database generated last time.

FIG. 9 is a diagram showing an example of a new database to which update information is added by update information adding means.

FIG. 10 is a diagram showing an example of a display screen of search results from a database generated according to the present invention.

FIG. 11 is a diagram showing an example of a display screen of search results from a database generated by a conventional database generation technique.

[Explanation of symbols]

10 Database generator 11 Data collection means 12 Attribute information extraction means 13 means of gathering people 14 Coupling means 15 Update information giving means 16 Database storage 17 Old Database 18 new database 20 Information server 30 user terminals 40 networks

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5B075 KK03 KK07 ND20 NK02 NK46                       NR03 NR14 NR20 UU40                 5B082 EA08 EA10

Claims (5)

[Claims] 1. An apparatus for collecting data from a plurality of points to generate a database, a storage unit for storing a database generated in the past (hereinafter referred to as an old database), a name, an address, etc. from each of the points. Data including the value of the attribute, identification ID of the data, data collection means for collecting update date / time information, and extracting the value of the attribute from each of the collected data, and each data is at least the value of the extracted attribute. Attribute information extraction means for generating a database (hereinafter referred to as a new database) having a structure consisting of the ID, the identification ID, and the update date and time, and a name grouping for classifying a data set in which the attribute values in the generated new database can be regarded as the same into the same group. Means, the new database and the old database, the data whose attribute values can be regarded as the same are judged to be the same, and Connecting means for associating each data between data sets, information such as identification ID and update date and time of data in the new database, and identification ID and update date and time of data in the old database associated with the data Update information adding means for adding update information to the corresponding data in the new database by comparing the information with the above information. 2. The database generation device according to claim 1, wherein the update information adding means uses the identification ID of the data of the new database and the information of the update date and time as the identification ID of the corresponding data of the old database and the information of the update date and time. If there is no match, update information indicating that there is an update, and if there is a match, update information is added to the corresponding data in the new database. If there is no data associated with the data in the new database in the old database, the update information in the new database is added. A database generation device characterized by adding update information indicating new to corresponding data. 3. A method for automatically generating a database by collecting data from a plurality of points, wherein data including attribute values such as names and addresses, identification IDs of the data, and update date / time information are collected from each of the points. A data collection process to be collected and an attribute value is extracted from each of the collected data, and each data is stored in a database (hereinafter referred to as a new database) composed of at least the extracted attribute value, identification ID, and update date and time. An attribute information extraction step of generating, a name identification step of classifying data sets in which the attribute values in the generated new database can be regarded as the same into the same group, and the old database generated and retained in the past with the new database And a combining process of determining that the data whose attribute values can be regarded as the same are the same and associating each data between both databases, Corresponding data in the new database by comparing information such as identification ID and update date and time of data in the database with information such as identification ID and update date and time of data in the old database associated with the data And a step of adding update information to the database, the method for generating a database. 4. The database generation method according to claim 3, wherein at least one of the name identification process and the combining process is omitted, and in the update information giving process, at least the combining process is omitted, A database generation method characterized by recognizing correspondence between new database data and old database data based on invariant information. 5. A computer-executable program for collecting data from a plurality of points to generate a database, the data including attribute values such as names and addresses from each of the points, and the data of the data. A data collection process for collecting identification IDs and update date / time information, and a database having a configuration in which attribute values are extracted from each of the collected data, and each data includes at least the extracted attribute values, identification IDs, and update dates / times. (Hereinafter referred to as a new database), an attribute information extraction process, a name identification process that classifies data sets that can be regarded as having the same attribute value in the generated new database into the same group, and the new database and the past Data that can be regarded as having the same attribute values as the old database are judged to be the same, and Comparing the information such as the identification ID and update date and time of the data in the new database with the information such as the identification ID and update date and time of the data in the old database associated with the data. By doing so, an update information giving process of giving update information to the corresponding data in the new database, a database generation program.