JP5710519B2 - Synonym extraction device, method, and program - Google Patents

Description

  The present invention relates to a synonym extraction device, method, and program, and more particularly, to a synonym extraction device, method, and program for extracting synonyms of words included in a description of a product on a site.

  In recent years, more and more companies have opened stores on EC sites in addition to actual stores. For this reason, the market size of EC sites will increase, and it is important to improve sales at EC sites. In order to improve sales at EC sites, it is important to purchase products that meet customer needs, so it is necessary to estimate customer needs.

  For this reason, it is expected that sales will be improved by presuming the conditions under which many or important customers have selected products as customer needs and purchasing products that meet the conditions.

  Hereinafter, a condition for selecting a product to be viewed is referred to as a product selection condition.

  As a method of extracting the product selection condition, there is a method of extracting a word that appears in common with the viewed product as the product selection condition.

  However, as shown in FIG. 12, the item selection condition of one piece may be different from a dress, a mini one piece, a shirt one piece, or the like. Such a word is called a synonym.

  If the description differs depending on the synonym, the frequency at which the words appear in common in the viewed product cannot be calculated. This is because the appearance frequency is calculated for each different word. Therefore, if the description differs depending on the synonym, the product selection condition cannot be extracted.

  Therefore, when the synonyms of the words included in the browsed product are complemented, the product selection conditions are those that frequently appear in the browsed products, and the product selection conditions can be correctly extracted.

  As a method for obtaining a synonym, there is a method of extracting a word having a high frequency that appears at the same time as a word in a document as a synonym. Here, the document includes arbitrary text, such as a page.

  However, when synonyms are extracted from the description on the product detail page on the EC site, it is unlikely that the description of the shirt dress will occur at the same time for the same product with the description of the dress. This is because the description of the product detail page is short, and different contents are often described rather than similar descriptions. Here, the product detail page is a page in which detailed information is described for an arbitrary product.

  Therefore, it is considered effective to calculate the frequency of simultaneous appearance from the description in the same session, rather than calculating the frequency of simultaneous appearance of the same product. Here, a series of browsing continuously browsed under the same intention is defined as a session.

  In Non-Patent Document 1, a search result including a certain word is used, but this method can be applied by regarding a session as a search result.

Takenobu Tokunaga, “Information Retrieval and Natural Language Processing”, University of Tokyo Press, P.A. 154-159, 1999

  When the number of sessions in which words co-occur within the same session is calculated as the co-occurrence frequency, there may be cases where there are sessions that browse different product categories within the same session.

  Thus, when the number of sessions in which words co-occur within the same session is calculated as the co-occurrence frequency, synonyms do not always have a high co-occurrence frequency. This is because even in the same session, it does not always have the same product selection condition, and a word with a high co-occurrence frequency may not be a synonym.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a synonym extraction device, method, and program capable of accurately extracting synonyms of words included in product descriptions on a site. To do.

In order to achieve the above object, a synonym extraction device of the present invention is a synonym extraction device for extracting synonyms of words included in a description of a product on a site, and at least one input for each session. Based on session information including one browsing product and a word group included in the description of each browsing product, a session having a predetermined number or more of browsing products including the extraction target word w1 is extracted for each word w1 to be extracted. An entire session extracted for the extraction target word w1 for each extraction target word w1 based on session extraction means and session information extracted by the session extraction means for each extraction target word w1 The number of sessions in which each word w2 appears is extracted as a co-occurrence frequency with the extraction target word w1. Based on the co-occurrence frequency of the frequency extraction means and the word w1 of the extraction target extracted by the co-occurrence frequency extraction means for each word w1 of the extraction target, the extraction target for each word w1 of the extraction target For each word w2 from which the co-occurrence frequency with the word w1 is extracted, the reciprocal of the sum of the number of the extraction target words w1 with the co-occurrence frequency equal to or higher than the first threshold and a predetermined positive constant IDF extraction means for extracting a certain word IDF, and for each word w1 to be extracted, the word to be extracted excluding the word w2 in which the word IDF extracted for the word w1 to be extracted is a second threshold value or less synonym extracting means for outputting a word w2 having a co-occurrence frequency with w1 equal to or higher than the first threshold as a synonym of the extraction target word w1.

The synonym extraction method of the present invention includes a session extraction unit, a co-occurrence frequency extraction unit, an IDF extraction unit, and a synonym extraction unit, and extracts synonyms of words included in the description of the product on the site. A synonym extraction method in an extraction device, wherein the synonym extraction device includes at least one browse product for each session and a word group included in the description of each browse product input by the session extraction unit. Based on the included session information, for each word w1 to be extracted, the step of extracting a session having a predetermined number or more of browsing products including the word w1 to be extracted, and the co-occurrence frequency extracting means Based on the session information of the session extracted by the session extraction means for the word w1, the word w1 to be extracted And extracting the number of sessions in which each word w2 appears in the entire session extracted for the word w1 to be extracted as a co-occurrence frequency with the word w1 to be extracted, and by the IDF extracting means, Based on the co-occurrence frequency of each word w1 to be extracted and the word w1 to be extracted extracted by the co-occurrence frequency extracting unit, the word w1 to be extracted is shared with the word w1 to be extracted. For each word w2 from which the occurrence frequency is extracted, a word IDF that is the reciprocal of the sum of the number of the extraction target words w1 with the co-occurrence frequency equal to or higher than a first threshold and a predetermined positive constant is extracted. And the word IDF extracted for the extraction target word w1 by the synonym extraction means for each extraction target word w1 is equal to or less than a second threshold value. Outputting a word w2 having a co-occurrence frequency with the extraction target word w1 excluding the word w2 as the synonym of the extraction target word w1. Features.

In addition, the program of the present invention is a program for extracting synonyms of words included in the description of products on a site, and the computer inputs at least one browse product for each session, and each browse Session extraction means for extracting, for each word w1 to be extracted, a session having a predetermined number or more of browsing products including the extraction target word w1, based on session information including a word group included in the description of the product, the extraction target A session in which each word w2 appears in the entire session extracted for the word w1 to be extracted for each word w1 to be extracted based on the session information of the session extracted by the session extraction means for each word w1 Is extracted as a co-occurrence frequency with the extraction target word w1. Means for each word w1 to be extracted for each word w1 to be extracted based on the co-occurrence frequency with the word w1 to be extracted extracted by the co-occurrence frequency extraction unit for each word w1 to be extracted For each word w2 from which the co-occurrence frequency is extracted, a word IDF that is the reciprocal of the sum of the number of words w1 to be extracted whose co-occurrence frequency is equal to or greater than a first threshold and a predetermined positive constant For each word w1 to be extracted and the word w1 to be extracted, excluding the word w2 in which the word IDF extracted for the word w1 to be extracted is equal to or less than a second threshold value A program for causing a word w2 having a co-occurrence frequency to be equal to or higher than the first threshold to function as a synonym extracting unit that outputs the word w2 as a synonym of the extraction target word w1.

  As described above, according to the synonym extraction device, method, and program of the present invention, for each word w1 to be extracted, a session having a predetermined number or more of browsing products including the word w1 is extracted, and each word w2 is extracted. By calculating the co-occurrence frequency with the word w1 for the word, and extracting the synonym of the word w1 except for the word w2 where the word IDF is less than or equal to the threshold, the synonym of the word included in the description of the product on the site The effect that it can extract well is acquired.

It is a block diagram which shows the functional structure of the synonym extraction apparatus of embodiment of this invention. It is a figure which shows the content of a session group input buffer. It is a figure which shows the content of a short-term session group input buffer. It is a figure which shows the content of the co-occurrence frequency preservation | save buffer. It is a figure which shows the content of the co-occurrence frequency input buffer. It is a figure which shows the content of IDF extraction buffer. It is a figure which shows the output of a synonym extraction apparatus. It is a flowchart which shows the content of the synonym extraction process routine in the synonym extraction apparatus of embodiment of this invention. It is a flowchart which shows the flow of the process which extracts the short-term session in the synonym extraction apparatus of embodiment of this invention. It is a flowchart which shows the flow of the process which extracts the co-occurrence frequency in the synonym extraction apparatus of embodiment of this invention. It is a flowchart which shows the flow of the process which extracts the word IDF in the synonym extraction apparatus of embodiment of this invention. It is a figure for demonstrating a synonym. It is a figure which shows the result of having calculated the co-occurrence frequency in all the sessions. It is a figure which shows the result of having calculated the co-occurrence frequency in a short-term session.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline of the present invention>
When the number of sessions in which words co-occur in the same session is calculated as the co-occurrence frequency in all sessions, there may be a session in which different product categories are viewed in the same session.

  Thus, when the number of sessions in which words co-occur within the same session is calculated as the co-occurrence frequency, synonyms do not always have a high co-occurrence frequency. This is because even a word that co-occurs in the same session does not necessarily have the same product selection condition, and a word having a high co-occurrence frequency may not be a synonym.

  FIG. 13 shows a case where the co-occurrence frequency is calculated in all sessions including one piece. It can be seen that synonyms such as shirt dress and mini dress do not come to the top, and words such as black that are easily included in the product detail page and words that are not synonymous with other categories come to the top.

  Therefore, in the present invention, synonyms are extracted by calculating the co-occurrence frequency only in sessions browsed under the same product selection condition.

  Browsing under different product selection conditions by defining a session viewed under the same product selection conditions as a session (referred to as a short-term session) that has N or more browsing products that contain words in common with the target word It is possible to reduce the influence of words.

  In addition, synonyms are more accurately extracted by removing words that co-occur in a session viewed under the same product selection conditions appear in a plurality of words.

  FIG. 14 shows an example in which the co-occurrence frequency is calculated and the synonyms are extracted only in the session browsed under the same product selection condition including one piece. Unlike FIG. 13, it can be seen that the synonyms of mini dress and shirt dress are included at the top. On the other hand, words such as black that are likely to be included in the product detail page remain at the top. Therefore, on the assumption that synonyms do not appear frequently together with a plurality of words, those words are removed by removing from the synonym candidates words whose word IDF, which is the reciprocal of the appearance frequency of the words, is less than or equal to a threshold value.

<System configuration>
A synonym extraction device 100 according to an embodiment of the present invention includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read that stores a program for executing a synonym extraction processing routine described later. Only memory). As shown in FIG. 1, this computer can be functionally represented by a configuration including a session group input unit 10, a synonym extraction unit 20, a word group dictionary database 30, and a synonym output unit 40. The synonym output unit 40 is an example of a synonym extraction unit.

  The session group input unit 10 receives input of session information for each session obtained at at least one EC site. Here, a session represents a log of browsing pages continuously browsed by the same viewer on the same site. As shown in FIG. 2, the session information includes a session ID that is a pre-defined session ID, a user ID indicating which user has browsed, a browsed product browsed during the session, and a description of the browsed product. It includes a word group that is a set of appearing words and an appearance frequency (0 or 1) of each word in the word group, and has a browsing product, a word group, and an appearance frequency of each word for each browsing product.

  The word group dictionary database 30 stores a large number of word data as a word group dictionary.

  The synonym extraction unit 20 includes a short-term session extraction unit 21, a co-occurrence frequency extraction unit 22, and an IDF extraction unit 23.

  The short-term session extraction unit 21 includes a session group input buffer 211.

  The session group input buffer 211 stores session information for each session shown in FIG. 2 input by the session group input unit 10.

  The short-term session extraction unit 21 starts an operation after writing to the session group input buffer 211 is completed, and, as will be described below, for each word in the word group dictionary database 30, a short-term session for the word is performed. Extract.

  First, one word w1 is extracted from the word group dictionary database 30, then one session information is extracted from the session group input buffer 211, and the contents of the session information are written in a session storage buffer (not shown). When the number of products including the word w1 among the browsed products in the session information stored in the session storage buffer is greater than or equal to a threshold value N, as shown in FIG. The word w1 and session information (session ID, user ID, browsing product, word group, frequency of appearance of each word) are written in.

  As described above, the short-term session extraction unit 21 determines, for each word w1 in the word group dictionary database 30, the number of browsing products including the word w1 from the set of all sessions in which session information is input as a threshold value. A subset of N sessions or more is extracted as a short-term session for the word w1, and information on each session included in the short-term session is associated with the word w1 as shown in FIG. Write to the group input buffer 221.

  The co-occurrence frequency extraction unit 22 includes a short-term session group input buffer 221 and a co-occurrence frequency storage buffer 222.

  The short-term session group input buffer 221 stores information on the short-term session shown in FIG. 2 written by the short-term session extraction unit 21.

  The co-occurrence frequency extraction unit 22 starts the operation after the writing to the short-term session group input buffer 221 is completed. As described below, for each word w1, the word w1 in the short-term session and the word w1 Is calculated for each word w2.

  All of the session information extracted as the short-term session of the word w1 is extracted from the short-term session group input buffer 221. Next, one word w 2 is extracted from the word group dictionary database 30. As shown in FIG. 4, the frequency at which the word w2 appears in the short-term session of the word w1 (the number of sessions including the word w2) is calculated, and the calculated appearance frequency is the co-occurrence frequency of the word w2 with the word w1. To the co-occurrence frequency storage buffer 222.

  Thus, for each word w1, in the short-term session for the word w1, the number of sessions in which each word w2 appears is calculated. As shown in FIG. 4, the co-occurrence frequency as the co-occurrence frequency with the word w1 is calculated. Stored in the save buffer 222.

  The IDF extraction unit 23 includes a co-occurrence frequency input buffer 231 and an IDF extraction buffer 232.

  The co-occurrence frequency input buffer 231 stores the same contents stored in the co-occurrence frequency storage buffer 222 as shown in FIG.

  The IDF extraction unit 23 starts the operation after the writing to the co-occurrence frequency input buffer 231 is completed, and extracts the word IDF of each word w2 and extracts synonyms for each word w1, as described below. .

  First, one word w 1 is extracted from the word group dictionary database 30. From the co-occurrence frequency input buffer 231, a word w 2 whose co-occurrence frequency with the word w 1 is equal to or greater than the threshold value M is extracted as a synonym candidate, and the co-occurrence of the word w 2 for each word w 2 extracted as a synonym candidate The number of words w1 whose frequency is greater than or equal to the threshold value M is calculated as the word appearance frequency (word DF) (see FIG. 6). The word IDF is calculated for the word w2 from the word appearance frequency calculated for the word w2 by using the following expression (1), and the word whose IDF is equal to or less than the threshold value L is removed from the synonym candidate of the word w1. .

  However, f is obtained by adding 1 to the word appearance frequency calculated for the word w2 whose co-occurrence frequency with the word w1 is equal to or greater than the threshold value M, and k is a normalization constant.

  For each word w1, the IDF extraction unit 23 removes the word w2 whose word IDF is equal to or less than the threshold value L from the synonym candidate (word that co-occurs with the word w1), and then extracts the synonym candidate as a synonym. Write to the output unit 40. The synonym output unit 40 outputs the synonym of each word w1, as shown in FIG. 7, based on the written synonym candidate for each word w1.

<Operation of synonym extraction device>
When session information for each session is input to the synonym extraction device 100 of the present embodiment, the synonym extraction device 100 stores the session information in the session group input buffer 211. Then, in the synonym extraction device 100, a synonym extraction processing routine shown in FIG. 8 is executed.

  First, in step S100, the short-term session extraction unit 21 creates a short-term subset of sessions in which the number of browsing products including the word w1 is N or more for each word w1 from the session set in the session group input buffer 211. Extract as a session. In step S101, the co-occurrence frequency extraction unit 22 calculates the appearance frequency of each word w2 in each session of the extracted word w1 for each word w1, and the entire short-term session of the extracted word w1. The co-occurrence frequency with the word w1 is calculated for each word w2. In step S102, the IDF extraction unit 23 calculates the word IDF of the word w2 for each word w2 of the synonym candidate of the word w1 for each word w1, and the word w2 for which the word IDF is less than the threshold value L. Are extracted from the synonym candidate of the word w1 as words co-occurring with many words w1, the synonym of the word w1 is extracted, and the synonym extraction processing routine is terminated.

  The step S100 is realized by a short-term session extraction processing routine shown in FIG.

  In step S110, one word w1 is extracted from the word group dictionary database 30. In step S111, one piece of session information is extracted from the session group input buffer 211 and written to the session storage buffer. In step S112, when N or more browsed products among the browsed products included in the session information extracted in step S111 include the word w1, the session information written in the session storage buffer is changed to the short-term word w1. As one of the sessions, the short-term session group input buffer 221 is written.

  In the next step S113, it is determined whether or not the session group input buffer 211 is empty. If the session group input buffer 211 is not empty, the process returns to step S111 and continues. On the other hand, if the session group input buffer 211 is empty, the process proceeds to step S114.

  In step S114, it is determined whether or not the above process has been executed for all the words in the word group dictionary database 30, and if not all the words in the word group dictionary database 30 have been processed, the process returns to step S110. Continue. On the other hand, if all the words in the word group dictionary database 30 have been processed, the short-term session extraction processing routine is terminated.

  Step S101 is realized by the co-occurrence frequency extraction processing routine shown in FIG.

  In step S120, information on each session included in the short-term session of the word w1 is extracted from the short-term session group input buffer 221, and one word w2 is extracted from the word group dictionary database 30 in step S121. In step S122, the appearance frequency of the word w2 in the short-term session of the word w1 extracted in step 120 (the number of sessions in which the word w2 appears) is calculated, and the co-occurrence frequency storage buffer 222 is calculated as the co-occurrence frequency with the word w1. Write to.

  In step S123, it is determined whether or not the above processing has been executed for all the words in the word group dictionary database 30, and if all the words in the word group dictionary database 30 have not been processed, the process returns to step S121. Continue processing. On the other hand, if all the words in the word group dictionary database 30 have been processed, the process proceeds to step S124.

  In step S124, it is determined whether or not the short-term session group input buffer 221 is empty. If the short-term session group input buffer 221 is not empty, the process returns to step S120 and the processing is continued. On the other hand, if the short-term session group input buffer 221 is empty, the co-occurrence frequency extraction processing routine is terminated.

  Step S102 is realized by the IDF extraction processing routine shown in FIG.

  In step S130, one word w1 is extracted from the word group dictionary database 30. In step S131, a word w2 whose co-occurrence frequency with the word w1 is equal to or greater than the threshold value M is extracted from the co-occurrence frequency input buffer 231 as a synonym candidate, and the word appearance frequency (DF) is set for each word w2 of the synonym candidate. ) Is calculated.

  In step S132, based on the word appearance frequency of each word w2 calculated in step S131, the word IDF of each word w2 is calculated, and the word w2 whose word IDF is equal to or less than the threshold L is the synonym candidate for word w1. Remove from. In the next step S133, the synonym candidate from which the word is removed in step S132 is written in the synonym output unit 40 as a synonym for the word w1.

  In step S134, it is determined whether or not the above process has been executed for all the words in the word group dictionary database 30, and if not all the words in the word group dictionary database 30 have been processed, the process returns to step S130. Continue processing. On the other hand, if all the words in the word group dictionary database 30 have been processed, the IDF extraction processing routine ends.

  And the synonym output part 40 outputs the synonym of each written word w1.

  As described above, according to the synonym extraction device according to the embodiment of the present invention, for each word w1 to be extracted, a session having a predetermined number or more of browsing products including the word w1 is extracted as a short-term session. In the short-term session, the co-occurrence frequency of each word w2 with the word w1 is calculated, and from the synonym candidates extracted based on the co-occurrence frequency, the word w1 having the word IDF equal to or lower than the threshold is excluded. Extract synonyms. In this way, by limiting the target for calculating the appearance frequency to sessions (short-term sessions) that are browsed under the same product selection conditions, synonyms have a high co-occurrence frequency especially with certain words. The synonym of the word included in the description of the product on the site can be accurately extracted.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, the above synonym extraction device has a computer system inside, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. .

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Session group input part 20 Synonym extraction part 21 Short-term session extraction part 22 Co-occurrence frequency extraction part 23 IDF extraction part 30 Word group dictionary database 40 Synonym output part 100 Synonym extraction apparatus

Claims (3)

A synonym extraction device that extracts synonyms of words included in a description of a product on a site,
Based on the input session information including at least one browsing product for each session and a word group included in the description of each browsing product, the browsing including the extraction target word w1 for each word w1 of the extraction target Session extraction means for extracting a session having a predetermined number of products;
Based on the session information of the session extracted by the session extraction unit for each word w1 to be extracted, for each word w1 to be extracted, each word w2 in the entire session extracted for the word w1 to be extracted is Co-occurrence frequency extraction means for extracting the number of appearing sessions as the co-occurrence frequency with the word w1 to be extracted;
Based on the co-occurrence frequency of each word w1 to be extracted and the word w1 to be extracted extracted by the co-occurrence frequency extraction unit, for each word w1 to be extracted, the word w1 to be extracted For each word w2 from which the co-occurrence frequency has been extracted, a word IDF that is the reciprocal of the sum of the number of the extraction target words w1 with the co-occurrence frequency equal to or higher than the first threshold and a predetermined positive constant is extracted. IDF extraction means for
For each word w1 to be extracted, the co-occurrence frequency with the word w1 to be extracted is the first threshold value except for the word w2 in which the word IDF extracted for the word w1 to be extracted is equal to or less than a second threshold value. Synonym extraction means for outputting the word w2 as above as a synonym of the word w1 to be extracted;
Synonym extraction device. A synonym extraction method in a synonym extraction device that includes a session extraction unit, a co-occurrence frequency extraction unit, an IDF extraction unit, and a synonym extraction unit, and extracts synonyms of words included in a description of a product on a site,
The synonym extraction device is:
For each word w1 to be extracted, based on session information including at least one browsing product for each session and a word group included in the description of each browsing product input by the session extraction unit, the extraction target Extracting a session having a predetermined number or more of browsing products including the word w1;
The co-occurrence frequency extraction unit extracts the extraction target word w1 for each extraction target word w1 based on the session information of the session extracted by the session extraction unit for each extraction target word w1. Extracting the number of sessions in which each word w2 appears in the entire session as a co-occurrence frequency with the word w1 to be extracted;
Based on the co-occurrence frequency of the extraction target word w1 extracted by the co-occurrence frequency extraction unit for each of the extraction target words w1 by the IDF extraction unit, the extraction is performed for each extraction target word w1. For each word w2 from which the co-occurrence frequency with the target word w1 has been extracted, the reciprocal of the sum of the number of the extraction target words w1 with the co-occurrence frequency equal to or higher than the first threshold and a predetermined positive constant Extracting a word IDF which is
For each word w1 to be extracted by the synonym extracting means, the word ID1 extracted with respect to the word w1 to be extracted is excluded from the word w1 to be extracted except for the word w2 having a second threshold value or less. Outputting a word w2 having an occurrence frequency equal to or higher than the first threshold as a synonym of the extraction target word w1,
The synonym extraction method characterized by including, and performing. A program for extracting synonyms of words in product descriptions on a site,
Computer
Based on the input session information including at least one browsing product for each session and a word group included in the description of each browsing product, the browsing including the extraction target word w1 for each word w1 of the extraction target Session extraction means for extracting a session having a predetermined number of products,
Based on the session information of the session extracted by the session extraction unit for each word w1 to be extracted, for each word w1 to be extracted, each word w2 in the entire session extracted for the word w1 to be extracted is Co-occurrence frequency extraction means for extracting the number of appearing sessions as the co-occurrence frequency with the word w1 to be extracted;
Based on the co-occurrence frequency of each word w1 to be extracted and the word w1 to be extracted extracted by the co-occurrence frequency extraction unit, for each word w1 to be extracted, the word w1 to be extracted For each word w2 from which the co-occurrence frequency has been extracted, a word IDF that is the reciprocal of the sum of the number of the extraction target words w1 with the co-occurrence frequency equal to or higher than the first threshold and a predetermined positive constant is extracted. Co-occurrence with the extraction target word w1 excluding the word w2 for which the word IDF extracted for the extraction target word w1 is equal to or less than a second threshold value for each extraction target word w1 A program for causing a word w2 having a frequency equal to or higher than the first threshold to function as a synonym extracting unit that outputs the word w2 as a synonym of the extraction target word w1.