JP7497384B2 - Text conversion support device and text conversion support method - Google Patents

Description

The present invention relates to a text conversion assistance device and a text conversion assistance method.

There is a need to check whether the contents of phone calls made by salespeople or call centers are appropriate from the perspective of compliance, etc. In recent years, instead of the traditional method of listening to and checking the recorded data of such phone calls, a method has been proposed in which the voice data is converted into text and then checked.
Examples of prior art related to such text conversion include a compliance check system and compliance check program (see Patent Document 1) that checks whether or not "prohibited expressions" are present and whether or not "required information" is included, based on data such as the content of explanations given to customers during business negotiations and sales activities.

This technology is a compliance check system that checks whether compliance is being observed with respect to each utterance made by a sales representative to a customer, and includes a text analysis unit that performs natural language analysis processing, including morphological analysis, on text data that has been converted into text using voice recognition technology from the content of each utterance made by the sales representative, and outputs the resulting analyzed text data; a determination unit that organizes each utterance in the analyzed text data relating to each utterance into blocks consisting of one or more consecutive utterances according to a predetermined criterion, and determines whether the content of first text data that is predefined as necessary matters that should be explained to the customer is explained in each block; and a determination unit that maps the analyzed text data relating to each utterance to second text data that is predefined as the content of prohibited expressions that should not be said to customers. The system has a keyword matching unit that determines that the prohibited expression of the target is mentioned in the target utterance if there is a keyword matching unit that matches the keyword matching unit, and a data recording unit that records the data of each utterance made by the sales representative to the customer in association with management information that identifies the sales representative and/or the customer, and the text analysis unit includes past utterances made by the sales representative to the customer that are extracted based on the management information in the data of each utterance made by the sales representative to the customer, and the determination unit, when it determines that the content of the first text data is explained in the block, assigns a category of the necessary items to the block and records it, and determines the degree to which each of the necessary items has been explained based on a predetermined evaluation criterion that has been set in advance.

JP 2018-120640 A

The accuracy of the above-mentioned text conversion has been improved through the advancement of deep learning technology, etc., and its utilization is progressing. For example, one example of the utilization of recorded telephone conversation data in the financial field is to check whether or not inappropriate words are used, and whether or not customer names and product names are pronounced correctly.
In many cases, this check involves keyword matching of the text of recorded conversation data. However, it has been found that there are calls where the accuracy of the text conversion is low (calls with many false positives) due to factors such as the recording conditions and the speaker's habits, making it difficult to perform accurate keyword matching on such calls.

In other words, even if keyword matching is performed on phone calls, where the accuracy of voice-to-text conversion tends to be low, the accuracy cannot be expected, which ultimately leads to missed checks.

The object of the present invention is to provide a technology that enables accurate keyword matching of the contents of a call, regardless of the characteristics of the speech-to-text conversion of the contents of the call.

The text conversion assistance device of the present invention that solves the above problem includes a storage device that stores master data that specifies correct phoneme information for each vocabulary expected to appear in each conversation scene or object , and information that specifies speech similarity between vowels , a process that applies call recording data obtained from a specified device to an acoustic model to extract phonemes, and a calculation of a match rate between the extracted phonemes and the correct phonemes of vocabulary expected to appear in the conversation scene or object of the call recording data among the vocabulary whose phonemes are specified in the master data. The method includes a process of calculating a matching rate between a phoneme and vowels contained in each of the extracted phonemes based on information about the speech similarity, calculating a matching rate between the correct phoneme and consonants contained in each of the extracted phonemes, weighting the matching rate between the vowels more heavily than the matching rate between the consonants, and calculating a matching rate between the phonemes based on the matching rates between the vowels and the consonants, and a calculation device that executes a process of identifying, as a keyword matching result, the vocabulary whose phonemes show a predetermined matching rate as a result of the calculation.
The text conversion assistance method of the present invention includes a process in which an information processing device stores master data defining correct phoneme information for each vocabulary expected to appear in each conversation scene or object , and information defining speech similarity between vowels, in a storage device, and applies call recording data obtained from a specified device to an acoustic model to extract phonemes; and when calculating a match rate between the correct phonemes of vocabulary expected to appear in relation to the conversation scene or object of the call recording data, among the vocabulary whose phonemes are defined in the master data, and the extracted phonemes , The method is characterized by executing a process of calculating a matching rate between vowels contained in each of the correct phonemes and the extracted phonemes based on the speech similarity information, calculating a matching rate between consonants contained in each of the correct phonemes and the extracted phonemes, weighting the matching rate between the vowels more heavily than the matching rate between the consonants, and calculating a matching rate between the phonemes based on the matching rates between the vowels and the consonants as a result of the calculation, and a process of identifying the vocabulary whose phonemes show a predetermined matching rate as a keyword matching result.

According to the present invention, it is possible to perform keyword matching of the contents of a call with high accuracy, regardless of the characteristics of the speech-to-text conversion of the contents of the call.

1 is a diagram showing a network configuration including a text conversion assistance device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of the text conversion assistance device according to the present embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of an operator terminal according to the present embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a call center system according to the present embodiment. 2 is a diagram illustrating an example of a hardware configuration of an administrator terminal according to the present embodiment. FIG. 2 is a diagram showing an example of the configuration of a call recording DB according to the present embodiment; FIG. FIG. 4 is a diagram showing an example of the configuration of a phoneme master table according to the present embodiment. FIG. 4 is a diagram illustrating an example of a configuration of an utterance similarity table according to the present embodiment. FIG. 2 is a diagram showing a first flow example of a text conversion assistance method according to the present embodiment. FIG. 11 is a diagram showing a second flow example of the text conversion assistance method in this embodiment. FIG. 11 is a diagram showing a flow example 3 of the text conversion assistance method in this embodiment.

<Network Configuration>
An embodiment of the present invention will be described in detail below with reference to the drawings. Fig. 1 is a network configuration diagram including a text conversion assistance device 100 according to the present embodiment. The text conversion assistance device 100 shown in Fig. 1 is a computer that can perform keyword matching of the contents of a telephone call with good accuracy, regardless of the characteristics of the speech-to-text conversion of the contents of the telephone call.

As shown in FIG. 1, the text conversion assistance device 100 of this embodiment is communicatively connected to an operator terminal 200, a call center system 300, and an administrator terminal 400 as necessary via an appropriate network 1, such as the Internet or a secure line within an organization. Therefore, these may be collectively referred to as the text conversion system 10.

The text conversion support device 100 of this embodiment can be said to be a support device that uses keyword matching to identify events such as whether the content of a conversation between an operator and a customer at a call center was appropriate from the standpoint of compliance or customer service, or whether inappropriate words appeared in the conversation or whether essential words did not appear.

Of course, the present invention is not limited to only situations where keyword matching is performed in conversations between operators and customers in call center operations, but can also be applied to any operations where there is an opportunity to verify the occurrence of necessary/prohibited keywords in voice data.

On the other hand, the operator terminal 200 is a terminal used by a person in charge of responding to customer inquiries regarding various products and services, or conducting telephone sales to potential customers, etc. Specifically, it can be a telephone terminal integrated with a PC, a smartphone, a tablet terminal, a personal computer, etc. Conversations between the person in charge and the customer on such an operator terminal 200 are recorded, and are managed and utilized as call recording data.

The call center system 300 also manages incoming and outgoing calls between the operator terminal 200 and the customer's telephone, and manages recorded call data, which is the content of conversations at the operator terminal 200. Therefore, the call center system 300 stores and manages the recorded call data in a storage device, and distributes it to the text conversion support device 100 as appropriate.

The administrator terminal 400 is a terminal operated by the administrator of the call center. The administrator terminal 400 instructs the text conversion assistance device 100 to perform keyword matching processing at an appropriate timing, such as when the business hours at the call center are finished, to check the call recording data for a predetermined period, such as one day, from a predetermined viewpoint, such as the above-mentioned compliance, and obtains the processing result.
<Hardware Configuration>
The hardware configuration of the text conversion assistance device 100 of this embodiment is as shown in FIG.

That is, the text conversion assistance device 100 includes a storage device 101, a memory 103, a computing device 104, and a communication device 105.

Of these, the storage device 101 is composed of an appropriate non-volatile storage element such as an SSD (Solid State Drive) or a hard disk drive.

In addition, memory 103 is composed of a volatile memory element such as RAM.

The calculation device 104 is a CPU that executes the program 102 stored in the storage device 101 by reading it into the memory 103, etc., and performs overall control of the device itself as well as various judgments, calculations, and control processes.

The communication device 105 is assumed to be a network interface card or the like that is connected to the network 1 and handles at least communication processing with the call center system 300.

If the text conversion assistance device 100 is a standalone machine, it is preferable for it to further include an input device that accepts key input or voice input from the user, and an output device such as a display that displays the processed data.

In addition to the program 102 for implementing the functions required for the text conversion assistance device of this embodiment, at least a call recording DB 125, a phoneme master table 126, and a speech similarity table 1276 are stored in the storage device 101. However, details of these databases will be described later.

The program 102 also includes an acoustic model 110 and a language model 111. The acoustic model 110 is a function for extracting phonemes that constitute the voice of a conversation between an operator and a customer from call recording data.

Therefore, the text conversion assistance device 100 performs an acoustic analysis in advance to analyze the voice features (frequency and volume) indicated by the call recording data and convert it into data that is easy to handle, and provides the features indicated by the results of this acoustic analysis to the acoustic model 110.

The acoustic model 110 is a model that defines the correspondence between the above-mentioned features and phonemes through appropriate deep learning, etc., and extracts phonemes, which are the smallest units of sound waves, by providing the above-mentioned voice features.

A phoneme is the smallest component of a sound wave that can be observed when a sound is produced. In Japanese, phonemes consist of three types: vowels (aiueo), onomatopoeia (n), and consonants (23 types). For example, the phonemes for "Tanaka-san" are "t-a-n-a-k-a-s-a-n."

The assistive text generation device 100 of this embodiment performs keyword matching based on the phonemes obtained by the acoustic model 110. In the above example, the phoneme "tanakasan"
is a process of identifying the Japanese vocabulary "Tanaka-san." More specifically, the phonemes are replaced with vocabulary by applying the text conversion assistance method of the present invention while appropriately using the phoneme master table 126 to determine which word each phoneme corresponds to.

On the other hand, the language model 111 is responsible for the process of appropriately converting the vocabulary clusters obtained by keyword matching into sentences. For example, it converts vocabulary clusters such as "Mr. Tanaka," "In Shinshu," "Snow," and "It's piled up" into meaningful sentences as examples of highly probable combinations based on statistical data about the relationship between the vocabulary clusters and correct (or frequently occurring) sentences.

The hardware configuration of the operator terminal 200 in this embodiment is as shown in Figure 3.

That is, the operator terminal 200 includes a storage device 201, a memory 203, a computing device 204, an input device 205, an output device 206, and a communication device 207.

Of these, the storage device 201 is composed of an appropriate non-volatile storage element such as an SSD (Solid State Drive) or a hard disk drive.

In addition, memory 203 is composed of a volatile memory element such as RAM.

The arithmetic unit 204 is a CPU that reads out the program 202 stored in the storage unit 201 into the memory 203 and executes the program, thereby controlling the device itself and carrying out various types of judgment, calculation and control processing.

The input device 205 is also composed of devices such as a keyboard and mouse that accept key input and voice input from the user/operator.

The output device 206 is composed of devices such as a display and a speaker that display the processing results of the calculation device 204.

The communication device 207 is assumed to be a network interface card or the like that is connected to the network 1 and handles communication processing with the call center system 300 and the administrator terminal 400 (or the text conversion assistance device 100).

The hardware configuration of the call center system 300 in this embodiment is as shown in Figure 4.

That is, the call center system 300 includes a storage device 301, a memory 303, a computing device 304, and a communication device 305.

Of these, the storage device 301 is composed of an appropriate non-volatile storage element such as an SSD (Solid State Drive) or a hard disk drive.

In addition, memory 303 is composed of a volatile memory element such as RAM.

The calculation device 304 is a CPU that executes the program 302 stored in the storage device 301 by reading it into the memory 303, etc., to perform overall control of the device itself, as well as various judgments, calculations, and control processes.

The communication device 305 is assumed to be a network interface card or the like that is connected to the network 1 and handles communication processing with at least the text conversion assistance device 100 and the operator terminal 200.

If the call center system 300 is a standalone machine, it is preferable for it to further include an input device that accepts key input or voice input from the user, and an output device such as a display that displays the processed data.

In addition to the program 302 for implementing the functions required for the call center system 300 of this embodiment, at least call recording data 325 is stored in the storage device 301. This call recording data 325 is data that becomes a record in the call recording DB 125 in the text conversion assistance device 100.

The hardware configuration of the administrator terminal 400 in this embodiment is as shown in Figure 5.

That is, the administrator terminal 400 includes a storage device 401, a memory 403, a computing device 404, an input device 405, an output device 406, and a communication device 407.

Of these, the storage device 401 is composed of an appropriate non-volatile storage element such as an SSD (Solid State Drive) or a hard disk drive.

In addition, memory 403 is composed of a volatile memory element such as RAM.

The calculation device 404 is a CPU that executes the program 402 stored in the storage device 401 by reading it into the memory 403, etc., to perform overall control of the device itself, as well as various judgments, calculations, and control processes.

The input device 405 is composed of devices such as a keyboard and a mouse that accept key input and voice input from the user/operator.

The output device 406 is composed of devices such as a display and a speaker that display the processing results of the calculation device 404.

The communication device 407 is assumed to be a network interface card or the like that is connected to the network 1 and handles communication processing with the text conversion assistance device 100 and the call center system 300 .
<Data structure example>
Next, various information used by the text conversion assistance device 100 of this embodiment will be described. Fig. 6 shows an example of the call recording DB 125 of this embodiment. The call recording DB 125 of this embodiment is a database that stores call recording data between an operator and a customer, obtained from the call center system 300 (or from the operator terminal 200), for example.

This call recording DB125 is a collection of records that link data such as the name of the customer, the name of the product or service specified by the customer, the ID of the operator who responded, and the recording data file, using, for example, the date and time of the call and a customer ID indicating the customer who was the subject of the call as keys.

Figure 7 shows an example of the configuration of the phoneme master table 126 in this embodiment. The phoneme master table 126 in this embodiment is a table that specifies the correct phonemes for each vocabulary.

This phoneme master table 126 is configured to, for example, use a conversation scene or subject as a key and specify the correct phoneme information for vocabulary that is expected to appear in a conversation related to that scene or subject.

Figure 8 also shows an example of the configuration of the speech similarity table 127 in this embodiment. The speech similarity table 127 in this embodiment is a table that specifies the similarity between each space when Japanese vowels are spoken.

This speech similarity table 127 is a matrix in which vowels are listed vertically and horizontally, and the similarity between each vowel is defined as a non-consecutive numerical value between a maximum value of 1 (perfect match) and a minimum value of 0 (zero similarity).
<Flow Example 1>
The actual procedure of the text conversion assistance method in this embodiment will be described below with reference to the drawings. The various operations corresponding to the text conversion assistance method described below are realized by a program that is read into a memory or the like and executed by the text conversion assistance device 100. This program is composed of codes for performing the various operations described below.

Figure 9 is a diagram showing a flow example 1 of the text conversion assistance method in this embodiment. In this case, the text conversion assistance device 100 acquires call recording data 325, for example, from the call center system 300 (or the operator terminal 200), and stores it in the call recording DB 125 (s1).

In addition, the text conversion assistance device 100 detects the arrival of a predetermined timing or receives an instruction from the administrator terminal 400, and extracts, for example, data relating to a predetermined period from the call recording data stored in the call recording DB 125, and applies this to the acoustic model 110 to extract phonemes (s2).

For example, suppose a phoneme sequence of "T-A-I-K-I-S-A-N" is extracted from a recorded call in which a call center operator, after a standard greeting, speaks to a customer named "Saeki," and then says "Saeki-san." Here, the customer's name is the subject of processing, but this is just one example, and it would also be appropriate to process, for example, the name of a financial product.

Next, the text conversion assistance device 100 identifies the customer of the call as "Saeki" from the customer ID linked to the above-mentioned call recording data, and extracts the phonemes of "Saeki" from the phoneme master table 126 as a vocabulary for keyword matching (s3). In this case, the phoneme sequence "S-A-E-K-I-S-A-N" is extracted from the record of customer ID "C018122: Saeki ***" in the phoneme master table 126.

Next, the text conversion assistance device 100 compares the phoneme sequences obtained in s2 and s3, respectively, and calculates the match rate (s4). In the above example, when the phoneme sequence "T-A-I-K-I-S-A-N" is compared with the phoneme sequence "S-A-E-K-I-S-A-N," six of the eight phonemes match, resulting in a match rate of 6/8 = 0.75.

If, as in the past, the phoneme sequence "T-A-I-K-I-S-A-N" obtained from the call recording data is applied to the language model 111 to obtain the text "Daiki-san," and this is compared with the text "Saeki-san," a vocabulary specified in the phoneme master table 126, the match rate will be calculated as 2/4 = 0.5, since two out of four characters match. If the pass/fail criterion for keyword matching is, for example, a match rate of 0.6, then even though the operator has certainly spoken the customer's name as "Saeki-san," it will be determined that they do not match due to the influence of the conversion accuracy in the language model 111.

On the other hand, according to the text conversion assistance device 100 of the present invention, it is possible to overcome the problem of conversion accuracy in the language model 111 and perform keyword matching based on the concordance rate between phoneme sequences, thereby enabling keyword matching with higher accuracy than in the past.
<Flow Example 2>
10 is a diagram showing a flow example 2 of the text conversion assistance method in this embodiment. Here, a method of calculating the degree of coincidence of phoneme sequences from the viewpoint of vowels will be described in order to further enhance the effect of the above-mentioned flow example 1. Note that in this flow, the steps up to s1 and s2 in the above-mentioned flow example 1 are the same, so the steps thereafter will be described.

The text conversion assistance device 100 extracts only the vowels (a, i, u, e, o) from the phoneme sequence extracted as in flow example 1 above (s10). In the above example, the vowel sequence "A, I, I, A" is extracted.

The text conversion assistance device 100 also extracts phonemes and vowels for the above-mentioned customer "Saeki" in the same manner as in s3 and s10 (s11). In this case, the vowel sequence "A, E, I, A" is extracted from the phoneme sequence "S-A-E-K-I-S-A-N."

Next, the text conversion assistance device 100 compares the vowels in the vowel sequences obtained in s10 and s11 against the speech similarity table 127, starting from the beginning of the sequence, and identifies the similarity between vowels at corresponding positions in the vowel sequences (s12).

For example, according to the speech similarity table 127, the vowel "A" and the vowel "A" have a similarity of "1", the vowel "A" and the vowel "I" have a similarity of "0", the vowel "A" and the vowel "U" have a similarity of "0", the vowel "A" and the vowel "E" have a similarity of "0.5", the vowel "A" and the vowel "O" have a similarity of "0.5", and so on.

As a result, in the above example, when "A, I, I, A" is matched with "A, E, I, A", the similarity between "A" and "A" is 1, between "I" and "E" is 0.5, between "I" and "I" is 1, and between "A" and "A" is 1.

Therefore, based on the similarity for each vowel obtained in s12, the text conversion assistance device 100 calculates the vowel similarity in the above-mentioned phoneme sequence to be (1 + 0.5 + 1 + 1) / 4 = 0.875 (s13).

The text conversion assistance device 100 also calculates the matching rate for consonants based on the phoneme sequence obtained in s2 and s3 (s14). In the above example, when comparing the consonants "T, K, S, N" in the phoneme sequence "T-A-I-K-I-S-A-N" with the consonants "S, K, S, N" in the phoneme sequence "S-A-E-K-I-S-A-N", three of the four phonemes match, resulting in a matching rate of 3/4 = 0.75.

Next, the text conversion assistance device 100 weights the vowel similarity obtained in s13, and calculates the match rate between the phoneme sequences by taking a weighted average with the consonant match rate (s15).

For example, if the weighting is set to "2", that is, the weight is set to twice the consonant matching rate and a weighted average is performed, the matching rate can be calculated as (consonant matching rate 0.75 + vowel similarity 0.875 x weighting 2)/3 = 0.83.
<Flow Example 3>
11 is a diagram showing a flow example 3 of the text conversion assistance method in this embodiment. Here, a method of calculating the degree of coincidence of phoneme sequences with a view to dealing with omitted characters and altered characters will be described in order to further enhance the effects of the above flow examples 1 and 2. Note that in this flow, steps s1 and s2 in the above flow example 1 and steps s10 and s11 in the flow example are the same, so the steps thereafter will be described.

The text conversion assistance device 100 determines the similarity between pairs of two consecutive vowels in the vowel sequence for each of the vowel sequences in the phoneme sequence in the call recording data extracted as described above and the vowel sequence in the phoneme sequence read from the corresponding record in the phoneme master table 126 based on the speech similarity table 127 (s20).

For example, in the vowel sequence "O, A, O, U, O, A, I, A, U, A, I, A" in the phoneme sequence "O-H-A-Y-O-U-G-O-Z-A-I-M-A-S-U-S-A-K-I-S-A-N" obtained from telephone call recording data, two vowels are selected from each vowel from the beginning to form a total of six pairs, such as pair (1) "O, A", pair (2) "O, U", pair (3) "O, A", pair (4) "I, A", pair (5) "U, A", and pair (6) "I, A", and based on the speech similarity table 127, it can be determined that pair (1) has a similarity of "0.5", pair (2) has a similarity of "0.5", pair (3) has a similarity of "0.5", pair (4) has a similarity of "0", pair (5) has a similarity of "0", and pair (6) has a similarity of "0".

Next, the text conversion assistance device 100 folds the pairs identified in s20 whose similarity is equal to or exceeds a criterion, such as 0.5, into a single predefined vowel (e.g., A, I, U), and for pairs whose similarity is below the criterion, it uses the first vowel and executes a process of forming the next pair with the last vowel in the phoneme sequence with an adjacent vowel, thereby generating a syllable sequence (s21).

In the above example, set (1) is aggregated to the vowel "A" (i.e., folded; same below), set (2) is aggregated to the vowel "U", set (3) is aggregated to the vowel "A", set (4) takes the first vowel "I" and combines the ending vowel "A" with the first vowel "U" of the original set (5) to form a new set (5)', and then reconstructs the sets for the subsequent vowel arrangements, and performs the aggregation based on similarity described above.

As a result, the syllable sequence remaining after combining each set is "A, U, A, I, A, U, A, I, A."

The text conversion assistance device 100 also performs this type of syllable sequence generation for the vowel sequence "A, A, I, A" in the phoneme sequence "S-A-S-A-K-I-S-A-N" of the corresponding record in the phoneme master table 126, thereby obtaining "A, I, A."

Next, in s21, the text conversion assistance device 100 compares the number of vowels in the syllable sequence derived from the phoneme master table 126 for the portion of the syllable sequence derived from the call recording data that matches the syllable sequence generated from the phoneme master table 126, and if the number of vowels is the same (s22: same number), it calculates the matching rate of the vowels in the corresponding vowel sequence between the above-mentioned portion and the syllable sequence derived from the phoneme master table 126 based on the speech similarity table 127 (s23).

For example, of the vowel sequence "O, A, O, U, O, A, I, A, U, A, I, A" in the phoneme sequence "O-H-A-Y-O-U-G-O-Z-A-I-M-A-S-U-S-A-K-I-S-A-N" in the recorded call data, the part where the phoneme sequence matches the syllable sequence "A, I, A" derived from the phoneme master table 126 (which is based on the vowel sequence "A, A, I, A") is "O, A, I, A".

Therefore, the text conversion assistance device 100 determines the similarity between each vowel between the vowel sequence "O, A, I, A" derived from the call recording data and the vowel sequence "A, A, I, A" derived from the phoneme master table 126 based on the speech similarity table 127, and calculates it to be, for example, (0.5 + 1 + 1 + 1) / 4 = 0.875.

On the other hand, if the comparison of the number of vowels in s22 above shows that the number of vowels derived from the master table 126 is greater than the number of vowels derived from the call recording data (s22: more), the text conversion assistance device 100 assumes that a character has been omitted, determines that the syllable sequence derived from the master table 126 is correct, and corrects the missing vowels in the syllable sequence derived from the call recording data by filling in the corresponding phonemes from the master table 126 (s24), and calculates the vowel matching rate between the corrected vowel sequence and the vowel sequence derived from the master table 126 based on the speech similarity table 127 (s25).

For example, the vowel sequence "A, I, A" in the phoneme sequence "S-A-K-I-S-A" of the call recording data matches the syllable sequence "A, I, A" (based on the vowel sequence "A, A, I, A") derived from the phoneme master table 126. However, the number of vowels in the corresponding vowel sequence is one more in the vowel sequence derived from the master table 126.

Therefore, the text conversion assistance device 100 converts the vowel sequence "A, I, A" from the call recording data into
Among them, the second "A" from the beginning is insufficient, i.e., is missing, compared to the vowel sequence "A, A, I, A" derived from the above-mentioned master table 126. Therefore, the text conversion assistance device 100 corrects the vowel sequence "A, I, A" derived from the call recording data by adding an "A" between the first "A" and the second "I".

The text conversion assistance device 100 also calculates the similarity between the vowel sequence that has been corrected as described above and the vowel sequence derived from the master table 126 based on the speech similarity table 127 as (1+1+1+1)/4=1, etc.

On the other hand, if the comparison of the number of vowels in s22 above shows that the number of vowels derived from the master table 126 is less than the number of vowels derived from the call recording data (s22: fewer), the text conversion support device 100 assumes that the syllable sequence derived from the master table 126 is correct, and corrects the syllable sequence derived from the call recording data by deleting the excess of vowels in the syllable sequence (s26), and calculates the vowel matching rate between the corrected vowel sequence and the vowel sequence derived from the master table 126 based on the speech similarity table 127 (s27).

For example, the vowel sequence "A, A, A, I, A" in the phoneme sequence "A-K-A-S-A-K-I-S-A" of the call recording data matches the syllable sequence "A, I, A" (based on the vowel sequence "A, A, I, A") derived from the phoneme master table 126. However, the number of vowels in the corresponding vowel sequence is one less in the vowel sequence derived from the master table 126.

Then, in the vowel sequence "A, A, A, I, A" derived from the call recording data, the first "A" is excessive compared to the vowel sequence "A, A, I, A" derived from the master table 126 described above. Therefore, the text conversion assistance device 100 corrects the vowel sequence "A, A, A, I, A" derived from the call recording data by deleting the first "A".

The text conversion assistance device 100 also calculates the similarity between the vowel sequence that has been corrected as described above and the vowel sequence derived from the master table 126 based on the speech similarity table 127 as (1+1+1+1)/4=1, etc.

Note that, as this has already been explained in Flow Example 2, the concept of calculating the match rate by taking into account the degree of agreement of consonant sequences in addition to the degree of similarity of vowel sequences will not be explained here.

The above describes in detail the best mode for carrying out the present invention, but the present invention is not limited to this, and various modifications are possible without departing from the spirit of the present invention.

According to this embodiment, it is possible to perform keyword matching of the contents of a call with high accuracy, regardless of the characteristics of the speech-to-text conversion of the contents of the call.

The description in this specification makes at least the following clear. That is, in the text conversion assistance device of this embodiment, the storage device may further hold information that specifies the speech similarity between vowels, and the arithmetic device, when calculating the matching rate, may perform a process of calculating the matching rate between the vowels included in each of the correct phoneme and the extracted phoneme based on the speech similarity information, a process of calculating the matching rate between the consonants included in each of the correct phoneme and the extracted phoneme, and a process of weighting the matching rate between the vowels more heavily than the matching rate between the consonants, and calculating the matching rate between the phonemes based on the matching rates between the vowels and between the consonants.

This allows priority to be given to vowels (which are few in number and easy to distinguish, i.e., less prone to false positives) as elements to be matched when matching between the above-mentioned phonemes, which makes it easier to improve the accuracy of the match rate. Ultimately, keyword matching of the contents of the phone call can be performed with higher accuracy, regardless of the characteristics of the speech-to-text conversion of the contents of the phone call.

In addition, in the text conversion assistance device of this embodiment, when calculating the matching rate, the calculation device may specify, for each of the extracted phoneme and the correct phoneme, the similarity between a pair of two consecutive vowels in the sequence of vowels contained in the phoneme using the speech similarity, convolve pairs with a predetermined specified vowel for pairs with the similarity equal to or higher than a standard, and use the first vowel for pairs with the similarity below the standard to form a next pair with the last vowel in the sequence with an adjacent vowel in the sequence, to generate a syllable sequence, compare the number of vowels between the syllable sequences generated for each of the extracted phoneme and the correct phoneme, and if the number of vowels is equal, calculate the vowel matching rate between the vowel sequences of the extracted phoneme and the correct phoneme that were the basis of the syllable sequence based on the speech similarity.

This makes it possible to deal appropriately with problems such as omissions and altered characters that tend to occur when two characters are pronounced as one syllable, when only one character is pronounced, when the first character is not pronounced, or when the same character is unnecessarily repeated, when vowels with high vowel similarity occur in Japanese. Ultimately, it becomes possible to perform keyword matching of the contents of a phone call with greater accuracy, regardless of the characteristics of the speech-to-text conversion of the contents of the phone call.

In addition, in the text conversion assistance device of this embodiment, the calculation device may compare the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is greater than that of the extracted phoneme, determine that the syllable sequence of the correct phoneme is correct, correct the missing vowels in the syllable sequence of the extracted phoneme by filling in the corresponding phoneme of the correct phoneme, and calculate the vowel matching rate between the vowel sequences of the extracted phoneme and the correct phoneme after the correction based on the speech similarity.

This allows the above-mentioned typo issue to be dealt with appropriately, and makes it possible to perform keyword matching of the contents of the call with greater accuracy, regardless of the characteristics of the speech-to-text conversion of the call content.

In addition, in the text conversion assistance device of this embodiment, the calculation device may compare the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is smaller than that of the extracted phoneme, determine that the syllable sequence of the correct phoneme is correct, delete the portion in which there are excess vowels in the syllable sequence of the extracted phoneme to correct it, and calculate the vowel matching rate between the vowel sequences of the corrected extracted phoneme and the correct phoneme based on the speech similarity.

This makes it possible to deal appropriately with the above-mentioned problem of altered characters and perform keyword matching of the contents of the phone call with greater accuracy, regardless of the characteristics of the speech-to-text conversion of the contents of the phone call.

In addition, in the text conversion assistance method of this embodiment, the information processing device may further store information in the storage device that specifies the speech similarity between vowels, and when calculating the matching rate, the information processing device may perform a process of calculating the matching rate between the vowels contained in each of the correct phoneme and the extracted phoneme based on the speech similarity information, a process of calculating the matching rate between the consonants contained in each of the correct phoneme and the extracted phoneme, and a process of weighting the matching rate between the vowels more heavily than the matching rate between the consonants to calculate the matching rate between the phonemes based on the matching rates between the vowels and between the consonants.

In addition, in the text conversion assistance method of this embodiment, when calculating the matching rate, the information processing device may specify, for each of the extracted phoneme and the correct phoneme, the similarity between a pair of two consecutive vowels in the sequence of vowels contained in the phoneme using the speech similarity, and for pairs whose similarity is equal to or greater than a standard, convolve them into one predetermined specified vowel, and for pairs whose similarity is below the standard, use the first vowel and execute a process of forming the next pair of the last vowel with the adjacent vowel in the sequence to generate a syllable sequence, compare the number of vowels between the syllable sequences generated for each of the extracted phoneme and the correct phoneme, and if the number of vowels is equal, calculate the vowel matching rate between the vowel sequences of the extracted phoneme and the correct phoneme that were the basis of the syllable sequence based on the speech similarity.

In addition, in the text conversion assistance method of this embodiment, the information processing device may compare the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is greater than that of the extracted phoneme, determine that the syllable sequence of the correct phoneme is correct, correct the missing vowel in the syllable sequence of the extracted phoneme by filling in the corresponding phoneme of the correct phoneme, and calculate the vowel matching rate between the vowel sequences of the extracted phoneme and the correct phoneme after the correction based on the speech similarity.

In addition, in the text conversion assistance method of this embodiment, the information processing device may compare the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is smaller than that of the extracted phoneme, determine that the syllable sequence of the correct phoneme is correct, delete the portion in which there are excess vowels in the syllable sequence of the extracted phoneme to correct it, and calculate the vowel matching rate between the vowel sequences of the corrected extracted phoneme and the correct phoneme based on the speech similarity.

1 Network 100 Text conversion support device 101 Storage device 102 Program 103 Memory 104 Arithmetic device 105 Communication device 110 Acoustic model 111 Language model 125 Call recording DB
126 Phoneme master table 127 Speech similarity table 200 Operator terminal 300 Call center system 400 Manager terminal

Claims (8)

A storage device that holds master data defining correct phoneme information for each vocabulary expected to appear in each conversation scene or subject , and information defining speech similarity between vowels ;
a calculation device for executing a process of applying the call recording data obtained from a specified device to an acoustic model to extract phonemes; a process of calculating a matching rate between the correct phonemes of vocabulary, of which phonemes are defined in the master data, that are expected to appear in the conversation scene or subject of the call recording data and the extracted phonemes, the process of calculating a matching rate between the vowels included in the correct phoneme and the extracted phoneme based on information on the speech similarity, calculating a matching rate between the consonants included in the correct phoneme and the extracted phoneme, and calculating a matching rate between the phonemes based on the matching rates between the vowels and the consonants by weighting the matching rate between the vowels more heavily than the matching rate between the consonants; and a process of specifying the vocabulary whose phonemes show a predetermined matching rate as a keyword matching result.
A text conversion assistance device comprising: The computing device includes:
When calculating the match rate,
For each of the extracted phonemes and the correct phonemes, a similarity between a pair of two consecutive vowels in a sequence of vowels included in the phoneme is specified by the speech similarity, and pairs having the similarity equal to or greater than a standard are folded into one predetermined defined vowel, and pairs having the similarity below the standard are folded into a next pair by adopting the first vowel and forming a next pair of the last vowel with an adjacent vowel in the sequence, thereby generating a syllable sequence;
a number of vowels is compared between the syllable sequences generated for the extracted phoneme and the correct phoneme, and if the number of vowels is equal, a vowel matching rate is calculated between the vowel sequences of the extracted phoneme and the correct phoneme, which are the basis of the syllable sequence, based on the speech similarity.
2. The text conversion assistance device according to claim 1 . The computing device includes:
comparing the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is greater than that of the extracted phoneme, determining that the syllable sequence of the correct phoneme is correct, correcting a portion where a vowel is missing in the syllable sequence of the extracted phoneme by filling in the corresponding phoneme of the correct phoneme;
A vowel matching rate between the vowel sequences of the extracted phonemes after the correction and the correct phonemes is calculated based on the speech similarity.
3. The text conversion assistance device according to claim 2 . The computing device includes:
comparing the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is smaller than that of the extracted phoneme, determining that the syllable sequence of the correct phoneme is correct, correcting the syllable sequence of the extracted phoneme by deleting a portion in which there are excess vowels;
A vowel matching rate between the vowel sequences of the extracted phonemes after the correction and the correct phonemes is calculated based on the speech similarity.
3. The text conversion assistance device according to claim 2 . An information processing device,
A storage device stores master data that specifies correct phoneme information for each vocabulary expected to appear in each conversation scene or subject , and information that specifies speech similarity between vowels ;
a process of applying the call recording data obtained from a specified device to an acoustic model to extract phonemes; a process of calculating a matching rate between the correct phonemes of vocabulary whose phonemes are defined in the master data and the extracted phonemes , the process of calculating a matching rate between the vowels contained in the correct phonemes and the extracted phonemes based on information on the speech similarity, calculating a matching rate between the consonants contained in the correct phonemes and the extracted phonemes, and calculating a matching rate between the phonemes based on the matching rates between the vowels and the consonants by weighting the matching rate between the vowels more heavily than the matching rate between the consonants; and a process of identifying the vocabulary whose phonemes show a predetermined matching rate as a result of the calculation as a keyword matching result.
A text conversion assistance method that performs the above. The information processing device,
When calculating the match rate,
For each of the extracted phonemes and the correct phonemes, a similarity between a pair of two consecutive vowels in a sequence of vowels included in the phoneme is specified by the speech similarity, and pairs having the similarity equal to or greater than a standard are folded into one predetermined defined vowel, and pairs having the similarity below the standard are folded into a next pair by adopting the first vowel and forming a next pair of the last vowel with an adjacent vowel in the sequence, thereby generating a syllable sequence;
comparing the number of vowels between the syllable sequences generated for the extracted phoneme and the correct phoneme, respectively, and, if the numbers of vowels are equal, calculating a vowel matching rate between the vowel sequences of the extracted phoneme and the correct phoneme, which are the basis of the syllable sequence, based on the speech similarity;
The text generation assistance method according to claim 5 . The information processing device,
comparing the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is greater than that of the extracted phoneme, determining that the syllable sequence of the correct phoneme is correct, correcting a portion where a vowel is missing in the syllable sequence of the extracted phoneme by filling in the corresponding phoneme of the correct phoneme;
calculating a vowel matching rate between the vowel sequences of the extracted phonemes after the correction and the correct phonemes based on the speech similarity;
The text generation assistance method according to claim 6 . The information processing device,
comparing the number of vowels between the syllable sequences, and if the number of vowels in the syllable sequence corresponding to the correct phoneme is smaller than that of the extracted phoneme, determining that the syllable sequence of the correct phoneme is correct, correcting the syllable sequence of the extracted phoneme by deleting a portion in which there are excess vowels;
calculating a vowel matching rate between the vowel sequences of the extracted phonemes after the correction and the correct phonemes based on the speech similarity;
The text generation assistance method according to claim 6 .

Images