JP2008233305A - Voice dialogue apparatus, voice dialogue method and program - Google Patents

Abstract

Provided is a voice dialogue apparatus that does not continue a wrong dialogue and does not perform an unnecessary response while improving the subjective evaluation of a user.
SOLUTION: A plurality of combination conditions of a first threshold that is a threshold for acceptance and confirmation and a second threshold that is a threshold for confirmation and rejection are set, and speech that converts input speech data into character data The recognizing means 23, the reliability calculating means 25 for calculating the reliability of the word included in the character data, and comparing the reliability with the first threshold value and the second threshold value, and accepting, confirming or rejecting The voice interactive apparatus 1 having response control means 27 for controlling whether to perform a response is subjected to an evaluation experiment, and the first threshold value and the second threshold value are determined based on the subject's subjective evaluation results, particularly the accuracy and efficiency evaluation results. The most preferable condition among the combination conditions with the threshold value is determined.
[Selection] Figure 2

Description

  The present invention relates to a voice recognition technique, and more particularly to a voice dialogue apparatus, a voice dialogue method, and a program that perform response control based on a voice recognition result.

In recent years, voice recognition technology has been widely used and used in car navigation, mobile phones, games, call centers, and the like. Various dictation programs exist as software on the PC.
As a mechanism for performing response control based on a speech recognition result, a mechanism for performing response control using the reliability of a word that is a speech recognition result is disclosed (see Patent Document 1 and Non-Patent Document 1).

  The method shown in Patent Document 1 realizes a voice interaction apparatus that performs response control so as to accept / confirm a voice recognition result using the reliability of a word that is a voice recognition result. Then, a threshold value for controlling acceptance / confirmation is provided for each semantic category, and the threshold value is corrected based on the conversation history data during the operation of the voice interaction apparatus. When an erroneously recognized word is accepted, the threshold is increased, and when the correct recognized word is confirmed, the threshold is corrected to be lower.

Further, the method shown in Non-Patent Document 1 realizes a voice interaction apparatus that performs response control so as to accept / confirm / reject a voice recognition result using the reliability of a word that is a voice recognition result. Then, a threshold value 1 for controlling acceptance / confirmation and a threshold value 2 for controlling confirmation / rejection are determined using various error rates that are objective indices. Here, the error rate used to determine the threshold 1 is FA1 = 1−the number of accepted correct words ÷ the total number of accepted words, SErr = 1−the number of accepted correct words ÷ the total number of correct words (= the total number of evaluation words). There are two types. Further, there are two types of error rates used for determining the threshold 2: FA2 = 1−the number of confirmed correct words ÷ the total number of confirmed words, FR = the number of rejected correct words ÷ the total number of rejected words.
JP 2005-181386 A Tatsuya Kawahara and Kazunori Komatani, “Utilization of Reliability of Speech Recognition Results in Spoken Dialogue Systems”, Proceedings of the Acoustical Society of Japan 2000 Autumn Meeting, p73, 2000.

  However, in the above-mentioned Patent Document 1, there is no response control regarding confirmation / rejection, and if a confirmation response is made with a misrecognition result, the user always answers “no”, and as a result, the user must speak again. Speaking efficiency is not good. If the reliability is sufficiently low, the possibility of misrecognition is very high, so response control should be performed so as to reject it.

  In Non-Patent Document 1 described above, the determination of the threshold value is an optimum value for objective evaluation, and subjective evaluation is not necessarily improved.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide a voice interactive apparatus that does not continue an erroneous conversation and does not perform an unnecessary response while improving the subjective evaluation of the user. It is.

In order to achieve the above-described object, the first invention provides speech recognition means for converting inputted speech data into character data, reliability calculation means for calculating the reliability of words included in the character data, Response that controls whether to accept or confirm or reject by comparing the reliability with the first threshold that is the threshold for acceptance and confirmation and the second threshold that is the threshold for confirmation and rejection And a control means, wherein the first threshold value and the second threshold value are determined based on subjective evaluation.
Moreover, it is desirable that the subjective evaluation is an evaluation on accuracy and efficiency.

According to a second aspect of the present invention, the input voice data is converted into character data, the step of calculating the reliability of a word included in the character data, and the reliability is a threshold value for acceptance and confirmation. A threshold value of 1 and a second threshold value that is a threshold value of confirmation and rejection, and controlling whether to accept, confirm, or reject the response. The threshold of 2 is a voice interaction method characterized by being determined based on subjective evaluation.
Moreover, it is desirable that the subjective evaluation is an evaluation on accuracy and efficiency.

  A third invention is a program for causing a computer to function as the voice interactive apparatus according to claim 1 or claim 2.

  According to the present invention, it is possible to provide a voice interactive apparatus that does not continue an erroneous conversation and does not perform an unnecessary response while improving the subjective evaluation of the user.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a hardware configuration diagram of a computer that realizes the voice interaction apparatus 1 according to the present embodiment.
As shown in FIG. 1, the voice interaction apparatus 1 includes a control unit 3, a storage unit 5, a media input / output unit 7, a communication control unit 9, an input unit 11, a display unit 13, a peripheral device I / F unit 15, and the like. It is connected to the bus 17.
In the following embodiment, an example of the voice interactive device 1 using a computer as hardware is shown. However, the present invention is not limited to a computer, and various electronic devices such as a car navigation device, a mobile phone terminal, a game device, etc. It can also be applied to equipment.

  The control unit 3 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

The CPU calls and executes a program stored in the storage unit 5, ROM, recording medium, etc. to a work memory area on the RAM, executes driving control of each device connected via the bus 17, and the voice interactive device 1 An information retrieval process (see FIG. 4) described later is performed.
The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like.
The RAM is a volatile memory, and temporarily stores programs, data, and the like loaded from the storage unit 5, ROM, recording medium, and the like, and includes a work area used by the control unit 3 for performing various processes.

The storage unit 5 is an HDD (hard disk drive), and stores a program executed by the control unit 3, data necessary for program execution, an OS (operating system), and the like. As for the program, a control program corresponding to an OS (operating system) and an application program corresponding to information search processing described later are stored.
Each of these program codes is read by the control unit 3 as necessary, transferred to the RAM, read by the CPU, and executed as various means.

  The media input / output unit 7 (drive device) inputs / outputs data, for example, floppy (registered trademark) disk drive, PD drive, CD drive (-ROM, -R, RW, etc.), DVD drive (-ROM, -R, -RW, etc.) and a media input / output device such as an MO drive.

  The communication control unit 9 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between the computer and the network 19, and performs communication control between other computers via the network 19.

  The input unit 11 inputs data and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. In addition, for example, a voice input device such as a microphone is included. An operation instruction, an operation instruction, data input, and the like can be performed on the computer via the input unit 11.

  The display unit 13 includes a display device such as a CRT monitor and a liquid crystal panel, and a logic circuit (such as a video adapter) for realizing a video function of the computer in cooperation with the display device.

  The peripheral device I / F (interface) unit 15 is a port for connecting a peripheral device to the computer, and the computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 15. The peripheral device I / F unit 15 is configured by USB, IEEE 1394, RS-232C, or the like, and usually has a plurality of peripheral devices I / F. The connection form with the peripheral device may be wired or wireless.

  The bus 17 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

Next, the configuration of the voice interaction apparatus 1 will be described with reference to FIG.
FIG. 2 is a block diagram showing functions of the voice interaction apparatus 1.

  The voice interactive apparatus 1 includes a voice input unit 21, a voice recognition unit 23, a reliability calculation unit 25, a response control unit 27, and the like.

  The voice input means 21 inputs voice spoken by the user as data. The voice data may be input via the input unit 11 of the voice interaction apparatus 1 or may be input from another computer or the like via the network 19.

  The voice recognition means 23 converts the input voice data into character data. First, a voice feature amount is extracted from the waveform of input voice data. Next, using the extracted speech features as input, use an acoustic model that defines acoustic features such as speaker characteristics and speech input environment, and a language model that defines linguistic features such as wording such as wording and words to be recognized. Perform the likelihood calculation. And the thing with high likelihood is selected from the recognition candidates, and it converts into character data. For example, there are many n-best methods that calculate all possibilities for an input sentence that is a speech analysis result, obtain a plurality of sentence candidates up to n, and output sentence candidates in descending order of likelihood as recognition results. It is used.

  The reliability calculation means 25 calculates the reliability of words included in the character data. Here, the confidence level (Confidence Measurement: CM) is a measure representing how much a word included in the character data as a speech recognition result can be trusted. A high reliability number indicates that no other candidate was found to compete with the words in the text data, and a low confidence number indicates that many other candidates were competing. .

ここで、単語ｗが第ｉ候補に含まれるときはδ（ｗ,ｉ）＝１、含まれないときはδ（ｗ,ｉ）＝０である。また、αはスムージング係数である。 As an example of the reliability calculation formula, a calculation formula using the results of n sentence candidates that are speech recognition results by the n-best method will be described. Intuitively, a word that appears consistently in any sentence candidate is considered to be reliable, the log likelihood of the i-th candidate is g (i), and the reliability CM (w of the word w ) Is obtained by the following calculation formula.

Here, when the word w is included in the i-th candidate, δ (w, i) = 1, and when it is not included, δ (w, i) = 0. Α is a smoothing coefficient.

  The response control means 27 compares the reliability with a first threshold value that is a threshold value for acceptance and confirmation and a second threshold value that is a threshold value for confirmation and rejection. Control what to do.

FIG. 3 is a diagram illustrating response control using reliability.
As shown in FIG. 3, when the reliability value is larger than the first threshold value 29, the control unit 3 makes an acceptance response. The acceptance response is to accept the speech recognition result as it is and continue the dialogue. When the voice recognition result is correct, a confirmation response described later can be omitted. Next, when the reliability value is equal to or less than the first threshold value 29 and greater than the second threshold value 31, the control unit 3 makes a confirmation response. The confirmation response is to confirm with the user whether the voice recognition result is correct. When the speech recognition result is incorrect, it is possible to avoid continuing the conversation with the incorrect recognition result. And when the value of reliability is below the 2nd threshold value 31, the control part 3 performs the rejection response. The rejection response is to ask the user the same question again without using the voice recognition result. If the result of speech recognition is incorrect, the confirmation response can be omitted.

Next, details of the operation of the voice interactive apparatus 1 will be described with reference to FIG.
FIG. 4 is a flowchart showing the procedure of the voice interaction process.

  As shown in FIG. 4, when voice data is input via the input unit 11 (step 101), the control unit 3 converts the input voice data into character data (step 102).

  Next, the control part 3 calculates the reliability of the word contained in character data (step 103). The calculation of the reliability is performed using, for example, the calculation formula described above with reference to FIG.

Next, the control unit 3 confirms whether or not the calculated reliability is greater than the first threshold (step 104).
When the reliability is larger than the first threshold, the control unit 3 makes an acceptance response (step 105).
When the reliability is equal to or lower than the first threshold value, the process proceeds to step 106.

Next, the control unit 3 confirms whether or not the calculated reliability is greater than the second threshold (step 105).
When the reliability is larger than the second threshold, the control unit 3 makes a confirmation response (step 106).
When the reliability is equal to or lower than the second threshold, the control unit 3 makes a rejection response (step 107).
The control unit 3 realizes a voice dialogue with the user by repeatedly performing a plurality of dialogue processings with the above processing as one dialogue processing.

  Next, how to determine the first threshold value and the second threshold value will be described with reference to FIGS.

FIG. 5 is a diagram illustrating a combination condition of the first threshold value and the second threshold value.
As shown in FIG. 5, for example, three types of combination conditions of a first threshold value and a second threshold value are set, and an evaluation experiment of the voice interaction apparatus 1 having each function shown in FIG. A first threshold value and a second threshold value are determined based on the evaluation result.
The combination conditions of the first threshold value and the second threshold value are not limited to three types, and three or more types of combination conditions may be set.

FIG. 6 is a diagram showing a response operation for each condition shown in FIG.
As shown in FIG. 6, the condition A sets a large range in which a confirmation response is made. Condition B sets a large range for the rejection response. Condition C has a large range for accepting responses. Thus, it can be seen that the response operation is greatly different between the conditions A to C.
Hereinafter, evaluation experiments actually performed will be described.

First, experimental specifications of this evaluation experiment will be described.
The speech recognition engine that realizes the speech recognition means 23 shown in FIG. 2 is Julius 3.5, which is open source software that is open to the public.
Next, the acoustic model is a PTM (Photonic Tied-Mixture) triphone model of an unspecified speaker that is publicly disclosed together with Julius 3.5.
The language model is a model of 3500 words learned from a recognized sentence list of 3 million sentences.
And the voice interactive apparatus 1 which performs the facility search service by the voice dialogue incorporating these modules was constructed, and this evaluation experiment was conducted. The number of subjects is 6 men and women in their 20s and 40s who are proficient in voice dialogue processing.

Next, the result of the subjective evaluation of this evaluation experiment will be described.
FIG. 7 is a diagram showing the subjective evaluation of the subject.
As shown in FIG. 7, in the subjective evaluation, answers were obtained from subjects on two viewpoints of accuracy and efficiency. The evaluation score shown in FIG. 7 is an average value of responses from subjects obtained by a five-step evaluation. It can be seen that Condition A obtained the highest evaluation in terms of accuracy. On the other hand, in terms of efficiency, the condition C obtained the highest evaluation, but it can be seen that the condition A also obtained substantially the same evaluation.
Here, determination of conditions will be described. For example, the condition C having the lowest evaluation in the accuracy evaluation result is removed. This is to ensure accuracy. Next, for example, the low-evaluation condition B is removed from the conditions A and B in the efficiency evaluation result. This is to improve efficiency. Then, among these three types of conditions, it can be determined that the condition A is the most preferable combination of the first threshold value and the second threshold value.

Next, how the condition A determined to be the most preferable combination of the first threshold value and the second threshold value is evaluated in the overall evaluation of the subject will be described.
FIG. 8 is a diagram illustrating a result of comparative evaluation with the interpersonal dialogue for each condition illustrated in FIG. 5.
The score shown in FIG. 8 is a comparative score when the evaluation when using the facility search service by voice dialogue with a manned operator is assumed to be 100 points.
As shown in FIG. 8, it can be seen that the condition A has a score that is about 5 points higher than the other conditions.

Next, the result of objective evaluation of this evaluation experiment will be described.
FIG. 9 is a diagram showing an acceptance error rate and an average required time for each condition shown in FIG.
The horizontal axis of FIG. 9 represents an acceptance error rate (= 1−the number of accepted correct words ÷ the total number of accepted words), which is the ratio of erroneously recognizing misrecognized words. Moreover, the vertical axis | shaft of FIG. 9 is an average required time. The average required time is an average of the time from the start of the voice dialogue until the input of all the search keywords, and does not include the time from when the system executes the search until the result is displayed. The acceptance error rate is an index of accuracy in objective evaluation, and the average required time can be said to be an index of efficiency in objective evaluation. If there is no acceptance error rate, the wrong dialogue is not continued. If the average required time is not extremely long, an unnecessary response is not performed.
As shown in FIG. 9, it can be seen that the condition C has an acceptance error, and the accuracy in objective evaluation cannot be secured. This confirms that it is appropriate to remove the condition C in the accuracy evaluation result by subjective evaluation.
On the other hand, the condition A and the condition B have no acceptance error, and the accuracy in objective evaluation can be secured. Further, it can be understood that the difference in average required time between the condition A and the condition B is smaller than that in the condition C, and an unnecessary response is not performed.

  As described above in detail, according to the present embodiment, an evaluation experiment of the voice interactive device 1 having each function shown in FIG. 2 is performed, and a subjective evaluation result of the subject, particularly an evaluation of accuracy and efficiency is performed. A first threshold value and a second threshold value are determined based on the result.

  In the description of FIG. 7, the first threshold value and the second threshold value are determined based only on the result of subjective evaluation. However, the first threshold value is determined based on both the result of subjective evaluation and the result of objective evaluation. And the second threshold may be determined. For example, when it is necessary to ensure 100% accuracy in objective evaluation, the combination of the first threshold value and the second threshold value shown in FIG. The combination of the first threshold value and the second threshold value with the highest evaluation in the efficiency evaluation result based on the subjective evaluation shown in FIG.

  According to the present embodiment, it is possible to provide the voice interactive apparatus 1 that improves the subjective evaluation of the user. Further, it is possible to prevent an unnecessary response from being performed while preventing a wrong dialogue from continuing. Then, the user feels that it is good to use various services provided by the voice interaction device 1, and can continue to use it with peace of mind.

  The preferred embodiments of the voice interactive apparatus and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

Hardware configuration diagram of a computer for realizing the voice interactive apparatus 1 Block diagram showing functions of the voice interactive apparatus 1 Diagram showing response control using reliability Flow chart showing the procedure of voice dialogue processing The figure which shows the combination conditions of a 1st threshold value and a 2nd threshold value The figure which shows the response operation | movement for every condition shown in FIG. Diagram showing subject's subjective evaluation The figure which shows the result of the comparative evaluation with the interpersonal dialogue for every condition shown in FIG. Figure showing acceptance error rate and average time required for each condition shown in Figure 5

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ......... Voice interaction apparatus 3 ......... Control part 5 ......... Storage part 7 ......... Media input / output part 9 ......... Communication control part 11 ......... Input part 13 ......... Display part 15 ......... Peripheral device I / F unit 17... Bus 19... Network 21... Voice input means 23 ... Voice recognition means 25 .... Reliability calculation means 27 ... Response control means

Claims (5)

Voice recognition means for converting input voice data into character data;
Reliability calculation means for calculating the reliability of words included in the character data;
The reliability is compared with a first threshold, which is a threshold for acceptance and confirmation, and a second threshold, which is a threshold for confirmation and rejection, to control whether to accept, confirm, or reject Response control means;
Comprising
The spoken dialogue apparatus according to claim 1, wherein the first threshold value and the second threshold value are determined based on subjective evaluation.   The spoken dialogue apparatus according to claim 1, wherein the subjective evaluation is an evaluation of accuracy and efficiency. Converting the input voice data into character data;
Calculating a reliability of a word included in the character data;
The reliability is compared with a first threshold, which is a threshold for acceptance and confirmation, and a second threshold, which is a threshold for confirmation and rejection, to control whether to accept, confirm, or reject Steps,
Including
The voice dialogue method according to claim 1, wherein the first threshold value and the second threshold value are determined based on subjective evaluation.   4. The spoken dialogue method according to claim 3, wherein the subjective evaluation is an evaluation of accuracy and efficiency.   A program for causing a computer to function as the voice interactive apparatus according to claim 1.

Images