JP6577930B2 - Feature extraction device, acoustic model learning device, acoustic model selection device, feature extraction method, and program - Google Patents

Description

  The present invention relates to a technique for extracting a feature amount from an impulse response representing an acoustic characteristic of a user environment and evaluating the similarity of the impulse response.

  In a system that handles audio as an input interface, its performance is greatly affected by the user usage environment. As a factor of performance deterioration, a factor due to speech and a factor due to sound environment can be considered. In order to improve the factors caused by the sound environment, it is necessary to reproduce the user environment on the system. As such conventional technologies, there are a method of reproducing using data recorded in an actual environment and a method of simulating a user environment on a computer simulation. Non-Patent Document 1 describes a method of learning an acoustic model from learning speech generated by convolving an artificial reverberation impulse response with clean speech and measuring speech recognition performance.

Kenta Nishigame, Junji Watanabe, Takuya Nishimoto, Junki Ono, Shigeki Hiyama, “Study on robust speech recognition in unknown reverberation environment using speech under multiple reverberation characteristics”, IEICE technical research Report, vol. 108 (66), pp. 43-48, May 2008

  When recording audio data in a real environment, there is a problem that it costs a lot. When simulating a user environment on a computer simulation, it is possible to reduce costs, but it is necessary to determine guidelines for simulating. If a feature amount is extracted from information (impulse response) acquired in the user environment and the similarity of the acoustic characteristics can be calculated based on the feature amount, appropriate data for simulating the user environment is selected. It is thought that you can. However, it has not been clear what feature quantity can appropriately evaluate the similarity of acoustic characteristics.

  In view of the above points, an object of the present invention is to provide a feature amount extraction technique that can select data suitable for a user environment based on a feature amount of an impulse response.

  In order to solve the above-described problem, the feature quantity extraction device according to the first aspect of the present invention includes a feature quantity calculation unit that calculates a feature quantity that represents a ratio of the power of the rear reverberation in the impulse response to the total power, An impulse response measurement unit that measures an impulse response of a user environment, a data storage unit that stores data related to a plurality of different acoustic environments and feature quantities calculated from the impulse responses measured in the acoustic environment, and a user And a data selection unit that selects data related to the acoustic environment corresponding to the user environment based on a distance between the feature amount calculated from the impulse response of the environment and the feature amount calculated from the impulse response of the acoustic environment.

  The acoustic model learning device according to the second aspect of the present invention includes a clean speech storage unit that stores clean speech, an impulse response storage unit that stores impulse responses measured in a plurality of different acoustic environments, and a posterior reverberation in the impulse response. A feature amount calculation unit that calculates a feature amount that represents a ratio of the total power to the power, an impulse response classification unit that classifies the impulse responses into a plurality of groups based on the distance between the feature amounts of the impulse response in the acoustic environment, and A learning data generation unit that generates a learning voice corresponding to each group by convolving each impulse response classified into each group into a clean voice, and a learning voice corresponding to each group is used to correspond to each group. An acoustic model learning unit that learns an acoustic model.

  According to a third aspect of the present invention, there is provided an acoustic model selection device including a feature amount calculation unit that calculates a feature amount that represents a ratio of the power of rear reverberation in an impulse response to the total power, and an impulse that measures an impulse response in a user environment. An acoustic model storage unit that associates and stores a response measurement unit, an acoustic model learned using learning speech corresponding to a plurality of different acoustic environments, and a feature amount calculated from an impulse response measured in the acoustic environment; An acoustic model selection unit that selects an acoustic model corresponding to the user environment based on a distance between the characteristic amount of the impulse response of the user environment and the characteristic amount of the impulse response corresponding to each acoustic model.

  According to the present invention, since the similarity of the acoustic characteristics can be evaluated based on the distance between the feature amounts calculated from the impulse response, data suitable for the user environment can be selected. In particular, since the acoustic model used for speech recognition can be learned and used according to the acoustic characteristics of the user environment, the accuracy of speech recognition can be improved.

FIG. 1 is a diagram illustrating a functional configuration of the feature quantity extraction device. FIG. 2 is a diagram illustrating a processing procedure of the feature amount extraction method. FIG. 3 is a diagram illustrating a functional configuration of the acoustic model learning device. FIG. 4 is a diagram illustrating a functional configuration of the acoustic model selection device. FIG. 5 is a diagram illustrating a processing procedure of the speech recognition method. FIG. 6 is a diagram showing experimental results. FIG. 7 is a diagram showing experimental results.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

<First embodiment>
1st Embodiment of this invention is the feature-value extraction apparatus and method which select the data for simulating the user environment based on the feature-value calculated from the impulse response measured in the user environment. As shown in FIG. 1, the feature quantity extraction device of the first embodiment includes an impulse response measurement unit 1, a feature quantity calculation unit 2, a data selection unit 3, and a data storage unit 9. The feature quantity extraction apparatus performs the process of each step shown in FIG. 2 to realize the feature quantity extraction method of the first embodiment.

  The feature quantity extraction device is, for example, a special configuration in which a special program is read into a known or dedicated computer having a central processing unit (CPU), a main storage device (RAM), and the like. Device. For example, the feature quantity extraction device executes each process under the control of the central processing unit. The data input to the feature quantity extraction device and the data obtained in each process are stored, for example, in the main storage device, and the data stored in the main storage device is read out as necessary and used for other processing. Is done. In addition, at least a part of each processing unit of the feature quantity extraction device may be configured by hardware such as an integrated circuit. Each storage unit included in the feature quantity extraction device includes, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory, or It can be configured with middleware such as a relational database or key-value store. Each storage unit included in the feature quantity extraction device may be logically divided, and may be stored in one physical storage device.

  The data storage unit 9 of the feature quantity extraction device stores data for simulating a plurality of acoustic environments in association with feature quantities calculated from impulse responses measured in each acoustic environment. This feature amount is calculated by the feature amount extraction unit 2 described later. The data for simulating the acoustic environment is data required for performing computer simulation in the prior art, and the content varies depending on the specifications of the computer simulation. Here, it is assumed that the waveform data is an impulse response measured in an acoustic environment.

  In step S1, the impulse response measurement unit 1 of the feature quantity extraction device measures an impulse response representing the spatial acoustic characteristics of the user environment. Hereinafter, the measured impulse response is represented as h (t). Note that t represents time. The impulse response may be measured using a known impulse response measurement technique. The measured impulse response h (t) is sent to the feature quantity calculation unit 2.

  In step S2, the feature amount calculation unit 2 of the feature amount extraction apparatus receives the impulse response h (t) from the impulse response measurement unit 1, and converts the impulse response h (t) into a direct sound component, an initial reflection component, and a rear reverberation component. Separately, the ratio of the power of the rear reverberation component to the total power is calculated as a feature amount. Hereinafter, the calculated feature amount is represented as D. The feature amount D can be calculated by, for example, Expression (1).

Where t ₁ represents the time at which rear reverberation starts. The calculated feature value D of the impulse response h (t) is sent to the data selection unit 3.

  In step S3, the data selection unit 3 of the feature amount extraction apparatus receives the feature amount D of the impulse response h (t) from the feature amount calculation unit 2, and calculates the feature amount D calculated from the impulse response h (t) and the data storage unit. The data for simulating the user environment is selected and output based on the distance from the feature amount calculated from each impulse response stored in FIG. The output data is used for simulating the user environment on the calculation simulation, and the user environment can be analyzed. As a method for calculating the distance between feature amounts, an absolute value of a difference between feature amounts may be used, or an Euclidean distance may be used. When selecting data, it is preferable to select data with a certain spread rather than selecting only data with a distance of zero. For example, even when the distance between feature amounts is shorter than a predetermined threshold value, the feature amounts are considered to match. Thereby, it is possible to prevent performance degradation due to a slight change in the user environment.

  With the configuration as described above, according to the feature quantity extraction device of the first embodiment, it is possible to extract a feature quantity that can evaluate the similarity of impulse responses representing spatial acoustic characteristics. Therefore, data for simulating the user environment can be appropriately selected from data prepared for simulating various acoustic environments prepared in advance based on the similarity (distance) between the feature quantities of the impulse response.

<Second embodiment>
The second embodiment of the present invention generates a learning speech that simulates various acoustic environments, and uses them to learn acoustic models corresponding to various acoustic environments, and various acoustics. An acoustic model selection apparatus and method for selecting an acoustic model suitable for a user environment from an acoustic model corresponding to an environment.

  As shown in FIG. 3, the acoustic model learning device according to the second embodiment includes an impulse response storage unit 11, a feature amount calculation unit 12, an impulse response classification unit 13, a learning data generation unit 14, an acoustic model learning unit 15, and an acoustic model. A storage unit 16 and a clean voice storage unit 19 are included. As shown in FIG. 4, the acoustic model selection device according to the second embodiment includes an acoustic model storage unit 16, an impulse response measurement unit 21, a feature amount calculation unit 22, and an acoustic model selection unit 23. The acoustic model learning device and the acoustic model selection device execute the steps shown in FIG. 5 to realize the speech recognition method of the second embodiment.

  The impulse response storage unit 11 of the acoustic model learning device stores a plurality of impulse responses measured in various acoustic environments.

  A clean voice prepared in advance is stored in the clean voice storage unit 19 of the acoustic model learning device.

  In step S <b> 12, the feature amount calculation unit 12 of the acoustic model learning device calculates, as a feature amount, the ratio of the power of the rear reverberation component to the total power from each impulse response stored in the impulse response storage unit 11. The feature amount calculation method is the same as the feature amount calculation unit 2 of the first embodiment. The calculated feature value of each impulse response is sent to the impulse response classification unit 13.

  In step S <b> 13, the impulse response classification unit 13 of the acoustic model learning device receives the feature amounts of the plurality of impulse responses from the feature amount calculation unit 12 and classifies the impulse responses into a plurality of groups based on the distances between the feature amounts. Each impulse response is sent to the learning data generation unit 14 for each group.

  In step S <b> 14, the learning data generation unit 14 of the acoustic model learning device receives an impulse response for each group from the impulse response classification unit 13, and clean speech stored in the clean speech storage unit 19 for each impulse response included in each group. The speech for learning corresponding to each group is generated. For example, if there are 10 clean sounds and there are 5 impulse responses included in the group, 50 learning sounds are generated. The generated learning voice is sent to the acoustic model learning unit 15 for each group.

  In step S15, the acoustic model learning unit 15 of the acoustic model learning device receives the learning speech for each group from the learning data generation unit 14, and learns the acoustic model for each group using the learning speech corresponding to each group. . The acoustic model corresponding to each learned group is stored in the acoustic model storage unit 16 in association with the impulse response included in each group.

  In step S21, the impulse response measurement unit 21 of the acoustic model selection device measures an impulse response h (t) representing the spatial acoustic characteristics of the user environment. The impulse response measurement method is the same as that of the impulse response measurement unit 1 of the first embodiment. The measured impulse response h (t) is sent to the feature quantity calculation unit 22.

  In step S22, the feature value calculation unit 22 of the acoustic model selection device receives the impulse response h (t) from the impulse response measurement unit 21, and converts the impulse response h (t) into a direct sound component, an initial reflection component, and a rear reverberation component. Separately, the ratio of the power of the rear reverberation component to the total power is calculated as the feature amount D. The feature amount calculation method is the same as the feature amount calculation unit 2 of the first embodiment. The calculated feature value D of the impulse response h (t) is sent to the acoustic model selection unit 23.

  In step S23, the acoustic model selection unit 23 of the acoustic model selection device receives the feature amount D of the impulse response h (t) from the feature amount calculation unit 22, and calculates the feature amount D calculated from the impulse response h (t) and the acoustic model. Based on the distance from the feature amount calculated from the impulse response corresponding to each acoustic model stored in the storage unit 16, the acoustic model corresponding to the user environment is selected and output. The output acoustic model is used to recognize voice collected in the user environment. The method for calculating the distance between feature amounts and the method for selecting data are the same as those of the data selection unit 3 of the first embodiment. In order to prevent performance degradation due to slight changes in the user environment, the selection is performed with a certain spread. To do.

  By configuring as described above, according to the acoustic model learning device and the acoustic model selection device of the second embodiment, an acoustic model suitable for the user environment can be obtained by selecting an acoustic model having a close impulse response feature amount. Therefore, it is possible to improve the accuracy of speech recognition.

<Experimental result>
Learning data corresponding to multiple acoustic environments was generated on a computer simulation and a speech recognition experiment was conducted. FIG. 6-7 shows the experimental results.

  FIG. 6 is an experimental result showing the relationship between the distance between feature amounts and the recognition rate. When learning acoustic models corresponding to each acoustic environment from learning speech corresponding to various acoustic environments (learning data), and recognizing speech (evaluation data) recorded in a user environment using each acoustic model The recognition rate was plotted on the graph. The horizontal axis is the distance between the feature amounts of the learning data and the evaluation data, and the vertical axis is the recognition rate. A high correlation is observed between the distance between the feature amounts and the recognition rate, and it can be seen that the recognition rate increases as the distance between the feature amounts is closer.

  FIG. 7 shows a change in recognition rate when speech (evaluation data) recorded in a certain user environment is recognized by speech using an acoustic model learned from learning speech (learning data) corresponding to a plurality of types of acoustic environments. It is an experimental result. Four types of sound (clean sound, sound in an acoustic environment with D = 0.028, sound in an acoustic environment with D = 0.056, sound in an acoustic environment with D = 0.075) The graph shows the recognition rate when speech recognition is performed between the learned acoustic model and the acoustic model learned with speech in an acoustic environment where the feature amounts do not match. The recognition rate level (80%) indicated by a thick dotted line represents the performance at which speech recognition is considered practical. For example, an acoustic model (clean speech model) trained with clean speech for an acoustic environment with D = 0.028 and an acoustic model (model with D = 0.028) trained with speech in an acoustic environment with D = 0.028 In the case of voice recognition, the latter has a higher recognition rate. That is, it can be seen that a higher recognition rate can be achieved by performing speech recognition using an acoustic model with matching or similar feature amounts. It can also be seen that practical speech recognition accuracy is achieved when the feature values match in all patterns. For the voice in the environment where D = 0.056, an experiment was performed using a clean voice model and a model with D = 0.075 (that is, none of the feature quantities match), but the feature quantity is closer to D = 0.075. It can be seen that the model has a higher recognition rate. This means that even if the feature quantity does not necessarily match, if an acoustic model is selected with a certain spread, a practically adequate acoustic model can be selected.

  With the configuration as described above, according to the feature amount extraction technique of the present invention, the user environment can be simulated only by the user selecting the usage environment, so that convenience is improved. When applied to a speech recognition system, the amount of speech data to be recorded in an actual user environment can be reduced, so the cost associated with learning an acoustic model can be greatly reduced. Moreover, since the acoustic model close to the actual user environment can be learned, the speech recognition rate in the actual environment can be improved.

  As described above, the embodiments of the present invention have been described, but the specific configuration is not limited to these embodiments, and even if there is a design change or the like as appropriate without departing from the spirit of the present invention, Needless to say, it is included in this invention. The various processes described in the embodiments are not only executed in time series according to the description order, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, this computer reads the program stored in its own recording medium and executes the process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

DESCRIPTION OF SYMBOLS 1 Impulse response measurement part 2 Feature quantity extraction part 3 Data selection part 9 Data storage part 11 Impulse response storage part 12 Feature quantity extraction part 13 Impulse response classification part 14 Learning data generation part 15 Acoustic model learning part 19 Clean speech storage part 21 Impulse Response measurement unit 22 Feature amount extraction unit 23 Acoustic model selection unit

Claims (5)

A feature quantity extraction device for extracting feature quantities in the user environment in order to appropriately select data for simulating the user environment from data prepared to simulate various acoustic environments,
A feature amount calculation unit that calculates a feature amount that represents the ratio of the power of the rear reverberation in the impulse response to the total power; and
An impulse response measurement unit for measuring the impulse response of the user environment;
A data storage unit that stores data related to a plurality of different acoustic environments and the feature amount calculated from the impulse response measured in the acoustic environment in association with each other;
A data selection unit that selects data related to the acoustic environment corresponding to the user environment based on a distance between the feature amount calculated from the impulse response of the user environment and the feature amount calculated from the impulse response of the acoustic environment; ,
A feature amount extraction device. An acoustic model learning device that learns an acoustic model suitable for each acoustic environment from learning speech generated using an impulse response selected from impulse responses measured in various acoustic environments prepared in advance,
A clean sound storage unit for storing clean sound;
An impulse response storage unit for storing impulse responses measured in a plurality of different acoustic environments;
A feature amount calculation unit that calculates a feature amount that represents the ratio of the power of the rear reverberation in the impulse response to the total power; and
An impulse response classifying unit that classifies the impulse responses into a plurality of groups based on the distance between the feature values of the impulse response of the acoustic environment;
A learning data generating unit that convolves each of the impulse responses classified into each group into the clean sound and generates a learning sound corresponding to each group;
An acoustic model learning unit that learns an acoustic model corresponding to each group using learning speech corresponding to each group;
An acoustic model learning device. An acoustic model selection device for selecting an acoustic model suitable for a user environment from acoustic models suitable for each acoustic environment learned using learning speech corresponding to various acoustic environments prepared in advance,
A feature amount calculation unit that calculates a feature amount that represents the ratio of the power of the rear reverberation in the impulse response to the total power; and
An impulse response measurement unit for measuring the impulse response of the user environment;
An acoustic model storage unit that associates and stores an acoustic model learned using learning speech corresponding to a plurality of different acoustic environments and the feature amount calculated from the impulse response measured in the acoustic environment;
An acoustic model selection unit that selects an acoustic model corresponding to the user environment based on a distance between the characteristic amount of the impulse response of the user environment and the characteristic amount of the impulse response corresponding to each acoustic model;
An acoustic model selection device including: A feature amount extraction method for extracting feature amounts in the user environment in order to appropriately select data for simulating the user environment from data prepared to simulate various acoustic environments,
The data storage unit stores data related to a plurality of different acoustic environments,
The impulse response measurement unit measures the impulse response of the user environment,
The feature amount calculation unit calculates a feature amount that represents the ratio of the power of the rear reverberation in the impulse response of the user environment to the total power,
The feature amount calculation unit calculates a feature amount that represents the ratio of the power of rear reverberation in the impulse response measured in the acoustic environment to the total power,
A data selection unit selects data related to the acoustic environment corresponding to the user environment based on a distance between the feature amount calculated from the impulse response of the user environment and the feature amount calculated from the impulse response of the acoustic environment. A feature extraction method including   A program for causing a computer to function as the feature quantity extraction device according to claim 1, the acoustic model learning device according to claim 2, or the acoustic model selection device according to claim 3.

Images