JP6784255B2 - Speech processor, audio processor, audio processing method, and program - Google Patents

Description

The present invention relates to a voice processing device, a voice processing system, a voice processing method, and a program for extracting a frequent pattern from voice data.

In recent years, forensic science-based scientific methods have been widely used in police criminal investigations. In the fingerprint test, which is a typical example, the fingerprint images collected at the crime scene are sequentially compared with a large number of known fingerprint images to estimate who was involved in the crime. A method similar to fingerprint test that handles voice is called voiceprint test or voice test.

Patent Document 1 describes a technique for extracting voice data of an unknown word that is a candidate for a keyword to be registered in a voice recognition dictionary from voice data. The technique described in Patent Document 1 detects a section in which a state in which the voice power value of voice data is greater than the threshold th1 is continuous for a certain period of time or longer, and a state in which the power value is greater than the threshold th2 is detected from each utterance section. Divide into sections that are continuous for a certain period of time or longer. Then, the technique described in Patent Document 1 acquires a phoneme sequence from the divided speech data, performs clustering, calculates an evaluation value, detects an unknown word, and registers it in a dictionary.

Patent Document 2 describes a technique for determining a factor that causes misrecognition and notifying the user. The technique described in Patent Document 2 divides a vector sequence of mel-cepstrum coefficients (hereinafter referred to as "MFCC") extracted by a feature extraction unit into segments for each phonetic element using a set of standard models. .. Then, the technique described in Patent Document 2 investigates the cause of the misrecognition, creates a character string of a message to be presented to the user according to the analysis result, and notifies the user by displaying the message on the display. ..

International Publication No. 2009/136440 Japanese Unexamined Patent Publication No. 2004-325635

M. Kotti, V.I. Moschou, and C.I. Kotropoulos, "Speaker segmentation and clustering," Signal Processing, Vol. 88, No. 5, pp. 1091-1124, May 2008. T. Anastasakos, J. Mol. McDonough, J. Mol. Makhoul, "Speaker adaptive training: a maximum likelihood approach to speaker normalization," Signal Processing, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 2, pp. 1043-1046, Apr. 1997. M. J. F. Gales, "Cruster adaptive training of hidden Markov models," IEEE Trans. on Speech and Audio Processing, Vol. 8, No. 4, pp. 417-428, Jul. 2000.

However, with the technique described in Patent Document 1, an unknown word that is a candidate for a keyword can be selected, but a phrase containing a sentence (for example, a sentence such as "Prepare a ransom") cannot be selected. The technique described in Patent Document 2 can analyze a vector sequence for each segment that causes erroneous recognition, but cannot select a desired phrase. That is, the techniques described in Patent Documents 1 and 2 have a problem that a desired phrase cannot be selected.

An object of the present invention is to provide a voice processing device, a voice processing system, a voice processing method, and a program capable of solving the above problems and selecting a desired phrase.

The voice processing apparatus according to one aspect of the present invention has a first generation means for generating a plurality of segments so that adjacent segments overlap at least partially from voice data, and the plurality of segments have a phonological similarity. A second generation means for classifying based on and generating clusters, a selection means for selecting clusters satisfying a predetermined condition based on the size of the cluster, and an extraction means for extracting segments included in the selected cluster. And.

In the speech processing method according to one aspect of the present invention, a plurality of segments in which adjacent segments overlap at least partially are generated from the speech data, and the plurality of segments are classified based on phonological similarity to generate a cluster. , A cluster satisfying a predetermined condition is selected based on the size of the cluster, and the segments included in the selected cluster are extracted.

The program according to one aspect of the present invention generates clusters by generating a plurality of segments in which adjacent segments overlap at least partially from audio data and classifying the plurality of segments based on phonological similarity. and processing, based on the size of the clusters, a process of selecting a predetermined condition is satisfied cluster, Ru to execute the processing for extracting the segments included in the selected cluster to the computer.

The present invention has an effect that a desired phrase can be selected in a voice processing device, a voice processing system, a voice processing method, and a program.

It is a block diagram which shows the structural example of the voice processing apparatus which concerns on 1st Embodiment of this invention. It is a schematic block diagram which shows the structural example of the computer in each embodiment and specific example of this invention. It is a flowchart which shows the operation example of the voice processing apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows an example of the method which the voice processing apparatus which concerns on 1st Embodiment of this invention extracts a phrase using HMM. It is a block diagram which shows the structural example of the voice processing apparatus which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the operation example of the voice processing apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the voice processing apparatus which concerns on 3rd Embodiment of this invention. It is a flowchart which shows the operation example of the voice processing apparatus which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the structural example of the voice processing system which concerns on 4th Embodiment of this invention. It is a figure which shows an example of the voice data which the external storage device stores in the specific example of this invention. It is a figure which shows an example of the method which the generation part in the specific example of this invention divides voice data. It is a figure which shows an example of the method which the clustering part in the specific example of this invention generates a cluster which put together a plurality of segments.

First, the background of the present invention will be described in order to facilitate understanding of the embodiments of the present invention.

In the voice appraisal method related to the present invention, for example, a phone call for a ransom request from a kidnapper or a terrorist's crime notice is recorded, and the recorded voice is compared with a known voice to try to identify the main voice of the phone.

Unlike fingerprints, which are lifelong, voice changes each time it is spoken. Therefore, in the voice appraisal method, a part (section) of the voice in which the same content is spoken is cut out and compared. In the voice appraisal method, for example, in the kidnapper's ransom request, it is expected that the phrase "prepare money" will often appear, so find and cut out such a phrase and also "prepare money". Compare with the voice spoken.

What phrase to use is on a case-by-case basis. In the case of a kidnapper, the above-mentioned "Prepare money" appears frequently, so it is considered appropriate. The phrase about money would be appropriate for wire fraud, but it's probably not the case for kidnappers. Also, in the case of terrorists, there will be better alternative phrases, and it would be better to use different phrases in military and other government intelligence activities. The selection of such frequently occurring phrases has relied on the experience and intuition of skilled appraisers. However, in such a case, it is necessary for a skilled appraiser to carefully observe the voice over time, and there is a problem that a large human cost is required to obtain the desired phrase necessary for the voice appraisal. ..

According to the embodiment of the present invention described below, the above-mentioned problems and the like can be solved, and a desired phrase can be selected.

Hereinafter, embodiments and specific examples of the present invention will be described with reference to the drawings. In addition, about each embodiment and a specific example, the same components are designated by the same reference numerals, and description thereof will be omitted as appropriate.

[First Embodiment]
Hereinafter, a first embodiment for carrying out the present invention (hereinafter, referred to as “first embodiment”) will be described in detail with reference to the drawings.

[Description of configuration]
FIG. 1 is a block diagram showing a configuration example of the voice processing device 10 according to the first embodiment of the present invention. Referring to FIG. 1, the voice processing apparatus 10 according to the first embodiment of the present invention includes a generation unit 11, a clustering unit 12, a selection unit 13, and an extraction unit 14. Here, the generation unit 11 is also described as a first generation unit. The clustering unit 12 is also described as a second generation unit.

The generation unit 11 generates a plurality of segments in which at least a part of adjacent segments overlaps from the voice data stored in the external storage device. For example, the generation unit 11 subdivides the voice data stored in the external storage device into short time units, and generates a plurality of segments using the subdivided voice data. Further, the time length of the plurality of segments generated by the generation unit 11 may be a constant time length. Further, the generation unit 11 may divide one voice data a plurality of times with different time lengths, and generate segments having various time lengths by using the divided voice data.

The clustering unit 12 classifies a plurality of segments based on a predetermined similarity index to generate clusters.

The selection unit 13 selects at least one cluster from the generated clusters based on the size of each cluster. The extraction unit 14 extracts the segments included in the selected cluster. Here, the size of the cluster is, for example, the total time length of the segment, the result obtained by multiplying the number of occurrences of the content of the segment (also referred to as a phrase) and the length of the segment, and the like.

FIG. 2 is a schematic block diagram showing a configuration example of the computer 1000 in each embodiment and specific example of the present invention. The computer 1000 includes a CPU (Central Processing Unit) 1001, a main storage device 1002, an auxiliary storage device 1003, an interface 1004, an input device 1005, and a display device 1006.

The voice processing device 10 and the like of each embodiment and the specific example are mounted on the computer 1000. The operation of the voice processing device 10 and the like is stored in the auxiliary storage device 1003 in the form of a program. The CPU 1001 reads a program from the auxiliary storage device 1003, expands it into the main storage device 1002, and executes the above processing according to the program. For example, the CPU 1001 reads the program from the auxiliary storage device 1003 and expands it to the main storage device 1002 to realize the functions of the generation unit 11, the clustering unit 12, the selection unit 13, and the extraction unit 14.

Auxiliary storage 1003 is an example of a non-temporary tangible medium. Other examples of non-temporary tangible media include magnetic disks, magneto-optical disks, CD (Compact Disk) -ROM (Read Only Memory), DVD (Digital Versailles Disk) -ROM, semiconductors connected via interface 1004. Memory and the like can be mentioned. When this program is distributed to the computer 1000 via a communication line, the distributed computer 1000 may expand the program to the main storage device 1002 and execute the above processing.

The interface 1004 is connected to the CPU 1001 and is connected to a network or an external storage medium. External data may be taken into the CPU 1001 via the interface 1004. The input device 1005 is, for example, a keyboard, a mouse, a touch panel, or a microphone. The display device 1006 displays a screen corresponding to drawing data processed by a CPU 1001 or a GPU (Graphics Processing Unit) (not shown), such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube) display. It is a device to do. The hardware configuration shown in FIG. 2 is only an example, and each part shown in FIG. 2 may be configured by an independent logic circuit.

Further, the program may be for realizing a part of the above-mentioned processing. Further, the program may be a difference program that realizes the above-mentioned processing in combination with another program already stored in the auxiliary storage device 1003.

[Explanation of operation]
The operation of this embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing an operation example of the voice processing device 10 according to the first embodiment of the present invention.

The generation unit 11 generates a plurality of segments from the voice data stored in the external storage device (step S101). At this time, the generation unit 11 generates a plurality of segments so that the adjacent segments overlap at least in time. The time length of the segment may be a constant value in the range of, for example, 1 second to several seconds, depending on the time length of the assumed phrase.

Further, the generation unit 11 may divide one voice data a plurality of times with different time lengths to generate segments having various time lengths. Further, the generation unit 11 divides the voice data at a predetermined change point, a silent section, or the like by using the method described in Non-Patent Document 1, and uses the divided plurality of voice data to generate a variable length segment. It may be generated.

The clustering unit 12 classifies a plurality of segments based on a predetermined similarity index to generate a cluster (step S102). That is, the clustering unit 12 clusters a plurality of segments. The clustering unit 12 calculates the similarity between each segment from the plurality of segments generated by the generation unit 11, and generates a cluster in which the segments having a high degree of similarity are grouped together. As a specific method of the similarity index and cluster generation by the clustering unit 12, for example, the method described in Non-Patent Document 1 can be used.

Here, the similarity index is an index for measuring the similarity of the phonemes constituting the segment. The similarity index is an index using statistics of acoustic features such as the Batacharya distance calculated from the mean and variance of the acoustic features series, the divergence of Calvac Libra, and the log-likelihood ratio. These indicators do not consider the order of the acoustic feature series within the segment.

Further, as a method using the similarity index, for example, an index considering the order, that is, the time order may be used. A method using the similarity index is, for example, a DP matching method in which the optimum correspondence between each acoustic feature is obtained by a dynamic programming method (hereinafter referred to as "DP") and the similarity is calculated. is there. Here, the acoustic feature amount is, for example, MFCC. MFCC is widely used in voice recognition and the like.

When the clustering in the clustering unit 12 converges (Yes in step S103), the selection unit 13 selects a cluster that satisfies a predetermined condition from the clusters generated by the clustering unit 12 based on the size of each cluster (step). S104). In this selection, the selection unit 13 compares the sizes of clusters from the viewpoint of finding frequently occurring phrases, and selects at least one cluster in descending order of size. Predetermined conditions include a larger number of segments, a longer total time length of the segments, and the like. That is, the selection unit 13 selects, for example, a cluster including more segments or a cluster having a longer total time length of the segments as a cluster satisfying a predetermined condition.

Here, the case where clustering converges is, for example, a situation in which steps S101 and S102 are executed a predetermined number of times, a situation in which an increase or decrease in a predetermined evaluation value related to clustering becomes a certain value or less. The case where the clustering converges may be a situation in which the movement of the segment between the clusters is stopped in association with the situation where the increase / decrease of the predetermined evaluation value related to the clustering becomes a certain value or less.

The extraction unit 14 extracts a segment from one or a plurality of segments included in the cluster selected by the selection unit 13 (step S105). As a result, the extraction unit 14 can extract the segment of the portion corresponding to the desired phrase from the voice data.

Here, if the clustering in the clustering unit 12 has not converged (No in step S103), the process returns to the process of step S101. This indicates that since steps S101 and S102 are interdependent, they may be repeated a predetermined number of times or until they converge.

The generation unit 11 and the clustering unit 12 can be collectively executed by using a hidden Markov model (hereinafter referred to as “HMM”) having the structure shown in FIG. FIG. 4 is a diagram showing an example of a method in which the voice processing device 10 extracts a phrase using the HMM. That is, the voice processing device 10 learns the HMM as shown in FIG. 4 based on the maximum likelihood estimation method or the like by using the voice data stored in the external storage device. As a result, the one-way HMM (Left-) expressing the first phrase (phrase 1 in FIG. 4), the second phrase (phrase 2 in FIG. 4), ..., the Nth phrase (phrase N in FIG. 4) To-right HMM) is automatically formed, and at the same time, the segments belonging to each are also acquired.

By listening to the relevant part of the extracted voice data, frequently occurring phrases can be confirmed and used for voice appraisal.

[Explanation of effect]
As described above, according to the voice processing device 10 according to the present embodiment, the generation unit 11 generates a plurality of segments from the voice data so that at least a part of the adjacent segments overlap, and the clustering unit 12 generates the phoneme. Create clusters by classifying multiple segments based on similarity. Then, according to the voice processing device 10 according to the present embodiment, the selection unit 13 selects at least one cluster from the clusters based on the size of each cluster. Further, according to the voice processing device 10 in the present embodiment, since the extraction unit 14 extracts the segment included in the selected cluster, it is possible to extract the segment corresponding to the desired phrase from the voice data. It will be possible. The reason is that since the generation unit 11 generates a plurality of segments from the voice data so that at least a part of the adjacent segments overlaps, a phrase shorter than the word to a phrase longer than the word is generated as one segment. Because it can be done.

Further, by using the voice processing device 10 in the present embodiment, it is possible to find and select frequently-used phrases necessary for voice appraisal at low cost even if the expert is not a skilled appraiser. The reason is that the voice processing device 10 generates a segment from the given voice data so that at least a part of the adjacent segments overlaps, clusters this segment, and selects a cluster containing a large number of similar segments. Because. This is because the voice processing device 10 extracts the segments included in the cluster selected in this way. The extracted segment is a segment generated by the generation unit 11, and is partial voice data including a desired phrase in the voice data. This is because the voice processing device 10 can automatically find phrases that frequently appear in the voice data.

Further, by using the voice processing device 10 in the present embodiment, it is possible to quantitatively find frequently used phrases, so that frequently used phrases useful for voice identification can be found with high reliability.

[Second Embodiment]
Hereinafter, the second embodiment of the present invention will be described in detail with reference to the drawings.

[Description of configuration]
FIG. 5 is a block diagram showing a configuration example of the voice processing device 20 according to the second embodiment of the present invention. Referring to FIG. 5, the voice processing device 20 according to the second embodiment of the present invention includes a normalization learning unit 15, a voice data normalization unit 16, a voice data processing unit 17, and first to Nth voice data storage. Unit (101-1-101-N (N is a positive integer)), unspecified acoustic model storage unit 102, and first to Nth parameter storage units (103-1-103-N (N is a positive integer). )) Is provided.
Here, the normalization learning unit 15 is also described as a third generation unit. In the present embodiment, when the first to Nth voice data storage units (101-1 to 101-N) are not distinguished from each other, or when they are generically referred to, they are referred to as voice data storage units 101. Further, when the first to first Nth parameter storage units (103-1 to 103-N) are not distinguished from each other, or when they are generically referred to, they are referred to as a parameter storage unit 103.

The voice data storage means 101 stores voice data having different properties. That is, the first voice data storage unit 101-1, the second voice data storage unit 101-2, ..., And the Nth voice data storage unit 101-N store voice data having different properties. Further, the voice data having different properties stored in the first voice data storage unit 101-1, the second voice data storage unit 101-2, ..., And the Nth voice data storage unit 101-N are stored. These are audio data classified based on their acoustic properties.

The unspecified acoustic model storage unit 102 stores the unspecified acoustic model learned by the normalized learning unit 15. The unspecified acoustic model is a model obtained by normalizing the difference in voice data having different properties stored in the voice data storage unit 101.

The parameter storage unit 103 stores each of the parameters for normalizing the difference in the voice data. That is, the first parameter storage unit 103-1, the second parameter storage unit 103-2, ..., And the Nth parameter storage unit 103-N provide parameters for normalizing the difference in voice data. Remember each one.

The normalization learning unit 15 performs normalization learning using voice data having different properties stored in the voice data storage unit 101.

Here, the normalization learning is, for example, a learning method of an acoustic model described in Non-Patent Document 2.
In the acoustic model, each tone i is defined by the average vector μ _i of the acoustic features, but in normalization learning, the average vector can change depending on the nature of the speech data. That is, in the present embodiment, the average vector (unspecified acoustic model) μ _i is expressed by an affine transformation as shown in the following equation (1).

Here, s = 1, 2, ..., N. Also, A _s and b _s are each an parameter for normalizing the difference in nature of the audio data.

The equation (1), and unspecified acoustic model mu _i which is not affected by the difference in nature of the audio data, the parameters A _S and b _S for normalizing the difference in nature of the audio data is obtained. Then, the normalization learning unit 15 stores the unspecified acoustic model μ _i in the unspecified acoustic model storage unit 102. Further, the normalization learning unit 15, the parameter A _S and b _S, stored in the parameter storage unit 103. Specifically, the normalization learning unit 15 stores the parameters A ₁ and b ₁ in the first parameter storage unit 103-1 and stores the parameters A _N and b _N in the Nth parameter storage unit 103-N. Store. Non-Patent Document 2 describes a method of normalizing the difference between speakers, assuming that the properties of voice data differ depending on the speaker, but the difference in the properties of voice data is not limited to the speaker, and background noise, Various assumptions such as microphones and communication lines are possible.

That is, the normalization learning unit 15 generates a normalization parameter for normalizing the difference in the voice data having different properties stored in the voice data storage unit 101, and stores the normalization parameter in the parameter storage unit 103. Further, the normalization learning unit 15 learns the unspecified acoustic model by normalizing the difference in the voice data having different properties stored in the voice data storage unit 101, and stores the learned unspecified acoustic model in the unspecified acoustic model 102. To do. Here, the normalization learning unit 15 generates the normalization parameter by estimating a normalization parameter for normalizing the difference of the voice data having different properties stored in the voice data storage unit 101. Further, the normalization learning unit 15 stores the unspecified acoustic model in the unspecified acoustic model 102 for each iteration, for example, when performing an iterative calculation.

The voice data normalization unit 16 refers to the parameters stored in the parameter storage unit 103, normalizes the voice data stored in the voice data storage unit 101, and sends the voice data to the voice data processing unit 17. Specifically, the s parameter is used for the time series x ₁ , x ₂ , ..., X _t , ... (t is a positive integer) of the acoustic features of the s voice data. , Equation (2), which is a transformation corresponding to the inverse transformation of equation (1), is applied.

The parameters that specify the normalization may be different depending on the phoneme class (fricative, plosive, etc.), or may be different depending on the preceding and following phonemes in consideration of context dependence. .. Further, the voice data normalization unit 16 may normalize not only the average vector of the acoustic features but also the variance. Further, not limited to these, various devices known for normalization learning may be applied.

The voice data processing unit 17 has the same configuration and effect as the voice processing device 10 in the first embodiment. That is, the voice data processing unit 17 executes the processing of the generation unit 11, the clustering unit 12, the selection unit 13, and the extraction unit 14 shown in FIG. 1 in the same manner as in the first embodiment, and is included in the normalized voice data. Outputs a segment containing phrases that frequently appear in.

[Explanation of operation]
The operation of this embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing an operation example of the voice processing device 20 according to the second embodiment of the present invention. Here, as shown in FIG. 6, the operation of the voice data processing unit 17 in the present embodiment, that is, steps S204 to S208 is the operation of the voice processing device 10 in the first embodiment, that is, steps S101 to S105. Since the same is true, the description thereof will be omitted.

The normalization learning unit 15 reads each voice data from the voice data storage unit 101, performs normalization learning, and stores the normalization parameters of each voice data in the parameter storage unit 103 (step S201).

The normalization learning unit 15 stores the unspecified acoustic model generated after performing normalization to eliminate the difference in the properties of the voice data in the unspecified acoustic model storage unit 102 (step S202).

The voice data normalization unit 16 refers to the normalization parameters stored in the parameter storage unit 103, and normalizes the voice data stored in the voice data storage unit 101 (step S203).

The voice data processing unit 17 executes the same processing as steps S101 to S105 of the voice processing device 10 in the first embodiment shown in FIG. 3, and outputs a segment including a phrase frequently appearing in the voice data (step). S204 to step S208).

[Explanation of effect]
As described above, according to the voice processing device 20 in the present embodiment, the normalization learning unit 15 reads each voice data from the voice data storage unit 101, performs normalization learning, and sets the normalization parameter of each voice data. It is stored in the parameter storage unit 103. The normalization learning unit 15 stores the unspecified acoustic model generated after normalizing and eliminating the difference in the acoustic properties of the respective voice data in the unspecified acoustic model storage unit 102. Further, the voice data normalization unit 16 refers to the normalization parameters stored in the parameter storage unit 103, and normalizes the voice data stored in the voice data storage unit 101, respectively. The voice data processing unit 17 outputs a segment including a phrase that frequently appears in the normalized voice data. Therefore, the voice processing device 20 in the present embodiment can normalize the non-normalized voice data and select a desired phrase.

Further, according to the voice processing device 20 in the present embodiment, the normalization learning unit 15 differs in the acoustic properties of the first voice data, the second voice data, ..., The Nth voice data (for example, the speaker). (Difference) is learned to normalize. After the voice data normalization unit 16 eliminates the difference in acoustic properties, the voice data processing unit 17 extracts a segment containing a phrase that frequently appears in the voice data. Therefore, the voice processing device 20 can more accurately extract phrases that frequently appear in the voice data. The reason is that in the voice processing device 20 of the present embodiment, the clustering unit 12 in the voice data processing unit 17 is affected by the difference in the properties of the voice data to generate an inappropriate cluster (for example, a cluster of speakers). This is because such a situation can be reduced.

[Third Embodiment]
Hereinafter, the third embodiment of the present invention will be described in detail with reference to the drawings.

[Description of configuration]
FIG. 7 is a block diagram showing a configuration example of the voice processing device 30 according to the third embodiment of the present invention. Referring to FIG. 7, the voice processing device 30 according to the third embodiment of the present invention includes the unclassified voice data storage unit 104 and the voice data classification unit 18 in addition to the configuration of the voice processing device 20 according to the second embodiment. And. Here, since the configuration of the voice processing device 20 in the second embodiment has already been described, the description thereof will be omitted. The voice data classification unit 18 is also described as a fourth generation unit.

The unclassified voice data storage unit 104 stores voice data.

The voice data classification unit 18 classifies the voice data stored in the voice data storage unit 104 based on acoustic properties and stores it in the voice data storage unit 101. The voice data classification unit 18 classifies the voice data stored in the unclassified voice data storage unit 104 into N clusters based on the difference in acoustic properties, for example, the difference in speakers, and the voice data storage unit 18. Each is stored in 101. That is, the voice data classification unit 18 generates N clusters by classifying the voice data stored in the unclassified voice data storage unit 104 based on the acoustic properties. Then, the voice data classification unit 18 has a first cluster in the first voice data storage unit, a second cluster in the second voice data storage unit, ..., A second in the Nth voice data storage unit. Store N clusters.

Here, the voice data stored in the unclassified voice data storage unit 104 may be a mixture of voice data having various acoustic properties. Further, N may be a predetermined constant, or may be automatically determined by the voice data classification unit 18 according to the processing target. These can be carried out by applying a known clustering method.

The voice data storage unit 101 stores each voice data classified by the voice data classification unit 18.

[Explanation of operation]
The operation of this embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing an operation example of the voice processing device 30 according to the third embodiment of the present invention. Here, as shown in FIG. 8, the operation of the voice data processing unit 17 in the present embodiment, that is, steps S306 to S310 is the operation of the voice processing device 10 in the first embodiment, that is, steps S101 to S105. Since the same is true, the description thereof will be omitted.

The voice data classification unit 18 classifies the voice data stored in the voice data storage unit 104 based on the acoustic property and stores it in the voice data storage unit 101 (step S301).

The normalization learning unit 15 reads each voice data from the voice data storage unit 101, performs normalization learning, and stores the normalization parameters of each voice data in the parameter storage unit 103 (step S302).

The normalization learning unit 15 stores the unspecified acoustic model generated after performing normalization to eliminate the difference in the properties of the voice data in the unspecified acoustic model storage unit 102 (step S303).

When the results of the voice data classification unit 18 and the normalization learning unit 15 converge (Yes in step S304), the voice data normalization unit 16 refers to the normalization parameters stored in the parameter storage unit 103, and each voice data. The audio data stored in the storage unit 101 is normalized (step S305).

Here, when the results of the voice data classification unit 18 and the normalization learning unit 15 have not converged (No in step S304), the process returns to the flow of step S301. As a result, the voice data classification unit 18 and the normalization learning unit 15 can be repeatedly executed alternately until the results converge.

The results output by the voice data classification unit 18 and the normalization learning unit 15 may depend on each other. Therefore, the speech data classification unit 18 and the normalization learning unit 15 may be repeatedly executed alternately until the number of executions reaches a predetermined threshold value or converges. Such an operation can be efficiently carried out based on an optimization criterion such as maximizing the likelihood, following the method described in Non-Patent Document 3.

The voice data processing unit 17 executes the same processing as steps S101 to S105 of the voice processing device 10 in the first embodiment shown in FIG. 6 and outputs a segment including a phrase frequently appearing in the voice data (step). S306 to step S310).

[Explanation of effect]
As described above, according to the voice processing device 30 in the present embodiment, the voice data classification unit 18 classifies the voice data stored in the voice data storage unit 104 based on the acoustic properties, and the voice data storage unit 101. Remember in. Then, the normalization learning unit 15 reads each voice data from the voice data storage unit 101, performs normalization learning, and stores the normalization parameters of each voice data in the parameter storage unit 103. The normalization learning unit 15 stores the unspecified acoustic model generated after normalizing and eliminating the difference in the acoustic properties of the respective voice data in the unspecified acoustic model storage unit 102. The voice data normalization unit 16 refers to the normalization parameters stored in the parameter storage unit 103, and normalizes the voice data stored in the voice data storage unit 101, respectively. The voice data processing unit 17 outputs a segment including a phrase that frequently appears in the normalized voice data. Therefore, the voice processing device 30 in the present embodiment can classify and normalize voice data that has not been classified and normalized, and select a desired phrase.

Further, according to the voice processing device 30 in the present embodiment, the voice data classification unit 18 classifies the voice data into N clusters based on the difference in acoustic properties, and the normalization learning unit 15 uses the result. Is configured to perform normalization learning. Therefore, the voice processing device 30 in the present embodiment can reduce the preparation cost of voice data as compared with the voice processing device 20 in the second embodiment. The reason is that the voice processing device 30 in the present embodiment does not need to divide the voice data in advance according to the difference in acoustic properties (for example, for each speaker), and collectively collects a set of miscellaneous voice data. This is because it can be given and processed with.

[Fourth Embodiment]
[Description of configuration]
Hereinafter, the fourth embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 9 is a block diagram showing a configuration example of the voice processing system 40 according to the fourth embodiment of the present invention. Referring to FIG. 9, the voice processing system 40 according to the fourth embodiment includes a voice processing device 41, a voice input device 42, an instruction input device 43, and an output device 44.

The voice processing device 41 processes the input voice with respect to the voice processing device 10 according to the first embodiment of the present invention, the processing of the voice processing device 20 according to the second embodiment, or the third embodiment. (Hereinafter referred to as "phrase extraction process according to the first to third embodiments of the present invention") of the voice processing device 30 in the above.

The voice input device 42 inputs voice. The audio input device 42 is an arbitrary device that acts as an interface for inputting arbitrary audio data to the audio processing device 41, that is, a microphone that receives audio signals as data, a memory that records audio data, and the like. The voice input device 42 is, for example, the input device 1005 shown in FIG.

The output device 44 outputs the result of the processing executed by the voice processing device 41. The output device 44 is an output device such as a monitor or a speaker that outputs the processing result of the voice processing device 42 by visual or auditory means according to the instruction input from the instruction input device 43 by the operator. The output method of the output device 44 is such that when the output device 44 is a monitor, for example, a list of clusters is displayed in order of size, the contents of a specific cluster are displayed by a waveform diagram, a spectrogram, etc., and a plurality of segments can be compared. Display them side by side, etc. When the output device 44 is a speaker, the output method of the output device 44 is to reproduce sound, and the like. The output device 44 is realized by, for example, the display device 1006.

The instruction input device 43 receives instruction information from the operator and controls the information displayed on the display device. The instruction input device 43 is a user interface for receiving operator's instruction information such as processing for information output by the output device 44 and processing of the voice processing device 41, and any input device such as a mouse, keyboard, or touch panel can be used. It is available.

[Explanation of operation]
Hereinafter, an operation example of the voice processing system 40 according to the fourth embodiment of the present invention will be described.

The instruction input device 43 receives instruction information from the operator and controls the voice processing device 41 to execute the process. The voice input device 42 inputs arbitrary voice data to the voice processing device 41. The voice processing device 41 executes the phrase extraction process according to the first to third embodiments of the present invention based on the input voice data, selects a cluster containing frequently appearing phrases, and further selects the cluster. Extract the segments contained in the cluster. The output device 44 outputs the processing result of the voice processing device 41 by visual or auditory means according to the instruction input from the instruction input device 43 by the operator. That is, the output device 44 outputs the processing result in the form desired by the operator to view.

[Explanation of effect]
As described above, according to the voice processing system 40 in the present embodiment, the instruction input device 43 controls the voice processing device 41 to execute the process according to the instruction information input from the operator. The voice input device 42 inputs arbitrary voice data to the voice processing device 41. Based on the input voice data, the voice processing device 42 executes the phrase extraction according to the first to third embodiments of the present invention, selects a cluster containing frequently appearing phrases (segments), and further. Extract the segments contained in the selected cluster. The output device 44 outputs the processing result of the voice processing device 41 by visual or auditory means according to the instruction input from the instruction input device 43 by the operator. Therefore, the voice processing system 40 in the present embodiment can output clusters and segments including frequently appearing phrases included in the voice data.

In addition, the voice processing system 40 in the present embodiment allows the operator to easily perform analysis work such as identifying a person from the voice. The reason is that the voice processing system 40 in the present embodiment is configured so that the processing result is output to the output device 44 in a form that the operator wants to view. Further, since the voice processing system 40 in the present embodiment visually and audibly outputs phrases that frequently appear, it is possible to analyze the habits and topics that a specific person often speaks.

(Concrete example)
Hereinafter, specific examples of the first embodiment of the present invention will be described. An example in which the voice processing device 10 extracts a phrase from the voice data will be described with reference to FIGS. 10 to 12.

Details of an example of extracting a phrase from the voice data stored in the external storage device will be described with reference to FIGS. 10 to 12. FIG. 10 is a diagram showing an example of voice data stored in an external storage device. Here, the external storage device is realized by, for example, the voice input device 42 in the fourth embodiment.

As shown in FIG. 10, the external storage device stores the voice data and the voice data ID which is an identifier of the voice data. When the voice data ID is "1", the external storage device stores the voice data such as "... I took care of my child. Prepare a ransom. The meeting place is ...". Here, the external storage device is not limited to the content of the audio data shown in FIG.

FIG. 11 is a diagram showing an example of a method in which the generation unit 11 generates a segment from voice data. As shown in FIG. 11, the generation unit 11 starts from the voice data shown in FIG. 10, that is, the voice data ID "1", "... I took care of my child. Prepare a ransom. The meeting place is ..." , Generate multiple segments. As shown in FIG. 11, segment 1 is “deposited” and segment 2 is “deposited. As shown in FIG. 11, the generation unit 11 arbitrarily subdivides the voice data (predetermined time, etc.) and uses these to generate a plurality of segments. Here, the generation unit 11 generates segments from the voice data so that the segments overlap each other. That is, as shown in FIG. 11, segment 1 is "deposited", segment 2 is "deposited. Substitute", and so on, "deposited" is duplicated in segments 1 and 2. As a result, the voice processing device 10 can extract the phrase required from the voice data.

FIG. 12 is a diagram showing an example of a method in which the clustering unit 12 generates a cluster in which a plurality of segments are grouped together. As shown in FIG. 12, in the cluster, for example, the cluster ID which is an identifier of the content (phrase) of the segment, the content of the segment, and the appearance of the content (phrase) of the segment appearing in all the voice data appear. Including the number of times. As shown in FIG. 12, in this specific example, the cluster ID and the segment number shown in FIG. 11 will be described as being the same. The cluster indicates, for example, that the phrase "deposited" with the cluster ID "1" appears 20 times in all the voice data. That is, as shown in FIG. 11, the clustering unit 12 calculates the similarity between each segment from the plurality of segments generated by the generation unit 11, and generates a cluster having a high degree of similarity, that is, the same segments are grouped together. ..

The selection unit 13 compares the clusters using the number of segments included in the cluster and the total time length, and selects a cluster that satisfies a predetermined condition. The selection unit 13 compares, for example, based on the number of segments included in each cluster among the plurality of clusters generated by the clustering unit 12, that is, the number of occurrences of the phrase. As shown in FIG. 12, the selection unit 13 selects "ransom" with 35 appearances and "prepare a ransom" with 30 appearances. Next, the selection unit 13 compares based on the size of each cluster. For example, the selection unit 13 sets the result of multiplication of the number of occurrences and the segment length, that is, the time length as the size of each cluster, and selects the cluster having the largest size of each cluster.

As shown in FIG. 12, the selection unit 13 compares, for example, a cluster having a cluster ID of 7 and a cluster having a cluster ID of 8. The selection unit 13 compares the result of multiplication of the number of appearances of 35 times with the time length of "ransom" and the result of multiplication of the number of appearances of 30 times with the time length of "prepare the ransom." Select the cluster whose content is "Prepare the ransom." That is, the phrase "prepare the ransom" is the desired phrase. Further, in the case of comparison between clusters having the same number of appearances, the selection unit 13 may select by comparing only the time lengths of the segments. The selection unit 13 is not limited to the above method, and the size may be defined and compared based on various indexes such as the number of appearances, the time length, and the number of phonemes.

Then, the extraction unit 14 extracts the segment from the selected cluster. As a result, the voice data, which is a segment whose content is "prepare the ransom", is extracted. From the voice data of this segment, it can be seen that the phrase "prepare the ransom" is frequently included in the voice data.

As described above, in the voice processing device 10 in this specific example, for example, the phrase "prepare the ransom" which is a frequent phrase from the voice data such as "... I took care of my child. Prepare the ransom. The meeting place is ...". It is possible to extract "white".

Although the present invention has been described above with reference to embodiments and specific examples, the present invention is not necessarily limited to the above embodiments and specific examples. Various changes and implementations of the present invention can be made and implemented within the scope of the present invention, which can be understood by those skilled in the art (within the scope of its technical ideas).

This application claims priority on the basis of Japanese application Japanese Patent Application No. 2015-061854 filed on March 25, 2015, and incorporates all of its disclosures herein.

10 Speech processing device 11 Generation section 12 Clustering section 13 Selection section 14 Extraction section 15 Normalization learning section 16 Voice data normalization section 17 Voice data processing section 18 Voice data classification section 20 Speech processing device 30 Speech processing device 40 Speech processing system 41 Voice processing device 42 Voice input device 43 Instruction input device 44 Output device 101 Voice data storage unit 102 Unspecified acoustic model storage unit 103 Parameter storage unit 1000 Computer 1001 CPU
1002 Main storage device 1003 Auxiliary storage device 1004 Interface 1005 Input device 1006 Display device

Claims (8)

A first generation means for generating a plurality of segments in which adjacent segments overlap at least partially from voice data,
A second generation means for classifying the plurality of segments based on phonological similarity to generate clusters related to the segments, and
Based on the size of the cluster according to the segment, and selecting means for selecting a cluster according to a predetermined condition is satisfied segments,
A voice processing device including an extraction means for extracting a segment included in a cluster related to the selected segment .
A third generation means for generating a plurality of normalization parameters for normalizing the difference in acoustic properties of the plurality of audio data based on the plurality of audio data, and
Further provided with a normalization means for normalizing the voice data by using the plurality of normalization parameters.
The first generation means generates the plurality of segments from the normalized voice data.
Further provided with a fourth generation means of classifying the voice data based on the difference in acoustic properties and generating clusters related to the voice data.
The third generation means generates a normalization parameter for the cluster related to the voice data.
Voice processing device . The voice processing device according to claim 1 , wherein the selection means compares and selects the clusters related to the segments by using the number of segments or the total time length included in the clusters related to the segments. The voice processing apparatus according to claim 1 or 2 , wherein the second generation means calculates the similarity between segments by comparing the acoustic features constituting the segment. The voice processing apparatus according to claim 1 , wherein the second generation means generates similarity by DP (Dynamic Programming) matching between the segments. Any of claims 1 to 4, wherein the fourth generation means and the third generation means are alternately and repeatedly executed until the results converge or the number of executions reaches a predetermined threshold value based on the mutual results . The voice processing device according to item 1 . From the audio data, generate multiple segments with at least some overlap of adjacent segments,
The plurality of segments are classified based on phonological similarity to generate clusters related to the segments .
Based on the size of the cluster in accordance with the segment, to select the cluster in accordance with a predetermined condition is satisfied segments,
A voice processing method for extracting segments included in a cluster related to the selected segment .
When the plurality of segments are generated, a plurality of normalization parameters for normalizing the difference in acoustic properties of the plurality of audio data are generated based on the plurality of audio data, and the plurality of normalization parameters are generated. To normalize the audio data and generate the plurality of segments from the normalized audio data.
When generating the plurality of normalization parameters, the voice data is classified based on the difference in acoustic properties to generate a cluster related to the voice data, and a normalization parameter is generated for the cluster related to the voice data. ,
Voice processing method . The process of generating multiple segments from audio data in which at least some of the adjacent segments overlap.
The process of classifying the plurality of segments based on phonological similarity to generate clusters related to the segments, and
Based on the size of the cluster according to the segment, the process of selecting one or more clusters according to a predetermined condition is satisfied segments,
A program that causes a computer to execute the process of extracting the segments included in the cluster related to the selected segment .
The process of generating the plurality of segments includes a process of generating a plurality of normalization parameters for normalizing the difference in acoustic properties of the plurality of audio data based on the plurality of audio data, and the process of generating the plurality of normalizations. It includes a process of normalizing the voice data using a normalization parameter and a process of generating the plurality of segments from the normalized voice data.
The process of generating the plurality of normalization parameters classifies the voice data based on the difference in acoustic properties to generate a cluster related to the voice data, and generates a normalization parameter for the cluster related to the voice data. To do,
Program . An instruction input device that receives the operator's instruction information,
A voice input device that inputs voice data to a voice processing device,
The voice processing device according to any one of claims 1 to 5 , which executes processing on the input voice data based on the instruction information.
An output device for outputting the processing result of the voice processing device is provided.
The output device is a voice processing system that outputs the processing result according to the instruction information.

Images