JP5644934B2 - Signal feature extraction apparatus and signal feature extraction method - Google Patents

Description

  The present invention relates to a signal feature extraction device and a signal feature extraction method, and more particularly to a signal feature extraction device and a signal feature extraction method for extracting features from speech, acoustic signals or other time series signals.

  Conventionally, various techniques for recognizing and identifying a sound source from voice or other acoustic signals have been proposed for voice recognition, machine failure diagnosis, and the like. Patent Document 1 below discloses an example of such a sound source identification method. In this sound source identification method, a frequency spectrum is obtained by performing a fast Fourier transform on an acoustic signal from a mechanical device, and then the frequency spectrum is subjected to filtering processing by a fuzzy rule to extract a characteristic portion of the frequency spectrum. .

  Next, the extracted spectrum data is input to the hierarchical neural network, and the calculation data calculated based on the spectrum data is compared with the judgment data stored in advance in the hierarchical neural network. When the comparison data is judged by a predetermined evaluation function to identify the sound source, the judgment data is switched according to the input spectrum data.

JP-A-8-44695

The above-described conventional signal feature extraction method has the following problems.
(1) The phase information of each frequency component included in the signal has not been extracted, and the identification accuracy of the signal characterized by the phase was low.
(2) Linear features are extracted from each frequency component, and few feature quantities that focus on the relationship between frequencies have been proposed.

  An object of the present invention is to provide a signal feature extraction apparatus and a signal feature extraction method for solving features of the prior art as described above and extracting features with high accuracy from speech, acoustic signals or other time series signals. It is in.

  The signal feature extraction apparatus of the present invention extracts a complex Fourier transform means for converting a digital input signal sampled for a predetermined period into a frequency axis, and extracts higher-order local correlation feature data from the data transformed by the complex Fourier transform means. The main feature is the provision of a feature extraction means.

  In the signal feature extraction apparatus described above, the higher-order local correlation feature focuses on one of a large number of data arranged two-dimensionally along the time axis and the frequency axis, and is determined in advance as the attention data. Another feature is that a correlation value with neighboring data determined by the mask pattern is calculated.

  In the signal feature extraction apparatus described above, the higher-order local correlation feature is characterized in that it includes a vector conversion unit that converts phase information of complex data input from the Fourier transform unit into a vector representation.

  In the signal feature extraction apparatus described above, a filter bank composed of a plurality of bandpass filters can be applied, and a plurality of bandpass filter means for multiplying and adding the input data to the weighted data and outputting the data, There is also a feature in that it is arranged between the Fourier transform means and the feature extraction means or after the feature extraction means.

  In the signal feature extraction apparatus described above, the filter means refers to the input signal, and the distribution smoothing in which the bandwidth of each bandpass filter is determined so that the frequency distribution of the time average value of the amplitude is uniform. Another characteristic is that it is a filter.

  Further, the signal feature extraction device described above is characterized in that logarithmic conversion means for amplitude information is arranged after the feature extraction means or after the filter means.

  The signal feature extraction method of the present invention includes a step of performing a complex Fourier transform process for converting a digital input signal sampled for a predetermined period into a frequency axis, and higher-order local correlation feature data from the data transformed in the complex Fourier transform process. And a step of performing a feature extraction process for extracting.

The signal feature extraction apparatus and signal feature extraction method of the present invention have the following effects.
(1) Features can be extracted with high accuracy from speech, acoustic signals, or other time-series signals, and the identification accuracy of signals having characteristics in phase is improved.
(2) The high-order local autocorrelation can effectively extract the relationship between frequencies, and the identification accuracy is improved.

FIG. 1 is a block diagram showing a hardware configuration of a signal feature extraction apparatus according to the present invention. FIG. 2 is a flowchart showing the contents of signal recognition processing using the signal feature extraction method of the present invention. FIG. 3 is an explanatory diagram showing the contents of the Fourier transform process. FIG. 4 is an explanatory diagram showing the contents of the mask pattern used in the feature extraction processing of the present invention. FIG. 5 is an explanatory diagram showing a quantization method (1) in the phase index HLAC of the present invention. FIG. 6 is an explanatory diagram showing the quantization method (2) in the phase index HLAC of the present invention. FIG. 7 is an explanatory diagram showing a characteristic example of the Mel filter.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, although the example which inputs a sound with a microphone is disclosed in the Example, this invention is applicable to the arbitrary electric signals which can be input into a computer.

  FIG. 1 is a block diagram showing a hardware configuration of a signal feature extraction apparatus according to the present invention. The microphone 10 converts, for example, audible sound generated from an object into an electric signal and outputs it to the computer 11. The computer 11 may be, for example, a known personal computer (PC) provided with an interface circuit (microphone input circuit: sampling, A / D conversion circuit) for capturing a sound signal. The present invention is realized by creating and installing a program for executing processing to be described later on any known computer 11 such as a personal computer.

  The monitor device 12 is a well-known output device of the computer 11 and is used, for example, to display a recognition result or the like of the type of sound emitted by the object to the operator. The keyboard 13 and the mouse 14 are well-known input devices used for input by the operator.

  FIG. 2A is a flowchart showing the contents of the signal recognition process (1) using the signal feature extraction method of the present invention. In S10, the analog signal input from the microphone 10 is sampled at a predetermined cycle, and A / D converted digital signal data is subjected to a known complex Fourier transform process using a moving time window.

  FIG. 3 is an explanatory diagram showing the contents of the Fourier transform process. The analog signal input from the microphone 10 is sampled at a predetermined period (for example, 50 μsec (sampling frequency 20 kHz)), A / D converted, and temporarily stored. This digital signal data is cut out using a time window of a predetermined length (for example, several seconds) and converted into a plurality of discrete complex values F on the frequency axis shown in the following equation 1 by a known short-time complex Fourier transform process. Is done. A represents the amplitude of the frequency component, and θ represents the phase.

  The time window is moved every one to a plurality of sampling periods, and Fourier transform processing is performed for a predetermined period (for example, several tens of seconds). As a result, a large number of complex data F arranged along the time axis and the frequency axis as shown on the right side of FIG. 3 is obtained.

  In S11, one feature data of (1) Fourier HLAC (Fourier HLAC, hereinafter referred to as FHLAC) or (2) Phase index HLAC (hereinafter referred to as PHLAC) is calculated as feature extraction processing. To do.

(1) FHLAC:
FHLAC is a Fourier higher order local autocorrelation feature invented by the inventors. First, pay attention to one of a large number of complex data F arranged along the time axis and the frequency axis shown on the right side of FIG. 3, and the neighborhood determined by the complex data of interest and a predetermined mask pattern. The correlation value with the complex number data is calculated. By executing this process for all mask patterns, feature data consisting of a set of a plurality of complex data X equal to the number of mask patterns is obtained for one target complex data.

  FIG. 4 is an explanatory diagram showing the contents of the mask pattern used in the feature extraction processing of the present invention. In determining the mask pattern, it is limited to the inside of a 3 × 3 square centering on the attention data, and the attention data at the center is always selected at least once. In addition, mask patterns that overlap by moving in parallel in the up / down / left / right and diagonal directions overlap if the data of interest is moved.

As a result of this de-duplication, there are four types of primary mask patterns (1) to (4) for selecting the center and another one point, and a secondary mask pattern for selecting the center and the other two points (1). 20 types of (20) remain. In addition, there are one type of primary mask pattern (5) for selecting the center twice, eight types of secondary mask patterns (21) to (28) for selecting the center twice and another point, and the center. There is one type of secondary mask pattern (29) selected three times.
Correlation values between complex number data at positions with black circles ● in the mask pattern are calculated. If there are two or more black circles ● at the same position, consider that there are two or more values and correlate like the others (multiply yourself twice or more). Note that only one of the primary and secondary masks may be used for extracting feature data, or both the primary and secondary masks may be used.

Equations 2 and 3 below show arithmetic expressions for primary and secondary correlations using primary and secondary mask patterns. In the calculation, a complex conjugate that reverses the sign of the imaginary number of one complex number is taken, and the correlation value (represented by a bar above F) is also a complex number. The phase information of the primary correlation value indicates the primary differential information of the phase of the frequency component, and the phase information of the secondary correlation value indicates the secondary differential information of the phase of the frequency component.
In the case of a mask pattern only for itself, such as the patterns (5) and (29) in FIG. 4, a real-valued feature is obtained, which matches the conventional power spectrum feature.

  In the above formulas 2 and 3, a plurality of amplitudes A are multiplied. However, in addition to multiplication, the calculation may be * (inner product), min (A, B) (the smaller of A and B), etc. Good. Therefore, if the calculation is an arbitrary function f, it can be expressed as the following formulas 4 and 5.

  The FHLAC is a set of complex data X obtained by scanning the complex data of interest in the frequency axis and time axis directions and performing the above processing on all the complex data F.

(2) PHLAC:
PHLAC is also a Fourier higher order local autocorrelation feature invented by the inventors. Attention is paid to one of a large number of complex data F arranged two-dimensionally along the time axis and the frequency axis shown on the right side of FIG. 3, and the vicinity of the neighborhood determined by the complex data of interest and a predetermined mask pattern. A correlation value with complex number data is calculated. At this time, the phase information is converted into a quantized expression (vector expression) for each complex number data, and then correlation is obtained.
In PHLAC, the correlation (mask patterns (5), (21) to (29) in FIG. 4) for selecting itself twice or more is not taken.

  FIG. 5 is an explanatory diagram showing a quantization method (1) in PHLAC (phase index HLAC) of the present invention. When the complex number F of Equation 1 is expressed on the complex plane, it can be expressed as shown on the left side of FIG. Here, the phase θ is determined by a weighted sum of two reference directions sandwiching a complex number F among a plurality (eight in FIG. 5) of reference directions (1 to 8) that are different in direction from each other by an equal angle. Can be represented.

  For example, in the case of the complex number F shown in FIG. 5, since θ is exactly in the middle of the vectors 2 and 3, the weights of the vectors 2 and 3 are 0.5 and the other weights are 0, respectively. Can be represented. Therefore, assuming that these eight weight values are an eight-dimensional vector h, the complex number F is expressed by the following Equation 6.

  Here, using the expression of Formula 6, the primary and secondary correlation calculation formulas using the primary and secondary mask patterns are shown in Formulas 7 and 8 below. The function f is the same as described above.

  The calculation is a vector outer product in Equation 7, and a tensor product in Equation 8. The primary correlation feature corresponding to one mask pattern of Formula 7 is a vector composed of 8 × 8 real values, and the secondary correlation feature corresponding to one mask pattern of Formula 8 is 8 × 8 × 8 real values. A vector consisting of

  Next, a modified example of PHLAC will be described. In the above-described PHLAC, the example in which the phase θ is converted into the quantized representation by the 8-dimensional vector h has been disclosed. However, the phase difference information of the complex number data X shown in Equation 4 or 5 is converted into the quantized representation using the vector h. It is also possible to do.

  In the following Expression 9, the phase difference information of Expression 4 is converted into a quantized expression by an 8-dimensional vector h. In Expression 10 below, the phase information of Expression 5 is transformed into the sum of two pieces of phase difference information, and converted into a quantized expression by the product of two 8-dimensional vectors h. In Equations 9 and 10, the dimensions are lower than those in Equations 4 and 5, and the data amount is reduced.

  Next, another modification of PHLAC will be described. In the above-described PHLAC, an example in which the phase information θ is converted into a quantized expression has been disclosed. Use.

  FIG. 6 is an explanatory diagram showing a quantization method when a group delay or an instantaneous frequency is used instead of phase information in the phase index HLAC of the present invention. Since the group delay or instantaneous frequency is a value having no periodicity, the minimum value and the maximum value are divided into a plurality of sections and quantized.

  The group delay or instantaneous frequency value (θ hat) is two reference values sandwiching the group delay or instantaneous frequency value (θ hat) among a plurality (eight in FIG. 6) of reference values (1 to 8). Can be represented by a weighted sum of Therefore, these eight weight values are set as the above-described 8-dimensional vector h. The following is the same as the processing described above. The method of dividing the minimum value, maximum value, and section may be determined from learning data (input signal data), or may be given as a parameter in advance.

  In S12, either (1) distribution smoothing filter processing or (2) Mel filter processing is performed as filter processing. In the filter processing, the calculation shown in the following Expression 11 is performed using a plurality of bandpass filter functions. Each bandpass filter multiplies the input data by the weight and adds (accumulates and adds) and outputs the result. W is the weight of the filter. As a result, the number of data in the frequency direction of the feature data X is reduced to the number of filters.

Mel filter processing:
FIG. 7 is an explanatory diagram showing a characteristic example of the Mel filter. A known Mel filter is a filter bank composed of a plurality of bandpass filters. The characteristic of each filter has a triangular shape as shown in FIG. 7, and the integrated value of the weight of each filter is the same.

  The intervals between the center frequencies of the filters become wider as the frequency becomes higher. When the frequency axis is expressed in a logarithmic scale, the center frequencies of the filters of the Mel filter are arranged at equal intervals. Also, the bandwidth of each filter is the same.

Distribution smoothing filter processing:
In the above-described Mel filter, the center frequencies of the bandpass filters are arranged at regular intervals on a logarithmic scale, but this is not always the optimal arrangement depending on the signal. Therefore, the present inventor has invented the following distribution smoothing filter. In this distributed smoothing filter, a filter bank composed of a plurality of band pass filters is used in the same manner as the Mel filter.

  The bandwidth of each bandpass filter of the distributed smoothing filter is determined as follows according to the characteristics of the input signal. First, H and q are obtained from the amplitude A of the complex number data F by the following Expression 12. H is a histogram value of frequency k (= time average value of amplitude), and q is a cumulative distribution function of H on the frequency axis. Further, g is an arbitrary function, and may be a logarithmic (log) function or a step function having a threshold value.

  This q is a function that monotonously increases from 0 to 1 as the frequency k increases. When the frequency axis is converted by q, the band with small amplitude becomes narrow and the band with large amplitude becomes wide. Therefore, when the converted frequency axis is equally divided, the added value of the amplitude in each section is the same (uniform distribution) A new frequency axis can be obtained. A filter group is formed in which the weights are even on the new frequency axis, that is, the intervals between the center frequencies of the bandpass filters are equal, and the filter characteristics have the same shape.

Specifically, for example, the frequency of a point that equally divides the value of q (for example, 0, 0.1, 0.2,... Is the boundary frequency or center frequency of each bandpass filter.
If the filter is configured in this manner, the bandwidth of the band-pass filter becomes narrow in the band having a large amplitude in accordance with the characteristics of the input signal, and the feature can be extracted finely. Therefore, recognition and identification accuracy are improved.

  In S13, logarithmic conversion is performed on the amplitude A as necessary. By this processing, for example, the influence of large noise can be suppressed.

In S14, known recognition and identification processing is performed based on the extracted feature data. For example, if xi is a feature vector group for learning obtained using a time window, learning data is obtained by performing principal component analysis (in the case of Fourier HLAC, a complex eigenvalue problem) on xi. A space V spanned by (normal) feature vectors frequently included in is obtained. Then, when the input feature vector is x, d ² is obtained by the following formula 13, and it is determined whether the noise is abnormal, that is, whether it is a failure or not by the magnitude of this value.

  FIG. 2B is a flowchart showing the contents of a second embodiment of the signal recognition process using the signal feature extraction method of the present invention. In the first embodiment described above, the example in which the filtering process in S12 is performed after the feature extraction process in S11 has been disclosed. However, in the second embodiment, the execution order of S11 and S12 in the first embodiment is switched to S16 (= S12 ), S17 (= S11). Other processes are the same as those in the first embodiment. In the second embodiment, the amount of data is reduced by the filter processing, so that the load of the feature extraction processing is reduced and the overall processing speed is improved.

  Although the embodiments have been described above, the following modifications can be considered for the apparatus of the present invention. In the embodiment, an example is disclosed in which after A / D conversion is temporarily stored and offline processing is performed, but processing may be performed in real time if the processing speed is in time.

  The present invention can be applied to the detection of abnormal sounds such as speech and other acoustic signals, as well as recognition and identification of arbitrary signals that can be input to a computer, and machine failure.

DESCRIPTION OF SYMBOLS 10 ... Microphone 11 ... Computer 12 ... Monitor apparatus 13 ... Keyboard 14 ... Mouse

Claims (7)

A complex Fourier transform means for transforming a digital input signal sampled for a predetermined period into a large number of complex data arranged two-dimensionally along a time axis and a frequency axis;
Focusing on one of the complex number data, a complex conjugate that inverts the sign of the imaginary number of the complex number of one data is taken between the target data and neighboring data determined by a predetermined mask pattern. The correlation value is calculated by calculating the product, and the calculation is performed for all the complex number data along the time axis and the frequency axis for each mask pattern, and the set of correlation values is determined for each mask pattern. A signal feature extraction apparatus comprising: feature extraction means for extracting Fourier higher-order local correlation feature data using When A is amplitude information and θ is phase information, when the complex data is expressed by the following Equation 1, the primary correlation value among the correlation values is expressed by the following Equation 2. The feature extraction device according to claim 1.

When A is amplitude information and θ is phase information, when the complex data is represented by the following Equation 1, the secondary correlation value among the correlation values is represented by the following Equation 3. The feature extraction device according to claim 1.

  The filter bank is composed of a plurality of bandpass filters, each bandpass filter multiplies the input data by multiplying the weights, and outputs the filter means between the complex Fourier transform means and the feature extraction means, Alternatively, the feature extraction apparatus according to claim 1, wherein the feature extraction device is disposed after the feature extraction unit.   The filter means is a distribution smoothing filter in which a bandwidth of each bandpass filter is determined with reference to an input signal so that a frequency distribution of time average values of amplitudes is uniform. Item 5. The feature extraction device according to Item 4. 5. The feature extraction apparatus according to claim 4 , wherein the filter means is disposed after the feature extraction means , and further , logarithmic conversion means for amplitude information is disposed after the filter means . Performing a complex Fourier transform process for converting a digital input signal sampled only for a predetermined period into a large number of complex data arranged two-dimensionally along a time axis and a frequency axis;
Focusing on one of the complex number data, a complex conjugate that inverts the sign of the imaginary number of the complex number of one data is taken between the target data and neighboring data determined by a predetermined mask pattern. The correlation value is calculated by calculating the product, and the calculation is performed for all the complex number data along the time axis and the frequency axis for each mask pattern, and the set of correlation values is determined for each mask pattern. And a step of performing feature extraction processing for extracting Fourier higher-order local correlation feature data.

Images