JP6090371B2 - Audio signal identification device and program - Google Patents

Description

  The present invention relates to a technology for identifying a consonant by analyzing an audio signal, and more particularly to a technology for identifying a consonant in the sound of broadcast content such as terrestrial digital broadcasting.

  In recent years, various broadcast contents such as movies, dramas, and sports broadcasts have been provided by terrestrial digital broadcasts and satellite digital broadcasts, and various audio devices that reproduce such broadcast contents with a high sense of presence, such as home theaters, have been proposed. ing. Among these types of audio equipment, in order to produce a sense of sound spread in the broadcast content, there is a device that reproduces the audio signal of the broadcast content by applying sound processing such as reverberation or equalizing. However, if the audio signal that is the target of audio processing represents a speech such as a movie dialogue, adding reverberation may impair clarity and nature, making it difficult to hear the dialogue. Such an event is conspicuous in consonants (particularly friction sounds such as “S” and “Sh”). Therefore, when performing audio processing such as adding reverberation or equalizing to an audio signal, the section corresponding to the sound waveform of the consonant in the audio signal (hereinafter referred to as consonant section) is identified, and audio processing is performed on the consonant section. Various techniques for avoiding this have been proposed (see Patent Documents 1 to 3).

  For example, Patent Document 1 discloses a technique for identifying a consonant section from band energy. Patent Document 2 discloses a technique for identifying consonants (friction sounds) from the number of zero crosses per unit time in an audio signal. Patent Document 3 narrows down speech section candidates using a discrimination coefficient obtained by weighted addition of a plurality of acoustic parameters (for example, autocorrelation coefficient, LPC cepstrum coefficient, LPC mel cepstrum coefficient, etc.) A technique for identifying a speech section, a stationary noise section corresponding to stationary noise, and a non-stationary noise section corresponding to sudden noise based on the time length of the candidate section is disclosed.

JP 2003-108172 A JP-A 62-65098 JP 2001-236085 A

  However, the techniques disclosed in Patent Documents 1 to 3 have a problem that it is difficult to apply to identification of consonant sections in broadcast content. For example, in a movie or the like, various sound effects are used for production, but some of these sound effects have an energy ratio equivalent to that of a consonant interval, and those having a zero cross number equivalent to that of a consonant interval. In the techniques disclosed in Document 1 and Patent Document 2, these sound effects may be mistakenly recognized as consonants. In the technique disclosed in Patent Document 3, complicated and high processing load signal processing (for example, Fourier transform to enable calculation of LPC cepstrum coefficients) is performed in order to increase resistance to various background noises such as stationary noise and non-stationary noise. Etc.). Since broadcast contents generally have a high S / N ratio, identification of consonant sections by the technique disclosed in Patent Document 3 will unnecessarily increase the processing load, and it has not been applied to home audio equipment. There is a problem that there is a lot of waste.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique that can reliably identify a consonant section in an audio signal by signal processing with a simple and light processing load.

  In order to solve the above problems, the present invention provides a buffer for accumulating a sample string representing a waveform of an audio signal, a read control unit for reading the sample string accumulated in the buffer into blocks of a predetermined number of samples, and the reading For each block read by the control unit, the energy of all frequency components of the audio signal in the block and the energy of frequency components belonging to a predetermined frequency band of the audio signal in the block are respectively calculated, and the ratio between the two is calculated. An energy ratio calculating means for calculating, a zero cross number counting means for counting the number of zero crosses per unit time for each block read by the read control unit, and the number of zero crosses among blocks sequentially read by the read control unit What is determined to be a sound effect from the increase / decrease mode of Whether or not based on the energy ratio of the determination target block, and the change state of the number of zero crosses in a predetermined number of blocks connected before or after the determination target block and the determination target block. There is provided an audio signal identification device comprising: determination means for determining whether a determination target block is included in a consonant section.

  Although details will be described later, in the case of a sound effect other than sound, the number of zero crosses per unit time is frequently increased and decreased as shown in FIG. For this reason, it can be determined whether it is a sound effect from the increase / decrease mode of the number of zero crosses, and the block determined to be a sound effect can be excluded from the determination target whether it is a consonant. Also, calculation of the energy ratio in each block and counting of the number of zero crosses per unit time are simple processes compared to Fourier transform. Therefore, according to the audio signal identification device, it is possible to identify a consonant section in an audio signal to be processed by audio signal processing such as reverberation or equalization by simple processing. Further, although details will be described later, in the audio signal identification device of the present invention, it is determined whether or not the block is included in the consonant section by using both the energy ratio and the variation of the number of zero crosses per unit time. Therefore, it is possible to prevent a sound effect having an energy ratio equivalent to that of the consonant section from being erroneously recognized as a consonant and to reliably identify the consonant section. That is, according to the audio signal identification device, it is possible to reliably identify a consonant section in an audio signal including a sound effect in addition to voice by simple and light signal processing.

  In a more preferable aspect, the block is determined to be a consonant section from a change aspect of the number of zero crosses per unit time in a block determined by the determination unit and included in a consonant section and a predetermined number of blocks connected before or after the block. It is further characterized by further comprising detailed determination means for determining whether it belongs to the rising section, belongs to the steady section of the consonant section, or belongs to the falling section of the consonant section. In recent studies, it has been reported that speech concealment is improved by performing speech speed conversion so as to extend both (or one) of the rising and stationary sections of the consonant sections. Therefore, according to the above aspect, it is possible to identify in detail a section in which speech speed conversion for improving the ease of listening to speech is to be performed.

It is a figure which shows the structural example of the consonant identification apparatus 1 of one Embodiment of this invention. It is a figure which shows an example of the amplitude spectrum of a vowel, and the amplitude spectrum of a consonant. It is a figure which shows an example of the time change of the number of zero crosses per unit time of a sound, and the time change of the number of zero crosses per unit time of sounds other than sound, such as a sound effect.

Embodiments of the present invention will be described below with reference to the drawings.
(A: Embodiment)
FIG. 1 is a diagram illustrating a configuration example of a consonant identification device 1 according to an embodiment of the audio signal identification device of the present invention.
The consonant identification device 1 is incorporated in an audio device that reproduces broadcast content, such as a home theater for home, for example, analyzes an audio signal of the broadcast content reproduced by the audio device, and determines a consonant section corresponding to the consonant. It is for identification. Although not shown in detail in FIG. 1, the audio device has an audio signal processing unit that performs audio signal processing for producing a sense of spread of sound. This audio signal processing unit is provided by a consonant identification device 1. The audio signal processing is performed only in the section determined to be other than the consonant section. As a result, it is possible to produce a sense of sound spread without impairing the clarity of the dialogue or the like.

  As shown in FIG. 1, the consonant identification device 1 includes a buffer 10, a read control unit 20, and a consonant segment identification processing unit 30. The buffer 10 is a ring buffer secured as a storage area in a volatile memory such as a RAM (Random Access Memory). In this buffer 10, a sample sequence of the audio signal of the broadcast content is written. If the audio signal is an analog signal, it may be written in the buffer 10 after A / D conversion by an A / D converter is performed and converted into a sample string. The read control unit 20 reads the sample sequence stored in the buffer 10 in order from the oldest one, dividing the sample sequence into blocks of the number of samples corresponding to a predetermined time such as 5 ms, and gives the sample sequence to the consonant segment identification processing unit 30.

The consonant section identification processing unit 30 includes, for example, a CPU (Central Processing Unit), a volatile memory such as a RAM, and a ROM (Read
Non-volatile memory such as “Only Memory” (not shown). This non-volatile memory stores in advance a consonant segment identification processing program that causes the CPU to execute a consonant segment identification process that significantly shows the features of the present embodiment. The volatile memory of the consonant segment identification processing unit 30 is used by the CPU as a work area when executing the consonant segment identification processing program. The CPU that operates according to the consonant segment identification processing program executes three processes: an energy ratio calculation process 310, a zero-cross number counting process 320, and a determination process 330. That is, the consonant segment identification process of the present embodiment includes three processes: an energy ratio calculation process 310, a zero cross number count process 320, and a determination process 330.

  The energy ratio calculation processing 310 calculates the energy of all frequency components and the energy of frequency components belonging to a predetermined frequency band (frequency band of 2 kHz to 7 kHz in the present embodiment) for each block read by the read control unit 20. This is a process of calculating each and further calculating a ratio between the two (the value obtained by dividing the latter by the former: hereinafter, energy ratio ER). As shown in FIG. 1, the energy ratio calculation process 310 includes energy calculation processes 312 a and 312 b, a bandpass filter process 314, and a multiplication process 316. The energy calculation process 312a is a process for calculating the sum of squares of the sample values of the samples included in one block as the energy of all frequency components in the block. The band-pass filter process 314 is a process that passes only the frequency components belonging to the predetermined frequency band in the sample sequence constituting the block. The energy calculation process 312b is a process for calculating the sum of squares of the sample values of the samples constituting the sample sequence that has undergone the bandpass filter process 314 as the energy of the frequency component belonging to the frequency band. The multiplication process 316 is a process of calculating the energy ratio ER by multiplying the value calculated by the energy calculation process 312b by the reciprocal of the value calculated by the energy calculation process 312a.

  The zero-cross number counting process 320 is a process of counting the zero-cross number NUM per unit time for each block read by the read control unit 20. A specific algorithm for counting the number of zero crosses may be appropriately used. Then, the determination process 330 sets each block sequentially read by the read control unit 20 as a determination target, the energy ratio ER calculated by the energy ratio calculation process 310 for the determination target block, and the determination target block and the previous or Each of a predetermined number of blocks connected in the back belongs to a consonant section based on a mode of time change between these blocks of the number of zero crosses per unit time counted by the zero cross number counting process 320 (hereinafter referred to as a change mode). In this process, it is determined whether or not the block is a block, and a determination result signal SS indicating the determination result is output to a subsequent audio signal processing unit. The reason why it is possible to determine whether or not it is a consonant section based on the energy ratio ER in each block and the change mode of the number of zero crosses per unit time is as follows.

  FIG. 2 is a diagram illustrating an example of an amplitude spectrum of a vowel and an amplitude spectrum of a consonant. In FIG. 2, the vowel amplitude spectrum is drawn as a solid line graph, and the consonant amplitude spectrum is drawn as a one-dot chain line graph. As shown in FIG. 2, the amplitude spectrum of the consonant is localized in the predetermined frequency band. For this reason, in the case of a block belonging to a consonant section, the energy values calculated in each of the energy calculation processes 312a and 312b are substantially the same value, and the energy ratio ER calculated in the multiplication process 316 is 1 (or close to 1). Value). On the other hand, the amplitude spectrum of the vowel section is distributed in a frequency band lower than the lower limit of the predetermined frequency band as shown in FIG. For this reason, in the case of a block belonging to a vowel section, the value of energy calculated in the energy calculation process 312b is zero or a value close to zero, and the energy ratio ER calculated in the multiplication process 316 is a value significantly lower than 1. Therefore, it is possible to determine whether each block belongs to a consonant section by referring to the energy ratio ER calculated by the energy ratio calculation processing 310 for each block.

  FIG. 3 is a diagram illustrating an example of a time change of the number of zero crosses per unit time of sound and a time change of the number of zero crosses per unit time of sounds other than sound such as sound effects. More specifically, FIG. 3 (a) is a diagram showing an example of a time change of the number of zero crosses per unit time of a voice composed of a consonant and a vowel following the consonant, and FIG. It is a figure which shows an example of the time change of the number of zero crosses per unit time of sounds other than a sound, such as a sound effect. In the case of speech, it is generally known that zero crosses occur periodically in the vowel section and zero crosses occur non-periodically in the consonant section. Since zero crosses occur periodically in the vowel section, the number of zero crosses per unit time is substantially constant as shown in FIG. On the other hand, when the consonant section is analyzed in more detail, a section in which the number of zero crosses per unit time monotonously increases (section A in FIG. 3A) and a section in which a substantially constant value is maintained (section B in FIG. 3A). ) And a section monotonously decreasing from the constant value (section C in FIG. 3A) is known to be classified into three sections. 3 corresponds to a consonant rising section (for example, a rising section from a silent state), and section B in FIG. 3A corresponds to a consonant steady section, and FIG. It is known that section C corresponds to a falling section of a consonant (a transition section from a consonant to a subsequent vowel). That is, the number of zero crosses per unit time is maintained at a substantially constant value over the entire vowel section in the case of a vowel section, and is substantially constant after monotonically increasing from 0 to a predetermined value in the case of a consonant section ( It becomes a higher value than that in the vowel interval), and then changes with time, such as decreasing monotonously.

  On the other hand, in the case of sound effects other than voice, the number of zero crosses per unit time is frequently increased and decreased as shown in FIG. Therefore, depending on whether or not the number of zero crosses NUM per unit time in the determination target block and a predetermined number of blocks connected before or after it repeats increasing or decreasing, whether the determination target block is a block corresponding to sound, or It is possible to determine whether the block corresponds to a sound effect other than voice, and if it is determined to correspond to voice, the number of zero crosses NUM per unit time is substantially constant and a predetermined threshold (vowel interval) The block belongs to a vowel section depending on whether or not a value smaller than the value in the steady section of the consonant and a value less than, for example, th) in FIG. 3A is maintained. Or whether it belongs to a consonant section.

  In the determination process 330 of this embodiment, (1) the number of zero crosses NUM per unit time monotonously increases or decreases in a block adjacent to the determination target block and the front or back thereof, or exceeds a predetermined threshold value. When the two conditions of maintaining a constant value and (2) the energy ratio ER being 1 or a value close to 1 are satisfied, the determination target block is determined to be a block belonging to a consonant section. As described above, whether or not the block belongs to the consonant section based on the energy ratio ER of the determination target block and the change mode of the number of zero crosses NUM per unit time in the determination target block and the block adjacent to the determination target block. When the determination is made, a ring buffer for storing the energy ratio ER calculated by the energy ratio calculation process ER and the zero cross number NUM counted by the zero cross number count process 320 for each block for 3 blocks is secured in the RAM. Then, it may be determined whether or not the above conditions (1) and (2) are satisfied based on the stored contents of the ring buffer.

  The point to be noted here is that it seems that it can be determined whether or not the block to be determined belongs to the consonant section only by either the energy ratio ER or the change mode of the number of zero crosses NUM per unit time. In the determination processing 330 of the present embodiment, the determination based on the energy ratio ER and the determination based on the change mode of the number of zero crosses NUM per unit time are used in combination. The reason why both are used together is as follows.

  For example, among various sound effects reproduced together with dialogue in movies, etc., the energy ratio in each block is close to 1, or the majority of frequency components belong to a frequency band different from the predetermined frequency band per unit time. The number of zero crosses varies in the same way as the consonant interval. The determination based on the energy ratio ER alone cannot identify the former sound effect and the consonant, and the determination based only on the time change of the number of zero crosses per unit time cannot identify the latter sound effect and the consonant. However, by using both in combination, these sound effects and consonants can be reliably identified. For example, even if the energy ratio in each block is a sound effect close to 1, if the number of zero crosses NUM per unit time changes as shown in FIG. 3B, the above condition (1) is not satisfied. In the determination process 330 of this embodiment, it is determined that these blocks do not belong to the consonant section. Similarly, even if the number of zero crosses NUM per unit time in the determination target block and blocks preceding or following the block is changed as shown in FIG. 3A, the energy ratio ER in each block is 1. If the sound effect is significantly lower than the above, the condition (2) is not satisfied, and therefore, in the determination process 330 of the present embodiment, it is determined that these blocks do not belong to the consonant section.

  As described above, according to the present embodiment, it is possible to distinguish between a sound effect and a consonant that are erroneously recognized as a consonant by only one of the determination by the energy ratio ER and the determination by the change mode of the number of zero crosses NUM per unit time. It is possible to reliably perform this. This is the reason why the determination by the energy ratio ER and the determination by the change mode of the number of zero crosses NUM per unit time are used together. In addition, the calculation of the energy ratio ER for each block and the calculation of the number of zero crosses NUM per unit time are processes with a lighter processing load than the Fourier transform. That is, according to the consonant identification device 1 of the present embodiment, it is possible to reliably identify a consonant section in an audio signal that is a target of audio signal processing such as reverberation or equalization by simple signal processing. .

(B: Modification)
Although the embodiment of the present invention has been described above, it goes without saying that the following modifications may be added to this embodiment.
(1) In the embodiment described above, is it a consonant interval based on the energy ratio ER in the determination target block and the change mode of the number of zero crosses per unit time in the determination target block and the block adjacent to the front or back of the determination target block? A determination of whether or not was made. However, a detailed determination process for determining whether the determination target block belongs to a rising section, a stationary section, or a falling section of a consonant section may be executed based on a change mode of the number of zero crosses per unit time. Specifically, it is determined that a block in which the number of zero crosses per unit time has increased by a predetermined threshold or more compared to the block immediately before (that is, the preceding block in time) belongs to the rising section, A block in which the increase / decrease width of the number of zero crosses with respect to the previous block is less than a predetermined threshold value is determined to belong to the stationary section, and a block with a decrease width of the number of zero crosses with respect to the previous block is equal to or greater than a predetermined threshold value The detailed determination process for determining that it belongs to the falling section is further executed by the consonant section identification processing unit 30. According to such an aspect, it is determined whether the block determined to belong to the consonant section belongs to the rising section, the stationary section, or the falling section, and the determination result signal indicating the determination result is processed in the subsequent audio signal processing. It is possible to cause the audio signal processing apparatus to execute the audio signal processing according to the determination result given to the apparatus.

  For example, when the speech signal processing device is a device that improves the ease of listening to speech through speech speed conversion, in recent research, the speech speed is extended so as to extend the rising interval or the steady interval (or both) of the consonant. Reports have been made that the conversion improves the ease of listening. Therefore, when the audio signal processing device in the subsequent stage of the audio signal identification device of the present invention is a device that improves the ease of listening to the voice by speech speed conversion, the audio signal identification device is caused to execute the detailed determination process. Accordingly, it is possible to appropriately determine a section where speech speed conversion is to be performed, and to notify the determination result to the audio signal processing device.

(2) In the above-described embodiment, the CPU of the consonant segment identification processing unit 30 has a consonant segment identification process (that is, the energy ratio calculation process 310, the zero cross number count process 320, and the determination process 330) that significantly shows the features of the embodiment. The consonant segment identification processing program for executing the process is stored in the nonvolatile memory of the consonant segment identification processing unit 30 in advance. However, the program may be written and distributed on a computer-readable recording medium such as a CD-ROM, or the program may be distributed by downloading via the Internet.

  In the above-described embodiment, the energy ratio calculation process 310, the zero cross number count process 320, and the determination process 330 are realized by software, but may be realized by hardware. Specifically, each of the energy ratio calculation means for executing the energy ratio calculation process 310, the zero cross number count means for executing the zero cross number count process 320, and the determination means for executing the determination process 330 is constituted by an electronic circuit. Of course, the consonant section identification processing unit 30 may be configured by combining the means. Further, in the aspect in which the detailed determination process is further executed, each means of the energy ratio calculation means, the zero cross number counting means, the determination means, and the detailed determination means for executing the detailed determination process is configured by an electronic circuit. What is necessary is just to comprise the consonant area identification process part 30 combining each means.

  DESCRIPTION OF SYMBOLS 1 ... Consonant identification apparatus, 10 ... Buffer, 20 ... Reading control part, 30 ... Consonant area identification process part, 310 ... Energy ratio calculation process, 312a, 312b ... Energy calculation process, 314 ... Band pass filter process, 316 ... Multiplication process 320 ... Zero cross count processing 330 ... Determination processing.

Claims (2)

A buffer for storing a sample sequence representing the waveform of an audio signal;
A read control unit for reading the sample sequence accumulated in the buffer into blocks of a predetermined number of samples; and
For each block read by the read control unit, the energy of all the frequency components of the audio signal in the block and the energy of the frequency components belonging to a predetermined frequency band of the audio signal in the block are calculated respectively. Energy ratio calculating means for calculating the ratio;
A zero cross number counting means for counting the number of zero crosses per unit time for each block read by the read control unit;
Among the blocks sequentially read out by the read control unit, those determined as sound effects from the increase / decrease mode of the number of zero crosses are excluded from the determination target whether or not they are consonants, the energy ratio for the determination target block, and Determining means for determining whether or not the determination target block is included in a consonant section based on a change aspect of the number of zero crosses in the determination target block and a predetermined number of blocks connected before or after the determination target block and,
Whether the block belongs to the rising section of the consonant section from the aspect of change of the number of zero crosses per unit time in the block determined by the determining means and included in the consonant section and a predetermined number of blocks connected before or after the block Detailed determination means for determining whether it belongs to a stationary section of a consonant section or a falling section of a consonant section;
An audio signal identification device comprising:   On the computer,
  For each of the blocks obtained by dividing the sample sequence of the audio signal input to the computer by a predetermined number of samples, the energy of all frequency components and the energy of frequency components belonging to a predetermined frequency band are respectively calculated, and the ratio between the two is calculated. Energy ratio calculation processing for calculating
  For each of the blocks, a zero cross number counting process for counting the number of zero crosses per unit time;
  Among the blocks, those determined as sound effects from the increase / decrease mode of the number of zero crosses are excluded from the determination target whether or not they are consonants, the energy ratio for the determination target block, and the determination target block and the determination A determination process for determining whether or not the determination target block is included in a consonant section based on a change aspect of the number of zero crosses in a predetermined number of blocks continuous before or after the target block;
  The block belongs to the rising section of the consonant section from the aspect of change in the number of zero crosses per unit time in the block determined in the determination process and the predetermined number of blocks connected before or after the block when included in the consonant section. A detailed determination process for determining whether it belongs to a stationary section of a consonant section or a falling section of a consonant section;
  A program characterized by having executed.

Images