JP2570395B2 - Sound recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial applications]

The present invention relates to a sound recognition device that recognizes a special sound from ambient sounds. In particular, it is used in a device that is mounted on a car or the like and recognizes an alarm sound of an emergency car or the like from ambient sounds outside the vehicle compartment and notifies the driver of the alarm sound.

[Prior art]

2. Description of the Related Art Conventionally, the following is known as an acoustic recognition device that is mounted on a vehicle and recognizes an alarm sound of an emergency vehicle, a level crossing alarm, or the like from an ambient sound outside the vehicle compartment and notifies the driving vehicle of the alarm sound. The first is an apparatus that determines whether or not the spectrum of the detected sound includes a reference frequency that is a feature of the alarm sound. Specifically, a band-pass filter having a center frequency of a reference frequency of 2 to 3 times, which is a feature of the alarm sound to be recognized,
Two or three characteristic frequency components are extracted from the acoustic signal, and if the magnitude of the component exceeds a predetermined threshold value, it is assumed that an alarm sound is present in the detected sound. (JP-A-58-221500). The second is to measure the period of the detected sound, that is, the pitch, check the ratio of each pitch with the pitch of a predetermined alarm sound, or recognize the alarm sound of an emergency vehicle using the pitch stability. (JP-A-60-219521). Third, a plurality of peak frequency components of the alarm sound are extracted from the detected sound using a plurality of band-pass filters each having a fixed peak frequency of the spectrum of the alarm sound to be recognized as a pass frequency. The alarm sound is recognized by comparing the temporal change shape of a plurality of peak frequency components with the temporal change shape of the peak frequency component of the alarm sound registered in advance (Japanese Patent Laid-Open No. Sho 62-175238). .

[Problems to be solved by the invention]

However, since the above-mentioned first device makes the determination by paying attention to the absolute values of a few fixed frequency components of the detected sound at each time, individual differences in the alarm sounding body, noise,
If the spectrum shape of the detected sound changes due to the Doppler effect, there is a problem that it cannot be recognized. Further, in the second device, when other sounds, for example, a running sound of a vehicle, a wind noise, a running sound of another vehicle, and the like are mixed, the pitch period is greatly affected, and it is not easy to recognize the alarm sound. In extracting the peak frequency component of the spectrum shape of the detected sound, the third device uses a plurality of bunt-pass filters in which the excess band is fixedly set in advance corresponding to each alarm sound. When the alarm sound shifted in frequency is included in the detection sound for the above reason, there is a problem that the shifted peak frequency component is not detected. or,
A large number of bandpass filters for detecting the peak frequency are required for each alarm sound, and the device becomes complicated. Furthermore, in order to directly match the temporal change shape of the extracted frequency component with the temporal change shape of the peak frequency of the pre-registered alarm sound,
There is also a problem that a large amount of data to be collated is required, and a long calculation time is required, and it takes time to recognize the alarm sound. In particular, for a sound source whose peak frequency changes with time, for example, for a siren of a fire engine, the change in the peak frequency cannot be detected because the extraction frequency of the above device is fixed. , Can not be accurate recognition. By the way, the ability to discriminate human sound is insensitive to the absolute values of frequency and sound pressure, but sensitive to relative changes in frequency and sound pressure. Therefore, while the conventional device performs the discrimination of the predetermined sound by focusing on the absolute value of the frequency and the sound pressure, the present invention focuses on such characteristics of the discrimination ability of human sound. , A temporal relative change in the peak frequency and its amplitude is extracted as a feature value, and the sound is recognized.

[Means for Solving the Problems]

As shown in FIG. 1, the means constituting the invention include an acoustoelectric converter M1, a frequency analyzing means M2, a peak extracting means M3, a grouping means M4, a feature extracting means M5, a discriminating means M6, and a reference feature storing. Means M7. The above-mentioned configuration means is achieved by means configured by hardware or software, and the frequency analysis means M2 comprises a frequency characteristic at each time of an acoustic signal whose time is output from an acoustoelectric transducer M1 such as a microphone as a variable. It is a means to ask for. Specifically, it is possible to obtain a frequency component while scanning the frequency using a digital bandpass filter having a steeply variable pass band, or to use a Fourier transformer that performs a Fourier transform on the acoustic signal. The peak extracting means M3 is a means for extracting a peak from the frequency characteristics. More specifically, the frequency characteristic can be constituted by a differentiator that differentiates with respect to the frequency by a difference operation. The grouping means M4 determines the continuity of the peak time from the peak time sequence output from the peak extracting means M3,
This is means for classifying peaks into continuous peak groups. Specifically, it is means for grouping peaks into consecutive peaks depending on whether or not the frequency divisions in which the peaks exist are continuous with respect to time. As is well known, similar to a line segment extractor in image processing, Means. The feature amount extracting unit M5 is a unit that extracts a feature amount of a characteristic shape associated with a temporal change of the peak group classified. Specifically, for each peak group, and for each partial shape of each peak group, the characteristic amount such as the amount of increase or decrease in frequency, the amount of increase or decrease in amplitude, and the duration of the partial shape is extracted. It is a means to do. The reference feature storage unit M7 is a unit that stores the feature of a predetermined person to be recognized as the reference feature in correspondence with the feature of the detected sound. The identification unit M6 is a unit that identifies a predetermined sound based on the feature amount extracted by the feature amount extraction unit M5 and the reference feature amount stored in the reference feature amount storage unit M7.

[Action]

The detected sound is converted into an acoustic signal, which is an electric signal, by the acoustic-electrical changer M1, and the acoustic signal is input to the frequency analyzing means M2, and the frequency characteristic at each time is obtained by the frequency analyzing means M2. The frequency characteristic given at each time is
The signal is input to the peak extracting means M3, and the peak of the frequency characteristic at each time is extracted by the peak extracting means M3. The peak information is input to the grouping means M4, and the grouping means M4
Thus, the data is grouped for each temporally continuous peak. The information on the group of peak groups is input to the feature amount extracting unit M5, and the feature amount extracting unit M5 extracts the feature amount of the characteristic shape of the continuous peak group. Then, the information of the characteristic amount is input to the identification unit M6, and the predetermined sound is identified based on the characteristic amount and the reference characteristic amount of the acoustic signal to be recognized stored in the reference characteristic amount storage unit M7.

【Example】

Hereinafter, the present invention will be described based on specific examples. FIG. 2 is a block diagram showing a configuration of the apparatus of the embodiment. The vehicle alarm device 2 of the present embodiment is configured to sample and store a sound signal from a microphone 4 for picking up sound outside the vehicle for a predetermined period of time, and to analyze the sampled sound signal. A high-speed arithmetic processing unit 8 for performing predetermined arithmetic processing at a high speed; an audio signal sampled by the audio signal input unit 6 being input to the high-speed arithmetic processing unit 8 to execute predetermined arithmetic processing; An alarm sound identification unit 10 that identifies various alarm sounds from emergency vehicles, level crossings, and the like from external sounds picked up by the microphone 4 and outputs an identification signal indicating the identification result.
And an output unit 74 that outputs a control signal to an alarm device 72 provided in the vehicle cabin according to the identification signal from the alarm sound identification unit 10 and notifies the vehicle driver of the identification result of various alarm sounds. A transmission unit 78 that transmits the identification result of the identification unit 10 to the vehicle control device 76 and executes vehicle control according to the identification result. Here, in the audio signal input unit 6, first, the audio signal from the microphone 4 is input to the pre-processing circuit 20, and the audio signal passing through the pre-processing circuit 20 is A / D converted by the A / D converter 22. The pre-processing circuit 20 is a circuit for processing an audio signal so that the A / D converter 22 can perform A / D conversion satisfactorily. An amplifier for amplifying the audio signal, an anti-aliasing filter, a sample-and-hold circuit, etc. Is provided. Also, the A / D converter 22 is controlled by the control circuit 24, automatically A / D converts the audio signal at a predetermined sampling cycle, and stores the result in the RAMa 26 or RAMb 28. The control circuit 24 starts with the A / D converter 22
Is connected to RAMa26 via the switch circuit 30, and A / A
The D conversion results are sequentially stored in the RAMa 26, and thereafter, when the storage area of the RAMa 26 is full, a storage signal indicating that fact is output to the CPU 40 of the alarm sound identification unit 10, and at the same time, the switch circuit 30 is switched, and the A / A The output of the D converter 22 is connected to the RAMb 28, and then the A / D converter 22
The result of the A / D conversion by the A / D converter 22 is alternately stored in the RAMb 26 and the RAMb 28 by a procedure such as sequentially storing the result in the RAMb 28. As a result, the A / D converter 22 can read the A / D converted data without stopping the operation of the A / D converter 22. The switch circuit 30 is actually configured by a TTL or CMOS logic circuit. The RAMa 26 and RAMb 28 which store the A / D conversion result by the A / D converter 22 have the A / D continuously for at least twice the period of the envelope of the alarm sound to be recognized.
The one having a capacity capable of storing the conversion result is used. Next, the alarm sound identification unit 10 includes a CPU 40, a ROM 42, and a RAM 44.
And executes a later-described alarm sound recognition process according to a later-described control program stored in the ROM 42. The alarm sound recognition processing is performed via the control circuit 24.
Controls the A / D converter 22 and RAMa 26
Alternatively, the A / D conversion data stored in the RAMb 28 is output to the high-speed arithmetic processing unit 8 to execute predetermined arithmetic processing, and based on the arithmetic result, the emergency vehicle is selected from the external sounds collected by the microphone 4. This is a process for identifying an alarm sound of a pedestrian crossing, an alarm sound of a pedestrian crossing, an alarm sound from a circuit breaker at a level crossing, an alarm sound from another vehicle (ie, a horn sound), and the like. In order to execute this processing, the ROM 42 stores in advance a reference feature amount extracted from a characteristic shape associated with a temporal change of a frequency characteristic peak obtained by frequency analysis of the various alarm sounds to be identified. It is remembered. Next, based on the input data from the alarm sound identification unit 10, the high-speed arithmetic processing unit 8 processes the audio signal input through the audio signal input unit 6 at a high speed, and performs the frequency characteristics of the audio signal at each time. DSP (Digital Signal Processor) 50 for processing a large amount of numerical operations at high speed, RAM 52 for storing input data from the alarm sound identification unit 10 and data after the arithmetic processing, It comprises a ROM 54 in which a control program for executing a high-speed operation is stored in advance, and a control circuit 56 for controlling the execution or stop of the operation of the DSP 50 in accordance with the operation command from the alarm sound discrimination unit 10. For this reason, the CPU 40 of the alarm sound discriminating unit 10 stops the arithmetic processing of the DSP 50 by the control circuit 56, transfers the data to be processed to the RAM 52, and thereafter, the control circuit 5
By executing the arithmetic processing of the DSP 50 via 6, desired arithmetic processing can be executed as needed. Next, the output unit 74 is for giving a signal to the alarm 72 when it is necessary to issue an alarm to the driver based on the result of the alarm sound identification by the alarm sound identification unit 10, and the alarm 72 is a speaker or an alarm lamp. Alternatively, the driver is notified of the presence / absence and type of the alarm by a display or the like. Further, the transmission unit 78 is for transferring the recognition result of the alarm sound to the vehicle control device 76 for controlling the traveling of the vehicle, and the vehicle control device 76 uses this information as one input element of the control. I do. That is, for example, if the vehicle control device 76 is an engine control device, the vehicle is decelerated when an alarm sound from the circuit breaker is recognized, or if the vehicle control device 76 is a steering control device, When the warning sound from the emergency vehicle is recognized in the vehicle, control can be performed such that the vehicle is brought closer to the road shoulder. The above is the hardware configuration of this apparatus. The acoustoelectric converter M1 is composed of the microphone 4, and the frequency analysis means M2 is composed of the main high-speed processing unit 8 and the high-speed processing unit 8
It comprises an audio signal input unit 6 and a warning sound identification unit 10 as sub-components for processing and controlling input data for the, and includes a peak extraction unit M3, a group classification unit M4, a feature amount extraction unit M5, and an identification unit M6.
Is constituted by the alarm sound identification unit 10, and the reference feature amount storage means M7 is constituted by the ROM 42 of the alarm sound identification unit 10. Next, the operation of the vehicle alarm device 2 will be described by the alarm sound recognition unit 10.
The processing procedure of the CPU 40 will be described with reference to FIGS. 3 and 4, and the explanatory views of FIGS. 5 to 12 showing the data processing. As shown in FIG. 3, when the alarm sound recognition process is started,
First, in step 100, initialization processing for initializing a memory and various peripheral elements is executed, and the process proceeds to step 110. In step 110, a drive signal is output to the control circuit 24 to start the A / D conversion operation of the A / D converter 22 in order to start the operation of the acoustic signal input unit 6. Then
As described above, the audio signal input unit 6 converts the audio signal input via the microphone 4 into an A / D converter 22.
A / D conversion is performed at a predetermined sampling cycle, and the A / D conversion data is sequentially stored in the RAMa 26 first, and when the RAMa 26 is full, that is, data for a time necessary for recognition of the alarm sound can be obtained. Then, the control circuit 24 outputs a stored signal to the CPU 40. Then, the subsequent sampling data by the A / D converter 22 is sequentially stored in the RAMb 28. According to such a procedure, the acoustic signal from the microphone 4 is sequentially stored in the RAMa26 and RAMb28 functioning as a buffer memory, and each time the RAMa26 or RAMb28 becomes full,
The storage signal is input to the CPU 40. Then, a time change characteristic of the frequency characteristic within a certain time is obtained in synchronization with the stored signal. Therefore, in the subsequent step 120, the process waits for the storage signal from the control circuit 24 to be input, and when the storage signal is input and the RAMa 26 or RAMb 28 is full, the process proceeds to step 130, where the A / D conversion is performed. Transfer data to RAMa26 or RAMb
28, and temporarily stored in the RAM 44 of the alarm sound identification unit 10. In the following step 140, a frequency analysis process is executed based on the stored A / D conversion data. This frequency analysis processing is executed as shown in FIG. As shown in FIG. 4, in the frequency analysis processing, first, in step 300, the A / D conversion data stored in the RAM 44 is transferred to the RAM 52 of the high-speed operation processing unit 8, and in the next step 310, the control circuit 56 The filter processing, which is a program related to the frequency analysis of the DSP 50, is started. Then, according to the program procedure stored in the ROM 54, the DSP 50 performs a filtering process for calculating a time change characteristic of the amplitude (power) of the specific frequency component during the predetermined time from the A / D conversion data stored in the RAM 52 for the predetermined time. And stores the processing result in an empty area of the RAM 52 to notify the CPU 40 of the end of the program through the control circuit 56. Therefore, in the subsequent step 320, the process waits for the end signal of the program to be input, and when the end signal is input, the process proceeds to step 330, where the time change characteristic data of one frequency component as a result of the filter processing is transferred from the RAM 52. Read, RAM4
Store in the empty area of 4. Then, in the following step 340, it is determined whether or not the filtering process has been completed for all preset extraction frequencies. If not, the process returns to step 310 to start the filtering process program again. Do. The DSP 50 changes the extraction frequency little by little every time the filter processing is started, and executes the frequency analysis processing over the entire frequency range. As a result, the time change characteristic data of each frequency component within the same time is stored in the RAM 44. That is, for example, the acoustic signal shown in FIG.
The frequency is analyzed by P50, and as shown in FIG. 6, a time change characteristic of the frequency characteristic within a certain time is obtained. This fixed time is a time required for recognizing the alarm sound, that is, a time about twice as long as the cycle of the envelope of the alarm sound. In addition, DSP50
The time interval of the frequency analysis data output by the CPU 40 is equal to the sampling period, but the data shown in FIG. 6 is obtained by the CPU 40 so that the average value in a certain time width is subjected to the frequency analysis data at that time. Average processing is applied. Next, the CPU 40 shifts the execution to step 150 in FIG. 3, extracts the peak information from the data as shown in FIG. 6, and creates the data as shown in FIG. That is, assuming that the frequency characteristic at a certain time is the characteristic shown in FIG. 11, a differential operation (actually, a difference operation) is performed with respect to the frequency, and the maximum value, that is, the peak is the set data of the frequency and the amplitude (f ₀ , P ₀ ), (F ₂ , P ₂ ). Such processing is performed at each time t ₁ , t ₂ , t ₂ .
This is performed for each frequency characteristic at t _n , and eventually, peak data as shown in FIG. 7 is obtained. Next, the CPU 40 shifts the execution to step 160, and determines the continuity of the peaks extracted as shown in FIG. This peak frequency f _i in the frequency characteristics of the arbitrary time t _i 1
Peak frequency f _i-1 of the frequency characteristic at time t _i-1 before One
If it is approaching with a certain width, the extracted peak f
Group _i as an extension of continuous line B. Further, on the contrary, if the peak frequency f _i and the peak frequency f _i-1 does not approach a constant width, the group-specific the peak f _i as a starting point of a new continuous line. Such a process is executed for the peak of the frequency characteristic at each time, so that the eighth
As shown in the figure, the extracted peaks are grouped into a continuous line B and a continuous line C. Incidentally, in an actual road environment, peaks that should be extracted may not be extracted due to noise or passage of a sound-insulating object. In such a case, a silent state occurs between the peaks before and after the peak and the sound is no longer detected as a continuous sound. You may do it. Next, in step 170 and step 180, the grouped peak groups are blocked for each common characteristic amount of the characteristic shape with time. In step 170,
As shown in FIG. 8, block division is performed by paying attention to the time change of the frequency of the peak data grouped into the continuous line B and the continuous line C. That is, as shown in FIG. 9, the peak sequence of the continuous line B can be divided into a block X having a higher frequency and a block Y having a lower frequency from the viewpoint of the time change of the frequency. it can. On the other hand, the peak sequence of the continuous line C is determined as one block Z as it is because the frequency is constant as a whole. Next, in step 180, block division focusing on a temporal change in amplitude is further performed. In FIG. 9, block X is determined as having a constant amplitude, block Y is determined as having a constant amplitude, and block Z is determined as having an amplitude attenuation. In this case, no further block division is performed. Incidentally, in the example as shown in FIG.
In the block division focusing on the time change of the frequency in 170, the frequency is constant and the whole is determined as one block. However, in this step 180, the time change of the amplitude is focused on, so that the block V and the block W , And one of the blocks is determined to have amplitude attenuation. Next, in step 190, the feature amounts of the blocks X, Y, and Z divided as described above are stored in the RAM 44 in the following format. {Block start time, block end time, amplitude time change form, block start frequency, block end frequency, frequency time change form} Therefore, in the tenth data, specifically, X = {T ₀ , T ₁ , constant, f ₀ , f ₁ , rise｝ Y = ｛T ₁ , T ₂ , constant, f ₁ , f ₀ , fall｝ Z = ｛T ₃ , T ₄ , fall, f ₂ , f ₂ , Is constant. As described above, the feature amount focusing on the time change of the frequency and the time change of the amplitude of the sound input from the microphone 4 is extracted for each block. Next, the program proceeds to step 200 and step 190
It is determined whether or not any of the feature amounts obtained in step 1 matches the reference feature amount of the person to be recognized. Each recognition target sound is divided into reference blocks for each reference feature amount in the same manner as in the above-described blocking. Then, a reference block name configured for each recognition target sound and a reference feature amount of each reference block are stored in the ROM 42 in a format similar to the above. For example, the alarm sound of a level crossing barrier is divided into two reference blocks α and β. The reference feature value of each reference _{block, α = {0, t e} ± Δ 1, _{lowering, f j ± Δ 2, f} j ± Δ 3, _{constant} β = {0 + Δ 4} , t f ± Δ 5, Descent, ± Δ ₆ , ± Δ ₇ , constant｝. Note that the reference feature values α and β
Are related by a certain relationship peculiar to the sounding body, so that the reference feature value β is defined as an allowable deviation from the reference feature value α. That is, the second sound corresponding to the reference feature β is
The first sound data corresponding to the reference feature alpha, start time and delta ₄ within After completion first ON, the frequency is in the allowable range range of ± delta ₆ or ± ₇ relative to the frequency of the first sound . Thus, the relative differences Δ ₆ and Δ ₇ between the frequencies of the first sound and the second sound,
Since the time-varying shape of the frequency and the amplitude and the sounding times t _e and t _f of the first and second sounds are limited, the allowable ranges Δ ₂ and Δ of the absolute values f _j of the frequencies of the first and second sounds are limited. ₃ has a very large value or infinity (that is, there is no limitation on the absolute value of the frequency), it does not erroneously recognize other sounds as a level crossing alarm sound, and there is a variation in frequency due to individual differences in sounding bodies, Recognition can be performed without being affected by the frequency shift due to the Doppler effect. Also, in the case of a siren of an ambulance, if the relative relationship between the two reference blocks corresponding to "Pee" and "Pee" of "Pee Pee Poo" is limited, recognition can be made in the same manner as in the case of a railroad crossing. The feature amount of the detected sound extracted as described above is compared with the reference feature amount as follows. A corresponding reference feature is selected by judging whether or not both the time variation of the amplitude and the time variation of the frequency match between the feature and the reference feature. Then, between the selected reference feature value and the feature value, whether or not the duration is within the allowable range, whether or not the frequency change amount is within the allowable range, the start frequency and the end frequency Is determined whether or not is within the allowable range. In this way, the reference feature values satisfying all the conditions are selected. Next, when one group of peaks is composed of a plurality of blocks or when two blocks are close to each other, it is determined whether the relationship between the blocks is equal to the relationship between the reference blocks. Is determined. When the relationship between the two becomes equal, it is finally recognized as an alarm sound composed of the reference block. Specifically, the continuous line segment B in FIG. 9 is a siren sound of a fire engine whose frequency changes with time, and the continuous line segment D in FIG. From this, it is recognized as a warning sound of a level crossing barrier. Next, proceeding to step 210, based on the recognition result in step 200, outputting a recognition signal to the output unit 74 and the transmission unit 78 in FIG. 2 to display the type of the recognized alarm sound, And make it sound inside the vehicle. In this way, one cycle of the sound recognition processing is completed, the process returns to step 120, and the sound recognition processing of the next cycle is executed in synchronization with the next storage signal.

【The invention's effect】

The acoustic recognition device of the present invention obtains frequency characteristics at each time of an acoustic signal, extracts peaks from the frequency characteristics, determines continuity of the peaks with respect to time, and classifies the peaks into groups of consecutive peaks. Then, the characteristic amount of the characteristic shape accompanying the temporal change of the peak group is extracted, and the predetermined person is identified based on the characteristic amount and the reference characteristic amount. Therefore, since the sound is recognized using the characteristic shape that accompanies the temporal change of the frequency characteristic, it is possible to accurately recognize even a sound that has a frequency shift or a frequency that changes with time. Becomes In addition, the characteristic shape is not compared with the shape as it is, but is compared with the feature amount of the shape. Therefore, the matching calculation is shortened, and the recognition speed is improved.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing the concept of the present invention. FIG. 2 is a block diagram showing a configuration of an apparatus according to a specific embodiment of the present invention. FIG. 3 and FIG. 4 are flowcharts showing processing procedures of the CPU of the apparatus of the embodiment. FIG. 5 is a waveform diagram showing an acoustic signal. FIG. 6 is an explanatory diagram showing a frequency analysis result. FIG. 7 is an explanatory diagram showing extraction of a peak. FIG. 8 is an explanatory diagram showing peaks classified by continuous line segments. FIG. 9 is an explanatory diagram showing block formation for each feature of a peak group divided into groups. FIG. 10 is an explanatory diagram showing extraction of a feature amount. FIG. 11 is a characteristic diagram showing frequency characteristics at one time. FIG. 12 is an explanatory view showing blocking of a peak group. 2 ... Vehicle alarm device 4 ... Microphone 6 ... Acoustic signal input unit 8 ... High speed processing unit 10 ... Alarm sound identification unit

Claims (1)

(57) [Claims] 1. A sound recognition device for recognizing a predetermined sound from a sound detected by an acoustoelectric converter, wherein: a frequency analysis means for obtaining a frequency characteristic at each time of an acoustic signal output from the acoustoelectric converter; Peak extracting means for extracting a peak from the frequency characteristic at each time obtained by the frequency analyzing means; determining continuity of the peak extracted by the peak extracting means with respect to time; Grouping means for separating, a characteristic amount extracting means for extracting a characteristic amount of a characteristic shape associated with a temporal change of the peak group classified by the grouping means, and recognition corresponding to the characteristic amount of the detected sound. A reference feature value storage unit that stores a feature value of a predetermined sound to be performed as a reference feature value; a feature value extracted by the feature value extraction unit and the reference feature value storage unit. On the basis of the reference feature values,
An acoustic recognition device, comprising: identification means for identifying a predetermined sound.