JP7287182B2 - SOUND PROCESSING DEVICE, SOUND PROCESSING PROGRAM AND SOUND PROCESSING METHOD - Google Patents

Description

The present invention relates to a sound processing device, a sound processing program, and a sound processing method, and for example, a sound masking process used as a technique for preventing the contents of a conversation from leaking out to third parties around the speaker who is speaking. can be applied to

In recent years, when a speaker has a conversation with a conversation partner at a reception counter, window, meeting space, etc. in a facility where an unspecified number of people are present (for example, hospitals, pharmacies, banks, etc.), the content of the conversation is Leakage to three parties is a problem.

Preventing the content of a conversation from being leaked to a third party is called speech privacy, and a sound masking effect is used to achieve speech privacy.

The sound masking effect is that when a certain sound (hereafter referred to as the target sound) is heard and there is another sound with similar acoustic characteristics (for example, frequency characteristics, pitch, formants, etc.), the target sound is masked. This is a phenomenon in which sounds become difficult to hear (masked). In general, the masked sound is called a masker, and the masked sound is called a maskee.

Patent documents 1 and 2 propose a sound masking device that uses this sound masking effect to prevent the leakage of conversation content to a third party (protect speech privacy).

The sound masking device described in Patent Document 1 analyzes the acoustic feature quantity of the speaker's speech signal and generates a masker signal based on the analysis result even when the speaker's speech signal, which is a masking signal, changes. , is a sound masking device capable of obtaining a high masking effect.

The speech processing method described in Patent Literature 2 extracts the spectral envelope and spectral fine structure of an audio signal, and deforms the extracted spectral envelope to generate a modified spectral envelope. Then, a modified spectrum is generated by synthesizing the modified spectrum envelope and the extracted spectral fine structure, and a signal generated based on the modified spectrum is output as a masker signal so that the contents of the conversational voice cannot be heard by a third party. It is an audio processing method that makes

JP 2012-88577 A JP 2006-243178 A

In the sound masking device described in Patent Document 1, voice signals of a plurality of people, including men and women, are stored in a database as general-purpose masker signals so that a certain degree of masking effect can be expected even for unspecified speakers. . Then, based on the analysis results of the acoustic features of the speaker's speech signal, the acoustic characteristics of the general-purpose masker signal stored in the database are changed (for example, the pitch of the general-purpose masker signal is converted to the pitch of the input speech signal, The masker signal is generated by converting the formant of the general-purpose masker sound into the formant of the input audio signal. Therefore, there is a possibility that the signal obtained by changing the general-purpose masker signal stored in the database will sound artificial and the masker signal will sound unpleasant. Furthermore, if the analysis result of the acoustic feature amount is wrong, the masking effect will be low and the content of the conversation cannot be masked because the acoustic feature amount of the speaker's voice and the acoustic feature amount of the masker signal are different.

Also in the speech processing method described in Patent Document 2, the spectral envelope of the extracted speech signal is deformed to generate a deformed spectral envelope, and the deformed spectral envelope and the spectral fine structure of the extracted speech signal are synthesized and used for masker signal generation. are doing. For this reason, the masker signal generated by transforming the voice signal of the speaker ends up sounding artificial, and there is a possibility that the masker signal will sound unpleasant.

In addition, in both the sound masking device described in Patent Document 1 and the sound processing method described in Patent Document 2, when the generated masker signal is output so that the speaker can hear it, the speaker also hears the masker signal. Because it is closed, it interferes with conversation, and it is not possible to have a smooth conversation.

In view of the above problems, even if the acoustic feature value of the speaker uttering speech (hereinafter referred to as "target speaker") is not analyzed, or the analysis result of the acoustic feature value is incorrect, An acoustic processing device, an acoustic processing program, and an acoustic processing method capable of realizing a high masking effect are desired. Furthermore, an acoustic processing device, an acoustic processing program, and an acoustic processing method capable of masking the voice uttered by the target speaker without interfering with the conversation of the target speaker are desired.

A sound processing apparatus according to a first aspect of the present invention comprises: (1) frame dividing means for dividing a microphone input signal supplied from a microphone for picking up a sound uttered by a target speaker into predetermined lengths; input signal storage means for storing frame-divided microphone input signals; and (3) a signal used for generating a masker signal is selected from past frame-divided microphone input signals stored in the input signal storage means. (4) a masker signal for generating and outputting the masker signal that makes it difficult to hear the voice uttered by the target speaker using the signal used for generating the masker signal; (5) pitch estimating means for estimating the pitch of the microphone input signal; (7) the masker signal generating means generates a masker signal using the microphone input signal of the class corresponding to the pitch estimated by the pitch estimating means from the input signal accumulating means; characterized by

The sound processing program of the second aspect of the present invention comprises: (1) a frame dividing means for dividing a microphone input signal supplied from a microphone for picking up a voice uttered by a target speaker into predetermined lengths; 2) input signal accumulation means for accumulating the frame-divided microphone input signal; and (3) a signal used for generating a masker signal from the past frame-divided microphone input signal accumulated in the input signal accumulation means. and (4) using the signal used to generate the masker signal, generates and outputs the masker signal that makes it difficult to hear the speech uttered by the target speaker. (5) functions as pitch estimation means for estimating the pitch of the microphone input signal; and (6) the input signal accumulation means stores the microphone input signal according to the pitch estimated by the pitch estimation means. (7) the masker signal generating means uses the microphone input signal of the class corresponding to the pitch estimated by the pitch estimating means from the input signal accumulating means to generate a masker signal; is characterized by generating

A sound processing method according to a third aspect of the present invention includes (1) frame division means, input signal accumulation means, signal selection means , masker signal generation means, and pitch estimation means ; (3) the input signal accumulation means accumulates the frame-divided microphone input signal, ( 4) the signal selection means selects a signal to be used for generating a masker signal from past frame-divided microphone input signals accumulated in the input signal accumulation means, and outputs the selection result ; The masker signal generating means uses the signal used to generate the masker signal to generate and output the masker signal that makes it difficult to hear the speech uttered by the target speaker, and (6) the pitch estimating means. (7) the input signal storage means sorts the microphone input signal into one of a plurality of classes according to the pitch estimated by the pitch estimation means and stores the class; (8) The masker signal generating means generates the masker signal using the microphone input signal of the class corresponding to the pitch estimated by the pitch estimating means from the input signal accumulating means.

According to the present invention, the signal used to generate the masker signal is generated using the accumulated past speech of the target speaker, so that the acoustic feature quantity is not analyzed, or the acoustic feature quantity is not analyzed. Even if the analysis result is wrong, a high masking effect can be achieved by generating a masker signal using a signal whose acoustic characteristics are not changed. Furthermore, the voice uttered by the target speaker can be masked without disturbing the conversation of the target speaker.

1 is a block diagram showing a functional configuration of a sound masking device according to a first embodiment; FIG. 1 is a block diagram showing an example of a hardware configuration of a sound masking device according to a first embodiment; FIG. FIG. 4 is an image diagram when the masker signal generated by the sound masking device according to the first embodiment is reflected on the floor surface and output. FIG. 4 is an image diagram of outputting a masker signal generated by the sound masking device according to the first embodiment; FIG. 11 is a block diagram showing the functional configuration of a sound masking device according to a second embodiment; FIG. FIG. 11 is a block diagram showing the functional configuration of a sound masking device according to a third embodiment; FIG. FIG. 12 is a block diagram showing the functional configuration of a sound masking device according to a fourth embodiment; FIG. FIG. 11 is a block diagram showing a configuration for accumulating a third party audio signal in a third party audio signal DB (database) of the sound masking device according to the fourth embodiment; FIG. 12 is a block diagram showing the functional configuration of a sound masking device according to a fifth embodiment; FIG. FIG. 12 is a block diagram showing the functional configuration of a sound masking device according to a sixth embodiment; FIG.

(A) First Embodiment Hereinafter, a first embodiment of a sound processing device, a sound processing program, and a sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to a sound masking device will be described.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing the functional configuration of a sound masking device 100 according to the first embodiment.

The sound masking device 100 has a microphone 101 , a microphone amplifier 102 , an AD converter 103 , a speaker 104 , a speaker amplifier 105 , a DA converter 106 and a sound masking processing section 200 .

The microphone 101 is a microphone that converts air vibration such as human voice and sound into an electric signal.

The microphone amplifier 102 amplifies an input signal received (collected) by the microphone 101 .

The AD converter 103 converts the input signal amplified by the microphone amplifier 102 from an analog signal to a digital signal. Hereinafter, the signal converted by the AD converter 103 will be referred to as "microphone input signal".

The sound masking processing unit 200 generates and outputs a masker signal from an input microphone input signal or past microphone input signals.

The DA converter 106 converts the sound signal output from the sound masking processing unit 200 from a digital signal to an analog signal.

A speaker amplifier 105 amplifies an analog signal.

The speaker 104 is a speaker that converts an electrical signal into air vibration and outputs it as sound.

Next, a detailed configuration of the sound masking processing section 200 will be described.

The sound masking processing unit 200 has a frame division unit 201, an input signal DB (database) 202, a signal selection unit 203, a masker signal generation unit 204, a sound input terminal IN, and a sound output terminal OUT.

A sound input terminal IN is an interface (audio interface) for inputting a microphone input signal to the sound masking processing unit 200 .

The frame dividing unit 201 divides the microphone input signal input to the sound masking processing unit 200 into predetermined lengths (processing frames) and outputs the divided frames. The frame division unit 201 generally divides the signal into lengths suitable for analyzing the voice, and for example, divides the microphone input signal into frames in units of 100 [milliseconds] to 200 [milliseconds].

The input signal DB 202 is storage means for accumulating frame-divided microphone input signals.

The signal selection unit 203 selects a signal to be used for generating a masker signal (hereinafter referred to as a “masker bare edge signal”) from past frame-divided microphone input signals stored in the input signal DB 202, and selects a selection result. to output

The masker signal generation unit 204 reads a plurality of frames of the selected masker side signal from the input signal DB 202, and generates and outputs a masker signal using the read masker side signals of the plurality of frames.

The sound output terminal OUT is an interface (audio interface) that outputs the generated masker signal to the DA converter 106 .

The sound masking processing unit 200 may be entirely configured in hardware (for example, constructed using a dedicated board or DSP (Digital Signal Processor)), or may be configured in software using a computer. can be The sound masking processing unit 200 may be configured by installing programs (including the sound processing program according to the embodiment) in a computer having a memory and a processor, for example. In this embodiment, the AD converter 103 and the DA converter 106 are arranged outside the sound masking processing unit 200, but the sound masking processing unit 200 is equipped with the AD converter 103 and the DA converter 106. It is good also as the composition which carried out.

Next, FIG. 2 shows a configuration when the sound masking processing unit 200 is implemented by software (computer).

The sound masking processing unit 200 shown in FIG. 2 is configured in software using the computer 300 . A program (a program including the sound processing program of the embodiment) is installed in the computer 300 . The computer 300 may be a computer dedicated to the sound processing program, or may be configured to be shared with programs of other functions.

A computer 300 shown in FIG. 2 has a processor 301, a primary storage unit 302, a secondary storage unit 303, a sound input terminal IN, and a sound output terminal OUT. A sound input terminal IN and a sound output terminal OUT are the same as the elements shown in FIG.

The primary storage unit 302 is storage means that functions as a working memory (work memory) for the processor 301, and for example, a high-speed memory such as a DRAM (Dynamic Random Access Memory) is applied.

The secondary storage unit 303 is storage means for recording various data such as OS (Operating System) and program data (including data of the sound processing program according to the embodiment). Drive), SSD (Solid State Drive), and other non-volatile memories are applied.

In the computer 300 of this embodiment, when the processor 301 is activated, the OS and programs (including the sound processing program according to the embodiment) recorded in the secondary storage unit 303 are read, and expanded on the primary storage unit 302. Execute. Note that the specific configuration of the computer 300 is not limited to the configuration in FIG. 2, and various configurations can be applied. For example, if the primary storage unit 302 is a non-volatile memory (for example, FLASH memory), the secondary storage unit 303 may be excluded.

(A-2) Operation of the First Embodiment Next, the operation of the sound masking device 100 (acoustic processing method of the embodiment) having the configuration described above according to the first embodiment will be described in detail.

When the sound masking device 100 starts operating and the user of the sound masking device 100 (target speaker U1 in FIG. 3) speaks into the microphone 101, a voice signal is input to the microphone 101. FIG.

An analog sound signal input to the microphone 101 is amplified by the microphone amplifier 102, converted from the analog signal to a digital signal by the AD converter 103, and supplied to the sound input terminal IN of the sound masking processing unit 200 as the microphone input signal x(n). ). Note that in the microphone input signal x(n), n is a parameter representing discrete times of the input signal.

When the microphone input signal x(n) starts to be input to the sound input terminal IN of the sound masking processing unit 200 , it is input to the frame dividing unit 201 .

Frame dividing section 201 divides microphone input signal x(n) into predetermined units. The frame division unit 201 divides each processing frame according to the following formula (1), for example.

In equation (1), x_fram (l; m) is a frame-divided microphone input signal, l is a frame number, m is a discrete time within a frame (m=0, 1, 2, . . . , M−1 ), M is the frame length. The frame dividing unit 201 outputs the frame-divided microphone input signal x_fram(l;m) to the input signal DB 202 .

The input signal DB 202 accumulates the frame-divided microphone input signal x_fram(l;m) in the input signal DB 202 frame by frame according to formulas (2) and (3).

In equation (2), DB(i;m) is the input signal DB, i is the index of the database (i=0, 1, 2, . , 1, 2, . . . , M-1), where M is the frame length and I is the database length. As shown in equation (3), i is incremented when data is accumulated in the input signal DB.

The signal selection unit 203 selects a masker bare edge signal from past frame-divided microphone input signals accumulated in the input signal DB 202 . The signal selection unit 203, for example, calculates the selection result T(k) as shown in equation (4).

In equation (4), k (k=1, 2, . , I) is the M O D function that returns the remainder when ik is divided by I. By returning the remainder when divided by I, the selection result T(k) becomes a value from 0 to I−1. For example, in equation (4), when K=5, microphone input signals for five frames accumulated in the input signal DB 202 are selected.

Various methods can be widely applied to the method of calculating the selection result T(k). For example, as shown in equation (5), the masker element signal may be randomly selected.

In equation (5), rand(k) is a function that generates a non-negative integer random number for a natural number k. Expression (5) returns the remainder when the random number generated by rand(k) using the MOD function is divided by I, and the selection result T(k) is a value from 0 to I−1. Signal selection section 203 outputs selection result T(k) to masker signal generation section 204 .

Based on the selection result T(k) of the signal selection unit 203, the masker signal generation unit 204 reads the masker side signal from the input signal DB 202 for K frames, and generates the masker signal from the read masker side signal of the K frames. Generate and output. The method of generating the masker signal is, for example, as shown in equation (6), by superimposing the masker edge signals of the read K frames.

In equation (6), k (k=1, 2, . ) is the masker signal. For example, in equation (6), when K=5, based on the selection result T(k), the past five frames are read from the input signal DB 202 as the masker edge signal, and the read masker edge signal is A masker signal h(l;m) is generated by superimposing the signals.

Various methods can be widely applied to generate the masker signal h(l;m). The masker signal h(l;m) may be generated by time-reversing and superimposing the microphone input signal as time processing, or the past frame-divided microphone input signal stored in the input signal DB 202 may be time-processed. , the masker signal h(l;m) may be generated by superimposing with a time delay.

Then, the masker signal generator 204 outputs the masker signal h(l;m) to the sound output terminal OUT of the sound masking processor 200 as the output signal y(n) according to equation (8).

A signal output from the sound output terminal OUT of the sound masking processing unit 200 is converted from a digital signal to an analog signal by the DA converter 106 , amplified by the speaker amplifier 105 and then output from the speaker 104 .

3 and 4 show a microphone 101, a target speaker U1 speaking into the microphone 101, and a person other than the target speaker U1 standing behind the target speaker U1 (a person speaking by the target speaker U1). 3A and 3B are diagrams showing an example of the arrangement relationship (arrangement configuration of the speaker 104) between a person U2 whose voice is to be made difficult to hear with the masker signal (hereinafter referred to as a “masking target person”) and the speaker 104. FIG. In Fig. 4, the directivity of the direct sound DS (Direct Sound) output from the speaker is indicated by a dotted line, and in Fig. 3, the reflected sound RS (Reflected Sound) generated by the reflection of the direct sound on the floor FR. Sound) is illustrated by a dashed line.

In FIG. 3, the speaker 104 is placed in front of the target speaker U1 at a knee height, the vibration plane (directivity) of the speaker 104 is directed downward, and the speaker 104 is installed obliquely to the surface of the floor FR. ing. Furthermore, the directivity is directed toward the floor FR portion behind the target speaker U1. Then, as shown in FIG. 3, the masker signal radiated from the speaker 104 is output toward the surface of the floor FR, and is reflected upon reaching the floor FR. As a result, the masker signal is reflected as shown in FIG. 3 and transmitted to the masking target person U2 behind the target speaker U1. At this time, the direct sound of the voice uttered by the target speaker U1 is also transmitted to the masking target U2, but is masked by the masker signal.

As for the installation method of the speaker 104, various installation methods can be widely applied as long as the speaker 104 is installed so that the target speaker U1 cannot hear the masker signal and the masking target U2 can hear the masker signal. For example, as shown in FIG. 4A, if there is a space that can be installed behind the target speaker U1, the vibration surface of the speaker 104 can be directed directly toward the masking target U2 to output the masker signal. Alternatively, as shown in FIG. 4B, the speaker 104 is embedded in the floor FR and the vibration surface of the speaker 104 is directed directly toward the masking target person U2 to output the masker signal. Alternatively, as shown in (c) of FIG. 4, a speaker 104 may be installed on the ceiling CE and the vibration surface of the speaker 104 may be directed directly toward the masking target person U2 to output the masker signal. good.

(A-3) Effects of First Embodiment According to the first embodiment, the following effects can be obtained.

The sound masking apparatus 100 of the first embodiment accumulates the speech of the target speaker U1 in an input signal DB, and uses a plurality of frames of past frame-divided microphone input signals accumulated in the input signal DB for masking. Generate and output a signal. As a result, in the sound masking device 100 of the first embodiment, the acoustic features of the masker signal become closer to the acoustic features of the voice of the target speaker U1, thereby improving the masking effect and preventing the leakage of the content of the conversation. be able to. In other words, the sound masking apparatus 100 of the first embodiment analyzes the acoustic characteristics of the target speaker U1 by generating the masker signal using the voice signal of the target speaker U1 stored in the input signal DB. A high masking effect can be obtained even without the masking signal because the acoustic features of the masker signal are close to the acoustic features of the speech signal of the target speaker U1.

Furthermore, in the sound masking apparatus 100 of the first embodiment, the speaker for reproducing the masker signal is installed so that the target speaker U1 cannot hear the masker signal and the masker target U2 can hear the masker signal. By doing so, the voice uttered by the target speaker U1 can be masked without interfering with the conversation of the target speaker U1.

(B) Second Embodiment Hereinafter, a second embodiment of the sound processing device, sound processing program, and sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to a sound masking device will be described.

(B-1) Configuration of Second Embodiment FIG. 5 is a block diagram showing the functional configuration of a sound masking device 100A according to the second embodiment. In FIG. 2, the same reference numerals or corresponding reference numerals are assigned to the same or corresponding portions as those in FIG.

In the following, the second embodiment will be described with a focus on differences from the first embodiment, and descriptions of portions that overlap with the first embodiment will be omitted.

The sound masking device 100A of the second embodiment differs from the first embodiment in that the sound masking processing section 200 is replaced with a sound masking processing section 200A. The sound masking processing unit 200A differs from the first embodiment in that the masker signal generation unit 204 is replaced with a masker signal generation unit 204A, and a speech section determination unit 205 and a DB accumulation determination unit 206 are added. ing.

In the sound masking processing unit 200A of the sound masking apparatus 100A of the second embodiment, the frame-divided microphone input signal and the masker input signal accumulated in the input signal DB due to the addition of the speech period determination unit 205 and the DB accumulation determination unit 206 are added. The sound masking apparatus 100 differs from the sound masking apparatus 100 of the first embodiment in that the method of signal generation is different, and the accumulation method of frame-divided microphone input signals and the masker signal method are different due to the use of the masker signal generation unit 204A.

A voice segment determination unit 205 determines whether the frame-divided microphone input signal is a voice segment or a non-voice segment (a segment other than a voice segment), and outputs the determination result.

If the frame-divided microphone input signal is determined to be in a voice segment based on the voice segment determination result of the voice segment determination unit 205, the DB accumulation determination unit 206 stores the frame-divided microphone input signal in the input signal DB 202. If it is determined as a non-speech section, the frame-divided microphone input signal is not output to the input signal DB 202 .

The masker signal generation unit 204A reads a plurality of frames of the selected masker side signal from the input signal DB 202 based on the result of the speech section determination and the selection result, and generates a masker signal from the read masker side signals of the plurality of frames. Generate and output.

(B-2) Operation of Second Embodiment Next, the operation of the sound masking device 100A (acoustic processing method according to the embodiment) according to the second embodiment having the configuration described above will be described in detail.

The basic operation of the sound masking process in the sound masking device 100A according to the second embodiment is the same as the sound masking process described in the first embodiment.

In the following, a detailed description will be given centering on the processing operations in the speech section determination unit 205, the DB accumulation determination unit 206, and the masker signal generation unit 204A, which are different from the first embodiment.

Frame dividing section 201 divides microphone input signal x(n) into processing frames, and outputs frame-divided microphone input signal x_fram(l;m) to voice section determining section 205 and DB accumulation determining section 206 .

The voice segment determination unit 205 uses the frame-divided microphone input signal x_fram(l;m) to determine whether it is a voice segment or a non-voice segment. The means for judging whether it is a speech segment or a non-speech segment makes a determination according to the equations (9) and (10), for example.

In equations (9) and (10), x_fram(l;m) is the frame-divided microphone input signal, x_fram_amp(l) is the average amplitude value of the frame-divided microphone input signal, and VAD(l) is the voice section determination result. , TH are thresholds used to determine the speech segment.

Formula (9) is a formula for obtaining the average amplitude value x_fram_amp(l) of the frame-divided microphone input signal x_fram(l;m). Equation (10) determines that it is a voice segment if the average amplitude value x_fram_amp(l) of the frame-divided microphone input signal x_fram(l;m) obtained by Equation (9) is greater than the threshold value TH. 1 is substituted for VAD(l), and if the value is smaller than the threshold value TH, it is determined as a non-speech section and 0 is substituted for the speech section determination result VAD(l).

The threshold value TH should be able to determine the presence or absence of voice, and various methods can be widely applied. , and a fixed threshold TH that uses the average amplitude value of the first several frames as the threshold TH may be used, or as shown in equation (12), x_fram_amp(l) is filtered using a time constant filter for each frame A threshold TH(l) that fluctuates to .

In the expression (12), a is a coefficient of the time constant filter and takes a value of 0 or more and 1 or less. In the equation (12), a value close to 1 is desirable for slow updating of the threshold value (for example, a=0.9), and a value close to 0 is desirable for speeding up updating of the threshold value (for example, value such as a=0.1).

Various methods can be widely applied as the means for judging whether it is a speech section or a non-speech section. It may be determined by a method such as determining whether it is a speech segment. The voice segment determination unit 205 outputs the determination result as to whether it is a voice segment or a non-voice segment to the DB accumulation determination unit 206 and the masker signal generation unit 204A.

The DB accumulation determination unit 206 extracts the The output frame-divided microphone input signal x_fram(l;m) is output to the input signal DB 202, and the frame-divided microphone input signal x_fram(l;m) is determined to be a non-voice segment by the voice segment determination unit 205. When (VAD(l)=0), the frame-divided microphone input signal x_fram(l;m) is not output.

The masker signal generation unit 204A extracts a plurality of selected masker edge signals from the input signal DB 202 based on the speech interval determination result VAD(l) of the speech interval determination unit 205 and the selection result T(k) of the signal selection unit 203. A frame is read, and a masker signal is generated and output from the read masker edge signals of a plurality of frames. The masker signal generation unit 204A outputs a masker signal according to formulas (6) and (13).

In equation (13), ha(l;m) is the masker signal generated by the masker signal generator 204A. Expression (13) expresses the selection result T (k) is used to read a plurality of frames of masker edge signals from the input signal DB 202, and the masker signal h(l;m) is generated using the readout masker edge signals of the plurality of frames, ha(l;m). ), and when the microphone input signal x_fram(l;m) is determined to be a non-speech section (when VAD(l)≠1), silence is substituted for ha(l;m).

The masker signal generator 204 outputs the output signal y(n) to the sound output terminal OUT in accordance with equation (14).

(B-3) Effects of Second Embodiment According to the second embodiment, the following effects can be obtained.

In the sound masking device 100A of the second embodiment, by accumulating the speech of the target speaker U1 in the input signal DB 202 only when it is determined to be in the speech period, noise unrelated to the speech of the target speaker U1 is input. Since it is no longer stored in the signal DB 202 and selected as a masker side signal, it is possible to generate a masker signal using only the voice of the target speaker U1, thereby maintaining a high masking effect.

Further, in the sound masking device 100A of the second embodiment, a masker signal is generated using a plurality of frames of past frame-divided microphone input signals stored in the input signal DB only when it is determined to be in a speech period. and output. Thereby, it is possible to configure so that the masker signal is output only when the voice is input.

(C) Third Embodiment Hereinafter, a third embodiment of the sound processing device, sound processing program and sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to a sound masking device will be described.

(C-1) Configuration of Third Embodiment FIG. 6 is a block diagram showing the functional configuration of a sound masking device 100B according to the third embodiment. In FIG. 6, the same reference numerals or corresponding reference numerals are given to the same or corresponding portions as those in FIG.

In the following, the third embodiment will be described with a focus on differences from the first and second embodiments, and descriptions of portions that overlap with the first and second embodiments will be omitted.

A sound masking device 100B of the third embodiment differs from that of the second embodiment in that the sound masking processing section 200A is replaced with a sound masking processing section 200B.

In the sound masking processing section 200B, the input signal DB 202, the signal selection section 203, and the masker signal generation section 204A are replaced with the input signal DB 202A, the signal selection section 203A, and the masker signal generation section 204B, respectively. It differs from the second embodiment in that a section 208 is added.

In the sound masking device 100B of the third embodiment, since the pitch estimation unit 205 and the class determination unit 208 are added, the pitch estimation of the frame-divided microphone input signal, the accumulation method of the frame-divided microphone input signal, the masker This embodiment differs from the second embodiment in that the method of selecting signals to be used for signal generation and the method of generating masker signals are different.

The pitch estimator 207 estimates the pitch (speech pitch) of the frame-divided microphone input signal from the frame-divided microphone input signal and the voice segment determination result, and outputs the estimated pitch.

Based on the result of the pitch estimated by the pitch estimation unit 207, the class determination unit 208 inputs the frame-divided microphone input signal only when it is determined that the frame-divided microphone input signal is stored in the input signal DB 202A. When it is determined not to output the frame-divided microphone input signal to the input signal DB 202A and store the frame-divided microphone input signal in the class corresponding to the pitch of the input signal DB 202A. No output.

The input signal DB 202A is storage means for accumulating frame-divided microphone input signals for each class corresponding to the pitch.

The signal selection unit 203A selects a masker bare edge signal from past frame-divided microphone input signals accumulated for each class, and outputs a selection result.

The masker signal generation unit 204B reads out a plurality of frames of the selected masker edge signal from the class corresponding to the pitch of the input signal DB 202A based on the result of the voice section determination and pitch estimation and the selection result, and converts the read-out plurality of frames. A masker signal is generated from the masker bare edge signal and output.

In addition, in the third embodiment, the configuration may be such that the speech segment determination unit 205 is excluded, as in the first embodiment.

(C-2) Operation of Third Embodiment Next, the operation of the sound masking device 100B (acoustic processing method according to the embodiment) of the third embodiment having the configuration described above will be described in detail.

The basic operation of the sound masking process in the sound masking device 100B according to the third embodiment is the same as the sound masking process described in the first and second embodiments.

In the following, the processing operations of the pitch estimation unit 207, class determination unit 208, input signal DB 202A, signal selection unit 203A, and masker signal generation unit 204B, which are different from the second embodiment, will be described in detail.

The frame dividing unit 201 divides the microphone input signal x(n) into processing frames, and divides the frame-divided microphone input signal x_fram(l;m) into the speech period determining unit 205, the DB accumulation determining unit 206, and the pitch estimating unit 207. output to

The speech section determination unit 205 uses the frame-divided microphone input signal x_fram(l;m) to determine whether it is a speech section or a non-speech section. , pitch estimation section 207 and masker signal generation section 204B.

Only when the voice segment determining unit 205 determines that the frame-divided microphone input signal x_fram(l;m) is a voice segment, the DB accumulation determining unit 206 performs frame-divided microphone input signal x_fram output from the frame dividing unit 201. (l; m) is output to the class determination unit 208, the signal selection unit 203A, and the masker signal generation unit 204B. , the frame-divided microphone input signal x_fram(l;m) is not output.

The pitch estimation unit 207 estimates the pitch of the frame-divided microphone input signal x_fram(l;m) only when the speech period determination unit 205 determines that the frame-divided microphone input signal x_fram(l;m) is in a voice period. do. For example, the pitch estimating means obtains the autocorrelation function x_fram_corr(l) of the frame-divided microphone input signal x_fram(l;m) according to equation (15), and calculates the autocorrelation function x_frame_corr(l) according to equation (16). You may make it estimate using.

In equation (16), pitch(l) is the estimated pitch and fs is the sampling frequency. Various methods can be widely applied as the pitch estimation method. You can calculate. Pitch estimation section 205 outputs the estimated pitch pitch(l) to class determination section 208 and masker signal generation section 204B.

Based on the pitch estimated by pitch estimation section 207, class determination section 208 determines whether or not to store the frame-divided microphone input signal in input signal DB 202A. In class determination section 208, the method of determining whether or not to store in input signal DB 202A is not limited. For example, the pitch pitch (l) estimated by the pitch estimation unit 207 is classified into classes at intervals (grids) of 100 Hz such as 100 Hz or less, 101 Hz to 200 Hz, 201 Hz to 300 Hz, 301 Hz to 400 Hz, 401 Hz to 500 Hz, and 500 Hz or more. . Then, when the frequency is 100 Hz or less or 500 Hz or more, it may be determined that the input signal DB 202A is not to be stored, and in other cases, it may be determined that the input signal DB 202A is to be stored. Further, for example, in the input signal DB 202A, the class frequency interval (grid) may be widened as the frequency increases.

Only when it is determined that the frame-divided microphone input signal is stored in the input signal DB 202A, the class determination unit 208 classifies the frame-divided microphone input signal x_fram(l;m) into a class according to the pitch of the input signal DB 202A. , and if it is determined not to store the frame-divided microphone input signal in the input signal DB 202A, the frame-divided microphone input signal x_fram(l;m) is not output to the class corresponding to the pitch of the input signal DB 202A. .

Only when the microphone input signal x_fram(l;m) is output from the class determination unit 208, the input signal DB 202A converts the output frame-divided microphone input signal x_fram(l;m) into the following equations (17) and (18): Accumulates in the input signal DB 202A for each class corresponding to the pitch according to the formula.

In equation (17), DB' (p; i; m) is the input signal DB, m is the discrete time in the frame (m = 0, 1, 2, ..., M-1), i (pitch (l)) is an index for each class of the database, and I is the length of the database. i(pitch(l)) is incremented when data is accumulated in the class as shown in equation (18).

The signal selection unit 203A selects a masker bare edge signal from past frame-divided microphone input signals accumulated for each class in the input signal DB 202A. The signal selection unit 203A selects the selection result Ta(k) as shown in equation (19), for example.

In the equation (19), k (k=1, 2, . , I) is the M O D function that returns the remainder when ik is divided by I. Expression (19) returns the remainder when divided by I, so that the selection result Ta(k) takes values from 0 to I−1.

Various methods can be widely applied to the method of calculating the selection result Ta(k). For example, as shown in equation (20), which frame to use may be randomly selected.

In equation (20), rand is a function that generates random numbers for natural number k. Expression (20) returns the remainder when the random number generated by rand(k) using the MOD function is divided by I, and the selection result Ta(k) becomes a value from 0 to I−1. The signal selection unit 203A outputs the selection result Ta(k) to the masker signal generation unit 204. FIG.

The masker signal generation unit 204B, based on the voice segment determination result VAD(l) of the voice segment determination unit 205, the pitch picth(l) estimated by the pitch estimation unit 207, and the selection result Ta(k) of the signal selection unit 203A, A plurality of frames of masker side signals are read from a class corresponding to the pitch of the input signal DB 202A, and a masker signal is generated from the read masker side signals of the plurality of frames and output. The masker signal generation unit 204B outputs a masker signal according to formulas (21) and (22).

In equation (21), hb(l;m) is the masker signal, F0_MAX is the maximum pitch value, and in equation (22), h'(l;m) is the masker signal K generated from the input signal DB. It is the number of selections of the masker bare side signal (the number of additions of the voice when generating the masker signal). Expression (21) expresses the pitch pitch estimated by the pitch estimating unit 207 when the microphone input signal x_fram(l;m) is determined to be in the speech interval by the speech period determining unit 205 (when VAD(l)=1). Only when (l) is greater than 0Hz and F0_MAX or less, the masker signal h'(l;m) is generated, and otherwise silence is generated and substituted into hb(l;m). . (22) is a method of superimposing and generating past frame-divided microphone input signals stored in the input signal DB 202A for each class corresponding to the pitch.

Various methods can be widely applied to the masker signal generation method in the masker signal generation unit 204B. For example, in the masker signal generation unit 204B, past frame-divided microphone input signals stored for each class in the input signal DB 202A are time-reversed and superimposed as time processing, and then the masker signal h'(l;m) is generated. Alternatively, the masker signal h'(l;m) may be generated by superimposing the past frame-divided microphone input signal accumulated in the input signal DB 202 with a time delay as time processing. However, which past frame to use may be randomly determined to generate the masker signal h'(l;m).

Then, the masker signal generator 204B outputs the output signal y(n) to the sound output terminal OUT according to the expression (23).

(C-3) Effects of Third Embodiment According to the third embodiment, the following effects can be obtained.

In the sound masking apparatus 100B of the third embodiment, the speech of the target speaker U1 is stored in the input signal DB 202A for each class according to pitch, and the past voices stored in the input signal DB for each class according to pitch are stored in the input signal DB 202A. A masker signal is generated and output using multiple frames of the microphone input signal. As a result, in the sound masking device 100B of the third embodiment, the acoustic features of the masker signal and the voice of the target speaker U1 are brought closer to each other, so that the masking effect can be further enhanced.

(D) Fourth Embodiment Hereinafter, a fourth embodiment of the sound processing device, sound processing program and sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to a sound masking device will be described.

(D-1) Configuration of the Fourth Embodiment FIG. 7 is a block diagram showing the functional configuration of a sound masking device 100C according to the fourth embodiment. In FIG. 7, the same reference numerals or corresponding reference numerals are given to the same or corresponding portions as those in FIG.

In the following, the fourth embodiment will be described with a focus on the differences from the first to third embodiments, and descriptions of portions that overlap with the first to third embodiments will be omitted.

A sound masking device 100C of the fourth embodiment differs from that of the third embodiment in that the sound masking processing section 200B is replaced with a sound masking processing section 200C.

In the sound masking processing unit 200C, the signal selection unit 203A and the masker signal generation unit 204B are replaced with the signal selection unit 203B and the masker signal generation unit 204C, and a third party audio signal DB 209 and a usage DB determination unit 210 are added. This point differs from the first to third embodiments.

In the sound masking device 100C of the fourth embodiment, since the number of the third party audio signal DB 209 and the usage DB determination unit 210 is increased, the method of accumulating the third party audio signal in the third party audio signal DB 209, the sound masking device It differs from the first to third embodiments in that the DB used when 100C operates, the method of selecting the signal used to generate the masker signal, and the method of generating the masker are different.

For example, the third-party audio signal DB 209 accumulates sample audio signals (hereinafter referred to as "third-party audio signals") in advance, divides the accumulated third-party audio signals into frames, and divides them into frames. This is a database in which divided third-party speech signals are classified into classes according to pitch and accumulated.

The use DB determination unit 210 determines whether or not a predetermined amount or more (sufficient) of frame-divided microphone input signals is accumulated in each class of the input signal DB 202A, and outputs the determination result.

The signal selection unit 203B selects a masker bare edge signal from past frame-divided microphone input signals stored for each class in the input signal DB 202A or the third-party audio signal DB 209, and outputs the selection result.

The masker signal generation unit 204C generates the input signal DB 202A and the input signal DB 202A when it is determined that the input signal DB 202A has accumulated a predetermined amount or more based on the results of the speech section determination and pitch estimation, the use DB determination result, and the selection result. When it is determined that the DB 202A does not store a predetermined amount or more, the third-party speech signal DB 209 is selected, and the pitch of the selected database (hereinafter, the selected database is referred to as the "selected database") of the masker base signal. A plurality of frames are read from a class corresponding to the class, and a masker signal is generated from a masker edge signal from the read plurality of frames and output.

In the fourth embodiment, the pitch estimator 205 may be excluded, and the input signal DB 202A or the third-party speech signal DB 209 may be stored without being classified. Also, in the fourth embodiment, the speech segment determination unit 205 may be excluded.

(D-2) Operation of Fourth Embodiment Next, the operation of the sound masking device 100C (acoustic processing method according to the embodiment) according to the fourth embodiment having the configuration described above will be described in detail.

The basic operation of the sound masking process in the sound masking device 100C according to the fourth embodiment is the same as the sound masking process described in the first to third embodiments.

In the following, a detailed description will be given centering on the processing operations of the third-party audio signal DB 209, the use DB determination unit 210, the signal selection unit 203B, and the masker signal generation unit 204C, which are different from the first to third embodiments.

The sound masking processing unit 200C of the sound masking device 100C accumulates the audio signal in the third party audio signal DB 209 before performing the sound masking process.

For example, as shown in FIG. 8, third-party audio signal sample data AS composed of a database in which audio signal samples are accumulated in advance (for example, a commercially available audio signal database) is processed by the sound masking processor. 200C to construct a third party audio signal DB 209. FIG.

In FIG. 8, an audio signal based on the third party audio signal sample data AS is input to the sound masking processing unit 200C, the sound masking device 100C starts operating, and the audio signal based on the third party audio signal sample data AS , frame division, voice section determination, pitch estimation, DB accumulation determination, and class determination are performed in the same manner as in the above embodiments, and stored in the third-party voice signal DB 209 .

Note that the third party audio signal DB 209 may be constructed by the same process as the accumulation process of the input signal DBs 202 and 202A in each of the above embodiments.

Also, the data recording medium on which the third party audio signal sample data AS is recorded is not limited.

Furthermore, as a sample for constructing the third party audio signal DB 209, the microphone 101, the microphone amplifier 102, and the AD converter 103 are connected to the sound input terminal IN instead of the prerecorded third party audio signal sample data AS. It is also possible to connect and store utterances to a plurality of people (samples of third-party voice signals are stored via the microphone 101), or data created by processing on another PC (third-party sample data of a voice signal) may be used (for example, copied by communication or a data recording medium).

Then, when the third party audio signal DB 209 has sufficiently accumulated data based on the third party audio signal (accumulated a predetermined amount or more of data), the sound masking device 100C prepares the third party audio signal DB 209. End the process and pause until the sound masking process begins.

It should be noted that when the third party audio signal DB 209 has sufficiently accumulated data based on the third party audio signal (a predetermined amount or more of data has been accumulated), the sound masking device 100C prepares the third party audio signal DB 209. Alternatively, the processing may be terminated and the sound masking processing may be started.

At this time, the method of determining whether or not a predetermined amount or more of data has been accumulated in the third party audio signal DB 209 is not limited. good.

The sound masking device 100C starts sound masking processing, and when the target speaker U1 speaks toward the microphone 101, the voice is input to the microphone 101. FIG.

An analog sound signal input to the microphone 101 is amplified by the microphone amplifier 102, converted from the analog signal to a digital signal by the AD converter 103, and is supplied to the sound input terminal IN of the sound masking processing unit 200C as the microphone input signal x(n). ).

When the microphone input signal x(n) starts to be input to the sound input terminal IN of the sound masking processing unit 200C, it is input to the frame division unit 201. FIG.

The frame dividing unit 201 divides the microphone input signal x(n) into processing frames, and divides the frame-divided microphone input signal x_fram(l;m) into the speech period determining unit 205, the DB accumulation determining unit 206, and the pitch estimating unit. 207.

The speech section determination unit 205 uses the frame-divided microphone input signal x_fram(l;m) to determine whether it is a speech section or a non-speech section. , the pitch estimation unit 207 and the masker signal generation unit 204C.

Only when the voice segment determining unit 205 determines that the frame-divided microphone input signal x_fram(l;m) is a voice segment, the DB accumulation determining unit 206 performs frame-divided microphone input signal x_fram output from the frame dividing unit 201. (l;m) is output to the class determination unit 208 signal selection unit 203B and the masker signal generation unit 204C, and the frame-divided microphone input signal x_fram(l;m) is determined as a non-speech interval by the voice interval determination unit 205. , the frame-divided microphone input signal x_fram(l;m) is not output.

The pitch estimation unit 207 estimates the pitch of the frame-divided microphone input signal x_fram(l;m) only when the speech period determination unit 205 determines that the microphone input signal x_fram(l;m) is in a voice period. The resulting pitch is output to masker signal generation section 204C and pitch estimation section 207 .

Based on the pitch estimated by pitch estimating section 207, class determining section 208 classifies the frame-divided microphone input signal into input signal DB 202A only when it is determined that the frame-divided microphone input signal is stored in input signal DB 202A. are output and stored in classes corresponding to the pitches.

Only when the microphone input signal x_fram(l;m) is output from the class determination unit 208, the input signal DB 202A converts the output frame-divided microphone input signal x_fram(l;m) into the following equations (17) and (18): Accumulates in the input signal DB 202A for each class corresponding to the pitch according to the formula.

The use DB determination unit 210 determines whether a predetermined amount or more of data has been accumulated (sufficiently accumulated) in past frame-divided microphone input signals x_fram(l;m) in each class of the input signal DB 202A. Output. The use DB determination unit 210 determines whether or not a predetermined amount or more of the microphone input signal x_fram(l;m) divided into frames is accumulated in the input signal DB 202A according to the following equation (24), for example.

In expression (24), flag(l) is the determination result. Formula (24) assigns 1 to the determination result flag(l) when a predetermined amount or more of data has been accumulated, and determines Assign 0 to the result flag(l).

It should be noted that various methods can be widely applied as a method of determining whether or not a predetermined amount of data or more is accumulated in the input signal DB 202A in the use DB determination unit 210. FIG. For example, the use DB determination unit 210 counts the number of times the microphone input signal x_fram(l;m) divided into frames is accumulated in the input signal DB, and when the count exceeds the threshold value, a predetermined amount of data or more is accumulated. Alternatively, the number of times accumulated for each class may be counted, and when the counted number exceeds the threshold value for all classes, it may be judged that sufficient accumulation is achieved.

Further, various methods can be widely applied as a method for starting determination in use DB determination section 210 as to whether or not a predetermined amount or more of data has been accumulated in input signal DB 202A. For example, the determination may be started after the operation of the sound masking device 100C is started, or the determination may be started when a predetermined time has passed since the operation of the sound masking device 100C is started. Then, the used DB determination unit 210 outputs the determination result flag(l) to the signal selection unit 203B.

The signal selection unit 203B selects from the input signal DB 202A from the determination result flag(l) output from the use DB determination unit 210, or from the past frame-divided microphone input signal accumulated for each class in the third-party audio signal DB 209. Select the masker bare edge signal. The signal selection unit 203A selects the selection result Tb(k) as shown in equation (25), for example.

(25), k (k=1, 2, . , I) is the M O D function that returns the remainder when ik is divided by I. By returning the remainder when dividing by I, the selection result Tb(k) becomes a value from 0 to I−1. Expression (25) is obtained by the use DB determination unit 210, when it is determined that the input signal DB 202A does not store a predetermined amount or more (when flag(l)=0), the third party audio signal DB 209 outputs masker elements. A masker edge signal is selected from the input signal DB 202A when it is determined that an edge signal is selected and accumulated in the input signal DB 202A by a predetermined amount or more (when flag(l) is other than 0). .

Various methods can be widely applied to the method of calculating the selection result Tb(k). For example, as shown in equation (26), which frame to use may be randomly selected.

In equation (26), rand is a function that generates random numbers for natural number k. Expression (26) returns the remainder when the random number generated by rand(k) using the MOD function is divided by I, and the selection result Tb(k) takes values from 0 to I−1. Signal selection section 203B outputs selection result Tb(k) to masker signal generation section 204 .

The masker signal generation unit 204C uses the voice segment determination result VAD(l) of the voice segment determination unit 205, the pitch picth(l) estimated by the pitch estimation unit 207, the selection result Tb(k) of the signal selection unit 203B, and the used DB determination. Based on the determination result flag(l) of the unit 210, when it is determined that the input signal DB 202A has accumulated the predetermined amount or more, the input signal DB 202A is determined not to have accumulated the predetermined amount or more. A third-person voice signal DB 209 is selected, and a plurality of frames of masker return signals are read out from the class corresponding to the pitch of the selected database. Then, it generates and outputs a masker signal from the plurality of read frames. The masker signal generator 204C outputs a masker signal according to, for example, equations (27) and (28).

In equation (27), hc(l;m) is the masker signal, F0_MAX is the maximum pitch value, and in equation (28), DB2(p;l;m) is the third party audio signal DB, h'' (l; m) is the masker signal generated from the third party audio signal DB and the input signal DB, and K is the number of selections of masker side signals (the number of audio additions when generating the masker signal). Expression (27) expresses the pitch pitch estimated by the pitch estimator 207 when the microphone input signal x_fram(l;m) is determined to be in the voice segment by the voice segment determination unit 205 (when VAD(l)=1). Only when (l) is greater than 0 Hz and equal to or less than F0_MAX, the masker signal h''(l;m) is generated, and otherwise silence is generated and substituted into hc(l;m). . (28) When the use DB determination unit 210 determines that the input signal DB 202A does not store a predetermined amount or more (flag(l) = 0), the masker bare edge signal is A plurality of frames are read out from the signal DB 209, a masker signal is generated from the read masker edge signals of the plurality of frames, and when it is determined that a predetermined amount or more is accumulated in the input signal DB 202A (flag(l) = other than 0). (time) reads a plurality of frames of masker side signals from the input signal DB 202A, and generates a masker signal from the read masker side signals of the plurality of frames.

In addition, in the masker signal generator 204C, various methods can be widely applied to the method of generating the masker signal. For example, in the masker signal generation unit 204C, past frame-divided microphone input signals accumulated for each class corresponding to the pitch of the selection database are time-reversed and superimposed as time processing, and then the masker signal h''(l ;m), or the masker signal h''( l;m) may be generated, or a past frame to be used may be randomly determined to generate the masker signal h''(l;m).

Then, the masker signal generator 204C outputs the generated masker signal hc(l;m) to the sound output terminal OUT as the output signal y(n) according to the equation (29).

(D-3) Effects of Fourth Embodiment According to the fourth embodiment, the following effects can be obtained.

The sound masking apparatus 100C of the fourth embodiment uses the third party audio signal DB 209 to generate and output a masker signal at the start of operation, and when the input signal DB 202A has accumulated enough input signals, the input signal DB 202A A masker signal is generated and output using a plurality of frames of accumulated past microphone input signals. As a result, the sound masking apparatus 100C can generate a masker signal whose acoustic features are close to the acoustic features of the voice of the target speaker U1 from the start of operation, so that the masking effect can be further enhanced.

(E) Fifth Embodiment Hereinafter, a fifth embodiment of the sound processing device, sound processing program, and sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to a sound masking device will be described.

(E-1) Configuration of Fifth Embodiment FIG. 9 is a block diagram showing the functional configuration of a sound masking device 100D according to the fifth embodiment. In FIG. 9, the same reference numerals or corresponding reference numerals are assigned to the same or corresponding portions as those in FIG.

In the following, the fifth embodiment will be described with a focus on the differences from the first embodiment, and descriptions of portions that overlap with the first embodiment will be omitted.

The sound masking device 100D of the fifth embodiment differs from the first embodiment in that the sound masking processing section 200 is replaced with a sound masking processing section 200D. The sound masking processor 200D differs from the first embodiment in that the masker signal generator 204 is replaced with a masker signal generator 204D.

The sound masking device 100D of the fifth embodiment differs from the sound masking device 100 of the first embodiment in that the method of generating the masker signal by the masker signal generator 204D is different.

The masker signal generation unit 204D reads a plurality of frames of the selected masker side signal from the input signal DB 202, generates a masker signal from the read masker side signals of the plurality of frames, and outputs the masker signal.

(E-2) Operation of Fifth Embodiment Next, the operation of the sound masking device 100D (acoustic processing method according to the embodiment) having the configuration described above according to the fifth embodiment will be described in detail.

The basic operation of the sound masking process in the sound masking device 100D according to the fifth embodiment is the same as the sound masking process described in the first embodiment.

In the following, a detailed description will be given centering on the processing operation in the masker signal generator 204D, which is different from the first embodiment.

The masker signal generation unit 204D generates a masker signal using the past frame-divided microphone input signal stored in the input signal DB 202 . As a masker signal generation method performed by the masker signal generation unit 204D, for example, a pseudo echo (hereinafter referred to as “pseudo echo ) and use it as a masker signal.

The masker signal generator 204D generates a pseudo echo and outputs the generated pseudo echo as a masker signal. A pseudo echo is generated according to, for example, equations (30) and (31).

In equations (30) and (31), c (c=1, 2, . 1)) is a parameter indicating how much the past frame-divided microphone input signal stored in the input signal DB 202 when generating a pseudo echo is to be delayed, and α is a reduction coefficient (0.0<α<1 .0). Equation (31) is a signal generated by multiplying the past frame-divided microphone input signal accumulated in the input signal DB 202 by an attenuation coefficient while shifting the readout of a plurality of frames, and then superimposing the signal. The delay time of the pseudo echo may be, for example, about 0.1 [seconds] to 1.0 [seconds] (approximately 4800 [samples] to 48000 [samples] at 48 kHz sampling). For example, in equation (30), when C=3, p=50, and α=0.5, the microphone input signal of the previous frame accumulated in the input signal DB 202 and the microphone input signal accumulated in the input signal DB 202 are A signal obtained by advancing the microphone input signal of the past two frames before by 50 samples and multiplying it by an attenuation coefficient α (=0.5), and advancing the microphone input signal of the past three frames accumulated in the input signal DB 202 by 100 samples. , a pseudo echo e(l;m) is generated by superimposing a signal multiplied by an attenuation coefficient α ² (=0.25).

Various methods can be widely applied to the pseudo echo generation method in the masker signal generation unit 204D. In the masker signal generation unit 204D, for example, as shown in equations (32) and (33), the past frame-divided microphone input signal accumulated in the input signal DB 202 is time-reversed as time-processed signals. Alternatively, the pseudo echo e(l;m) may be generated by randomly determining which past frame to use.

Then, the masker signal generator 204D outputs the generated pseudo echo e(l;m) to the sound output terminal OUT as the output signal y(n) according to the equation (34).

(E-3) Effects of Fifth Embodiment According to the fifth embodiment, the following effects can be obtained.

The sound masking device 100D of the fifth embodiment stores the voice of the target speaker U1 in the input signal DB, and uses the past frame-divided microphone input signal stored in the input voice signal DB for a plurality of frames. A pseudo echo is generated and the pseudo echo is output as a masker signal. As a result, in the sound masking device 100D, the acoustic features of the masker signal are brought closer to the acoustic features of the voice of the target speaker U1, thereby improving the masking effect and preventing the leakage of the content of the conversation. In other words, the sound masking apparatus 100 of the fifth embodiment also analyzes the acoustic characteristics of the target speaker U1 by generating the masker signal using the voice signal of the target speaker U1 stored in the input signal DB. Even if the above is not performed, the acoustic features of the masker signal are close to the acoustic features of the voice signal of the target speaker U1, so that a high masking effect can be obtained.

(F) Sixth Embodiment Hereinafter, a sixth embodiment of the sound processing device, sound processing program, and sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to a sound masking device will be described.

(F-1) Configuration of Sixth Embodiment FIG. 10 is a block diagram showing the functional configuration of a sound masking device 100E according to the sixth embodiment. In FIG. 10, the same reference numerals or corresponding reference numerals are assigned to the same or corresponding portions as in FIG. 9 described above.

In the following, the fifth embodiment will be described with a focus on differences from the fifth embodiment, and descriptions of portions that overlap with the fifth embodiment will be omitted.

A sound masking device 100E of the sixth embodiment differs from that of the fifth embodiment in that the sound masking processing section 200D is replaced with a sound masking processing section 200E. The sound masking processing unit 200E includes a frame division unit 201, a first input signal DB 211, a second input signal DB 212, a first signal selection unit 213, a second signal selection unit 214, a first masker generation unit 215, It has a second masker generator 216 and a masker signal mixer 217 .

The sound masking device 100E of the sixth embodiment differs from the first and fifth embodiments in the method of generating the masker signal. Specifically, the sound masking processing unit 200E generates two types of masker signals from the input microphone input signal, and outputs the superimposed signal as the masker signal.

Since the first input signal DB211 and the second input signal DB212 are similar to the input signal DB202 of the first embodiment, detailed description thereof will be omitted. Also, the first signal selection unit 213 and the second signal selection unit 214 are similar to the signal selection unit 203 of the first embodiment except that they have different names, so detailed description thereof will be omitted.

The first masker generator 215 generates and outputs a masker signal from the first input signal DB 211 by a method different from that of the second masker generator 216, which will be described later.

The second masker generator 216 generates and outputs a masker signal from the second input signal DB 212 by a method different from that of the first masker generator 215 .

The masker signal mixing unit 217 mixes the masker signals output from the respective masker signal generation units to generate a masker signal to be finally output.

For the first input signal DB211 and the second input signal DB212, the same data as the DB of both methods (for example, the first input signal DB211 and the second input signal DB212 have the same data as the input signal DB202 in the first embodiment). , or different data (for example, the first input signal DB211 is the input signal DB202 in the first embodiment, and the second input signal DB212 is the third embodiment). (data similar to the input signal DB 202A in the form) may be accumulated.

(F-2) Operation of Sixth Embodiment Next, the operation of the sound masking device 100E (acoustic processing method according to the embodiment) of the sixth embodiment having the configuration described above will be described in detail.

The basic operation of the sound masking process in the sound masking device 100E according to the sixth embodiment is the same as the sound masking process described in the fifth embodiment.

The operation of the sound masking device 100E according to the sixth embodiment of the invention will be described in detail.

The first masker generator 215 uses the past frame-divided microphone input signal stored in the first input signal DB 211 to generate a masker signal by a method different from that of the second masker generator 216 .

The second masker generation unit 216 generates a masker signal by a method different from that of the first masker generation unit 215 using the past frame-divided microphone input signal stored in the second input signal DB 212 .

For example, the first masker generator 215 generates the masker signal h(l;m) as shown in equation (6) or (7), and the second masker generator 216 generates the masker signal h(l;m) as shown in equation (32) , or a pseudo echo e(l;m) as shown in equation (34) may be generated as a masker signal.

The masker signal mixing unit 217 mixes the masker signals output from the first masker generation unit 215 and the second masker generation unit 216, and outputs a masker signal mix(l;m). The masker signal mixing section 217 may mix the masker signals output from the first masker generation section 215 and the second masker generation section 216, for example, based on the equation (35).

In equation (35), β (0.0≤β≤1.0) is a parameter indicating which masker signal is used more. When the masker signal of the first masker generator 215 is desired to be used more, β is preferably close to 1 (for example, a value such as β=0.9), and the masker signal of the second masker generator 216 is used more often. β should be close to 1 (for example, β=0.1).

The masker signal mixing unit 217 outputs the mixed masker signal mix(l;m) as the output signal y(n) according to the equation (36).

(F-3) Effects of Sixth Embodiment According to the sixth embodiment, the following effects can be obtained.

In the sound masking device 100E of the sixth embodiment, the voice of the target speaker U1 is accumulated in the first input signal DB 211 and the second input signal DB 212, and the past microphone input signals accumulated in each input signal DB are are used for a plurality of frames, masker signals are generated by different methods, the amount of mixing is adjusted, and the signals are mixed and output. As a result, the sound masking apparatus 100E of the sixth embodiment can adjust the mixing amount of the masker sound of the method having a high masking effect on the target speaker U1, so that the masking effect can be further enhanced.

(G) Other Embodiments The present invention is not limited to the above-described embodiments, and modified embodiments such as those illustrated below can also be included.

(G-1) For example, the sound masking device of the present invention may be installed in a device that prevents the contents of a conversation from leaking out to people other than the target audience in a teleconference. In this case, in the sound masking device, the target speaker U1 is the person speaking in the conference call.

(G-2) In each of the above embodiments, the sound masking unit of the sound masking device may be configured to be processed by a processing device (for example, a server, etc.) on the network.

(G-3) In each of the above embodiments, the sound masking device includes an audio device (microphone, microphone amplifier, AD converter, speaker, speaker amplifier, and DA converter). The masking device may be manufactured without the audio device, and the audio device may be separately connected at the site of actual use. That is, the sound masking device may include at least a sound masking processing unit.

100, 100A, 100B, 100C, 100D, 100E... Sound masking device, 101... Microphone, 102... Microphone amplifier, 103... AD converter, 104... Speaker, 105... Speaker amplifier, 106... DA converter, 107... Speaker, 200, 200A, 200B, 200C, 200D, 200E... Sound masking processing unit, 201... Frame division unit, 202, 202A... Input signal DB, 203, 203A, 203B... Signal selection unit, 204, 204A, 204B, 204C, 204D Masker signal generation unit 205 Speech section determination unit 206 DB accumulation determination unit 207 Pitch estimation unit 208 Club determination unit 209 Third party voice signal DB 210 Usage DB determination unit 211 First input signal DB, 212... Second input signal DB, 213... First signal selector, 216... Second signal selector, 215... First masker generator, 216... Second masker generator Part 217... Masker signal mixing part 300... Computer 301... Processor 302... Primary storage part 303... Secondary storage part.

Claims (10)

a frame dividing means for dividing a microphone input signal supplied from a microphone for picking up a voice uttered by a target speaker into predetermined lengths;
input signal accumulation means for accumulating the frame-divided microphone input signal;
signal selection means for selecting a signal to be used for generating a masker signal from past frame-divided microphone input signals accumulated in the input signal accumulation means, and for outputting a selection result;
a masker signal generating means for generating and outputting the masker signal that makes the speech uttered by the target speaker difficult to hear, using the signal used to generate the masker signal ;
pitch estimation means for estimating the pitch of the microphone input signal;
The input signal accumulation means sorts and accumulates the microphone input signal into one of a plurality of classes according to the pitch estimated by the pitch estimation means,
The masker signal generating means generates a masker signal using a microphone input signal of a class corresponding to the pitch estimated by the pitch estimating means from the input signal accumulating means.
An acoustic processing device characterized by: 2. The sound processing apparatus according to claim 1, further comprising a speaker for emitting said masker signal output by said masker signal generating means toward a person to be masked other than said target speaker. The masker signal output by the masker signal generating means is reflected by a reflecting surface, and a speaker is arranged so that the reflected sound reflected by the reflecting surface is directed toward a person to be masked other than the target speaker. The sound processing device of claim 1, further comprising: further comprising a speech section determination unit that determines whether the microphone input signal is a speech section or a non-speech section,
3. The sound processing apparatus according to claim 1, wherein the input signal accumulation means accumulates the microphone input signal only when the speech period is determined. a third party signal storage means for storing a third party voice signal obtained by picking up a voice uttered by a third party different from the target speaker;
further comprising accumulation determination means for determining whether or not a predetermined amount or more of the microphone input signal is accumulated in the input signal accumulation means,
The masker signal generation means accumulates in the third party signal accumulation means only while the accumulation judgment means judges that the input signal accumulation means does not accumulate microphone input signals of a predetermined amount or more. 5. The sound processing apparatus according to any one of claims 1 to 4 , wherein the masker signal is generated using a third party audio signal that has been recorded. The input signal accumulation means accumulates microphone input signals divided into a plurality of frames,
The masker signal generating means time-processes a signal obtained by superimposing a plurality of frames of the microphone input signal accumulated in the input signal accumulation means, or time-processes the input signal of a plurality of frames accumulated in the input signal accumulation means. 2. The sound processing device according to claim 1, wherein the signal superimposed on the signal is output as a masker signal. The masker signal generation means delays the microphone input signal stored in the input signal storage means by a predetermined amount to generate a pseudo echo, and outputs the generated pseudo echo as the masker signal. Item 1. The acoustic processing device according to item 1. The input signal accumulation means accumulates microphone input signals divided into a plurality of frames,
The masker signal generating means is
A signal obtained by superimposing a plurality of frames of the microphone input signal accumulated in the input signal accumulation means, or a signal obtained by temporally processing and superimposing the plurality of frames of the input signal accumulated in the input signal accumulation means, as a first signal. generated as a masker signal of
delaying the microphone input signal accumulated in the input signal accumulation means by a predetermined amount to generate a pseudo echo, and generating the generated pseudo echo as a second masker signal;
The sound processing apparatus according to claim 1, wherein a signal obtained by superimposing the first masker signal and the second masker signal is generated as a masker signal and output. the computer,
a frame dividing means for dividing a microphone input signal supplied from a microphone for picking up a voice uttered by a target speaker into predetermined lengths;
input signal accumulation means for accumulating the frame-divided microphone input signal;
signal selection means for selecting a signal to be used for generating a masker signal from past frame-divided microphone input signals accumulated in the input signal accumulation means, and for outputting a selection result;
a masker signal generating means for generating and outputting the masker signal that makes the speech uttered by the target speaker difficult to hear, using the signal used to generate the masker signal ;
functioning as pitch estimation means for estimating the pitch of the microphone input signal,
The input signal accumulation means sorts and accumulates the microphone input signal into one of a plurality of classes according to the pitch estimated by the pitch estimation means,
The masker signal generating means generates a masker signal using a microphone input signal of a class corresponding to the pitch estimated by the pitch estimating means from the input signal accumulating means.
A sound processing program characterized by: In the acoustic processing method,
having frame division means, input signal accumulation means, signal selection means , masker signal generation means and pitch estimation means ,
The frame dividing means divides a microphone input signal supplied from a microphone for picking up a voice uttered by a target speaker into predetermined lengths,
The input signal accumulation means accumulates the frame-divided microphone input signal,
The signal selection means selects a signal to be used for generating a masker signal from past frame-divided microphone input signals accumulated in the input signal accumulation means, and outputs a selection result;
The masker signal generating means uses the signal used to generate the masker signal to generate and output the masker signal that makes the speech uttered by the target speaker difficult to hear ,
The pitch estimation means estimates the pitch of the microphone input signal,
The input signal accumulation means sorts and accumulates the microphone input signal into one of a plurality of classes according to the pitch estimated by the pitch estimation means,
The masker signal generating means generates a masker signal using a microphone input signal of a class corresponding to the pitch estimated by the pitch estimating means from the input signal accumulating means.
An acoustic processing method characterized by:

Images