JP2021135361A - Sound processing device, sound processing program and sound processing method - Google Patents

Abstract

To provide a sound processing device, a sound processing program and a sound processing method that reduce discomfort of a listener having listened to a masker signal generated with a voice signal.SOLUTION: A sound processing device includes: means for dividing a microphone input signal supplied from a microphone for collecting the sound of a voice spoken by an object speaker, into prescribed lengths; means for creating long-time frames of prescribed lengths together with the frame-divided microphone input signal; means for accumulating the generated long-time frame signals; means for carrying out frame signal selection processing for selecting a signal to be used in generating a masker signal from the past frame-divided microphone input signals that have been accumulated; means for restricting frames to be selected, when carrying out the frame signal selection processing; and means for generating and outputting a masker signal for making listening difficult of the voice spoken by the object speaker, by using a signal to be used in generating the masker signal.SELECTED DRAWING: Figure 1

Description

The present invention relates to an acoustic processing device, an acoustic processing program, and an acoustic processing method. For example, a sound masking process used as a method for preventing the contents of a conversation from being leaked to a third party around the speaker who is speaking. Can be applied to.

In recent years, when a speaker talks with a conversation partner at a reception counter, a window, a meeting space, etc. of a facility (for example, a hospital, a pharmacy, a bank, etc.) where an unspecified number of people exist, the content of the conversation becomes the surrounding number. Leakage to three parties has become a problem.

Preventing the leakage of conversation content to a third party is called speech privacy, and the sound masking effect is used to realize speech privacy.

The sound masking effect is a state in which a certain sound (hereinafter, also referred to as "target sound") is heard, and another sound having acoustic characteristics (for example, frequency characteristics, pitch, formant, etc.) close to the target sound exists. This is a phenomenon in which the target sound becomes difficult to hear (masked). Generally, the masked sound is called "masker", and the masked sound (target sound) is also called "muskellunge".

Patent Document 1 proposes a sound masking device that uses this sound masking effect to prevent leakage of conversation content to a third party (protect speech privacy).

Japanese Unexamined Patent Publication No. 2006-2431178

However, in the voice processing method of Patent Document 1, the spectral envelope of the input voice signal of the microphone is extracted, the spectral envelope is deformed to generate a deformed spectral envelope, and the signal is combined with the spectral microstructure to be used for masker signal generation. It is used as. Therefore, in the technique described in Patent Document 1, the masker signal generated by transforming the speaker's voice signal becomes an artificial sound, and the masker signal may become an unpleasant sound.

Further, in the voice processing method described in Patent Document 1, since the masker signal is generated by transforming the input voice signal of the microphone, the words of the input voice signal of the microphone and the words of the masker signal have similar contents. For those who hear the audio and masker signals, you will hear an unpleasant sound like an echo.

In view of the above problems, an acoustic processing device, an acoustic processing program, and an acoustic processing method that reduce the discomfort of the listener listening to the generated masker signal are desired.

The first sound processing apparatus of the present invention includes (1) a frame dividing means for dividing a microphone input signal supplied from a microphone that picks up the sound spoken by the target speaker into a predetermined length, and (2) the above. A long-time frame signal creating means that combines the microphone input signals frame-divided by the frame-dividing means to create a long-time frame of a predetermined length, and (3) a long-time frame signal generated by the long-time frame signal creating means. And (4) a frame signal selection process for selecting a signal to be used for generating a masker signal from a past frame-divided microphone input signal stored in the input signal storage means. Used for frame signal selection means to be performed, (5) frame selection limiting means for limiting the frames to be selected when the frame signal selection means performs the frame signal selection process, and (6) generation of the masker signal. It is characterized by having a masker signal generation means for generating and outputting the masker signal that makes it difficult to hear the voice spoken by the target speaker using the signal.

The second sound processing program of the present invention comprises (1) a frame dividing means for dividing a microphone input signal supplied from a microphone that picks up the sound spoken by the target speaker into a predetermined length, and (1) 2) A long-time frame signal creating means for converting the microphone input signal frame-divided by the frame-dividing means into a time frame of a predetermined length, and (3) a long-time frame signal generated by the long-time frame signal creating means. Performs a frame signal selection process for selecting the input signal storage means to be stored and (4) a signal to be used for generating a masker signal from the past frame-divided microphone input signals stored in the input signal storage means. The frame signal selection means, (5) the frame selection limiting means for limiting the frame to be selected when the frame signal selection means performs the frame signal selection process, and (6) the signal used to generate the masker signal. It is characterized in that it has a masker signal generation means for generating and outputting the masker signal that makes it difficult to hear the voice spoken by the target speaker.

A third aspect of the present invention is the acoustic processing method performed by the acoustic processing apparatus. (1) The acoustic processing apparatus includes a frame dividing means, a long-time frame signal creating means, an input signal accumulating means, a frame selection limiting means, and a frame signal selection. It has means and a masker signal generating means, and (2) the frame dividing means divides a microphone input signal supplied from a microphone that picks up the sound spoken by the target speaker into a predetermined length, and (2) 3) The long-time frame signal creating means creates a long-time frame having a predetermined length by combining the microphone input signals frame-divided by the frame-dividing means, and (4) the input signal accumulating means has the length. The long-time frame signal generated by the time frame signal creating means is accumulated, and (5) the frame signal selecting means generates a masker signal from the past frame-divided microphone input signal stored in the input signal storing means. A frame signal selection process for selecting a signal to be used is performed, and (6) the frame selection limiting means limits the frames to be selected when the frame signal selection means performs the frame signal selection process. (7) The masker signal generation means uses the signal used to generate the masker signal to generate and output the masker signal that makes it difficult to hear the voice spoken by the target speaker. ..

According to the present invention, it is possible to provide an acoustic processing device, an acoustic processing program, and an acoustic processing method that reduce discomfort to a listener listening to an audio signal and a generated masker signal.

It is a block diagram which shows the functional structure of the sound masking apparatus which concerns on 1st Embodiment. It is a block diagram which showed the example of the hardware composition of the sound masking apparatus which concerns on 1st Embodiment. It is an image diagram which outputs the masker signal generated by the sound masking apparatus which concerns on 1st Embodiment. It is a block diagram which shows the functional structure of the sound masking apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the functional structure of the sound masking apparatus which concerns on 3rd Embodiment. It is a block diagram which shows the functional structure of the sound masking apparatus which concerns on 4th Embodiment.

(A) First Embodiment Hereinafter, the first embodiment of the acoustic processing apparatus, acoustic processing, and acoustic processing method of the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to the sound masking device will be described.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing a functional configuration of the sound masking device 100 according to the first embodiment.

The sound masking device 100 includes a microphone 101, a microphone amplifier 102, an AD converter 103, a DA converter 104, a speaker amplifier 105, a speaker 106, and a sound masking processing unit 200.

The microphone 101 converts air vibrations such as human voice and sound into electric signals.

The microphone amplifier 102 amplifies the electric signal received (sound picked up) by the microphone 101.

The AD converter 103 converts an electric signal (analog signal) amplified by the microphone amplifier 102 into a digital signal. Hereinafter, the digital signal output from the AD converter 103 will be referred to as a “microphone input signal”.

The sound masking processing unit 200 generates a masker signal from the input microphone input signal and outputs it.

The DA converter 104 converts an output signal (digital signal) output from the sound masking processing unit 200 into an electric signal (analog signal).

The speaker amplifier 105 amplifies the electric signal output from the DA converter 104.

The speaker 106 converts an electric signal into vibration of air and outputs it as sound.

Next, the detailed configuration of the sound masking processing unit 200 will be described.

The sound masking processing unit 200 includes a frame dividing unit 201, a long-time frame signal creating unit 202, a DB (database) writing unit 203, an input signal DB 204, a frame signal DB 205, a frame selection limiting unit 206, a frame signal selection unit 207, and a masker signal. It has a generation unit 208, a sound input terminal IN, and a sound output terminal OUT.

The sound input terminal IN is an interface (audio interface) for inputting a microphone input signal to the sound masking processing unit 200.

The frame dividing unit 201 divides the microphone input signal input to the sound masking processing unit 200 into frames having a predetermined length (hereinafter referred to as “frame length L1”) (hereinafter referred to as “divided frames”). Output. It is generally desirable to apply a frame length L1 suitable for analyzing voice. For example, in the frame dividing unit 201, the frame length L1 may be 100 to 200 msec. Then, the frame dividing unit 201 outputs the divided frame signal (hereinafter, referred to as “divided frame signal”).

The long-time frame signal creation unit 202 combines the divided frame signal with a frame having a predetermined length (hereinafter, referred to as “frame length L2”) (hereinafter, referred to as “long-time frame”) and outputs the signal. The frame length L2 (the length of combining divided frames; the number of combined divided frames) is a length that can be recognized as a word of an audio signal or a sentence (a word of an audio signal when heard by the human ear, or It is desirable to apply the length that can be judged as a sentence). For example, in the long-time frame signal creation unit 202, the frame length L2 may be a length obtained by combining 3 frames to 5 frames of the divided frame signal (for example, L2 = L2 × 3 to L2 = L2 × 5), and the sound segmentation. It may be combined so as to have a length of a unit (for example, a length in a mora unit of 1 to 2 mora), and as a time unit (for example, a length in the range of 300 to 1000 msec). Is also good. Then, the long-time frame signal creation unit 202 outputs the combined long-time frame signal (hereinafter, referred to as “long-time frame signal”).

The DB writing unit 203 writes the long-time frame signal into the frame signal DB 205 of the input signal DB 204.

The input signal DB 204 is a storage means for accumulating (holding) each long-time frame signal in the past for each long-time frame. The data format in the input signal DB 204 is not limited, but here, it is assumed that the input signal DB 204 is composed of a frame signal DB 205 that stores a past long-time frame signal.

The frame selection limiting unit 206 determines the number of frames to be limited (hereinafter, referred to as "limited number of frames"), and outputs the limited number of frames.

The frame signal selection unit 207 uses the past long-time frame signal stored in the frame signal DB 205 of the input signal DB 204 to generate a masker signal from frames before the limit number of frames of the frame selection restriction unit 206. A plurality of frames are selected as a signal (hereinafter referred to as "masker element signal"), and the selected frame is output.

Based on the selection result of the frame signal selection unit 207, the masker signal generation unit 208 reads out a plurality of frames of the masker element signal selected by the frame signal selection unit 207 from the frame signal DB 205 of the input signal DB 204, and reads out a plurality of frames. A masker signal is generated and output using the masker fragment signal of the frame.

The sound output terminal OUT is an interface (audio interface) that outputs the masker signal generated by the masker signal generation unit 208 to the DA converter 104.

The sound masking processing unit 200 may be configured entirely in terms of hardware (for example, constructed using a dedicated board or DSP (Digital Signal Processor)), or configured in terms of software using a computer. You may. Further, the sound masking processing unit 200 may be configured by installing a program (including the sound processing program according to the embodiment) on a computer having a memory and a processor, for example.

In this embodiment, the AD converter 103 and the DA converter 104 are arranged outside the sound masking processing unit 200, but the sound masking processing unit 200 is equipped with the AD converter 103 and the DA converter 104. It may be configured.

FIG. 2 shows a configuration when the sound masking processing unit 200 is realized as software (computer).

The sound masking processing unit 200 shown in FIG. 2 is configured by software using a computer 300. A program (a program including the sound processing program of the embodiment) is installed in the computer 300. The computer 300 may be a computer dedicated to an audio processing program, or may be shared with a program having another function.

The computer 300 shown in FIG. 2 has a processor 301, a primary storage unit 302, a secondary storage unit 303, a sound input terminal IN, and a sound output terminal OUT. The sound input terminal IN and the sound output terminal OUT are the same as the elements shown in FIG.

The primary storage unit 302 is a storage means that functions as a work memory (work memory) of the processor 301, and for example, a memory that operates at high speed such as a DRAM (Dynamic Random Access Memory) is applied.

The secondary storage unit 303 is a storage means for recording various data such as an OS (Operating System) and program data (including data of an acoustic processing program according to an embodiment), and is, for example, a FLASH memory or an HDD (Hard Disk). A non-volatile memory such as Drive) or SSD (Solid State Drive) is applied.

In the computer 300 of this embodiment, when the processor 301 is started, the OS and programs (including the sound processing program according to the embodiment) recorded in the secondary storage unit 303 are read and expanded on the primary storage unit 302. Run. The specific configuration of the computer 300 is not limited to the configuration shown in FIG. 2, and various configurations can be applied. For example, if the primary storage unit 302 is a non-volatile memory, the secondary storage unit 303 may be excluded.

FIG. 3 shows a microphone 101, a speaker speaking into the microphone 101 (hereinafter referred to as “target speaker”) U1, and a person other than the target speaker standing behind the target speaker U1 (target). The person to be masked for listening to the voice spoken by the speaker U1: hereinafter referred to as the "masking target person") is a diagram showing an example of the arrangement relationship between the U2 and the speaker 106 (arrangement configuration of the speaker 106). ..

In FIG. 3, the directivity of the direct sound DS (Direct Sound) output from the speaker 106 is illustrated by a dotted line. Further, in FIG. 3A, the directivity of the reflected sound RS (Reflected Sound) generated by the direct sound reflected on the floor FR (FLOOR) is illustrated by a dashed line.

In FIG. 3A, the speaker 106 is installed in front of the target speaker U1 at a height of about knee, the vibration surface of the speaker 106 facing downward, and diagonally to the surface of the floor FR. The direct sound DS is reflected on the floor FR, and the reflected reflected sound RS is directed so as to be transmitted to the masking target person U2 behind the target speaker U1. Then, the masker signal radiated from the speaker 106 is output toward the surface of the floor FR, and is reflected when it reaches the floor FR. As a result, the masker signal reflected by the floor FR is transmitted to the masking target person U2 behind the target speaker U1. At this time, the direct sound of the voice uttered by the target speaker U1 is also transmitted to the masking target U2, but is masked by the masker signal.

As described above, as the installation method of the speaker 106, if the masker signal can be installed so that the target speaker U1 cannot hear the masker signal and the masker signal can be heard by the masking target person U2, various installation methods can be widely applied. Can be done. For example, as shown in FIG. 3B, if there is a space in which the speaker 106 can be installed behind the target speaker U1, the speaker 106 is installed behind the target speaker U1 and the speaker 106 is directly installed. The vibrating surface of the speaker may be output toward the masking target person U2, or as shown in FIG. 3C, there is a space for embedding the speaker 106 in the floor FR near the masking target person U2. For example, the speaker 106 may be embedded in the floor FR so that the vibration surface of the speaker 106 is directly directed to the masking target person U2 to output the masker signal, as shown in FIG. 2 (c). If there is a space in the ceiling CE (CEILING) near the masking target person U2, the speaker 106 can be installed in the ceiling CE, and the vibration surface of the speaker 106 is directly directed toward the masking target person U2. The masker signal may be output.

(A-2) Operation of the First Embodiment Next, the operation of the sound masking device 100 of the first embodiment having the above configuration (the sound processing method of the embodiment) will be described.

When the operation of the sound masking device 100 starts and the target speaker U1 of the sound masking device 100 speaks a voice toward the microphone 101, a voice signal is input to the microphone 101.

The analog audio signal input to the microphone 101 is converted into an electric signal (analog signal), amplified by the microphone amplifier 102, converted from an analog signal to a digital signal by the AD converter 103, and the sound of the sound masking processing unit 200. It is input to the input terminal IN as a microphone input signal x (n). In the microphone input signal x (n), n is a parameter indicating a discrete time series of the input signal.

When the microphone input signal x (n) starts to be input to the sound input terminal IN of the sound masking processing unit 200, it is input to the frame dividing unit 201.

The frame division unit 201 divides the microphone input signal x (n) into the frame length L1 of the division frame signal. For example, the frame dividing unit 201 divides the microphone input signal x (n) into each divided frame according to the equation (1).
x_fram (l; m) = x (l · L1 + m) ... (1)

In the equation (1), x_fram (l; m) is the divided frame signal, l is the frame number, and m is the time in the divided frame (m = 0, 1, 2, ..., L1-1).

The frame division unit 201 outputs the division frame signal x_fram (l; m) to the long-time frame signal creation unit 202.

The long-time frame signal creation unit 202 combines the divided frame signal x_fram (l; m) with the frame length L2. The specific method by which the long-time frame signal creating unit 202 creates a long-time frame is not limited, and various methods can be applied. For example, when the divided frame signal is combined by the divided frame, the long-time frame signal creating unit 202 may create the long-time frame signal x_fram_long (s) according to the equation (2).
x_fram_long (i ・ L1 + m)
= X_fram (l-((I-1) -i); m) ... (2)

In equation (2), i is the index (i = 0, 1, 2, ..., I-1), and I is the number of divided frames used for the long-time frame signal (hereinafter referred to as "the number of frames used"). There is (I = L2 / L1).

Further, as a method for creating the long-time frame signal x_fram_long (s), for example, the long-time frame signal x_fram_long (s) may be used according to the equations (3) and (4).
x_fram_long (s) = x_fram_long (s + L1) ... (3)
x_fram_long (L2-L1 + m) = x_fram (l; m) ... (4)

In equation (3), s is the time in the frame for a long time (s = 0, 1, 2, ..., L2-L1-1). In the equation (3), the long-time frame signal x_fram_long (s) is shifted forward by the split frame length L1, and in the equation (4), the long-time frame signal x_fram_long (s) is followed by the split frame signal x_fram (l; m; m). ) Is stored. The long-time frame signal creation unit 202 may combine the divided frame signals x_fram (l; m) in time units.

Then, the long-time frame signal creation unit 202 outputs the long-time frame signal x_fram_long (s) to the DB writing unit 203.

The DB writing unit 203 writes the long-time frame signal x_fram_long (s) into the frame signal DB 205 of the input signal DB 204. For example, the DB writing unit 203 writes the long-time frame signal x_fram_long (s) into the frame signal DB205DB_singal (j; t) of the input signal DB204, respectively, according to the equations (5) and (6).

In equation (5), t is the time in the frame for a long time (t = 0, 1, 2, ..., L2-1), and j is an index when the frame signal DB205 of the input signal DB204 is written for a long time. The index to be processed (j = 0, 1, 2, ..., DB_LEN-1; primary key; long-time frame identifier), DB_LEN is the database length. As shown in the equations (5) and (6), the frame signal DB 205 writes the long-time frame signal x_fram_long (s) in the DB_singal (j; t).

The input signal DB 204 is a storage means for accumulating (holding) each past long-time frame signal.

As described above, the input signal DB 204 of this embodiment includes the frame signal DB 205. Here, each long-time frame signal x_frame_long (s) is accumulated in the frame signal DB 205.

The frame selection limiting unit 206 determines the number of limited frames. The specific method for limiting the frame signal selected by the frame selection limiting unit 206 is not limited, and various methods can be applied. For example, the frame selection limiting unit 206 determines the number of limited frames for preventing the frame signal selection unit 207, which will be described later, from selecting the long-time frame signal just accumulated in the input signal DB 204 according to the equation (7).
Limit_Fream_NUM = a × DB_LEN… (7)

In the equation (7), Limit_Fream_NUM is the number of limited frames (Limit_Fream_NUM <DB_LEN), and a is the ratio to the database length DB_LEN, which is 0.1 or more and 0.5 or less. In equation (7), when it is desired to shorten the number of limited frames, a is preferably a value close to 0.1 (for example, a value such as a = 0.1), and when it is desired to increase the number of limited frames, a is close to 0.5. A value is desirable (eg, a value such as a = 0.5).

The method for determining the limited number of frames in the frame selection limiting unit 206 is not limited, and various methods can be applied. In the frame selection limiting unit 206, for example, a fixed value Flame_NUM_CONST having a length that prevents the frame signal selection unit 207 from selecting the newly accumulated frame signal and pitch information (frame signal and pitch information within the latest predetermined time) is set. It may be set as Limit_Fream_NUM (for example, Limit_Fream_NUM = 10), it may be set as a fixed time value TIME_CONST (for example, Dream_NUM_CONST = (fs · TIME_CONST) / L2), or it may be masked by an experiment or the like in advance. While maintaining the masking effect of the masker sound based on the masker signal at the position of, set a suitable value (a value that does not make the masker sound unpleasant) that does not sound like an echo as Limit_Fream_NUM (for example, sound masking). After installing the device 100 and the speaker 106 in an actual environment, a suitable value may be set to Limit_Fream_NUM).

Then, the frame selection limiting unit 206 outputs the limited number of frames Limit_Fream_NUM to the frame signal selection unit 207.

The frame signal selection unit 207 selects a plurality of past long-time frame signals stored in the frame signal DB 205 of the input signal DB 204 as masker element signal from frames before the limit frame Limit_Fream_NUM of the frame selection restriction unit 206. .. The specific method for selecting the masker element signal by the frame signal selection unit 207 is not limited, and various methods can be applied. The frame signal selection unit 207 selects a frame according to, for example, Eq. (8).

In equation (8), T (p) is the selected frame, p (p = 0, 1 ..., SEL_NUM-1) is the index of the selected frame T (p), and SEL_NUM (SEL_NUM <= DB_LEN-1) is the masker. The number of selected elementary signals, j, is the index of the database in Eq. (6). In the equation (8), the long-time frame signal held in the frame signal DB 205 of the input signal DB 204 is selected from the frames before the limit number of frames Limit_Fream_NUM in the order of new time, and the selected long-time frame signal is selected. The formula is to assign the index number of the retained database to T (p).

Further, as a method for selecting the masker element signal, for example, the masker element signal may be randomly selected from the frames before the limited number of frames Limit_Fream_NUM.

As described above, the frame signal selection unit 207 selects a plurality of frames from the long-time frame signals held in the frame signal DB 205, and outputs the selected frame T (p) to the masker signal generation unit 208.

The masker signal generation unit 208 reads a plurality of frames from the frame signal DB 205 of the input signal DB 204 based on the frame T (p) selected by the frame signal selection unit 207, generates a masker signal, and outputs the masker signal. The specific method by which the masker signal generation unit 208 generates the masker signal is not limited, and various methods can be applied. The masker signal generation unit 208 generates the masker signal h (l; t) according to the equation (9), for example.

In the equation (9), a plurality of masker element signals selected by the frame signal selection unit 207 are read from the frame signal DB 205 of the input signal DB 204, and the read masker element signals are superimposed to superimpose the masker signal h (l; t; t). ) Is an expression to generate.

Then, the masker signal generation unit 208 performs overlap addition processing of the masker signal h (l; t) to obtain an output signal y (n) according to the equation (10), and outputs the output signal y (n) from the sound output terminal OUT.

The masker signal output from the sound output terminal OUT of the sound masking processing unit 200 is converted from a digital signal to an analog signal by the DA converter 104, amplified by the speaker amplifier 105, and then output from the speaker 106.

(A-3) Effect of First Embodiment According to the first embodiment, the following effects can be obtained.

In the sound masking device 100 of the first embodiment, a plurality of masker element signals are selected from the frames before the limited number of frames for the past long-time frame signals stored in the frame signal DB of the input signal DB. A masker signal (masking sound) is generated. As a result, in the sound masking device 100 of the first embodiment, the new long-time frame signal is not selected as the masker element signal, and the masker signal does not sound like an echo. Therefore, the generated masker signal is heard. It becomes comfortable (the discomfort for the masking target U2 is reduced).

(B) Second Embodiment Hereinafter, a second embodiment of the acoustic processing apparatus, the acoustic processing program, and the acoustic processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to the sound masking device will be described.

(B-1) Configuration of Second Embodiment FIG. 4 is a block diagram showing a functional configuration of the sound masking device 100A according to the second embodiment. In FIG. 4, the same parts or corresponding parts as those in FIG. 1 are designated by the same reference numerals or corresponding reference numerals.

In the following, the second embodiment will be mainly described with respect to the difference from the first embodiment, and the description of the part overlapping with the first embodiment will be omitted.

The sound masking device 100A of the second embodiment is different from the first embodiment in that the sound masking processing unit 200 is replaced with the sound masking processing unit 200A.

In the sound masking processing unit 200A, a voice section determination unit 209, a DB accumulation determination unit 210, and a masker signal generation determination unit 211 are added, and further, a DB writing unit 203, a frame signal selection unit 207, and a masker signal generation unit 208 are added. , DB writing unit 203A, frame signal selection unit 207A, and masker signal generation unit 208A are replaced, which is different from the first embodiment.

In the sound masking device 100A of the second embodiment, the voice section determination unit 209 and the DB accumulation determination unit 210 are added and replaced with the DB writing unit 203A, so that a long-time frame accumulated in the frame signal DB 205 of the input signal DB 204 The signal storage method is different, the masker signal generation method is different due to the addition of the masker signal generation determination unit 211, and the masker piece signal selection method is different because it is replaced by the frame signal selection unit 207A. It differs from the sound masking apparatus 100 of the first embodiment in that it differs from the sound masking apparatus 100 of the first embodiment in that the masker signal generation method is different due to the replacement with the masker signal generation unit 208A. The detailed configuration of the sound masking processing unit 200A will be described.

The sound masking processing unit 200A includes a frame division unit 201, a voice section determination unit 209, a DB accumulation determination unit 210, a long-time frame signal creation unit 202, a DB writing unit 203A, an input signal DB 204, a frame signal DB 205, and a frame selection restriction unit 206. , A frame signal selection unit 207A, a masker signal generation unit 208A, a sound input terminal IN, and a sound output terminal OUT.

The voice section determination unit 209 determines whether the divided frame signal is a voice section or a non-voice section (a section other than the voice section), and outputs a determination result.

The DB accumulation determination unit 210 determines whether or not to accumulate the frame signal for a long time in the DB based on the determination result of the voice section determination unit 209, and outputs the determination result.

The DB writing unit 203A writes the long-time frame signal into the frame signal DB 205 of the input signal DB 204 based on the determination result of the DB accumulation determination unit 210.

The masker signal generation determination unit 211 determines whether or not to generate a masker signal based on the determination result of the voice section determination unit 209, and outputs the determination result.

Based on the determination result of the masker signal generation determination unit 211, the frame signal selection unit 207A selects the past long-time frame signal stored in the input signal DB 204 from the frames before the limit number of frames of the frame selection restriction unit 206. Multiple frames are selected as masker element signals, and the selected frames are output.

In the second embodiment, the masker signal generation determination unit 211 may be excluded as in the second embodiment.

(B-2) Operation of the Second Embodiment Next, the operation of the sound masking device 100A (the sound processing method according to the embodiment) in the second embodiment having the above configuration will be described in detail.

The basic operation of the sound masking process in the sound masking device 100A according to the second embodiment is the same as the sound masking process described in the first embodiment.

In the following, the operations in the voice section determination unit 209, the DB accumulation determination unit 210, the DB writing unit 203A, the masker signal generation determination unit 211, and the frame signal selection unit 207A, which are different from the first embodiment, will be described in detail. do.

The frame division unit 201 divides the microphone input signal x (n) into processing frames, and outputs the divided frame signal x_fram (l; m) to the voice section determination unit 209 and the long-time frame signal creation unit 202.

The voice section determination unit 209 determines whether it is a voice section or a non-voice section by using the divided frame signal x_fram (l; m). The specific method for determining whether the voice section determination unit 209 is a voice section or a non-voice section is not limited, and various methods can be applied. The voice section determination unit 209 may determine, for example, whether it is a voice section or a non-voice section according to the equations (11) and (12).

In equations (11) and (12), x_fram_amp (l) is the average amplitude value of the divided frame signal, VAD (l) is the voice section determination result, and TH is the threshold value used for the determination of the voice section. Equation (11) is an equation for obtaining the average amplitude value x_fram_amp (l) of the divided frame signal x_fram (l; m). In the equation (12), if the average amplitude value x_fram_amp (l) of the divided frame signal x_fram (l; m) obtained by the equation (11) is larger than the threshold value TH, it is determined as a voice section and the voice section determination result VAD (l). ) Is substituted, and if the value is smaller than the threshold value TH, it is determined to be a non-voice section, and 0 is substituted into the voice section determination result VAD (l).

As for the threshold value TH of the equation (12), it suffices if the presence or absence of voice can be determined, and various methods can be widely applied. For example, as shown in equation (13), a frame (hereinafter referred to as “initial frame”) having a predetermined length (hereinafter referred to as “frame length L3”) after the sound masking device 100A starts operating is a silent section. A fixed threshold value TH may be used, in which the average amplitude value of the initial frame is used as the threshold value TH. Further, according to the equation (14), a threshold value TH (l) that fluctuates for each divided frame may be used for the average amplitude value x_fram_amp (l) of the divided frame signal x_fram (l; m) by using a time constant filter.

In equation (14), b is the coefficient of the time constant filter, which is a value of 0 or more and 1 or less. In equation (14), when it is desired to delay the update of the threshold value, b is preferably a value close to 1 (for example, a value such as b = 0.9), and when it is desired to speed up the update of the threshold value, b is preferably a value close to 0. (For example, a value such as b = 0.1).

In the voice section determination unit 209, various methods can be widely applied as a means for determining whether the voice section or the non-voice section is used. For example, the autocorrelation of the divided frame signal x_fram (l; m) is obtained. It may be determined by a method such as determining whether it is a voice section or a non-voice section.

Then, the voice section determination unit 209 outputs the voice section determination result VAD (l) to the DB accumulation determination unit 210 and the masker signal generation determination unit 211.

The DB accumulation determination unit 210 determines whether or not to accumulate the divided frame signal x_fram (l; m) in the frame signal DB 205 of the input signal DB 204 based on the audio section determination result VAD (l) of the audio section determination unit 209. do. The DB accumulation determination unit 210 determines, for example, whether or not to accumulate the divided frame signal x_fram (l; m) in the frame signal DB 205 of the input signal DB 204 according to the equation (15).

In equation (15), DB_flag (l) is a determination result of whether or not to accumulate. In the equation (15), when the voice section determination result VAD (l) is 1, it is determined that the data is accumulated in the DB, 1 is substituted for the determination result DB_flag (l), and the voice interval determination result VAD (l) is 0. , It is determined that the data is not accumulated in the DB, and 0 is substituted into the determination result DB_flag (l).

Then, the DB accumulation determination unit 210 outputs the determination result DB_flag (l) as to whether or not to accumulate in the DB to the DB writing unit 203A.

The DB writing unit 203A inputs a long-time frame signal x_frame_long (s) to the input signal DB 204 only when the determination result DB_flag (l) of the DB accumulation determination unit 210 is 1, for example, according to the equations (5) and (6). Write to the frame signal DB205. On the other hand, when the determination result DB_flag (l) of the DB accumulation determination unit 210 is 0, the long-time frame signal x_fram_long (s) is not written to the frame signal DB 205 of the input signal DB 204.

The masker signal generation determination unit 211 determines whether or not to generate a masker signal based on the voice section determination result VAD (l) of the voice section determination unit 209. The determination means determines, for example, whether or not to generate a masker signal according to the equation (16).

In equation (16), mask_flag (l) is a determination result of whether or not to generate a masker signal.

In the equation (16), when the voice interval determination result VAD (l) is 1, it is determined that a masker signal is generated, 1 is substituted for the determination result mask_flag (l), and the voice interval determination result VAD (l) is 0. At this time, it is determined that the masker signal is not generated, and 0 is substituted into the determination result mask_flag (l).

Then, the masker signal generation determination unit 211 outputs the determination result mask_flag (l) as to whether or not to generate the masker signal to the frame signal selection unit 207A.

The frame signal selection unit 207A follows the equations (7) and (8) only when the determination result mask_flag (l) of whether or not to generate the masker signal output from the masker signal generation determination unit 211 is 1. , Select a frame. On the other hand, when the determination result mask_flag (l) of whether or not to generate a masker signal is 0, the frame signal selection unit 207A does not select a frame.

As described above, the frame signal selection unit 207A has a plurality of frames from the past long-time frame signal held in the frame signal DB 205 only when the determination result mask_flag (l) of whether or not to generate the masker signal is 1. The selected frame T (p) is output to the masker signal generation unit 208A.

The masker signal generation unit 208A has a long time in the past from the frame signal DB 205 of the input signal DB 204 based on the determination result mask_flag (l) of the masker signal generation determination unit 211 and the frame T (p) selected by the frame signal selection unit 207A. Multiple frames are read out from the frame signal as a masker element signal, and a masker signal is generated and output. The masker signal generation unit 208A generates the masker signal ha (l; t) according to the equation (17), for example.

In the equation (17), the masker signal h (l; s) is generated and substituted into ha (l; t) only when the determination result mask_flag (l) of the masker signal generation determination unit 211 is 1, and the masker signal is used. When the determination result mask_flag (l) of whether or not to generate is 0, 0 (silence) is substituted for ha (l; t).

As described above, the masker signal generation unit 208A generates the masker signal only when the determination result mask_flag (l) of whether or not to generate the masker signal is 1. Then, the masker signal generation unit 208A performs overlap addition processing of the masker signal ha (l; t) according to the equation (18) and outputs the output signal y (n) to the sound output terminal OUT.

(B-3) Effect of Second Embodiment According to the second embodiment, the following effects can be obtained as compared with the first embodiment.

In the sound masking device 100A of the second embodiment, the voice of the target speaker U1 is stored in the frame signal DB 205 of the input signal DB 204 only when it is determined to be the voice section, so that only the voice section is stored in the input signal DB 204. Therefore, the masker signal can be generated only by voice, and a high masking effect can be maintained.

Further, in the sound masking device 100A of the second embodiment, since the masker signal is generated only when the voice section is determined, the masker signal is generated and output only when the voice of the target speaker U1 is input. Therefore, it can be configured so that the masker signal is output only when the voice is input.

(C) Third Embodiment Hereinafter, a third embodiment of the sound processing apparatus, the sound processing program, and the sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to the sound masking device will be described.

(C-1) Configuration of Third Embodiment FIG. 5 is a block diagram showing a functional configuration of the sound masking device 100B according to the third embodiment. In FIG. 5, the same reference numerals or corresponding reference numerals are given to the same portions or corresponding portions as those in FIGS. 1 and 4 described above.

In the following, the third embodiment will be mainly described with respect to the difference from the second embodiment, and the description of the part overlapping with the second embodiment will be omitted.

The sound masking device 100B of the third embodiment is different from the second embodiment in that the sound masking processing unit 200A is replaced with the sound masking processing unit 200B.

In the sound masking processing unit 200B, the pitch estimation unit 212 is added, and the DB accumulation determination unit 210 and the masker signal generation determination unit 211 are replaced by the DB accumulation determination unit 210B and the masker signal generation determination unit 211B, respectively. It is different from the second embodiment.

In the sound masking device 100B of the third embodiment, the pitch of the divided frame signal is estimated differently due to the addition of the pitch estimation unit 212, and the input signal DB204 is replaced by the DB accumulation determination unit 210B. The sound masking device 100A of the second embodiment is different in the method of accumulating the long-time frame signal stored in the frame signal DB 205 and the method of generating the masker signal by being replaced by the masker signal generation determination unit 211B. Is the difference.

Next, the detailed configuration of the sound masking processing unit 200B will be described.

The sound masking processing unit 200B includes a frame division unit 201, a voice section determination unit 209, a pitch estimation unit 212, a DB accumulation determination unit 210B, a long-time frame signal creation unit 202, a DB writing unit 203A, an input signal DB 204, and a frame signal DB 205. It has a masker signal generation determination unit 211B, a frame selection limiting unit 206, a frame signal selection unit 207A, a masker signal generation unit 208A, a sound input terminal IN, and a sound output terminal OUT.

The pitch estimation unit 212 estimates the pitch (speech pitch) of the divided frame signal only when it is determined to be a voice section based on the result of the voice section determination output from the voice section determination unit 209, and pitch estimates. The result (hereinafter referred to as "pitch estimate") is output.

The DB accumulation determination unit 210B determines whether or not to accumulate the divided frame signal and the pitch estimation value in the frame signal DB 205 of the input signal DB 204 based on the pitch estimation value of the pitch estimation unit 212, and outputs the determination result. do.

The masker signal generation determination unit 211B determines whether or not to generate a masker signal based on the estimated value of the pitch of the pitch estimation unit 212, and outputs the determination result.

In the third embodiment, the masker signal generation determination unit 211B may be excluded as in the first embodiment.

(C-2) Operation of the Third Embodiment Next, the operation of the sound masking device 100B (the sound processing method according to the embodiment) in the third embodiment having the above configuration will be described in detail.

The basic operation of the sound masking process in the sound masking device 100B according to the third embodiment is the same as the sound masking process described in the first and second embodiments.

Hereinafter, the processing operations in the pitch estimation unit 212, the DB accumulation determination unit 210B, and the masker signal generation determination unit 211B, which are different from the second embodiment, will be described in detail.

For example, the voice section determination unit 209 determines whether the divided frame signal x_fram (l; m) is a voice section or a non-voice section according to the equations (13) to (15), and determines the voice section determination result VAD (l). Output to the pitch estimation unit 212.

The pitch estimation unit 212 estimates the pitch of the divided frame signal x_fram (l; m) only for the divided frame (divided frame with VAD (l) = 1) determined by the voice section determination unit 209 as the voice section. The specific method for estimating the pitch by the pitch estimation unit 212 is not limited, and various methods can be applied. The pitch estimation unit 212 may estimate the pitch according to the equations (19) to (21), for example.

In equation (19), τ (τ = 0, 1 ..., L1-1) is the delay amount of autocorrelation, in equation (20), fs is the sampling frequency, and tp_pitch (l) is the estimated value of pitch temporarily. In the variable to be held, equation (21), pitch (l) indicates the estimated value of the pitch. In equation (21), fs is the sampling frequency.

In the equation (19), the autocorrelation function x_fram_corr (l; i) of the divided frame signal x_fram (l; m) is obtained.

In equation (20), the pitch is estimated by finding the delay amount τ that maximizes the autocorrelation function x_fram_corr (l; i) and dividing it by the sampling frequency fs, and temporarily assigning it to tp_pitch (l). In the equation (20), when the voice section determination result VAD (l) is 1 (when the voice section is used), the pitch estimation value is substituted into the pitch estimation value pitch (l), and the voice section determination result VAD (l) is obtained. When it is 0 (in the non-voice section), 0 is substituted for the estimated pitch (l) of the pitch.

The pitch estimation method in the pitch estimation unit 212 is not limited, and various methods can be applied. In the pitch estimation unit 212, for example, the pitch may be calculated by performing discrete Fourier transform or fast Fourier transform on the divided frame signal x_fram (l; m) and then performing cepstrum analysis.

Then, the pitch estimation unit 212 outputs the pitch estimation value pitch (l) to the DB accumulation determination unit 210B and the masker signal generation determination unit 211B.

The DB accumulation determination unit 210B determines whether or not to accumulate the long-time frame signal x_fram_long (s) in the frame signal DB 205 of the input signal DB 204 based on the pitch estimation value pitch (l) of the pitch estimation unit 212. For example, the DB accumulation determination unit 210B may determine whether or not to accumulate according to the equation (22).

In the equation (22), DB_flag (l) is a determination result of whether or not to accumulate, and TH_PITCH is a threshold value used for determining whether or not to accumulate DB. In equation (22), if the pitch estimated value pitch (l) is larger than the threshold value TH_PITCH, it is determined that the pitch is accumulated in the DB, 1 is substituted into the determination result DB_flag (l), and the pitch estimated value pitch (l) is the threshold value. If it is smaller than TH_PITCH, it is determined that it will not be accumulated in the DB, and 0 is substituted into the determination result DB_flag (l).

As long as it can be determined whether or not the threshold value TH_PITCH is accumulated in the DB, various methods can be widely applied. For example, in the pitch estimation unit 212, when pitch (l) is other than 0, TH_PITCH = 0 may be set as it is stored in the DB, or TH_PITCH = 100 as the lower limit value (for example, 100 Hz) of the fundamental frequency of human voice. May be.

Then, the DB accumulation determination unit 210B outputs the determination result DB_flag (l) as to whether or not to accumulate in the DB to the DB writing unit 203A.

The masker signal generation determination unit 211B determines whether or not to generate a masker signal based on the pitch estimation value pitch (l) of the pitch estimation unit 212. The determination means determines, for example, whether or not to generate a masker signal according to the equation (23).

In equation (23), mask_flag (l) is a determination result of whether or not to generate a masker signal, and TH2_PITCH is a threshold value used for determining whether or not to accumulate.

In equation (23), when the estimated pitch value pitch (l) is larger than the threshold value TH2_PITCH, it is determined that the pitch is accumulated in the DB, 1 is substituted into the determination result DB_flag (l), and when it is smaller than the threshold value TH2_PITCH, a masker signal is generated. It is an expression that it is determined not to be performed and 0 is substituted into the determination result mask_flag (l).

As the threshold value TH2_PITCH, it suffices if it can be determined whether or not it is accumulated in the DB, and values calculated by various methods can be widely applied. For example, when pitch (l) is other than 0 in the pitch estimation unit 212, TH2_PITCH = 0 may be set as it is stored in the DB, or TH2_PITCH = 100 as the lower limit value (for example, 100 Hz) of the fundamental frequency of human voice. Is also good. Further, TH2_PITCH = TH_PITCHDB may be the same as the threshold value TH_PITCH of the formula (22) used in the DB accumulation determination unit 210B.

Then, the masker signal generation determination unit 211B outputs the determination result mask_flag (l) as to whether or not to generate the masker signal to the frame signal selection unit 207A.

(C-3) Effect of Third Embodiment According to the third embodiment, the following effects can be obtained as compared with the first and second embodiments.

In the sound masking device 100B of the third embodiment, the pitch of the voice of the target speaker U1 is estimated, and the estimated value of the pitch is used for the DB accumulation determination unit 210B and the masker signal generation determination unit 211B in the voice section. Since only the voiced sound is stored in the input signal DB 204, the masker signal can be generated only by the voice, and a high masking effect can be maintained.

Since the sound masking device 100B of the third embodiment generates a masker signal only when it is determined to be a voiced sound in the voice section, the masker signal is generated and output only when the voice of the target speaker U1 is input. doing. As a result, the sound masking device 100B of the third embodiment can be configured so that the masker signal is output only when the sound is input.

(D) Fourth Embodiment Hereinafter, a fourth embodiment of the sound processing apparatus, the sound processing program, and the sound processing method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound processing device, the sound processing program, and the sound processing method of the present invention are applied to the sound masking device will be described.

(D-1) Configuration of Fourth Embodiment FIG. 6 is a block diagram showing a functional configuration of the sound masking device 100C according to the fourth embodiment. In FIG. 6, the same code or the corresponding code is attached to the same part or the corresponding part as in FIG. 5 described above.

In the following, the fourth embodiment will be mainly described with respect to the difference from the third embodiment, and the description of the part overlapping with the third embodiment will be omitted.

The sound masking device 100C of the fourth embodiment is different from the third embodiment in that the sound masking processing unit 200B is replaced with the sound masking processing unit 200C.

In the sound masking processing unit 200C, the long-time pitch estimation information creation unit 213 and the pitch estimation information DB are added, and the DB writing unit 203A, the input signal DB 204, and the frame signal selection unit 207A input the DB writing unit 203C. It differs from the third embodiment in that it replaces the signal DB 204C and the frame signal selection unit 207C.

In the sound masking device 100C of the fourth embodiment, the long-time pitch estimation information creation unit 213 and the pitch estimation information DB 214 are added and replaced with the input signal DB 204C, so that the estimated value of the pitch is accumulated in the input signal DB. The first point is that the method of accumulating the input signal DB204C is different due to the replacement with the DB writing unit 203C, and the method of selecting the masker element signal is different due to the replacement with the frame signal selection unit 207C. This is a difference from the sound masking device 100B of the third embodiment.

Next, the detailed configuration of the sound masking processing unit 200C will be described.

The sound masking processing unit 200 includes a frame division unit 201, a voice section determination unit 209, a pitch estimation unit 212, a DB accumulation determination unit 210B, a long-time frame signal creation unit 202, a long-time pitch estimation information creation unit 213, and an input signal DB 204C. It has a frame signal DB 205, a pitch estimation information DB 214, a frame selection limiting unit 206, a frame signal selection unit 207C, a masker signal generation unit 208A, a sound input terminal IN, and a sound output terminal OUT.

The long-time pitch estimation information creation unit 213 is based on the determination result of the DB accumulation determination unit 210B, and based on the estimated value of the pitch estimated by the pitch estimation unit 212, the pitch estimation information of the long-time frame (hereinafter, "long-time"). "Pitch estimation information") is created, and the created long-time pitch estimation information is output.

The DB writing unit 203C inputs the long-time frame signal to the frame signal DB 205 of the input signal DB 204C and the long-time pitch estimation information to the input signal DB 204C based on the determination result of whether or not to store the long-time frame signal in the DB of the DB storage determination unit 210B. It is written in the pitch estimation information DB 214.

The input signal DB204C is a storage means that stores (holds) the past long-time frame signal and the past long-time pitch estimation information in association with each other for each long-time frame. The data format in the input signal DB204C is not limited, but here, the input signal DB204C has accumulated at least the frame signal DB205 that has accumulated the past long-time frame signal and the past long-time pitch estimation information. It is assumed that the pitch estimation information DB 214 is provided.

The frame signal selection unit 207C contains the determination result of the masker signal generation determination unit 211B, the pitch estimation value of the pitch estimation unit 212, and the past long-time pitch estimation information stored in the pitch estimation information DB 214 of the input signal DB 204C. Based on the comparison result, the past long-time frame signal stored in the input signal DB204C is selected as the masker element signal from the frames before the limited number of frames of the frame selection limiting unit 206, and the selected frame is output.

In the third embodiment, the masker signal generation determination unit 211B may be excluded as in the second embodiment.

(D-2) Operation of the Fourth Embodiment Next, the operation of the sound masking device 100C (the sound processing method of the embodiment) in the fourth embodiment having the above configuration will be described in detail.

The basic operation of the sound masking process in the sound masking device 100C according to the fourth embodiment is the same as the sound masking process described in the third embodiment.

In the following, in the fourth embodiment, the long-time pitch estimation information creation unit 213, the DB writing unit 203C, the pitch estimation information DB 214 of the input signal DB 204C, and the frame signal selection unit 207C, which are different from the third embodiment. The processing operation will be described in detail.

The pitch estimation unit 212 estimates the pitch of the divided frame signal x_fram (l; m), and sets the pitch estimation value pitch (l) to the DB accumulation determination unit 210B, the long-time pitch estimation information creation unit 213, and the masker signal generation determination unit. Output to 211A and frame signal selection unit 207C.

The long-time pitch estimation information creation unit 213 combines the pitch estimation value pitch (l) of the divided frame signal x_fram (l; m) estimated by the pitch estimation unit 212 to obtain the long-time frame signal x_fram_long (s). Create long-time pitch estimation information. For example, the long-time pitch estimation information creating unit 204 may create the long-time pitch estimation information pitch_long (i) by combining the estimated values of the pitches according to the equation (24).
pix_long (i)
= Pitch (l-((I-1) -i)) ... (24)

Equation (24) is an equation (24) in which the estimated value pitch (l) of the pitch of the divided frame signal used when creating the long-time frame signal is combined to create the long-time pitch estimation information pitch_long (i). It has become.

Then, the long-time pitch estimation information creation unit 213 outputs the created long-time pitch estimation information pitch_long (i) to the DB writing unit 203C.

Based on the determination result DB_flag (l) output from the DB accumulation determination unit 210B, the DB writing unit 203C inputs the long-time frame x_frame_long (s) and the long-time pitch estimation information pitch_long (i) into the frame signal of the input signal DB204. It is written in association with DB 205 and pitch estimation information DB 214.

The DB writing unit 203C writes the long-time frame signal x_fram_long (s) to the DB_singal (j; t) of the frame signal DB 205 only when the determination result DB_flag (l) output from the DB accumulation determination unit 210B is 1, and at the same time. The long-time pitch estimation information pitch_long (i) is written in the pitch estimation information DB214DB_pitch (i). When the determination result DB_flag (l) output from the DB accumulation determination unit 210B is 1, the DB writing unit 203C follows, for example, equations (5), (25), and (6) for a long time frame signal x_fram_long ( s) is written in the DB_singal (j; t) of the frame signal DB 205, and at the same time, the long-time pitch estimation information pitch_long (i) is written in the pitch estimation information DB214DB_pitch (i). On the other hand, when the determination result DB_flag (l) of the DB accumulation determination unit 210B is 0, the DB writing unit 203C inputs the long-time frame signal x_frame_long (s) and the long-time pitch estimation information pitch_long (i) to the frame signal of the input signal DB204C. Do not write to DB205 and pitch estimation information DB214.
DB_pitch (j; i) = pitch_long (i) ... (25)

The input signal DB204C stores (holds) the long-time frame x_fram_long (s) and the long-time pitch estimation information pitch_long (i) in association with the frame signal DB205 of the input signal DB204 and the pitch estimation information DB214.

As described above, the input signal DB 204C of this embodiment includes the frame signal DB 205 and the pitch information DB 214. Here, each long-time frame signal x_frame_long (s) is recorded in the frame signal DB 205, and the long-time pitch estimation information pitch_long (i) is recorded in the pitch information DB 214.

The frame signal selection unit 207C estimates the pitch of the pitch estimation unit 212 (current division frame) based on the determination result mask_flag (l) of whether or not to generate the masker signal output from the masker signal generation determination unit 211. (Pitch based on) is compared with the past long-time pitch estimation information stored in the pitch information DB 214 of the input signal DB 204C, and has pitch information (close pitch value) close to the pitch estimation value of the pitch estimation unit 212. Select the long-time frame signal as the masker element signal. The frame signal selection unit 207C selects the masker fragment signal by using the input signal DB204C, the pitch estimated value pitch (l), and the limited number of frames Limit_Fream_NUM for the element data used for masker signal generation.

The frame signal selection unit 207C selects a frame only when the determination result mask_flag (l) of the masker signal generation determination unit 211 is 1, and when the determination result mask_flag (l) of the masker signal generation determination unit 211 is 0, the frame is framed. Do not select. When the determination result mask_flag (l) of the masker signal generation determination unit 211 is 1, the frame signal selection unit 207C selects a frame according to, for example, equations (28) and (29).

In equation (26), Tc (p) is a variable that holds the selected frame, and tp_T (p) is a variable that holds the temporarily selected frame. Further, in the equation (26), DB_pitch_ave (i) is an average value of past long-time pitch estimation information. Further, in the equation (27), Sub_pitch (j) is an absolute value of the difference between the pitch estimated value pitch (l), the average value of the past long-time pitch estimation information, and DB_pitch_ave (i). Furthermore, in equation (28), Tc (p) is the selected frame number, and p (p = 0, 1 ..., SEL_NUM-1) is the number of selected frames. Further, the small (x (k), p) in the equation (28) is a function that outputs the index k_p of the p-th smallest x (k_p) in the array x (k).

In the equation (26), the long-time frame signal held in the frame signal DB205 of the input signal DB204C is selected from the frames before the limit number of frames Limit_Fream_NUM in the order of new time, and the selected long-time frame signal is selected. The formula is to assign the index number of the retained database to tp_T (p). Further, the formula (27) is a formula for calculating the average value DB_pitch_ave (i) of the past long-time pitch estimation information stored in the pitch estimation information DB 214 for each tp_T (p). Further, the equation (28) is a process of calculating the absolute value Sub_pitth (i) of the difference between the pitch estimation value pitch (l) of the pitch estimation unit 212 and the average value DB_pitch_ave (i) of the past long-time pitch estimation information. Shown. Furthermore, the equation (29) is an equation in which a plurality of frames are selected as the masker element signal of the index j (long-time frame) having the smallest Sub_picth (i).

The frame signal selection unit 206C is a value indicating the pitch of the entire long-time frame based on the long-time pitch estimation information for each past long-time frame stored in the pitch information DB 214 (hereinafter, "long-time frame pitch"). The long-time frame to be selected as the masker element signal may be selected based on the comparison result between the estimated value of the pitch of the pitch estimation unit 212 and the long-time frame pitch. For example, the frame signal selection unit 207C may select a long-time frame having a long-time frame pitch having a value close to the estimated value of the pitch of the pitch estimation unit 212 as a masker element signal.

As described above, the frame signal selection unit 207C selects and selects a long-time frame signal (index) having pitch information (near pitch value) close to the pitch estimation value of the pitch estimation unit 212 as the masker element signal. The frame Tc (p) is output.

(D-3) Effect of Fourth Embodiment According to the fourth embodiment, the following effects can be obtained as compared with the first to third embodiments.

In the sound masking device 100C of the fourth embodiment, the pitch information stored in the pitch estimation information DB 214 and the signal close to the pitch estimation value estimated by the pitch estimation unit 212 are stored in the frame signal DB 205. You are selecting from. As a result, in the sound masking device 100C of the fourth embodiment, the frequency characteristic of the masker signal becomes close to the pitch of the voice of the target speaker U1, and a higher masking effect can be maintained.

(E) Other Embodiments The present invention is not limited to each of the above embodiments, and modified embodiments as illustrated below can also be mentioned.

(E-1) For example, the sound masking device of the present invention may be mounted on a device for preventing the content of a conversation from being leaked to a person other than the surrounding subjects in a conference call. In this case, in the sound masking device, the target speaker U1 is a person speaking in a conference call.

(E-2) In each of the above embodiments, the sound masking unit of the sound masking device may be configured to be processed by a processing device (for example, a server or the like) on the network.

(E-3) In each of the above embodiments, the sound masking device has been described as a configuration including an audio device (microphone, microphone amplifier, AD converter, speaker, speaker amplifier, and DA converter), but the sound has been described. The masking device may be manufactured as a configuration excluding the audio device, and the audio device may be connected separately at the actual site of use. That is, the sound masking device may be configured to include at least a sound masking processing unit.

100, 100A, 100B, 100C, ... Sound masking device, 101 ... Microphone, 102 ... Microphone amplifier, 103 ... AD converter, 104 ... DA converter, 105 ... Speaker amplifier, 106 ... Speaker, 200, 200A, 200B, 200C ... Sound masking device, 201 ... Frame division unit, 202 ... Long-time frame signal creation unit, 203, 203A, 203C ... DB writing unit, 204, 204C ... Input signal DB, 205 ... Frame signal DB, 206 ... Frame selection restriction unit , 207, 207A, 207C ... Frame signal selection unit, 208, 208A ... Masker signal generation unit, 209 ... Sound section determination unit, 210, 210B ... DB accumulation determination unit, 211, 211B ... Masker signal generation determination unit, 212 ... Pitch Estimate 213 ... Long-time pitch estimation information creation unit, 214 ... Pitch estimation information DB, IN ... Sound input terminal, OUT ... Sound output terminal, 300 ... Computer, 301 ... Processor, 302 ... Primary storage unit, 303 ... Secondary storage Department.

Claims (7)

A frame dividing means for dividing the microphone input signal supplied from the microphone that collects the voice spoken by the target speaker into a predetermined length, and
A long-time frame signal creating means for creating a long-time frame having a predetermined length by combining the microphone input signals frame-divided by the frame-dividing means.
An input signal storage means for accumulating a long-time frame signal generated by the long-time frame signal creation means,
A frame signal selection means for performing a frame signal selection process for selecting a signal to be used for generating a masker signal from a past frame-divided microphone input signal stored in the input signal storage means.
When the frame signal selection means performs the frame signal selection process, the frame selection limiting means for limiting the frames to be selected, and the frame selection limiting means.
An audio processing device having a masker signal generation means for generating and outputting the masker signal that makes it difficult to hear the voice uttered by the target speaker by using the signal used for generating the masker signal. .. The acoustic processing according to claim 1, wherein the frame selection limiting means limits the frame signal selecting means so as not to select a signal of a predetermined time before when performing the frame signal selecting process. Device. A voice section determining means for determining whether the microphone input signal frame-divided by the frame dividing means is a voice section or a non-voice section,
Based on the result of the voice section determination means, an input signal storage determination means for determining whether or not to accumulate the frame-divided microphone input signal in the input signal storage means, and an input signal storage determination means.
A masker signal generation determining means for determining whether or not to generate the masker signal based on the result of the voice section determining means is further provided.
The long-time frame signal creating means stores the microphone input signal frame-divided by the frame-dividing means in the input signal accumulating means only when it is determined to be the voice section.
The sound processing apparatus according to claim 1 or 2, wherein the masker signal generation means generates the masker signal only when it is determined to be the voice section. Further, a pitch estimation means for estimating the pitch of the microphone input signal frame-divided by the frame division means is provided.
Based on the result of the pitch estimation means, an input signal storage determination means for determining whether or not to store the frame-divided microphone input signal in the input signal storage means, and an input signal storage determination means.
A masker signal generation determining means for determining whether or not to generate the masker signal based on the result of the pitch estimating means is further provided.
The long-time frame signal creating means stores the microphone input signal frame-divided by the frame dividing means in the input signal accumulating means based on the pitch estimated by the pitch estimating means.
The sound processing apparatus according to claim 1 or 2, wherein the masker signal generation means generates the masker signal based on the pitch estimated by the pitch estimation means. A pitch accumulating means capable of accumulating the pitch estimated by the pitch estimating means for each frame, and a pitch accumulating means.
The pitch storing means is further provided with a pitch information creating means for accumulating the pitch estimated by the pitch estimating means.
The frame selection restriction is performed by using the past frame-divided microphone input signal stored in the input signal storage means, the past pitch information stored in the pitch storage means, and the pitch estimated by the pitch estimation means. The acoustic processing apparatus according to claim 4, wherein a signal used for generating a masker signal is selected from frames prior to the number of frames limited by means. Computer,
A frame dividing means for dividing the microphone input signal supplied from the microphone that collects the voice spoken by the target speaker into a predetermined length, and
A long-time frame signal creating means for converting a microphone input signal frame-divided by the frame-dividing means into a time frame of a predetermined length,
An input signal storage means for accumulating a long-time frame signal generated by the long-time frame signal creation means,
A frame signal selection means for performing a frame signal selection process for selecting a signal to be used for generating a masker signal from a past frame-divided microphone input signal stored in the input signal storage means.
When the frame signal selection means performs the frame signal selection process, the frame selection limiting means for limiting the frames to be selected, and the frame selection limiting means.
It is characterized by having a masker signal generation means for generating and outputting the masker signal that makes it difficult to hear the voice spoken by the target speaker by using the signal used for generating the masker signal. Characteristic sound processing program. In the sound processing method performed by the sound processing device,
The sound processing device includes a frame dividing means, a long-time frame signal creating means, an input signal storing means, a frame selection limiting means, a frame signal selecting means, and a masker signal generating means.
The frame dividing means divides the microphone input signal supplied from the microphone that picks up the voice spoken by the target speaker into a predetermined length.
The long-time frame signal creating means creates a long-time frame having a predetermined length by combining the microphone input signals frame-divided by the frame-dividing means.
The input signal storage means stores the long-time frame signal generated by the long-time frame signal creation means, and the input signal storage means stores the long-time frame signal.
The frame signal selection means performs a frame signal selection process for selecting a signal to be used for generating a masker signal from a past frame-divided microphone input signal stored in the input signal storage means.
The frame selection limiting means limits the frames to be selected when the frame signal selecting means performs the frame signal selection process.
The sound processing method is characterized in that the masker signal generation means uses a signal used to generate the masker signal to generate and output the masker signal that makes it difficult to hear the voice spoken by the target speaker. ..

Images