JP2964897B2 - Sound control device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the position of the center of gravity and the physical and The present invention relates to a sound control device that controls sound according to a state of a user such as a state.

[0002]

2. Description of the Related Art Conventionally, there has been known an apparatus which detects a position of a center of gravity of a user and changes an image to be presented. By using this device, it is possible to detect the center-of-gravity position information that changes according to the operation of the user, and it is possible to change the video presented to the user based on the operation of the user.

On the other hand, a sound generated in the virtual world to indicate the position of the user in the virtual world is presented in a three-dimensional sound field, or an alarm is issued when the load of the user exceeds a predetermined value. Those that emit sound are known.

[0004]

However, when the conventional apparatus for presenting the position of the center of gravity in an image is applied to the field of an artificial reality system in which a sense of realism is important, it is used for the position of the center of gravity of the user. Information presented to the user is only a video, and there is a problem that the user cannot be given a sufficient sense of reality.

Further, there is a problem that an apparatus using a three-dimensional sound field to indicate the position of a user in a virtual world cannot be reflected in the sound presenting the user's individual actions or mental and physical states. Furthermore, in the device that generates a warning sound when the load of the user exceeds a predetermined value, there is a problem that the operation of the user cannot be presented as a change in sound.

[0006] In summary, in the conventional device, it was not possible to present as an acoustic change directly corresponding to a user's movement or a state of mind and body. For this reason, it is impossible for the user to associate the presented sound with the user's own motion or mental / physical state, and it is difficult for the user to give meaning to the presented sound. Was. As a result, with the conventional device, it is insufficient to give the user a sense of realism or to give a virtual operating feeling.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide an acoustic control apparatus used for controlling sound for presenting biological information such as load information, posture information, and physiological indices of a user. It is another object of the present invention to convey the meaning of the presented sound to the user in an easy-to-understand manner, and to consequently give the user a sense of realism and operation.

[0008]

According to a first aspect of the present invention, there is provided a sound control apparatus for controlling a sound to be presented in accordance with a state of a user. A correspondence information database that holds correspondence information between load information and sound effects and outputs the correspondence information; a sound effect information calculation unit that obtains sound effect information from the load information and the correspondence information; And a sound effect adding means for changing the sound signal according to the sound effect information as an input and outputting the sound signal to the outside.

According to a second aspect of the present invention, there is provided a sound control device for controlling a sound to be presented in accordance with a state of a user. A correspondence information database that retains correspondence information and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the load information and the correspondence information, and a plurality of sound effect information and a plurality of externally provided A sound effect adding unit that receives a sound signal as input and obtains a sound effect added sound to which a sound effect based on the sound effect information is added, and holding selection information indicating correspondence between a load and a sound to be selected, Sound selection information database for outputting sound selection information, sound selection information calculating means for obtaining sound selection information from the load information and the selection information, the sound selection information and the sound effect addition The seeking an input audio signal to be output by the acoustic selection information and having an acoustic selection means for outputting to the outside.

According to a third aspect of the present invention, there is provided a sound control apparatus for controlling sound to be presented in accordance with a state of a user, a posture measuring means for measuring a posture of the user and outputting posture information, A correspondence information database that holds the correspondence information and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the posture information and the correspondence information, and the sound effect information and an externally supplied sound signal. Sound effect adding means for changing an audio signal according to the sound effect information as an input and outputting the signal to the outside.

According to a fourth aspect of the present invention, there is provided an acoustic control device for controlling a sound to be presented in accordance with a state of a user, a posture measuring means for measuring a posture of the user and outputting posture information; A correspondence information database for holding the correspondence information and outputting the correspondence information; a sound effect information calculation unit for obtaining sound effect information from the posture information and the correspondence information; A sound effect adding unit that receives a signal and obtains a sound effect added sound to which a sound effect based on the sound effect information is added, and holds selection information indicating a correspondence between a posture and a sound to be selected; A sound selection information database to be output, sound selection information calculation means for obtaining sound selection information from the posture information and the selection information, and the sound selection information and the sound effect added sound. And having an acoustic selecting means for outputting to the outside seeking sound signal outputted by said acoustic selection information and force.

According to a fifth aspect of the present invention, there is provided a sound control apparatus for controlling a presented sound in accordance with a state of a user, a physiological index measuring means for measuring a physiological index of the user and outputting a physiological index, a physiological index and a sound. A correspondence information database that holds correspondence information with effects and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the physiological index and the correspondence information, and a sound provided by the sound effect information and an externally provided sound. A sound effect adding means for receiving a signal as input, changing the sound signal according to the sound effect information, and outputting the sound signal to the outside.

According to a sixth aspect of the present invention, there is provided a sound control apparatus for controlling a presented sound in accordance with a state of a user, a physiological index measuring means for measuring a physiological index of the user and outputting a physiological index, A correspondence information database that holds correspondence information with effects and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the physiological index and the correspondence information, and a plurality of sound effect information and externally provided Sound effect input means for inputting the sound signal and obtaining a sound effect added sound to which a sound effect based on the sound effect information is added, and holding selection information indicating correspondence between a physiological index and a sound to be selected, A sound selection information database for outputting selection information, sound selection information calculation means for obtaining sound selection information from the physiological index and the selection information, the sound selection information and the sound And having an acoustic selecting means for outputting to the outside seeking sound signal outputted by said acoustic selection information as input fruit addition sound.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention 1 will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment 1 of the present invention. In this embodiment, a load measuring unit 1 that measures the position of the center of gravity and outputs load information, a correspondence information database 2 that stores correspondence information between the load information and the sound effect, and obtains sound effect information from the load information and the correspondence information. Sound effect information calculation means 3
And sound effect adding means 4 for changing sound signals according to sound effect information.

The load measuring means 1 measures the position of the center of gravity of the user to obtain load information. The load information includes, for example, the position of the center of gravity of the user, the deviation between the position of the center of gravity of the user and the center position, and the deviation between the position of the center of gravity of the user and the ideal position of the center of gravity of the user. To obtain the load information, for example, one or more load sensors are arranged on the floor surface, the position of the center of gravity of the user is measured from the outputs of the load sensors, and the load information is obtained. A method of determining the position of the center of gravity of the user by disposing one or a plurality of load sensors on a table on which is placed (Japanese Patent Laid-Open No. 3-212263).

The correspondence information database 2 holds correspondence information, which is a correspondence between load information and acoustic effects. There is, for example, a method as shown in FIG. 2 for associating the load information with the sound effect. This correspondence can be held in a function format or a table format. For example, FIG.
The correspondence between the load information x and the sound effect y in (A) can be expressed by a mathematical expression such as y = ax + b in a functional form. Here, a and b are parameters representing the correspondence between the load information and the sound effect. The parameters of the function representing such correspondence are previously determined by the system designer as a = 2.0, b = 0.
5 is set. An example of this parameter is
If no load is applied and the load information is 0,
0.5 is given as an acoustic effect.
When a predetermined reference value is taken, 2.5 is given as a sound effect. Further, in the example of the function format, when the load information increases, the sound effect is added in proportion to the degree of the increase in the load information.
FIG. 3 shows an example in which FIG. 2A is represented in a table format. The width of the load information, the range of the load information, the value of the corresponding sound effect, the accuracy of the sound effect value, and the like used in the table are set in advance by the system designer. Correspondence information database 2
Can be realized using a storage device such as a ROM, a RAM with a power supply device, a magnetic card, a magnetic disk device, and a magneto-optical disk device. The correspondence information database 2 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The sound effect information calculating means 3 receives the load information from the load measuring means 1 and the corresponding information from the corresponding information database 2, respectively, and refers to the sound effect information representing the sound effect corresponding to the load information and the corresponding information. Ask. If the correspondence information is in the form of a table in FIG. 3 and the load information obtained from the load measuring means 1 is 3.0, the acoustic effect information is obtained as 2.5 by searching the table. If there is no load information column in the table corresponding to the measured load information, the sound effect information can be obtained by appropriately interpolating from the values before and after the table or by using the sound effect information with the closest value to the load information. Can be dealt with. The correspondence information is represented by a function y = 2x + 0.5 between the load information x and the sound effect information y. If the load information obtained from the load measuring means 1 is 3.0, the sound effect information is 6. A calculation is performed so as to make 5 The sound effect information calculation means 3 can be implemented, for example, as a program operating on a personal computer PC-9801 series manufactured by NEC Corporation.

The sound effect adding means 4 receives sound signals from the outside and sound effect information from the sound effect information calculating means 3, respectively, and changes the sound signal in accordance with the sound effect information. Output to As a method of changing the sound signal, the sound volume is changed according to the sound effect information, the sound in a predetermined range is amplified or attenuated according to the sound effect information, or the sound is amplified or attenuated according to the sound effect information There are methods of changing a sound range, changing an additional sound such as an echo reverb in accordance with sound effect information, changing a pitch in accordance with sound effect information, and changing a tempo in accordance with sound effect information. The sound effect adding means 4 includes an AD converter, a DS
It is realized by a combination of P and DA converters, and various sound effects can be added by a program on the DSP. For the sound effect addition calculation, see Musical Applications of Microprocessors (Musica
l Applications of Microprocessors, 1985, Heiden Book Company, USA; Also, with a configuration in which an A / D converter and a D / A converter are added to a personal computer, it is also possible to describe and realize a program that causes the CPU of the personal computer to perform the sound effect addition operation. Further, a commercially available sound adding device can be used as the sound effect adding means 4. An example of a commercially available sound adding device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound effect calculating means 3 is implemented as a program on a personal computer, and the sound effect adding means 4 is configured to use an external sound adding device, the communication between the sound effect calculating means 3 and the musical instrument or the computer is performed by music. The sound effect information can be communicated by a communication method conforming to the MIDI standard defined by the MIDI Standards Council as a standard for communicating performance information. For the MIDI standard, M
Details of the IDI 1.0 standard, 1989, MIDI Standards Council (hereinafter referred to as Document 2).

Next, with reference to FIGS. 1, 3 and 4, the present embodiment uses a load measuring means 1 which uses the position of the center of gravity obtained from a load sensor placed on the floor as load information. The volume information is used as the acoustic information to be changed, and the load information and the volume information are associated with each other using the table shown in FIG. 3 and the correspondence information database 2 stored in the ROM is used. Means 3 is calculated by a personal computer program as means 3,
The digital mixing processor DMP7 manufactured by Yamaha Corporation is used as the sound effect adding means 4, and the sound effect information is MIDI.
The operation will be described using a specific example, taking the case of communication using a standard as an example.

Digital mixing processor DMP7
Has a MIDI receiving terminal and converts the input sound signal to M
The output volume is set to 12 according to the message obtained by IDI.
The output can be changed in eight stages. Prior to the operation, the MIDI reception channel of DMP7 is set to 1 or OMNI so that DMP7 can receive the data output to MIDI channel1. In this setting, in order to change the output volume of DMP7, M
This can be realized by changing the stereo level of parameter number 135 of DMP7 with control number 4 of the control change of IDI. M for making control change of MIDI channel 1
The IDI message is a message having a 3-byte length of B0, control number, and control value in hexadecimal notation. After all, M for changing the volume of DMP7
The IDI message is a message having a 3-byte length of B0, 4, and a volume value in hexadecimal notation. This volume value is a value from 0 to 127, and the output volume increases as the value increases.

The load measuring means 1 uses the position of the center of gravity obtained from the load sensor as load information (FIG. 4, step A).
1). Next, the sound effect calculation means 3 generates volume information corresponding to the measured load information from the load information and the correspondence information held in the correspondence information database 2 (FIG. 4, step A2). Next, DMP7 which is the sound effect adding means 4
Is a sound effect information calculation means 3 through a MIDI cable.
And receives the sound signal from the sound signal input terminal, increases or decreases the sound volume of the received sound signal based on the sound volume information, and outputs the sound signal to the sound signal output terminal (FIG. 4, step A3).

The operation of the sound effect calculation means 3 in the above description will be described in detail with reference to FIG. First, load information is received from the load measuring means 1 (FIG. 5, step A21).
Next, the sound volume is searched from the correspondence table of FIG. 3 held in the correspondence information database 2. (FIG. 5, Step A2
2). For example, if the load information is 1.0, 1.5 is set as the search result. Next, the volume obtained from the table is normalized to the range of the volume handled by the DMP 7 (FIG. 5, step A2).
3). If the range of the volume in the correspondence table of FIG. 3 is 0.0 to 15.0, for example, the volume obtained from the table is multiplied by 8 and rounded down to a whole number so that it can be normalized. . If the volume obtained from the table is 1.5, the normalized volume value is 24. Next, a MIDI message is generated based on the normalized volume (step A24 in FIG. 5). When the normalized volume value is 24, the MIDI message is represented in hexadecimal notation as B0, 4,
It becomes 18. Next, the generated MIDI message is
It transmits to MP7 (FIG. 5, step A25).

In the above description of the operation, the volume obtained by searching the correspondence information database 2 is normalized,
Although the description has been made so as to generate the message, a normalized volume value may be put in the correspondence information database in advance. In this case, the process of step A23 in FIG. 5 can be omitted.

Further, the sound effect information calculating means 3 and the sound effect adding means 4 can be implemented as a program of a personal computer having an AD converter and a DA converter. In this case, the sound effect information can be communicated via a temporary storage area or the like of the personal computer, and the sound addition processing includes each sample value and sound volume value of the sound signal input by AD conversion from the outside. And can be obtained by multiplication. By subjecting the obtained value to DA conversion again, a sound signal with sound added can be output to the outside.

Further, the sound effect information calculating means 3 can be mounted as a program on a personal computer, and the sound adding means 4 can be mounted on a signal processing dedicated DSP board having an AD converter and a DA converter. In this case, the sound effect information can be communicated through the bus connecting the personal computer and the DSP board.

Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. FIG. 6 is a block diagram showing an embodiment 2 of the present invention. In this embodiment, a load measuring unit 11 that measures the movement of the center of gravity and outputs load information, a correspondence information database 12 that stores correspondence information between the load information and the sound effect, and obtains sound effect information from the load information and the correspondence information. Sound effect information calculating means 13 and sound effect adding means 14 for changing sound signals based on sound effect information to obtain sound effect added sounds 14
A sound selection information database 15 for holding selection information indicating a correspondence between the load information and the sound to be selected; a sound selection information calculation unit 16 for obtaining sound selection information from the load information and the selection information; And sound selecting means 17 for obtaining a sound signal to be output from the effect-added sound.

The load measuring means 11 calculates the load information by measuring the position of the center of gravity of the user. As load information, for example,
There are a position of the center of gravity of the user, a deviation between the position of the center of gravity of the user and the center position, a deviation between the position of the center of gravity of the user and an ideal position of the center of gravity of the user, and the like. To obtain the load information, for example, one or more load sensors are arranged on the floor surface, the position of the center of gravity of the user is measured from the outputs of the load sensors, and the load information is obtained. A method of determining the position of the center of gravity of the user by disposing one or a plurality of load sensors on a table on which is placed (Japanese Patent Laid-Open No. 3-212263).

The correspondence information database 12 holds correspondence information which is a correspondence between load information and acoustic effects.
For example, as shown in FIG.
There is something like This correspondence can be held in a function format or a table format. For example, FIG.
The correspondence between the load information x and the sound effect y in (A) can be expressed by a mathematical expression such as y = ax + b in a functional form. Here, a and b are parameters representing the correspondence between the load information and the sound effect. The parameters of the function representing such correspondence are previously determined by the system designer as a = 2.0, b = 0.
5 is set. An example of this parameter is
If no load is applied and the load information is 0,
0.5 is given as an acoustic effect.
When a predetermined reference value is taken, 2.5 is given as a sound effect. Further, in the example of the function format, when the load information increases, the sound effect is added in proportion to the degree of the increase in the load information.
FIG. 3 shows an example in which FIG. 2A is represented in a table format. The width of the load information, the range of the load information, the value of the corresponding sound effect, the accuracy of the sound effect value, and the like used in the table are set in advance by the system designer. Correspondence information database 12
Can be realized using a storage device such as a ROM, a RAM with a power supply device, a magnetic card, a magnetic disk device, and a magneto-optical disk device. The correspondence information database 12 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The sound effect information calculation means 13 receives the load information from the load measurement means 11 and the correspondence information from the correspondence information database 12, respectively, and refers to the sound effect information representing the sound effect corresponding to the load information and the correspondence information. Ask. The correspondence information is in the form of a table in FIG.
If the load information obtained from is 3.0, the acoustic effect information is obtained as 2.5 by searching the table. If there is no load information column in the table corresponding to the measured load information, the sound effect information can be obtained by appropriately interpolating from the values before and after the table or by using the sound effect information with the closest value to the load information. Can be dealt with. The correspondence information is expressed by a function y = 2x + 0.5 between the load information x and the sound effect information y. If the load information obtained from the load measuring means 11 is 3.0, the sound effect information is 6. A calculation is performed so as to obtain a value of 5. The sound effect information calculation means 13 is, for example, a personal computer PC-980 manufactured by NEC Corporation.
It can be implemented as a program that runs on one series.

The sound effect adding means 14 receives a plurality of sound signals from the outside and the sound effect information from the sound effect information calculating means 13, respectively, and changes the plurality of received sound signals in accordance with the sound effect information. Generate sound. As a method of changing the sound signal, the sound volume is changed according to the sound effect information, the sound in a predetermined range is amplified or attenuated according to the sound effect information, or the sound is amplified or attenuated according to the sound effect information There are methods of changing a sound range, changing an additional sound such as an echo reverb in accordance with sound effect information, changing a pitch in accordance with sound effect information, and changing a tempo in accordance with sound effect information. The sound effect adding means 14 includes an AD converter, DS
It is realized by a combination of P and DA converters, and various sound effects can be added by a program on the DSP. The calculation of the sound effect addition is described in detail in Reference 1. Also, with a configuration in which an A / D converter and a D / A converter are added to a personal computer, it is also possible to describe and realize a program that causes the CPU of the personal computer to perform the sound effect addition operation. Further, a commercially available sound adding device can be used as the sound effect adding means 14. An example of a commercially available sound adding device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound effect information calculating means 13 is implemented as a program on a personal computer, and the sound effect adding means 14 is configured to use an external sound adding device, the communication between the sound effect information calculating means 13 and the sound effect adding means 14 is performed in accordance with the MIDI standard. Sound effect information can also be communicated. The MIDI standard is described in detail in Reference 2.

The selection information database 15 holds selection information which is information for associating the load information with the sound selection information. There is, for example, a method as shown in FIG. 7 for associating the load information with the sound selection information. FIG. 7A is an example of sound selection information when only one sound to be output is selected according to the load information, and FIG. 7B is a diagram in which a plurality of input sounds are mixed and output. In this case, the correspondence between the mixing ratio of the input sound and the load information is used as the sound selection information.
The range of the load information, the value of the corresponding sound selection information, and the like are set in advance by the system designer. The selection information database 15 can be realized using a storage device such as a ROM, a RAM with a power supply device, a magnetic card, a magnetic disk device, and a magneto-optical disk device. The selection information database 15 can be realized by storing the type of function, the function parameter, or the value of each table in these storage devices as data.

The sound selection information calculation means 16 receives the load information from the load measurement means 11 and the corresponding information from the selection information database 15, respectively, and refers to the selection information for the sound selection information representing the sound selection corresponding to the load information. Ask. If the selection information is in the form of a table in FIG. 7A and the load information obtained from the load measuring means 11 is 3.0, the acoustic selection information is determined to be 1 by searching the table. When there is no load information column in the table corresponding to the measured load information, in order to obtain the sound selection information, it is necessary to interpolate appropriately from the values before and after the table or to select the sound selection information with the closest value to the load information. Can be dealt with. The sound selection information calculation means 16 can be implemented as, for example, a program that operates on a personal computer PC-9801 series manufactured by NEC Corporation.

In the sound selecting means 17, the sound adding means 14
, A plurality of sound additional sounds from the sound selection information calculation means 16, and a sound signal to be output from the plurality of sound additional sounds received according to the sound selection information is obtained and output to the outside. As the sound selection, there are a method of outputting only one of a plurality of input sounds, and a method of changing the volume level of the plurality of input sounds and mixing and outputting them. The sound selection unit 17 is realized by a combination of a plurality of AD converters, DSPs, and DA converters, and can perform various sound selections by a program on the DSP. The calculation of the sound selection information is described in detail in Reference 1. Also, with a configuration in which a plurality of A / D converters and D / A converters are added to a personal computer, it is also possible to write and realize a program that causes the CPU of the personal computer to perform the sound selection calculation. Further, a commercially available sound selection device can be used as the sound selection means 17. An example of a commercially available sound selection device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound selection information calculation means 16 is implemented as a program on a personal computer, and the sound selection means 17 has a configuration using an external sound selection device,
During this time, the sound effect information can be communicated by a communication method conforming to the MIDI standard. The MIDI standard is described in detail in Reference 2.

Next, with reference to FIGS. 3, 6, 7 and 8, the load measuring means 11 in this embodiment uses the position of the center of gravity obtained from the load sensor placed on the floor surface as load information. , The bass equalization amount is used as the acoustic information to be changed, FIG. 3A is used to associate the load information with the bass equalization amount, and FIG. 3A is used to associate the load information with the sound selection information. 7 (B), the one stored in the ROM as the correspondence information database 12 and the selection information database 15 and the sound effect information calculation means 13
The sound selection information calculation means 16 uses a program calculated by a personal computer program. The sound effect addition means 14 and the sound selection means 17 use a digital mixing processor DMP7 manufactured by Yamaha Corporation. The sound effect information and the sound selection information are MIDI. The operation will be described using a specific example, taking the case of communication using a standard as an example.

Digital mixing processor DMP7
Has a MIDI receiving terminal, and can set the bass equalization amount and the output volume of a plurality of input audio signals in accordance with the message obtained by MIDI, and can mix and output inputs of up to eight channels. it can.
Prior to the operation, the MID of the DMP 7 is set so that the DMP 7 can receive the data output to the MIDI channel 1.
The I reception channel is set to 1 or OMNI.
In this setting, in order to set the bass equalization amount of the input 1 of the DMP 7, it is necessary to use MIDI note-on 24 to set the bass equalization amount of the input 2.
The value can be set for each of the note-on 25th. The message to make the MIDI channel 1 note-on is in hexadecimal notation, 80, note number,
The message is a 3-byte long message of the velocity value.
Therefore, the MIDI message for setting the bass equalization amount of the input 1 of the DMP 7 is 80, 1 in hexadecimal notation.
A message having a length of 3 bytes, ie, C, equalization amount. Similarly, a MIDI message for setting the bass equalization amount of the input 2 of the DMP 7 is 80, in hexadecimal notation.
The message has a length of 3 bytes, ie, 1D, equalization amount. These bass equalization values are values from 0 to 127. The higher the value, the greater the output of the bass output. Also, to change the volume of input 1,
Control number 3 of MIDI control change
In order to change the volume of input 2, MIDI
Control number 33 of the control change,
Each value can be set. M for making control change of MIDI channel 1
The IDI message is a message having a 3-byte length of B0, control number, and control value in hexadecimal notation. Therefore, the MIDI message for setting the volume of the input 1 of the DMP 7 is represented by hexadecimal notation B0, B2,
The message has a length of 3 bytes of 0 and a volume value.
Similarly, the MIDI message for setting the volume of the input 2 of the DMP 7 is a message having a 3-byte length of B0, 21, and a volume value in hexadecimal notation. These sound volume values are values from 0 to 127, and the higher the value, the higher the output sound volume.

The load measuring means 11 calculates the load information based on the center of gravity information obtained from the load sensor (FIG. 8,
Step B1). Next, the sound effect information calculation means 13 generates a MIDI message for setting a bass equalization amount corresponding to the measured load information from the load information and the correspondence information held in the correspondence information database 12 (FIG. 8, FIG. Step B2). Next, the DMP 7 serving as the sound effect adding means 14 transmits the M for setting the bass equalization amount from the sound effect information calculating means 13 through the MIDI cable.
An IDI message is received from an audio signal input terminal, and the amount of bass equalization for the received audio signal is increased or decreased (FIG. 8, step B3). Next, the sound selection information calculation means 16 generates sound selection information corresponding to the measured load information from the load information and the selection information held in the selection information database 15 (FIG. 8, step B4). Next, the DMP 7
Receives the sound selection information from the sound selection information calculation means 16, increases or decreases the volume of the sound to which the sound is added, mixes and outputs the mixed sound to the sound signal output terminal (FIG. 8, step B5).

The sound effect information calculating means 1 in the above description
The operation 3 will be described in detail with reference to FIG. First, load information is received from the load measuring means 11 (FIG. 5, step A).
21). Next, the bass equalization amount is searched from the correspondence table of FIG. 3 held in the correspondence information database 12 (FIG. 5, step A22). For example, if the load information is 1.0
, 1.5 is set as the search result. Next, the bass equalization amount obtained from the table is normalized to the range of the bass equalization amount handled by the DMP 7 (FIG. 5, step A23).
The range of the sound volume in the correspondence table of FIG.
If it is 0, it can be normalized by, for example, multiplying the volume obtained from the table by 8 and rounding down the decimal part so that it becomes an integer. If the volume obtained from the table is 1.5, the normalized bass equalization amount is 24. Next, a MIDI message is generated based on the normalized bass equalization amount (step A24 in FIG. 5). The MIDI message when the normalized bass equalization amount is 24 is
80, 1C, 1C, 80, 1D, 1C in hexadecimal notation
6 bytes. Next, the generated MIDI message is transmitted to the DMP 7 (FIG. 5, step A25).

Next, the operation of the sound selection information calculation means 16 in the above description will be described in detail with reference to FIG. First,
The load information is received from the load measuring means 11 (FIG. 9, step B41). Next, the volume is searched from the correspondence table of FIG. 7B held in the selection information database 15 (FIG. 9, step B42). For example, when the load information is 1.0, sound 1 is 0.0 and sound 2 is 1.0 as search results. Next, the selection information obtained from the table is normalized to the range of the volume handled by the DMP 7 (FIG. 9, step B43). If the range of the volume in the correspondence table of FIG. 3 is 0.0 to 1.0, for example, the volume obtained from the table is multiplied by 127 and rounded down to a whole number so that it can be normalized. . If the sound 1 obtained from the table is 0.0, the volume value of the normalized sound 1 is 0, and if the sound 2 obtained from the table is 1.0, the volume value of the normalized sound 2 is 1.0 127. next,
A MIDI message is generated based on the normalized volume (FIG. 9, step B44). When the volume value of the normalized sound 1 is 0 and the volume value of the normalized sound 2 is 127, the MIDI message is expressed in hexadecimal notation as B0, 2
0, 0, B0, 21, 7F. Next, the generated M
An IDI message is transmitted to the DMP 7 (FIG. 5, step B45).

Further, the sound effect information calculating means 13, the sound effect adding means 14, the sound selection information calculating means 16, and the sound selecting means 17 can be implemented as a program of a personal computer having an AD converter and a DA converter. . In this case, the sound effect information and the sound selection information are
Communication can be performed via a temporary storage area or the like of a personal computer. The sound addition processing and the sound selection processing include filtering processing and mixing processing for each sample value of an audio signal input by AD conversion from the outside. Can be determined by: The obtained value is returned to DA
By performing the conversion, a sound signal in which sound is added and sound is selected can be output to the outside.

Further, the sound effect information calculation means 13 and the sound selection information calculation means 16 are implemented as a program on a personal computer, and the sound addition means 14 and the sound selection means 17 are converted into a signal having a plurality of AD converters and DA converters. It is also possible to use a dedicated DSP board for processing. In this case, the sound effect information and the sound selection information can be communicated through a bus connecting the personal computer and the DSP board.

Hereinafter, a third embodiment of the present invention will be described with reference to the drawings. FIG. 10 is a block diagram showing an embodiment 3 of the present invention. In the present embodiment, a posture measuring means 21 for measuring posture to obtain posture information, a correspondence information database 22 for storing correspondence information between posture information and sound effects, and a sound for obtaining sound effect information from posture information and correspondence information. It is composed of effect information calculating means 23 and sound effect adding means 24 for changing sound signals according to sound effect information.

The posture measuring means 21 measures the posture of the user to obtain posture information. The posture information is information indicating what posture the user is in, and can be represented by angles of joints such as ankle joints and knee joints. The difference between the angle of these joints and the angle of the ideal user's joint can also be used as the posture information. To obtain the posture information, for example, a method using a device in which a sensor for measuring the position and orientation of the footrest is attached to a movable footrest (Japanese Patent Application No. 6-21)
No. 7472). There is also a method of measuring a posture by attaching one or more of a magnetic sensor, an optical sensor, an ultrasonic sensor, or the like or a combination thereof to a user's body. These attitude measurement methods are described in detail in Artificial Reality Generation Technology and its Application, Science Co., Ltd., 1992 (hereinafter referred to as Document 3).

The correspondence information database 22 holds correspondence information, which is a correspondence between attitude information and sound effects.
For example, as shown in FIG.
There is something like 1. This correspondence can be held in a function format or a table format. For example, FIG.
The correspondence between the attitude information x of 1 (A) and the sound effect y is as follows.
In a functional form, it can be represented by a mathematical expression such as y = ax + b. Here, a and b are parameters representing the correspondence between the posture information and the sound effect. The parameters of the function representing such correspondence are set in advance by the system designer as a = 2.0, b =
It is set as 0.5. As an example of this parameter, for example, when posture information such as a knee angle is 0 while the posture is standing, 0.5 is given as a sound effect, and for example, posture information such as a knee angle is 1.0. When a predetermined reference value is taken, 2.5 is given as a sound effect. Further, in the example of the function format, when the posture information increases, the sound effect is added in proportion to the degree of the increase in the posture information. FIG. 11A
Is shown in FIG. 12 in a table format. The width of the posture information, the range of the posture information, the value of the corresponding sound effect, the accuracy of the sound effect value, and the like used in the table are set in advance by the system designer. The correspondence information database 22 includes a ROM, a RAM with a power supply, a magnetic card, a magnetic disk device,
This can be realized using a storage device such as a magneto-optical disk device. The correspondence information database 22 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The sound effect information calculation means 23 receives the posture information from the posture measurement means 21 and the correspondence information from the correspondence information database 22, respectively, and refers to the sound effect information representing the sound effect corresponding to the posture information and the correspondence information. Ask. The correspondence information is in the form of a table in FIG.
If the posture information obtained from 1 is 3.0, the acoustic effect information is obtained as 2.5 by searching the table. In order to obtain the sound effect value from the posture information having no posture value in the table, the sound effect value can be dealt with by appropriately interpolating the values before and after the table or using the sound effect value having the closest posture information. Also, the correspondence information is represented by a function y = 2x + 0.5 between the posture information x and the sound effect y. If the posture information obtained from the posture measurement means 21 is 3.0, the sound effect is 6.5. Is calculated. Sound effect information calculation means 23
Can be implemented as, for example, a program that operates on a personal computer PC-9801 series manufactured by NEC Corporation.

The sound effect adding means 24 receives the sound signal from the outside and the sound effect information from the sound effect information calculating means 23, and changes the sound signal in accordance with the sound effect information. Output to As a method of changing the sound signal, the sound volume is changed according to the sound effect information, the sound in a predetermined range is amplified or attenuated according to the sound effect information, or the sound is amplified or attenuated according to the sound effect information There are methods of changing a sound range, changing an additional sound such as an echo reverb in accordance with sound effect information, changing a pitch in accordance with sound effect information, and changing a tempo in accordance with sound effect information. The sound effect adding means 24 includes an AD converter,
It is realized by a combination of an SP and a DA converter, and various sound effects can be added by a program on the DSP. The calculation of the sound effect addition is described in detail in Reference 1. Also, with a configuration in which an A / D converter and a D / A converter are added to a personal computer, it is also possible to describe and realize a program that causes the CPU of the personal computer to perform the sound effect addition operation. Further, a commercially available sound adding device can be used as the sound effect adding means 24. An example of a commercially available sound adding device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound effect calculating means 23 is implemented as a program on a personal computer, and the sound effect adding means 24 has a configuration using an external sound adding device,
During this time, the sound effect information can be communicated by a communication method conforming to the MIDI standard. The MIDI standard is described in detail in Reference 2.

Next, with reference to FIGS. 10, 12 and 13, of this embodiment, a posture measuring means 21 for obtaining the angle of an ankle joint as posture information is used, and volume information is changed as acoustic information. The table of FIG. 12 is used for associating the posture information with the sound volume information, the one stored in the ROM as the correspondence information database 22, and the sound effect information calculation means 23 is calculated by a personal computer program. Sound effect adding means 2
4 as Yamaha digital mixing processor DM
The operation will be described using a specific example, taking as an example a case where P7 is used and the sound effect information is communicated using the MIDI standard.

Digital mixing processor DMP7
Has a MIDI receiving terminal and converts the input sound signal to M
The output volume is set to 12 according to the message obtained by IDI.
The output can be changed in eight stages. Prior to the operation, the MIDI reception channel of DMP7 is set to 1 or OMNI so that DMP7 can receive the data output to MIDI channel1. In this setting, in order to change the output volume of DMP7, M
This can be realized by changing the stereo level of parameter number 135 of DMP7 with control number 4 of the control change of IDI. M for making control change of MIDI channel 1
The IDI message is a message having a 3-byte length of B0, control number, and control value in hexadecimal notation. As a result, the MIDI message for changing the volume of the DMP 7 is a message having a 3-byte length of B0, 4, and a volume value in hexadecimal notation. This volume value is a value from 0 to 127, and the output volume increases as the value increases.

The posture measuring means 21 measures the angle of the joint of the ankle and uses it as posture information (FIG. 13, step C1). Next, the sound effect calculating means 23 generates volume information corresponding to the measured posture from the posture information and the correspondence information held in the correspondence information database 22 (FIG. 13, step C).
2). Next, DMP7, which is the sound effect adding means 24,
Volume information is received from the sound effect information calculation means 23 through the MIDI cable, sound signals are respectively received from the sound signal input terminals, and the volume of the received sound signal is increased or decreased based on the sound volume information and output to the sound signal output terminal (FIG. 13, Step C3).

The operation of the sound effect calculation means 23 in the above description will be described in detail with reference to FIG. First, posture information is received from the posture measuring means 21 (FIG. 14, step C).
21). Next, the volume is searched from the correspondence table of FIG. 12 held in the correspondence information database 22 (FIG. 1).
4. Step C22). For example, if the posture information is 1.0, 1.5 is set as the search result. Next, the volume obtained from the table is normalized to the range of the volume handled by DMP7 (FIG. 1).
4. Step C23). If the range of the volume in the correspondence table of FIG. 12 is 0.0 to 15.0, for example, the volume obtained from the table is multiplied by 8 and rounded down to a whole number so that it can be normalized. . If the volume obtained from the table is 1.5, the normalized volume value is 24. Next, a MIDI message is generated based on the normalized volume (step C24 in FIG. 14). The MIDI message when the normalized volume value is 24 is B0, 4, or 18 in hexadecimal notation. Next, the generated M
An IDI message is transmitted to the DMP 7 (FIG. 14, step C25).

In the above description of the operation, the volume obtained by searching the correspondence information database 22 is normalized, and then the MID
Although the description has been made so as to generate the I message, a normalized volume value may be previously stored in the correspondence information database. In this case, the process of step C23 in FIG. 14 can be omitted.

Further, the sound effect information calculation and the sound effect adding process can be implemented as a program of a personal computer having an AD converter and a DA converter. In this case, the sound effect information can be communicated via a temporary storage area or the like of a personal computer, and the sound addition calculation is performed by externally performing A / D conversion on each sample value and volume value of the sound signal input by AD conversion. And can be obtained by multiplication. The obtained value is returned to DA
By performing the conversion, the sound signal to which sound is added can be output to the outside.

Further, it is also possible to mount the sound effect information calculation means 23 as a program of a personal computer and to mount the sound addition calculation means 24 using a DSP board dedicated to signal processing having an AD converter and a DA converter. is there. In this case, the personal computer and D
Sound effect information can be communicated through a bus to which the SP board is connected.

Hereinafter, a fourth embodiment of the present invention will be described with reference to the drawings. FIG. 15 is a block diagram showing one embodiment of the present invention 4. In the present embodiment, a posture measuring unit 31 that measures posture and obtains posture information, a correspondence information database 32 that stores correspondence information between posture information and sound effects, and a sound that obtains sound effect information from posture information and correspondence information. Effect information calculating means 33 and sound effect adding means 34 for changing sound signals according to sound effect information to obtain sound effect added sounds
A sound selection information database 35 for holding selection information indicating a correspondence between a posture and a sound to be selected; a sound selection information calculation unit 36 for obtaining sound selection information from the posture information and the selection information; And sound selection means 37 for obtaining a sound signal to be output from the additional sound.

The posture measuring means 31 measures the posture of the user to obtain posture information. The posture information is information indicating what posture the user is in, and can be represented by angles of joints such as ankle joints and knee joints. The difference between the angle of these joints and the angle of the ideal user's joint can also be used as the posture information. To obtain the posture information, for example, a method using a device in which a sensor for measuring the position and orientation of the footrest is attached to a movable footrest (Japanese Patent Application No. 6-21)
No. 7472). There is also a method of measuring a posture by attaching one or more of a magnetic sensor, an optical sensor, an ultrasonic sensor, or the like or a combination thereof to a user's body. These attitude measurement methods are described in detail in Reference 3.

The correspondence information database 32 holds correspondence information which is a correspondence between the posture information and the sound effect.
For example, as shown in FIG.
There is something like 1. This correspondence can be held in a function format or a table format. For example, FIG.
The correspondence between the attitude information x of 1 (A) and the sound effect y is as follows.
In a functional form, it can be represented by a mathematical expression such as y = ax + b. Here, a and b are parameters representing the correspondence between the posture information and the sound effect. The parameters of the function representing such correspondence are set in advance by the system designer as a = 2.0, b =
It is set as 0.5. As an example of this parameter, for example, when posture information such as a knee angle is 0 while the posture is standing, 0.5 is given as a sound effect, and for example, posture information such as a knee angle is 1.0. When a predetermined reference value is taken, 2.5 is given as a sound effect. Further, in the example of the function format, when the posture information increases, the sound effect is added in proportion to the degree of the increase in the posture information. FIG. 11A
Is shown in FIG. 12 in a table format. The width of the posture information, the range of the posture information, the value of the corresponding sound effect, the accuracy of the sound effect value, and the like used in the table are set in advance by the system designer. The correspondence information database 32 includes a ROM, a RAM with a power supply, a magnetic card, a magnetic disk device,
This can be realized using a storage device such as a magneto-optical disk device. The correspondence information database 32 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The sound effect information calculation means 33 receives the posture information from the posture measurement means 31 and the correspondence information from the correspondence information database 32, and refers to the sound effect information representing the sound effect corresponding to the posture information and the correspondence information. Ask. The correspondence information is in the form of a table in FIG.
If the posture information obtained from 1 is 3.0, the acoustic effect information is obtained as 2.5 by searching the table. In order to obtain the sound effect value from the posture information having no posture value in the table, the sound effect value can be dealt with by appropriately interpolating the values before and after the table or using the sound effect value having the closest posture.
In addition, a function y = correspondence between the attitude x and the sound effect y
If the posture information obtained from the posture measuring means 31 is 3.0, the sound effect is calculated to be 6.5. The sound effect information calculation means 33 is, for example, a personal computer PC-98 manufactured by NEC Corporation.
It can be implemented as a program that runs on the 01 series.

The sound effect adding means 34 receives a plurality of sound signals from the outside and the sound effect information from the sound effect information calculating means 33, respectively, and changes the received sound signals in accordance with the sound effect information. Generate sound. As a method of changing the sound signal, the sound volume is changed according to the sound effect information, the sound in a predetermined range is amplified or attenuated according to the sound effect information, or the sound is amplified or attenuated according to the sound effect information There are methods of changing a sound range, changing an additional sound such as an echo reverb in accordance with sound effect information, changing a pitch in accordance with sound effect information, and changing a tempo in accordance with sound effect information. The sound effect adding means 34 is an AD converter, a DS
It is realized by a combination of P and DA converters, and various sound effects can be added by a program on the DSP. The calculation of the sound effect addition is described in detail in Reference 1. Also, with a configuration in which an A / D converter and a D / A converter are added to a personal computer, it is also possible to describe and realize a program that causes the CPU of the personal computer to perform the sound effect addition operation. Further, a commercially available sound adding device can be used as the sound effect adding means 34. An example of a commercially available sound adding device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound effect information calculating means 33 is implemented as a program on a personal computer, and the sound effect adding means 34 is configured to use an external sound adding device, the communication between the sound effect information calculating means 33 and the communication means conforming to the MIDI standard is performed. Sound effect information can also be communicated. The MIDI standard is described in detail in Reference 2.

The selection information database 35 holds selection information which is information for associating posture information with sound selection information. FIG. 7 shows an example of a method of associating the posture information with the sound selection information. FIG. 16 (A)
FIG. 16B shows an example of sound selection information in a case where only one sound to be output is selected according to the posture information. This is an example in which the correspondence between the ratio and the posture information is used as the selection information.
The range of the posture information, the value of the corresponding sound selection information, and the like are set in advance by the system designer. The selection information database 35 can be realized using a storage device such as a ROM, a RAM with a power supply device, a magnetic card, a magnetic disk device, and a magneto-optical disk device. The selection information database 35 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The sound selection information calculation means 36 stores the posture information from the posture measurement means 31 into the sound selection information database 35.
, And obtains sound selection information representing sound selection corresponding to the posture information with reference to the selection information. If the selection information is in the form of a table shown in FIG. 16A and the posture information obtained from the posture measuring means 31 is 3.0, the sound selection information is determined to be 1 by searching the table. If there is no column of attitude information in the table corresponding to the measured attitude information, to obtain sound selection information,
This can be dealt with by appropriately interpolating the values before and after the table, or by using the sound selection information having the closest value to the posture information. The sound selection information calculation means 36 can be implemented as, for example, a program operating on a personal computer PC-9801 series manufactured by NEC Corporation.

In the sound selecting means 37, the sound adding means 34
And a plurality of sound additional sounds from the sound selection information calculating means 36, respectively, and a sound signal to be output from the plurality of sound additional sounds received according to the sound selection information is obtained and output to the outside. As the sound selection, there are a method of outputting only one of a plurality of input sounds, and a method of changing the volume level of the plurality of input sounds and mixing and outputting them. The sound selection means 37 is realized by a combination of a plurality of AD converters, DSPs, and DA converters, and can perform various sound selections by a program on the DSP. The sound selection process is described in detail in Document 1 and the like. Also, with a configuration in which a plurality of A / D converters and D / A converters are added to a personal computer, it is also possible to describe and realize a program that causes the CPU of the personal computer to perform the sound selection processing. Further, a commercially available sound selection device can be used as the sound selection means 37. An example of a commercially available sound selection device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound selection information calculation means 36 is implemented as a program on a personal computer, and the sound selection means 37 has a configuration using an external sound selection device,
During this time, the sound effect information can be communicated by a communication method conforming to the MIDI standard. The MIDI standard is described in detail in Reference 2.

Next, FIG. 12, FIG. 15, FIG. 16 and FIG.
In this embodiment, the attitude measuring means 31 of the present embodiment uses the one that determines the angle of the ankle joint as attitude information, uses the bass equalization amount as the acoustic information to be changed, and associates the attitude with the bass equalization amount. FIG. 12A is used for associating the posture with the selection information, and FIG. 16B is used for associating the posture with the selection information. Information calculation means 33 and sound selection information calculation means 3
6, a digital computer processor DMP7 is used as the sound effect adding means 34 and the sound selecting means 37, and the sound effect information and the sound selection information are M.
The operation will be described using a specific example, taking a case where communication is performed using the IDI standard as an example.

Digital mixing processor DMP7
Has a MIDI receiving terminal, and can set the bass equalization amount and the output volume of a plurality of input audio signals in accordance with the message obtained by MIDI, and can mix and output inputs of up to eight channels. it can.
Prior to the operation, the MID of the DMP 7 is set so that the DMP 7 can receive the data output to the MIDI channel 1.
The I reception channel is set to 1 or OMNI.
In this setting, in order to set the bass equalization amount of the input 1 of the DMP 7, it is necessary to use MIDI note-on 24 to set the bass equalization amount of the input 2.
The value can be set for each of the note-on 25th. The message to make the MIDI channel 1 note-on is in hexadecimal notation, 80, note number,
The message is a 3-byte long message of the velocity value.
Therefore, the MIDI message for setting the bass equalization amount of the input 1 of the DMP 7 is 80, 1 in hexadecimal notation.
A message having a length of 3 bytes, ie, C, equalization amount. Similarly, a MIDI message for setting the bass equalization amount of the input 2 of the DMP 7 is 80, in hexadecimal notation.
The message has a length of 3 bytes, ie, 1D, equalization amount. These bass equalization values are values from 0 to 127. The higher the value, the greater the output of the bass output. Also, to change the volume of input 1,
Control number 3 of MIDI control change
In order to change the volume of input 2, MIDI
Control number 33 of the control change,
Each value can be set. M for making control change of MIDI channel 1
The IDI message is a message having a 3-byte length of B0, control number, and control value in hexadecimal notation. Therefore, the MIDI message for setting the volume of the input 1 of the DMP 7 is represented by hexadecimal notation B0, B2,
The message has a length of 3 bytes of 0 and a volume value.
Similarly, the MIDI message for setting the volume of the input 2 of the DMP 7 is a message having a 3-byte length of B0, 21, and a volume value in hexadecimal notation. These sound volume values are values from 0 to 127, and the higher the value, the higher the output sound volume.

The posture measuring means 31 measures the angle of the joint of the ankle and uses it as posture information (FIG. 17, step D1). Next, the sound effect information calculating means 33 generates a MIDI message for setting a bass equalization amount corresponding to the measured posture from the posture information and the correspondence information held in the correspondence information database 32 (FIG. 17, step D2). . next,
The DMP 7 serving as the sound effect adding means 34 receives a MIDI message for setting the amount of bass equalization from the sound effect information calculating means 33 through the MIDI cable, receives sound signals from the sound signal input terminals, and performs bass equalization on the received sound signals. The amount is increased or decreased (FIG. 17, step D3). Next, sound selection information calculation means 3
6 generates sound selection information corresponding to the measured posture from the posture information and the selection information held in the selection information database 35 (FIG. 17, step D4). Next, the DMP 7 which is the sound selection means 37 is a sound selection information calculation means 36.
And the sound selection information is received, and the volume of the sound to which the sound is added is increased / decreased, mixed, and output to the sound signal output terminal (FIG. 17, step D5).

The sound effect information calculation means 3 in the above description
Operation 3 will be described in detail with reference to FIG. First, posture information is received from the posture measuring means 31 (FIG. 14, step C21). Next, the bass equalization amount is searched from the correspondence table of FIG. 3 held in the correspondence information database 32 (FIG. 14, step C22). For example, if the posture information is 1.0, 1.5 is set as the search result. Next, the bass equalization amount obtained from the table is normalized to the range of the bass equalization amount handled by the DMP 7 (FIG. 14, step C).
23). The range of the volume in the correspondence table of FIG.
From 15.0 to 15.0, the sound volume obtained from the table can be normalized by, for example, multiplying the volume by 8 to round it down to the next integer. If the volume obtained from the table is 1.5, the normalized bass equalization value is 24. Next, a MIDI message is generated based on the normalized bass equalization amount (step C24 in FIG. 14). MID when the normalized bass equalization amount is 24
The I message is expressed in hexadecimal notation, 80, 1C, 1C,
80 bytes, 1D, and 6 bytes. Next, the generated MIDI message is transmitted to the DMP 7 (FIG. 14, step C25).

Next, the operation of the sound selection calculating means 36 in the above description will be described in detail with reference to FIG. First, posture information is received from the posture measuring means 31 (FIG. 18, step D41). Next, the sound volume is searched from the correspondence table of FIG. 16B stored in the selection information database 35 (FIG. 18, step D42). For example, if the posture information is 1.
If it is 0, sound 1 is set to 0.0 and sound 2 is set to 1.0 as a search result. Next, the selection information obtained from the table is normalized to the range of the volume handled by the DMP 7 (FIG. 18, step D4).
3). If the range of the volume in the correspondence table of FIG. 12 is 0.0 to 1.0, for example, the volume obtained from the table is multiplied by 127 and rounded down to a whole number, and can be normalized. . If the sound 1 obtained from the table is 0.0, the volume value of the normalized sound 1 is 0, and the sound 2 obtained from the table is
Is 1.0, the volume value of the normalized sound 2 is 127
Becomes Next, a MIDI message is generated based on the normalized volume (FIG. 18, step D44). When the volume value of the normalized sound 1 is 0 and the volume value of the normalized sound 2 is 127, the MIDI message becomes B0, 20, 0, B0, 21, 7F in hexadecimal notation. . Next, the generated MIDI message is transmitted to the DMP 7 (FIG. 18, step D45).

Further, the sound effect information calculating means 33, the sound effect adding means 34, the sound selection information calculating means 36, and the sound selecting means 37 can be implemented as a program of a personal computer having an AD converter and a DA converter. . In this case, the sound effect information and the sound selection information are
Communication can be performed via a temporary storage area or the like of a personal computer. The sound addition calculation and sound selection processing include filtering processing and mixing processing for each sample value of an audio signal input by AD conversion from the outside. Can be determined by: The obtained value is returned to DA
By performing the conversion, the audio signal can be output to the outside as an audio signal subjected to the audio addition processing and the audio selection processing.

Further, the sound effect information calculating means 33 and the sound selection information calculating means 34 are implemented as a program of a personal computer, and the sound adding means 34 and the sound selecting means 35 are used for signal processing having a plurality of AD converters and DA converters. It can also be implemented as a dedicated DSP board. In this case, personal computer and DS
The sound effect information and the sound selection information can be communicated through a bus to which the P board is connected.

Hereinafter, a fifth embodiment of the present invention will be described with reference to the drawings. FIG. 19 is a block diagram showing a fifth embodiment of the present invention. In the present embodiment, a physiological index measuring unit 41 that measures a physiological index, a correspondence information database 42 that stores correspondence information between a physiological index and an acoustic effect, and a sound effect information calculation that obtains acoustic effect information from the physiological index and the corresponding information Means 43
And sound effect adding means 44 for changing sound signals according to sound effect information.

The physiological index measuring means 41 obtains a physiological index of the user. One of the physiological indices is the degree of tension. For obtaining the physiological index, for example, there is a method in which one or more physiological index measuring sensors are attached to the user and the physiological index of the user is measured from the outputs of these sensors. In order to measure the degree of tension, there is a method of attaching a pulse wave sensor to a user's fingertip. For a method of measuring physiological indices, see JP-A-6-2966.
See No. 13 or Reference 3.

The correspondence information database 42 holds correspondence information indicating correspondence between physiological indices and acoustic effects.
For example, FIG.
Something like 0. This correspondence can be held in a function format or a table format. For example, FIG.
The correspondence between the physiological index x of 0 (A) and the sound effect y is
In a functional form, it can be represented by a mathematical expression such as y = ax + b. Here, a and b are parameters representing the correspondence between the physiological index and the sound effect. The parameters of the function representing such correspondence are set in advance by the system designer as a = 2.0, b =
It is set as 0.5. An example of this parameter is, for example, when the user is not nervous and the degree of tension, which is one of the physiological indices, is 0, 0.5 is given as an acoustic effect, for example, the degree of tension, which is one of the physiological indices. Takes a predetermined reference value of 1.0, 2.5 is given as a sound effect. Further, in the example of the function form, when the physiological index increases, the acoustic effect is added in proportion to the degree of increase of the physiological index. Also,
FIG. 21 shows an example in which FIG. 20A is represented in a table format. The width of the physiological index used in the table, the range of the physiological index, the value of the corresponding acoustic effect, the accuracy of the acoustic effect value, and the like are set in advance by the system designer. Correspondence information database 42
Can be realized using a storage device such as a ROM, a RAM with a power supply device, a magnetic card, a magnetic disk device, and a magneto-optical disk device. The correspondence information database 42 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The sound effect information calculating means 43 stores the physiological index from the physiological index measuring means 41 into the correspondence information database 42.
, And obtains sound effect information representing a sound effect corresponding to the physiological index with reference to the corresponding information. If the correspondence information is in the form of a table in FIG. 21 and the physiological index obtained from the physiological index measuring means 41 is 3.0, the acoustic effect information is obtained as 2.5 by searching the table. In order to obtain the sound effect value from the physiological index having no physiological index value in the table, it can be dealt with by appropriately interpolating the values before and after the table or using the acoustic effect value closest to the physiological index. Further, the correspondence information is represented by a function y = 2x + 0.5 between the physiological index x and the acoustic effect y. If the physiological index obtained from the physiological index measuring means 41 is 3.0, the acoustic effect is 6.5. Is calculated. The sound effect information calculation means 43 can be implemented as a program that operates on, for example, a personal computer PC-9801 series manufactured by NEC Corporation.

The sound effect adding means 44 receives the sound signal from the outside and the sound effect information from the sound effect information calculating means 43, and changes the sound signal in accordance with the sound effect information. Output to As a method of changing the sound signal, the sound volume is changed according to the sound effect information, the sound in a predetermined range is amplified or attenuated according to the sound effect information, or the sound is amplified or attenuated according to the sound effect information There are methods of changing a sound range, changing an additional sound such as an echo reverb in accordance with sound effect information, changing a pitch in accordance with sound effect information, and changing a tempo in accordance with sound effect information. The sound effect adding means 44 includes an AD converter,
It is realized by a combination of an SP and a DA converter, and various sound effects can be added by a program on the DSP. The calculation of the sound effect addition is described in detail in Reference 1. Also, with a configuration in which an A / D converter and a D / A converter are added to a personal computer, it is also possible to describe and realize a program that causes the CPU of the personal computer to perform the sound effect addition operation. Further, a commercially available sound adding device can be used as the sound effect adding means 44. An example of a commercially available sound adding device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound effect calculating means 43 is implemented as a program on a personal computer and the sound effect adding means 44 has a configuration using an external sound adding device,
During this time, the sound effect information can be communicated by a communication method conforming to the MIDI standard. The MIDI standard is described in detail in Reference 2.

Next, with reference to FIGS. 19, 21 and 22, the degree of tension is calculated from the pulse wave obtained from the pulse wave sensor attached to the user as the physiological index measuring means 41 in this embodiment. Using the volume information as the acoustic information to be changed, using the table of FIG. 21 for the correspondence between the physiological index and the volume information, and using the information stored on the ROM as the correspondence information database 42, The sound effect information calculation means 43 uses a personal computer program, the sound effect addition means 44 uses a Yamaha digital mixing processor DMP7, and the sound effect information communicates using the MIDI standard. The operation will be described using a specific example.

The digital mixing processor DMP7 has a MIDI receiving terminal, and converts the input audio signal into MID.
In response to the message obtained in I, the output volume can be changed in 128 steps and output. Prior to the operation, the MIDI reception channel of DMP7 is set to 1 or OMNI so that DMP7 can receive the data output to MIDI channel1. In this setting, in order to change the output volume of DMP7, MI
This can be realized by changing the stereo level of the DMP7 parameter number 135 with the control number 4 of the DI control change. M for making control change of MIDI channel 1
The IDI message is a message having a 3-byte length of B0, control number, and control value in hexadecimal notation. As a result, the MIDI message for changing the volume of the DMP 7 is a message having a 3-byte length of B0, 4, and a volume value in hexadecimal notation. This volume value is a value from 0 to 127, and the output volume increases as the value increases.

The physiological index measuring means 41 calculates the degree of tension as a physiological index based on the pulse wave obtained from the pulse wave sensor (FIG. 22, step E1). Next, the sound effect calculation means 43 generates volume information corresponding to the measured physiological index from the physiological index and the corresponding information held in the corresponding information database 42 (FIG. 22, step E2). Next, the DMP 7 serving as the sound effect adding means 44 receives the sound volume information from the sound effect information calculating means 43 and the sound signal from the sound signal input terminal through the MIDI cable, and determines the sound volume of the received sound signal based on the sound volume information. It is increased or decreased and output to the audio signal output terminal (FIG. 22, step E3).

The operation of the sound effect calculation means 43 in the above description will be described in detail with reference to FIG. First, a physiological index is received from the physiological index measuring means 41 (FIG. 23, step E21). Next, the volume is searched from the correspondence table of FIG. 21 held in the correspondence information database 42 (FIG. 2).
3. Step E22). For example, if the physiological index is 1.0, 1.5 is set as the search result. Next, the volume obtained from the table is normalized to the range of volume handled by DMP7 (FIG. 2).
3, Step E23). If the range of the sound volume in the correspondence table of FIG. 21 is 0.0 to 15.0, the sound volume obtained from the table can be normalized by, for example, multiplying the sound volume obtained from the table by eight and truncating the decimal part so that it becomes an integer. . If the volume obtained from the table is 1.5, the normalized volume value is 24. Next, a MIDI message is generated based on the normalized volume (FIG. 23, step E24). The MIDI message when the normalized volume value is 24 is B0, 4, or 18 in hexadecimal notation. Next, the generated M
An IDI message is transmitted to the DMP 7 (FIG. 23, step E25).

In the above description of the operation, the volume obtained by searching the correspondence information database 42 is normalized,
Although the description has been made so as to generate the I message, a normalized volume value may be previously stored in the correspondence information database. In this case, the process of step E23 in FIG. 23 can be omitted.

Further, the sound effect information calculating means 43 and the sound effect adding means 44 can be implemented as a program of a personal computer having an AD converter and a DA converter. In this case, the sound effect information can be communicated via a temporary storage area or the like of the personal computer, and the sound addition processing includes each sample value and sound volume value of the sound signal input by AD conversion from the outside. And can be obtained by multiplication. By subjecting the obtained value to DA conversion again, a sound signal with sound added can be output to the outside.

Further, the sound effect information calculating means 43 can be mounted as a program of a personal computer, and the sound adding means 44 can be mounted by using a DSP board dedicated to signal processing having an AD converter and a DA converter. . In this case, the sound effect information can be communicated through the bus connecting the personal computer and the DSP board.

Hereinafter, a sixth embodiment of the present invention will be described with reference to the drawings. FIG. 24 is a block diagram showing a sixth embodiment of the present invention. In the present embodiment, a physiological index measuring means 51 for measuring a physiological index, a corresponding information database 52 for storing corresponding information between the physiological index and the acoustic effect, and a sound effect information calculation for obtaining the acoustic effect information from the physiological index and the corresponding information. Means 53, sound effect adding means 54 for obtaining sound effect added sound by changing the sound signal according to sound effect information, and sound selection information database 55 holding selection information indicating correspondence between physiological indices and sounds to be selected. Selection information calculation means 56 for obtaining sound selection information from physiological indices and selection information
And sound selecting means 57 for obtaining a sound signal to be output from the sound selection information and the sound effect added sound.

The physiological index measuring means 51 obtains the physiological index of the user. One of the physiological indices is the degree of tension. For obtaining the physiological index, for example, there is a method in which one or more physiological index measuring sensors are attached to the user and the physiological index of the user is measured from the outputs of these sensors. In order to measure the degree of tension, there is a method of attaching a pulse wave sensor to a user's fingertip. For a method of measuring physiological indices, see JP-A-6-2966.
See No. 13 or Reference 3.

The correspondence information database 52 holds correspondence information which is a correspondence between physiological indices and acoustic effects.
For example, FIG.
Something like 0. This correspondence can be held in a function format or a table format. For example, FIG.
The correspondence between the physiological index x of 0 (A) and the sound effect y is
In a functional form, it can be represented by a mathematical expression such as y = ax + b. Here, a and b are parameters representing the correspondence between the physiological index and the sound effect. The parameters of the function representing such correspondence are set in advance by the system designer as a = 2.0, b =
It is set as 0.5. An example of this parameter is, for example, when the user is not nervous and the degree of tension, which is one of the physiological indices, is 0, 0.5 is given as an acoustic effect, for example, the degree of tension, which is one of the physiological indices. Takes a predetermined reference value of 1.0, 2.5 is given as a sound effect. Further, in the example of the function form, when the physiological index increases, the acoustic effect is added in proportion to the degree of increase of the physiological index. Also,
FIG. 21 shows an example in which FIG. 20A is represented in a table format. The width of the physiological index used in the table, the range of the physiological index, the value of the corresponding acoustic effect, the accuracy of the acoustic effect value, and the like are set in advance by the system designer. Correspondence information database 52
Can be realized using a storage device such as a ROM, a RAM with a power supply device, a magnetic card, a magnetic disk device, and a magneto-optical disk device. The correspondence information database 52 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The sound effect information calculating means 53 stores the physiological index from the physiological index measuring means 51 into the correspondence information database 52.
, And obtains sound effect information representing a sound effect corresponding to the physiological index with reference to the corresponding information. If the correspondence information is in the form of a table shown in FIG. 21 and the physiological index obtained from the physiological index measuring means 51 is 3.0, the acoustic effect information is obtained as 2.5 by searching the table. In order to obtain the sound effect value from the physiological index having no physiological index value in the table, it can be dealt with by appropriately interpolating the values before and after the table or using the acoustic effect value closest to the physiological index. The correspondence information is represented by a function y = 2x + 0.5 between the physiological index x and the acoustic effect y. If the physiological index obtained from the physiological index measuring means 51 is 3.0, the acoustic effect is 6.5. Is calculated. The sound effect information calculation means 53 can be implemented as, for example, a program that operates on a personal computer PC-9801 series manufactured by NEC Corporation.

The sound effect adding means 54 receives a plurality of sound signals from the outside and the sound effect information from the sound effect information calculating means 53, respectively, and changes the received sound signals according to the sound effect information. Generate sound. As a method of changing the sound signal, the sound volume is changed according to the sound effect information, the sound in a predetermined range is amplified or attenuated according to the sound effect information, or the sound is amplified or attenuated according to the sound effect information There are methods of changing a sound range, changing an additional sound such as an echo reverb in accordance with sound effect information, changing a pitch in accordance with sound effect information, and changing a tempo in accordance with sound effect information. The sound effect adding means 54 includes an AD converter, a DS
It is realized by a combination of P and DA converters, and various sound effects can be added by a program on the DSP. The calculation of the sound effect addition is described in detail in Reference 1. Also, with a configuration in which an A / D converter and a D / A converter are added to a personal computer, it is also possible to describe and realize a program that causes the CPU of the personal computer to perform the sound effect addition operation. Further, a commercially available sound adding device can be used as the sound effect adding means 54. An example of a commercially available sound adding device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound effect information calculating means 53 is implemented as a program on a personal computer, and the sound effect adding means 54 is configured to use an external sound adding device, communication between the sound effect adding means 53 and the sound effect adding means 54 is performed in accordance with the MIDI standard. Sound effect information can also be communicated. The MIDI standard is described in detail in Reference 2.

[0093] The selection information database 55 holds selection information that is information for associating physiological indices with sound selection information. As a method of associating the physiological index with the sound selection information, for example, there is a method as shown in FIG. FIG. 25 (A)
FIG. 25B shows an example of the sound selection information when only one sound to be output is selected according to the physiological index. FIG. This is an example in which the correspondence between the mixing ratio and the physiological index is used as the sound selection information. The range of the physiological index, the value of the corresponding sound selection information, and the like are set in advance by the system designer. The selection information database 55 includes a ROM and an R
It can be realized using a storage device such as an AM, a magnetic card, a magnetic disk device, and a magneto-optical disk device. The selection information database 55 can be realized by storing the type of function, the function parameter, or each value of the table as data in these storage devices.

The acoustic selection information calculating means 56 stores the physiological index from the physiological index measuring means 51 into the selection information database 55.
, And obtains sound selection information representing sound selection corresponding to the physiological index with reference to the selection information. If the selection information is in the form of a table shown in FIG. 25A and the physiological index obtained from the physiological index measuring means 51 is 3.0, the acoustic selection information is determined to be 1 by searching the table. If there is no column of the physiological index in the table corresponding to the measured physiological index, in order to obtain the sound selection information, it is necessary to appropriately interpolate from the values before and after the table, or to select the acoustic selection information of the value closest to the physiological index. Can be dealt with. The sound selection information calculation means 56 can be implemented, for example, as a program operating on a personal computer PC-9801 series manufactured by NEC Corporation.

In the sound selecting means 57, the sound adding means 54
And the sound selection information from the sound selection report calculation means 56, and obtains an audio signal to be output from the received sound addition sounds in accordance with the sound selection information, and outputs it to the outside. As the sound selection, there are a method of outputting only one of a plurality of input sounds, and a method of changing the volume level of the plurality of input sounds and mixing and outputting them. The sound selecting means 57 is realized by a combination of a plurality of AD converters, DSPs and DA converters, and can perform various sound selections by a program on the DSP. The sound selection calculation is described in detail in Reference 1. Also, with a configuration in which a plurality of A / D converters and D / A converters are added to a personal computer, it is also possible to write and realize a program that causes the CPU of the personal computer to perform the sound selection calculation. Further, a commercially available sound selection device may be used as the sound selection means 57. An example of a commercially available sound selection device is a digital mixing processor DMP7 manufactured by Yamaha Corporation.

When the sound selection information calculation means 56 is implemented as a program on a personal computer, and the sound selection means 57 has a configuration using an external sound selection device,
During this time, the sound effect information can be communicated by a communication method conforming to the MIDI standard. The MIDI standard is described in detail in Reference 2.

Next, FIG. 21, FIG. 24, FIG. 25 and FIG.
With reference to the embodiment, the physiological index measuring means 51 of the present embodiment uses, as a physiological index, the degree of tension from a pulse wave obtained from a pulse wave sensor attached to the user's fingertip, as the acoustic information to be changed. FIG. 21 (A) is used for associating the physiological index with the bass equalization amount, and FIG. 25 is used for associating the physiological index with the sound selection information.
(B), the correspondence information database 52 and the selection information database 55 are those stored on the ROM, and the sound effect information calculation means 53 and the sound selection information calculation means 56 are those calculated by a personal computer program. The sound effect adding means 56 and the sound selecting means 57 use a digital mixing processor DMP7 manufactured by Yamaha Corporation.
The operation will be described using a specific example, taking a case where communication is performed using the IDI standard as an example.

Digital mixing processor DMP7
Has a MIDI receiving terminal, and can set the bass equalization amount and the output volume of a plurality of input audio signals in accordance with the message obtained by MIDI, and can mix and output inputs of up to eight channels. it can.
Prior to the operation, the MID of the DMP 7 is set so that the DMP 7 can receive the data output to the MIDI channel 1.
The I reception channel is set to 1 or OMNI.
In this setting, in order to set the bass equalization amount of the input 1 of the DMP 7, it is necessary to use MIDI note-on 24 to set the bass equalization amount of the input 2.
The value can be set for each of the note-on 25th. The message to make the MIDI channel 1 note-on is in hexadecimal notation, 80, note number,
The message is a 3-byte long message of the velocity value.
Therefore, the MIDI message for setting the bass equalization amount of the input 1 of the DMP 7 is 80, 1 in hexadecimal notation.
A message having a length of 3 bytes, ie, C, equalization amount. Similarly, a MIDI message for setting the bass equalization amount of the input 2 of the DMP 7 is 80, in hexadecimal notation.
The message has a length of 3 bytes, ie, 1D, equalization amount. These bass equalization values are values from 0 to 127. The higher the value, the greater the output of the bass output. Also, to change the volume of input 1,
Control number 3 of MIDI control change
In order to change the volume of input 2, MIDI
Control number 33 of the control change,
Each value can be set. M for making control change of MIDI channel 1
The IDI message is a message having a 3-byte length of B0, control number, and control value in hexadecimal notation. Therefore, the MIDI message for setting the volume of the input 1 of the DMP 7 is represented by hexadecimal notation B0, B2,
The message has a length of 3 bytes of 0 and a volume value.
Similarly, the MIDI message for setting the volume of the input 2 of the DMP 7 is a message having a 3-byte length of B0, 21, and a volume value in hexadecimal notation. These sound volume values are values from 0 to 127, and the higher the value, the higher the output sound volume.

In the physiological index measuring means 51, the degree of tension is calculated as a physiological index based on the pulse wave obtained from the pulse wave sensor (FIG. 26, step F1). Next, the sound effect information calculation means 53 generates a MIDI message for setting the bass equalization amount corresponding to the measured physiological index from the physiological index and the corresponding information held in the corresponding information database 52 (FIG. 26, Step F2). Next, the DMP 7 as the sound effect adding means 54 receives a MIDI message for setting the amount of bass equalization from the sound effect information calculating means 53 through the MIDI cable, receives a sound signal from the sound signal input terminal, and receives the sound signal. Increase or decrease the amount of bass equalization for the audio signal (FIG. 2)
6, Step F3). Next, the sound selection information calculation means 56
Generates selection information corresponding to the measured physiological index from the physiological index and the sound selection information held in the selection information database 55 (FIG. 26, step F4). Next, the DMP 7 as the sound selection means 57
6, the volume of the sound to which the sound is added is increased or decreased, mixed, and output to the sound signal output terminal (FIG. 26, step F5).

Sound effect information calculation means 5 in the above description
Operation 3 will be described in detail with reference to FIG. First, a physiological index is received from the physiological index measuring means 51 (FIG. 23, step E21). Next, the bass equalization amount is retrieved from the correspondence table of FIG. 3 held in the correspondence information database 52 (FIG. 23, step E22). For example, if the physiological index is 1.0, 1.5 is set as the search result. Next, the bass equalization amount obtained from the table is normalized to the range of the bass equalization amount handled by the DMP 7 (FIG. 23, step E).
23). The range of the volume in the correspondence table of FIG.
From 15.0 to 15.0, the sound volume obtained from the table can be normalized by, for example, multiplying the volume by 8 to round it down to the next integer. If the volume obtained from the table is 1.5, the normalized bass equalization amount is 24. Next, a MIDI message is generated based on the normalized bass equalization amount (FIG. 23, step E24). MID when the normalized bass equalization amount is 24
The I message is expressed in hexadecimal notation, 80, 1C, 1C,
80 bytes, 1D, and 6 bytes. Next, the generated MIDI message is transmitted to the DMP 7 (FIG. 23, step E25).

Next, the operation of the sound selection calculating means 56 in the above description will be described in detail with reference to FIG. First, a physiological index is received from the physiological index measuring means 51 (FIG. 27,
Step F41). Next, the sound volume is searched from the correspondence table of FIG. 25B held in the selection information database 55 (FIG. 27, step F42). For example, when the physiological index is 1.0, sound 1 is 0.0 and sound 2 is 1.0 as search results. Next, the selection information obtained from the table is normalized to the volume range handled by the DMP 7 (FIG. 27, step F).
43). The range of the volume in the correspondence table of FIG.
From 1.0 to 1.0, the sound volume obtained from the table can be normalized by, for example, multiplying the volume by 127 to round it down to the nearest integer. If the sound 1 obtained from the table is 0.0, the volume value of the normalized sound 1 is 0, and the sound 2 obtained from the table is
Is 1.0, the volume value of the normalized sound 2 is 127
Becomes Next, a MIDI message is generated based on the normalized volume (step F44 in FIG. 27). When the volume value of the normalized sound 1 is 0 and the volume value of the normalized sound 2 is 127, the MIDI message becomes B0, 20, 0, B0, 21, 7F in hexadecimal notation. . Next, the generated MIDI message is transmitted to the DMP 7 (FIG. 27, step F45).

The sound effect information means 53, sound effect adding means 54, sound selection information calculation means 56, and sound selection means 57 can be implemented as a program for a personal computer having an AD converter and a DA converter. In this case, the sound effect information and the sound selection information can be communicated via a temporary storage area or the like of a personal computer. It can be obtained by filtering and mixing for each sample value of the signal. By subjecting the obtained value to DA conversion again, a sound signal on which sound addition and sound selection processing has been performed can be output to the outside.

Further, the sound effect information calculation means 53 and the sound selection information calculation means 56 are implemented as a program of a personal computer, and the sound addition means 54 and the sound selection means 55 are used as signal processing having a plurality of AD converters and DA converters. It can also be mounted on a dedicated DSP board. In this case, the sound effect information and the sound selection information can be communicated through a bus connecting the personal computer and the DSP board.

[0104]

As described above, according to the present invention, it is possible to present an acoustic effect directly corresponding to a user's movement and a physical and mental state. Therefore, by using the present invention, it is possible to convey the meaning of the presented sound to the user in an easy-to-understand manner, as compared with the conventional device that cannot be presented as the acoustic change directly corresponding to the user's motion and the physical and mental state. As a result, it is possible to give the user a sense of realism and operation.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention 1. FIG.

FIG. 2 is an explanatory diagram for explaining an operation of one embodiment of the present invention 1;

FIG. 3 is an explanatory diagram for explaining an operation of one embodiment of the present invention 1;

FIG. 4 is a flowchart for explaining one embodiment of the present invention 1.

FIG. 5 is a flowchart for explaining one embodiment of the present invention 1.

FIG. 6 is a block diagram showing an embodiment 2 of the present invention.

FIG. 7 is an explanatory diagram for explaining an operation of one embodiment of the present invention 2;

FIG. 8 is a flowchart for explaining one embodiment of the present invention 2;

FIG. 9 is a flowchart for explaining one embodiment of the present invention 2.

FIG. 10 is a block diagram showing an embodiment 3 of the present invention.

FIG. 11 is an explanatory diagram for explaining an operation of one embodiment of the present invention 3;

FIG. 12 is an explanatory diagram for explaining the operation of one embodiment of the present invention 3;

FIG. 13 is a flow chart for explaining one embodiment of the present invention 3;

FIG. 14 is a flowchart for explaining one embodiment of the present invention 3;

FIG. 15 is a block diagram showing one embodiment of the present invention 4.

FIG. 16 is an explanatory diagram for explaining the operation of one embodiment of the present invention 4.

FIG. 17 is a flowchart for explaining one embodiment of the present invention 4.

FIG. 18 is a flowchart for explaining one embodiment of the present invention 4.

FIG. 19 is a block diagram showing an embodiment 5 of the present invention.

FIG. 20 is an explanatory diagram for explaining the operation of one embodiment of the present invention 5.

FIG. 21 is an explanatory diagram for explaining the operation of one embodiment of the present invention 5.

FIG. 22 is a flowchart for explaining one embodiment of the present invention 5.

FIG. 23 is a flowchart for explaining one embodiment of the present invention 6.

FIG. 24 is a block diagram showing one embodiment of the present invention 6.

FIG. 25 is an explanatory diagram for explaining the operation of one embodiment of the present invention 6.

FIG. 26 is a flowchart for explaining one embodiment of the present invention 6.

FIG. 27 is a flowchart for explaining one embodiment of the present invention 6.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Load measuring means 2 Corresponding information database 3 Sound effect calculating means 4 Sound effect adding means 11 Load measuring means 12 Corresponding information database 13 Sound effect calculating means 14 Sound effect adding means 15 Selection information database 16 Sound selecting information calculating means 17 Sound selecting means Reference Signs List 21 attitude information measuring means 22 correspondence information database 23 sound effect information calculating means 24 sound effect adding means 31 attitude information measuring means 32 correspondence information database 33 sound effect information calculating means 34 sound effect adding means 35 selection information database 36 sound selection information calculating means 37 sound selection means 41 physiological index measurement means 42 correspondence information database 43 sound effect information calculation means 44 sound effect addition means 51 physiological index measurement means 52 correspondence information database 53 sound effect information calculation means 54 with sound effect Means 55 selects information database 56 sound selection information calculating unit 57 sound selection means

Continuation of the front page (51) Int.Cl. ⁶ Identification symbol FI H04R 3/00 310 A61B 5/10 310 H04S 5/02 G10K 15/00 M (56) References JP-A-1-274780 (JP, A) JP-A-6-78998 (JP, A) JP-A-62-213732 (JP, A) JP-A-2-209169 (JP, A) JP-A-6-43890 (JP, A) JP-A-6-47170 (JP, A) JP-A-8-107600 (JP, A) JP-A-6-40804 (JP, U) JP-A-6-34691 (JP, U) "Sports of artificial reality using biological information" Application to Image Trainer ", Information Processing Society of Japan, vol. 92. no. 31. (H1-42) pp. 117-122 "Artificial Reality System Using Biological Information Development of Human Body Information Presenting Apparatus Human Interface Symposium, vol. 8th. ⁶ , DB name) A61B 5/10 H04R 7/00

Claims (13)

(57) [Claims] 1. A sound control device for controlling a sound to be presented in accordance with a state of a user, a load measuring means for measuring a position of a center of gravity of the user and outputting load information, and correspondence information between the load information and the sound effect. A correspondence information database for holding and outputting the correspondence information, a sound effect information calculation means for obtaining sound effect information from the load information and the correspondence information, and the sound effect information and an externally supplied sound signal as inputs. A sound control device comprising: a sound effect adding unit that changes a sound signal according to sound effect information and outputs the sound signal to the outside. 2. A sound control device for controlling a sound to be presented in accordance with a state of a user, a load measuring means for measuring a position of a center of gravity of the user and outputting load information, and correspondence information between the load information and the sound effect. A correspondence information database for storing the correspondence information and outputting the correspondence information; a sound effect information calculating unit for obtaining sound effect information from the load information and the correspondence information; and inputting the sound effect information and a plurality of sound signals given from outside. A sound effect adding unit for obtaining a sound effect added sound to which a sound effect is added based on the sound effect information; and a sound holding selection information indicating a correspondence between a load and a sound to be selected and outputting the selection information. A selection information database, sound selection information calculation means for obtaining sound selection information from the load information and the selection information, and inputting the sound selection means and the sound effect added sound Sound control apparatus characterized by having a sound selection means for outputting to the outside seeking to acoustic signals to be output by the sound selection information. 3. The acoustic control device according to claim 1, wherein the load measuring means includes a means for measuring an amount of movement of the center of gravity due to the movement of the center of gravity of the user. 4. The acoustic control device according to claim 1, wherein the correspondence information database holds correspondence information indicating a correspondence between the load information and the sound effect in a function form or a table form. 5. A sound control apparatus for controlling sound to be presented according to a state of a user, a posture measuring means for measuring a posture of the user and outputting posture information, and correspondence information between the posture information and the sound effect. A correspondence information database that holds and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the posture information and the correspondence information, and a sound signal that receives the sound effect information and a sound signal given from outside. A sound control device comprising: a sound effect adding unit that changes a sound signal according to effect information and outputs the sound signal to the outside. 6. A sound control device for controlling a sound to be presented according to a state of a user, a posture measuring means for measuring a posture of the user and outputting posture information, and correspondence information between the posture information and the sound effect. A correspondence information database that holds and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the posture information and the correspondence information, and a plurality of sound signals given from outside as the sound effect information. Sound effect adding means for obtaining a sound effect added sound to which a sound effect is added based on the sound effect information; and sound selection for holding selection information indicating a correspondence between a posture and a sound to be selected and outputting the selection information. An information database, sound selection information calculation means for obtaining sound selection information from the posture information and the selection information, and the sound selection means and the sound effect added sound as inputs. Serial sound control apparatus characterized by having a sound selection means for outputting to the outside determined acoustic signal output by the acoustic selection information. 7. The posture measuring means includes means for measuring a joint angle of a lower body of a user to obtain posture information.
Or the acoustic control device according to 6. 8. The acoustic control device according to claim 7, wherein the posture measuring means is means for measuring an angle of a joint of a user's ankle or an angle of a knee. 9. The acoustic control device according to claim 5, wherein the correspondence information database holds the correspondence information indicating the correspondence between the posture information and the sound effect in a function form or a table form. 10. An acoustic control device for controlling a presented sound according to a state of a user, a physiological index measuring means for measuring a physiological index of the user and outputting a physiological index, and a correspondence between the physiological index and an acoustic effect. A correspondence information database that retains information and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the physiological index and the correspondence information, and an input of the sound effect information and an externally supplied sound signal. A sound effect adding means for changing a sound signal according to the sound effect information and outputting the sound signal to the outside. 11. A sound control device for controlling a presented sound according to a state of a user, a physiological index measuring means for measuring a physiological index of the user and outputting posture information, and a correspondence between the physiological index and the sound effect. A correspondence information database that retains information and outputs the correspondence information, a sound effect information calculation unit that obtains sound effect information from the physiological index and the correspondence information, and a plurality of sound signals provided from outside and the sound effect information. Sound effect adding means for obtaining a sound effect-added sound to which a sound effect is added based on the sound effect information as input, and holding selection information indicating correspondence between a physiological index and a sound to be selected, and outputting the selection information Sound selection information database, sound selection information calculation means for obtaining sound selection information from the physiological index and the selection information, the sound selection means and the sound effect Sound control apparatus characterized by having a sound selection means for outputting to the outside seeking sound signal outputted by said acoustic selection information as input sound. 12. The physiological index measuring means is means for measuring a user's pulse wave to determine the user's degree of tension.
Or the acoustic control device according to 11. 13. The acoustic control device according to claim 10, wherein the correspondence information database holds correspondence information indicating a correspondence between the posture information and the sound effect in a function format or a table format.