JP4484570B2 - Acoustic information processing apparatus and acoustic information providing method - Google Patents

Description

  The present invention relates to an apparatus and method for processing acoustic information, and more particularly to an apparatus and method for providing acoustic information to a user in a new form.

  Conventionally, various sounds such as music and environmental sounds have been processed by a computer to enable sound output. In recent years, it is possible to generate and output sound by, for example, synthesizing a plurality of sounds. An example of this conventional acoustic information processing will be described using the screen display example shown in FIG.

  FIG. 5 shows a sound source 2 as a sound transmission source in a virtual space displayed on the screen, an audible area 4 in which a sound output from each sound source 2 can be heard, and a recording point 6 for inputting the transmitted sound. Is shown. In FIG. 5, the sound source 2 is installed in five places. In addition, the audible area 4 is formed in a circular shape centering on each sound source 2.

  As shown in FIG. 5, when the recording point 6 is not included in any audible area 4, nothing is heard. When the user moves the recording point 6 to, for example, the audible area 4a by operating the mouse or the like, the sound output from the sound source 2a located at the center of the audible area 4a is output from the built-in speaker or an external speaker. Further, when the recording point 6 is moved to an area 4c where the audible area 4a and the audible area 4 overlap, sounds output from the sound sources 2a and 2b located at the centers of the audible areas 4a and 4b are synthesized. Is output. As described above, conventionally, the sound output from one or a plurality of sound sources is provided to the user so that the user can visually confirm the sound.

  On the other hand, conventionally, an apparatus that provides information by touch is proposed. For example, Patent Document 1 discloses an apparatus that provides sound information by tactile sensation by generating tactile information based on sound information.

JP 2003-99177 A

  However, in the conventional information processing apparatus, the change in sound and the state of the synthesized sound can be provided by hearing or vision, but not by tactile sense. On the other hand, Patent Document 1 can generate tactile information based on audio information and provide the information to the user, but does not refer to generating tactile information based on the positional relationship between the sound source and the recording points.

  SUMMARY An advantage of some aspects of the invention is to provide an acoustic information processing apparatus and an acoustic information providing method capable of providing a user with an acoustic state by touch.

In order to achieve the above-described object, the acoustic information processing apparatus according to the present invention includes a presentation part that is touched by a user and is movable in a two-dimensional or three-dimensional direction, and one or more sound sources in a virtual space. distance between the point in rather based, and transmitting audio information generating means for generating audio information including information on the sound pressure when the audio information is transmitted to the recording point outputted from the respective sound sources, included in the audio information Drive means for driving the presenting part in a direction based on the direction of the sound source with respect to the recording point with a driving force according to information relating to the sound pressure to be recorded .

  The image processing apparatus further includes display processing means for displaying a positional relationship between each sound source and recording points in the virtual space.

Further, the display processing means displays the acoustic information output from each sound source on a screen so that ripples spread radially from each sound source .

Further, the display processing means displays a ripple from each sound source on the screen at a timing when the sound is emitted from each sound source according to the sound information, and the drive means is connected to each sound source after the sound from each sound source is emitted. The presenting part is driven with a delay corresponding to the distance from the recording point .

  In addition, it has an operation means for changing the position of the sound source or the recording point in the virtual space displayed on the screen.

Further, the driving means drives the presenting part by reflecting the change of the position of the sound source or the recording point .

Acoustic information providing method according to the present invention, more touched to the user is carried out in an information processing apparatus tactile information presentation devices are connected with the possible presentation site moves in a two-dimensional or three-dimensional direction, the user audio information in acoustic information providing method for providing, including information regarding sound pressure when rather based on the distance between the recording points and the one or more sound sources in the virtual space, sound information output from the sound source is transmitted to the recording point A transmission acoustic information generation step for generating acoustic information, and a driving step for driving the presenting part in a direction based on the direction of the sound source with respect to the recording point with a driving force according to information on sound pressure included in the acoustic information. It is characterized by including.

  According to the present invention, acoustic information can be presented to a user by touch. Thereby, the range of how to enjoy sound can be expanded.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a hardware configuration diagram showing an embodiment of an acoustic information processing apparatus according to the present invention. As shown in FIG. 1, the acoustic information processing apparatus 10 according to the present embodiment can be realized by a general-purpose computer having a display screen 12 such as a notebook personal computer. A tactile information presentation apparatus 14 is connected to the computer. Composed. The tactile information presentation device 14 can be realized with the same configuration as the tactile information presentation device disclosed in Patent Document 1, for example. In short, the tactile information presentation device 14 has a structure in which a presentation part 16 that can be moved in a two-dimensional or three-dimensional direction is mounted on an existing mouse. By driving, acoustic information and the like can be provided to the user by touch.

  FIG. 2 is a block configuration diagram of the acoustic information processing apparatus 10 according to the present embodiment. The acoustic information processing apparatus 10 includes an acoustic information generation processing unit 20, a tactile information generation processing unit 22, a presentation site driving unit 24, an operation reception unit 26, a position detection unit 28, a virtual space information management unit 30, a sound output processing unit 32, and A display processing unit 34 is included. In the present embodiment, the source sound source storage unit 40 and the virtual space information storage unit 42 are provided. The source sound source storage unit 40 stores acoustic information and the virtual space information storage unit 42 stores information related to the virtual space in advance. It is registered. The acoustic information registered in the source sound source storage unit 40 is acoustic information output from each sound source in the virtual space. In the present embodiment, it is referred to as a “source sound source” in order to distinguish it from the acoustic information generated by the acoustic information generation processing unit 20. If it demonstrates using the example of a display screen shown in FIG. 5, in the information regarding the virtual space registered into the virtual space information memory | storage part 42, the virtual space represented by coordinate data and movably installed in the virtual space are installed. Each sound source 2 and each position information of the recording point 6 are included. The source sound source storage unit 40 and the virtual space information storage unit 42 are realized by an external storage device such as a hard disk drive device. However, in the virtual space information, each position information of the sound source 2 and the recording point 6 is updated as needed by movement and is used for tactile information generation processing to be performed at high speed. It is desirable to store all information in a memory or the like. In addition, since the acoustic information and the tactile information are sequentially updated as the sound source 2 and the recording point 6 move, and are used for sound output processing that should be performed at high speed, the storage units 44 and 46 described later are It is desirable to implement with a storage device such as a memory suitable for high-speed processing.

  The acoustic information generation processing unit 20 generates acoustic information when the acoustic information output from each sound source in the virtual space is transmitted to the recording point based on the virtual space information stored in the virtual space information storage unit 42. And stored in the acoustic information storage unit 44. The tactile information generation processing unit 22 generates tactile information based on the acoustic information registered in the source sound source storage unit 40 and the virtual space information stored in the virtual space information storage unit 42 and stores it in the tactile information storage unit 46. To do. The presentation site driving unit 24 drives the presentation site 16 based on the haptic information stored in the haptic information storage unit 46. Different acoustic patterns can be presented to the user with different tactile sensations by varying the driving pattern of the presenting portion 16. The user can move the recording point 6 or the sound source 2 on the display screen by operating the tactile information presentation device 14 also serving as a mouse or a keyboard provided as an operation means. Accept the contents of user operations. The operation content is mainly a change in the position of the recording point 6 or the sound source 2 in the virtual space, and the position detection unit 28 determines the position of the recording point 6 or the sound source 2 in the virtual space moved according to the user operation. Is detected. The virtual space information management unit 30 updates the position information of the moving point set in the virtual space information storage unit 42 based on the position of the recording point 6 or the sound source 2 detected by the position detection unit 28. The sound output processing unit 32 outputs acoustic information from a built-in speaker or an externally connected speaker. The display processing unit 34 generates visible information from each of the source sound source, the acoustic information, and the virtual space information and displays the information on the display screen 12. Note that the display screen 12 shown in FIG. 1 shows a display example based on the virtual space information.

  The present embodiment is characterized in that the sound when the source sound source is heard at the recording point 6 is generated and the sound is provided to the user by touch. Hereinafter, this characteristic tactile presentation process will be described with reference to the flowchart shown in FIG. This embodiment will be described using a virtual space similar to FIG.

  The user displays the virtual space on the display screen 12 by performing a predetermined operation. A predetermined source sound source is output from the sound source 2 displayed in the virtual space. As shown in FIG. 1, when the recording point 6 is not included in any audible area 4, the source sound source does not reach and nothing can be heard. Here, it is assumed that the user moves the recording point 6 by operating a mouse or the like and enters the audible area 4a, for example, in any audible area (steps 101 and 102). The positional relationship among the sound source 2a, the audible area 4a and the recording point 6 at this time is shown in FIG. The acoustic information generation processing unit 20 acquires the acoustic information output from the sound source 2a from the source sound source storage unit 40, and further virtual information on the virtual space, specifically the position information of the sound source 2a and the recording point 6 in the virtual space. Obtained from the spatial information storage unit 42 (step 103). Then, the acoustic information generation processing unit 20 obtains the distance and direction from the sound source 2a to the recording point 6 based on the position information of the sound source 2a and the recording point 6. Note that the virtual space here is a narrow space such as a room, the distance between the sound source 2a and the recording point 6 is short, and the time lag that the source sound source emitted from the sound source 2 reaches the recording point 6 can be ignored. . Accordingly, the acoustic information generation processing unit 20 generates information as acoustic information by adding information on the reception state such as sound pressure when the source sound source is transmitted to the source sound source (step 104). Then, the generated acoustic information is registered in the acoustic information storage unit 44.

  Similarly, the tactile information generation processing unit 22 acquires acoustic information output from the sound source 2a from the source sound source storage unit 40, and acquires position information of the sound source 2a and the recording point 6 in the virtual space from the virtual space information storage unit 42. (Step 103). The tactile information generation processing unit 22 generates tactile information based on the acquired information (step 106). Before describing this generation processing, the tactile information generation processing unit 22 is driven by the presentation site driving unit 24 based on the generated tactile information. The operation of the presenting part 16 will be described.

  As shown in FIG. 4A, the sound in the arrow A direction is transmitted to the recording point 6 from the sound source 2a located in the upper right direction of the drawing. Therefore, the presentation part drive unit 24 drives the presentation part 16 provided in the tactile information presentation device 14 in the direction of the arrow B in the same direction as the arrow A from the neutral position, as shown in FIG. The driving timing is driven in the direction of arrow B according to the rhythm, for example. It should be noted that the presentation part 16 operates to return to the neutral position at the timing when it is not driven. The driving force to the presentation site 16 is determined by the sound pressure. That is, the presentation part 16 is driven with a strong driving force when the transmitted sound pressure is strong, and is driven with a weak driving force when the sound pressure is weak. In this way, the sound pressure level is expressed. Thereby, the user obtains a tactile sensation in which sound is transmitted from the sound source 2. In addition, the internal process regarding the drive of this presentation site | part 16 is demonstrated later.

  As described above, the haptic information generation processing unit 22 generates haptic information based on the direction of the sound source 2a obtained from the virtual space information and the sound pressure obtained from the acoustic information, and the generated haptic information is stored in the haptic information storage unit 46. Register with. And as mentioned above, the presentation site | part drive part 24 provides acoustic information to a user by a tactile sense by driving the presentation site | part 16 based on the produced | generated tactile information (step 107).

  In addition, the acoustic information is provided by hearing simultaneously with the driving of the presentation site 16 based on the acoustic information. That is, the sound output processing unit 32 outputs sound from the speaker based on the sound information registered in the sound information storage unit 44 (step 105). The sound output processing itself in the sound output processing unit 32 may be the same as the conventional one. Further, after entering the audible area 4, if the display processing unit 34 displays the screen so that ripples spread radially from the sound source 2 based on the acoustic information, it is possible to visually confirm how the sound is transmitted. it can. The speed at which the ripples spread and the magnitude and interval of the wave phase are changed in accordance with acoustic information such as sound pressure, acoustic frequency and amplitude, or changes in acoustic information.

  As described above, the acoustic information processing apparatus 10 can present acoustic information to the user by touch. The user can feel the sound by tactile sensation, but can enjoy more by receiving the sound both by tactile sense and by hearing. Furthermore, since the sound can be enjoyed through vision by changing the screen display according to the sound, it is more effective.

  By repeatedly executing the above processing (steps 101 to 107), the acoustic information processing apparatus 10 continuously provides acoustic information by tactile sense. For example, the recording point 6 moves in a direction approaching the sound source 2a. Then, since the distance between the sound source 2 and the recording point 6 approaches, the direction in which the presentation part 16 is driven is driven so strongly that the sound pressure increases even in the same B direction. As a result, the user obtains “movement” that the recording point 6 approaches the sound source 2 a, in other words, “change in positional relationship” by tactile sense.

  For example, if the recording point 6 is moved in the direction of the audible area 4b and moved to the area 4c where the audible area 4a and the audible area 4b overlap, the acoustic information generation processing unit 20 is output from the sound sources 2a and 2b. Are acquired from the virtual space information storage unit 42 (step 103), and acoustic information relating to the sound source 2a and acoustic information relating to the sound source 2b are generated. When the sound source is included only in the single audible area 4, the source sound source may be included in the acoustic information as it is as described above. To generate acoustic information at the recording point 6 located in the region 4c (step 104). At the same time, the tactile information generation processing unit 22 generates tactile information based on the acoustic information from each sound source and the positional relationship between the sound source that outputs the acoustic information and the recording point. Provides the user with a tactile sensation generated by synthesizing a tactile sensation generated by the source sound source from the sound source 2a located in the lower right direction of the screen and a tactile sensation generated by the source sound source from the sound source 2b located in the upper left direction of the screen. Will do. In addition, the sound output processing unit 32 provides the user with a synthesized sound of each sound from the sound sources 2a and 2b.

  Here, the generation processing in the tactile information generation processing unit 22 will be described. The tactile information generation processing unit 22 generates tactile information based on the input acoustic information and virtual space information. First, the acoustic information is extracted from the acoustic information storage unit 44, and the acoustic data included in the acoustic information is converted to frequency. To divide. Then, the time change of the specific frequency band is always detected, and the tactile information is generated so that the presentation site 16 is driven according to the detected time change. The driving timing (driving pattern) is determined by the amount of time change. The specific frequency band described above is set in advance. For example, when the presentation portion 16 is driven in response to a rhythm base sound, the frequency of the rhythm base sound is set to be sampled. . Of course, it may be possible to change which band is the specific frequency band. The driving force is determined by sound pressure data included in the acoustic information as described above.

  As described above, the driving pattern and driving force of the presentation site 16 are determined based on the acoustic information. In the present embodiment, the driving force is further adjusted according to the distance between the sound source 2 and the recording point 6. . That is, the tactile information generation processing unit 22 extracts the position information of the sound source 2 and the recording point 6 that form the audible area 4 including the recording point 6 from the input virtual space information, and the sound source 2 and the recording point 6. Find the distance to. Then, the determined driving force is increased or decreased according to the distance. More specifically, the attenuation rate of the source sound source output from the sound source 2 is determined according to the distance between the sound source 2 and the recording point 6, and the driving force is determined based on the attenuation rate.

  Next, in this embodiment, it is necessary to determine the direction in which the presentation site 16 is driven based on the positional relationship between the sound source 2 and the recording point 6. That is, the tactile information generation processing unit 22 determines the transmission direction of the sound emitted from the sound source 2 to the recording point 6 based on the extracted position information of the sound source 2 and the recording point 6. This direction is the driving direction of the presentation site 16. When included in a plurality of audible areas 4, the direction in which the directions are combined is the driving direction as described above.

  By the way, in the above description, since the virtual space is assumed to be a narrow space such as a room, the time lag when the source sound source emitted from the sound source 2 reaches the recording point 6 is ignored. Here, a case where this time lag is taken into account will be described.

  For example, consider a case where a concert is held in a wide outdoor area. The virtual space in this case is a wide outdoor place, and the sound emitted from the sound source 2 reaches the recording point 6 with some delay. In this case, the acoustic information generation processing unit 20 obtains a distance between the sound source 2 and the recording point 6 based on the virtual space information, and generates acoustic information with a delay corresponding to the distance. As a result, the drive unit presentation part 16 drives the presentation part 16 with a delay from the output of the source sound source. Thereby, the presentation part drive part 24 drives the presentation part 16, and the sound output process part 32 outputs a sound from a speaker, after a sound is emitted from the source sound source.

  However, if the user cannot grasp the timing when the sound is emitted from the source sound source, the user cannot recognize the time lag. Therefore, the display processing unit 34 displays the screen so that the output timing of the source sound source from the sound source 2 can be known. Thereby, the user can recognize the sound output timing from the sound source visually, and can recognize the occurrence of the time lag by tactile sense and hearing.

  In addition, although the explanation was divided into the case where time lag was considered and the case where it was ignored without being considered, the processing itself does not need to be separated, and tactile information is generated by always considering the distance between the sound source 2 and the recording point 6. What should I do? When the virtual space is a narrow space such as a room, even if the time lag is reflected in the tactile information, the time lag is very small and the user can only feel the time lag.

  According to the present embodiment, since sound can be presented not only by hearing and vision but also by tactile sense, the range of how to enjoy sound can be expanded.

  In the present embodiment, as described above, basically, the sound state that the source sound source output from the sound source 2 is transmitted to the recording point 6 at what sound pressure and in what direction is described. Although provided by tactile sense, information based on the positional relationship between each sound source and recording points in the virtual space may be provided as well as the acoustic state. For example, in the above description, it has been described that “movement” (“change in positional relationship”) that the recording point 6 approaches the sound source 2a is provided by tactile sense. In other words, the change in sound pressure accompanying the movement of the recording point 6 is reflected in the tactile sense, but if it moves around the sound source 2 while maintaining the distance from the sound source 2, the position changes due to the movement. It may be possible to provide information indicating that the user is in touch. In addition, the relative positional relationship between the sound source 2 and the recording point 6 is changed even when the sound source 2 is moved without being limited to the movement of the recording point 6. You may make it drive.

It is a hardware block diagram which showed one Embodiment of the acoustic information processing apparatus which concerns on this invention. It is a block block diagram of the acoustic information processing apparatus in this Embodiment. It is the flowchart which showed the process which presents the acoustic information in this Embodiment by a tactile sense. It is the figure which showed each positional relationship of a sound source, an audible area | region, and a recording point in this Embodiment. It is the figure which showed each positional relationship of the sound source in a virtual space, an audible area, and a recording point.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Acoustic information processing apparatus, 12 Display screen, 14 Tactile information presentation apparatus, 16 Presentation part, 20 Acoustic information generation process part, 22 Tactile information generation process part, 24 Presentation part drive part, 26 Operation reception part, 28 Position detection part, 30 virtual space information management unit, 32 sound output processing unit, 34 display processing unit, 36 acoustic information storage unit, 40 source sound source storage unit, 42 virtual space information storage unit, 44 acoustic information storage unit, 46 tactile information storage unit.

Claims (7)

A presentation part touched by a user and movable in a two-dimensional or three-dimensional direction ;
Based rather on the distance between the recording points and the one or more sound sources in the virtual space, transfer audio information for generating audio information including information on the sound pressure when the audio information output from the sound sources is transmitted to the recording point Generation processing means ;
Driving means for driving the presenting part in a direction based on the direction of the sound source with respect to the recording point with a driving force according to information on sound pressure included in the acoustic information ;
An acoustic information processing apparatus comprising: The acoustic information processing apparatus according to claim 1,
An acoustic information processing apparatus comprising display processing means for displaying on screen a positional relationship between each sound source and recording points in a virtual space. In the acoustic processing device according to claim 2,
The display processing means displays acoustic information output from each sound source on the screen so that ripples spread radially from each sound source . The acoustic information processing apparatus according to claim 2 ,
The display processing means displays a ripple from each sound source on the screen at a timing when sound is emitted from each sound source according to the sound information,
The acoustic information processing apparatus , wherein the driving means drives the presentation site with a delay corresponding to the distance between each sound source and a recording point after sound from each sound source is emitted . The acoustic information processing apparatus according to claim 1,
An acoustic information processing apparatus having an operation means for changing a position of a sound source or a recording point in a virtual space displayed on a screen. The acoustic information processing apparatus according to claim 5,
The acoustic information processing apparatus , wherein the driving means drives the presentation site by reflecting a change in a position of a sound source or a recording point . More touched to the user is carried out in an information processing apparatus tactile information presentation devices are connected with a movable presentation site in a two-dimensional or three-dimensional directions, the acoustic information providing method for providing sound information to the user,
Based rather on the distance between the recording points and the one or more sound sources in the virtual space, transfer audio information for generating audio information including information on the sound pressure when the audio information output from the sound sources is transmitted to the recording point Generation step ;
A driving step of driving the presenting part in a direction based on a direction of the sound source with respect to the recording point with a driving force according to information on sound pressure included in the acoustic information ;
A method for providing acoustic information, comprising:

Images