CN115407908A - Sound effect display method and device and computer readable storage medium - Google Patents

Abstract

The application discloses a sound effect display method, a device and a computer readable storage medium, which are applied to the technical field of augmented reality, wherein the sound effect display method is applied to AR equipment, and comprises the following steps: displaying a virtual audio device and an audio adjustment icon of the virtual audio device through an AR device; receiving an adjusting operation of a user for the audio adjusting icon, wherein the adjusting operation at least comprises one of a gesture interaction operation, a touch interaction operation, a voice interaction operation and an eye movement interaction operation; and responding to the adjusting operation, and displaying the first sound effect image corresponding to the adjusting operation. The technical problem that the perceptibility of the sound effect adjusting process is poor due to the fact that the sound effect of the AR equipment is abstracted in the prior art is solved.

Description

Sound effect display method, device and computer-readable storage medium

technical field

The present application relates to the field of augmented reality technology, and in particular to a sound effect display method, device and computer-readable storage medium.

Background technique

AR (Augmented Reality, Augmented Reality) technology is a human-computer interaction technology that superimposes digital information such as virtual 3D model animation, video, text, and pictures into real scenes in real time, and realizes natural interaction with real objects or users. Along with the development of AR technology, audio technology also plays a role that cannot be ignored in AR devices. However, in the prior art, AR devices usually use buttons set on the AR device to directly adjust the sound effect of the audio. However, this In this way, the sound effect of the AR device is relatively abstract, and it is difficult for the user to intuitively perceive the change process of different sound effects, and the adjustment process of the sound effect is less perceivable.

Contents of the invention

The main purpose of this application is to provide a sound effect display method, an electronic device and a computer-readable storage medium, aiming to solve the technical problem in the prior art that the sound effect of the AR device is relatively abstract, resulting in poor perceptibility of the sound effect adjustment process.

In order to achieve the above purpose, this application provides a method for displaying sound effects, which is applied to AR devices. The method for displaying sound effects includes:

Displaying a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device;

receiving a user's adjustment operation on the audio adjustment icon, wherein the adjustment operation includes at least one of gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement interaction operation;

In response to the adjustment operation, a first sound effect image corresponding to the adjustment operation is displayed.

Optionally, before the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, the method includes:

Obtain the current audio signal of the AR device;

Generate and display a second sound effect image corresponding to the current audio signal according to the current audio signal.

Optionally, the audio adjustment icon includes a volume adjustment icon, and the step of displaying a first sound effect image corresponding to the adjustment operation in response to the adjustment operation further includes:

In response to the user's adjustment operation on the volume adjustment icon, determine a volume adjustment amount corresponding to the adjustment operation;

Adjust the volume display indicator in the second sound effect image according to the volume adjustment amount, and display the adjusted second sound effect as the first sound effect image corresponding to the adjustment operation.

Optionally, the audio adjustment icon includes a sound effect mode icon, and the step of displaying a first sound effect image corresponding to the adjustment operation in response to the adjustment operation further includes:

In response to the user's adjustment operation on the sound effect mode icon, determine a target sound effect mode corresponding to the adjustment operation;

Adjust the sound effect display logo in the second sound effect image according to the sound effect parameters of the target sound effect mode, and display the adjusted second sound effect image as the first sound effect image corresponding to the adjustment operation .

Optionally, after the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, the sound effect presentation method further includes:

Acquiring the user's interactive operation information, and determining the idle time of the audio adjustment icon according to the interactive operation information;

When the idle duration reaches a preset duration threshold, the presentation of the virtual audio device and the audio adjustment icon is canceled.

Optionally, the AR device includes a display lens, and the step of displaying a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device includes:

Obtaining a target display area in a real scene, wherein the target display area is used to display the virtual audio device and the audio adjustment icon;

controlling the AR device to display the virtual audio device and the audio adjustment icon on a display area corresponding to the target display area on the display glasses.

Optionally, the step of controlling the AR device to display the virtual audio device and the audio adjustment icon on the display area corresponding to the target display area on the display lens further includes:

determining a new display area of the target display area on the display lens according to the real-time pose information of the AR device;

Moving the virtual audio device to the new display area, so that the virtual audio device is always displayed in the display area corresponding to the target display area.

Optionally, after the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, the sound effect presentation method further includes:

Detecting the user's real-time gaze direction by performing eye tracking on the user;

According to the real-time gaze direction, determine the real-time gaze area corresponding to the real-time gaze direction;

Judging whether the display area of the virtual audio device is within the real-time gaze area;

If the display area is not within the real-time gaze area, cancel the presentation of the virtual audio device and the audio adjustment icon.

In addition, in order to achieve the above purpose, the present application also provides a sound effect display device, the sound effect display device is applied to AR equipment, and the sound effect display device includes:

The first display module is used to display a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device;

A receiving module, configured to receive a user's adjustment operation on the audio adjustment icon, wherein the adjustment operation includes at least one of gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement interaction operation;

The second display module is configured to display the first sound effect image corresponding to the adjustment operation in response to the adjustment operation.

The present application also provides an electronic device. The electronic device includes: a memory, a processor, and a program of a sound effect display method stored in the memory and operable on the processor. When the program of the sound effect display method is executed by the processor, the above-mentioned The steps of the sound effect display method.

The present application also provides a computer-readable storage medium, on which a program for realizing the sound effect display method is stored. When the program of the sound effect display method is executed by a processor, the steps of the above-mentioned sound effect display method are realized.

The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for displaying sound effects are realized.

The present application provides a sound effect display method, device, and computer-readable storage medium. Compared with the prior art in which the user adjusts the sound effect through the buttons on the AR device, and then the user simply perceives the adjusted sound effect through hearing, the present application The application first displays the virtual audio device and the audio adjustment icon of the virtual audio device through the AR device; receives the user's adjustment operation on the audio adjustment icon, wherein the adjustment operation includes at least gesture interaction operation, touch interaction operation, and voice interaction One of operation and eye movement interaction operation; in response to the adjustment operation, displaying a first sound effect image corresponding to the adjustment operation. That is, the virtual audio device and the audio adjustment icon of the virtual audio device are displayed in the real scene through the AR device, so that the user can control the sound effect of the AR device as if controlling a real speaker device, effectively improving the AR device when adjusting the sound effect. Convenience. Then, in response to the user's adjustment operation, the first sound effect image corresponding to the adjustment operation is displayed. Therefore, when adjusting the sound effect of the AR device, the user can use the first sound effect image to assist the user to perceive the change of the sound effect during the adjustment process, so that this application presents the more abstract sound effect in a concrete way, which is more conducive to user perception , improving the perceptibility of the sound effect adjustment process.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, In other words, other drawings can also be obtained from these drawings without paying creative labor.

FIG. 1 is a schematic flow chart of the first embodiment of the sound effect display method of the present application;

FIG. 2 is an example diagram of a virtual audio device and an audio adjustment icon in the sound effect display method of the present application;

FIG. 3 is a schematic flow diagram of the second embodiment of the sound effect display method of the present application;

FIG. 4 is a schematic flow chart of the third embodiment of the sound effect display method of the present application;

Fig. 5 is another scene example diagram showing a virtual audio device in the sound effect display method of the present application;

FIG. 6 is a schematic diagram of a device structure of a hardware operating environment involved in a method for displaying sound effects in an embodiment of the present application.

The realization of the purpose, functions and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

In order to make the above objects, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Apparently, the described embodiments are only some of the embodiments of this application, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

Embodiment one

The embodiment of the present application provides a sound effect display method, which is applied to AR (Augmented Reality, augmented reality) equipment (such as AR glasses). In the first embodiment of the sound effect display method of the present application, referring to FIG. 1, the sound effect display method includes:

Step S10, displaying a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device;

Step S20, receiving a user's adjustment operation on the audio adjustment icon, wherein the adjustment operation includes at least one of gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement interaction operation;

Step S30, in response to the adjustment operation, displaying a first sound effect image corresponding to the adjustment operation.

In this embodiment, it should be noted that the AR device also includes a display lens, which can be used to display virtual objects, and the user can observe a real display scene through the display lens. In addition, the AR The device includes a speaker for outputting audio information.

In addition, it should be noted that the virtual audio device may be a virtual image such as a speaker, a phonograph, or a music box, and the specific image of the virtual audio device may be set according to requirements, which is not limited in this embodiment. The audio adjustment icon of the virtual audio device may be a three-dimensional icon, of course, it may also be a two-dimensional icon.

Wherein, the adjustment operation includes at least one of gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement interaction operation. For example, refer to FIG. 2. FIG. 2 is an example diagram of a virtual audio device and an audio adjustment icon in the sound effect display method of the present application. In the field of view of the AR device, a virtual audio device 10 is included, and the audio adjustment icon includes a volume increase The large icon 11 and the volume down icon 12, the user can click the volume up icon 11 to increase the output volume of the AR device through gesture interaction, and click the volume down icon 12 to decrease the output volume of the AR device.

As another example, steps S10 to S40 include: displaying a virtual audio device and an audio adjustment icon of the virtual audio device through an AR device, and then the user can perform an adjustment operation on the virtual audio device and the audio adjustment icon interact. When the AR device receives a user's adjustment operation on the audio adjustment icon, it may display a first sound effect image corresponding to the adjustment operation in response to the adjustment operation.

As another example, the step of displaying a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device includes:

When a user's trigger interaction operation is received, a virtual audio device and an audio adjustment icon of the virtual audio device are displayed at a preset display position on the display glasses of the AR device.

Wherein, the preset display position may be the center position of the field of view, the lower right corner position, the lower left corner position and other positions on the display lens of the AR device, and the preset display position may be set according to specific requirements. The triggering interaction operation may be a gesture interaction operation, a touch interaction operation, a voice interaction operation, an eye movement interaction operation, and the like. The user can trigger the AR device to display the virtual audio device and the audio adjustment icon by means of gesture, touch, voice, eye movement and the like. Exemplarily, when it is detected that the user wears the AR device, it may be determined that a user's trigger interaction operation is received.

As another example, the audio adjustment icon is a volume adjustment icon, and the step of displaying a first sound effect image corresponding to the adjustment operation in response to the adjustment operation includes:

In response to the user's adjustment operation on the volume adjustment icon, display the volume display logo of the AR device, adjust the volume display logo according to the adjustment operation, and use the adjusted volume display logo as the adjustment operation corresponding The image of the first sound effect is displayed.

Wherein, the volume adjustment icon may be a knob icon, and the user's adjustment operation is to decrease the volume when turning it to the left, and the user's adjustment operation is to increase the volume when turning it to the right. The volume adjustment icon may also include a volume up icon and a volume down icon, the user's adjustment operation is to decrease the volume when the volume down icon is clicked, and the user's adjustment operation is to increase the volume when the volume up icon is clicked. Exemplarily, the volume display indicator may be a volume progress bar, and the user's adjustment operation is to elongate the volume progress bar when the volume up icon is clicked, and the volume progress bar is elongated when the user's adjustment operation is to click the volume down icon. bar shortened.

As another example, when the audio adjustment icon is a sound effect mode icon, the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation includes:

determining a target sound effect mode in response to the user's adjustment operation on the sound effect mode icon;

The sound effect display identifier corresponding to the target sound effect mode is displayed as the first sound effect image corresponding to the adjustment operation.

It is understandable that different sound effect modes (such as transparent mode, noise reduction mode, electronic sound mode, live mode, etc.) bring different auditory experiences to users. Therefore, the sound effect characteristics of different sound effect modes ( That is, the amplitude, frequency, pitch, timbre and other characteristics of the audio in different sound effect modes), and generate corresponding sound effect display signs to represent the sound effect characteristics of different sound effect modes. For example, light blue or light green slightly fluctuating light and shadow can be used in transparent mode, and disordered lines in red and black color or metal color can be used in electronic sound mode. Therefore, the user can visually understand the sound effect characteristics of the target sound effect mode selected by the adjustment operation through the sound effect characteristic image displayed on the AR device.

Wherein, after the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation in step S30, the sound effect presentation method further includes:

Step A10, acquiring the user's interactive operation information, and determining the idle time of the audio adjustment icon according to the interactive operation information;

Step A20, when the idle duration reaches a preset duration threshold, cancel the presentation of the virtual audio device and the audio adjustment icon.

As an example, steps A10 to A20 include: determining an idle time period during which the audio adjustment icon does not interact with the user by acquiring interactive operation information of the user and analyzing it according to the interactive operation information. Then it is judged whether the idle time reaches a preset time threshold (such as 2 seconds, 3 seconds, 5 seconds, etc.). When the idle time reaches the preset time threshold, it means that the user has completed the audio adjustment operation, and then cancels the display of the virtual audio device and the audio adjustment icon, so as not to block the user's field of vision for a long time and interfere with the user's viewing of the AR device. normal use.

The first embodiment of the present application provides a sound effect display method. Compared with the prior art in which the user adjusts the sound effect through the buttons on the AR device, and then the user simply perceives the adjusted sound effect through hearing, the present application first uses the AR The device displays a virtual audio device and an audio adjustment icon of the virtual audio device; and receives a user's adjustment operation on the audio adjustment icon, wherein the adjustment operation includes at least gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement One of the interactive operations; in response to the adjustment operation, displaying a first sound effect image corresponding to the adjustment operation. That is, the virtual audio device and the audio adjustment icon of the virtual audio device are displayed in the real scene through the AR device, so that the user can control the sound effect of the AR device as if controlling a real speaker device, effectively improving the AR device when adjusting the sound effect. Convenience. Then, in response to the user's adjustment operation, the first sound effect image corresponding to the adjustment operation is displayed. Therefore, when adjusting the sound effect of the AR device, the user can use the first sound effect image to assist the user to perceive the change of the sound effect during the adjustment process, so that this application presents the more abstract sound effect in a concrete way, which is more conducive to user perception , improving the perceptibility of the sound effect adjustment process.

Embodiment two

Further, referring to FIG. 3 , based on the first embodiment of the present application, in another embodiment of the present application, for the same or similar content as the first embodiment above, reference may be made to the introduction above, and details will not be repeated hereafter. Before the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation in step S30, the steps include:

Step B10, acquiring the current audio signal of the AR device;

Step B20, generating and displaying a second sound effect image corresponding to the current audio signal according to the current audio signal.

In this embodiment, it can be understood that the sound effect image is an effect image representing audio features of the current audio signal, where the audio features may be at least one of amplitude, frequency, pitch, and timbre. Exemplarily, the second sound effect image may include a sound wave waveform corresponding to the audio feature of the current audio signal, and the sound wave waveform may be expressed in a graph, a histogram, or the like. It can be understood that the timbre and pitch in the audio feature of the current audio signal can also be expressed by the color and saturation of the sound wave waveform. In addition, the second sound effect image may be a two-dimensional image or a three-dimensional image.

As an example, steps B10 to B20 include: acquiring a current audio signal of the AR device; and then generating and displaying a second sound effect image corresponding to the current audio signal according to the current audio signal.

Wherein, the audio adjustment icon includes a volume adjustment icon, and step S30 responds to the adjustment operation, and the step of displaying the first sound effect image corresponding to the adjustment operation further includes:

Step C10, in response to the user's adjustment operation on the volume adjustment icon, determine the volume adjustment amount corresponding to the adjustment operation;

Step C20, adjust the volume display indicator in the second sound effect image according to the volume adjustment amount, and display the adjusted second sound effect as the first sound effect image corresponding to the adjustment operation.

It can be understood that when the audio adjustment icon that the user interacts with through the adjustment operation is a volume adjustment icon, it means that the sound effect that the user desires to adjust is the volume of the current audio signal. The volume display mark in the second sound effect image may be the amplitude of the sound wave in the second sound effect image.

As an example, steps C10 to C20 include: in response to the user's adjustment operation on the volume adjustment icon, determining the volume adjustment amount corresponding to the adjustment operation, where the volume adjustment amount may include volume increase and Volume reduction. Then, according to the volume adjustment amount, the volume display mark (ie, the amplitude of the sound wave waveform) in the second sound effect image is adjusted, and the adjusted second sound effect image is used as the first sound effect corresponding to the adjustment operation image for display. Thus, the user can visually and intuitively perceive the adjustment and change process of the sound effect through the first sound effect diagram.

Wherein, the audio adjustment icon includes a sound effect mode icon, and step S10 responds to the adjustment operation, and the step of displaying the first sound effect image corresponding to the adjustment operation further includes:

Step D10, in response to the user's adjustment operation on the sound effect mode icon, determine the target sound effect mode corresponding to the adjustment operation;

Step D20, adjusting the sound effect display logo in the second sound effect image according to the sound effect parameters of the target sound effect mode, and using the adjusted second sound effect image as the first sound effect corresponding to the adjustment operation image for display.

It can be understood that when the audio adjustment icon that the user interacts with through the adjustment operation is a sound effect mode icon, it means that the sound effect that the user desires to adjust is the sound effect mode of the current audio signal. The sound effect display identifier may be the amplitude and frequency of the sound wave in the second sound effect graph.

As an example, steps D10 to D20 include: in response to the user's adjustment operation on the sound effect mode icon, determining a target sound effect mode corresponding to the adjustment operation. Due to different sound effect modes, both the amplitude and the frequency of the sound wave will change. Therefore, according to the sound effect parameters of the target sound effect mode, the sound effect display logo (amplitude and frequency of the sound wave waveform) in the second sound effect image is adjusted, and the adjusted second sound effect image is used as the adjusted The first sound effect image corresponding to the operation is displayed. Thus, the user can visually and intuitively perceive the adjustment and change process of the sound effect through the first sound effect diagram.

In the second embodiment of the present application, by acquiring the current audio signal of the AR device; according to the current audio signal, a second sound effect image corresponding to the current audio signal is generated and displayed. In response to the user's adjustment operation on the volume adjustment icon, determine a volume adjustment amount corresponding to the adjustment operation; adjust the volume display logo in the second sound effect image according to the volume adjustment amount, and set The adjusted second sound effect is displayed as the first sound effect image corresponding to the adjustment operation. In response to the user's adjustment operation on the sound effect mode icon, determine the target sound effect mode corresponding to the adjustment operation; according to the sound effect parameters of the target sound effect mode, carry out the sound effect display logo in the second sound effect image Adjust, and display the adjusted second sound effect image as the first sound effect image corresponding to the adjustment operation. Therefore, in this embodiment, after the user adjusts the audio adjustment icon, he can intuitively perceive the adjustment and change of the sound effect through the displayed first sound effect image. Compared with the simple and fixed image, the first sound effect in this embodiment A sound effect image is closer to the actual change of the sound effect, which is more conducive to the user's perception of the change of the sound effect during the sound effect adjustment process.

Embodiment Three

Further, referring to FIG. 4 , in another embodiment of the present application, for content that is the same as or similar to the above embodiment, reference may be made to the introduction above, and details will not be repeated hereafter. On this basis, the step S10 shows the virtual audio device and the audio adjustment icon of the virtual audio device through the AR device, including:

Step S11, acquiring a target display area in a real scene, wherein the target display area is used to display the virtual audio device and the audio adjustment icon;

Step S12, controlling the AR device to display the virtual audio device and the audio adjustment icon on the display area corresponding to the target display area on the display lens.

As an example, steps S11 to S13 include: acquiring a target display area in a real scene, wherein the target display area is used to display the virtual audio device and the audio adjustment icon, wherein the target display area uses For displaying the virtual audio device and the audio adjustment icon, such as a free desk, bookshelf, bed, etc. By collecting real-time images of the real scene for analysis, determine the target display area where the preset audio device model can be placed in the real scene, of course, the user can also designate a certain area in the real scene as the target display area. Then, according to the relative position information between the target display area and the AR device, it is determined that the target display area displays a corresponding display area on the lens in the AR device. Then control the AR device to display the virtual audio device and the audio adjustment icon on the display area corresponding to the target display area on the display glasses. Wherein, the audio adjustment icon is used to represent the adjustable sound effect information of the AR device (that is, the sound effect mode that can be set by the AR device). You can refer to FIG. 5, which is an example diagram of another scene showing a virtual audio device in the sound effect display method of the present application. In the figure, the display area on the display lens in the AR device renders a virtual audio device 10, and the user can observe it through the AR device. to the virtual audio device 10 displayed on the target display area of the desk 20 in the real scene.

Further, the step of rendering a virtual audio device in the AR device according to the target display area and the preset audio device model in step S10 also includes:

Step E10, according to the real-time pose information of the AR device, determine a new display area of the target display area on the display lens;

Step E20, moving the virtual audio device to the new display area, so that the virtual audio device is always displayed in the display area corresponding to the target display area.

As an example, steps E10 to E20 include: acquiring real-time pose information of the AR device, wherein the real-time pose information includes the real-time position and real-time attitude of the AR device, according to the real-time pose information of the AR device posture information, can determine the posture change between the AR device and the target display area, determine the new display area of the target display area on the display lens, and then move the virtual audio device to the new display area, so that the virtual audio device is always displayed in the display area corresponding to the target display area. In this embodiment, the display area of the virtual audio device in the AR device is updated in real time according to the real-time pose information of the AR device, so that the virtual audio device is placed closer to the target The real device on the display area avoids the virtual audio device always floating in a fixed position in the field of view of the AR device, causing interference to the user, and also improves the authenticity of the user's interaction with the virtual audio device through adjustment operations.

Further, after the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation in step S30, the sound effect presentation method further includes:

Step F10, by tracking the user's eyeballs, detecting the real-time gaze direction of the user;

Step F20, according to the real-time gaze direction, determine the real-time gaze area corresponding to the real-time gaze direction;

Step F30, judging whether the display area of the virtual audio device is within the real-time gaze area;

Step F40, if the display area is not within the real-time gaze area, cancel the presentation of the virtual audio device and the audio adjustment icon.

As an example, steps F10 to F40 include: detecting the real-time gaze direction of the user by tracking the user's eyeballs, so that the real-time gaze direction can be determined and displayed on the lens in the AR device according to the real-time gaze direction The corresponding real-time gaze area. Because the display area of the virtual audio device in the AR device moves according to the real-time pose information of the AR device. Therefore, it can be determined whether the display area of the virtual audio device is within the real-time gaze area, and whether the user still pays attention to the virtual audio device can be determined. If the display area is not within the real-time gaze area, it means that the user no longer pays attention to the virtual audio device, and cancels the presentation of the virtual audio device and the audio adjustment icon. If the display area is within the real-time gaze area, it means that the user is still paying attention to the virtual audio device, and then keep displaying the virtual audio device and the audio adjustment icon. In this embodiment, by judging whether the display area of the virtual audio device is within the real-time gaze area, it is determined whether the user still pays attention to the virtual audio device, thereby determining whether to continue to maintain the virtual audio device and the audio adjustment Display of icons. Under the premise of preventing the user from being disturbed by the virtual audio device and the audio adjustment icon, it is also ensured that when the user no longer pays attention to the virtual audio device and the audio adjustment icon, the display is canceled to avoid occupying the AR computing resources of the device.

Embodiment Four

The embodiment of the present application also provides a sound effect display device, the sound effect display device is applied to sound effect display equipment, and the sound effect display device includes:

The first display module is used to display a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device;

A receiving module, configured to receive a user's adjustment operation on the audio adjustment icon, wherein the adjustment operation includes at least one of gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement interaction operation;

The second display module is configured to display the first sound effect image corresponding to the adjustment operation in response to the adjustment operation.

Optionally, the second display module is also used for:

Obtain the current audio signal of the AR device;

Generate and display a second sound effect image corresponding to the current audio signal according to the current audio signal.

Optionally, the audio adjustment icon includes a volume adjustment icon, and the second display module is also used for:

In response to the user's adjustment operation on the volume adjustment icon, determine a volume adjustment amount corresponding to the adjustment operation;

Adjust the volume display indicator in the second sound effect image according to the volume adjustment amount, and display the adjusted second sound effect as the first sound effect image corresponding to the adjustment operation.

Optionally, the audio adjustment icon includes a sound effect mode icon, and the second display module is also used for:

In response to the user's adjustment operation on the sound effect mode icon, determine a target sound effect mode corresponding to the adjustment operation;

Adjust the sound effect display logo in the second sound effect image according to the sound effect parameters of the target sound effect mode, and display the adjusted second sound effect image as the first sound effect image corresponding to the adjustment operation .

Optionally, the sound effect display device also includes a cancel display module, which is used for:

Acquiring the user's interactive operation information, and determining the idle time of the audio adjustment icon according to the interactive operation information;

When the idle duration reaches a preset duration threshold, the presentation of the virtual audio device and the audio adjustment icon is canceled.

Optionally, the first display module is also used to:

Obtaining a target display area in a real scene, wherein the target display area is used to display the virtual audio device and the audio adjustment icon;

controlling the AR device to display the virtual audio device and the audio adjustment icon on a display area corresponding to the target display area on the display glasses.

Optionally, the first display module is also used to:

determining a new display area of the target display area on the display lens according to the real-time pose information of the AR device;

Moving the virtual audio device to the new display area, so that the virtual audio device is always displayed in the display area corresponding to the target display area.

Furthermore, the undisplay module is also used to:

Detecting the user's real-time gaze direction by performing eye tracking on the user;

According to the real-time gaze direction, determine the real-time gaze area corresponding to the real-time gaze direction;

Judging whether the display area of the virtual audio device is within the real-time gaze area;

If the display area is not within the real-time gaze area, cancel the presentation of the virtual audio device and the audio adjustment icon.

The sound effect display device provided by the present application adopts the sound effect display method in the above-mentioned embodiment 1 or embodiment 2, and solves the technical problem in the prior art that the sound effect of the AR device is relatively abstract, resulting in poor perceptibility of the sound effect adjustment process. Compared with the prior art, the beneficial effect of the sound effect display device provided by the embodiment of the present application is the same as that of the sound effect display method provided by the above embodiment, and other technical features in the sound effect display device are the same as those disclosed by the method of the above embodiment The features are the same, and will not be repeated here.

Embodiment five

An embodiment of the present application provides an electronic device, and the electronic device includes: at least one processor; and a memory connected in communication with the at least one processor; wherein, the memory stores instructions executable by at least one processor, and the instructions are executed by at least one processor. The processor executes, so that at least one processor can execute the sound effect display method in the first embodiment above.

Referring now to FIG. 6 , it shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure. The electronic equipment in the embodiment of the present disclosure may include but not limited to such as mobile phone, notebook computer, digital broadcast receiver, PDA (personal digital assistant), PAD (tablet computer), PMP (portable multimedia player), vehicle terminal (such as mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers and the like. The electronic device shown in FIG. 6 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.

As shown in FIG. 6, an electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which may be loaded into a random access memory (RAM) according to a program stored in a read-only memory (ROM) or loaded from a storage device. Various appropriate actions and processing are performed by the programs in the program. In RAM, various programs and data necessary for the operation of electronic equipment are also stored. The processing means, ROM, and RAM are connected to each other via a bus. Input/output (I/O) interfaces are also connected to the bus.

Typically, the following systems can be connected to the I/O interface: input devices including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; including, for example, liquid crystal displays (LCDs), speakers, vibrators output devices such as; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. A communication device may allow an electronic device to communicate with other devices wirelessly or by wire to exchange data. While an electronic device is shown with various systems in the figures, it should be understood that implementing or having all of the systems shown is not a requirement. More or fewer systems may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means, or installed from a storage means, or installed from a ROM. When the computer program is executed by the processing device, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

The electronic device provided by the present application adopts the sound effect display method in Embodiment 1 or Embodiment 2 above, which solves the technical problem in the prior art that the sound effect of the AR device is relatively abstract, resulting in poor perceptibility of the sound effect adjustment process. Compared with the prior art, the beneficial effect of the electronic device provided by the embodiment of the present application is the same as that of the sound effect display method provided by the first embodiment above, and other technical features of the electronic device are the same as those disclosed by the method of the above embodiment The same, will not be repeated here.

It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in an appropriate manner.

The above is only the specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the application, and should be covered Within the protection scope of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Embodiment six

This embodiment provides a computer-readable storage medium, which has computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the audio adjustment method in the first embodiment above.

The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, system or device. Program code embodied on a computer readable storage medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable storage medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable storage medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: displays the virtual audio device and the audio adjustment icon of the virtual audio device through the AR device; receives An adjustment operation performed by the user on the audio adjustment icon, wherein the adjustment operation includes at least one of gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement interaction operation; in response to the adjustment operation, displaying the The first sound effect image corresponding to the adjustment operation.

Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the module does not constitute a limitation of the unit itself under certain circumstances.

The computer-readable storage medium provided by the present application stores computer-readable program instructions for executing the above-mentioned sound effect display method, which solves the technology in the prior art that the sound effect of the AR device is relatively abstract, resulting in poor perceptibility of the sound effect adjustment process. question. Compared with the prior art, the beneficial effect of the computer-readable storage medium provided by the embodiment of the present application is the same as the beneficial effect of the sound effect display method provided by the first or second embodiment above, and will not be repeated here.

Embodiment seven

The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for displaying sound effects are realized.

The computer program product provided by the present application solves the technical problem in the prior art that the sound effect of the AR device is relatively abstract, resulting in poor perceptibility of the sound effect adjustment process. Compared with the prior art, the beneficial effect of the computer program product provided by the embodiment of the present application is the same as the beneficial effect of the sound effect display method provided by the above-mentioned embodiment 1 or embodiment 2, and will not be repeated here.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structures or equivalent process transformations made by using the description of the application and the accompanying drawings are directly or indirectly used in other related technical fields. , are all included in the patent processing scope of the present application in the same way.

Claims (10)

1. A sound effect display method, characterized in that it is applied to an AR device, and the sound effect display method comprises: Displaying a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device; receiving a user's adjustment operation on the audio adjustment icon, wherein the adjustment operation includes at least one of gesture interaction operation, touch interaction operation, voice interaction operation, and eye movement interaction operation; In response to the adjustment operation, a first sound effect image corresponding to the adjustment operation is displayed. 2. The sound effect display method according to claim 1, wherein, before the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, the steps include: Obtain the current audio signal of the AR device; Generate and display a second sound effect image corresponding to the current audio signal according to the current audio signal. 3. The sound effect display method according to claim 2, wherein the audio adjustment icon comprises a volume adjustment icon, and the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, Also includes: In response to the user's adjustment operation on the volume adjustment icon, determine a volume adjustment amount corresponding to the adjustment operation; Adjust the volume display indicator in the second sound effect image according to the volume adjustment amount, and display the adjusted second sound effect as the first sound effect image corresponding to the adjustment operation. 4. The sound effect display method according to claim 2, wherein the audio adjustment icon comprises a sound effect mode icon, and the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, Also includes: In response to the user's adjustment operation on the sound effect mode icon, determine a target sound effect mode corresponding to the adjustment operation; Adjust the sound effect display logo in the second sound effect image according to the sound effect parameters of the target sound effect mode, and display the adjusted second sound effect image as the first sound effect image corresponding to the adjustment operation . 5. The sound effect display method according to claim 1, wherein after the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, the sound effect display method further comprises: Acquiring the user's interactive operation information, and determining the idle time of the audio adjustment icon according to the interactive operation information; When the idle duration reaches a preset duration threshold, the presentation of the virtual audio device and the audio adjustment icon is canceled. 6. The sound effect display method according to claim 1, wherein the AR device includes a display lens, and the step of displaying a virtual audio device and an audio adjustment icon of the virtual audio device through the AR device includes: Obtaining a target display area in a real scene, wherein the target display area is used to display the virtual audio device and the audio adjustment icon; controlling the AR device to display the virtual audio device and the audio adjustment icon on a display area corresponding to the target display area on the display glasses. 7. The sound effect display method according to claim 6, wherein the controlling the AR device displays the virtual audio device and the audio adjustment icon on the display area corresponding to the target display area on the display lens The steps also include: determining a new display area of the target display area on the display lens according to the real-time pose information of the AR device; Moving the virtual audio device to the new display area, so that the virtual audio device is always displayed in the display area corresponding to the target display area. 8. The sound effect display method as claimed in claim 6, wherein after the step of displaying the first sound effect image corresponding to the adjustment operation in response to the adjustment operation, the sound effect display method further comprises : Detecting the user's real-time gaze direction by performing eye tracking on the user; According to the real-time gaze direction, determine the real-time gaze area corresponding to the real-time gaze direction; Judging whether the display area of the virtual audio device is within the real-time gaze area; If the display area is not within the real-time gaze area, cancel the presentation of the virtual audio device and the audio adjustment icon. 9. An electronic device, characterized in that the electronic device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can perform any one of claims 1 to 8 The steps of the sound effect display method. 10. A computer-readable storage medium, characterized in that a program for implementing a sound effect display method is stored on the computer-readable storage medium, and the program for implementing the sound effect display method is executed by a processor to implement the following claims 1 to 10. Steps of the sound effect display method described in any one of 8.

Images