CN118646925A - Terminal device and sound processing method - Google Patents

Abstract

Some embodiments of the present application illustrate a terminal device and a sound processing method, the method comprising: after receiving a user input instruction to switch sound effect parameters, obtaining the direct sound of the original sound signal; processing the direct sound through a reverberation echo algorithm to obtain a first effect sound; performing gain processing on the first effect sound to obtain a second effect sound, and switching sound effect parameters; mixing the direct sound, the second effect sound, and the music accompaniment sound to obtain an output sound; and controlling an audio output device to play the output sound. The embodiments of the present application only perform gain processing on the effect sound when switching the sound effect parameters, and the direct sound is still played normally. While reducing the noise generated by adjusting the sound effect parameters, it does not cause the user to feel uncomfortable due to sudden silence.

Description

Terminal device and sound processing method

Technical Field

The present application relates to the technical field of audio signal processing, and in particular to a terminal device and a sound processing method.

Background Art

With the improvement of living standards, TVs have become a common electronic consumer product. In the home, karaoke, various games or live broadcasts can effectively improve the entertainment and usage frequency of TVs. Because everyone's voice timbre is different and their requirements for sound effects are different, it is difficult for engineers to adjust the sound effects to suit everyone. Many karaoke products provide professional sound effect parameters for professional users, allowing users to adjust them themselves.

However, when the sound effect parameters are adjusted, the output audio signal will be discontinuous, causing POP noise. The general solution is to place a gain module at the output end of the audio, that is, before output to the power amplifier, for fade-in and fade-out processing. When the TV sound effect processing module receives an instruction to adjust the sound effect parameters, for example, adjust the reverberation time to 6. The sound effect processing module slowly sets the coefficient of the gain module to a linear value of 0. Then execute the switch of the sound effect parameters. After the sound effect parameters are set, the sound effect processing module slowly sets the coefficient of the gain module to a linear value of 1. Therefore, there will be a period of silence when switching the sound effect parameters, which will make the user feel uncomfortable.

Another method to solve the problem of adjusting sound effect parameters is to detect whether the microphone signal is close to silence, and switch the sound effect parameters if it is a silent segment. Since the microphone signal is close to silence and the amplitude is about 0, although noise will be generated, the noise generated is very small and cannot be heard by the human ear. However, this method has the following two disadvantages: (1) If the user is still using the microphone after entering the instruction to switch the sound effect parameters, the sound effect parameter switching operation cannot be performed. That is, the user adjusts the sound effect parameters on the terminal device, but because silence is not detected, the sound effect parameters have not been switched. This may cause the user to feel that the adjusted sound effect parameters are not enough and continue to adjust the sound effect parameters. (2) After the silent segment is detected, the sound effect parameters are switched, but during the switching process, the microphone picks up the sound of the user singing and cannot stop the switching of the sound effect parameters. The amplitude of the user's voice signal is large, resulting in a large noise.

Summary of the invention

Some embodiments of the present application provide a terminal device and a sound processing method, which only performs gain processing on the effect sound when adjusting the sound effect parameters, while the direct sound is played normally. While reducing the noise generated by adjusting the sound effect parameters, it will not cause the user to feel uncomfortable due to sudden silence.

In a first aspect, some embodiments of the present application provide a terminal device, including:

The sound collector is configured to: collect the original sound signal input by the user;

The audio output device is configured to: play output sound;

The controller is configured as:

After receiving the sound effect parameter switching instruction input by the user, obtaining the direct sound of the original sound signal;

Processing the direct sound with a reverberation echo algorithm to obtain a first effect sound;

Performing gain processing on the first effect sound to obtain a second effect sound, and switching sound effect parameters;

Mixing the direct sound, the second effect sound and the music accompaniment sound to obtain output sound;

Control the audio output device to play the output sound.

In some embodiments, the controller performs acquisition of the direct sound of the original sound signal and is further configured to:

Receiving the original sound signal collected by the sound collector;

The original sound signal is pre-processed to obtain direct sound, wherein the pre-processing includes noise gate processing, expander processing, gain control, howling suppression and/or pop-out prevention processing.

In some embodiments, the controller performs gain processing on the first effect sound to obtain a second effect sound, and switches sound effect parameters, and is further configured as follows:

multiplying the first effect sound by a gain coefficient to obtain a second effect sound, wherein the gain coefficient is used to characterize the gain of the first effect sound;

After the gain coefficient is reduced from a first preset value to a second preset value, switching the sound effect parameter, the first preset value being greater than the second preset value;

After the sound effect parameter is switched, the gain coefficient is increased from the second preset value to the first preset value.

In some embodiments, after the controller reduces the gain coefficient from the first preset value to the second preset value, the sound effect parameter is switched, and is further configured as follows:

After reducing the gain coefficient from a first preset value to a second preset value at a first preset rate, switching the sound effect parameter;

The controller increases the gain coefficient from the second preset value to the first preset value, and is further configured to:

The gain coefficient is increased from the second preset value to the first preset value at a second preset rate.

In some embodiments, the controller performs mixing processing on the direct sound, the second effect sound and the music accompaniment sound to obtain output sound, and is further configured as follows:

Mixing the direct sound, the second effect sound and the music accompaniment sound to obtain a mixed sound;

The mixed sound is subjected to target sound processing to obtain an output sound, wherein the target sound processing includes frequency division processing, equalizer processing and/or dynamic gain control.

In a second aspect, some embodiments of the present application provide a sound processing method, comprising:

After receiving the sound effect parameter switching instruction input by the user, obtaining the direct sound of the original sound signal, wherein the original sound signal is collected by the sound collector;

After receiving the sound effect parameter switching instruction input by the user, obtaining the direct sound of the original sound signal, wherein the original sound signal is collected by the sound collector;

Processing the direct sound with a reverberation echo algorithm to obtain a first effect sound;

Performing gain processing on the first effect sound to obtain a second effect sound, and switching sound effect parameters;

Mixing the direct sound, the second effect sound and the music accompaniment sound to obtain output sound;

Control the audio output device to play the output sound.

In some embodiments, the step of obtaining the direct sound of the original sound signal includes:

Receiving the original sound signal collected by the sound collector;

The original sound signal is pre-processed to obtain direct sound, wherein the pre-processing includes noise gate processing, expander processing, gain control, howling suppression and/or pop-out prevention processing.

In some embodiments, the step of performing gain processing on the first effect sound to obtain a second effect sound and switching sound effect parameters includes:

Determine the second effect sound as the product of the first effect sound and a gain coefficient, wherein the gain coefficient is used to characterize the gain of the first effect sound;

After reducing the gain coefficient from a first preset value to a second preset value, switching the sound effect parameter, the first preset value being greater than the second preset value;

After the sound effect parameter is switched, the gain coefficient is increased from the second preset value to the first preset value.

In some embodiments, the step of switching the sound effect parameters after reducing the gain coefficient from the first preset value to the second preset value includes:

After reducing the gain coefficient from a first preset value to a second preset value at a first preset rate, switching the sound effect parameter;

The step of increasing the gain coefficient from the second preset value to the first preset value comprises:

The gain coefficient is increased from the second preset value to the first preset value at a second preset rate.

In some embodiments, the step of mixing the direct sound, the second effect sound and the music accompaniment sound to obtain the output sound includes:

Mixing the direct sound, the second effect sound and the music accompaniment sound to obtain a mixed sound;

The mixed sound is subjected to target sound processing to obtain an output sound, wherein the target sound processing includes frequency division processing, equalizer processing and/or dynamic gain control.

Some embodiments of the present application provide a terminal device and a sound processing method. After receiving the instruction to switch sound effect parameters input by the user, the direct sound of the original sound signal is obtained; the direct sound is processed by the reverberation echo algorithm to obtain the first effect sound; the first effect sound is subjected to gain processing to obtain the second effect sound, and the sound effect parameters are switched; the direct sound, the second effect sound and the music accompaniment sound are mixed to obtain the output sound, and the output sound is played by the audio output device. When the embodiment of the present application receives the instruction to switch sound effect parameters input by the user, it is not necessary to wait for the microphone sound to be detected as silent before switching the sound effect parameters. Instead, the sound effect parameters are switched during the process of gain processing of the effect sound after being processed by the reverberation echo algorithm, thereby effectively ensuring the real-time nature of the switching parameters. By only performing gain processing on the effect sound when switching the sound effect parameters, the direct sound is still played normally, and while reducing the noise generated by adjusting the sound effect parameters, the user will not feel uncomfortable due to sudden silence.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

FIG2 is a schematic diagram of a hardware configuration of a display device provided in some embodiments of the present application;

FIG3 is a schematic diagram of software configuration of a display device provided in some embodiments of the present application;

FIG4 is a flow chart of a sound processing method provided by some embodiments of the present application;

FIG5 is a schematic diagram of switching audio parameters in a silent segment provided by some embodiments of the present application;

FIG6 is a flow chart of another sound processing method provided in some embodiments of the present application;

FIG7 is a schematic diagram of a singing homepage provided by some embodiments of the present application;

FIG8 is a schematic diagram of an audio and video playback page provided in some embodiments of the present application;

FIG9 is a schematic diagram of an audio setting page provided in some embodiments of the present application;

FIG10 is a schematic diagram of a sound effect parameter setting page provided in some embodiments of the present application;

FIG11 is a flow chart of a method for processing sound before receiving an instruction to switch sound effect parameters provided by some embodiments of the present application;

FIG. 12 is a flowchart of a method for processing sound after receiving an instruction to switch sound effect parameters, provided in some embodiments of the present application.

DETAILED DESCRIPTION

The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and/or software code that is capable of performing the functions associated with that element.

In the embodiments of the present application, terminal equipment generally refers to electronic equipment with communication processing control functions. Terminal equipment includes product forms including sound collector hardware, as well as storage media for debugging audio parameter functions through an interface in the form of software or storage media for debugging audio parameter functions in the form of key lights. Terminal equipment includes but is not limited to single product forms or combined product forms such as handheld microphones, display devices, headphones, and speakers.

The display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

FIG1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. As shown in FIG1, a user can operate a display device 200 through touch operation, a mobile terminal 300, and a control device 100. The control device 100 is used to receive an operation instruction input by a user and convert the operation instruction into a control instruction that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote control, a stylus, a handle, etc.

The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

1, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function that provides a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

FIG. 2 is a hardware configuration block diagram of the display device 200 in FIG. 1 provided in some embodiments of the present application.

In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input/output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

In some embodiments, the user input interface 280 may be used to receive instructions from a user.

In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for a display device.

The operating system can be divided into different modules or levels according to the functions implemented. For example, as shown in Figure 3, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Applications can access system resources and obtain system services during execution through the API interface.

As shown in FIG3 , the application framework layer in the embodiment of the present application includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C/C++ instruction library, to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, as shown in FIG3 , a hardware driver can be configured in the kernel layer, and the driver included in the kernel layer can be at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a WIFI driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver, etc.

It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

With the improvement of living standards, TVs have become a common electronic consumer product. In the home, karaoke, various games or live broadcasts can effectively improve the entertainment and usage frequency of TVs. Because everyone's voice timbre is different and their requirements for sound effects are different, it is difficult for engineers to adjust the sound effects to suit everyone. Many karaoke products provide professional sound effect parameters for professional users, allowing users to adjust them themselves.

However, when the sound effect parameters are adjusted, the output audio signal will be discontinuous, causing POP noise.

In some embodiments, a gain module is placed at the output end of the audio, that is, before output to the power amplifier, for fade-in and fade-out processing. As shown in Figure 4, when the sound effect processing module of the terminal device receives an instruction to switch the sound effect parameters, for example, the reverberation time is adjusted to 6. The sound effect processing module obtains the direct sound after pre-processing the original sound signal collected by the sound collector. The direct sound is processed by the reverberation and echo algorithm to obtain the effect sound. The direct sound, the effect sound and the music accompaniment sound are mixed to obtain the mixed sound. The mixed sound is gain-processed by the gain module, and after the gain processing, it is played by the audio output device through the power amplifier. After slowly setting the coefficient of the gain module to a linear value of 0, the operation of switching the sound effect parameters is executed. After the sound effect parameters are switched, the coefficient of the gain module is slowly set to a linear value of 1. Therefore, there will be a period of silence when switching the sound effect parameters, which will make the user feel uncomfortable.

In some embodiments, when the sound effect processing module of the terminal device receives an instruction to switch the sound effect parameters, it detects whether the amplitude of the microphone signal is less than a preset value, that is, it determines whether the microphone signal is in a silent state or a near silent state; if the microphone signal is in a silent state or a near silent state, the sound effect parameters are switched, as shown in FIG5 . If the microphone signal is not in a silent state or a near silent state, the operation of switching the sound effect parameters is not performed. Since the signal of the microphone is close to silent, the amplitude is about 0. Therefore, although noise will be generated when the sound effect parameters are switched at this time, the noise generated is very small and cannot be heard by the human ear.

However, this method has the following two disadvantages: (1) If the user is still using the microphone after entering the instruction to switch the sound effect parameters, the sound effect parameter switching operation cannot be performed. That is, the user adjusts the sound effect parameters on the terminal device, but because no silence is detected, the sound effect parameters are not switched. This may cause the user to feel that the adjusted sound effect parameters are not enough and continue to adjust the sound effect parameters. (2) After the silence segment is detected, the sound effect parameters are switched, but during the switching process, the microphone picks up the sound of the user singing and cannot stop the switching of the sound effect parameters. The amplitude of the user's voice signal is large, resulting in a large noise.

In order to solve the above technical problems, an embodiment of the present application provides a terminal device, which includes a sound collector, an audio output device and a controller. As shown in FIG6 , the controller runs an application program to enable the terminal device to perform the following steps:

Step S601: receiving a sound effect parameter switching instruction input by a user;

In some embodiments, an instruction to switch sound effect parameters is received from a user by pressing a button on a terminal device.

Exemplarily, the terminal device is installed with a sound effect parameter switching button, an up button and a down button. The user can switch to different sound effect parameters by pressing the sound effect parameter switching button, and adjust the specific value corresponding to the sound effect parameter by pressing the up button or the down button, thereby inputting the instruction to switch the sound effect parameter to the terminal device.

In some embodiments, an instruction to switch sound effect parameters is received from a user through interaction with a user interface.

Taking the display device 200 as an example, the step of receiving the instruction of switching sound effect parameters input by the user includes:

In response to a user inputting an instruction to open an audio setting page, controlling the display 260 to display the audio setting page, wherein the audio setting page includes a sound effect mode setting control;

In some embodiments, the display 260 displays a singing (Karaoke) application homepage, which includes at least one song control. After receiving a user input command to select a song control, the audio and video corresponding to the song control are played. The sound collector can collect the user's singing sound and play it after being processed by the sound effect module. The user can switch the current page to the audio setting page by inputting a command to open the audio setting page.

In some embodiments, the display 260 displays the user interface of the game application. After the user turns on the sound collector, the user sound collected by the sound collector can be processed by the sound effect module and sent to the server, and the server sends the processed sound to the terminal device of the designated user for playback. The user can switch the current page to the audio settings page by inputting an instruction to open the audio settings page.

In some embodiments, the display 260 displays the user interface of the video live broadcast or audio/video call application. After receiving the user input to start the video live broadcast or audio/video call, the user sound collected by the sound collector can be processed by the sound effect module and sent to the server, and the server sends the processed sound to the terminal device of the specified user for playback. The user can switch the current page to the audio settings page by inputting the command to open the audio settings page.

In some embodiments, in response to a user inputting an instruction to select the sound effect mode setting control, the display 260 is controlled to display a sound effect mode setting page, wherein the sound effect mode setting page includes at least one sound effect mode control;

In some embodiments, the sound effect mode refers to the sound effect parameters in a specific scene. Exemplarily, the sound effect modes include KTV mode, concert mode, studio mode, and theater mode. The sound effect parameters include dry sound volume, reverberation gain, reverberation time, and echo gain, and the values of the sound effect parameters are in the range of 0-10.

A mapping table of sound effect modes and sound effect parameters is stored in the local memory, and the mapping table represents the corresponding relationship between the sound effect modes and the sound effect parameters. After receiving a sound effect mode control selected by the user, the sound effect parameters can be changed to the sound effect parameters corresponding to the sound effect mode.

Exemplarily, a mapping table of sound effect modes and sound effect parameters is shown in Table 1.

Table 1

Sound Effect Parameter Controls Dry Volume Reverb Gain Reverberation time Echo Gain … KTV 5 1 2 2 … Concert 6 2 3 2 … Recording Studio 7 3 4 2 … … … … … … …

Receive an instruction from a user to select the sound effect mode control.

In some embodiments, the sound effect mode also includes a custom mode, i.e., a professional sound effect, which requires the user to adjust various sound effect parameters by himself.

In response to the user input of an instruction to select the professional sound effect control, the control display 260 displays a sound effect parameter setting page, the sound effect parameter setting page includes at least one sound effect parameter control, the sound effect parameter control includes a sound effect parameter adjustment bar, the sound effect parameter adjustment bar is provided with a slider, the position of the slider in the sound effect parameter adjustment bar represents the numerical value corresponding to the sound effect parameter.

An instruction for adjusting the position of the slider on the sound effect parameter adjustment bar is received from a user.

Exemplarily, after receiving an instruction from a user to open a singing application, or after selecting a singing control in the homepage navigation bar, the singing homepage page is displayed, as shown in FIG7 . The karaoke page includes a plurality of song controls 71. After receiving an instruction from a user to select a song control 71, the audio and video of the song is played, as shown in FIG8 . The user is received to input an instruction to open the audio settings page by pressing the lower key of the control device 100, and the audio settings page is displayed, as shown in FIG9 . The audio settings page includes a console control 91, a sound effect mode setting control 92, a song request station control 93, a clarity control 94, an automatic recording control 95, and a focus 96. The current focus 96 indicates that the sound effect mode setting control 92 is selected. When the sound effect mode setting control 92 is selected, the sound effect mode setting box is displayed. The sound effect mode setting box includes a KTV control 921, a concert control 922, a recording studio control 923, a theater control 924, and a professional sound effect control 925.

By receiving user input and selecting the concert control 922, an instruction to change the sound effect parameters to the sound effect parameters corresponding to the concert mode is input to the display device 200.

After receiving the user input to select the professional sound effect control 925, the sound effect parameter setting page is displayed, as shown in Figure 10. The sound effect parameter setting page includes a dry sound volume control 1011, a reverberation gain control 1012, a reverberation time control 1013, an echo gain control 1014, an echo interval control 1015, an echo duration control 1016, a bass control 1017, a midrange control 1018, a treble control 1019, a save control 102, a reset control 103 and a focus 96. The reset control 103 is used to restore the sound effect parameters to the default values. After the user selects the dry sound volume control 1011, the position of the slider in the sound effect parameter adjustment bar is adjusted by pressing the up and down keys of the control device 100 to modify the value corresponding to the dry sound volume. The switching sound effect parameter instruction is input to the display device 200 by receiving the user input to select the save control 102 instruction.

Step S602: obtaining the direct sound of the original sound signal;

As shown in Figure 11, when the user input switching sound effect parameter instruction is not received, the sound collector sends the collected user sound, that is, the original sound signal, to the pre-processing module. The pre-processing module pre-processes the original sound signal to obtain the direct sound, and sends the direct sound to the reverberation echo module and the mixing module respectively. The reverberation echo module processes the direct sound with a reverberation/echo algorithm to obtain the effect sound, and sends the effect sound to the mixing module. The mixing module mixes the direct sound, the effect sound and the music accompaniment sound to obtain the output sound. The output sound is played by the audio output device after passing through the power amplifier. Or the effect sound passes through the gain module before being sent to the mixing module, but the gain coefficient of the gain module is 1 and remains unchanged.

After receiving the sound effect parameter switching instruction input by the user, receiving the original sound signal collected by the sound collector, that is, the voice signal input by the user;

The original sound signal is pre-processed to obtain direct sound, wherein the pre-processing includes noise gate processing, expander processing, gain control, howling suppression and/or pop-out prevention processing.

A noise gate is an audio processor used to control and suppress the noise in the input signal, that is, the original sound signal. It is a dynamic volume controller that can automatically open or close the channel of the audio signal. When the volume of the input signal is greater than the set threshold, the noise gate will open, allowing the signal to pass through and maintaining the original volume in the output. When the volume of the input signal is lower than the threshold, the noise gate will close, shielding or weakening the signal to a set level to suppress the spread of noise.

The expander amplifies the original sound signal. While the original sound signal is amplified, it performs some appropriate processing and finally changes the line to transmit the original sound signal.

Gain control, or Automatic Gain Control (AGC), is an automatic control method that automatically adjusts the gain of the amplifier circuit according to the signal strength. Automatic gain control is a type of limited output that uses an effective combination of linear amplification and compression amplification to adjust the original sound signal. When a weak signal is input, the linear amplifier circuit works to ensure the strength of the output signal; when the original sound signal reaches a certain strength, the compression amplifier circuit is activated to reduce the output amplitude. In other words, the AGC function can automatically control the gain amplitude by changing the input-output compression ratio.

Howling suppression algorithms include frequency shifting, notch filter and adaptive filter. The frequency shifting method increases or decreases the frequency of the input audio signal. The output signal with the changed frequency will not be superimposed with the original signal frequency when it enters the system again, thus achieving the effect of suppressing howling. The notch filter method destroys the gain condition for howling by reducing the gain at the howling frequency point. The adaptive filter method filters out the feedback signal collected by the microphone from the microphone signal by using adaptive filters such as the least mean square algorithm and the normalized least mean square algorithm.

In order to prevent the mouth from being too close to the microphone when singing or recording, causing the microphone to make a popping sound due to the air sprayed out of the mouth, affecting the sound effect, the level can be adjusted to prevent the original sound signal from being overloaded. The essence of anti-pop microphone processing is also a noise elimination method.

The sound collector sends the collected original sound signal to the pre-processing module, and the pre-processing module pre-processes the original sound signal to generate direct sound.

Step S603: performing reverberation echo algorithm processing on the direct sound to obtain a first effect sound;

After the direct sound is obtained, the direct sound is divided into two paths, one of which is processed by the reverberation echo algorithm and then mixed, and the other is directly mixed.

Reverberation echo algorithms are a series of mathematical formulas and algorithms used to simulate or generate reverberation effects. Reverberation refers to the phenomenon that sound persists in space and gradually decays due to reflections caused by obstacles during the propagation process. In the fields of audio processing and acoustics, reverberation algorithms are used to simulate reverberation effects in natural environments, or to adjust reverberation in recordings or musical works. Reverberation algorithms are usually based on acoustic principles, including the reflection, absorption, and attenuation of sound. A commonly used method for calculating reverberation time is the Sabine formula, which calculates the reverberation time based on the volume and sound absorption of the room. Another method is the Airun formula, which also calculates the reverberation time based on the sound energy attenuation ratio.

The sound effect module includes a reverberation echo module and a mixing module. Upon receiving the switching sound effect parameter instruction input by the user, the application module sends the switched sound effect parameters to the reverberation echo module. The reverberation echo module does not immediately perform the sound effect parameter switching operation after receiving the switched sound effect parameters.

The pre-processing module sends the original sound signal after pre-processing, that is, the direct sound, to the reverberation echo module and the mixing module respectively. Although the reverberation echo module receives the sound effect parameters after switching, it still performs reverberation echo processing on the direct sound with the sound effect parameters before switching to obtain the first effect sound.

Step S604: performing gain processing on the first effect sound to obtain a second effect sound, and switching sound effect parameters;

The purpose of gain processing is to change the volume of the first effect sound through the gain coefficient, so that when the output volume of the first effect sound is low, the operation of switching the sound effect parameters is executed, thereby achieving a fade-in and fade-out effect.

multiplying the first effect sound by a gain coefficient to obtain a second effect sound, wherein the gain coefficient is used to characterize the gain of the first effect sound;

After the gain coefficient is reduced from a first preset value to a second preset value, switching the sound effect parameter, the first preset value being greater than the second preset value;

After the sound effect parameter is switched, the gain coefficient is increased from the second preset value to the first preset value.

Exemplarily, the first preset value is 1, and the second preset value is 0.

c(n)=Coef(n)*b(n);

Wherein, c(n) is the n-point data in the second effect sound; b(n) is the n-point data in the first effect sound, and the first effect sound is the effect sound of the direct sound after reverberation and echo algorithms. Coef(n) is the coefficient of the gain module.

In some embodiments, the step of reducing the gain coefficient from the first preset value to the second preset value includes:

The gain coefficient is reduced from a first preset value to a second preset value by a first preset amplitude.

Exemplarily, the gain coefficient Coef(n)=Coef(n-1)-A1, wherein A1 represents the amplitude of the gain coefficient reduction, which is a constant, and Coef(n-1) represents the previous value of the gain coefficient Coef(n). The initial value of Coef(n-1) is the first preset value.

Determine the current gain coefficient according to the previous gain coefficient and the first preset amplitude, and the initial value of the gain coefficient is the first preset value;

Determine whether the current gain coefficient is less than or equal to a second preset value;

If the current gain coefficient is greater than the second preset value, the first effect sound is multiplied by the current gain coefficient to obtain the second effect sound, and the step of determining the current gain coefficient according to the previous gain coefficient and the first preset amplitude is continued.

If the current gain coefficient is less than or equal to the second preset value, the first effect sound is multiplied by the second preset value to obtain the second effect sound, and the sound effect parameters are switched.

During the process of switching the sound effect parameters, the first effect sound is continuously multiplied by the second preset value to obtain the second effect sound.

Switching the sound effect parameters includes: replacing the sound effect parameters with the sound effect parameters parsed from the switching sound effect parameter instruction, that is, performing a new sound effect parameter initialization operation.

In some embodiments, the step of increasing the gain coefficient from the second preset value to the second preset value includes:

The gain coefficient is increased from the second preset value to the first preset value by a second preset amplitude.

Exemplarily, the gain coefficient Coef(n)=Coef(n-1)+A2, wherein A2 represents the amplitude of the gain coefficient increase, which is a constant. The initial value of Coef(n-1) is the second preset value.

After the sound effect parameter is switched, the current gain coefficient is determined according to the previous gain coefficient and the second preset amplitude, and the initial value of the gain coefficient is the second preset value;

Determine whether the current gain coefficient is greater than or equal to a first preset value;

If the current gain coefficient is less than the first preset value, the first effect sound is multiplied by the current gain coefficient to obtain a second effect sound, and the step of determining the current gain coefficient according to the previous gain coefficient and the second preset amplitude is continued.

If the current gain coefficient is greater than or equal to the second preset value, the first effect sound is multiplied by the first preset value to obtain the second effect sound, or the first effect sound is directly sent to the mixing module.

The first preset amplitude and the second preset amplitude may be the same or different.

In the embodiment of the present application, after receiving the instruction to switch the sound effect parameters input by the user, the gain coefficient is gradually reduced, and the gain coefficient is gradually increased after switching the sound effect parameters, which can effectively reduce the noise generated by the change of the gain coefficient.

In some embodiments, the step of reducing the gain coefficient from the first preset value to the second preset value includes:

The gain coefficient is reduced from a first preset value to a second preset value by a first target amplitude, and the first target amplitude shows an increasing trend.

Exemplarily, the gain coefficient Coef(n)=Coef(n-1)-A1, A1=x+at, or A1=x+x0, where A1 represents the amplitude of the gain coefficient reduction; x is the initial reduction amplitude, t is the time from the receipt of the user input of the switching sound effect parameter instruction, a is the amplitude increase rate, and x0 is the gain coefficient growth amplitude. The initial value of Coef(n-1) is the first preset value.

In some embodiments, the step of increasing the gain coefficient from the second preset value to the first preset value includes:

The gain coefficient is increased from the second preset value to the first preset value by a second target amplitude, and the second target amplitude shows a downward trend.

Exemplarily, the gain coefficient Coef(n)=Coef(n-1)+A2, A2=x-at, or A2=x-x0, where A2 represents the amplitude of the gain coefficient increase; x is the initial increase amplitude, t is the time taken for the distance sound effect parameter to be switched, a is the amplitude reduction rate, and x0 is the gain coefficient reduction amplitude. The initial value of Coef(n-1) is the second preset value.

In the embodiment of the present application, after receiving the user input of the switching sound effect parameter instruction, the gain coefficient gradually decreases and the decrease amplitude gradually increases; after switching the sound effect parameter, the gain coefficient gradually increases and the increase amplitude gradually decreases. When the sound signal is relatively large, the change amplitude is reduced, and when the sound signal is relatively small, the change amplitude is increased, which effectively reduces the noise generated by the change of the gain coefficient and makes the sound change smoother and more fluent.

In some embodiments, the step of reducing the gain coefficient from the first preset value to the second preset value includes:

The gain factor is reduced from a first preset value to a second preset value at a first preset rate.

Exemplarily, the gain coefficient Coef(n)=Smooth1*Target+(1-Smooth1)*Coef(n-1), where Smooth1 represents the rate of change of the coefficient, that is, the first preset rate, and the value range is 0＜Smooth1＜1. The larger the value, the faster the change, and the greater the noise that may be generated; Target represents the target gain coefficient, which is a constant. When the user inputs the switching sound effect parameter instruction, Target is 0 and Coef(n-1) is 1. Coef(n-1) represents the previous value of the gain coefficient Coef(n).

It should be noted that when Coef(n) is less than a certain fixed value, Coef(n) is considered to be 0. For example, the fixed value is 0.0000001.

In some embodiments, the step of increasing the gain coefficient from the second preset value to the first preset value includes:

The gain factor is increased from a second preset value to a first preset value at a second preset rate.

Exemplarily, the gain coefficient Coef(n)=Smooth2*Target+(1-Smooth2)*Coef(n-1), where Smooth2 represents the rate of change of the coefficient, i.e., the second preset rate, and the value range is 0＜Smooth2＜1. The larger the value, the faster the change, and the greater the noise that may be generated; Target represents the target gain coefficient, which is a constant. When the sound effect parameter switching is completed, Target is 1 and Coef(n-1) is 0. Coef(n-1) represents the previous value of the gain coefficient Coef(n).

In the embodiment of the present application, after receiving the user input of the switching sound effect parameter instruction, the gain coefficient gradually decreases and the decrease amplitude gradually increases; after switching the sound effect parameter, the gain coefficient gradually increases and the increase amplitude gradually decreases. When the sound signal is relatively large, the change amplitude is reduced, and when the sound signal is relatively small, the change amplitude is increased, which effectively reduces the noise generated by the change of the gain coefficient and makes the sound change smoother and more fluent.

The reverberation echo module sends the first effect sound to the gain module, and the gain module performs gain processing on the first effect sound to obtain the second effect sound, and sends the second effect sound to the mixing module.

Step S605: Mixing the direct sound, the second effect sound and the music accompaniment sound to obtain output sound;

Mixing is the process of integrating sounds from multiple sources into a stereo or mono track. During the mixing process, the frequency, dynamics, sound quality, positioning, reverberation and sound field of each original signal are adjusted separately to optimize each track and then superimposed on the final output sound.

The output sound signal y(n)=a(n)+c(n)+d(n), where a(n) is the n-point data in the direct sound, which is the signal after pre-processing of the original sound signal x(n) collected by the sound collector; c(n) is the second effect sound, which is the signal after the first effect sound b(n) is processed by the gain module, and d(n) is the n-point data in the music accompaniment sound.

It should be noted that if the current scene is not a singing scene, such as an audio or video call, live broadcast or game scene, there is no need to add musical accompaniment to the mixing process.

In some embodiments, the direct sound, the second effect sound and the music accompaniment sound are mixed to obtain a mixed sound;

The mixed sound is subjected to target sound processing to obtain an output sound, wherein the target sound processing includes frequency division processing, equalizer processing and/or dynamic gain control.

Crossover processing is a technology used in audio systems to divide audio signals into different frequency bands to ensure that each speaker unit can effectively reproduce audio in a specific frequency range.

An equalizer is an electronic device that can adjust the amplification of electrical signals of various frequency components separately. It compensates for the defects of speakers and sound fields, compensates and modifies various sound sources and has other special functions by adjusting electrical signals of different frequencies.

Dynamic gain control, also known as dynamic range control (DRC), aims to adjust the dynamic range of the signal, that is, the logarithm of the ratio between the maximum and minimum amplitudes of the signal. In audio processing, DRC is usually used as the last module of sound processing to compress or limit the dynamic range of the audio signal to avoid distortion and hardware overload caused by large signals. It can be implemented through different types of controllers, such as dynamic range compressors, limiters and expanders, as well as noise gates, to attenuate signal amplitudes above or below a certain threshold.

The mixing module mixes the direct sound, the second effect sound and the music accompaniment sound, and can directly send the mixed sound as the output sound to the audio output device, or send the mixed sound to the target signal processing module. The target signal processing module performs target sound processing on the mixed sound to obtain the output sound, and sends the output sound to the audio output device.

Step S606: Control the audio output device to play output sound.

In some embodiments, before the audio output device plays the output sound, the output sound signal may be amplified by a power amplifier.

As shown in FIG12 , the sound collector collects the original sound signal and sends the original sound signal to the pre-processing module. The pre-processing module pre-processes the original sound signal to become a direct sound. The pre-processing module divides the direct sound into two paths, one of which is sent to the reverberation echo module. The reverberation echo module processes the direct sound through the reverberation echo algorithm to become the first effect sound. The other path is directly input into the mixing module, mixed with the music accompaniment and the effect sound, and then output to the power amplifier and audio output device. Among them, before mixing, the first effect sound will pass through the gain module. The gain module multiplies the first effect sound by a gain coefficient to become the second effect sound. When receiving an instruction to switch the sound effect parameters, the gain module slowly changes the gain coefficient from 1 to 0. After ensuring that the effect sound is not output, switch the sound effect parameters. After the new sound effect parameter initialization is completed, the gain module will slowly change the gain coefficient from 0 to 1 to restore the volume of the effect sound. At the same time, after the new sound effect parameter initialization is completed, the sound effect module sends an instruction to the whole machine to complete the switching of the sound effect parameters.

The embodiment of the present application adopts the method of only performing gain on the effect sound, that is, fade-in and fade-out processing, and there is no need to perform fade-in and fade-out processing on the output sound of the entire channel, so that there will not be a moment of silence, which will make the user feel that there is a problem when adjusting the parameters. Since only the effect sound is faded in and fade-out, the direct sound of the microphone is still output and played, and the user will not feel the discomfort of sudden silence.

Compared with the solution of switching parameters when silence is detected, the embodiment of the present application directly switches parameters. There is no need to detect whether the sound of the microphone is silent, which reduces the amount of calculation and does not delay switching parameters, effectively ensuring the real-time nature of switching parameters.

Some embodiments of the present application provide a sound processing method, which is applicable to a terminal device, and the terminal device includes a sound collector, an audio output device and a controller, wherein the controller is configured to: after receiving a switching sound effect parameter instruction input by a user, obtain the direct sound of the original sound signal; process the direct sound by a reverberation echo algorithm to obtain a first effect sound; perform gain processing on the first effect sound to obtain a second effect sound, and switch the sound effect parameters; perform mixing processing on the direct sound, the second effect sound and the music accompaniment sound to obtain an output sound; and control the audio output device to play the output sound. When receiving a switching sound effect parameter instruction input by a user, the embodiment of the present application does not need to wait for the microphone sound to be detected as silent before switching the sound effect parameters, but switches the sound effect parameters during the process of gain processing of the effect sound processed by the reverberation echo algorithm, thereby effectively ensuring the real-time nature of the switching parameters. By only performing gain processing on the effect sound when switching the sound effect parameters, the direct sound is still played normally, and while reducing the noise generated by adjusting the sound effect parameters, the user will not feel uncomfortable due to sudden silence.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims (10)

1. A terminal device, comprising:

A sound collector configured to: collecting an original sound signal input by a user;

an audio output device configured to: playing the output sound;

a controller configured to:

after receiving a sound effect switching parameter instruction input by a user, acquiring direct sound of the original sound signal;

Performing reverberation echo algorithm processing on the direct sound to obtain first effect sound;

Gain processing is carried out on the first effect sound so as to obtain a second effect sound and switch sound effect parameters;

mixing the direct sound, the second effect sound and the music accompaniment sound to obtain output sound;

and controlling the audio output device to play the output sound.

2. The terminal device of claim 1, wherein the controller performing the acquisition of the direct sound of the original sound signal is further configured to:

receiving an original sound signal acquired by the sound acquisition device;

And preprocessing the original sound signal to obtain direct sound, wherein the preprocessing comprises noise gate processing, expander processing, gain control, howling suppression and/or blowout prevention processing.

3. The terminal device of claim 1, wherein the controller performs gain processing on the first effect sound to obtain a second effect sound, and switching sound effect parameters is further configured to:

multiplying the first effect sound by a gain coefficient to obtain a second effect sound, wherein the gain coefficient is used for representing the gain of the first effect sound;

After the gain coefficient is reduced from a first preset value to a second preset value, switching the sound effect parameter, wherein the first preset value is larger than the second preset value;

After the sound effect parameter is switched, the gain coefficient is increased from the second preset value to the first preset value.

4. A terminal device according to claim 3, wherein the controller, after performing the reduction of the gain factor from the first preset value to the second preset value, switches the sound effect parameter, is further configured to:

reducing the gain coefficient from a first preset value to a second preset value at a first preset rate, and switching the sound effect parameters;

The controller performs increasing the gain factor from the second preset value to the first preset value, and is further configured to:

And increasing the gain coefficient from the second preset value to the first preset value at a second preset rate.

5. The terminal device according to claim 1, wherein the controller performs a mixing process of the direct sound, the second effect sound, and the musical accompaniment sound to obtain an output sound, and is further configured to:

Mixing the direct sound, the second effect sound and the music accompaniment sound to obtain mixed sound;

And performing target sound processing on the mixed sound to acquire output sound, wherein the target sound processing comprises frequency division processing, equalizer processing and/or dynamic gain control.

6. A sound processing method, comprising:

after receiving a sound effect parameter switching instruction input by a user, acquiring direct sound of an original sound signal, wherein the original sound signal is acquired by a sound acquisition device;

after receiving a sound effect parameter switching instruction input by a user, acquiring direct sound of an original sound signal, wherein the original sound signal is acquired by a sound acquisition device;

Performing reverberation echo algorithm processing on the direct sound to obtain first effect sound;

Gain processing is carried out on the first effect sound so as to obtain a second effect sound and switch sound effect parameters;

mixing the direct sound, the second effect sound and the music accompaniment sound to obtain output sound;

and controlling the audio output device to play the output sound.

7. The method of claim 6, wherein the step of obtaining the direct sound of the original sound signal comprises:

receiving an original sound signal acquired by a sound acquisition device;

And preprocessing the original sound signal to obtain direct sound, wherein the preprocessing comprises noise gate processing, expander processing, gain control, howling suppression and/or blowout prevention processing.

8. The method of claim 6, wherein the step of gain processing the first effect sound to obtain a second effect sound, and switching the sound effect parameter comprises:

Determining the second effect sound as a product of the first effect sound and a gain coefficient, wherein the gain coefficient is used for representing the gain magnitude of the first effect sound;

After the gain coefficient is reduced from a first preset value to a second preset value, switching the sound effect parameter, wherein the first preset value is larger than the second preset value;

After the sound effect parameter is switched, the gain coefficient is increased from the second preset value to the first preset value.

9. The method of claim 8, wherein the step of switching the sound effect parameter after reducing the gain factor from the first preset value to the second preset value comprises:

reducing the gain coefficient from a first preset value to a second preset value at a first preset rate, and switching the sound effect parameters;

The step of increasing the gain factor from the second preset value to the first preset value includes:

And increasing the gain coefficient from the second preset value to the first preset value at a second preset rate.

10. The method of claim 6, wherein the step of mixing the direct sound, the second effect sound, and the music accompaniment sound to obtain an output sound comprises:

Mixing the direct sound, the second effect sound and the music accompaniment sound to obtain mixed sound;

And performing target sound processing on the mixed sound to acquire output sound, wherein the target sound processing comprises frequency division processing, equalizer processing and/or dynamic gain control.