CN112492340B - Live broadcast audio acquisition method, mobile terminal and computer readable storage medium - Google Patents
Live broadcast audio acquisition method, mobile terminal and computer readable storage medium Download PDFInfo
- Publication number
- CN112492340B CN112492340B CN202011361903.4A CN202011361903A CN112492340B CN 112492340 B CN112492340 B CN 112492340B CN 202011361903 A CN202011361903 A CN 202011361903A CN 112492340 B CN112492340 B CN 112492340B
- Authority
- CN
- China
- Prior art keywords
- audio
- live
- audio data
- mobile terminal
- live broadcast
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Telephone Function (AREA)
Abstract
The application provides a live broadcast audio acquisition method, which comprises the following steps: acquiring current live audio data and multimedia data of an application program to be live broadcast of a mobile terminal, wherein the current live audio data are audio data generated by an external environment where the mobile terminal is located; respectively sending the current live audio data and the multimedia data to terminal equipment for screen casting, and playing in a live client by a live server; detecting whether an audio acquisition device corresponding to current live audio data has a fault or not; determining a target audio acquisition device according to the detection result, wherein the target audio acquisition device is determined from the audio acquisition device of the mobile terminal and the audio acquisition device of the terminal device; and acquiring audio data through the target audio acquisition equipment to be used as live broadcast audio data sent to the live broadcast client by the live broadcast server. By the mode, the problem that if the mobile terminal audio acquisition equipment appears, the live broadcasting room has no sound is solved.
Description
Technical Field
The present application relates to the field of network communications technologies, and in particular, to a live broadcast audio acquisition method, a mobile terminal, and a computer readable storage medium.
Background
With the use of mobile terminals becoming more common, the screen-throwing function is also widely applied, especially the game screen throwing and live broadcast screen throwing are favored by the vast users. Therefore, the sound live broadcast processing in the process of game cooperation and live broadcast screen throwing of the athletic team is important, and if the problems of intermittent, no sound, blocking and the like occur, the user experience is greatly influenced. For example, the microphone is carelessly covered by hands, or the microphone sound is not generated due to the fault of the terminal, at the moment, the sound of the user is not generated in a live broadcasting room, and in the live broadcasting process, the emotion of the user is excited, so that the live broadcasting passion is influenced.
Disclosure of Invention
The application mainly aims to provide a live broadcast audio acquisition method, a mobile terminal and a computer readable storage medium, and aims to solve the problem of no sound fault when live broadcast is carried out through the mobile terminal so as to improve user experience.
In order to achieve the above object, the present application provides a live broadcast audio acquisition method, which respectively establishes a communication connection between a mobile terminal and a terminal device, and a communication connection between the mobile terminal and a live broadcast server, the method comprising:
Acquiring current live audio data and multimedia data of an application program to be live broadcast of the mobile terminal, wherein the current live audio data are audio data generated by an external environment where the mobile terminal is located;
the current live broadcast audio data and the multimedia data are respectively sent to the terminal equipment for screen projection, and the live broadcast server plays the current live broadcast audio data and the multimedia data in a live broadcast client;
detecting whether the audio acquisition equipment corresponding to the current live audio data has a fault or not;
determining a target audio acquisition device according to a detection result, wherein the target audio acquisition device is determined from the audio acquisition device of the mobile terminal and the audio acquisition device of the terminal device;
and acquiring audio data through the target audio acquisition equipment to be used as the live broadcast audio data sent to the live broadcast client by the live broadcast server.
Optionally, the step of detecting whether the audio collection device corresponding to the current live audio data has a fault includes:
determining corresponding audio parameters according to the current live audio data;
and detecting whether the audio parameters meet an audio parameter threshold value or not so as to determine whether the audio acquisition equipment corresponding to the current live audio data has a fault or not.
Optionally, when the current live audio data is an analog signal, the step of detecting whether the audio parameter meets an audio parameter threshold is at least one of:
detecting whether the peak value of the signal curve of the current live audio data is smaller than a preset value; and
and detecting whether the waveform of the signal curve of the current live audio data meets a preset waveform or not.
Optionally, when the current live audio data is a digital signal, the step of detecting whether the audio parameter meets an audio parameter threshold is to detect whether a vertex value of a signal curve of the current live audio data is smaller than a preset value.
Optionally, the audio collection device corresponding to the current live audio data is an audio collection device of the mobile terminal, and the step of determining the target audio collection device according to the detection result includes:
when the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the terminal equipment as target audio acquisition equipment.
Optionally, the audio collection device corresponding to the current live audio data is an audio collection device of the terminal device, and the step of determining the target audio collection device according to the detection result includes:
When the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the mobile terminal as target audio acquisition equipment.
Optionally, the step of collecting, by the target audio collecting device, audio data to be sent as live audio data in the live client by the live server includes:
collecting audio data through an audio collecting device of the terminal device;
transmitting the audio data acquired by the audio acquisition equipment of the terminal equipment to the mobile terminal;
and sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server through the mobile terminal so as to be used as the live broadcast server to send the live broadcast audio data in the live broadcast client.
Optionally, the method further comprises: establishing communication connection between the terminal equipment and the live broadcast server, wherein the step of acquiring audio data through the target audio acquisition equipment to serve as live broadcast audio data sent by the live broadcast server to the live broadcast client comprises the following steps of:
collecting audio data through an audio collecting device of the terminal device;
And sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server to serve as the live broadcast server to send the live broadcast audio data in the live broadcast client.
The application also provides a mobile terminal, which comprises: a touch screen; a processor;
and the memory is connected with the processor and contains a control instruction, and when the processor reads the control instruction, the mobile terminal is controlled to realize the live broadcast audio acquisition method.
The present application also provides a computer readable storage medium having one or more programs executed by one or more processors to implement the live audio collection method described above.
According to the live broadcast audio acquisition method, the mobile terminal and the computer readable storage medium, communication connection is respectively established between the mobile terminal and the terminal equipment as well as between the mobile terminal and the live broadcast server, so that the mobile terminal can perform live broadcast screen casting with the terminal equipment, meanwhile, live broadcast with the live broadcast server can be performed, audio data and multimedia data which need to be sent to the live broadcast server are acquired when live broadcast data acquisition is performed, whether the audio acquisition equipment corresponding to the current audio data is detected to have faults or not in the process of acquiring the audio data, and the corresponding target audio acquisition equipment is determined to switch the audio acquisition equipment if the faults exist, thereby ensuring good audio effect in the live broadcast process.
The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.
Drawings
FIG. 1 is a schematic diagram of an alternative mobile terminal hardware architecture for implementing various embodiments of the present application;
FIG. 2 is a schematic diagram of a communication network system of the mobile terminal shown in FIG. 1;
fig. 3 is a flowchart of a live audio collection method according to an embodiment of the present application;
fig. 4 is a schematic diagram of an audio transmission flow when current live audio data is collected by an audio collection device of a mobile terminal according to an embodiment of the present application;
fig. 5 is a schematic diagram of an audio transmission flow when an audio collection device of a terminal collects current live audio data according to an embodiment of the present application;
fig. 6 is a schematic diagram of an audio transmission flow when an audio collection device of a terminal collects current live audio data according to another embodiment of the present application;
fig. 7 is a schematic structural diagram of a mobile terminal according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present application, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.
The terminal may be implemented in various forms. For example, the terminals described in the present application may include mobile terminals such as cell phones, tablet computers, notebook computers, palm computers, personal digital assistants (PersonalDigital Assistant, PDA), portable media players (PortableMediaPlayer, PMP), navigation devices, wearable devices, smart bracelets, pedometers, and the like, as well as stationary terminals such as digital TVs, desktop computers, and the like.
The following description will be given taking a mobile terminal as an example, and those skilled in the art will understand that the configuration according to the embodiment of the present application can be applied to a fixed type terminal in addition to elements particularly used for a moving purpose.
Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention, the mobile terminal 100 may include: RF (radio frequency) unit 101, wiFi module 102, audio output unit 103, a/V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power source 111. Those skilled in the art will appreciate that the mobile terminal structure shown in fig. 1 is not limiting of the mobile terminal and that the mobile terminal may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
The following describes the components of the mobile terminal in detail with reference to fig. 1:
the radio frequency unit 101 may be used for receiving and transmitting signals during the information receiving or communication process, specifically, after receiving downlink information of the base station, processing the downlink information by the processor 110; and, the uplink data is transmitted to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to GSM (global system for mobile communications), GPRS (general packet radio service), CDMA2000 (code division multiple access2000 ), WCDMA (wideband code division multiple access), TD-SCDMA (time division-SynchronousCodeDivisionMultiple Access), FDD-LTE (frequency division duplex-longterm evolution), TDD-LTE (TimeDivision Duplexing-longterm evolution), and the like.
WiFi belongs to a short-distance wireless transmission technology, and a mobile terminal can help a user to send and receive e-mails, browse web pages, access streaming media and the like through the WiFi module 102, so that wireless broadband Internet access is provided for the user. Although fig. 1 shows a WiFi module 102, it is understood that it does not belong to the necessary constitution of a mobile terminal, and can be omitted entirely as required within a range that does not change the essence of the invention.
The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output (e.g., a call signal reception sound, a message reception sound, etc.) related to a specific function performed by the mobile terminal 100. The audio output unit 103 may include a speaker, a buzzer, and the like.
The a/V input unit 104 is used to receive an audio or video signal. The a/V input unit 104 may include a graphic processor (GraphicsProcessingUnit, GPU) 1041 and a microphone 1042, the graphic processor 1041 processing image data of still pictures or videos obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound into audio data. The processed audio (voice) data may be converted into a format output that can be transmitted to the mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting the audio signal.
The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1061 and/or the backlight when the mobile terminal 100 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; as for other sensors such as fingerprint keys, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured in the mobile phone, the detailed description thereof will be omitted.
The display unit 106 is used to display information input by a user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display (LiquidCrystal Display, LCD), an organic light-emitting diode (organic light-EmittingDiode, OLED), or the like.
The user input unit 107 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the mobile terminal. In particular, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 1071 or thereabout by using any suitable object or accessory such as a finger, a stylus, etc.) and drive the corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends the touch point coordinates to the processor 110, and can receive and execute commands sent from the processor 110. Further, the touch panel 1071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 107 may include other input devices 1072 in addition to the touch panel 1071. In particular, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc., as specifically not limited herein.
Further, the touch panel 1071 may overlay the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or thereabout, the touch panel 1071 is transferred to the processor 110 to determine the type of touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of touch event. Although in fig. 1, the touch panel 1071 and the display panel 1061 are two independent components for implementing the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 may be integrated with the display panel 1061 to implement the input and output functions of the mobile terminal, which is not limited herein.
The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.
Memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and a storage data area; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by running or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.
The mobile terminal 100 may further include a power source 111 (e.g., a battery) for supplying power to the respective components, and preferably, the power source 111 may be logically connected to the processor 110 through a power management system, so as to perform functions of managing charging, discharging, and power consumption management through the power management system.
Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described herein.
In order to facilitate understanding of the embodiments of the present invention, a communication network system on which the mobile terminal of the present invention is based will be described below.
Referring to fig. 2, fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention, where the communication network system is an LTE system of a general mobile communication technology, and the LTE system includes a UE (user equipment) 201, an e-UTRAN (Evolved UMTSTerrestrialRadioAccessNetwork ) 202, an epc (evolved packet core) 203, and an IP service 204 of an operator, which are sequentially connected in communication.
Specifically, the UE201 may be the terminal 100 described above, and will not be described herein.
The E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. The eNodeB2021 may be connected with other eNodeB2022 by a backhaul (e.g., an X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide access from the UE201 to the EPC 203.
EPC203 may include MME (mobility management entity) 2031, hss (home subscriber server) 2032, other MME2033, SGW (serving gateway) 2034, pgw (PDNGate Way, packet data network gateway) 2035, PCRF (PolicyandChargingRules Function, policy and tariff function entity) 2036, and the like. The MME2031 is a control node that handles signaling between the UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location registers (not shown) and to hold user specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034 and PGW2035 may provide IP address allocation and other functions for UE201, PCRF2036 is a policy and charging control policy decision point for traffic data flows and IP bearer resources, which selects and provides available policy and charging control decisions for a policy and charging enforcement function (not shown).
IP services 204 may include the internet, intranets, IMS (IPMultimediaSubsystem ), or other IP services, etc.
Although the LTE system is described above as an example, it should be understood by those skilled in the art that the present application is not limited to LTE systems, but may be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems.
Based on the above mobile terminal hardware structure and the communication network system, various embodiments of the method of the present application are provided.
Fig. 3 is a flowchart of an embodiment of a method for collecting a commercial audio. Once triggered by a user, the process in this embodiment automatically operates through the mobile terminal, where each step may be performed sequentially as shown in the flowchart, or may be performed simultaneously according to a plurality of steps in actual situations, which is not limited herein. The live broadcast audio acquisition method provided by the application comprises the following steps:
step S310, current live audio data and multimedia data of an application program to be live broadcast of the mobile terminal are obtained, wherein the current live audio data is audio data generated by an external environment where the mobile terminal is located;
step S320, the current live audio data and the multimedia data are respectively sent to the terminal device for screen projection, and the live server plays the current live audio data and the multimedia data in the live client;
Step S330, detecting whether the audio acquisition equipment corresponding to the current live audio data has a fault or not;
step S340, determining a target audio acquisition device according to a detection result, wherein the target audio acquisition device is determined from the audio acquisition device of the mobile terminal and the audio acquisition device of the terminal device;
and step S350, collecting audio data through the target audio collecting device to be used as the live broadcast server to send the live broadcast audio data to the live broadcast client.
By the mode, the short board of the mobile terminal audio acquisition equipment is made up, and the problem that if the mobile terminal audio acquisition equipment appears, the live broadcasting room has no sound is solved.
The above steps will be specifically described with reference to specific embodiments.
In step S310, current live audio data and multimedia data of an application program to be live broadcast of the mobile terminal are obtained, where the current live audio data is audio data generated by an external environment where the mobile terminal is located.
Specifically, the mobile terminal comprises various application programs and a live broadcast main broadcasting end, communication connection between the mobile terminal and a live broadcast server is established through the live broadcast main broadcasting end, a live broadcast client is installed in other mobile terminals, the mobile terminal sends live broadcast contents to the live broadcast server through the live broadcast main broadcasting end, and then the live broadcast server sends the received live broadcast contents to the live broadcast client, so that the live broadcast client can see the live broadcast contents of the live broadcast main broadcasting end.
In this embodiment, the live content is multimedia data of an application program to be live determined in the mobile terminal, for example, the application program to be live may be a game application, and the multimedia data is picture content and corresponding application audio of the application program to be live acquired by the mobile terminal, for example, when the game application runs, game audio data in a current game picture.
The current live audio data is audio data in an external environment in which the mobile terminal is located, such as, for example, a commentary audio of a host. The live audio data are audio data collected by an audio collection device, wherein the audio collection device can be an audio collection device configured on a mobile terminal, such as a microphone; the audio acquisition device may also be a configured audio acquisition device on other terminal devices. In this embodiment, the other terminal device is a device that is communicatively connected to the mobile terminal, such as a notebook computer.
In step S320, the current live audio data and the multimedia data are respectively sent to the terminal device for screen casting, and the live server performs playing in the live client.
Specifically, the mobile terminal establishes communication connection with the terminal device, and in this embodiment, the terminal device is a notebook computer. The mobile terminal establishes wireless connection with the terminal equipment, the mobile terminal sends the multimedia data of the application program to be live broadcast in the mobile terminal to the terminal equipment through a screen throwing technology, and the terminal equipment analyzes the received multimedia data to obtain a live broadcast picture and corresponding audio data, so that a host broadcast or other players in the same physical space with the host broadcast can watch the content, such as game content, of the application program in the mobile terminal through the terminal equipment.
Meanwhile, as shown in fig. 4, the mobile terminal packages the live audio data and the multimedia data to form a live data packet, and sends the live data packet to the live server, and the live server distributes the received live data packet to all live clients watching the current live content, so that the live clients can see the live content and hear the explanation audio carried out by the host.
In step S330, it is detected whether the audio collection device corresponding to the current live audio data has a fault.
Specifically, since the current live audio data is audio data collected in real time through the audio collection device, a problem occurs in the live audio data sent to the live server due to the fact that the audio collection device fails, for example, audio data meeting requirements are not successfully or previously collected, and further, a live client connected with the live server cannot obtain audio when analyzing the live audio data, for example, a situation that a live broadcasting room does not have a main broadcasting sound occurs. To solve this problem, in this step, it is determined in real time whether or not there is a malfunction in the acquisition device for acquiring the current live audio data. In this embodiment, the mobile terminal determines whether the corresponding audio collection device has a fault by determining the quality of the current live audio data, and step S330 may include the following steps:
Step S3301, corresponding audio parameters are determined according to the current live audio data;
in step S3302, it is detected whether the audio parameter meets an audio parameter threshold, so as to determine whether the audio acquisition device corresponding to the current live audio data has a fault.
The audio parameter is an audio signal waveform corresponding to the current audio data. In an alternative embodiment, step S33022 includes the following steps:
detecting whether the peak value of the signal curve of the current live audio data is smaller than a preset value; and
and detecting whether the waveform of the signal curve of the current live audio data meets a preset waveform or not.
Specifically, when the current live audio data is an analog signal, the signal curve of the current live audio data is a continuous signal curve, whether the audio acquisition equipment has a fault is judged by judging whether the peak value of the signal curve is smaller than a preset value, for example, when the peak value of the signal curve is smaller than the preset value, the audio acquisition equipment is determined to have the fault, otherwise, the audio acquisition equipment has no fault. In other embodiments, whether the waveform of the signal curve of the current live audio data meets a preset waveform is detected to determine whether the audio acquisition device has a fault, for example, if the preset waveform is sinusoidal in the upper half of the signal curve, then the audio acquisition device is determined to have a fault.
In an alternative embodiment, step S33022 includes the following steps: and detecting whether the peak value of the signal curve of the current live audio data is smaller than a preset value. In particular, optionally, when the current live audio data is a digital signal, the signal profile is a discrete signal in both amplitude and time.
In other embodiments, the determination may be made according to other causes that may cause the malfunction of the audio capturing device, for example, determining whether the circuit signals of the audio capturing device are connected, or the like.
Through the embodiment, the mobile terminal can timely detect whether the current audio acquisition equipment has a problem in the live broadcast process so as to timely solve the problem and avoid great adverse effect on the live broadcast effect.
In step S340, a target audio capturing device is determined according to the detection result, wherein the target audio capturing device is determined from the audio capturing device of the mobile terminal and the audio capturing device of the terminal device.
In particular, there may be many devices for capturing live audio data, as long as the device is capable of capturing audio and finally transmitting the audio to a live server. In an alternative embodiment, step S340 may include the steps of:
When the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the terminal equipment as target audio acquisition equipment.
Specifically, as shown in fig. 4, when live broadcasting is performed through the mobile terminal, current live broadcasting audio data is acquired by default through the audio acquisition device of the mobile terminal. At this time, as shown in fig. 5, when a problem is detected in the audio collection device of the mobile terminal, the audio collection device in the terminal device connected to the mobile terminal is used as a target audio collection device, and the current live audio data, such as the comment audio of the anchor, is collected through the audio collection device in the terminal device.
In an alternative embodiment, step S340 may include the steps of:
when the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the mobile terminal as target audio acquisition equipment.
Specifically, if the audio collection device corresponding to the current live audio data is the audio collection device of the terminal device, when detecting that the audio collection device of the terminal device has a fault, the audio collection device of the mobile terminal is used as a target audio collection device to collect the current live audio data.
In step S350, audio data is collected by the target audio collection device, so as to be sent as live audio data in the live client by the live server.
Specifically, in an alternative embodiment, step S350 may include the steps of:
step S3501, collecting audio data by the audio collecting device of the terminal device;
step S3503, transmitting the audio data collected by the audio collection device of the terminal device to the mobile terminal;
step S3503, transmitting, by the mobile terminal, the audio data collected by the audio collection device of the terminal device to the live broadcast server, so as to be transmitted as the live broadcast server to live broadcast audio data in the live broadcast client.
Specifically, as shown in fig. 5, when the mobile terminal detects a microphone failure of the mobile terminal, the microphone of the computer terminal is adopted to receive sound, then the computer terminal sends microphone sound back to the mobile terminal through socket communication, and the mobile terminal packages the microphone sound and the multimedia data together and sends the microphone sound and the multimedia data to the live broadcast server. Through the embodiment, the short board for making up the radio of the audio acquisition equipment of the mobile terminal is replaced by the audio acquisition equipment of the terminal equipment, so that the scheme is feasible, the cost is low, and the mobile terminal is easy to apply to products.
In an alternative embodiment, step S350 may include the steps of:
step S3504, collecting audio data by the audio collecting device of the terminal device;
step S3505, transmitting the audio data collected by the audio collection device of the terminal device to the live broadcast server, so as to be used as the live broadcast server to transmit the live broadcast audio data in the live broadcast client.
Specifically, as shown in fig. 6, a communication connection between the terminal device and the live broadcast server is established, when an audio collection device of the mobile terminal fails, live broadcast audio data is collected through the audio collection device of the terminal device, multimedia data is collected through the mobile terminal, finally the mobile terminal sends the multimedia data to the live broadcast server, the terminal device sends the collected live broadcast audio data to the live broadcast server, the multimedia data and the live broadcast audio data are combined into live broadcast data through the live broadcast server, and then the live broadcast data are sent to the live broadcast client for the live broadcast client to analyze and obtain live broadcast contents.
Fig. 7 is a schematic structural diagram of a mobile terminal 100 according to an embodiment of the present application, where the mobile terminal 100 includes: a touch panel 1071; a processor 110; a memory 109 connected to the processor 110, wherein the memory 109 contains control instructions, and when the processor 110 reads the control instructions, the mobile terminal 100 is controlled to implement the following steps:
Acquiring current live audio data and multimedia data of an application program to be live broadcast of the mobile terminal, wherein the current live audio data are audio data generated by an external environment where the mobile terminal is located;
the current live broadcast audio data and the multimedia data are respectively sent to the terminal equipment for screen projection, and the live broadcast server plays the current live broadcast audio data and the multimedia data in a live broadcast client;
detecting whether the audio acquisition equipment corresponding to the current live audio data has a fault or not;
determining a target audio acquisition device according to a detection result, wherein the target audio acquisition device is determined from the audio acquisition device of the mobile terminal and the audio acquisition device of the terminal device;
and acquiring audio data through the target audio acquisition equipment to be used as the live broadcast audio data sent to the live broadcast client by the live broadcast server.
Optionally, the step of detecting whether the audio collection device corresponding to the current live audio data has a fault includes:
determining corresponding audio parameters according to the current live audio data;
and detecting whether the audio parameters meet an audio parameter threshold value or not so as to determine whether the audio acquisition equipment corresponding to the current live audio data has a fault or not.
Optionally, when the current live audio data is an analog signal, the step of detecting whether the audio parameter meets an audio parameter threshold is at least one of:
detecting whether the peak value of the signal curve of the current live audio data is smaller than a preset value; and
and detecting whether the waveform of the signal curve of the current live audio data meets a preset waveform or not.
Optionally, when the current live audio data is a digital signal, the step of detecting whether the audio parameter meets an audio parameter threshold is to detect whether a vertex value of a signal curve of the current live audio data is smaller than a preset value.
Optionally, the audio collection device corresponding to the current live audio data is an audio collection device of the mobile terminal, and the step of determining the target audio collection device according to the detection result includes:
when the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the terminal equipment as target audio acquisition equipment.
Optionally, the audio collection device corresponding to the current live audio data is an audio collection device of the terminal device, and the step of determining the target audio collection device according to the detection result includes:
When the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the mobile terminal as target audio acquisition equipment.
Optionally, the step of collecting, by the target audio collecting device, audio data to be sent as live audio data in the live client by the live server includes:
collecting audio data through an audio collecting device of the terminal device;
transmitting the audio data acquired by the audio acquisition equipment of the terminal equipment to the mobile terminal;
and sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server through the mobile terminal so as to be used as the live broadcast server to send the live broadcast audio data in the live broadcast client.
Optionally, the method further comprises: establishing communication connection between the terminal equipment and the live broadcast server, wherein the step of acquiring audio data through the target audio acquisition equipment to serve as live broadcast audio data sent by the live broadcast server to the live broadcast client comprises the following steps of:
collecting audio data through an audio collecting device of the terminal device;
And sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server to serve as the live broadcast server to send the live broadcast audio data in the live broadcast client.
Through above-mentioned mobile terminal, through establishing communication connection respectively mobile terminal and terminal equipment and live broadcast server, make mobile terminal can carry out live broadcast with terminal equipment and throw the screen, simultaneously, also can carry out live broadcast with live broadcast server, when carrying out live broadcast data acquisition, obtain audio data and the multimedia data that need send to live broadcast server, in the in-process of gathering audio data, detect and gather the audio acquisition equipment that current audio data corresponds and have the trouble, if have the trouble then confirm corresponding target audio acquisition equipment, with the switching of audio acquisition equipment, guaranteed that the audio effect is good in live broadcast in-process, through above-mentioned mode, make up the short slab of mobile terminal audio acquisition equipment, solve if mobile terminal audio acquisition equipment appears the event, lead to the problem of silent sound in live broadcast room.
The embodiment of the present application also provides a computer-readable storage medium having one or more programs executed by one or more processors to implement the steps of:
Acquiring current live audio data and multimedia data of an application program to be live broadcast of the mobile terminal, wherein the current live audio data are audio data generated by an external environment where the mobile terminal is located;
the current live broadcast audio data and the multimedia data are respectively sent to the terminal equipment for screen projection, and the live broadcast server plays the current live broadcast audio data and the multimedia data in a live broadcast client;
detecting whether the audio acquisition equipment corresponding to the current live audio data has a fault or not;
determining a target audio acquisition device according to a detection result, wherein the target audio acquisition device is determined from the audio acquisition device of the mobile terminal and the audio acquisition device of the terminal device;
and acquiring audio data through the target audio acquisition equipment to be used as the live broadcast audio data sent to the live broadcast client by the live broadcast server.
Optionally, the step of detecting whether the audio collection device corresponding to the current live audio data has a fault includes:
determining corresponding audio parameters according to the current live audio data;
and detecting whether the audio parameters meet an audio parameter threshold value or not so as to determine whether the audio acquisition equipment corresponding to the current live audio data has a fault or not.
Optionally, when the current live audio data is an analog signal, the step of detecting whether the audio parameter meets an audio parameter threshold is at least one of:
detecting whether the peak value of the signal curve of the current live audio data is smaller than a preset value; and
and detecting whether the waveform of the signal curve of the current live audio data meets a preset waveform or not.
Optionally, when the current live audio data is a digital signal, the step of detecting whether the audio parameter meets an audio parameter threshold is to detect whether a vertex value of a signal curve of the current live audio data is smaller than a preset value.
Optionally, the audio collection device corresponding to the current live audio data is an audio collection device of the mobile terminal, and the step of determining the target audio collection device according to the detection result includes:
when the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the terminal equipment as target audio acquisition equipment.
Optionally, the audio collection device corresponding to the current live audio data is an audio collection device of the terminal device, and the step of determining the target audio collection device according to the detection result includes:
When the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the mobile terminal as target audio acquisition equipment.
Optionally, the step of collecting, by the target audio collecting device, audio data to be sent as live audio data in the live client by the live server includes:
collecting audio data through an audio collecting device of the terminal device;
transmitting the audio data acquired by the audio acquisition equipment of the terminal equipment to the mobile terminal;
and sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server through the mobile terminal so as to be used as the live broadcast server to send the live broadcast audio data in the live broadcast client.
Optionally, the method further comprises: establishing communication connection between the terminal equipment and the live broadcast server, wherein the step of acquiring audio data through the target audio acquisition equipment to serve as live broadcast audio data sent by the live broadcast server to the live broadcast client comprises the following steps of:
collecting audio data through an audio collecting device of the terminal device;
And sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server to serve as the live broadcast server to send the live broadcast audio data in the live broadcast client.
Through the computer readable storage medium, through establishing communication connection respectively with mobile terminal and terminal equipment and live broadcast server for mobile terminal can live broadcast with terminal equipment throws the screen, simultaneously, also can live broadcast with live broadcast server, when carrying out live broadcast data acquisition, acquire audio data and the multimedia data that need send to live broadcast server, in the in-process of gathering audio data, detect and gather the audio acquisition equipment that current audio data corresponds and have the trouble, if have the trouble then confirm corresponding target audio acquisition equipment, with the switching of audio acquisition equipment, guaranteed that audio effect is good in live broadcast in-process, through above-mentioned mode, make up the short board of mobile terminal audio acquisition equipment, solve if mobile terminal audio acquisition equipment appears the event, lead to the problem of silence between live broadcast.
The embodiment of the application also provides a computer readable storage medium. The computer-readable storage medium here stores one or more programs. Wherein the computer readable storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid state disk; the memory may also comprise a combination of the above types of memories.
The corresponding technical features in the above embodiments can be used mutually without causing contradiction between schemes or incapacitation.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.
The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.
Claims (7)
1. The live broadcast audio acquisition method is characterized by respectively establishing communication connection between a mobile terminal and terminal equipment and communication connection between the mobile terminal and a live broadcast server, and comprises the following steps:
acquiring current live audio data and multimedia data of an application program to be live broadcast of the mobile terminal, wherein the current live audio data are audio data generated by an external environment where the mobile terminal is located;
transmitting the current live audio data and the multimedia data to the terminal equipment for screen casting, and transmitting the current live audio data and the multimedia data to the live server for playing in a live client;
detecting whether the audio acquisition equipment corresponding to the current live audio data has a fault or not comprises the following steps:
Determining corresponding audio parameters according to the current live audio data;
detecting whether the audio parameter meets an audio parameter threshold value to determine whether an audio acquisition device corresponding to the current live audio data has a fault, including:
when the current live audio data is an analog signal, the step of detecting whether the audio parameter meets an audio parameter threshold is at least one of:
detecting whether the peak value of the signal curve of the current live audio data is smaller than a preset value; and
detecting whether the waveform of the signal curve of the current live audio data meets a preset waveform or not;
when the current live audio data is a digital signal, the step of detecting whether the audio parameter meets an audio parameter threshold value is to detect whether the peak value of a signal curve of the current live audio data is smaller than a preset value;
determining a target audio acquisition device according to a detection result, wherein the target audio acquisition device is determined from the audio acquisition device of the mobile terminal and the audio acquisition device of the terminal device;
and acquiring audio data through the target audio acquisition equipment to serve as live audio data sent to the live client by the live server.
2. The method of claim 1, wherein the audio collection device corresponding to the current live audio data is an audio collection device of the mobile terminal, and the step of determining the target audio collection device according to the detection result includes:
when the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the terminal equipment as target audio acquisition equipment.
3. The method of claim 1, wherein the audio collection device corresponding to the current live audio data is an audio collection device of the terminal device, and the step of determining the target audio collection device according to the detection result includes:
when the audio acquisition equipment corresponding to the current live audio data has faults, determining the audio acquisition equipment of the mobile terminal as target audio acquisition equipment.
4. The method of claim 2, wherein the step of collecting audio data by the target audio collection device for transmission as live audio data by the live server into the live client comprises:
collecting audio data through an audio collecting device of the terminal device;
Transmitting the audio data acquired by the audio acquisition equipment of the terminal equipment to the mobile terminal;
and sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server through the mobile terminal so as to be used as the live broadcast audio data sent to the live broadcast client by the live broadcast server.
5. The method of claim 2, wherein the method further comprises: establishing communication connection between the terminal equipment and the live broadcast server; the step of collecting audio data through the target audio collection device to serve as live audio data sent to the live client by the live server includes:
collecting audio data through an audio collecting device of the terminal device;
and sending the audio data acquired by the audio acquisition equipment of the terminal equipment to the live broadcast server to serve as live broadcast audio data sent to the live broadcast client by the live broadcast server.
6. A mobile terminal, the mobile terminal comprising:
a touch screen;
a processor;
and the memory is connected with the processor and contains a computer program, and when the processor reads the computer program, the mobile terminal is controlled to realize the live broadcast audio acquisition method of any one of claims 1-5.
7. A computer readable storage medium having one or more programs for execution by one or more processors to implement the live audio collection method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011361903.4A CN112492340B (en) | 2020-11-27 | 2020-11-27 | Live broadcast audio acquisition method, mobile terminal and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011361903.4A CN112492340B (en) | 2020-11-27 | 2020-11-27 | Live broadcast audio acquisition method, mobile terminal and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112492340A CN112492340A (en) | 2021-03-12 |
| CN112492340B true CN112492340B (en) | 2023-08-11 |
Family
ID=74936345
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011361903.4A Active CN112492340B (en) | 2020-11-27 | 2020-11-27 | Live broadcast audio acquisition method, mobile terminal and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112492340B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115225917B (en) * | 2021-04-15 | 2024-08-27 | 北京字节跳动网络技术有限公司 | Recording plug-flow method, device, equipment and medium |
| CN113596502A (en) * | 2021-08-03 | 2021-11-02 | 广州繁星互娱信息科技有限公司 | Live broadcast room sound effect adjusting method and device, electronic equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101114942A (en) * | 2006-05-08 | 2008-01-30 | 奥科有限公司 | Switching Between Secure Media Devices |
| JP2018007053A (en) * | 2016-07-04 | 2018-01-11 | アルパイン株式会社 | On-vehicle equipment and processing method in on-vehicle equipment |
| CN107707728A (en) * | 2017-07-25 | 2018-02-16 | 努比亚技术有限公司 | A kind of audio transmission switching method, terminal and computer-readable recording medium |
| CN110856080A (en) * | 2019-12-13 | 2020-02-28 | 中科海微(北京)科技有限公司 | Full-automatic intelligent microphone switching system and device |
-
2020
- 2020-11-27 CN CN202011361903.4A patent/CN112492340B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101114942A (en) * | 2006-05-08 | 2008-01-30 | 奥科有限公司 | Switching Between Secure Media Devices |
| JP2018007053A (en) * | 2016-07-04 | 2018-01-11 | アルパイン株式会社 | On-vehicle equipment and processing method in on-vehicle equipment |
| CN107707728A (en) * | 2017-07-25 | 2018-02-16 | 努比亚技术有限公司 | A kind of audio transmission switching method, terminal and computer-readable recording medium |
| CN110856080A (en) * | 2019-12-13 | 2020-02-28 | 中科海微(北京)科技有限公司 | Full-automatic intelligent microphone switching system and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112492340A (en) | 2021-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112004173B (en) | Bluetooth headset switching method, mobile terminal and computer readable storage medium | |
| CN110784898B (en) | Network switching method, mobile terminal and computer readable storage medium | |
| CN112394896B (en) | Reverse screen-throwing frame rate control method, device and computer readable storage medium | |
| CN109890022B (en) | Data transmission control method, mobile terminal and computer readable storage medium | |
| CN109195213B (en) | Mobile terminal screen control method, mobile terminal and computer readable storage medium | |
| CN112689191B (en) | Screen projection control method, terminal and computer readable storage medium | |
| CN112423076B (en) | Audio screen-throwing synchronous control method, equipment and computer readable storage medium | |
| CN111479309B (en) | Service processing method, user equipment and computer readable storage medium | |
| CN109982402B (en) | Terminal voice calling method, mobile terminal and computer readable storage medium | |
| CN113032065B (en) | Screen refresh control method, mobile terminal and computer readable storage medium | |
| CN112866685B (en) | Screen projection delay measurement method, mobile terminal and computer readable storage medium | |
| CN112433690B (en) | Data processing method, terminal and computer readable storage medium | |
| CN112929943B (en) | Electronic price tag relay communication method, electronic price tag relay communication equipment and computer readable storage medium | |
| CN112600815B (en) | Video display method, terminal and computer readable storage medium | |
| CN112667339B (en) | Screen refresh rate setting method, terminal and computer readable medium | |
| CN108579081B (en) | Event processing method, device and computer storage medium based on game | |
| CN112492340B (en) | Live broadcast audio acquisition method, mobile terminal and computer readable storage medium | |
| CN110052024B (en) | Game vibration adjusting method, adjusting device, mobile terminal and storage medium | |
| CN112954755B (en) | Optimized network switching method and device and computer readable storage medium | |
| CN111970738B (en) | Network switching control method, device and computer readable storage medium | |
| CN112578978B (en) | Window switching control method, device and computer readable storage medium | |
| CN112770279B (en) | Network sharing method, mobile terminal and computer readable storage medium | |
| CN112437472B (en) | Network switching method, equipment and computer readable storage medium | |
| CN109462829B (en) | Call transfer method, device and computer readable storage medium | |
| CN109388947B (en) | Background management method, device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |