CN114928763A - Playing detection, starting up and echo processing method and device, electronic equipment and product - Google Patents

Abstract

The present application relates to a playback detection, power-on, echo processing method, device, electronic device and product, wherein the sound playback detection processing method includes: acquiring a first audio signal and an audio and video source detected by a sound pickup module of an audio and video source output device The second audio signal sent by the output device to the audio and video playback device; according to the audio signal correlation and phase difference between the first audio signal and the second audio signal, detect whether the audio and video playback device plays the second audio signal; if the audio and video playback device If the second audio signal is not played, the second audio signal is played through the speaker of the audio and video source output device. The embodiment of the present invention realizes intelligent switching by analyzing and judging the correlation and phase difference between the first audio signal retrieved by the sound pickup module of the audio and video source output device and the second audio signal sent to the audio and video playback device. The device that plays the audio, the detection of the power-on state, and the processing of echo cancellation.

Description

Playback detection, startup, echo processing method, device, electronic device and product

technical field

The present application relates to a method, device, electronic device and product for playback detection, startup, and echo processing, belonging to the technical field of computers.

Background technique

In the prior art, TV boxes are generally used as audio and video source output devices, and TV devices are generally used as audio and video playback devices. Compared with TV devices, the sound source provided by TV boxes may only be one of the sound sources input by TV devices. When the device plays signals from other sound sources, the sound channel of the TV device is occupied, and the audio signal output by the TV box to the TV device may not be played. In addition, when there is a problem with the sound signal transmission line between the TV device and the TV box, the audio signal output by the TV box to the TV device may not be played. Such a situation will seriously affect the video playback effect, and the user often needs to manually adjust or use the device to solve the problem, causing great inconvenience to the user.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a method, device, electronic device, and product for playback detection, startup, and echo processing, so as to intelligently select audio and video playback devices to play sound source signals and reduce the occurrence of situations where sound source signals cannot be played.

In order to achieve the above purpose, an embodiment of the present invention provides a sound playback detection and processing method, including:

Obtain the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device;

Detecting whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference of the first audio signal and the second audio signal;

If the audio and video playback device does not play the second audio signal, the second audio signal is played through the speaker of the audio and video source output device.

Embodiments of the present invention also provide a method for detecting and processing a power-on state of an audio and video playback device, including:

Send a detection audio signal to the audio and video playback device, and obtain the first audio signal detected by the sound pickup module of the audio and video source output device;

Detecting whether the audio and video playback device plays the detected audio signal according to the audio signal correlation and phase difference between the first audio signal and the detected audio signal;

If the audio and video playback device plays the detected audio signal, output the audio and/or video content signal to the audio and video playback device.

The embodiment of the present invention also provides an echo cancellation processing method, including:

Obtain the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device;

Comparing the signal characteristics of the first audio signal and the second audio signal, and obtaining the part of the audio signal whose correlation is greater than a preset correlation threshold;

determining a phase difference between the associated audio signal portion of the first audio signal and the associated audio signal portion of the second audio signal;

The phase difference-eliminated second audio signal is removed from the first audio signal.

The embodiment of the present invention also provides a sound playback detection and processing device, including:

an audio signal acquisition module for acquiring the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device;

A playback detection module, configured to detect whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference of the first audio signal and the second audio signal;

An audio playback switching processing module, configured to play the second audio signal through the speaker of the audio and video source output device when it is detected that the audio and video playback device does not play the second audio signal.

The embodiment of the present invention also provides a power-on state detection and processing device for an audio and video playback device, including:

The detection signal sending module is used to send the detection audio signal to the audio and video playback device;

an audio signal acquisition module for acquiring the first audio signal detected by the sound pickup module of the audio and video source output device;

A playback detection module, configured to detect whether the audio and video playback device plays the detected audio signal according to the audio signal correlation and phase difference of the first audio signal and the detected audio signal;

A playback processing module, configured to output audio and/or video content signals to the audio and video playback device when it is detected that the audio and video playback device plays the detected audio signal.

The embodiment of the present invention also provides an echo cancellation processing device, including:

an audio signal acquisition module for acquiring the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device;

a correlation processing module, configured to compare the signal characteristics of the first audio signal and the second audio signal, and obtain the part of the audio signal whose correlation is greater than a preset correlation threshold;

a phase difference determination module for determining a phase difference between a relevant audio signal portion of the first audio signal and a relevant audio signal portion of the second audio signal;

The echo cancellation module is configured to remove the second audio signal after the phase difference has been eliminated from the first audio signal.

The embodiment of the present invention also provides a projection playback processing method, including:

Output audio and video signals to projection equipment;

The sound pickup module detects whether the projection device plays the audio signal in the audio and video signal, and if it is determined that the projection device does not play the audio signal, the audio signal is played through the speaker of the audio and video source output device.

The embodiment of the present invention also provides a remote conference collaborative playback processing method, including:

Obtain the same-screen image signal of the remote conference and the audio and video signals of the conference site;

Compare the video signal in the audio-video signal with the image signal on the same screen, and determine the audio signal corresponding to the image signal on the same screen;

The same-screen image signal and the corresponding audio signal are played synchronously.

The embodiment of the present invention also provides an audio and video source output device, including:

The audio and video signal output module is used to output audio and video signals to the audio and video playback device;

The audio signal playing module is used for playing the audio signal output to the audio and video playing device according to the instruction of the audio playing detection module;

The audio playback detection module is used to detect whether the audio playback device plays the audio signal, and if it is determined that the audio playback device does not play the audio signal, trigger the audio signal playback module to play the audio Signal.

The embodiment of the present invention also provides an electronic device, including:

memory for storing programs;

The processor is used to run the program stored in the memory to execute the aforementioned sound playback detection processing method, the power-on state detection processing method of the audio and video playback equipment, the echo cancellation processing method, the projection playback processing method and the remote conference coordination Play any one or more of the processing methods.

An embodiment of the present invention also provides a computer program product, comprising a computer program or an instruction, characterized in that, when the computer program or instruction is executed by a processor, the processor is caused to implement the aforementioned sound playback detection processing method, Any one or more methods of a power-on state detection processing method, an echo cancellation processing method, a projection playback processing method, and a remote conference cooperative playback processing method of an audio and video playback device.

The playback detection, power-on, and echo processing methods, devices, electronic devices, and products of the embodiments of the present invention are sent to the audio and video playback device through the first audio signal retrieved by the sound pickup module of the audio and video source output device and the audio and video source output device. The correlation and phase difference between the second audio signals are analyzed and judged, which realizes the intelligent switching of the audio-playing device, the detection of the power-on state of the audio-video playback device and the processing of echo cancellation, thereby improving the user experience of audio and video playback. experience.

The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.

Description of drawings

1 is a schematic diagram of an application scenario of a sound playback detection processing method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of an application scenario of a processing method for starting a TV according to an embodiment of the present invention;

3 is a schematic diagram of an application scenario of an echo cancellation processing method according to an embodiment of the present invention;

4 is a schematic flowchart of a sound playback detection processing method according to an embodiment of the present invention;

5 is a schematic flowchart of a method for detecting a power-on state of an audio and video playback device according to an embodiment of the present invention;

6 is a schematic flowchart of an echo cancellation processing method according to an embodiment of the present invention;

7 is a schematic structural diagram of a sound playback detection processing apparatus according to an embodiment of the present invention;

8 is a schematic structural diagram of a power-on state detection and processing device of an audio and video playback device according to an embodiment of the present invention;

9 is a schematic structural diagram of an echo cancellation processing apparatus according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

In some application scenarios, the audio and video source output device is wired or wirelessly connected to the audio and video playback device, and provides playback content to the audio and video playback device, and then the audio and video playback device's speakers play. However, if the channel of the audio and video playback device is occupied by other playback sources, or there is a problem with the transmission line of the playback content, it will cause the situation that the sound cannot be played. In view of this situation, the embodiments of the present invention provide a sound playback detection and processing method, which collects and recognizes the sound played by the audio and video playback device through a sound pickup module (such as a microphone) set on the audio and video source output device, and judges Whether the audio and video playback device is playing the audio signal output by the audio and video source output device, if not, it will be played through the speakers on the audio and video source output device.

As shown in FIG. 1, it is a schematic diagram of an application scenario of the sound playback detection processing method according to the embodiment of the present invention. The scene shown in FIG. 1 uses a TV device as an example of an audio and video playback device, and a TV box as an example of an audio and video source output device. . As shown in the figure, the TV box accesses the audio and video content providing platform through the Internet, and obtains audio and video content data from the platform. Hereinafter, the audio and video content may include audio content, video content, and combined audio and video content. The TV box has a system for processing audio and video sources. After the processing system obtains the audio and video content data from the platform for decoding and other processing, it can pass the HDMI (High Definition Multimedia Interface, high-definition multimedia interface) signal output port to the TV. device to output. The TV device and the TV box can be connected in a wired or wireless manner. As an example, the TV device and the TV box in FIG. 1 are connected through the HDMI transmission line.

In the embodiment of the present invention, it mainly involves the processing of audio signals. The audio signals mentioned here can be independent audio content, such as music, broadcasting, etc., or audio content combined with video, such as film and television dramas. For TV equipment, the TV box is the audio and video source output device. Under normal circumstances, the TV device obtains the HDMI signal sent by the TV box through the HDIM signal input port, and plays it through the audio and video playback processing system. Among them, the audio The signal is played through speaker 1 on the TV device. In addition, the TV box can also send pure audio signals to the TV device through the HDMI channel, such as playing online music, etc. In this case, the TV device only acts as a device for playing audio. However, in some cases, the channel that sends the audio signal to the speaker may be occupied by other audio sources, it may appear that the TV device plays the video but does not play the corresponding audio, or, in the case of playing music, although, The TV device receives the audio signal transmitted to it by the TV box, but cannot play it, for example, the TV device is playing audio content from the internal memory, or the channel is occupied by another HDMI signal input port.

In view of this situation, in the embodiment of the present invention, the sound pickup module on the TV box can be used to detect the audio signal of the speaker 1 of the TV device. If the audio signal of the speaker 1 and the audio signal sent by the TV box to the TV device are detected If the signals are consistent, it means that the TV device is playing the audio signal sent by the TV box normally, otherwise, it means that the TV device is playing the audio signal sent by the TV box to the TV device, then turn on the speaker 2 on the TV box to play the audio signal.

Specifically, whether the speaker 1 of the TV device plays the audio signal sent by the TV box to the TV device can be detected in the following manner. As shown in Figure 1, this part of the processing can be implemented by the audio playback detection module in the figure. The audio playback detection module can be implemented by hardware and/or software, and can exist in the TV box as a system for processing audio and video sources. middle. On the one hand, the audio playback detection module can detect the audio signal in the environment through the microphone. For the convenience of distinguishing the representation, it is called the first audio signal here. It should be noted that the first audio signal here is obtained by receiving the sound in the environment. The generated audio signal may include the audio signal emitted by the speaker 1, may also include the audio signal emitted by other devices in the environment, and may also include the audio signal emitted by the speaker 2 of the TV box itself; on the other hand, the audio playback detection module The audio signal output to the TV device can be obtained from the HDMI signal output port, which is referred to as the second audio signal here. The second audio signal is relatively determined. The audio signal can be read from the hardware system or software system of the TV box. However, in order to calculate the accuracy of the phase difference, it is better to obtain the signal from the HDMI signal output port, so that the signal delay inside the TV box of the second audio signal can be ignored.

After acquiring the first audio signal and the second audio signal, the audio playback detection module analyzes the audio signal correlation and phase difference between the first audio signal and the second audio signal, and determines whether the television device is in the Play the second audio signal output by the TV box, and further determine which device speaker is used to play the second audio signal. Specifically, the following situations may exist:

Situation 1: If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than the preset phase difference threshold, then It is determined that the TV device is playing the second audio signal, and no processing is required in this case. The audio signal correlation degree indicates whether the first audio signal contains components of the second audio signal, and the audio phase difference indicates the lag between the correlated audio signal components. In the case where the TV device and the TV box are connected by an HDMI line, the transmission from the TV box to the device and the processing of the HDMI signal received by the TV device will cause a certain delay, and the sound is captured through the microphone of the TV box and then converted. A certain processing delay is also required for the audio signal. These delays are much larger than the delay caused by directly broadcasting the second audio signal through the speakers of the TV box. The delay between related audio signal components in the second audio signal output by the box is about several hundreds of milliseconds, while the delay generated by directly broadcasting the second audio signal through the speakers of the TV box is about several tens of milliseconds. Therefore, through the delay, it can be distinguished which speaker is playing the sound corresponding to the second audio signal.

Scenario 2: If the audio signal correlation between the first audio signal and the second audio signal is less than the preset correlation threshold, it means that the TV device does not play the second audio signal, and the second audio signal needs to be passed through the speakers of the TV box. play.

Situation 3: If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, but the phase difference between the first audio signal and the second audio signal is less than the preset phase difference threshold, It means that the speaker of the TV box is playing the second audio signal, and no processing is required in this case. Scenario 3 occurs in a subsequent situation of Scenario 2, that is, the TV device is still in a situation where the second audio signal cannot be played normally, and the speakers of the TV box continue to play.

Case 4: If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, and the first audio signal includes sub audio signals and sub audio signals with different phase differences, one of the sub audio signals and The phase difference between the second audio signals is greater than the preset phase difference threshold, and the phase difference between the other sub-audio signal and the second audio signal is less than the phase difference threshold, then it is determined that both the TV equipment and the speakers of the TV box are playing the The second audio signal. In this case, the speaker of the audio and video source output device can stop playing the second audio signal. Situation 4 generally occurs when the TV device cannot play the second audio signal normally, and switches to play the second audio signal through the speakers of the TV box, that is, the follow-up state of the above-mentioned situation 2. However, after a period of time, the TV The device can play the second audio signal normally, so that the speakers of both devices are playing. In this case, the second audio signal is preferably played by the speakers of the TV The speaker plays the second audio signal.

The above-mentioned correlation between audio signals can be determined by extracting frequency domain features of the audio signals, and then performing frequency domain analysis and comparison. Each time detection is required, the audio playback detection module can simultaneously collect audio signals from the HDMI signal output port and the microphone, respectively, to form multiple frames of the second audio signal and the first audio signal. Since the audio signal collected through the microphone will have a certain delay, it is possible to compare the audio signals of multiple frames within a certain period of time. audio signal and 5 frames of the first audio signal. For a certain frame of audio signal, it can be compared with the second audio signal of the corresponding frame in the same time period and the second audio signal of the previous frames, so as to determine whether the first audio signal contains the same frequency as the second audio signal. The audio signal components whose correlation degree exceeds a preset correlation degree threshold value are used to determine whether the audio signal collected by the microphone contains the content of the second audio signal.

The phase difference between the above-mentioned audio signals can be determined on the basis of the components of the first audio signal including the second audio signal, and the feature point of the part with a higher degree of correlation in the first audio signal and the second audio signal can be found, Based on these feature points, the phase difference, such as the phase difference between the peak points of the audio signal, is calculated. Determining the phase difference between the audio signals also determines the delay between the audio signals, so that it can be further determined whether the component of the second audio signal contained in the first audio signal is played by the speaker of the TV device or the TV box of speakers.

The application of the embodiments of the present invention in detection of sound playback and switching processing between a TV device and a TV box has been described above. Next, the application for the booting scene of the TV device will be introduced.

As shown in FIG. 2 , it is a schematic diagram of an application scenario of the processing method for turning on a TV according to an embodiment of the present invention. In the figure, a TV box is used as an example of an audio and video source output device, and a TV device is used as an example of an audio and video playback device for description. In the following cases, the TV box can detect the user's voice command through the microphone, and trigger the audio and video playback processing. For example, the user sends a "play XX video" command through the voice. In this case, the TV box needs to confirm whether the TV device is in It is turned on, and then sends audio and video signals, so as to avoid the TV equipment not being turned on or in the process of turning on, resulting in the inability to play audio and video content for a period of time, thus affecting the user experience. The power-on operation of the TV device can be performed independently of the TV box. For example, the user can perform the boot-up operation through the remote control or voice commands. In addition, since the TV box is generally directly connected with the TV device, the power-on operation of the TV device can also be performed through the TV box. Realization, that is, the TV box can control the power-on of the TV device through the control command. In this case, after receiving the user's audio and video playback voice command, the TV box can perform the power-on state detection of the TV device, and after confirming that the TV device has been turned on, send the audio and video content signal to be played. It is detected that the TV device has not been turned on, and a boot control command can be sent to the TV device to trigger the TV to turn on. In addition, the TV box can also directly send a boot control command to the TV device after receiving the user's audio and video playback voice command, and Constantly detect whether the TV device has finished booting.

It generally takes a period of time for the TV device to start up and wait. During this process, the system of the TV device is starting up, so the audio or video content sent by the TV box cannot be played. In addition, some TV devices may play a boot-up video or boot-up advertisement after the system is started, so that the audio and video content sent by the TV box cannot be played. During the time period when the TV device cannot play the audio and video content, if the TV box starts to send the audio and video content to the TV device, the user will miss the playback content during this time. In response to this situation, the embodiment of the present invention determines whether the TV device has been turned on by detecting the audio signal sent to the TV device and performing sound retrieval and comparison through the microphone of the TV box.

Specifically, as shown in FIG. 2 , the TV box sends a detection audio signal to the TV device through the HDMI signal output port, and detects the audio signal through the microphone to generate the first audio signal. Then, according to the audio signal correlation degree and phase difference between the first audio signal and the power-on detection audio signal, it is detected whether the television device plays the power-on detection audio signal. The specific detection and determination method is the same as the detection principle in the sound playback detection processing method described above, that is, if the correlation between the first audio signal and the audio signal of the detected audio signal is greater than the preset correlation threshold, and the first audio signal If the phase difference between the audio signal and the detected audio signal is greater than the preset phase difference threshold, it can be considered that the TV device has played the detected audio signal, so that the video content signal to be played can be transmitted to the TV device, as shown in Figure 2 (b). ) shown. In addition, if the TV device does not play the power-on detection audio signal, for example, it may be that the TV device has not completed startup, as shown in Figure 2(a) The speaker of the TV device does not play any sound or plays a boot-up video or advertisement other than the detection audio, etc. Sound, in view of this situation, it is necessary to send the power-on detection audio signal to the TV equipment again, and perform a new round of processing to detect whether the TV equipment plays the power-on detection audio signal. There can be a certain time interval between each round of detection processing. Through such repeated detection, video playback can be automatically started after the TV device is powered on, and the user will not miss part of the audio and video content due to the boot process of the TV device.

Hereinafter, another application scenario of using the above-mentioned technical principle of sound playback detection will be introduced. As shown in FIG. 3 , it is a schematic diagram of an application scenario of the echo cancellation processing method according to the embodiment of the present invention. Still take a TV box as an example of an audio and video source output device, and a TV device as an example of an audio and video playback device. The echo cancellation processing method is applied to the situation of recognizing the user's voice command. As shown in the figure, the TV box transmits audio and video content signals to the TV device through the HDMI signal output port. For the convenience of description, the audio part is called the second part. Audio signal, the second audio signal will be played through the speaker 1 of the TV device. In order to distinguish it from the second audio signal output by the original TV box (there is a phase difference), the audio played by the speaker 1 is called content audio. On the other hand, the user may issue a voice command to the TV box, and the voice command and the content audio played by the speaker 1 are mixed in the environment to form mixed audio. After the mixed audio is collected by the microphone of the TV box, a first audio signal is formed. That is to say, when the user sends a voice command, the first audio signal includes the second audio signal played after a delay and the audio signal sent by the user. voice commands. The echo cancellation processing method of the embodiment of the present invention is to eliminate the first audio signal to the second audio signal, so as to obtain relatively pure user voice commands for precise control processing.

The echo cancellation is still implemented by utilizing the characteristics of the audio signal correlation and the phase difference between the first audio signal and the second audio signal. Specifically, the echo cancellation processing can be implemented by the echo cancellation module in the figure. The basic principle is similar to that shown in FIG. 1. The echo cancellation module can obtain the first audio signal from the microphone, and obtain the first audio signal from the HDMI signal output port. The second audio signal from the TV box to the TV device. After that, compare the signal characteristics of the first audio signal and the second audio signal, obtain the relevant audio signal part of which the correlation degree of the audio signal is greater than the preset correlation degree threshold, and determine the relevant audio signal part of the first audio signal and the second audio signal. The phase difference between the relevant audio signal parts of the audio signal, and then according to the determined phase difference, the second audio signal after eliminating the phase difference is removed from the first audio signal. The process of eliminating the phase difference mentioned here can be specific In order to align the signals on the time axis according to the determined phase difference, for example, if the determined phase difference is 100ms, that is, the correlation component in the first audio signal lags behind the second audio signal by 100ms, then the current first audio signal can be The second audio signal before 100ms is aligned and then subtracted, so that the component corresponding to the second audio signal can be eliminated.

The above-mentioned calculation process of determining the phase difference does not need to be calculated every time the echo cancellation process is performed, and can be performed at a predetermined time interval, that is, after calculating the phase difference once, within this time interval, the phase difference can be used. for echo cancellation. Since the transmission delay of the HDMI signal may fluctuate, in practical applications, a reasonable time interval can be determined according to the delay fluctuation of the HDMI signal. In addition, in some cases, the above-mentioned second audio signal may be played through the speaker 2 of the TV box itself. The principle of echo processing for this case is the same as that described above, except that the determined phase difference will be relatively small.

The technical solutions of the present invention are further described below through some specific embodiments.

Example 1

As shown in FIG. 4 , which is a schematic flowchart of a sound playback detection processing method according to an embodiment of the present invention, the method can be applied to an audio and video source output device such as a TV box, and the audio and video source output device itself is used for collecting audio signals. The sound pickup module (such as a microphone) and a speaker are connected between the audio and video source output device and an audio and video playback device such as a TV device through a wired or wireless manner, so as to transmit audio and video signals. Specifically, the method includes:

S101: Acquire a first audio signal detected by a sound pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device. The first audio signal is an audio signal generated by collecting sounds in the environment through the sound pickup module of the audio and video source output device, which may include audio signals sent by the audio and video playback device, or may include audio signals sent by other devices in the environment. It may also include audio signals from the speakers of the audio and video source output device itself. The second audio signal is the audio content that the audio and video source output device sends to the audio and video playback device and is expected to be played by the audio and video playback device. Compared with the first audio signal, the second audio signal is a definite signal. When the audio and video source output device is a TV box, the second audio signal can be obtained from the HDMI output port of the audio and video source output device.

S102: Detect whether the audio and video playback device plays the second audio signal according to the audio signal correlation and the phase difference between the first audio signal and the second audio signal. Specifically, the determination method may include: if the audio signal correlation of the first audio signal and the second audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than a preset phase If the difference threshold is exceeded, it is determined that the audio and video playback device plays the second audio signal; otherwise, it is determined that the audio and video playback device does not play the second audio signal.

Specifically, the above-mentioned correlation between audio signals can be determined by extracting frequency domain features of the audio signals, and then performing frequency domain analysis and comparison. The phase difference between the audio signals can be determined based on the components of the first audio signal including the second audio signal, and the feature points of the part with high correlation between the first audio signal and the second audio signal can be found. feature points to calculate the phase difference, such as the phase difference between the peak points of an audio signal.

Further, when the audio and video playback device does not play the second audio signal, the following specific situations may exist:

If the audio signal correlation between the first audio signal and the second audio signal is less than the preset correlation threshold, it means that the first audio signal does not contain components related to the second audio signal, and of course it also means that the audio and video playback device does not play the second audio signal;

If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, however, the phase difference between the first audio signal and the second audio signal is smaller than the preset phase difference threshold. This shows that the first audio signal collected from the sound pickup module of the audio and video source output device contains the relevant components of the second audio signal, but the relevant components are not emitted by the audio and video playback device, but by the audio and video signals. The source output device's own speaker emits, so the phase difference is small.

If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, and the first audio signal includes sub audio signals and sub audio signals with different phase differences, one of the sub audio signals and the second audio signal The phase difference between the signals is greater than the preset phase difference threshold, and the phase difference between the other sub-audio signal and the second audio signal is less than the phase difference threshold, it means that both the audio and video playback device and the audio and video source output device are playing the And the audio signal.

S103: If the audio and video playback device does not play the second audio signal, play the second audio signal through the speaker of the audio and video source output device, and if the audio and video playback device plays the second audio signal, it means that the audio and video playback device is in a normal state state, no switching is required.

In addition, for the situation where it is detected that the audio and video playback device is not playing the second audio signal, but the speaker of the audio and video source output device is playing the second audio signal, it means that the audio and video playback device has not returned to normal, then the audio and video source output device is used. The loudspeaker continues to play the second audio signal, so no processing is required in this case either. In the case where the speakers of the audio and video playback device and the audio and video source output device are both playing the second audio signal, the speaker of the audio and video source output device should stop playing the second audio signal.

The above-mentioned sound playback detection processing can be repeated for multiple rounds at a certain time interval, so as to continuously adjust the device for playing the second audio signal according to the state of the audio and video playback device, which can ensure that the user can always hear the audio and video source output. The audio content output by the device is not affected by the state of the audio and video playback device.

Embodiment 2

As shown in FIG. 5 , which is a schematic flowchart of a method for detecting a power-on state of an audio and video playback device according to an embodiment of the present invention, the method can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module for collecting audio signals, and the audio and video source output device and audio and video playback devices such as TV equipment are connected by wired or wireless means, so as to perform audio and video signal acquisition. transmission. In the application scenario of this embodiment, when the audio and video source output device needs to transmit the audio and video playback content to the audio and video playback device, it is hoped to first confirm whether the audio and video playback device is turned on, and then send the audio and video signals, thereby Avoid the TV device is not powered on or is in the process of booting, which will cause the audio and video content transmitted by the audio and video source output device to be unable to be played for a period of time, thus affecting the user experience. Specifically, the method includes:

S201: Send a detection audio signal to an audio and video playback device, and obtain a first audio signal detected by a sound pickup module of the audio and video source output device. The detected audio signal can be a simple piece of audio content, such as "an exciting content is about to be played for you". If the audio and video playback device is in a normal power-on state, the audio should be played soon, and the audio and video source output device can pass the pickup. The audio module collects the first audio signal including the audio segment. In addition, the audio and video source output device can also control the audio and video playback device through a control protocol. For example, the audio and video source output device can control the audio and video playback device on and off through control instructions. The audio and video source output device may also have the ability to detect the user's voice command through the pickup module to trigger the audio and video playback processing capability. Therefore, in some application scenarios, the user may trigger the audio and video source output device to initiate audio and video playback through a voice command, while the audio and video playback device is still in a shutdown state. The audio signal may also include an audio and video source output device sending a power-on command.

S202: Detect whether the audio-video playback device plays the detected audio signal according to the correlation degree and phase difference of the audio signal between the first audio signal and the detected audio signal. The specific detection and determination method is the same as the detection principle in the sound playback detection processing method described above, that is, if the correlation between the first audio signal and the audio signal of the detected audio signal is greater than the preset correlation threshold, and the first audio signal If the phase difference between the audio signal and the detected audio signal is greater than the preset phase difference threshold, it is determined that the audio and video playback device has played the detected audio signal; otherwise, it is determined that the audio and video playback device has not played the detected audio signal.

S203: If the audio and video playback device plays the detected audio signal, output the audio and/or video content signal to the audio and video playback device. If the TV device does not play the detection audio signal, send the detection audio signal to the TV device again, and perform a new round of processing to detect whether the TV device plays the detection audio signal, until the predetermined round is detected or the audio and video playback is detected. The device played a detection audio signal. In addition, the reason why the audio and video playback device does not play the detected audio signal may be because the audio and video playback device is not powered on. Issue a boot command to trigger it to perform boot processing.

The method for detecting and processing the power-on state of an audio and video playback device according to the embodiment of the present invention sends a detection audio signal to the audio and video playback device, and then uses a sound pickup module to perform sound retrieval, and judges the retrieved audio signal according to the audio signal correlation and phase difference. Whether the detection audio signal is included in it, so as to realize the detection of the power-on state of the audio and video playback device. In this way, it can be avoided that the user misses the audio and video playback content for a period of time because the audio and video playback device is not powered on or is in the process of booting.

Embodiment 3

As shown in FIG. 6 , which is a schematic flowchart of an echo cancellation processing method according to an embodiment of the present invention, the method can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module for collecting audio signals, and can receive user's voice commands through the pickup module. The audio and video source output device and audio and video playback devices such as TV equipment are connected by cable Or connect wirelessly to transmit audio and video signals. In some scenarios, when the user sends a voice command to the audio and video source output device, the audio and video playback device may be playing audio and video content, and these sounds may interfere with the recognition of the voice command. The echo cancellation processing method provided by the embodiment of the present invention is as follows: to eliminate these disturbances. Specifically, the method includes:

S301: Acquire a first audio signal detected by a sound pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device. The second audio signal corresponds to the audio content sent by the audio and video source output device to the audio and video playback device, that is, the audio content that needs to be played by the speaker of the audio and video playback device, and the first audio signal is transmitted through the audio and video source output device. The audio signal collected by the sound pickup module includes the second audio signal played by the speaker of the audio and video playback device after a delay. When the user sends out a voice command, it may also include a voice command sent by the user. In addition, in some cases, the above-mentioned second audio signal may be played through the speaker of the audio and video source output device itself, so that the first audio signal includes the first audio signal played by the speaker of the audio and video source output device after a delay. The second audio signal, when the user sends a voice command, may also include the voice command sent by the user. The difference between the two cases is that the delay in playing the second audio signal will be different.

S302 : Compare the signal features of the first audio signal and the second audio signal, and obtain the part of the audio signal whose correlation degree of the audio signal is greater than a preset correlation degree threshold. If the audio signal correlation is greater than the preset correlation threshold, it indicates that the first audio signal contains the content of the second audio signal. The audio signal correlation between the above audio signals can be determined by extracting frequency domain features of the audio signals, and then performing frequency domain analysis and comparison.

S303: Determine the phase difference between the relevant audio signal part of the first audio signal and the relevant audio signal part of the second audio signal. The above-mentioned phase difference between the audio signals can be based on determining that the first audio signal contains the components of the second audio signal, and find out the characteristics of the audio signal part with a higher degree of correlation in the first audio signal and the second audio signal. From these feature points, the phase difference, such as the phase difference between the peak points of the relevant audio signal parts, is calculated.

S304: Remove the second audio signal after the phase difference has been eliminated from the first audio signal. The process of eliminating the phase difference mentioned here may specifically be aligning the signals on the time axis according to the determined phase difference. For example, the determined phase difference is 100ms, that is, the correlation components in the first audio signal are higher than those in the second audio signal. If the delay is 100ms, the current first audio signal can be aligned with the second audio signal 100ms ago and then subtracted, so that the component corresponding to the second audio signal can be eliminated.

The above-mentioned process of determining the phase difference can be performed at a preset time interval, that is, the processes of steps S302 and S303 are performed at a preset time interval. Correspondingly, within the time interval, in step S304, the newly determined phase difference , the second audio signal after the phase difference has been eliminated is removed from the first audio signal.

In addition, determining the phase difference between the audio signals also determines the delay between the audio signals, so that it can be further determined that the component of the second audio signal contained in the first audio signal is played by the speaker of the audio and video playback device. However, in the scenario of the embodiment of the present invention, as long as the phase difference is determined, the echo cancellation processing can be performed, and there is no need to distinguish whether the currently playing audio is the audio and video playback device or the audio and video playback device. Video source output device.

In the echo cancellation processing method of the embodiment of the present invention, the phase difference is calculated by the first audio signal recovered by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device, and then, using The phase difference is used for echo cancellation, so that the influence of the second audio signal output by the output device of the audio and video source can be shielded, so that relatively pure voice commands can be extracted, and then accurate voice control processing can be performed.

Embodiment 4

As shown in FIG. 7 , which is a schematic structural diagram of a sound playback detection and processing device according to an embodiment of the present invention, the device can be applied to an audio and video source output device such as a TV box, and the audio and video source output device itself is used for collecting audio signals. The audio and video source output device and audio and video playback devices such as TV devices are connected in a wired or wireless manner, so as to transmit audio and video signals. Specifically, the device includes:

The audio signal acquisition module 11 is configured to acquire the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device. The first audio signal is an audio signal generated by collecting sounds in the environment through the sound pickup module of the audio and video source output device, which may include audio signals sent by the audio and video playback device, or may include audio signals sent by other devices in the environment. It may also include audio signals from the speakers of the audio and video source output device itself. The second audio signal is the audio content that the audio and video source output device sends to the audio and video playback device and is expected to be played by the audio and video playback device. Compared with the first audio signal, the second audio signal is a definite signal. When the audio and video source output device is a TV box, the second audio signal can be obtained from the HDMI output port of the audio and video source output device.

The playback detection module 12 is configured to detect whether the audio and video playback device plays the second audio signal according to the audio signal correlation and the phase difference between the first audio signal and the second audio signal. Specifically, the determination method may include: if the audio signal correlation of the first audio signal and the second audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than a preset phase If the difference threshold is exceeded, it is determined that the audio and video playback device plays the second audio signal; otherwise, it is determined that the audio and video playback device does not play the second audio signal.

Specifically, the above-mentioned correlation between audio signals can be determined by extracting frequency domain features of the audio signals, and then performing frequency domain analysis and comparison. The phase difference between the audio signals can be determined based on the components of the first audio signal including the second audio signal, and the feature points of the part with high correlation between the first audio signal and the second audio signal can be found. feature points to calculate the phase difference, such as the phase difference between the peak points of an audio signal.

Further, when the audio and video playback device does not play the second audio signal, the following specific situations may exist:

If the audio signal correlation between the first audio signal and the second audio signal is less than the preset correlation threshold, it means that the first audio signal does not contain components related to the second audio signal, and of course it also means that the audio and video playback device does not play the second audio signal;

If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, however, the phase difference between the first audio signal and the second audio signal is smaller than the preset phase difference threshold. This shows that the first audio signal collected from the sound pickup module of the audio and video source output device contains the relevant components of the second audio signal, but the relevant components are not emitted by the audio and video playback device, but by the audio and video signals. The source output device's own speaker emits, so the phase difference is small.

If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, and the first audio signal includes sub audio signals and sub audio signals with different phase differences, one of the sub audio signals and the second audio signal The phase difference between the signals is greater than the preset phase difference threshold, and the phase difference between the other sub-audio signal and the second audio signal is less than the phase difference threshold, it means that both the audio and video playback device and the audio and video source output device are playing the And the audio signal.

The audio playback switching processing module 13 is used to play the second audio signal through the speaker of the audio and video source output device when it is detected that the audio and video playback device does not play the second audio signal, and when it is detected that the audio and video playback device has played the second audio signal. The second audio signal indicates that the audio and video playback device is in a normal state and does not need to be switched.

In addition, for the situation where it is detected that the audio and video playback device is not playing the second audio signal, but the speaker of the audio and video source output device is playing the second audio signal, it means that the audio and video playback device has not returned to normal, then the audio and video source output device is used. The loudspeaker continues to play the second audio signal, so no processing is required in this case either. In the case where the speakers of the audio and video playback device and the audio and video source output device are both playing the second audio signal, the speaker of the audio and video source output device should stop playing the second audio signal.

The above-mentioned sound playback detection processing can be repeated for multiple rounds at a certain time interval, so as to continuously adjust the device for playing the second audio signal according to the state of the audio and video playback device, which can ensure that the user can always hear the audio and video source output. The audio content output by the device is not affected by the state of the audio and video playback device.

Embodiment 5

As shown in FIG. 8 , it is a schematic structural diagram of a power-on state detection and processing device of an audio and video playback device according to an embodiment of the present invention, and the device can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module for collecting audio signals, and the audio and video source output device and audio and video playback devices such as TV equipment are connected by wired or wireless means, so as to perform audio and video signal acquisition. transmission. In the application scenario of this embodiment, when the audio and video source output device needs to transmit the audio and video playback content to the audio and video playback device, it is hoped to first confirm whether the audio and video playback device is turned on, and then send the audio and video signals, thereby Avoid the TV device is not powered on or is in the process of booting, which will cause the audio and video content transmitted by the audio and video source output device to be unable to be played for a period of time, thus affecting the user experience. Specifically, the device includes:

The detection signal sending module 21 is used for sending the detection audio signal to the audio and video playback device. The detected audio signal can be a simple piece of audio content, such as "an exciting content is about to be played for you". If the audio and video playback device is in a normal power-on state, the audio should be played soon, and the audio and video source output device can pass the pickup. The audio module collects the first audio signal including the audio segment. In addition, the audio and video source output device can also control the audio and video playback device through a control protocol. For example, the audio and video source output device can control the audio and video playback device on and off through control instructions. The audio and video source output device may also have the ability to detect the user's voice command through the pickup module to trigger the audio and video playback processing capability. Therefore, in some application scenarios, the user may trigger the audio and video source output device to initiate audio and video playback through a voice command, while the audio and video playback device is still in a shutdown state. Before sending the detection audio signal, the video playback device may also include the audio and video source output device sending a power-on instruction to the device.

The audio signal acquisition module 22 is configured to acquire the first audio signal detected by the sound pickup module of the audio and video source output device. The first audio signal is an audio signal generated by collecting the sound in the environment through the sound pickup module of the audio and video source output device, which may include the audio signal sent by the audio and video playback device, and may also include the audio signal sent by other devices in the environment, It may also include audio signals from the audio and video source output device's own speakers.

The playback detection module 23 is configured to detect whether the audio and video playback device plays the detected audio signal according to the correlation and phase difference between the first audio signal and the audio signal of the detected audio signal. The specific detection and determination method is the same as the sound playback detection processing method and the detection principle in the device described above, that is, if the audio signal correlation between the first audio signal and the detected audio signal is greater than the preset correlation threshold, and If the phase difference between the first audio signal and the detected audio signal is greater than the preset phase difference threshold, it is determined that the audio and video playback device has played the detected audio signal, otherwise it is determined that the audio and video playback device has not played the detected audio signal.

The playback processing module 24 is configured to output audio and/or video content signals to the audio and video playback device when it is detected that the audio and video playback device plays the detected audio signal. In addition, if the television equipment does not play the detection audio signal, the detection audio signal is sent to the television equipment again, and the power-on state detection processing device is triggered to perform a new round of processing of detecting whether the television equipment plays the detection audio signal until a predetermined amount of time is detected. round or detected that the audio and video playback device plays the detected audio signal. In addition, the reason why the audio and video playback device does not play the detected audio signal may be because the audio and video playback device is not powered on. Issue a boot command to trigger it to perform boot processing.

The power-on state detection and processing device of an audio and video playback device according to the embodiment of the present invention sends a detection audio signal to the audio and video playback device, and then uses a sound pickup module to perform sound retrieval, and judges the retrieved audio signal according to the audio signal correlation and phase difference. Whether the detection audio signal is included in it, so as to realize the detection of the power-on state of the audio and video playback device. In this way, it can be avoided that the user misses the audio and video playback content for a period of time because the audio and video playback device is not powered on or is in the process of booting.

Embodiment 6

As shown in FIG. 9 , which is a schematic structural diagram of an echo cancellation processing apparatus according to an embodiment of the present invention, the apparatus can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module for collecting audio signals, and can receive user's voice commands through the pickup module. The audio and video source output device and audio and video playback devices such as TV equipment are connected by cable Or connect wirelessly to transmit audio and video signals. In some scenarios, when a user sends a voice command to the audio and video source output device, the audio and video playback device may be playing audio and video content, and these sounds may interfere with the recognition of the voice command. The echo cancellation processing device provided by the embodiment of the present invention is to eliminate these disturbances. Specifically, the device includes:

The audio signal acquisition module 31 is configured to acquire the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device. The second audio signal corresponds to the audio content sent by the audio and video source output device to the audio and video playback device, that is, the audio content that needs to be played by the speaker of the audio and video playback device, and the first audio signal is transmitted through the audio and video source output device. The audio signal collected by the sound pickup module includes the second audio signal played by the speaker of the audio and video playback device after a delay. When the user sends out a voice command, it may also include a voice command sent by the user. In addition, in some cases, the above-mentioned second audio signal may be played through the speaker of the audio and video source output device itself, so that the first audio signal includes the first audio signal played by the speaker of the audio and video source output device after a delay. The second audio signal, when the user sends a voice command, may also include the voice command sent by the user. The difference between the two cases is that the delay in playing the second audio signal will be different.

The correlation processing module 32 is configured to compare the signal features of the first audio signal and the second audio signal, and obtain the relevant audio signal parts whose audio signal correlation is greater than a preset correlation threshold. If the audio signal correlation is greater than the preset correlation threshold, it indicates that the first audio signal contains the content of the second audio signal. The audio signal correlation between the above audio signals can be determined by extracting frequency domain features of the audio signals, and then performing frequency domain analysis and comparison.

The phase difference determination module 33 is configured to determine the phase difference between the relevant audio signal part of the first audio signal and the relevant audio signal part of the second audio signal. The above-mentioned phase difference between the audio signals can be based on determining that the first audio signal contains the components of the second audio signal, and find out the characteristics of the audio signal part with a higher degree of correlation in the first audio signal and the second audio signal. From these feature points, the phase difference, such as the phase difference between the peak points of the relevant audio signal parts, is calculated.

The echo cancellation module 34 is configured to remove the second audio signal after the phase difference has been eliminated from the first audio signal. The process of eliminating the phase difference mentioned here may specifically be aligning the signals on the time axis according to the determined phase difference. For example, the determined phase difference is 100ms, that is, the correlation components in the first audio signal are higher than those in the second audio signal. If the delay is 100ms, the current first audio signal can be aligned with the second audio signal 100ms ago and then subtracted, so that the component corresponding to the second audio signal can be eliminated.

The above-mentioned process of determining the phase difference can be performed at preset time intervals, that is, the processing performed by the correlation processing module 32 and the phase difference determination module 33 is performed at preset time intervals. The cancellation module 34 performs a process of removing the phase difference-eliminated second audio signal from the first audio signal with the newly determined phase difference.

In addition, determining the phase difference between the audio signals also determines the delay between the audio signals, so that it can be further determined that the component of the second audio signal contained in the first audio signal is played by the speaker of the audio and video playback device. However, in the scenario of the embodiment of the present invention, as long as the phase difference is determined, the echo cancellation processing can be performed, and there is no need to distinguish whether the currently playing audio is the audio and video playback device or the audio and video playback device. Video source output device.

The echo cancellation processing apparatus of the embodiment of the present invention calculates the phase difference by using the first audio signal recovered by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device, and then using The phase difference is used for echo cancellation, so that the influence of the second audio signal output by the output device of the audio and video source can be shielded, so that relatively pure voice commands can be extracted, and then accurate voice control processing can be performed.

Embodiment 7

This embodiment relates to the application of the technical solutions of the embodiments of the present invention in some scenarios to realize the cooperative playback of audio and video. For example, in some multi-person conferences or education and training scenarios, video images are often played through projection devices. Some projection devices may have their own audio playback function, while some projection devices may only have image playback functions. The projection equipment with the same function may also be unable to play audio normally due to signal transmission failure or equipment settings.

In view of this situation, this embodiment provides a projection playback processing method, which can be applied to an audio and video source output device such as a TV box, and the audio and video source output device itself has a pickup module for collecting audio signals (for example, Microphone), the audio and video source output device and the projection device are connected in a wired or wireless manner, so as to transmit audio and video signals. Specifically, the method includes: an audio and video source output device outputs an audio and video signal to the projection device; detecting whether the projection device plays an audio signal in the audio and video signal through a sound pickup module, if the detection determines that the projection device does not play an audio signal, The speaker of the video source output device plays the audio signal. If the detection determines that the projection device plays the audio signal normally, the projection device is used preferentially to play the audio signal.

For the detection method of whether the projection device plays audio signals, the method described in the previous embodiment can be adopted. The audio signals in the environment are collected by the sound pickup module, and then the correlation and phase difference between the signals are analyzed and judged to determine whether the projection device plays the audio signal. The audio output from the audio and video source output device is displayed.

In addition, in the remote conference scenario, on the one hand, the shared processing mechanism based on the cloud platform can realize remote image display on the same screen. On the other hand, the audio and video signals at the conference site can also be transmitted to the equipment in the remote conference site through the network. superior. The requirements of the remote conference venue are: the video played on the same screen is highlighted, and the content of the audio played is the audio signal transmitted by the conference site, and it is hoped that the two can be played together as much as possible.

To this end, an embodiment of the present invention provides a remote conference collaborative playback processing method, which can be applied to an audio and video source output device such as a TV box, and the audio and video source output device itself has a pickup module for collecting audio signals. (eg microphone), the audio and video source output device and the projection device or large-scale display device in the remote meeting place are connected by wire or wireless, so as to transmit image signals and audio signals. include:

S401: Acquire the same-screen image signal of the remote conference and the audio and video signals of the conference site. In the remote conference venue, the video signal on the same screen at the conference site and the audio and video signals at the conference site can be received through the network through the audio and video source output device such as the TV box. Among them, the same-screen image signal can come from the cloud platform, that is, the meeting host at the meeting site can share the plane based on the office system of the cloud platform, so as to generate the same-screen image signal on the cloud platform and transmit it to the audio and video source of the remote meeting place. in the output device. On the other hand, audio and video recording equipment can be set up at the conference site, and the audio and video signals can be transmitted to the audio and video source output equipment of the remote conference site synchronously.

S402: Compare the video signal in the audio-video signal with the image signal on the same screen, and determine the audio signal corresponding to the image signal on the same screen. In the audio and video source output device of the remote conference venue, the synchronization between the two signals can be determined by comparing and analyzing the video signal captured at the conference site and the image signal on the same screen. The signals are corresponding. After the synchronization between the video signal and the image signal on the same screen is determined, the synchronization state of the audio signal is also indirectly determined. That is, in this embodiment, the main function of the video signal captured on site is to determine Correspondence between the audio signal and the image signal on the same screen.

S403: Synchronously play the same-screen image signal and the corresponding audio signal. Based on the corresponding relationship between the same-screen image signal and the audio signal determined in the previous steps, the audio signal and the same-screen image signal are cooperatively played.

In addition, an embodiment of the present invention also provides an audio and video source output device, which may be specifically a TV box or other device. Specifically, the device includes:

The audio and video signal output module is used to output audio and video signals to the audio and video playback device. Specifically, the audio and video signal output module can transmit audio and video signals to audio and video playback devices such as smart TVs and projectors by wire (eg, HDMI signal line) or wirelessly (local area network).

The audio signal playing module is used for playing the audio signal output to the audio and video playing device according to the instruction of the audio playing detection module. The audio signal playback module may be a sound card and a speaker module set by the audio and video source output device itself.

The audio playback detection module is used for detecting whether the audio playback device plays the audio signal, and if it is determined that the audio playback device does not play the audio signal, the audio signal playback module is triggered to play the audio signal. The specific detection method can adopt the method introduced in the foregoing embodiment, the audio signal in the environment is collected through the sound pickup module, and then the correlation and phase difference between the signals are analyzed and judged to determine whether the audio and video playback device has played the audio and video source. The audio output by the output device.

In addition, in the case that both the audio and video playback device and the audio and video source output device can perform audio playback, the user can select through a voice command. And according to the voice command, the audio signal playback switching process is performed between the audio and video playback device and the audio signal playback module.

Embodiment 8

The previous embodiment describes the process of playback detection, power-on, echo cancellation processing, projection playback processing, and remote conference collaborative playback processing methods and the corresponding device structure. As shown in FIG. 10 , it is a schematic structural diagram of an electronic device according to an embodiment of the present invention, which specifically includes: a memory 110 and a processor 120 .

The memory 110 is used to store programs.

In addition to the above-described programs, the memory 110 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

Memory 110 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

The processor 120, coupled to the memory 110, is configured to execute the program in the memory 110 to perform one or any of the playback detection, power-on, echo cancellation processing, projection playback processing, and teleconference collaborative playback processing methods described in the foregoing embodiments. Operation steps for multiple methods.

In addition, the processor 120 may also include various modules described in the foregoing embodiments to perform one or any of the processing methods of playback detection, power-on, and echo cancellation, and the memory 110 may be used, for example, to store these modules to perform operations. required data and/or output data.

The specific description of the above-mentioned processing process, the detailed description of the technical principle, and the detailed analysis of the technical effect have been described in detail in the foregoing embodiments, and will not be repeated here.

Further, as shown in the figure, the electronic device may further include: a communication component 130 , a power supply component 140 , an audio component 150 , a display 160 and other components. Only some components are schematically shown in the figure, which does not mean that the electronic device only includes the components shown in the figure.

Communication component 130 is configured to facilitate wired or wireless communications between electronic devices and other devices. Electronic devices can access wireless networks based on communication standards, such as WiFi, mobile communication networks such as 2G, 3G, 4G/LTE, and 5G, or a combination thereof. In one exemplary embodiment, the communication component 130 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 130 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

The power supply assembly 140 provides power for various components of the electronic device. Power components 140 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic devices.

Audio component 150 is configured to output and/or input audio signals. For example, audio component 150 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device is in operating modes, such as calling mode, recording mode, and voice recognition mode. The received audio signal may be further stored in the memory 110 or transmitted via the communication component 130 . In some embodiments, the audio component 150 also includes a speaker for outputting audio signals.

The display 160 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. A touch sensor can sense not only the boundaries of a touch or swipe action, but also the duration and pressure associated with the touch or swipe action.

In addition, an embodiment of the present invention also provides a computer program product, including a computer program or an instruction, characterized in that, when the computer program or instruction is executed by the processor, the processor is caused to implement the aforementioned sound playback detection processing method, audio and video One or any of a plurality of methods for detecting and processing a power-on state of a playback device, an echo cancellation processing method, a projection playback processing method, and a remote conference collaborative playback processing method.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by program instructions related to hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps including the above method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention. scope.

Claims (21)

1. A sound playback detection and processing method, comprising: Obtain the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device; Detecting whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference of the first audio signal and the second audio signal; If the audio and video playback device does not play the second audio signal, the second audio signal is played through the speaker of the audio and video source output device. 2. The method according to claim 1, wherein, according to the audio signal correlation degree and phase difference of the first audio signal and the second audio signal, it is determined whether the audio and video playback device normally plays the second audio signal. Audio signals include: If the audio signal correlation of the first audio signal and the second audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than a preset If the phase difference threshold is determined, it is determined that the audio and video playback device plays the second audio signal; otherwise, it is determined that the audio and video playback device does not play the second audio signal. 3. The method according to claim 1, wherein, according to the audio signal correlation degree and the phase difference of the first audio signal and the second audio signal, it is determined whether the audio and video playback device normally plays the second audio signal. Audio signals include: If the audio signal correlation of the first audio signal and the second audio signal is greater than a preset correlation threshold, and the first audio signal includes sub audio signals and sub audio signals with different phase differences, one of the sub audio signals The phase difference between the audio signal and the second audio signal is greater than a preset phase difference threshold, and the phase difference between another sub-audio signal and the second audio signal is less than the phase difference threshold, then it is determined that the The speakers of both the audio and video playback device and the audio and video source output device are playing the second audio signal, and the speakers of the audio and video source output device are stopped from playing the second audio signal. 4. The method according to claim 1, wherein, according to the audio signal correlation degree and the phase difference of the first audio signal and the second audio signal, it is determined whether the audio and video playback device normally plays the second audio signal. Audio signals include: If the audio signal correlation of the first audio signal and the second audio signal is greater than a preset correlation threshold, but the phase difference between the first audio signal and the second audio signal is less than a preset The phase difference threshold is determined as the audio and video playback device is not playing the second audio signal, and the speaker of the audio and video source output device is playing the second audio signal. 5. The method according to claim 1, wherein, the audio and video source output device is a TV box, the audio and video playback device is a TV device, and the first audio and video source output device sent by the audio and video source output device to the audio and video playback device is obtained. The second audio signal includes: obtaining the second audio signal from the HDMI output port of the audio and video source output device. 6. A method for detecting and processing a power-on state of an audio and video playback device, comprising: Send a detection audio signal to the audio and video playback device, and obtain the first audio signal detected by the sound pickup module of the audio and video source output device; Detecting whether the audio and video playback device plays the detected audio signal according to the audio signal correlation and phase difference between the first audio signal and the detected audio signal; If the audio and video playback device plays the detected audio signal, output the audio and/or video content signal to the audio and video playback device. 7. The method according to claim 6, wherein, if the audio-video playback device does not play the detection audio signal, then the detection audio signal sent to the audio-video playback device again, and carries out a new round of detection of the audio-video signal Whether the playback device plays the process of detecting the audio signal. 8. The method according to claim 7, further comprising: if the audio and video playback device does not play the detected audio signal, sending a power-on instruction to the audio and video playback device. 9. The method according to claim 6, wherein, according to the audio signal correlation degree and phase difference of the first audio signal and the detected audio signal, detecting whether the audio and video playback device plays the detected audio signal comprises: : If the audio signal correlation between the first audio signal and the detected audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the detected audio signal is greater than the preset phase difference If the threshold is set, it is determined that the audio and video playback device plays the detected audio signal; otherwise, it is determined that the audio and video playback device does not play the detected audio signal. 10. An echo cancellation processing method, comprising: Obtain the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device; Comparing the signal characteristics of the first audio signal and the second audio signal, and obtaining the part of the audio signal whose correlation is greater than a preset correlation threshold; determining a phase difference between the associated audio signal portion of the first audio signal and the associated audio signal portion of the second audio signal; The phase difference-eliminated second audio signal is removed from the first audio signal. 11. The method according to claim 10, wherein the signal feature comparison of the first audio signal and the second audio signal is performed at a preset time interval, and the obtained audio signal correlation degree is greater than a preset time interval. a process of determining the phase difference between the correlated audio signal portion of the correlation threshold and the correlated audio signal portion of the first audio signal and the correlated audio signal portion of the second audio signal, Removing the phase difference-eliminated second audio signal from the first audio signal includes: in the preset time interval, removing the phase difference from the first audio signal with the newly determined phase difference The second audio signal after eliminating the phase difference. 12. A sound playback detection and processing device, comprising: an audio signal acquisition module for acquiring the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device; A playback detection module, configured to detect whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference of the first audio signal and the second audio signal; An audio playback switching processing module, configured to play the second audio signal through the speaker of the audio and video source output device when it is detected that the audio and video playback device does not play the second audio signal. 13. The apparatus according to claim 12, wherein, according to the audio signal correlation degree and the phase difference of the first audio signal and the second audio signal, it is determined whether the audio and video playback device normally plays the second audio signal. Audio signals include: If the audio signal correlation of the first audio signal and the second audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than a preset If the phase difference threshold is determined, it is determined that the audio and video playback device plays the second audio signal; otherwise, it is determined that the audio and video playback device does not play the second audio signal. 14. A power-on state detection and processing device for an audio and video playback device, comprising: The detection signal sending module is used to send the detection audio signal to the audio and video playback device; an audio signal acquisition module for acquiring the first audio signal detected by the sound pickup module of the audio and video source output device; A playback detection module, configured to detect whether the audio and video playback device plays the detected audio signal according to the audio signal correlation and phase difference of the first audio signal and the detected audio signal; A playback processing module, configured to output audio and/or video content signals to the audio and video playback device when it is detected that the audio and video playback device plays the detected audio signal. 15. An echo cancellation processing device, comprising: an audio signal acquisition module for acquiring the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device; a correlation processing module, configured to compare the signal characteristics of the first audio signal and the second audio signal, and obtain the part of the audio signal whose correlation is greater than a preset correlation threshold; a phase difference determination module for determining a phase difference between a relevant audio signal portion of the first audio signal and a relevant audio signal portion of the second audio signal; The echo cancellation module is configured to remove the second audio signal after the phase difference has been eliminated from the first audio signal. 16. A projection playback processing method, comprising: Output audio and video signals to projection equipment; The sound pickup module detects whether the projection device plays the audio signal in the audio and video signal, and if it is determined that the projection device does not play the audio signal, the audio signal is played through the speaker of the audio and video source output device. 17. A remote conference collaborative playback processing method, comprising: Obtain the same-screen image signal of the remote conference and the audio and video signals of the conference site; Compare the video signal in the audio-video signal with the image signal on the same screen, and determine the audio signal corresponding to the image signal on the same screen; The same-screen image signal and the corresponding audio signal are played synchronously. 18. An audio and video source output device, comprising: The audio and video signal output module is used to output audio and video signals to the audio and video playback device; The audio signal playing module is used for playing the audio signal output to the audio and video playing device according to the instruction of the audio playing detection module; The audio playback detection module is used to detect whether the audio playback device plays the audio signal, and if it is determined that the audio playback device does not play the audio signal, trigger the audio signal playback module to play the audio Signal. 19. The apparatus of claim 18, further comprising: The switching control module is used for receiving a user's voice command, and according to the voice command, performs switching processing of audio signal playback between the audio and video playback device and the audio signal playback module. 20. An electronic device comprising: memory for storing programs; A processor, for running the program stored in the memory, to execute the sound playback detection processing method described in any one of claims 1 to 5, and the booting of the audio and video playback device described in any one of claims 6 to 9 Any one or more of the state detection processing method, the echo cancellation processing method of claim 10 or 11, the projection playback processing method of claim 16, and the teleconference collaborative playback processing method of claim 17. 21. A computer program product, comprising a computer program or an instruction, characterized in that, when the computer program or instruction is executed by a processor, the processor is caused to implement the sound playback detection according to any one of claims 1 to 5 The processing method, the power-on state detection processing method of the audio and video playback device according to any one of claims 6 to 9, the echo cancellation processing method according to claim 10 or 11, the projection playback processing method according to claim 16, and the processing method according to claim 17. Any one or more of the remote conference collaborative playback processing methods.

Images