TW202325008A - Method for automatically switching on/off sound receiving channel of video conference and electronic device - Google Patents

Abstract

A method for automatically switching on/off sound receiving channel of video conference and an electronic device are provided. The method for automatically switching on/off sound receiving channel of video conference includes the following steps. First, a volume information of an electronic is detected. If the volume information is greater than a threshold condition, a video image of a participant participating in the video conference of the electronic device is received. Then, whether the participant is in a speaking state is determined according to the video image. If the participant is not in the speaking state, a sound receiving channel of the video conference is switched off.

Description

Automatic switching method and electronic device for video conferencing radio

The present invention relates to an automatic switching method and an electronic device, and in particular to an automatic switching method and an electronic device for video conferencing radio.

In response to the demand for remote work, the usage rate of many online conference software is increasing. Employees of the company can conduct work discussions through online meetings to reduce the chance of face-to-face contact.

Considering the impact of the surrounding environment of the participants on the quality of the meeting, the participants often need to switch the microphone of the conference to mute or radio mode. However, it often happens that participants forget to switch the microphone to radio mode when they want to speak, resulting in the situation that other participants cannot hear the sound; Go through the meeting and be heard by other participants.

The invention relates to an automatic switching method and an electronic device for video conference audio, which can judge whether the participant is currently speaking through the video image of the participant, and then automatically turn on/off the audio channel of the video conference. In addition, the present invention further assists in confirming whether the participant speaks at present through double judging steps.

According to one aspect of the present invention, a method for automatically switching audio in a video conference is proposed. The method for automatic switching of video conference audio includes the following steps. Firstly, the volume information of the electronic device is detected. If the volume information is greater than the threshold condition, the video images of the participants of the video conference of the electronic device are received. Then, it is determined whether the participant is speaking according to the video image. If the participant is not speaking, the audio channel of the video conference is turned off.

According to another aspect of the present invention, an electronic device is provided. The electronic device includes a radio unit, a processing unit and a camera unit. The radio unit is used for detecting volume information. The processing unit is used for performing video conferencing and receiving volume information. The camera unit is used for obtaining video images of participants in the video conference. If the processing unit judges that the volume information is greater than the threshold condition, it receives the video image, and judges whether the participant is speaking according to the video image. If the processing unit judges that the participant is not speaking, then close the audio channel of the video conference.

In order to have a better understanding of the above-mentioned and other aspects of the present invention, the following specific examples are given in detail with the accompanying drawings as follows:

Various embodiments of the present invention will be described in detail below and illustrated with accompanying drawings. In addition to these detailed descriptions, the present invention can also be widely implemented in other embodiments, and any easy replacement, modification, and equivalent changes of any of the described embodiments are included in the scope of the present invention, and are defined in the following patent scope prevail. In the description of the specification, many specific details and implementation examples are provided in order to enable readers to have a more complete understanding of the present invention; however, these specific details and implementation examples should not be regarded as limitations of the present invention. Also, well-known steps or elements have not been described in detail in order to avoid unnecessarily limiting the invention.

Please refer to FIG. 1 , which is a block diagram illustrating an electronic device 100 according to an embodiment of the present invention. The electronic device 100 is, for example, a notebook computer, a desktop computer, a tablet computer or a smart phone. The electronic device 100 includes a sound receiving unit 110 , a camera unit 120 , a display unit 130 and a processing unit 140 . The sound receiving unit 110 is a sound receiving element installed on the electronic device 100 for receiving various sounds to detect volume information, such as a microphone. The camera unit 120 is used for shooting various images, such as a camera. The display unit 130 is used to display various information, such as a display panel. The processing unit 140 is used to execute various processing procedures, such as a circuit board, a chip, a circuit, a computer program product, or a computer-readable recording medium. The processing unit 140 includes a judgment module 141 , an image processing module 142 , an identification module 143 and a radio channel switching module 144 . The judging module 141 is used for receiving the volume information of the sound receiving unit 110 and performing a judging process. The image processing module 142 is used for performing image processing on the image of the camera unit 120 . The recognition module 143 is used for performing image recognition on the processed image by using an artificial intelligence algorithm. The audio channel switching module 144 is used for opening or closing the audio channel of the video conference according to the identification result of the identification module 143 .

FIG. 2 is a flow chart illustrating a method for automatically switching audio in a video conference according to an embodiment of the present invention. Referring to FIG. 1 and FIG. 2 , in step S110 , the electronic device 100 executes a video conference. The user can, for example, click on a program of the electronic device 100 to make the processing unit 140 execute the video conference and display the video conference image on the display unit 130 .

In step S120 , the sound receiving unit 110 detects the volume information of the electronic device 100 . Then, in step S130, the judging module 141 judges whether the volume information is greater than the threshold condition. If the judging module 141 judges that the volume information is greater than the threshold condition, then enter step 140 ; if the judging module 141 judges that the volume information is not greater than the threshold condition, then enter step 120 . Here, the judging module 141 detects whether there is an input signal according to the volume information of the sound receiving unit 110 . For example, when the input signal is greater than a certain volume level, for example, when the input signal is greater than the threshold condition of 5% volume, it can be determined that someone is currently speaking (but it is not sure whether it is the participant speaking or another participant speaking) , or the ambient noise is too loud.

In step S140, the processing unit 140 receives the video images of the participants of the video conference. Next, in step S150 , the processing unit 140 determines whether the participant is speaking according to the video image. In one embodiment, the processing unit 140 can determine whether the participant is speaking by detecting the change of the participant's mouth shape or gesture in the video image. Specifically, the processing unit 140 can use an artificial intelligence algorithm to perform image recognition on the participant's mouth shape changes or gestures in the video image, so as to determine whether the participant is in a speaking state. For example, in step S140, the image processing module 142 can receive the video image, and then in step S150, first perform image processing on the video image, for example, process the video image to extract the simple Character screen. Next, the recognition module 143 uses artificial intelligence algorithms to perform image recognition of mouth shape changes or gestures on the participant's simple character picture to determine whether the participant is in a speaking state.

The manner in which the recognition module 143 uses the artificial intelligence algorithm for training is further described below.

FIG. 3 is a flow chart illustrating training using an artificial intelligence algorithm according to an embodiment of the present invention. Please refer to FIG. 1 and FIG. 3 , in step S210 , the camera unit 120 is used to capture an image with only the background. Next, in step S220 , the camera unit 120 is used to capture an image including the background and the person. Then, in step S230 , the image processing module 142 captures a simple character frame, for example, extracts a simple character frame through back removal processing. Afterwards, steps S210 - S230 can be performed repeatedly under different backgrounds and/or different character conditions to obtain multiple simple character pictures, which can be used as image training sets for the recognition module 143 . In step S240, the recognition module 143 uses artificial intelligence algorithms for training, such as convolutional neural networks (Convolutional Neural Networks, CNN), similar neural network structures (such as Softmax function) and/or other suitable artificial intelligence algorithms method to train the model.

The aforementioned different character conditions can include the same character/different characters opening their mouths to speak, shutting their mouths and not speaking, gestures like "OK" when wearing a mask (indicating that they are currently speaking), or gestures like "cross" when wearing a mask gesture (indicating that you are not speaking at the moment). For example, Fig. 4A to Fig. 4G show the embodiments of different shooting images IMGa-IMGg, wherein Fig. 4A-Fig. 4D and Fig. 4E-Fig. 4G are the same person (such as participant H) respectively Images IMGa~IMGg taken under two different backgrounds. Figures 4A and 4E show the characters opening their mouths to speak. These images IMGa and IMGe can be captured through the image processing module 142 to capture the pure character images, and then marked "start speaking" by the user and sent to the recognition module 143 for training; Figures 4B and 4F show that the characters shut their mouths and do not speak. These images IMGb and IMGf can be captured by the image processing module 142 to capture the pure character images, and then the user marks "Stop talking" and sends them to the recognition module 143 for further processing. Training; Fig. 4C shows the gesture of a character saying "OK" while wearing a mask. This image IMGc can capture a simple character picture through the image processing module 142, and then mark "start talking" by the user and send it to the recognition module Group 143 conducts training; Figures 4D and 4G show the gestures of people making "cross" gestures while wearing a mask. These images IMGd and IMGg can be captured by the image processing module 142 after capturing simple images of people, and then marked by the user "End the speech" and send it to the recognition module 143 for training. In this way, the recognition module 143 after training can be based on the mouth shape changes M (such as Figures 4A, 4B, 4E, 4F) or gestures G (such as Figures 4C, 4D, 4F) of people (such as participants H). 4G map) to determine whether it is speaking.

Please refer to Fig. 1 and Fig. 2. In step S150, when it is judged that the participant is not in the state of speaking, then enter step S160, and the audio channel switching module 144 automatically closes the audio channel of the video conference, so as to prevent the sound of the surrounding environment from passing through. When it is judged that the participant is in the state of speaking, then enter step S170, the radio channel switching module 144 automatically opens the radio channel of the video conference, without adjustment by the participant himself.

FIG. 5 shows an embodiment of a video conference VC. Please refer to FIG. 1 , FIG. 2 and FIG. 5 , when the judgment module 141 judges that the volume information is greater than the threshold condition, the camera unit 120 sends the video image of the participant H to the image processing module 142 for image processing. The recognition module 143 performs image recognition on the processed image, and judges that the participant H is not currently speaking according to the change M of the participant H's mouth shape, and the volume information may come from the sound of the participant's surrounding environment. The radio channel switching module 144 automatically turns off the radio channel MIC of the video conference VC and sets it as mute, so as not to cause interference to the video conference VC.

In addition, in another embodiment, if the participant H wears a mask on his face and forgets to make gestures, so that the recognition module 143 cannot detect the change M of participant H's mouth shape and the gesture G, it can be displayed on the The unit 130 displays a prompt message to remind the participant H to make an appropriate gesture G.

The method and electronic device for automatically switching audio in a video conference proposed by the present invention first detect the volume information of the electronic device, and then determine the source of the volume information through the video images of the participants when the volume information is greater than a preset threshold condition Whether it is the participant himself or not. If so, the audio channel of the video conference is automatically turned on; if not, the audio channel of the video conference is automatically turned off, so as to improve the experience of the user video conference.

Although the present invention has been disclosed above with the embodiments, it is not intended to limit the present invention. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the scope of the appended patent application.

100: Electronic device 110: Radio unit 120: camera unit 130: display unit 140: processing unit 141: Judgment module 142: Image processing module 143: Identification module 144:Radio channel switching module VC: video conferencing G: Gestures H: Participants IMGa~IMGg: Image M: mouth shape change MIC: radio channel S110~S170, S210~S240: steps

FIG. 1 is a block diagram illustrating an electronic device according to an embodiment of the present invention; FIG. 2 is a flow chart illustrating an automatic switching method for video conferencing radio according to an embodiment of the present invention; FIG. 3 is a flow chart illustrating training using an artificial intelligence algorithm according to an embodiment of the present invention; Figures 4A to 4G illustrate examples of different captured images; and FIG. 5 illustrates an embodiment of a video conference.

S110~S170: steps

Claims (10)

An automatic switching method for audio conferencing, comprising: detecting a volume message of an electronic device; if the volume message is greater than a threshold condition, receiving a video image of a participant of a video conference on the electronic device; judging whether the participant is in a state of speaking according to the video image; and If the participant is not in the speaking state, then close the audio channel of the video conference. The automatic switching method of video conference radio as described in request item 1 further includes: If the participant is in the speaking state, the audio channel of the video conference is opened. The method for automatically switching audio in a video conference as described in claim item 1, wherein in the step of judging whether the participant is in the speaking state according to the video image, it is by detecting a change of the participant's mouth shape in the video image Or a gesture action to determine whether the participant is in the speaking state. The automatic switching method of video conference radio as described in request item 3, wherein if the change of the mouth shape and the gesture of the participant cannot be detected from the video image, a prompt message is displayed on the electronic device to remind The participant makes the gesture. The method for automatically switching audio in a video conference as described in request item 3, wherein the judgment of whether the participant is in the speaking state is performed by using an artificial intelligence algorithm to perform the change of the mouth shape or the gesture of the participant in the video image image recognition. An electronic device comprising: a radio unit for detecting a volume message; a processing unit for performing a video conference and receiving the volume message; and a camera unit for obtaining a video image of one of the participants of the video conference; Wherein, if the processing unit judges that the volume message is greater than a threshold condition, then receive the video image, and judge whether the participant is in a state of speaking according to the video image; If the processing unit determines that the participant is not in the speaking state, then close the audio channel of the video conference. The electronic device according to claim 6, wherein if the processing unit determines that the participant is in the speaking state, then open the audio channel of the video conference. The electronic device according to claim 6, wherein the processing unit determines whether the participant is in the speaking state by detecting a change in the participant's mouth shape or a gesture in the video image. The electronic device as described in claim 8, wherein if the processing unit cannot detect the change of the mouth shape and the gesture of the participant from the video image, then make a display unit display a prompt message to remind the participant make the gesture. The electronic device as described in claim 8, wherein the processing unit uses an artificial intelligence algorithm to perform image recognition on the change of the participant's mouth shape or the gesture in the video image to determine whether the participant is in the speech state.

Images