Detailed Description
Embodiments of the present invention will be described below with reference to the accompanying drawings. In the drawings, the same reference numerals indicate the same or similar components or process flows.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, values, method steps, operations, components, and/or components, but do not preclude the presence or addition of further features, values, method steps, operations, components, and/or groups thereof.
It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is described as being "directly connected" or "directly coupled" to another element, there are no intervening elements present.
Please refer to fig. 1, which is a block diagram illustrating an exemplary embodiment of a multimedia interactive system according to the present application. As shown in fig. 1, the multimedia interaction system 1 includes: the wireless broadcast device 11, the terminal equipment 12 and the cloud server 13. The terminal device 12 may be selectively connected to the wireless broadcasting device 11 wirelessly, and the cloud server 13 is connected to the terminal device 12 through a network (shown in fig. 1 as a cloud shape).
In practical implementations, the wireless broadcasting device 11 may be, but is not limited to, a wireless headset or a wireless electronic device with broadcasting function, such as: intelligent glasses or wireless sound equipment capable of broadcasting; terminal device 12 may be, but is not limited to, a mobile terminal, a tablet computer, a desktop computer, a laptop/notebook computer, or a smart television; the wireless communication protocol between the wireless broadcaster 11 and the terminal equipment 12 may include, but is not limited to, WiFi, bluetooth or ZigBee; the network of the terminal device 12 connected to the cloud server 13 may include, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a VPN network, or a wireless Ad Hoc network (Ad Hoc network).
In this embodiment, the terminal device 12 is configured to establish a connection with the radio broadcasting apparatus 11 and transmit a transcription instruction to the radio broadcasting apparatus 11.
In one embodiment, terminal equipment 12 cannot directly transmit instructions to the wireless broadcaster 11 to which it is wirelessly connected, and therefore, terminal equipment 12 may download and install an application program for controlling wireless broadcaster 11 to transmit the transcription instructions to wireless broadcaster 11 to which it is connected via the application program.
In this embodiment, in the process of playing the video by the terminal device 12, the wireless broadcast device 11, which is connected to the terminal device 12 and receives the transcription instruction, continuously receives and compresses the voice information of the video, so as to continuously return the compressed information to the terminal device 12; the terminal device 12 decompresses the compressed information received in real time to obtain the information to be translated; the terminal device 12 transmits the information to be translated to the cloud server 13 for translation to obtain the text information with the target language; the terminal device 12 displays the text information on the screen of the video played by it. Therefore, the multimedia interactive system 1 can perform the translation function by using the wireless broadcast device 11 as a medium for capturing and transmitting the voice information of the video and the cloud server 13, so that the terminal device 12 can provide the subtitles in the familiar language corresponding to the video watched by the user, thereby improving the experience of the user in watching the video.
In one embodiment, the wireless communication protocol between the wireless broadcasting device 11 and the terminal device 12 may be bluetooth, and therefore, the terminal device 12 may establish a bluetooth Audio Distribution Profile (A2DP) connection with the wireless broadcasting device 11, so that the terminal device 12 may transmit the voice information of the video it is playing to the wireless broadcasting device 11 based on the A2DP connection.
In one embodiment, since the wireless broadcast device 11 needs to return the compressed information to the terminal device 12 while receiving the voice information of the video from the terminal device 12, in order to prevent the wireless broadcast device 11 from returning the compressed information to the terminal device 12, which occupies too much communication bandwidth between the wireless broadcast device 11 and the terminal device 12, and jam the transmission of the voice information transmitted to the wireless broadcast device 11 by the terminal device 12, the transmission of the compressed information from the wireless broadcast device 11 to the terminal device 12 needs to be performed in the time interval of the voice information transmitted to the wireless broadcast device 11 by the terminal device 12. Specifically, the terminal device 12 encapsulates the data of the voice information into individual data packets, and then sequentially transmits the data packets of the voice information to the radio broadcasting device 11 using a communication protocol between the terminal device 12 and the radio broadcasting device 11, since the transmission of the data packet of the voice message has a time gap (that is, the terminal device 12 does not transmit the data packet of the voice message uninterruptedly), the wireless broadcasting device 11 may decapsulate the received data packet of the voice message and compress the information, then encapsulate the compressed information into individual data packets, then, in the time gap of the transmission of the data packet of the voice information, the data packet of the compressed information is sequentially transmitted back to the terminal device 12 by using the communication protocol between the terminal device 12 and the wireless broadcasting device 11 (i.e. the transmission of the data packet of the compressed information is interspersed between the transmission of the data packet of the voice information). That is, the wireless broadcasting device 11 continuously transmits back the packet of the compressed information to the terminal device 12 during the interval of continuously receiving the packet of the voice information.
In one embodiment, the wireless broadcaster 11 may be configured to continuously receive the voice message and compress the voice message with a default compression ratio through a voice compression algorithm. The speech compression algorithm may be any algorithm that can meet the hearing requirement (user satisfaction), such as: g.711a, g.711u, g.729, 324M, etc., the default compression ratio may be, but is not limited to, 16: 1, the actual speech compression algorithm and the default compression ratio can be adjusted according to actual requirements.
In an embodiment, each packet of the compressed information may include a CRC16 check code, and the terminal device 12 is further configured to continuously determine whether the CRC16 check code included in each packet of the compressed information received in real time passes the check, to decompress the packet of the compressed information received in real time that passes the check, and to notify the wireless broadcasting device 11 of retransmission of the packet of the compressed information that fails the check. Therefore, it is possible to prevent the occurrence of a situation in which a subsequent decompression failure occurs due to a transmission error of the compressed information.
In one embodiment, the wireless communication protocol between the wireless broadcasting apparatus 11 and the terminal device 12 may be bluetooth, so that the wireless broadcasting apparatus 11 may also continuously transmit the compressed information back to the terminal device 12 based on Serial Port Profile (SPP) protocol.
In an embodiment, when the terminal device 12 transmits the information to be translated to the cloud server 13 for translation, a language instruction (that is, the language instruction is used to specify the target language) is also transmitted to the cloud server 13, so that the cloud server 13 translates the received information to be translated into the text information in the target language based on the language instruction. Wherein, the target language may be, but not limited to, chinese. In an example, the process of the cloud server 13 translating the received information to be translated into the text information in the target language may include: converting the received information to be translated into character information in the same language as the voice information by a voice to Text (Speech to Text) technology; and translating the text information in the same language as the voice information into the text information in the target language by a text translation technology.
In another embodiment, the target language may be a default language, and therefore, the terminal device 12 only needs to transmit the information to be translated to the cloud server 13 (i.e., does not need to transmit the language instruction to the cloud server 13), and can obtain the text information with the target language returned by the cloud server 13.
In one embodiment, terminal device 12 may display the text message in the form of a floating window on the screen of the video played by the terminal device. In another embodiment, terminal device 12 may directly superimpose the text information onto the picture of the played video in a manner similar to Augmented Reality (Augmented Reality) technology.
Please refer to fig. 2, which is a flowchart illustrating a multimedia processing method according to an embodiment of the present application. As shown in fig. 2, the multimedia processing method 2 can be applied to the multimedia interaction system 1 of fig. 1, and the multimedia processing method 2: the terminal device 12 establishes a connection with the radio broadcasting apparatus 11 and transmits a transcription instruction to the radio broadcasting apparatus 11 (step 21); in the process of playing the video by the terminal device 12, the wireless broadcasting device 11 connected to the terminal device 12 continuously receives and compresses the voice information of the video, so as to continuously return the compressed information to the terminal device 12 (step 22); the terminal device 12 decompresses the compressed information received in real time to obtain the information to be translated (step 23); the terminal device 12 transmits the information to be translated to the cloud server 13 for translation to obtain the text information with the target language (step 24); and terminal device 12 displays the text information on the screen of the video it plays (step 25). That is, the multimedia interactive system 1 can use the wireless broadcast device 11 as a medium for capturing and transmitting the voice information of the video, and further use the cloud server 13 to perform the translation function, so that the terminal device 12 can provide the user with the subtitles in the familiar language corresponding to the video he watches.
In one embodiment, step 21, the terminal device 12 transmitting the transcription instruction to the radio broadcasting apparatus 11 may include: the terminal device 12 downloads and installs an application program for controlling the radio broadcasting device 11 to transmit the transcription instruction to the radio broadcasting device 11 connected thereto through the application program. The detailed description is already described in the above paragraphs, and will not be repeated herein.
In one embodiment, the step 21 of establishing a connection between the terminal device 12 and the wireless broadcasting apparatus 11 may include: the terminal device 12 and the wireless broadcasting apparatus 11 establish a bluetooth audio transmission model (A2DP) agreement connection. The detailed description is already described in the above paragraphs, and will not be repeated herein.
In one embodiment, step 22 may comprise: the wireless broadcasting device 11 continuously transmits back the data packet of the compressed information to the terminal equipment 12 during the interval of continuously receiving the data packet of the voice information. The detailed description is already described in the above paragraphs, and will not be repeated herein.
In one embodiment, step 22 may comprise: the wireless broadcaster 11 continuously transmits the compressed information back to the terminal device 12 based on the serial port Specification (SPP) protocol. The detailed description is already described in the above paragraphs, and will not be repeated herein.
In one embodiment, step 22 may comprise: the wireless broadcasting device 11 continuously receives the voice information and compresses the voice information by a voice compression algorithm according to a default compression ratio. The detailed description is already described in the above paragraphs, and will not be repeated herein.
In an embodiment, each data packet of the compressed information may include a CRC16 check code, and step 23, the terminal device 12 decompresses the compressed information received in real time, which may include: the terminal device 12 continuously determines whether the CRC16 check code included in each packet of the compressed information received in real time passes the check, so as to decompress the packet of the compressed information received in real time that passes the check, and notifies the wireless broadcasting apparatus 11 of retransmitting the packet of the compressed information that fails the check. The detailed description is already described in the above paragraphs, and will not be repeated herein.
In one embodiment, step 24 may comprise: when the terminal device 12 transmits the information to be translated to the cloud server 13 for translation, the language instruction is also transmitted to the cloud server 13, so that the cloud server 13 translates the received information to be translated into the text information with the target language based on the language instruction. The detailed description is already described in the above paragraphs, and will not be repeated herein.
In one embodiment, step 25 may comprise: the terminal device 12 displays the text information on the picture of the video played by the terminal device in the form of a floating window.
In another embodiment, step 25 may comprise: the terminal device 12 directly superimposes the text information on the picture of the played video in a manner similar to the augmented reality technology.
To sum up, in this application embodiment, the wireless broadcast device obtains the video speech information of terminal equipment real-time broadcast to the compressed information that corresponds speech information of passback gives terminal equipment, makes terminal equipment translate through the high in the clouds server after decompressing compressed information, and obtains the text information that has the target language that the video of broadcast corresponds (namely the text information of the language that the user is familiar with), and then shows on the video picture of broadcast text information, in order to improve the user and watch the video experience. In addition, because the voice information of the video played by the terminal device in real time is acquired through the wireless broadcast device, the multimedia processing method and the multimedia interaction system in the embodiment of the application are suitable for being applied to the situation that the terminal device playing the video cannot directly acquire the voice information of the video watched by the user (for example, the terminal device cannot directly acquire the voice information of the video watched by the user based on the authority problem).
Although the above-described elements are included in the drawings of the present application, it is not excluded that more additional elements may be used to achieve better technical results without departing from the spirit of the invention.
While the invention has been described using the above embodiments, it should be noted that these descriptions are not intended to limit the invention. Rather, this invention encompasses modifications and similar arrangements as would be apparent to one skilled in the art. The scope of the claims is, therefore, to be construed in the broadest manner to include all such obvious modifications and similar arrangements.