CN116501919A - Prompting method, device, equipment and storage medium - Google Patents

Abstract

The application discloses a prompting method, device, equipment and storage medium. The method includes: acquiring historical audio data of a target object; determining the speech rate text of the target object according to the historical audio data; acquiring real-time audio data of the target object; determining that the text corresponding to the real-time audio data is in the target The location information of the speech rate text of the object; determine the content to be prompted according to the location information and the speech rate text of the target object, and display the content to be prompted through the prompt display screen. Through the technical solution of this application, it is possible to quickly locate in the pre-recorded audio according to the content of the speaker or singer's speech or singing, and then determine the content that needs to be prompted, improve the convenience of the teleprompter device, and accurately provide the speaker or singer Singers provide personalized teleprompter messages and tempo based on any content from pre-recorded audio.

Description

Prompt method, device, equipment and storage medium

technical field

The embodiments of the present application relate to the technical field of audio processing, and in particular, to a prompt method, device, device, and storage medium.

Background technique

Teleprompter is a common technology in life. For example, when we sing karaoke, we need prompts. In addition to singing, speeches also require prompts. Usually, the teleprompter used for speech is a kind of display screen, which displays the content of the document through a high-brightness display device, and reflects the content to a special coated glass at a 45-degree angle in front of the camera lens, reflecting the lines, making The speaker can also face the camera while reading the speech. It is supported on the same axis with the speaker, camera, and tripod, so that the intimacy that the speaker is always facing the audience improves the quality of the speech. Everyone actually has their own unique speaking speed. However, no matter whether it is speech or singing, the current teleprompter cannot provide personalized teleprompter service.

Contents of the invention

In view of this, the embodiment of this application provides a reminder method, including:

Obtain historical audio data of the target object;

determining the speech rate text of the target object according to the historical audio data;

Acquiring real-time audio data of the target object;

Determining the position information of the text corresponding to the real-time audio data in the speech rate text of the target object;

The content to be prompted is determined according to the location information and the speech rate text of the target object, and the content to be prompted is displayed on a prompter display screen.

The embodiment of the present application also provides a reminder device, which includes:

The first obtaining module is used to obtain the historical audio data of the target object;

A first determination module, configured to determine the speech rate text of the target object according to the historical audio data;

The second obtaining module is used to obtain the real-time audio data of the target object;

The second determination module is used to determine the position information of the text corresponding to the real-time audio data in the speech rate text of the target object;

The obtaining and prompting module is configured to determine the content to be prompted according to the location information and the speech rate text of the target object, and display the content to be prompted through the prompt display screen.

The embodiment of the present application also provides an electronic device, and the electronic device includes:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the method described in any embodiment of the present application. Prompt method.

The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the prompting method described in any embodiment of the present application. .

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the application, nor is it intended to limit the scope of the application. Other features of the present application will be easily understood from the following description.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings that are required in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and thus It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

FIG. 1 is a flow chart of a prompting method in an embodiment of the present application;

Fig. 2 is a schematic diagram of a prompting method in the embodiment of the present application;

Fig. 3 is a schematic structural diagram of a prompting device in an embodiment of the present application;

Fig. 4 is a schematic structural diagram of an electronic device implementing a prompting method according to an embodiment of the present application.

Detailed ways

In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the application. Obviously, the described embodiment is only It is an embodiment of a part of the application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

It should be noted that the terms "first" and "second" in the description and claims of the present application and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

Embodiment one

The teleprompter is a professional device for the speaker or actor to prompt the speaker or actor with lines during speech or singing. It is generally arranged in front of the speaker or actor, but the content of the teleprompter needs to be manually input, and a special person is required to be behind the scenes when using it Manually control the display order. In addition, the teleprompter is relatively bulky and difficult to deploy, and it is inconvenient to arrange the teleprompter in some important occasions such as reporting to the superior. In the existing technology, there is a subtitle technology similar to the teleprompter. The difference between the two is that the subtitle uses sound as the original input and outputs the corresponding text, with the purpose of providing information to the viewers; and the purpose of the teleprompter is to help the speech The speaker or singer masters the content and rhythm of the speech or singing, focusing on prompting the speaker or singer's speech speed and intonation through text fade-in and highlight positions. This function is very important in speech or singing scenes.

The embodiment of the present application provides a prompting method, device, device and storage medium, so as to quickly locate in the pre-recorded audio according to the content of the speaker or singer's speech or singing, and then determine the content that needs to be prompted, and improve the prompting. The convenience of the equipment can accurately provide the speaker or singer with personalized teleprompter information and speed for any content.

Fig. 1 is a flow chart of a reminder method in the embodiment of the present application. This embodiment is applicable to the situation of prompting. The method can be executed by the prompting device in the embodiment of the present application. The device can use software and/or hardware As shown in Figure 1, the method specifically includes the following steps:

S101. Acquire historical audio data of a target object.

In this embodiment, the target object may be a user such as a speaker or a singer, and the number of specific target objects may be one or more, which is not limited in this embodiment.

Wherein, the historical audio data may be pre-recorded audio file data corresponding to the target object. Exemplarily, the historical audio data of the target object may be the audio data when the speaker recites the speech content such as poems and poems, or may be the audio data when the singer performs the singing of folk art content such as songs and operas.

Specifically, in one embodiment, the target object can record historical audio data in advance when giving a speech or singing, and the recording tool can be any recording device (for example, a recording device in a recording studio or a terminal device such as a smart phone). The historical audio data recorded by the recording device when the target object is speaking or singing may also be stored in the audio storage device. The pre-recorded historical audio data of the target object speaking or singing can be obtained from the recording device or the audio storage device.

S102. Determine the speech rate text of the target object according to the historical audio data.

It should be noted that the speech rate text may be extracted from the audio data, and is the text corresponding to the speech rate of the target object when speaking or singing.

Extracting text from audio data can be obtained by any speech technology. In one embodiment, AI speech recognition technology (such as DNN-HMM deep neural network technology) can be used to perform speech recognition on historical audio data, and at the same time, the speech rate text corresponding to historical audio data can be transcribed through AI grammar and semantic recognition, which can Simplify the difficulty of extracting prompt content and reduce the time for manual production.

Specifically, in one embodiment, after obtaining the pre-recorded historical audio data from the recording device or audio storage device, the speech rate text corresponding to the historical audio data is transcribed through AI grammar and semantic recognition, and recorded in the lrc (lyric The abbreviation of the corresponding English word lyric is used as the extension of the lyrics file) in the file.

Exemplarily, a piece of historical audio data contains two lines of poem "The sun is at the end of the mountain, and the Yellow River flows into the sea". The format of the corresponding speech rate text in the converted lrc file can be expressed as follows:

[00:00.000]<1050>white<2300>day<800>by<500>mountain<2800>,

[00:07.450]<1500>Yellow<1200>River<1800>Enter<750>Sea<530>Current.

Among them, [00:00.000] can be the start time of the first sentence "Bai Sun Yi Shan Jin", [00:07.450] can be the start time of the second sentence "Yellow River flows into the sea", and the number in front of each word can be duration of each word. For example, <1050> takes 1.050 seconds for the word "白", <2300> takes 2.300 seconds for the word "日", and so on.

In an actual operation process, the format of the above lrc file may also be expressed as a time period divided by intervals. In addition, the content of the above-mentioned lrc file may also be a song, a speech, or a drama.

When generating the lrc file, it is not only necessary to identify the content of the speech based on historical audio data, but also to separate the duration of each text, so that the color can be changed or the prompt text can be highlighted during the speech to remind the user of the progress of the speech.

S103. Acquire real-time audio data of the target object.

In this embodiment of the present application, the real-time audio data is opposite to the historical audio data, and the real-time audio data may be audio file data corresponding to a target object recorded in real time. Exemplarily, the real-time audio data of the target object may be the audio data when the speaker recites the speech content such as poems and songs, or may be the audio data when the singer performs the singing of folk art content such as songs and operas.

Specifically, in one embodiment, the target object may collect real-time audio data when speaking or singing, and the collection tool may be, for example, AR glasses worn by the target object or other devices with microphones. The real-time audio data of the target object when speaking or singing can be obtained from the AR glasses worn by the target object or other devices with microphones.

S104. Determine the position information of the text corresponding to the real-time audio data in the speech rate text of the target object.

In various embodiments, the text corresponding to the real-time audio data of the target object may be compared with the speech rate text of the target object to obtain position information of the text corresponding to the real-time audio data in the speech rate text of the target object.

Specifically, in one embodiment, the recording device or audio storage device synchronizes the speech rate text of the target object to the AR glasses end worn by the target object, and the AR glasses converts the real-time audio data of the target object to the corresponding text and the speech rate text of the target object Compare and find out the position information of the text corresponding to the real-time audio data in the speech rate text of the target object.

Exemplarily, the speech rate text of the target object is "<1050>Bai<2300>Day<800>Yi<500>Mountain<2800>End", and the corresponding text of the real-time audio data is "Day Day", then the real-time audio data corresponds to The position information of the text in the speech rate text of the target object may be the positions of the first two characters.

S105. Determine the content to be prompted according to the location information and the speech rate text of the target object, and display the content to be prompted through the prompt display screen.

It should be explained that the content to be prompted may be the content to be displayed on the prompt display screen.

In an embodiment of the present application, the teleprompter display may be a wearable device worn by the target object, such as AR glasses.

AR is a technology that calculates the position and angle of camera images in real time, and adds corresponding images, videos, and 3D models. It obtains real environment information through cameras, and then projects virtual objects (such as images, scenes, or system prompts, etc.) Superimposed on the obtained real environment information, and then displayed to the user, so that the user can simultaneously see the visual effect of the virtual object existing in the real environment, that is, to realize the "enhancement" of the real scene.

Specifically, in one embodiment, the wearable device worn by the target object, such as AR glasses, determines the content to be prompted according to the position information of the text corresponding to the real-time audio data in the target object's speech rate text and the target object's speech rate text. Exemplarily, the speech rate text of the target object may be a text containing time (speech rate) information "The sun is at the end of the mountain, and the Yellow River flows into the sea", and the position information of the text corresponding to the real-time audio data in the speech rate text of the target object may be The position of the first two characters, that is, "Bai Ri", then the determined content that needs to be prompted can be the content after "Bai Ri", for example, it can be "Yi", "Yi Shan Jin" or "Yi Shan Jin". , the Yellow River flows into the sea", the number of specific prompt texts can be preset by the user, which is not limited in this embodiment. After determining the content that needs to be prompted, the display screen of the AR glasses worn by the target object displays the content that needs to be prompted, and accurately provides the speaker or singer with any content prompt information, which improves the convenience of the teleprompter device, and at the same time provides The word content is projected on the inner screen of the AR glasses, and cannot be viewed from the outside, which improves the privacy of use and protects the privacy of the speaker.

In the actual operation process, when there are multiple target objects, that is, when multiple users are speaking or singing at the same time, in one embodiment, the voices of all target objects can be recorded into one audio data, and each target object Displayed on the teleprompter display, each user can view everyone's speech or singing content at the same time (the target text corresponding to each user can be distinguished by different colors or other effects), when the user wants to only view his own speech or singing When viewing content, you can select only the part of your speech or singing to display. In another embodiment, each target object's own speech or singing content can be made into a separate audio data, and then displayed through the respective teleprompter display.

In the embodiment of the present application, by acquiring the historical audio data of the target object, determining the speech rate text of the target object according to the historical audio data, obtaining the real-time audio data of the target object, and determining the position information of the text corresponding to the real-time audio data in the speech rate text of the target object, Determine the content that needs to be prompted according to the location information and the speech rate text of the target object, and display the content that needs to be prompted through the teleprompter display. Through the technical solution of this application, it is possible to quickly locate in the pre-recorded audio according to the content of the speaker or singer's speech or singing, and then determine the content that needs to be prompted, improve the convenience of the teleprompter device, and accurately provide the speaker or singer Singers provide personalized teleprompter messages and tempo based on any content from pre-recorded audio.

In some embodiments, the content to be prompted is determined according to the location information and the speech rate text of the target object, including:

Get the original text corresponding to the historical audio data.

It should be noted that the original text may be an accurate speech or sung by a speaker or singer when collecting historical audio data, without typos or missing characters.

Exemplarily, the original text corresponding to the historical audio data may be text content such as a lecturer's speech, or text content such as a singer's lyrics.

The content to be prompted is determined according to the location information, the original text, and the speech rate text of the target object.

In one embodiment, when historical audio data and/or real-time audio data are recognized for speech, grammar, and semantics, there may be inaccurate recognition such as typos or multi-word omissions, for example, the original text corresponding to historical audio data It can be "the sun is at the end of the mountain, the Yellow River flows into the sea", but typos may appear when recognizing historical audio data and/or real-time audio data, such as "a hundred days are at the end of the mountain, the Yellow River flows into the sea", at this time, you can use the The recognition result of the real-time audio data finds the most suitable position in the original text corresponding to the historical audio data to determine the content that needs prompting.

Specifically, the content to be prompted is determined according to the position information of the text corresponding to the real-time audio data in the speech rate text of the target object, the original text, and the speech rate text of the target object.

In some embodiments, determining the speech rate text of the target object according to historical audio data includes:

Obtain audio amplitude information and audio frequency information of historical audio data.

Specifically, the amplitude information and frequency information of the audio signal in the pre-recorded historical audio data are obtained.

Identify the audio amplitude information and audio frequency information, and obtain the text and speech rate information corresponding to the historical audio data.

In one embodiment, AI speech recognition technology (such as DNN-HMM deep neural network technology) can be used to perform speech recognition on historical audio data, and at the same time, the text and speech rate information corresponding to historical audio data can be transcribed through AI grammar and semantic recognition , which can simplify the difficulty of extracting the prompt content and reduce the time for manual production. However, the implementation of the technical solution is not limited to AI speech, grammar and semantic recognition.

In one embodiment, obtain pre-recorded historical audio data, identify sentence breaks, word interval positions, and corresponding time according to the amplitude and frequency changes of the audio signal in the historical audio data, and record them in lrc (the abbreviation of the English word lyric corresponding to the lyrics) , is used as the extension of the lyrics file) file, the duration of each sentence or even each word or word is detected by analyzing the amplitude change of the human voice in the historical audio data.

The speech rate text of the target object is determined according to the text and speech rate information corresponding to the historical audio data.

Specifically, the speech rate text of the target object can be determined according to information such as words, sentences, word interval positions and corresponding time corresponding to the historical audio data.

In some embodiments, the speech rate information includes: at least one of duration corresponding to each character, duration corresponding to each word, duration corresponding to each sentence, and start time corresponding to each sentence.

Exemplarily, the interval time between two sentences in the historical audio data may be determined according to the start time corresponding to each sentence in the historical audio data and the duration corresponding to each sentence in the historical audio data. Specifically, according to the sum of the start time corresponding to a sentence in the historical audio data plus the duration corresponding to the sentence in the historical audio data, the duration corresponding to the sentence in the historical audio data and the duration corresponding to the sentence in the historical audio data can be determined. The end time corresponding to the sentence can be determined according to the end time corresponding to the sentence in the historical audio data and the corresponding start time of the next sentence in the historical audio data. The interval between sentences. In one embodiment, the prompt method corresponding to the interval between two sentences can be preset, for example, it can be a progress bar, specifically several dots, and the prompt method such as gradually changing color or size as time progresses , in another embodiment, the prompting manner corresponding to the interval between two sentences may also be other prompting manners that can be used to indicate the time progress, which is not limited in this embodiment. Obtain the prompt mode corresponding to the interval between two sentences, determine the prompt mode corresponding to each text in the historical audio data according to the duration corresponding to each text in the historical audio data, and determine the corresponding prompt mode for each text in the historical audio data according to the corresponding duration of each text in the historical audio data The prompt mode determines the prompt mode corresponding to each sentence in the historical audio data, and determines the prompt mode corresponding to the historical audio data according to the prompt mode corresponding to the interval between every two sentences and the prompt mode corresponding to each sentence in the historical audio data.

Exemplarily, in one embodiment, when a speaker or a singer makes a speech or sings, there may be prelude music, or blank space, before the first sentence of the speech or sing. The start time of the historical audio data is obtained, and the time interval is determined according to the start time of the historical audio data and the start time of the speech rate text. The prompt method corresponding to the time interval can be preset, for example, it can be a progress bar, specifically it can be several dots, and the prompt method gradually changes in color or size as time progresses. In addition, it can also be other prompts that can be used to represent time. The way of prompting progress is not limited in this embodiment. The prompt mode corresponding to the content to be prompted is determined according to the prompt mode corresponding to the time interval, the prompt mode corresponding to each sentence in the historical audio data, and the prompt mode corresponding to the interval between every two sentences.

In some embodiments, the content to be prompted is determined according to the location information, the original text, and the speech rate text of the target object, including:

The text to be pronounced is determined based on the location information and the original text.

It should be noted that the text to be pronounced may be the text to be pronounced by the target object such as a speaker or a singer when speaking or singing.

In one embodiment, the original text may be "Bai Ri ends at the end of the mountain, the Yellow River flows into the sea", and the target object such as a speaker or a singer has just finished pronouncing "Bai Ri", and the position information of "Bai Ri" is the most recent in the original text. The position of the first two words at the beginning, then the determined text to be pronounced can be the text behind "Bai Ri", for example, it can be "Yi", "Yi Shan Jin" or "Yi Shan Jin, the Yellow River flows into the sea", specifically The number of prompt texts can be preset by the user, which is not limited in this embodiment.

The prompting method of the text to be pronounced is determined according to the speech rate text of the target audience.

Specifically, after the text to be pronounced is determined, a prompting manner for the text to be pronounced may be determined according to the speech rate text of the target object.

In some embodiments, the prompting method includes: using at least one of a cursor, a highlight and a marquee to prompt the text to be pronounced.

In one embodiment, the prompting method corresponding to the text to be pronounced may be, for example, a cursor. In another embodiment, the prompting method corresponding to the text to be pronounced may also be, for example, a highlight scrolling type. In addition, the text to be pronounced The corresponding prompt manner may also be, for example, a marquee style, or a thickness change prompt, etc., which are not limited in this embodiment.

In some embodiments, the prompting method of the text to be pronounced is determined according to the speech rate of the target object, including:

The duration corresponding to each word in the text to be pronounced and the interval time between adjacent words are determined according to the speech rate text of the target object.

In one embodiment, the speech rate text corresponding to the target object can be obtained when the historical video data is identified, and the speech rate text includes the duration corresponding to each word and the interval between adjacent words, from the target object The speech rate text obtains the duration corresponding to each word in the text to be pronounced and the interval between adjacent words.

The prompt mode of the text to be pronounced is determined according to the duration corresponding to each character in the text to be pronounced and the interval time between adjacent words.

In one embodiment, the prompting method of the text to be pronounced on the prompt display screen can be determined according to the duration corresponding to each character in the text to be pronounced and the interval between adjacent words. In the actual operation process, the duration corresponding to each character in the historical audio data may be different, and the prompting method corresponding to each character in the characters to be pronounced may also be different. For example, in the case that the prompt mode is a highlight rolling type, the corresponding highlight duration of each character in the text to be pronounced is also different, for example, it may be that the corresponding highlight duration of each character in the text to be pronounced is to be The duration corresponding to each character in the to-be-pronounced text, and finally, the prompt mode corresponding to the to-be-pronounced text can be determined according to the prompt mode corresponding to each character in the to-be-pronounced text.

In some embodiments, after displaying the content to be prompted through the prompt display screen, it further includes:

Receive control instructions on the currently displayed content.

In this embodiment, the control instruction may be an instruction sent by the user to adjust the content being displayed on the display screen of the wearable device. In one embodiment, the control instruction can be an instruction to adjust the playback speed of the content being displayed on the display screen of the wearable device. An instruction to adjust the playback progress of the content being displayed.

Wherein, the currently displayed content may be the content being displayed on the display screen of the wearable device worn by the target object when the control instruction is received.

Specifically, the user can directly issue a control command on the wearable device. In one embodiment, the user can use a control button on the wearable device or a virtual button on a display screen of the wearable device. In another embodiment, the user may also issue control instructions through operations such as gesture recognition. In addition, users can also issue control commands through terminal devices such as smart phones. In the actual operation process, when sending control commands to wearable devices worn by target objects through terminal devices such as smart phones, some interconnected communication technologies widely used at present can be used. One or more of them: for example, wifi, bluetooth, or cloud terminal, etc., which are not limited in this embodiment.

The current display content is updated according to the control instruction, the current display content and the speech rate text of the target object.

Specifically, after the wearable device receives the control instruction for the currently displayed content, it updates the current displayed content corresponding to the teleprompter display according to the control instruction, the content being displayed on the teleprompter display, and the speech rate text of the target object. . In one embodiment, for example, the playback speed may be updated, and in another embodiment, for example, the playback progress may be updated.

In one embodiment, the current display content is updated according to the control instruction, the current display content and the speech rate text of the target object, including:

It is determined that the control instruction is a playback speed adjustment instruction.

It should be noted that the playback speed adjustment instruction may be an instruction issued by the user to adjust the playback speed of the content being displayed on the display screen of the wearable device.

Specifically, it is detected whether the control instruction issued by the user is a playback speed adjustment instruction.

Obtain the speed adjustment parameter corresponding to the playback speed adjustment command and the current playback speed.

In this embodiment, the speed adjustment parameter may be to adjust the multiple of the playback speed of the currently displayed content corresponding to the display screen of the wearable device, for example, it may be 0.5 times, 0.75 times, 1.5 times or 2 times. Specifically, the speed adjustment parameter may be preset, and the user may select the playback speed to be adjusted.

Wherein, the current playback speed may be the playback speed of the currently displayed content corresponding to the display screen of the wearable device worn by the target object when the playback speed adjustment instruction is received.

Specifically, if the received control instruction is a playback speed adjustment instruction, the user-selected speed adjustment parameter corresponding to the playback speed adjustment instruction and the current playback speed of the currently displayed content corresponding to the display screen of the wearable device worn by the target object are acquired.

Determine the target playback speed according to the speed adjustment parameter and the current playback speed.

Wherein, the target playback speed may be an adjusted playback speed of the content displayed on the display screen of the wearable device worn by the target object.

Specifically, the result of the product of the current playback speed and the speed adjustment parameter may be used as the target playback speed. Exemplarily, the current playback speed of the currently displayed content corresponding to the display screen of the wearable device worn by the target object is 1 times the speed, and the speed adjustment parameter corresponding to the playback speed adjustment instruction is 0.5 times, then the target playback speed can be 1×0.5= Play at 0.5x speed.

The current display content is updated according to the target playback speed, the current display content and the speech rate text of the target object.

Specifically, update the current display content corresponding to the display screen of the wearable device worn by the target object to the text to be pronounced at the target playback speed, and at the same time, prompt the text to be pronounced according to the prompt method corresponding to the text to be pronounced.

By setting the speed adjustment parameter corresponding to the playback speed adjustment command, the user can control the prompt to accelerate or decelerate the prompt according to the actual situation, so as to conveniently and accurately provide the prompt information of any content for the speaker or singer.

In another embodiment, the current display content is updated according to the control instruction, the current display content and the speech rate text of the target object, including:

It is determined that the control instruction is a playback progress adjustment instruction.

It should be noted that the playback progress adjustment instruction may be an instruction issued by the user to adjust the playback progress of the content being displayed on the display screen of the wearable device. Exemplarily, the playback progress adjustment instruction may be to play the previous sentence or to play the next sentence, etc., and the progress adjustment options such as playing the previous sentence or the next sentence may be preset, and the user may select the desired playback progress.

Specifically, it is detected whether the control instruction issued by the user is a playback progress adjustment instruction.

If the target instruction is a playback progress adjustment instruction, the text to be prompted is determined according to the playback progress adjustment instruction, the currently displayed content, and the speech rate text of the target object.

Wherein, the speech rate text of the target object includes the text to be prompted, that is, the text to be prompted is the text included in the speech rate text of the target object.

It should be explained that the text to be prompted may be the text to be prompted after the display screen of the wearable device worn by the target object is updated.

Specifically, if the received control instruction is a playback progress adjustment instruction, update the current display content corresponding to the display screen of the wearable device to the text to be prompted according to the playback progress adjustment instruction and the speech rate text of the target object. Exemplarily, the current display content corresponding to the display screen of the wearable device is "The day is beyond the mountains", and the user chooses to play the next sentence. At this time, the determined playback progress adjustment instruction can be "Play the next sentence", then it can be determined The text to be prompted on the display screen of the wearable device may be "Yellow River flows into the sea".

The current display content of the display screen of the wearable device worn by the target object is updated with the text to be prompted.

Specifically, the current display content of the display screen of the wearable device worn by the target object is updated with the text to be prompted.

By setting the playback progress adjustment command, the user can control to return to the previous sentence or skip a sentence according to the actual situation, so as to conveniently and accurately provide the speaker or singer with prompt information of any content.

As an exemplary description of the embodiment of the present application, FIG. 2 is a schematic diagram of a prompting method in the embodiment of the present application. As shown in Figure 2, the content displayed on the display screen of the wearable device AR glasses worn by the target object is "The sun is at the end of the mountain, and the Yellow River flows into the sea". It can be seen that the current progress of the speech has been completed. According to", the next word to be spoken is "mountain". The font of the word that has been spoken is thicker, and the font of the word that has not been spoken is thinner, which can accurately and intuitively prompt the user. At the same time, there are also buttons for 0.5 times slower, previous sentence, continue/pause, next sentence and 0.5 times faster at the bottom of the display. Users can control the playback speed or progress, and play or pause according to the actual situation. According to the technical solution of the embodiment of this application, the user can control the prompt to accelerate and decelerate the prompt according to the actual situation, or go back to the previous sentence, skip a sentence, etc. The wearable device AR glasses can accurately provide the speaker or singer with any content. The teleprompter information improves the convenience of the teleprompter device. At the same time, the content of the teleprompter is projected on the inner screen of the AR glasses and cannot be viewed outside, which improves the privacy of use and protects the privacy of the speaker.

In the technical solution of the embodiment of the present application, the audio data recorded in advance or in real time is analyzed to detect the duration of each sentence or even each word or word by analyzing the amplitude change and frequency change of the human voice in the audio, and at the same time through the grammar and semantic recognition The corresponding text is transcribed. After the recognition is completed, the text and time information will be assembled into an lrc file, which simplifies the difficulty of prompting content and reduces the time for manual production. Users can synchronize lrc files to AR glasses through smartphones and other terminal devices during formal speeches or concerts, which solves the problem of clumsy and difficult deployment of teleprompters. According to the time information in the lrc file, the AR glasses will synchronously prompt the text content on the display screen of the AR glasses to prevent the leakage of the prompt content and the perception of others, and automatically fade or highlight the reminder according to the speed and order during display, and more In order to conveniently prompt the user to adjust the speech rate and intonation of the speech or singing, at the same time, the user can control the prompt speed and progress through the mobile phone or AR glasses device during the prompt process.

Embodiment two

Fig. 3 is a schematic structural diagram of a prompting device in an embodiment of the present application. This embodiment is applicable to the situation of prompting, and the device can be realized by software and/or hardware, and the device can be integrated in any device that provides a prompting function, as shown in Figure 3, the prompting device specifically includes: A first acquisition module 201 , a first determination module 202 , a second acquisition module 203 , a second determination module 204 and an acquisition and prompt module 205 .

Wherein, the first acquisition module 201 is used to acquire the historical audio data of the target object;

The first determination module 202 is configured to determine the speech rate text of the target object according to the historical audio data;

The second obtaining module 203 is used to obtain the real-time audio data of the target object;

The second determination module 204 is configured to determine the position information of the text corresponding to the real-time audio data in the speech rate text of the target object;

The obtaining and prompting module 205 is configured to determine the content to be prompted according to the location information and the speech rate text of the target object, and display the content to be prompted through the prompt display screen.

In some embodiments, the obtaining and prompting module 205 includes:

a first acquisition unit, configured to acquire the original text corresponding to the historical audio data;

The first determining unit is configured to determine the content to be prompted according to the location information, the original text, and the speech rate text of the target object.

In some embodiments, the first determination module 202 includes:

a second acquiring unit, configured to acquire audio amplitude information and audio frequency information of the historical audio data;

An identification unit, configured to identify the audio amplitude information and audio frequency information, and obtain text and speech rate information corresponding to the historical audio data;

The second determination unit is configured to determine the speech rate text of the target object according to the text and speech rate information corresponding to the historical audio data.

In some embodiments, the speech rate information includes: at least one of the duration corresponding to each character, the duration corresponding to each word, the duration corresponding to each sentence, and the start time corresponding to each sentence .

In some embodiments, the first determination unit includes:

a first sub-determining unit, configured to determine a character to be pronounced according to the location information and the original text;

The second sub-determining unit is configured to determine the prompting mode of the text to be pronounced according to the speech rate text of the target object.

In some embodiments, the prompting method includes: using at least one of a cursor, a highlight and a marquee to prompt the text to be pronounced.

In some embodiments, the second sub-determining unit is specifically configured to:

Determine the duration corresponding to each word in the text to be pronounced and the interval between adjacent words according to the speech rate text of the target object;

The prompt mode of the to-be-pronounced text is determined according to the duration corresponding to each character in the to-be-pronounced text and the interval time between adjacent words.

In some embodiments, the prompting device also includes:

A receiving module, configured to receive a control instruction for the currently displayed content after displaying the content to be prompted through the teleprompter display;

An update module, configured to update the currently displayed content according to the control instruction, the currently displayed content, and the speech rate text of the target object after displaying the content to be prompted through the teleprompter display screen, including :

Determining that the control instruction is a playback speed adjustment instruction;

Obtaining the speed adjustment parameter corresponding to the playback speed adjustment instruction and the current playback speed;

determining a target playback speed according to the speed adjustment parameter and the current playback speed;

The currently displayed content is updated according to the target playback speed, the currently displayed content, and the speech rate text of the target object.

The above-mentioned product can execute the prompting method provided by any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the prompting method.

Embodiment Three

FIG. 4 shows a schematic structural diagram of an electronic device 30 that can be used to implement the embodiments of the present application. Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (eg, helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the applications described and/or claimed herein.

In some embodiments, the electronic device is a wearable device worn by the target object, and the teleprompter display screen is a screen generated by an optical machine of the wearable device. In one embodiment, the electronic device may be a wearable device such as head-mounted AR glasses, and the teleprompter display screen may be, for example, an AR optical machine (that is, a display and optical system) of a wearable device such as head-mounted AR glasses. The screen produced by the projection device integrated together.

As shown in FIG. 4 , the electronic device 30 includes at least one processor 31, and a memory connected in communication with the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores There is a computer program executable by at least one processor, and the processor 31 can operate according to a computer program stored in a read-only memory (ROM) 32 or loaded from a storage unit 38 into a random access memory (RAM) 33. Various appropriate actions and processes are performed. In the RAM 33, various programs and data necessary for the operation of the electronic device 30 are also stored. The processor 31 , ROM 32 and RAM 33 are connected to each other through a bus 34 . An input/output (I/O) interface 35 is also connected to the bus 34 .

Multiple components in the electronic device 30 are connected to the I/O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a magnetic disk, an optical disk etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 39 allows the electronic device 30 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Processor 31 may be various general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 31 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 31 executes the various methods and processes described above, such as the prompting method:

Obtain historical audio data of the target object;

determining the speech rate text of the target object according to the historical audio data;

Acquiring real-time audio data of the target object;

Determining the position information of the text corresponding to the real-time audio data in the speech rate text of the target object;

The content to be prompted is determined according to the location information and the speech rate text of the target object, and the content to be prompted is displayed on a prompter display screen.

In some embodiments, the prompting method can be implemented as a computer program, which is tangibly embodied in a computer-readable storage medium, such as the storage unit 38 . In some embodiments, part or all of the computer program may be loaded and/or installed on the electronic device 30 via the ROM 32 and/or the communication unit 39 . When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the prompting method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured in any other suitable manner (for example, by means of firmware) to execute the prompting method.

Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that the computer program causes the functions/operations specified in the flowcharts and/or block diagrams to be implemented when executed by the processor. A computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus or device. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

In order to provide interaction with the user, the systems and techniques described herein can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display)) for displaying information to the user. monitor); and a keyboard and pointing device (eg, a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

A computing system can include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the problems of difficult management and weak business expansion in traditional physical hosts and VPS services. defect.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the various steps described in this application may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution of this application can be achieved, there is no limitation herein.

The above specific implementation methods are not intended to limit the protection scope of the present application. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims (10)

1. A method of prompting, comprising:

acquiring historical audio data of a target object;

determining the speech rate text of the target object according to the historical audio data;

acquiring real-time audio data of the target object;

determining the position information of the text corresponding to the real-time audio data in the speech speed text of the target object;

and determining the content to be prompted according to the position information and the speech speed text of the target object, and displaying the content to be prompted through a prompter display screen.

2. The method of claim 1, wherein determining content to be prompted based on the location information and the speech rate text of the target object comprises:

acquiring an original text corresponding to the historical audio data;

and determining the content to be prompted according to the position information, the original text and the speech speed text of the target object.

3. The method of claim 2, wherein determining the speech rate text of the target object from the historical audio data comprises:

acquiring audio amplitude information and audio frequency information of the historical audio data;

identifying the audio amplitude information and the audio frequency information to obtain text and speech rate information corresponding to the historical audio data;

and determining the speech rate text of the target object according to the text and the speech rate information corresponding to the historical audio data.

4. The method of claim 3, wherein the speech rate information comprises: at least one of a duration corresponding to each word, a duration corresponding to each sentence, and a start time corresponding to each sentence.

5. The method of claim 2, wherein determining content to be prompted based on the location information, the original text, and the pace text of the target object comprises:

Determining the text to be pronounced according to the position information and the original text;

and determining the prompting mode of the words to be pronounced according to the speech speed text of the target object.

6. The method of claim 5, wherein the prompting means comprises: the text to be pronounced is prompted with at least one of a cursor, a highlight, and a ticker.

7. The method of claim 5, wherein determining the alert mode of the word to be pronounced based on the pace text of the target object comprises:

determining the duration time corresponding to each word in the words to be pronounced and the interval time between adjacent words according to the speech rate text of the target object;

and determining the prompting mode of the word to be pronounced according to the duration time corresponding to each word in the word to be pronounced and the interval time between adjacent words.

8. The method as recited in claim 1, further comprising:

receiving a control instruction for the current display content;

updating the current display content according to the control instruction, the current display content and the language speed text of the target object, including:

Determining the control instruction as a play speed adjustment instruction;

acquiring a speed adjustment parameter and a current playing speed corresponding to the playing speed adjustment instruction;

determining a target playing speed according to the speed adjusting parameter and the current playing speed;

and updating the current display content according to the target playing speed, the current display content and the language speed text of the target object.

9. An electronic device, the electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the prompting method of any one of claims 1-8.

10. The electronic device of claim 9, wherein the electronic device is a wearable device worn by a target object and the tagout display is a screen generated by an optical engine of the wearable device.