CN105388786B - A kind of intelligent marionette idol control method - Google Patents

Abstract

The invention discloses an intelligent string puppet control method. The intelligent string puppet control method includes: playing audio, accompanied by body movements; sending commands through voice to realize basic action control and oral communication of the puppet; Instructions to ask questions, search out the answers online and feed back the results; real-time remote monitoring, monitoring and protection in hidden places. The present invention combines the marionette and the preschool education escort robot, realizes the interaction by using the mobile phone, the tablet computer or the client of the computer, establishes the close contact between the parents and the infant, and realizes the early education and escort of the infant more appropriately; provides an open puppet Exclusive customized services make escorts more personalized; introducing national non-material culture into the family has invisibly edified the excellent traditional Chinese culture for users, and helps to cultivate Chinese sons and daughters with more sound personalities.

Description

An intelligent string puppet control method

technical field

The invention belongs to the technical field of accompanying early education, and in particular relates to an intelligent marionette control method.

Background technique

In recent years, with the continuous growth of my country's population, the resulting demographic dividend has disappeared, the fertility rate has decreased, birth sex ratio imbalance, population aging and a series of socio-economic problems have gradually emerged. In order to solve a series of problems brought about by my country's population It is imminent to adjust the population policy, especially the birth policy, among which the open two-child policy came into being. With the gradual opening of the two-child policy, the number of infants and young children in our country is about to usher in a small peak. And with the continuous improvement of people's living standards in our country, parents pay more and more attention to children's physical and mental development. More and more parents realize the importance of early education and family care for infants. According to statistics, more than 95% of parents believe that early education is important for children. However, due to busy work, business trips and high economic pressure, most parents do not have time to provide early education for their children or accompany their children, which affects their physical and mental development. The problem is getting worse. At present, the most popular early education products for families are early education institutions, educational books and electronic audio-visual products. Due to the limitations of the product itself, more and more parents choose early childhood education toys and robots.

Contents of the invention

The present invention combines a marionette with a preschool education escort robot, and aims to provide an intelligent marionette control method, aiming at solving the problem of single function and low intelligence of existing early education products.

The present invention is achieved in this way, an intelligent string puppet control method, the intelligent string puppet control method includes:

Play audio, accompanied by body movements;

Send commands by voice to realize the basic action control and oral communication of the puppet;

According to the instruction of the question, after searching the answer online, the result will be fed back;

Real-time remote monitoring, monitoring and protection in hidden places;

Use the mobile phone, tablet computer or computer client to leave a message to achieve interaction.

Further, the playback of the audio is decoded by MediaPlayer and the audio file is displayed using SurfaceView. When using MediaPlayer to decode and play the audio, the corresponding action command at the moment can be obtained from the progress of the audio playback, and the progress can be sent by dragging the progress to a certain position. Corresponding action instructions, the puppet makes corresponding actions according to the action instructions;

The sending commands by voice to realize the basic action control and oral communication of the puppet is divided into two parts: voice recognition and voice synthesis;

Use the speech recognition interface, the question answering interface and the speech synthesis interface to process the results and generate speech; first use the speech recognition interface to recognize the voice input, use the provided question answering interface to obtain intelligent answers from the recognition results, and then call the speech generation interface to process the returned results to generate speech, and get Voice answer.

Further, the remote monitoring is divided into three parts: the monitoring end, the monitored end and the intermediate server; when the application of the monitored end is started, the instruction is obtained from the heartbeat module, if it is an opening instruction, the camera is started in the background, and the video is sent to the intermediate server stream; if it is a close command, stop sending video streams to the intermediate server; each monitored terminal corresponds to a unique UUID to identify the device, and the video stream it sends is identified by this UUID, when the monitoring terminal wants to monitor , log in to the intermediate server and select the monitoring terminal, and then click Monitor to start receiving video streams for real-time monitoring, and modify the monitoring status of the monitoring terminal to monitoring.

Further, the heartbeat module maintains a data table in the database of the server, which stores the status of each device, and the data items include device UUID, online (0/1), monitoring (0/1), information (0/1), 0 means offline, unmonitored and no information; 1 means online, monitoring and with information, the heartbeat module reads the device status table, and encapsulates the data into a JSON packet for sending.

Further, the client leaves a message using a mobile phone, a tablet computer or a computer to realize interaction and use audiorecord to record the voice, and the background uses the HTTP protocol to upload to the server, and the server is forwarded and sent to the destination; after the destination receives the information, download the voice to the Local database, users can view and play voice;

The client message using a mobile phone, a tablet computer or a computer specifically includes:

The sender sends the information to the server, and the server saves the sender's information, information content and recipient's address. The recipient's address has been converted to the UUID of the device by default, uniquely identifies a recipient, and modifies the recipient's status The information status bit of the table is 1;

The recipient uses the heartbeat module to detect the status of the status table, and when there is information detected, the recipient is reminded to receive;

After receiving the information reminder, the recipient makes an information receiving request to the server, the server queries the relevant information content, and encapsulates it as a JSON package and sends it to the recipient, the application parses the JSON package, stores the data in the local database, and the sender Send information to the server and modify state table information.

Further, download or search for local audio files from the Internet, set the background music, and use the action arrangement method to arrange the program. After the arrangement is completed, the corresponding action instructions and the composite file of the audio file corresponding to each instruction are encapsulated into a JSON package, sent to the server for shared use;

The background music acquisition network resource access process is as follows: firstly send an HTTP request to the server, the server sends back a JSON packet, the client parses the JSON packet and accesses the server resource, and then downloads it.

Another object of the present invention is to provide a control system for the smart marionette control method, the control system includes:

The playback module is used to play audio and accompany body movements;

The communication module is used to send commands through voice to realize the basic movement control and oral communication of the puppet;

The feedback module is used to feed back the results after the answers are searched online according to the instructions of the questions;

The remote monitoring module is used for real-time remote monitoring and monitoring and protection in hidden places;

The interactive module is used to realize interaction by using a mobile phone, a tablet computer or a computer client to leave a message.

Further, the communication module includes:

a speech recognition unit for recognizing speech input using a speech recognition interface;

The speech synthesis unit is used to use the provided question-and-answer interface to obtain intelligent answers from the recognition results, and then call the speech generation interface to process the returned results to generate speech and obtain speech answers.

Further, the remote monitoring module includes: a monitoring terminal, a monitored terminal and an intermediate server;

The monitoring terminal and the monitored terminal communicate with the intermediate server.

Further, the interactive module further includes:

Information status unit, the sender sends the information to the server, the server saves the sender information, information content and recipient address, the recipient address has been converted to the UUID of the device by default, uniquely identifies a recipient, modify The information status bit of the recipient status table is 1;

Information reminder unit, the recipient uses the heartbeat module to detect the status of the status table, and when there is information detected, the recipient is reminded to receive;

The status table information modification unit, after the recipient receives the information reminder, sends an information receiving request to the server, the server queries the relevant information content, and encapsulates it as a JSON package and sends it to the recipient, the application parses the JSON package, and stores the data in the The local database, the sender sends information to the server and modifies the state table information.

Compared with the prior art, the intelligent string puppet control method provided by the present invention has the following advantages:

1. Establish a close relationship between parents and children, and more closely realize the early education and companionship of children; realize the functions of early education and companionship for infants and young children more vividly, and give children a happier childhood while enhancing product functions; The combination of marionettes and voice quizzes breaks through the limitation that traditional marionettes require additional dubbing by professional actors. Traditional puppets realize the movements of the puppet's eyes, mouth, and limbs, and provide open puppet exclusive customization services to make the escort more personalized; combine toys with security guards to make the protection more concealed, and can stay behind when thieves invade More evidence, but also has a certain effect on alleviating the feeling of being monitored; the introduction of national non-material culture into the family has invisibly nurtured the excellent traditional Chinese culture for users, and helps to cultivate Chinese sons and daughters with more sound personalities.

2. The present invention can tell stories, Chinese studies, and English, and use the built-in early education software to play early education games. At the same time, it can push the learning content of professional early education institutions, and can also remotely transmit the parents' educational voice and video. The real meaning It realizes the early education of young children, which is rich in performance and appreciation.

3. The present invention has a built-in speech recognition function. On the one hand, this function can be used to realize the action control of the puppet in the present invention. On the other hand, according to this function, audio and video files such as songs and stories can be downloaded online for playback. At the same time, this function can be used to Search the questioner's answer online for man-machine dialogue; with voice transmission function, the image and video collected by the built-in microphone are transmitted to the supporting mobile phone and tablet client software for remote monitoring.

4. The present invention has the general function of a network camera, and can transmit the images and videos collected by the built-in camera to the matching mobile phone and tablet computer client software for remote monitoring.

5. The present invention uses the three-dimensional simulation doll to edit the repertoire of the marionette, and only needs to drag the three-dimensional simulation doll on the touch screen to realize the control of the marionette.

Description of drawings

Fig. 1 is a schematic diagram of the workflow of the monitored terminal provided by the embodiment of the present invention.

Fig. 2 is a schematic diagram of the workflow of the monitoring terminal provided by the embodiment of the present invention.

FIG. 3 is a schematic diagram of a voice intercom interaction process provided by an embodiment of the present invention.

Fig. 4 is a schematic diagram of a recipient's workflow in communication provided by an embodiment of the present invention.

Fig. 5 is a flow chart of the action control flow for dolls through the 3D model on the application side provided by the embodiment of the present invention.

Fig. 6 is a schematic diagram of a network resource access flow provided by an embodiment of the present invention.

Fig. 7 is a flow chart of the implementation of the intelligent string puppet control method provided by the embodiment of the present invention.

FIG. 8 is a flow chart of implementing resource access provided by an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The application principle of the present invention will be described in detail below in conjunction with the accompanying drawings.

The functions realized by the intelligent string puppet control method of the present invention are as follows:

Can sing and dance: the present invention can tell stories, Chinese studies, and English to children through a large number of selected content to realize early education for infants and young children. While strengthening the effect of early education, use the fun of the product to accompany the happy childhood, indirectly reduce the time spent watching TV, playing games and playing mobile phones, and protect children's eyesight.

Audio playback: mainly through MediaPlayer decoding and using SurfaceView to display audio files. When using MediaPlayer to decode and play the audio, you can get the corresponding action command from the progress of the audio playback, drag the progress to a certain position, and then send the corresponding action command to the puppet, and the puppet will make corresponding actions according to the action command The action of the video is similar to the implementation process.

Voice control: In order to enhance the man-machine interaction, entertainment and interest of the present invention, a voice control function is specially added. The user only needs to send commands to the invention through voice to realize the basic action control of the puppet. After the product is connected to the Internet, it can realize the instruction control of the puppet singing and telling stories. The method of voice control is closer to real human interaction, which helps to cultivate children's cheerful and optimistic personality. The realization of voice control function is mainly divided into two parts: voice recognition and voice synthesis. Speech recognition and speech synthesis technology is a hot spot in technology research now. Compared with text communication, people often prefer verbal communication when communicating with robots and puppets, which makes the development of this technology in full swing. The present invention mainly uses the speech recognition interface, the question answering interface and the speech synthesis interface to process the results to generate speech. First, use the voice recognition interface to recognize the voice input, use the provided question-and-answer interface to obtain intelligent answers from the recognition results, and then call the voice generation interface to process the returned results to generate a voice and get a voice answer, so that people can speak a word and communicate with a puppet who speaks a language .

This module can not only be used to send commands by voice to instruct dolls to do corresponding tasks, but also can be used for English learning. Face-to-face oral communication is of great help to children's oral English learning, thereby improving children's interest in English learning.

Smart Q&A: The function realizes the encyclopedia, "100,000 whys" and chat functions. The user asks the puppet a question, and the puppet searches for the answer online and then feeds back the result to the user. For children whose parents are often busy outside and lack the company of Mufu, this invention can realize the simple educational function for children, and at the same time, the user can also interact with the puppet Conduct conversations and chats, avoiding the loneliness of child users and elderly users, and preventing depression.

Security Guard: Real-time remote monitoring through the camera, monitoring and protection in hidden places. On the one hand, family members and guests will not have the feeling of being watched, while thieves will not know that they are being watched, so they can better protect the family; on the other hand, when parents and children are not together, whether it is sleeping at night, going to work or When traveling on business, you can see the child's dynamics in real time, so that parents can feel at ease. To realize the real-time remote monitoring function, the present invention realizes by dividing the functional module into three parts: a monitoring terminal, a monitored terminal and an intermediate server. When the monitored application is started, it obtains the command from the heartbeat module. If it is an open command, the camera is started in the background and sends a video stream to the intermediate server; if it is a close command, it stops sending the video stream to the intermediate server. Each monitored terminal corresponds to a unique UUID to identify the device, and the video stream it sends is identified by this UUID. Therefore, the video stream corresponding to each monitored terminal is unique. When the monitoring terminal wants to monitor, log in to the intermediate server and select the monitoring terminal, then click Monitor to start receiving video streams to achieve real-time monitoring, and modify the monitoring status of the monitoring terminal to monitoring. Here, Nginx and RTMP are used to complete the transmission of video streams. The workflow of the monitored terminal is shown in Figure 1, including:

S101: start an application;

S102: Obtain instructions from the heartbeat module;

S103: If it is an opening command, start the camera in the background to execute S104, and if it is a closing instruction, execute S105;

S104: Send the video stream to the intermediate server;

S105: Stop sending the video stream.

The working flow chart of the monitoring terminal is shown in Figure 2, which specifically includes:

S201: Log in and select monitoring flow;

S202: Make a judgment on whether the monitoring flow exists, if it exists, execute step S203; if it does not exist, return to step S201;

S203: Click on the monitor and send an instruction;

S204: Modify the control state of the monitored terminal;

S205: Receive a surveillance video stream.

As shown in FIG. 7 , the implementation process of the intelligent string puppet control method according to the embodiment of the present invention includes the following steps:

S701: waiting to receive a function instruction;

S702: Make a judgment on whether to execute the instruction, if the instruction is executed, execute step S703; if the instruction is not executed, execute step S701;

S703: Make a judgment on whether it is a shutdown command, if it is a shutdown command, then end; if it is not a shutdown command, then execute step S704;

S704: Execute a function instruction;

S705: Make a judgment on whether the execution is completed; if the execution is completed, return to step S701; if not completed, return to step S704.

The realization of the real-time remote monitoring module also depends on the realization of a heartbeat module. The realization of Heartbeat module among the present invention, mainly is by maintaining a data table in the database of server, deposits the state of each equipment in this data table, and data item comprises equipment UUID, online (0/1), monitoring (0/1 ), information (0/1), 0 means offline, unmonitored and no information; 1 means online, monitoring and with information. The heartbeat module reads the device status table and encapsulates the data into JSON packets for sending.

By setting this table and checking the information of this table through heartbeat, it is convenient to monitor the status of the device in real time.

Voice intercom: As shown in Figure 3, parents can use the client of the mobile phone, tablet computer or computer to leave a message to the child through the present invention to care for the child or remind the child to do homework, etc., and the child can also directly leave a message to the parent through the present invention , parents can receive it after opening the client. Similarly, when the voice message is played, the puppet will make corresponding actions, no matter where it goes, the family members can communicate with each other with zero distance.

As shown in Figure 3, voice intercom specifically includes:

S301: recording voice;

S302: Upload to the server;

S303: read message request;

S304: Remind the arrival of the message and download the message to the local database;

S305: Remind the arrival of the message and download the message to the local database;

S306: read message request;

S307: parse the message;

S308: if the reply is otherwise, end; if the reply is yes, return to step S301;

S309: Analyzing the information downloaded in step S304;

S310: if the reply is otherwise, end; if the reply is yes, return to step S311;

S311: recording voice;

S312: Upload to the server.

In the present invention, parents or children use audiorecord to record the voice, and the background uses HTTP protocol to upload to the server, and the server transmits it to the destination after forwarding; after receiving the information, the destination downloads the voice to the local database, and the user checks and plays the voice.

The communication/message part of the present invention depends on the realization of the information interaction module, and the content of the message can be text or voice. Communication can be understood as a kind of instant message, and communication can be divided into two parts, sending and receiving. The two processes are only different in direction, and there is no difference in essence. Therefore, the following is only the process of sending messages through communication. Be explained. The sender sends the information to the server, and the server saves the sender’s information, information content and recipient’s address. The recipient’s address here has been converted to the UUID of the device by default, uniquely identifying a recipient, and modifying the recipient’s address. The information status bit of the person status table is 1. The recipient also constantly uses the heartbeat module to detect the status of the status table, and when there is information detected, the recipient is reminded to receive the mail. After receiving the information reminder, the recipient makes an information receiving request to the server. The server queries the relevant information content and encapsulates it into a JSON package and sends it to the recipient. The application parses the JSON package and stores the data in the local database. The sender sends information to the server and modifies the status table information. The recipient's workflow is shown in Figure 4, including:

S401: The heartbeat module detects information;

S402: Click to receive information;

S403: Receive and parse the JSON package returned by the server;

S404: saving the information content to a local database;

S405: Check the information content from the local database.

Custom program: built-in special program editing software for marionettes, using three-dimensional simulation puppet model, only need to drag the three-dimensional simulation puppet model on the touch screen to set the action to realize the puppet of the present invention to make corresponding actions, in addition, the voice The control function can assist in controlling the action of the puppet, and at the same time, you can add background music to complete the program arrangement, and upload the program you have arranged to the company website to share with others.

The application end of the present invention is equipped with a three-dimensional model that is the same as the actual puppet, and the puppet can be controlled by manually controlling the model, such as moving the arm of the model, and the puppet will move the arm in the same way. The 3D model information of the puppet is stored in an OBJ file, and the OBJ file is analyzed, parsed and displayed with OpenGL, so that whenever the user rotates part of the limbs of the model, it is converted into instructions according to the angle of rotation, and the limbs of the puppet are controlled through Bluetooth Turn accordingly. The use of this module can help realize the flexible control of puppet actions, and the control process is shown in Figure 5; specifically includes:

S501: Rotate the 3D model;

S502: Calculate the rotation angle;

S503: encapsulating as a control command;

S504: Controlling the puppet.

Download or search for local audio files from the Internet, set the background music, and use the above action arrangement method to arrange the program. After the arrangement is completed, the corresponding action instructions and the composite file of the audio file corresponding to each instruction are encapsulated into a JSON Package, sent to the server, shared for everyone to use. Among them, the acquisition of background music depends on the realization of the resource access module. The invention can realize the access of local resources and network resources. To access the local resources, it is only necessary to search and find the files for access; to access the network resources, it needs to be downloaded to the local area first, and then accessed.

The network resource access process is as follows: firstly, an HTTP request is sent to the server, the server sends back a JSON packet, the client parses the JSON packet and accesses the server resources, and then downloads, as shown in Figure 6 and Figure 8.

As shown in Figure 6, the specific steps include:

S601: Send a resource access request to the server;

S602: Receive the JSON packet returned by the server;

S603: Parse the JSON package and access server resources;

S604: Download the required resources to the local.

As shown in Figure 8, the specific steps include:

S801: Enter related functions of resource access;

S802: Determine whether it is a local resource, if it is a local resource, execute step S804; if it is not a local resource, execute step S803;

S803: Download the network resources to the local;

S804: Access local resources.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (5)

1. An intelligent string puppet control method, characterized in that, the intelligent string puppet control method comprises: Play audio, accompanied by body movements; Send commands by voice to realize the basic action control and oral communication of the puppet; According to the instruction of the question, after searching the answer online, the result will be fed back; Real-time remote monitoring, monitoring and protection in hidden places; Use the mobile phone, tablet computer or computer client to leave a message to achieve interaction; The audio is played through MediaPlayer decoding and using SurfaceView to display the audio file. When using MediaPlayer to decode and play the audio, obtain the corresponding action command at the moment from the audio playback progress, drag the progress to a certain position, and send the corresponding action command , according to the action instructions, the puppet makes corresponding actions; The sending commands by voice to realize the basic action control and oral communication of the puppet is divided into two parts: voice recognition and voice synthesis; Use the speech recognition interface, the question answering interface and the speech synthesis interface to process the results and generate speech; first use the speech recognition interface to recognize the voice input, use the provided question answering interface to obtain intelligent answers from the recognition results, and then call the speech generation interface to process the returned results to generate speech, and get voice response; The remote monitoring is divided into three parts: the monitoring terminal, the monitored terminal and the intermediate server; when the application of the monitored terminal is started, an instruction is obtained from the heartbeat (Heartbeat) module, and if it is an opening instruction, the camera is started in the background and sent to the intermediate server Video stream; if it is a close command, stop sending the video stream to the intermediate server; each monitored terminal corresponds to a unique UUID to identify the device, and the video stream it sends is identified by the UUID, when the monitoring terminal wants to perform When monitoring, log in to the intermediate server and select the monitoring terminal, then click Monitoring to start receiving video streams to achieve real-time monitoring, and modify the monitoring status of the monitoring terminal to monitoring. 2. the intelligent string puppet control method as claimed in claim 1, is characterized in that, described heartbeat module is by maintaining a data table in the database of server, deposits the state of each equipment in this data table, data Items include device UUID, online (0/1), monitoring (0/1), information (0/1), 0 means offline, unmonitored, and no information; 1 means online, monitoring, and information, read by the heartbeat module Get the device status table, and encapsulate the data into a JSON packet for sending. 3. The intelligent string puppet control method as claimed in claim 1, characterized in that, the client leaves a message using a mobile phone, a tablet computer or a computer, realizes interactive use of audiorecord to record voice, and the background uses HTTP protocol to upload to the server , the server sends it to the target end after forwarding; after the target end receives the information, it downloads the voice to the local database, and the user checks and plays the voice; The client message using a mobile phone, a tablet computer or a computer specifically includes: The sender sends the information to the server, and the server saves the sender's information, information content and recipient's address. The recipient's address has been converted to the UUID of the device by default, uniquely identifies a recipient, and modifies the recipient's status The information status bit of the table is 1; The recipient uses the heartbeat module to detect the status of the status table, and when there is information detected, the recipient is reminded to receive; After receiving the information reminder, the recipient makes an information receiving request to the server, the server queries the relevant information content, and encapsulates it as a JSON package and sends it to the recipient, the application parses the JSON package, stores the data in the local database, and the sender Send information to the server and modify state table information. 4. The intelligent string puppet control method as claimed in claim 3, characterized in that, downloading or searching local audio files from the network, setting background music, and using the action arrangement mode to carry out program arrangement, the user self-edits the action or Select an existing action from the action library, and after the arrangement is completed, package the corresponding action command and the composite file of the audio file corresponding to each command into a JSON package, send it to the server for shared use; The background music acquisition network resource access process is as follows: firstly send an HTTP request to the server, the server sends back a JSON packet, the client parses the JSON packet and accesses the server resource, and then downloads it. 5. A control system of an intelligent string puppet control method as claimed in claim 1, wherein said control system comprises: The playback module is used to play audio and accompany body movements; The communication module is used to send commands through voice to realize the basic movement control and oral communication of the puppet; The feedback module is used to feed back the results after the answers are searched online according to the instructions of the questions; The remote monitoring module is used for real-time remote monitoring and monitoring and protection in hidden places; The interactive module is used to use the mobile phone, tablet computer or computer client to leave a message to realize interaction; The communication module includes: a speech recognition unit for recognizing speech input using a speech recognition interface; The speech synthesis unit is used to use the provided question-and-answer interface to obtain intelligent answers from the recognition results, and then call the speech generation interface to process the returned results to generate speech and obtain speech answers; The remote monitoring module includes: a monitoring terminal, a monitored terminal and an intermediate server; The monitoring terminal and the monitored terminal communicate with the intermediate server; The interactive module further includes: Information status unit, the sender sends the information to the server, the server saves the sender information, information content and recipient address, the recipient address has been converted to the UUID of the device by default, uniquely identifies a recipient, modify The information status bit of the recipient status table is 1; Information reminder unit, the recipient uses the heartbeat module to detect the status of the status table, and when there is information detected, the recipient is reminded to receive; The status table information modification unit, after the recipient receives the information reminder, sends an information receiving request to the server, the server queries the relevant information content, and encapsulates it as a JSON package and sends it to the recipient, the application parses the JSON package, and stores the data in the The local database, the sender sends information to the server and modifies the state table information.