CN118170038A - A smart home control method, control model, storage medium and electronic device - Google Patents

Abstract

The present application relates to the field of smart home, and more specifically, to a smart home control method, storage medium and electronic device. The smart home control method of the present application includes: obtaining user's speech information; judging whether the device to be operated can be determined based on the speech information; if so, controlling the device to be operated to perform an operation adapted to the speech information; if not, obtaining the user's context information, and determining the device to be operated based on the speech information and the user's context information. The present application greatly reduces the misoperation of home device control and improves the user's experience in the process of using smart home.

Description

A smart home control method, control model, storage medium and electronic device

Technical Field

The present application relates to the field of smart home, and more specifically, to a smart home control method, storage medium and electronic device.

Background technique

Smart home is a system that uses residential housing as a platform and integrates facilities related to home life using integrated wiring technology, network communication technology, security technology, automatic control technology, and audio and video technology to build an efficient management system for residential facilities and family schedule affairs. It can connect various devices in the home (such as audio and video equipment, lighting systems, curtain control, air conditioning control, security systems, digital cinema systems, audio and video servers, video cabinet systems, network appliances, etc.) through the Internet of Things technology, and provide a variety of functions and means such as home appliance control, lighting control, telephone remote control, indoor and outdoor remote control, anti-theft alarm, environmental monitoring, HVAC control, infrared forwarding, and programmable timing control.

Smart homes have a wide range of applications and can be intelligently controlled and managed according to different scenarios and needs. For example, smart home systems can be used to remotely control home appliances, such as turning on or off air conditioners, televisions and other devices. In addition, smart homes can also realize functions such as smart lighting and smart security, such as automatically adjusting indoor lighting brightness according to light and time, and realizing real-time monitoring and alarms through security systems.

The emergence of smart home not only improves the safety, convenience, comfort and artistry of home, but also realizes an environmentally friendly and energy-saving living environment. It can not only save money for various energy costs, but also provides a full range of information interaction functions, making people's lives more intelligent and convenient.

In existing smart home systems, when users try to use voice assistants to control home appliances, if the user's instructions involve multiple devices, the system often cannot determine the specific device the user wants to operate. This directly leads to inconvenience in user operation and reduces user experience.

Traditional smart home control systems mostly operate based on specific instructions, such as "turn on the bedroom light". This method can work normally when the instructions are clear. However, once the user's instructions involve too many devices, or the instructions do not clearly specify the device to be operated, this system will not be able to correctly execute the user's commands. For example, in a home environment, there are multiple bedrooms, each with a light. The user only issues the command "turn on the bedroom light", but it is unclear which specific light is being turned on.

Current technology cannot accurately determine which device the user wants to operate in the control instructions of multiple electrical appliances. Once the instruction involves multiple devices, the system usually needs to go through multiple rounds of inquiries to determine the user's intention, which has a serious impact on the user experience. For example, when a user issues the instruction "turn on the bedroom light", when it is not clear which specific light is being referred to, it is necessary to ask questions such as "Is it the light in the master bedroom that is turned on?", "Is it the light on the desk in the master bedroom that is turned on?", etc. Multiple rounds of inquiries greatly reduce the efficiency of intelligent control. On the other hand, this solution is not only complicated and requires repeated clarification to the user, resulting in interactive devices, but also easily causes trouble to the user.

Application Contents

In order to overcome the above-mentioned defects, the present application proposes a smart home control method, control model, storage medium and electronic device, which greatly reduce the misoperation of home appliance control and enhance the user experience in the process of using smart home.

In a first aspect, the present application provides a smart home control method, comprising:

Get the user's speaking information;

Determining whether a device to be operated can be determined according to the speech information;

If yes, controlling the device to be operated to perform an operation adapted to the speaking information;

If not, the user's context information is obtained, and the device to be operated is determined according to the speech information and the user's context information.

In a technical solution of the above-mentioned smart home control method, the acquiring of the user's context information and determining the device to be operated according to the speech information and the user's context information comprises:

Acquire the language information of the user before the utterance information;

Determining whether a device to be operated can be determined according to the speech information and the language information;

If yes, the device to be operated is controlled to perform an operation adapted to the speaking information.

In a technical solution of the above-mentioned smart home control method, after the step of determining whether the device to be operated can be determined according to the speech information and the language information, the method further includes:

If the device to be operated cannot be determined according to the speech information and the language information, obtaining information of all devices, wherein all devices include the device to be operated;

Determine whether the device to be operated can be determined based on the speech information, language information and information of all devices;

If yes, the device to be operated is controlled to perform an operation adapted to the speaking information.

In a technical solution of the above-mentioned smart home control method, the judgment whether it is possible to

After the step of determining the device to be operated based on the information, language information and information of all devices, the method further includes:

If the device to be operated cannot be determined based on the speech information, language information and information of all devices, obtaining the user's usage habits;

Determine whether the device to be operated can be determined based on the speech information, language information, information of all devices and user usage habits;

If yes, the device to be operated is controlled to execute an action adapted to the speaking information.

In a technical solution of the above-mentioned smart home control method, the judgment whether it is possible to

After the language information, the information of all devices and the user's usage habits determine the device steps to be operated, the method further includes:

If the device to be operated cannot be determined based on the speech information, language information, information of all devices and the user's usage habits, then the user's environment information is obtained;

The device to be operated is determined according to the speech information, language information, information of all devices, user usage habits and user environment information.

In a technical solution of the above-mentioned smart home control method, after the step of obtaining the user's context information and determining the device to be operated according to the speech information and the user's context, the method further includes:

Get feedback from users.

In a second aspect, the present application provides a training method for a smart home control model, comprising:

Obtain the user's speech information, context information, and the device to be operated;

The control model to be trained is trained according to the speech information, situation information and the device to be operated of the user.

In a technical solution of the above training method, the training of the control model to be trained according to the user's speech information, situation information and operating device includes:

Inputting the user's speech information, situation information, and the device to be operated into the control model to be trained;

Converting the speech information and the situation information into feature vectors respectively by encoding;

The device to be operated is obtained by decoding the feature vector through a recurrent neural network;

Taking the device to be operated as a true value, calculating the loss of the device to be operated by using a preset loss function;

Back propagation is performed based on the loss to update the parameters of the control model to complete the current round of iteration.

In a third aspect, the present application provides a method for predicting a device to be operated using a trained control model, comprising:

Obtain the user's speech information and user context;

According to the trained control model, the speech information and the user's context information are input, and the predicted device to be operated is output.

In a fourth aspect, the present application provides a smart home control device, comprising a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the smart home control method or the training method of the smart home control method or the method of predicting the device to be operated.

In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein the smart home control method is executed when the program is run.

In a sixth aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute a smart home control method through the computer program.

The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

In the technical solution for implementing the present application, the present application combines the user's speech information with the user's situational information to comprehensively judge and determine the device to be operated. The present application can enhance the user's experience in the process of using the smart home, accurately determine the device to be operated to achieve device disambiguation, and greatly reduce the misoperation of home devices.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic flow chart of main steps of a smart home control method according to an embodiment of the present application;

2 is a schematic flow chart of the main steps of obtaining context information of a user and determining a device to be operated according to the speech information and the context information of the user according to an embodiment of the present application;

3 is a flow chart showing a step of determining a device to be operated according to the speech information, language information and information of all devices according to an embodiment of the present application;

4 is a flowchart of a step of determining a device to be operated according to speech information, language information, information of all devices and user usage habits according to an embodiment of the present application;

5 is a flowchart of a step of determining a device to be operated according to the speech information, language information, information of all devices, user usage habits, and user environment information according to an embodiment of the present application;

FIG6 is a flow chart showing a step of obtaining user feedback according to an embodiment of the present application;

FIG7 is a flow chart showing the main steps of a method for training a smart home control model according to an embodiment of the present application;

8 is a schematic flow chart of the main steps of training a control model to be trained according to the user's speech information, context information and operating device according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a hardware environment of an interaction method of a smart device according to an embodiment of the present application.

Detailed ways

In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatuses.

The present application provides a smart home control method, referring to FIG1 , including:

S1, obtain the user's speaking information;

S2, determining whether the device to be operated can be determined according to the voice information;

If yes, controlling the device to be operated to perform an operation adapted to the speaking information;

If not, the user's context information is obtained, and the device to be operated is determined according to the speech information and the user's context information.

Smart home refers to connecting various devices in the home through the Internet, and realizing the automatic control of home devices through the application or voice control of smart terminal devices, including smart lighting, smart speakers, smart TVs, smart refrigerators, smart air conditioners, smart security systems, etc. With the continuous development of technology, the smart home market is also constantly improving, and various smart home devices are becoming more and more popular.

If the user's command (speech information) involves multiple devices, the system often cannot determine the specific device the user wants to operate. For example, if the user says "turn on the light", because there are multiple lights in the room, it does not know which specific light should be turned on. If the smart home system can only determine the user's intention through multiple rounds of inquiries, this will have a serious impact on the user's experience. Therefore, in this application, after obtaining the user's speech information, no further inquiries are made. Instead, when the obtained user speech information cannot determine the device to be operated, the speech information is combined with the user's context information, and the device to be operated is determined after analysis.

The present application abandons the process of multiple rounds of questioning of the user, and adopts the method of analyzing and judging the user's context information in combination with the user's spoken information to draw a conclusion on the device to be operated.

The scenario includes (specific occasion) situations and scenes. In this application, it refers to the contextual language environment in which the user issues instructions in a home environment, the user's preferences and usage habits for the device, the status of home appliances, and the indoor and outdoor environment of the room.

By adding the user's context information, the corresponding device to be operated in the user's speaking information can be analyzed more accurately.

In one embodiment, obtaining the user's speech information includes: obtaining the user's control instructions and/or identifying the user's expectations.

The user issues a control request, such as "turn on the air conditioner". After the user's voice information acquisition device, such as a voice assistant, receives the request, it converts the voice command into text through voice recognition technology and analyzes the semantics to realize command reception.

In an application scenario, a user's control command is obtained, such as the user saying "turn on the air conditioner", "turn on the bedroom light", etc. "Turn on the air conditioner" means that the user requires the air conditioner to be turned on, and "turn on the bedroom light" means that the user requires the bedroom light to be turned on.

In this case, if it is not clear which specific device is being targeted based only on the user's spoken information, such as not knowing which air conditioner or light to turn on, further analysis and judgment is performed in combination with the user's context information.

In an application scenario, the user's expectations are identified, such as when the user says: "Adjust the light" or "The room is too cold". "Adjust the light" means that the user wants to increase the brightness of the room, and "The room is too cold" means that the user wants to increase the temperature of the room.

In this case, if it is unclear which specific device operation is being performed based solely on the user's spoken information, such as dimming the light and not knowing whether to turn on the light or open the curtains, further analysis and judgment will be required based on the user's situational information.

Or, when a user says "it's too cold in the room", it is not clear which specific device is being operated based on the user's spoken information. Therefore, further analysis and judgment based on the user's situational information is needed to determine whether to turn on the air conditioner or the heater.

In one embodiment, referring to FIG. 2 , the acquiring of the user's context information and determining the device to be operated according to the speech information and the user's context information includes:

S2-1, obtaining the language information of the user before the speaking information;

Specifically, the user's speech content is obtained through speech recognition and natural language processing, and the

Output the key information of the corresponding device;

S2-2, determining whether the device to be operated can be determined based on the speech information and the language information;

If yes, the device to be operated is controlled to perform an operation adapted to the speaking information.

In this embodiment, not only the user's speech information is collected this time, but also the speech information before the speech information is collected. In this way, combined with the context, especially the previous speech information, the device to be operated corresponding to the current speech information can be analyzed.

In an application scenario, the user's voice message is "turn on the light". After hearing this command, the smart home device cannot determine which light to turn on, so it combines the information before the user speaks "turn on the light" and obtains the language information before the voice message, which is "It's 7 o'clock, time to cook dinner, prepare to cut vegetables first." According to the above language information, it can be known that the user is cooking in the kitchen, so it is very likely that the kitchen light is turned on, so the kitchen light is controlled to turn on, without asking the user "Which light do you want to turn on?" according to the content of the prior art.

This embodiment collects a series of speech histories before the user's current speech to more accurately understand the user's current needs.

In one embodiment, referring to FIG. 3, the step S2-2 determines whether the calling information can be used.

After determining the device to be operated according to the language information, the method further includes:

S2-3, if the device to be operated cannot be determined based on the speech information and the language information, then

Get the information of all devices, including the devices to be operated;

S2-4, determining whether the device to be operated can be determined based on the speech information, language information and information of all devices;

If yes, the device to be operated is controlled to perform an operation adapted to the speaking information.

The acquisition of information of all devices includes:

Get the capabilities, operating status, and location of all devices.

The information of all these devices is collected, and the device to be operated is one or more of these devices.

Specifically, the functions, operating status and location of these devices are collected. Specifically, the home appliance information includes the name, status (offline/online, on/off), function status (on/off), location (obtained through the device's binding location or Wi-Fi signal), and supported functions of each device.

In one application scenario, the user’s smart home includes smart air conditioners, smart lights, smart refrigerators, and smart heating stoves.

The smart air conditioners in the user's home include: master bedroom air conditioner, living room air conditioner and second bedroom air conditioner.

First, S1 is performed to obtain the user's speech information, and the user's speech content is "turn on the air conditioner";

However, it is not clear which air conditioner to turn on.

Then, it is determined whether the device to be operated can be determined based on the speech information and the language information. Before the user speaks the content, there is no other valid language information, so the device to be operated still cannot be determined.

So continue to judge, the information of all devices is obtained, and specifically the usage status and location information of all air conditioners in the user's home are judged, and the result is: the air conditioner in the master bedroom is turned on, the air conditioner in the second bedroom is turned on, and the air conditioner in the living room is turned on.

Based on the positions and on states of the three air conditioners, it can be inferred that the air conditioner to be operated corresponding to the user's command “turn on the air conditioner” is the air conditioner in the living room.

In one application scenario, the user's smart home includes a smart refrigerator, smart lights, smart fans, and smart heaters.

First, S1 is performed to obtain the user's spoken information. The user's spoken content is "start heating"; however, it does not explicitly turn on the device. Then, it is determined whether the device to be operated can be determined based on the spoken information and the language information. Before the user's spoken content, there is no other valid language information, so the device to be operated cannot be determined.

So continue to judge, the information of all devices is obtained, and the result is:

Only a heater has the function of heating, so it can be inferred that the device to be operated corresponding to the user's command "start heating" is the smart heater.

In an application scenario, the user's smart home has a smart refrigerator, smart light, smart fan, and smart heater. First, S1 is performed to obtain the user's speech information. The user's speech content is "start self-cleaning"; however, it does not explicitly turn on the device. Then, it is determined whether the device to be operated can be determined based on the speech information and the language information. Before the user's speech content, there is no other valid language information, so the device to be operated cannot be determined.

So continue to judge, the information of all devices is obtained, and the result is:

Only air conditioners have the self-cleaning function, so it can be inferred that the device to be operated corresponding to the user's command "start self-cleaning" is the air conditioner.

In one embodiment, referring to FIG. 4 , after the step S2-4 of determining whether the device to be operated can be determined based on the speech information, language information and information of all devices, the method further includes:

S2-5, if the device to be operated cannot be determined based on the speech information, language information and information of all devices, obtaining the user's usage habits;

S2-6, determining whether the device to be operated can be determined based on the speech information, language information, information of all devices and user usage habits;

If yes, the device to be operated is controlled to execute an action adapted to the speaking information.

The acquisition of user usage habits specifically involves extracting the user usage habits based on historical data of the user's past operations on home devices.

Specifically, obtaining the user's usage habits includes:

Get the user's usage frequency of all devices respectively. The higher the usage frequency of a device, the higher the probability that the device will be controlled by the user. Based on the usage frequency of the device, the device with high usage frequency can be controlled preferentially.

This application provides a step-by-step judgment method. If the device to be operated cannot be determined based on the speech information, language information and information of all devices, the user's usage habits are obtained to help deduce and determine the device to be operated.

In an application scenario, when the user returns home at night, he wants to turn on the lights to illuminate the home, so he issues a command to the smart home system, "Turn on the lights."

First, S1 is performed to obtain the user's speech information, and the user's speech content is "turn on the light";

However, it is not clear to turn on the device and then determine whether the device to be operated can be determined based on the speaking information and the language information. Before the user speaks the content, there is no other valid language information, so the device to be operated still cannot be determined.

Continue to judge and obtain information of all devices, including all devices to be operated. In this step, the on status information of all lights in the home is obtained. The result is that none of the lights are turned on, so at this point, it is still impossible to determine the device to be operated.

Continue to judge and obtain user usage habits.

In this application scenario, the user's usage habit is obtained as follows: the frequency of controlling the lighting at the entrance is the highest, so it is inferred that the user's voice information command is likely to be the control of turning on the lighting at the entrance.

In one embodiment, referring to FIG. 5, in step S2-6, after determining whether the device to be operated can be determined based on the speech information, language information, information of all devices and user usage habits, the method further

include:

S2-7, if the device to be operated cannot be determined based on the speech information, language information, information of all devices and the user's usage habits, then obtaining the user's environment information;

S2-8, determining the device to be operated according to the speech information, language information, information of all devices, user usage habits and user environment information.

In one embodiment, in S2-7, obtaining the user's environment information includes:

Obtain the user's indoor and/or outdoor environment information.

The step S2-8, determining the device to be operated according to the speech information, language information, information of all devices, user usage habits and user environment information, includes:

The device to be operated is determined according to the speech information, language information, information of all devices, user usage habits, and indoor and/or outdoor environment information where the user is located.

In one embodiment, obtaining the indoor and/or outdoor environment information of the user includes:

Obtain the user's indoor temperature, humidity, light intensity, air quality, weather and season, and/or

Get the user's outdoor temperature, humidity, light intensity, air quality, weather and season. Indoor environment information: such as temperature, humidity, air quality, etc., can be obtained by indoor sensors.

Outdoor environmental information: such as geographic location, season, weather, temperature, humidity, etc., can be obtained from Internet weather services.

In one application scenario, the user sends a message saying "The room is too dark".

The user's expectation is to turn up the light in the room. So the language information, the information of all devices and the user's usage habits are not able to determine the device to be operated. So the environment is judged. The weather is sunny and the season is spring. Combined with the device information, the curtains are in the state of covering the windows, the lights are in the state of off, the indoor light intensity is weak, and the outdoor light intensity is strong. After comprehensive judgment, it is analyzed that controlling the curtains to open and letting the outdoor light shine into the room can increase the indoor light and save energy and protect the environment. In this application scenario, it is finally determined that the device to be operated is the smart curtain, and the smart curtain is controlled to open.

In the above application scenario, if the weather is rainy and the season is autumn, combined with the device information, the curtains are in the state of covering the windows, the lights are in the state of being off, the indoor light intensity is weak, and the outdoor light intensity is also weak.

After comprehensive judgment, it is concluded that in this application scenario, the intelligent lighting of the device to be operated is finally determined.

Control the smart lighting to turn on.

In one embodiment, referring to FIG. 6 , in S2, after determining whether the device to be operated can be determined based on the speech information, the method further includes:

S3, obtain user feedback.

In an application scenario, when the user returns home at night, he wants to turn on the lights to illuminate the home, so he issues a command to the smart home system, "Turn on the lights."

First, S1 is performed to obtain the user's speech information, and the user's speech content is "turn on the light";

However, it is not clear to turn on the device and then determine whether the device to be operated can be determined based on the speaking information and the language information. Before the user speaks the content, there is no other valid language information, so the device to be operated cannot be determined.

Continue to judge and obtain information of all devices, including the device to be operated. In this step, the on status information of all lights in the home is obtained. The result is that none of the lights are turned on, so at this point, it is still impossible to determine the device to be operated.

Continue to judge and obtain user usage habits.

In this application scenario, the user's usage habit is obtained as follows: the frequency of controlling the lighting at the entrance is the highest, so it is inferred that the user's voice information command is likely to be the control of turning on the lighting at the entrance.

After controlling the lighting at the entrance, if the user feedback is "not this light", the user habit will be adjusted based on the feedback.

The present application also provides a training method for a smart home control model, referring to FIG. 7 , comprising:

Obtain the user's speech information, context information, and the device to be operated;

The control model to be trained is trained according to the speech information, situation information and the device to be operated of the user.

The known data: speech information, situation information and the to-be-operated devices corresponding to the speech information and situation information are used as sample data to train the control model to be trained.

In one embodiment, referring to FIG. 8 , the training of the control model to be trained according to the user's speech information, context information, and operation device includes:

Inputting the user's speech information, situation information, and the device to be operated into the control model to be trained;

Converting the speech information and the situation information into a format suitable for model processing, such as converting them into feature vectors by encoding and then normalizing the converted data;

The device to be operated is obtained by decoding the feature vector through a recurrent neural network. In one application scenario, the recurrent neural network is a recurrent neural network (RNN). The characteristic of RNN is that it can process sequence data, so it is particularly suitable for processing data with time series, such as the user's speech history and usage habits.

Taking the device to be operated as a true value, calculating the loss of the device to be operated by using a preset loss function;

Back propagation is performed based on the loss to update the parameters of the control model to complete the current round of iteration.

The present application also proposes a method for predicting a device to be operated using the above-mentioned trained control model, including:

Obtain the user's speech information and user context;

According to the trained control model, the speech information and the user's context information are input, and the predicted device to be operated is output.

The model outputs the device with the highest probability.

The model can also output the probability corresponding to each possible device, and the device with the highest probability is considered to be the device that the user expects to operate.

The control model established in the present application can be considered as a device disambiguation model, because after the model is applied, the device to be operated can be determined to avoid misunderstanding.

After the model determines the device that the user expects to operate, it generates the corresponding control command and sends it to the device via a wireless network (such as Wi-Fi, Bluetooth, 5G, etc.) to allow the device to perform the corresponding operation.

If the user has feedback on the operation of the system, for example, "I'm not talking about this air conditioner", this feedback can be used to readjust the model weights and perform online learning.

1. This application combines the user's speech information with the user's context information to comprehensively determine the device to be operated.

2. This application uses deep learning models or rule engines to predict the equipment to be operated;

3. This application generates device control commands based on the prediction results, which can effectively avoid the troubles caused by the failure of device disambiguation, reduce invalid interactions, and enhance the user experience in the process of using smart homes. Accurate device disambiguation can greatly reduce misoperation of the home.

The present application also proposes a smart home control device, including a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the smart home control method or the training method of the smart home control method described in the execution item or the method for predicting the device to be operated.

In an embodiment of a control device according to the present application, the control device includes a processor and a storage device, the storage device can be configured to store a program for executing the training method of the smart home control method of the above method embodiment or the method for predicting the device to be operated, and the processor can be configured to execute the program in the storage device, which includes but is not limited to executing the training method of the smart home control method of the above method embodiment or the method for predicting the device to be operated. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The control device can be a control device device formed by various electronic devices.

It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

Furthermore, the present application also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium can be configured to store a program for executing the sample data acquisition method of the above-mentioned method embodiment or the training method of the poster generation model or the poster generation method, and the program can be loaded and run by the processor to implement the training method of the above-mentioned smart home control method or the method of predicting the device to be operated. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present application, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

It is understood by those skilled in the art that each module in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of this application, and therefore, the technical solutions after splitting or merging will fall within the protection scope of this application.

So far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

According to one aspect of an embodiment of the present application, a method for interacting with a smart home device is provided. The method for interacting with a smart home device is widely used in whole-house intelligent digital control application scenarios such as smart home, smart home, smart home device ecology, and smart house ecology.

The present application provides an interactive system for smart home devices, including:

A user speech information acquisition device, used to acquire the user's speech information, such as a speaker, a voice assistant of a mobile terminal, etc.;

The server is used to receive the speaking information of the user and determine the device to be operated according to the speaking information and the context information of the user.

The terminal device is used to perform an operation adapted to the speaking information according to the device to be operated determined by the server.

The server is also called the cloud, and the terminal device is connected to the cloud (server) through the network. The user speech information acquisition device is connected to the cloud (server) through the network. The above network may include but is not limited to at least one of the following: wired network, wireless network. The above wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the above wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth.

After the cloud determines the device to be operated, it controls the corresponding device to perform the operation adapted to the speech information.

In this embodiment, the above-mentioned interaction method of smart home devices can be applied to the hardware environment composed of the terminal device 102 and the server 104 as shown in FIG9. As shown in FIG9, the server 104 is connected to the terminal device 102 through a network, and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal, and a database or cloud can be set on the server or independently of the server to provide data storage services or data analysis services for the server 104, and cloud computing and/or edge computing services can be configured on the server or independently of the server to provide data computing services or data analysis services for the server 104.

The terminal device 102 may be but is not limited to a PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projection equipment, smart TV, smart clothes drying rack, smart curtains, smart audio and video, smart socket, smart audio, smart speaker, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart sweeping robot, smart window cleaning robot, smart mopping robot, smart air purification equipment, smart steamer, smart microwave oven, smart kitchen treasure, smart purifier, smart water dispenser, smart door lock, etc.

The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims (10)

1. The intelligent home control method is characterized by comprising the following steps of:

acquiring the speech information of a user;

judging whether the equipment to be operated can be determined according to the speaking information;

If yes, controlling the equipment to be operated to execute the operation adapted to the speaking information;

if not, acquiring the context information of the user, and determining the equipment to be operated according to the speech information and the context information of the user.

2. The method of claim 1, wherein the obtaining context information of the user, and determining the device to be operated according to the speech information and the context information of the user comprises:

Acquiring language information of a user before the speech information;

judging whether the equipment to be operated can be determined according to the speaking information and the language information;

if yes, controlling the equipment to be operated to execute the operation adapted to the speaking information.

3. The method according to claim 2, wherein after the step of determining whether the device to be operated can be determined based on the speech information and the language information, the method further comprises:

If the equipment to be operated cannot be determined according to the speaking information and the language information, acquiring information of all equipment, wherein all equipment comprises the equipment to be operated;

Judging whether the equipment to be operated can be determined according to the speech information, the language information and the information of all the equipment;

if yes, controlling the equipment to be operated to execute the operation adapted to the speaking information.

4. A method according to claim 3, wherein after the step of determining whether the device to be operated can be determined based on the speech information, language information, and information of all devices, the method further comprises:

If the equipment to be operated cannot be determined according to the speech information, the language information and the information of all the equipment, acquiring the use habit of the user;

Judging whether the equipment to be operated can be determined according to the speech information, the language information, the information of all the equipment and the use habit of the user;

if yes, controlling the equipment to be operated to execute the action adapted to the speaking information;

If the equipment to be operated cannot be determined according to the speech information, the language information, the information of all the equipment and the use habit of the user, the environment information of the user is acquired;

And determining the equipment to be operated according to the speech information, the language information, the information of all the equipment, the use habit of the user and the environmental information of the user.

5. The training method of the intelligent home control model is characterized by comprising the following steps of:

acquiring speech information, scenario information and equipment to be operated of a user;

And training the control model to be trained according to the speech information, the scenario information and the equipment to be operated of the user.

6. The training method according to claim 5, wherein training the control model to be trained according to the speech information, the scenario information, and the operation device of the user comprises:

Inputting the speech information, the scenario information and the equipment to be operated of the user into a control model to be trained;

respectively converting the speaking information and the scene information into feature vectors through coding;

The feature vector is decoded through a cyclic neural network to obtain equipment to be operated;

taking the equipment to be operated as a true value, and calculating loss of the equipment to be operated through a preset loss function;

and back-propagating based on the loss to update parameters of the control model, and completing current round iteration.

7. A method for predicting a device to be operated using the trained control model of any one of claims 5-6, comprising:

acquiring speech information of a user and a scene of the user;

the trained control model according to any one of claims 5-6 inputs speech information and user context information and outputs predicted equipment to be operated.

8. Smart home control apparatus comprising a processor and a storage means adapted to store a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by the processor to perform the smart home control method of any one of claims 1 to 4 or to perform the training method of the smart home control method of any one of claims 5 to 6 or the method of predicting a device to be operated of claim 7.

9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program when run performs the method of any one of claims 1 to 4.

10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method according to any of claims 1 to 4 by means of the computer program.