CN105700389B - Intelligent home natural language control method - Google Patents

Abstract

The invention discloses a natural language control method for an intelligent home, which comprises the following steps: (1) the voice receiving device receives a voice control command; (2) the control unit analyzes the command; (3) executing the command; in the step (2), the following sub-steps are further included: (21) analyzing the integrity of the voice control command, and directly entering the step (3) if the voice control command comprises the complete electric appliance type and the action command for controlling execution; otherwise, entering the step (22); (22) if the type of the electric appliance is not specified in the voice control command, the position of the user is located, the electric appliance which is closest to the user is found out according to the position of the user, the electric appliance is judged to be the specified electric appliance, and the step (3) is executed. The intelligent home natural language control method adopts a mode of indoor positioning for the user, and intelligently analyzes and judges the information implicitly contained in the incomplete voice control command of the user according to the position of the user, thereby improving the natural language recognition capability of the voice control system.

Description

A kind of intelligent home natural language control method

technical field

The invention relates to the technical field of smart homes, in particular to a natural language control method for smart homes.

Background technique

Smart home is the product of informationization and post-industrial age. In the post-industrial era, industry and informatization are closely integrated. Advanced computing methods such as computer vision, speech recognition, and artificial intelligence are widely used. Home appliances generally have primary artificial intelligence. Recognition is gradually replacing the remote control. Voice control is the most natural and convenient way to control. At present, the voice recognition functions of smart homes are scattered and scattered. For example, users can only speak the voice commands that the TV can understand to the TV, and the voice commands that the air conditioner can understand to the air conditioner. They need to remember the voice commands of different smart home appliances. This fragmented voice control system makes users feel a sense of uncertainty, users always use these new functions carefully, operate them with very simple and fixed verbs or phrases, and keep their eyes on whether the target appliance is in accordance with the instructions. Your own commands act, the whole process is full of confusion and no natural language interaction at all.

SUMMARY OF THE INVENTION

In order to solve the problem that the existing intelligent home voice control system does not support natural language control, the user's natural voice recognition and judgment ability is poor, resulting in inability to recognize and execute, or judgment errors cause execution errors, or the system will issue many confirmation prompts, instead giving The user brings many inconvenient problems, and a smart home natural language control method is proposed. The ability of the system to recognize natural language.

In order to solve the above-mentioned technical problems, the present invention adopts the following technical solutions to be realized:

A natural language control method for an intelligent home, comprising the following steps: (1), a voice receiving device receives a voice control command; (2), a control unit analyzes the command; (3), executes the command; in the step (2), Also includes the following sub-steps:

(21), analyze the integrity of the voice control command, if the voice control command contains the complete electrical appliance type and the action command to control the execution, then directly enter step (3); otherwise, enter step (22);

(22) If the type of electrical appliance is not specified in the voice control command, locate the user's location, find out the electrical appliance with the closest distance to the user according to the user's location, determine that the electrical appliance is the designated electrical appliance, and execute step (3).

Further, at the initial stage of the intelligent family natural language control method, it also includes the step of setting user authority, collecting the voice characteristics of different users of family members, assigning corresponding user authority levels to them, and establishing a user authority level table;

Before entering step (3) in the step (21), it also includes the step of analyzing the command conflict,

a. If a voice control command from a user with a different authority level is received at the same time or within a limited time, the voice control command with a higher authority level is executed;

b. If different voice control commands of users at the same authority level conflict with each other, determine whether the conflicting voice control commands come from the same user, and if they come from the same user, send a voice prompt for the user to confirm.

Still further, in the step b of analyzing the command conflict, if the conflicting voice control commands do not come from the same user, continue to detect and receive voice control commands, and if the conflicting voice control commands continue to appear frequently , then it is judged that there are differences of opinion between the two users, and the method of executing the command at a discount is used to delay the execution of the new voice control command, and each time it is executed, the delay time increases accordingly until the contradictory voice control no longer occurs. Order.

Still further, in the step b of analyzing the command conflict, if the conflicting voice control commands do not come from the same user, and the repetition frequency of the conflicting voice control commands is lower than the set limit, it is determined as One of the users has prank control, locate the user's location, find out the user closest to the controlled electrical appliance according to the user's location, and execute the user's voice control command.

Preferably, in the step (22), the method for locating the user's position is as follows: using IPV6 address positioning, binding the IPV6 address of each electrical appliance with its actual space address, and locating the IPV6 address of each voice receiving device The address is bound to the actual space address where it is located, and the location of the user is determined according to the IPV6 address of the voice receiving device that receives the voice control signal, and then the electrical appliance closest to the user's location is controlled to execute corresponding control commands.

Further, in the step (22), if multiple voice receiving devices receive the voice control command at the same time, then according to the strength of the voice signal received by each voice receiving device, compare the voice receiving device with the strongest received voice signal strength, The location of the user is determined by the IPV6 address of the voice receiving device.

Preferably, before the step (1), the step of presetting a trigger vocabulary for triggering the voice control operation is further included, and the step (1) or the subsequent steps are performed only when the user utters the voice of the trigger vocabulary.

Preferably, in the step (3), if the voice command does not contain specific parameter information, the user command is executed according to default parameters, and each electrical appliance is set with default parameters.

Compared with the prior art, the advantages and positive effects of the present invention are as follows: the smart home natural language control method of the present invention adopts the method of indoor positioning of the user, and intelligently analyzes and judges the user's incomplete voice control command according to the user's position. The implicitly contained information improves the voice control system’s ability to recognize natural language, and avoids the inconvenience caused by the system not performing actions due to inability to recognize or asking more confirmation questions for users to confirm due to inability to recognize accurately. problems, and help improve the user's enjoyment and feeling of using the smart home system.

Other features and advantages of the present invention will become more apparent upon reading the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a flowchart of an embodiment of a method for controlling natural language in a smart home proposed by the present invention;

FIG. 2 is a schematic diagram of discount execution steps of the natural language control method for a smart home proposed by the present invention.

Detailed ways

Voice control is the most natural and convenient control method, and it is also the hotspot of technological development in recent years. For example, the Siri voice control function of Apple mobile phones is considered to have opened a new round of artificial intelligence technology revolution. Through this intelligent voice system, users can read text messages, introduce restaurants, inquire about the weather, set alarm clocks by voice, etc. through their mobile phones. Siri can also call the weather forecast, schedule, search data and other applications that come with the system, and can continuously learn new sounds. and intonation, providing a conversational response.

The domestic voice technology manufacturer Keke launched the voice assistant "Lingxi". Say "Central set" to Lingxi, and the TV channel will automatically switch to the central set; say "Make me a cup of coffee" to Lingxi, the coffee machine will start to work; Yes Lingxi said, "Turn down the temperature by two degrees", and the air conditioner will start to adjust the temperature... At present, many home appliance manufacturers cooperate with Internet hardware manufacturers to explore the application of voice technology in smart home scenarios. Some simple voice control and recognition can already be realized, but In the smart home industry, due to the wide variety of controlled home appliances involved, combined with the current application environment controlled by the user, many scenarios and many possibilities will be involved, thus bringing great difficulties and difficulties to the speech recognition system. The challenge is that the current voice recognition control needs to issue clear and complete voice control commands to a specific controlled object, or can perform fuzzy recognition on some incomplete commands, but it cannot fully realize the recognition and judgment of the user's natural language. Based on the above, the present invention proposes a natural language control method for an intelligent home.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Embodiment 1, this embodiment proposes a natural language control method for a smart home, including the following steps:

S1, the voice receiving device receives the voice control command;

S2, the control unit analyzes the command;

S3. Execute the command;

In the step S2, the following sub-steps are also included:

S21, analyze the completeness of the voice control command, if the voice control command includes the complete electrical type and the action command to control execution, then directly enter step S3; otherwise, enter step S22;

S22. If the electrical appliance type is not specified in the voice control command, locate the user's location, find out the electrical appliance closest to the user according to the user's location, determine that the electrical appliance is a designated electrical appliance, and execute step S3.

An important problem faced by smart home speech recognition is the omission of language. Hidden in our natural language are a plethora of known preconditions, such as "turn it down a little", does it mean the temperature is lower, or the curtains lower, or the screen brightness lower? In daily life, people may know which device the speaker wants to turn down with just one look, but smart appliances are still unable to perceive this expressive expression. The intelligent family natural language control method of the present invention adopts the method of indoor positioning of the user, and intelligently analyzes and judges the information contained in the incomplete voice control command of the user according to the user's position, thereby improving the recognition of the natural language by the voice control system It avoids that the system does not perform actions because it cannot recognize it, or asks more confirmation questions for the user to confirm because it cannot recognize it accurately, which brings inconvenience to the user and reduces the user's enjoyment and feeling of the smart home system. According to the judgment of incomplete voice control commands based on the user positioning method, the hidden Markov model is actually used to solve the problem of language simplification. With the help of indoor positioning information, if the user is closer to the air conditioner, the command is considered to be sent to the air conditioner. ; if the user is closer to the window, the command is considered to be sent to the window. In fact, this is also completely in line with the normal usage habits of human beings, which helps to improve the accuracy of speech judgment.

So how does the air conditioner know that "bigger" is asking for "colder"? Or "hotter"? Or a little more ventilation? It is not enough to rely only on the user's location information, and it is also necessary to use the hidden Markov model algorithm for judgment.

According to human's logical reasoning method, we can infer the real intention of the user when they say "bigger":

1. If the window has just been opened, then "bigger" should refer to the larger window.

2. If the air conditioner has just been turned on, then it depends on whether the air conditioner is cooling or heating. When cooling, it means "cooler"; when heating, it means "heater". Generally speaking, the possibility of adjusting the ventilation volume of the air conditioner is not high.

3. If the user has just adjusted the ventilation volume of the air conditioner, the possibility of requesting to adjust the ventilation volume again becomes very high. At this point "bigger" most likely means a little more ventilation.

Turning this process into something a computer can do requires the use of Hidden Markov Models. Hidden Markov Model is a mathematical model such as the following, also known as "quintuple":

S: hidden state set;

N: observation state set;

A: The transition probability matrix between hidden states;

B: output matrix (i.e. the probability of hidden state to output state);

PI: initial probability distribution (initial probability distribution of hidden states);

Taking the bedroom as an example, we can define a quintuple as follows:

S = {window opening, A/C cold, A/C hot, A/C air volume}

N = {

Window opening: {Cooler: 0.0, Warmer: 0.0, Larger: 0.5, Smaller: 0.5},

Air conditioning refrigeration: {cold: 0.4, warmer: 0.0, bigger: 0.3, smaller: 0.3},

Air-conditioning heating: {Cooler: 0.0, Warmer: 0.4, Larger: 0.3, Smaller: 0.3},

Air conditioning air volume: {Cooler: 0.0, Warmer: 0.0, Larger: 0.5, Smaller: 0.5},

}

A = {

Window opening: {window opening: 0.7, air-conditioning cooling: 0.1, air-conditioning heating: 0.1, air-conditioning air volume: 0.1},

Air-conditioning cooling: {window opening: 0.1, air-conditioning cooling: 0.5, air-conditioning heating: 0.3, air-conditioning air volume: 0.1},

Air-conditioning heating: {window opening: 0.1, air-conditioning cooling: 0.3, air-conditioning heating: 0.5, air-conditioning air volume: 0.1},

Air conditioning air volume: {window opening: 0.1, air conditioning cooling: 0.2, air conditioning heating: 0.2, air conditioning air volume: 0.5},

}

PI = {Window opening: 0.0, A/C cold: 0.0, A/C hot: 0.0, A/C air volume: 1.0}

Among them, in the N of the quintuple, the observation state set represents the probability of each operation. If the user wants to operate the opening of the window, then he must not say "cold" or "hot", so these two The probabilities are all 0, and the user is half as likely to say "bigger" or "smaller".

In the A of the quintuple, the transition probability matrix between the hidden states refers to the probability that the four states occur again after each state in the S set. Taking the window opening as an example, it is known that the last command of the user is to adjust If the window opening is adjusted, the probability of this command to adjust the window opening is 70%, and the probability of adjusting the cooling, heating, and air volume of the air conditioner is very low, both of which are 10%.

PI is the initial probability distribution, which refers to the last judgment result. We assume that the last calculation result is to adjust the air volume of the air conditioner.

After defining the above three arrays, the Viterbi algorithm (Viterbi algorithm) can then be used to obtain the calculation result. The Viterbi algorithm is a commonly used algorithm in the hidden Markov model, and details of the algorithm are not repeated in this embodiment.

For example, to set the input sequence:

Input = {bigger, smaller, colder, bigger}

After the Viterbi algorithm is calculated, the output results are as follows:

Output = {A/C air volume, A/C air volume, A/C cooling, A/C cooling}

The calculation result is in complete agreement with the previous logical reasoning result.

As a preferred embodiment, at the initial stage of the smart home natural language control method, it further includes the step of setting user authority, collecting the voice characteristics of different users of family members, assigning corresponding user authority levels to them, and establishing user authority grade table;

Before entering step (3) in the step (21), it also includes the step of analyzing the command conflict,

a. If a voice control command from a user with a different authority level is received at the same time or within a limited time, the voice control command with a higher authority level is executed;

b. If different voice control commands of users at the same authority level conflict with each other, determine whether the conflicting voice control commands come from the same user, and if they come from the same user, send a voice prompt for the user to confirm. This step is to prevent users from joking about smart home controls. Some users may use voice commands too casually out of curiosity. For example, the user just asked to "lower the temperature by 10 degrees", and then quickly issued "Turn up the temperature by 10 degrees." degrees" command. If the voice recognition system operates strictly according to the user's command, it may damage the air conditioner. Therefore, the system needs to issue a prompt for the user to confirm, which is conducive to prompting the user to return to rationality and improving the seriousness of voice control.

This step improves the level of intelligence by assigning different permissions to different users. For example, the system administrator has the highest authority to completely control the system, the parent user is set by the system administrator, and the parent user can also specify a new parent user. , the power of the guardian user is smaller than that of the parent user. When the guardian user's command conflicts with the parent user's command, the system will execute the parent user's command first. The guest user is mainly the guest who visits the family temporarily, or the housekeeping service personnel. The smart home system provides the visitor with Minimal convenience utilities and lighting services. Of course, the voice feature information of the guest user may not be collected and stored in advance. If the system cannot find a user whose voice feature information matches in the stored data, it will be classified as a guest user. By restricting the permission level, for example, the guardian user is generally young, mentally unsound, intoxicated, in a bad mood, or has a mental illness, the control permission can be limited to 1, and commands that violate the rules cannot be used, such as homework time Ask to turn on the TV or open the door to go out and play. 2. Do not use commands that affect neighbors, such as turning on the room's audio to maximum volume at night. For another example, visitors are generally guests, housekeepers, etc., and their control rights can be limited to: 1. Voice commands related to family privacy cannot be used, such as opening the family mailbox. 2. System parameters cannot be set or modified. 3. Except for lights, air conditioners, and water heaters, other household appliances are not allowed. By adopting the above restriction, the management and control of the main user are facilitated, and corresponding services can be provided intelligently, which brings great convenience to the user.

In this embodiment, a method for handling the conflict of voice control commands from the same authority level is also given. In the step b of analyzing the command conflict, if the conflicting voice control commands do not come from the same user, Then continue to detect and receive the voice control command, if the conflicting voice control commands continue to appear at high frequency, then it is judged that there are differences in the opinions of the two users, then the method of discounting the execution command is adopted to delay the execution of the new voice control command, and Each time it is executed, the delay time increases accordingly until the contradictory voice control commands no longer appear. This step can solve the problem of parent-user conflict. For example, what if the male host and the hostess quarrel? The male host said "turn on the air conditioner" and the hostess said "turn off the air conditioner", what should I do? In this embodiment, the “most good” solution is adopted—the exact opposite order is executed at a discount until one party gives up. The schematic diagram of "discount execution" is shown in Figure 2. In short, the more A and B "toss", the smaller the slope of the line segment representing the command execution process, which means that the system's response is getting slower and slower. The parameter discount is getting higher and higher. When A and B stop arguing, and the system delays for 10 seconds to confirm that there is no more objection command, it will return to the normal state of rapid response and thorough execution.

In addition, the present embodiment also provides a step of judging mischief. In the step b of analyzing the command conflict, if the conflicting voice control commands are not from the same user, and the conflicting voice control commands If the command repetition frequency is lower than the set limit, it is judged that one of the users has prank control, then locate the user's location, find out the user who is closest to the controlled electrical appliance according to the user's location, and execute the user's voice control command . For example, the hostess is taking a shower in the bathroom, and the host shouts "stop the hot water supply" outside. At this time, it can be judged whether to execute the command according to the position of the host and hostess. Indoor location information is often the best decision-support data to judge whether an action is a mischief. This embodiment proposes the simplest indoor positioning method, by binding the IPV6 address of the electrical appliance and its spatial location into a table, as shown in Table 1:

ID Home appliance IPV6 address Appliance type Location information of home appliances 1 2001:0DB8::1428:0950 television master bedroom, east wall, central 2 2001:0DB8::1428:0951 air conditioner Master bedroom, east wall, south side 3 2001:0DB8::1428:0952 Electric control window master bedroom, south wall, central ... ... ... ...

Table 1 IPV6 address and location comparison table of smart home appliances

Since there are enough IPv6 addresses, there is no need to worry about address exhaustion. Generally speaking, the main household appliances are installed on the wall, on the ceiling or on the floor, so it is not necessary to use the Cartesian coordinate system to accurately describe their spatial position, which can greatly reduce the amount of data.

The present invention is not limited to this indoor positioning method. The positioning information formed by other indoor positioning methods, such as smart floor, Bluetooth beacon positioning, sound source positioning, and wifi positioning, can be used to participate in the above calculation based on position information.

Preferably, in this embodiment, in the step S22, the method for locating the user's position is as follows: using IPV6 address positioning, binding the IPV6 address of each electrical appliance to the actual space address where it is located, and connecting each voice receiving device The IPV6 address of the device is bound to the actual space address where it is located, and the location of the user is determined according to the IPV6 address of the voice receiving device that receives the voice control signal, and then the electrical appliance closest to the user's location is controlled to execute corresponding control commands. The binding method can be seen in Table 1.

Further, since multiple voice receiving devices may be set up in the room, in the step S22, if multiple voice receiving devices receive the voice control command at the same time, the received voice signals are compared according to the strength of the voice signals received by each voice receiving device. The voice receiving device with the strongest strength, and the IPV6 address of the voice receiving device is used to determine the location of the user.

Preferably, before the step S1, the step of presetting a trigger vocabulary for triggering the voice control operation is further included, and the step S1 or the steps after the step S1 are executed only when the user utters the voice of the trigger vocabulary. This is to prevent the user's daily conversations from being separated from the voice control commands sent to the smart home, and the system will only analyze and execute the voice when the user enables the voice control function.

In daily life, it is possible that the voice control command issued by the user will not contain specific parameter information. For example, "Turn up the temperature of the air conditioner a little", the system only receives the command to increase the temperature of the air conditioner, but it does not know how to adjust the temperature of the air conditioner. How many degrees, another example, "turn on the TV", the system does not know which station should be turned on, therefore, in order to be able to handle the above situation, in the step S3, if the voice command does not contain specific parameter information, the default Parameters execute user commands, and each appliance is set with default parameters. Taking TV as an example, the default parameter of "Watch TV" is to watch the station where the TV was last turned off. If that station has no signal, choose the station with the highest viewing frequency. Taking an air conditioner as an example, the default parameter is plus or minus 1 degree. Other types of smart home appliances or smart devices should be set to appropriate default parameters.

To sum up, the smart home natural language control method of this embodiment, combined with the indoor positioning technology and the hidden Markov principle, can judge incomplete voice control commands according to people's normal habits and logical thinking, which improves the accuracy of judgment. It can also judge a variety of different application control environments, avoid users' mischief and lead to damage to electrical appliances, and at the same time provide a solution to user command conflicts, making smart home control more intelligent and more in line with human beings Logical thinking habits and living habits, and at the same time, in the entire control method, the system needs to ask the user very few questions, which will not bring more inconvenience to the user.

Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by those of ordinary skill in the art within the scope of the present invention should also belong to protection scope of the present invention.

Claims (8)

1. A natural language control method for an intelligent home comprises the following steps: (1) the voice receiving device receives a voice control command; (2) the control unit analyzes the command; (3) executing the command; characterized in that, in the step (2), the following substeps are also included:

(21) analyzing the integrity of the voice control command, and directly entering the step (3) if the voice control command comprises the complete electric appliance type and the action command for controlling execution; otherwise, entering the step (22); the method for analyzing the integrity of the voice control command comprises the following steps:

defining a quintuple comprising:

s: the hidden state set comprises a plurality of hidden states of the electric appliance to be adjusted;

n: observing a state set, wherein the observation state set represents the probability of each operation, and comprises the probability of each operation corresponding to each hidden state;

a: the transition probability matrix among the hidden states represents the probability that each state occurs again after the state, and comprises the probability that each hidden state in the hidden state set occurs again after any hidden state in the hidden state set occurs;

b: an output matrix which is the probability from a hidden state to an output state;

PI: an initial probability distribution which is an initial probability distribution of a hidden state;

the output matrix B is obtained by inputting the array S, N, A into a Viterbi algorithm for calculation;

(22) if the type of the electric appliance is not specified in the voice control command, the position of the user is located, the electric appliance which is closest to the user is found out according to the position of the user, the electric appliance is judged to be the specified electric appliance, and the step (3) is executed.

2. The intelligent home natural language control method according to claim 1, further comprising a step of setting user authority when the intelligent home natural language control method is initiated, collecting voice characteristics of different users of the family members, assigning corresponding user authority levels to the voice characteristics, and establishing a user authority level table;

before step (3) is entered in step (21), a step of analyzing a command conflict is further included,

a. if the voice control commands of users with different authority levels are received at the same time or within a limited time, executing the voice control command with high authority level;

b. if the different voice control commands of the users at the same authority level conflict with each other, judging whether the conflicting voice control commands are from the same user, and if the conflicting voice control commands are from the same user, sending a voice prompt to confirm the user.

3. The intelligent home natural language control method according to claim 2, wherein in the step b of analyzing the command conflict, if the conflicting voice control commands are not from the same user, the received voice control commands are continuously detected, if the conflicting voice control commands continue to appear, it is determined that there is a divergence between opinions of the two users, a method of performing the commands by discounting is employed, the new voice control commands are delayed, and the time of the delay is correspondingly increased every time the new voice control commands are performed until the conflicting voice control commands do not appear any more.

4. The method according to claim 3, wherein in the step b of analyzing the command conflict, if the conflicting voice control commands are not from the same user and the conflicting voice control command repetition frequency is lower than a set limit, it is determined that one of the users has prank control, the location of the user is located, the user closest to the controlled appliance is found according to the location of the user, and the voice control command of the user is executed.

5. The smart home natural language control method according to any one of claims 1 to 4, wherein in the step (22), the method for locating the position of the user comprises: the IPV6 address positioning is adopted, the IPV6 address of each electric appliance is bound with the actual space address where the electric appliance is located, the IPV6 address of each voice receiving device is bound with the actual space address where the electric appliance is located, the position where a user is located is judged according to the IPV6 address of the voice receiving device which receives voice control signals, and then the electric appliance which is closest to the position of the user is correspondingly controlled to execute a corresponding control command.

6. The intelligent home natural language control method according to claim 5, wherein in the step (22), if a plurality of voice receiving devices receive the voice control command at the same time, the voice receiving device with the strongest received voice signal strength is compared according to the strength of the voice signal received by each voice receiving device, and the position of the user is determined according to the IPV6 address of the voice receiving device.

7. The smart home natural language control method according to any one of claims 1 to 4, further comprising a step of presetting a trigger vocabulary for triggering a voice control operation before the step (1), and performing the step (1) or the subsequent steps when a user utters a voice of the trigger vocabulary.

8. The smart home natural language control method according to any one of claims 1 to 4, wherein in the step (3), if the voice command does not include specific parameter information, the user command is executed according to default parameters, and each appliance is provided with default parameters.

Images