CN103257627A - Control system and control method of Internet of things based on computer vision - Google Patents

Abstract

A control system and method based on computer vision, the system includes an embedded computer, each signal end of the embedded computer is respectively connected to at least one camera, an Internet of Things coordinator, a wired router and/or a wireless router, and an electromagnetic lock , a door switch, a microphone, and a speaker. Based on the control method of the above control system, the control system provides a camera to collect video signals, and the image recognition software in the embedded computer can perform identity recognition, state recognition, feature recognition, expression recognition, Body language recognition. After the recognition is completed, the Internet of Things products are controlled according to the set control plan to realize operations such as unlocking, opening the door, turning on the light, turning on the air conditioner, and alarming. The above control can also be realized through a mobile phone or computer connected to the Internet. Improve the convenience of people's life and work.

Description

A computer vision-based Internet of things control system and method

technical field

The invention relates to the field of Internet of Things control, in particular to a computer vision-based control system and method, in which an Internet of Things camera controls related Internet of Things units according to image recognition results. the

Background technique

The rich diversity of information contained in the image determines the breadth of the application field of the camera. The Internet of Things has entered people's lives, and the camera, as a sensor in the Internet of Things, has not yet been fully utilized. the

Contents of the invention

In order to overcome the defects of the above-mentioned prior art, the object of the present invention is to provide a control system and method based on computer vision, which can realize identity recognition and state recognition through monocular (single camera) or binocular (dual camera) vision , feature recognition, facial expression recognition, etc., and then control and automatically control other units in the Internet of Things according to the recognition results, so as to improve the convenience of people's life and work. the

To achieve the above object, the present invention is achieved by taking the following technical solutions:

A control system based on computer vision, comprising an embedded computer, each signal end of the embedded computer is respectively connected to at least one camera, an IoT coordinator, a wired router and/or a wireless router, an electromagnetic lock, and a switch door mechanism, a microphone, and a speaker. the

The control method based on the above-mentioned control system is characterized in that, comprising the following steps:

Step 1. Select one or more IoT products according to the needs. IoT products include IoT power switch, IoT air conditioner controller, IoT door lock, IoT switch door controller, IoT computer boot device and IoT alarm Device; the embedded computer communicates with each IoT product through the IoT coordinator, sends a control signal to the IoT product and receives the status signal sent back by the IoT product;

Step 2. The embedded computer contains video image recognition software, so that plane images correspond to monocular vision or stereo images correspond to binocular vision recognition. Specifically, there are the following types of recognition functions: identity recognition, state recognition, feature recognition, expression recognition, body language recognition; the embedded computer also includes control function setting software, so that the user can set the control scheme of the intelligent control device through a mobile phone or computer connected to the Internet,

The control scheme includes two items, one is which type of recognition is performed after the embedded computer collects the image, and the types of recognition include: identity recognition, status recognition, feature recognition, expression recognition and body language recognition; There are three types of control operations. The first type is direct control operation, which specifically includes the following: unlocking, closing, opening, closing, voice prompts, and voice alarms; the second type is sending control commands through the IoT coordinator To control the functional operation of the IoT products, including the following: the opening and closing of the IoT power switch; the control operation of the IoT air conditioner controller, including opening, closing, operating mode setting, temperature setting and air volume setting The opening and closing of the door lock of the Internet of Things; the opening and closing of the door controlled by the switch door controller of the Internet of Things; The following types: send images or video signals to designated mobile phones or computers through the Internet, realize two-way voice communication between the owner and visitors through the Internet, alarm through the Internet, and operate computers to play music through the Internet;

The above three types of control operations are automatically controlled by the computer vision-based control system according to the set control scheme, or the mobile phone or computer sends commands to the control system after password authentication through the Internet;

Step 3. The camera installed at the door of the home, apartment, garage or office, identifies the person who arrives at the door, and proceeds to the next step according to the recognition result: if the owner is the owner, unlock/open the door, and open it according to the set control plan lights, turn on the air conditioner, turn on the computer, play music, etc.; if it is not the owner, notify the owner, and open the door, or ask, or call the police according to the owner's instructions; the camera installed at the garage door can also recognize the license plate. The license plate number can control the opening of the garage door; the camera can also be installed in front of the safe to identify the person who opens the safe, and if it is not the owner, the owner will be notified and the police will be called. the

The advantages of the present invention are as follows:

(1) Through the computer vision function, a series of operations such as automatically opening the door, turning on the lights, turning on the air conditioner, turning on the humidifier, turning on the water dispenser, turning on the computer, and playing music can be realized, which greatly increases the convenience of life. the

(2) Through the computer vision function, it is possible to know the security situation of the home and the visiting situation of the guests at any time, and to call the police in time, which is conducive to safety. the

(3) Through mobile phones (or computers), users can control various power sources at home anytime and anywhere, which is conducive to energy saving. the

Description of drawings

Fig. 1 is the functional block diagram of the intelligent control device of the method of the present invention. the

Fig. 2 is a block diagram of a household application example of the method of the present invention. the

Fig. 3 is a block diagram of a safety alarm application example of the method of the present invention. the

Detailed ways

The present invention is described in detail below in conjunction with accompanying drawing. the

Referring to Figure 1, a control system based on computer vision includes an embedded computer, and each signal terminal of the embedded computer is respectively connected to at least one camera, an Internet of Things coordinator, a wired router and/or a wireless router, and an electromagnetic A lock, a door opening mechanism, a microphone, a speaker. the

The control method based on the above-mentioned control system is characterized in that, comprising the following steps:

Step 1. Select one or more IoT products according to the needs. IoT products include IoT power switch, IoT air conditioner controller, IoT door lock, IoT switch door controller, IoT computer boot device and IoT alarm Device; the IoT coordinator communicates with each IoT product through radio frequency signals, sends control signals to IoT products and receives status signals sent back by IoT products;

The functions of each IoT product are briefly described as follows: The IoT power switch not only has the function of an ordinary power switch, but also can accept the instructions of the IoT coordinator, control the power on and off according to the instructions, and return the power on and off status signal to the object Networking coordinator; IoT air conditioner controller not only has the function of ordinary air conditioning controller, but also can control the air conditioner according to the instructions of the IoT coordinator, and returns the air conditioner operating status signal to the IoT coordinator; IoT door lock It not only has the manual switch function of ordinary door locks, but also accepts the instructions of the IoT coordinator, controls the unlocking and closing of the door locks according to the instructions, and returns the switch status signal of the lock to the IoT coordinator; the IoT door switch control The function of the device is to accept the instructions of the Internet of Things coordinator, control the opening or closing of the door according to the instructions, and return the switch status signal of the door to the Internet of Things coordinator; the function of the Internet of Things computer boot device is to accept the instructions of the Internet of Things coordinator, Control the opening of the computer according to the instructions, and return the computer's switch status signal to the IoT coordinator; the function of the IoT alarm device is to accept the instructions of the IoT coordinator, and send out an audible and visual alarm signal according to the instructions;

Step 2. The embedded computer contains video image recognition software, which can realize that the plane image corresponds to monocular vision or the stereoscopic image corresponds to binocular vision recognition. Specifically, there are the following types of recognition functions: ① Identity recognition, through face recognition Judging the identity of the visitor; ②Status recognition, judging the state of the visitor (tired, excited, etc.) through face recognition; ③Feature recognition, judging the car license plate through image recognition; ④Expression recognition, judging the expression of the visitor through face recognition (emotions, sorrows, joys, etc.); ⑤ body language recognition, judge the visitor's body language through image recognition; the embedded computer also includes control function setting software, so that users can set Determine the control scheme of the intelligent control device,

The control scheme includes two items, one is which type of recognition is performed after the embedded computer collects the image, and the types of recognition include: identity recognition, status recognition, feature recognition, expression recognition and body language recognition; There are three types of control operations. The first type is direct control operation, which specifically includes the following: unlocking, closing, opening, closing, voice prompts, and voice alarms; the second type is sending control commands through the IoT coordinator To control the functional operation of the IoT products, including the following: the opening and closing of the IoT power switch; the control operation of the IoT air conditioner controller, including opening, closing, operating mode setting, temperature setting and air volume setting The opening and closing of the door lock of the Internet of Things; the opening and closing of the door controlled by the switch door controller of the Internet of Things; The following types: send images or video signals to designated mobile phones or computers through the Internet, realize two-way voice communication between the owner and visitors through the Internet, alarm through the Internet, and operate computers to play music through the Internet;

The above three types of control operations are automatically controlled by the computer vision-based control system according to the set control scheme, or the mobile phone or computer sends commands to the control system after password authentication through the Internet;

Step 3. The camera installed at the door of the home, apartment, garage or office, identifies the person who arrives at the door, and proceeds to the next step according to the recognition result: if the owner is the owner, unlock/open the door, and open it according to the set control plan lights, turn on the air conditioner, turn on the computer, play music, etc.; if it is not the owner, notify the owner, and open the door, or ask, or call the police according to the owner's instructions; the camera installed at the garage door can also recognize the license plate. The license plate number can control the opening of the garage door; the camera can also be installed in front of the safe to identify the person who opens the safe, and if it is not the owner, the owner will be notified and the police will be called. the

Embodiment one

A computer vision-based IoT control method home application example. the

As shown in Figure 2, the intelligent control device in this example adopts monocular vision, wireless router access to the Internet, and omits the automatic door opening device; six IoT products are selected, and three of them are used to control the lighting of the living room. Lights, drinking fountains and humidifiers, an IoT air conditioner controller to control the air conditioner, an IoT computer boot device to control the home computer, and an IoT alarm device. the

The camera is installed at the door of the house. When someone arrives at the door, the intelligent control device first completes the identification. If it is the owner, it controls the electromagnetic door lock to unlock; controls the IoT power switch 1, 2, 3 to connect the living room lighting, water dispenser, humidifier Control the Internet of Things air conditioner controller to turn on the air conditioner according to the set conditions; control the Internet of Things computer boot device to turn on the computer; further carry out facial expression recognition, and control computer playback and mood through the Internet according to the facial expression recognition results and pre-set programs suitable music. the

If a guest visits, the image of the visitor will be sent to the set mobile phone or computer, and the owner can have a voice conversation with the visitor through the mobile phone or computer, and the owner decides whether to unlock. If the visitor has abnormal behavior (such as masking, picking the lock), the owner can give a voice warning and activate the alarm device. the

Example two

A security alarm application example of a computer vision-based Internet of Things control method

As shown in Figure 3, the intelligent control device in this example adopts binocular vision, accesses the Internet through a wired router, and omits the electromagnetic door lock and automatic door opening device; only one IoT product—the IoT alarm device is selected. the

The dual cameras are concealedly installed in areas that need to be secured (such as in front of a safe). The binocular vision has a stereoscopic recognition function. The use of binocular vision recognition can effectively prevent the use of photos instead of real people to deceive the intelligent control device. the

When someone arrives at the area protected by the camera, the identification is first carried out, and if it is the owner, the body language recognition is further carried out, and several specific body languages indicating the alarm are agreed in advance. Once the alarm body language is recognized, immediately according to the prior settings Start the alarm device and send a video signal to a predetermined computer; if it is not the owner, send a video signal to the owner's mobile phone or computer, and at the same time perform body language recognition, and immediately start an alarm if there is an abnormal behavior (such as unlocking). the

Claims (2)

1. control system based on computer vision, it is characterized in that each signal end that comprises an embedded computer, embedded computer connects at least one camera, an Internet of Things telegon, a cable router and/or a wireless router, an electromagnetic lock, a door opening/closing apparatus, a microphone, a loudspeaker respectively.

2. based on the control method of the described control system of claim 1, it is characterized in that, may further comprise the steps:

Step 1, select one or more Internet of Things products, Internet of Things product to comprise Internet of Things power switch, Internet of Things air-conditioner controller, Internet of Things door lock, Internet of Things switch door controller, Internet of Things booting computer device and Internet of Things warning device according to demand; Embedded computer is communicated by letter with each Internet of Things product by the Internet of Things telegon, to the status signal that the Internet of Things product transmits control signal and the receiver networked product is beamed back;

Comprise video image identification software in step 2, the embedded computer, the realization plane picture is namely namely identified corresponding to binocular vision corresponding to monocular vision or stereo-picture, and following a few class recognition function is specifically arranged: identification, state recognition, feature identification, Expression Recognition, body language identification; Also comprise control function setting software in the described embedded computer, the control scheme that makes mobile phone that the user can be by being connected into the internet or computing machine set described intelligence controlling device,

The control scheme comprises two contents, and the one, which kind of identification embedded computer carries out after collecting image, and identification types comprises: identification, state recognition, feature identification, Expression Recognition and body language identification; The 2nd, identification is carried out any control operation after finishing, and control operation has three classes, and the first kind is direct control operation, specifically comprises following several: unblank, shut, open the door, close the door, voice suggestion, audio alert; Second class is to control the Internet of Things product by Internet of Things telegon transmission control command to carry out feature operation, specifically comprises following several: the Kai Heguan of Internet of Things power switch; The control operation of Internet of Things air-conditioner controller comprises that open and close, operational mode are set, temperature is set and air quantity is set; The Kai Heguan of Internet of Things door lock; What Internet of Things switch door controller was controlled opens the door and closes the door; The warning of Internet of Things booting computer device control booting computer, Internet of Things warning device; The 3rd class is the operation of being undertaken by the internet, specifically comprises following several: send image or vision signal, get in touch with, report to the police, play music by internet operation computing machine by the internet by the double-directional speech between internet realization owner and the visiting person by mobile phone or the computing machine of internet to appointment;

Above-mentioned three class control operations by based on the control system of computer vision according to the control scheme of setting automatically control finish, perhaps behind cipher authentication, send order to control system by the internet by mobile phone or computing machine and control and finish;

Step 3, be installed in the camera on family, apartment, garage or office doorway, the people who arrives the doorway is carried out identification, carry out next step operation according to recognition result: if owner then unblanks/opens the door, and turn on light, turn on the aircondition, open computer, play operations such as music according to the control scheme of setting; If not owner then notifies owner, and according to owner's instruction or open the door or inquire or report to the police; The camera that is installed in the doorway, garage also can be identified car plate, can control the unlatching garage door for the license plate number of setting; Before described camera also can be installed in safety cabinet, carry out identification to opening the safety cabinet person, if not owner then notifies owner and reports to the police.

Images