CN112360285B - Automatic door stereoscopic machine vision sensor and sensing method thereof - Google Patents

Abstract

The utility model discloses an automatic door stereoscopic machine vision sensor and a sensing method thereof, wherein the sensing method of the automatic door stereoscopic machine vision sensor comprises the following steps of S1: installing a binocular stereoscopic machine vision sensor comprising a first camera and a second camera on an automatic door, wherein the first camera and the second camera are electrically connected with a controller through data lines; step S2: parallax comparison is performed through different picture positions of imaging of the object on the first camera and the second camera. The utility model discloses an automatic door stereoscopic machine vision sensor and a sensing method thereof, which detect and judge the motion state of an object through a binocular stereoscopic machine vision sensor, thereby avoiding two conditions of opening the door of the automatic door when the object is static and opening the door of the automatic door when the object moves parallel to the automatic door, avoiding unnecessary starting of the automatic door, prolonging the service time of the automatic door and improving the economic benefit.

Description

Automatic door stereo machine vision sensor and its sensing method

technical field

The invention belongs to the technical field of automatic door sensors, and in particular relates to a three-dimensional machine vision sensor for automatic doors and a sensing method for the three-dimensional machine vision sensor for automatic doors.

Background technique

At present, there are two problems in the opening and closing of the automatic door. One is that the door will automatically open when the person is still in front of the door, and the other is that the automatic door will open when the person walks in parallel. In both cases, there is no need to open the door. The frequent operation of automatic doors will increase the power consumption and the depreciation rate of automatic doors, shorten the life of automatic doors, and greatly reduce the economic benefits.

The publication number is: CN105869241, the subject name is an invention patent of an adaptive automatic door system based on Doppler radar and its control method, and its technical scheme discloses that "the system includes a control unit, a human-computer interaction terminal, a Doppler Radar, actuator, motor encoder, manual button switch, visual and audible alarm device, the Doppler radar transmits dynamic target information to the control unit, and the motor encoder is installed on the motor shaft to transmit the motor speed signal To the control unit, the control unit outputs door opening and closing information to the actuator that controls the opening and closing of the automatic door, the manual button switch and the human-computer interaction terminal output control signals to the control unit, and the control unit outputs alarm signals to the visual and Audible alarm device".

Taking the above invention patent as an example, although it mentions the automatic door, its technical solution is different from the present invention. Therefore, in view of the above problems, further improvements are made.

Contents of the invention

The main purpose of the present invention is to provide an automatic door stereo machine vision sensor and its sensing method, which detects and judges the motion state of an object through a binocular stereo machine vision sensor, thereby avoiding the automatic door opening when the object is still and the object moving parallel to the automatic door The two situations of opening the automatic door avoid unnecessary actuation of the automatic door, thereby prolonging its service time and improving economic benefits.

Another object of the present invention is to provide a three-dimensional machine vision sensor for automatic doors and its sensing method, which can also detect the movement of multiple objects, then the door opening and closing time can be calculated by accumulating the earliest door opening time and the latest door closing time of multiple objects. It can be solved by multi-threaded programming.

In order to achieve the above object, the present invention provides a sensing method of a three-dimensional machine vision sensor for an automatic door, which is used to detect the motion state of an object (preferably a human body) so that the automatic door judges whether to open the door, comprising the following steps:

Step S1: installing a binocular stereo machine vision sensor including a first camera and a second camera on the automatic door, and both the first camera and the second camera are electrically connected to the controller through a data line;

Step S2: Parallax comparison is performed at different image positions of the object on the first camera and the second camera to obtain the distance between the center positions of the first camera and the second camera and the object;

Step S3: The binocular stereo machine vision sensor continuously takes pictures of the monitored area at a rate of several frames per second to obtain the motion state of the object in the monitored area, and the automatic door judges whether to open the door according to the motion state of the object.

As a further preferred technical solution of the above technical solution, step S1 is specifically implemented as the following steps:

Step S1.1: The first camera and the second camera are fixedly placed at a preset distance in the same direction;

Step S1.2: The preset distance between the first camera and the second camera is set according to the monitoring area of the binocular stereo machine vision sensor;

Step S1.3: The binocular stereo machine vision sensor performs camera calibration according to the calibration method.

As a further preferred technical solution of the above technical solution, step S2 is specifically implemented as the following steps:

Step S2.1: After the camera is calibrated, the binocular stereo machine vision sensor obtains the image position of the same object imaged by the first camera and the second camera respectively;

Step S2.2: Due to the preset distance between the first camera and the second camera, a certain parallax is generated in the image position of the same object after the first camera and the second camera respectively;

Step S2.3: The binocular stereo machine vision sensor performs parallax comparison according to the pixel position of the camera imaging, and matches the calibrated parallax with the distance to obtain the distance between the center position of the first camera and the second camera and the object, And calibrated as the sensing distance.

As a further preferred technical solution of the above technical solution, step S3 is specifically implemented as the following steps:

Step S3.1: The binocular stereo machine vision sensor detects that the sensing distance is s1 at time t1, and the abscissa on the imaging is x1;

Step S3.2: The binocular stereo machine vision sensor detects that the sensing distance is s2 at time t2, and the abscissa on the imaging is x2;

Step S3.3: judging whether the offset between x1 and x2 reaches the set threshold X, so as to judge the motion state of the object;

Step S3.4: According to the data obtained in step S3.1 and step S3.2 and by the formula v=(s2-s1)/(t2-t1), the moving speed v of the object is obtained to judge whether the object continues to move or remains still ( When the person is still, s2=s1, speed v=0, when calculating the time when the object will reach the door, an error will be reported because the denominator is 0, so as to achieve the purpose of not opening the door. In addition, because when the object is close to the door, s2<s1 , then v<0, if v>0, it is because s2>s1, that is, the object is moving away from the door and does not open the door);

Step S3.5: Obtain the time t when the object reaches the automatic door according to the moving speed v and the sensing distance s2;

Step S3.6: Set the opening time t-Δt and closing time t+Δt of the automatic door according to the preset time amount Δt;

Step S3.7: When multiple objects are moving, calculate the door opening time and closing time by accumulating the earliest opening time and the latest closing time of multiple objects (multiple objects can be solved by multi-threaded programming).

As a further preferred technical solution of the above technical solution, step S3.3 is specifically implemented as the following steps:

Step S3.3.1: If the offset between x1 and x2 reaches the set threshold X, it is judged that the object is moving parallel to the automatic door, and the door opening action is not performed;

Step S3.3.2: If the offset between x1 and x2 does not reach the set threshold X, it is judged that the object is moving forward (s2<s1) or moving backward (s2>s1) or stationary (s2) in the direction of the automatic door = s1), if the object is moving forward, execute step S3.4.

In order to achieve the above object, the present invention also provides a stereoscopic machine vision sensor for automatic doors, which is installed on the automatic door and includes a binocular stereoscopic machine sensor. The binocular stereoscopic machine sensor includes a first camera and a second camera, wherein:

A binocular stereo machine vision sensor including a first camera and a second camera is installed on the automatic door, and the first camera and the second camera are electrically connected to the controller through a data line;

Parallax comparison is performed by different image positions of the object on the first camera and the second camera to obtain the distance between the center position of the first camera and the second camera and the object;

The binocular stereo machine vision sensor continuously takes pictures of the monitored area at a rate of several frames per second to obtain the motion status of objects in the monitored area, and the automatic door judges whether to open the door according to the motion status of the object.

As a further preferred technical solution of the above technical solution, the first camera and the second camera are fixedly placed at a preset distance in the same direction; the preset distance between the first camera and the second camera is based on binocular stereoscopic machine vision The monitoring area of the sensor is set; the binocular stereo machine vision sensor is calibrated according to the calibration method.

As a further preferred technical solution of the above technical solution, after the camera is calibrated, the binocular stereo machine vision sensor obtains the picture position of the same object after the first camera and the second camera are imaged respectively; The same object has a certain parallax in the image position of the first camera and the second camera respectively; the binocular stereo machine vision sensor performs a parallax comparison according to the pixel position of the camera imaging, and matches the calibrated parallax with the distance , to obtain the distance between the center position of the first camera and the second camera and the object, and calibrate it as the sensing distance.

As a further preferred technical solution of the above technical solution, the binocular stereo machine vision sensor detects that the sensing distance is s1 at the moment t1, and the abscissa on the imaging is x1; The sensing distance is s2, and the abscissa on the imaging is x2; judge whether the offset between x1 and x2 reaches the set threshold X to judge the motion state of the object; according to the acquired data and through the formula v=( s2-s1)/(t2-t1) to obtain the moving speed v of the object to judge whether the object continues to move or remain still (when the person is still, s2=s1, speed v=0, when calculating the time when the object will reach the door, it will Because the denominator is 0, an error is reported, so as to achieve the goal of not opening the door. In addition, because when the object is close to the door, s2<s1, then v<0, if v>0, it is because s2>s1, that is, the object is moving away from the door. do not open the door); obtain the time t when the object reaches the automatic door according to the motion speed v and the sensing distance s2; set the opening time t-Δt and closing time t+Δt of the automatic door according to the preset time amount Δt; when there are multiple When the object is moving, the door opening time and closing time are calculated by accumulating the earliest opening time and the latest closing time of multiple objects (multiple objects can be solved by multi-threaded programming).

As a further preferred technical solution of the above technical solution, if the offset between x1 and x2 reaches the set threshold X, it is judged that the object is moving parallel to the automatic door, and the door opening action is not performed;

If the offset between x1 and x2 does not reach the set threshold X, it is judged that the object is moving forward (s2<s1) or moving backward (s2>s1) or stationary (s2=s1) towards the automatic door.

Description of drawings

Fig. 1 is a flow chart of the automatic door stereo machine vision sensor and its sensing method of the present invention.

Fig. 2 is a schematic diagram of object movement of the automatic door stereo machine vision sensor and its sensing method of the present invention.

Detailed ways

The following description serves to disclose the present invention to enable those skilled in the art to carry out the present invention. The preferred embodiments described below are only examples, and those skilled in the art can devise other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents and other technical solutions without departing from the spirit and scope of the present invention.

Referring to Figure 1 of the accompanying drawings, Figure 1 is a flowchart of the automatic door stereo machine vision sensor and its sensing method of the present invention, and Figure 2 is a schematic diagram of object movement of the automatic door stereo machine vision sensor and its sensing method of the present invention.

In the preferred embodiment of the present invention, those skilled in the art should note that the multi-thread programming, camera calibration and camera involved in the present invention can be regarded as prior art.

preferred embodiment.

The utility model discloses a sensing method of a three-dimensional machine vision sensor for an automatic door, which is used for detecting the motion state of an object (preferably a human body) so that the automatic door can judge whether to open the door. The method comprises the following steps:

Step S1: installing a binocular stereo machine vision sensor including a first camera and a second camera on the automatic door, and both the first camera and the second camera are electrically connected to the controller through a data line;

Step S2: Parallax comparison is performed at different image positions of the object on the first camera and the second camera to obtain the distance between the center positions of the first camera and the second camera and the object;

Step S3: The binocular stereo machine vision sensor continuously takes pictures of the monitored area at a rate of several frames per second to obtain the motion state of the object in the monitored area, and the automatic door judges whether to open the door according to the motion state of the object.

Specifically, step S1 is specifically implemented as the following steps:

Step S1.1: The first camera and the second camera are fixedly placed at a preset distance in the same direction;

Step S1.2: The preset distance between the first camera and the second camera is set according to the monitoring area of the binocular stereo machine vision sensor;

Step S1.3: The binocular stereo machine vision sensor performs camera calibration according to the calibration method.

More specifically, step S2 is specifically implemented as the following steps:

Step S2.1: After the camera is calibrated, the binocular stereo machine vision sensor obtains the image position of the same object imaged by the first camera and the second camera respectively;

Step S2.2: Due to the preset distance between the first camera and the second camera, a certain parallax is generated in the image position of the same object after the first camera and the second camera respectively;

Step S2.3: The binocular stereo machine vision sensor performs parallax comparison according to the pixel position of the camera imaging, and matches the calibrated parallax with the distance to obtain the distance between the center position of the first camera and the second camera and the object, And calibrated as the sensing distance.

Further, step S3 is specifically implemented as the following steps:

Step S3.1: The binocular stereo machine vision sensor detects that the sensing distance is s1 at time t1, and the abscissa on the imaging is x1;

Step S3.2: The binocular stereo machine vision sensor detects that the sensing distance is s2 at time t2, and the abscissa on the imaging is x2;

Step S3.3: judging whether the offset between x1 and x2 reaches the set threshold X, so as to judge the motion state of the object;

Step S3.4: According to the data obtained in step S3.1 and step S3.2 and by the formula v=(s2-s1)/(t2-t1), the moving speed v of the object is obtained to judge whether the object continues to move or remains still ( When the person is still, s2=s1, speed v=0, when calculating the time when the object will reach the door, an error will be reported because the denominator is 0, so as to achieve the purpose of not opening the door. In addition, because when the object is close to the door, s2<s1 , then v<0, if v>0, it is because s2>s1, that is, the object is moving away from the door and does not open the door);

Step S3.5: Obtain the time t when the object reaches the automatic door according to the moving speed v and the sensing distance s2;

Step S3.6: Set the opening time t-Δt and closing time t+Δt of the automatic door according to the preset time amount Δt;

Step S3.7: When there are multiple objects moving, calculate the opening time and closing time by accumulating the earliest opening time and the latest closing time of multiple objects (multiple objects can be solved by multi-threaded programming), if there are no multiple targets , the controller directly controls the automatic door to open and close the door.

Furthermore, step S3.3 is specifically implemented as the following steps:

Step S3.3.1: If the offset between x1 and x2 reaches the set threshold X, it is judged that the object is moving parallel to the automatic door, and the door opening action is not performed;

Step S3.3.2: If the offset between x1 and x2 does not reach the set threshold X, it is judged that the object is moving forward (s2<s1) or moving backward (s2>s1) or stationary (s2) in the direction of the automatic door = s1), if the object is moving forward, execute step S3.4.

Preferably, step S3.6 is specifically implemented as the following steps:

Step S3.6.1: The door opening time is: t-Δt;

Step S3.6.2: The closing time is: t+Δt.

The invention also discloses a three-dimensional machine vision sensor for automatic doors, which is installed on the automatic door and includes a binocular three-dimensional machine sensor. The binocular three-dimensional machine sensor includes a first camera and a second camera, wherein:

A binocular stereo machine vision sensor including a first camera and a second camera is installed on the automatic door, and the first camera and the second camera are electrically connected to the controller through a data line;

Parallax comparison is performed by different image positions of the object on the first camera and the second camera to obtain the distance between the center position of the first camera and the second camera and the object;

The binocular stereo machine vision sensor continuously takes pictures of the monitored area at a rate of several frames per second to obtain the motion status of objects in the monitored area, and the automatic door judges whether to open the door according to the motion status of the object.

Specifically, the first camera and the second camera are fixedly placed at a preset distance in the same direction; the preset distance between the first camera and the second camera is set according to the monitoring area of the binocular stereo machine vision sensor; The binocular stereo machine vision sensor performs camera calibration according to the calibration method.

More specifically, after the camera is calibrated, the binocular stereo machine vision sensor obtains the picture position of the same object after the first camera and the second camera respectively; due to the preset distance between the first camera and the second camera, the same object A certain parallax is generated at the image position of the first camera and the second camera respectively; the binocular stereo machine vision sensor performs a parallax comparison according to the pixel position of the camera imaging, and matches the calibrated parallax with the distance to obtain the first camera and the distance between the center position of the second camera and the object, and calibrated as the sensing distance.

Further, the binocular stereo machine vision sensor detects that the sensing distance is s1 at time t1, and the abscissa on the imaging is x1; the binocular stereo machine vision sensor detects that the sensing distance is s2 at time t2, and at The abscissa on the imaging is x2; judge whether the offset between x1 and x2 reaches the set threshold X to judge the motion state of the object; according to the acquired data and through the formula v=(s2-s1)/(t2 -t1) Obtain the moving speed v of the object to judge whether the object continues to move or stay still (when the person is still, s2=s1, speed v=0, when calculating the time when the object will reach the door, an error will be reported because the denominator is 0, So as to achieve the purpose of not opening the door, in addition, because when the object is close to the door, s2<s1, then v<0, if v>0, it is because s2>s1, that is, the object is moving away from the door, and the door will not be opened); according to the movement The speed v and the sensing distance s2 obtain the time t when the object reaches the automatic door; set the opening time t-Δt and closing time t+Δt of the automatic door according to the preset time amount Δt; when there are multiple objects moving, pass multiple Objects accumulate the earliest opening time and the latest closing time to calculate the opening time and closing time (multiple objects can be solved by multi-threaded programming).

Furthermore, if the offset between x1 and x2 reaches the set threshold X, it is judged that the object is moving parallel to the automatic door, and the door opening action is not performed;

If the offset between x1 and x2 does not reach the set threshold X, it is judged that the object is moving forward (s2<s1) or moving backward (s2>s1) or stationary (s2=s1) towards the automatic door.

Preferably, the automatic door stereo machine vision sensor of the present invention can also perform face recognition, and then open the door according to the recognition result; at the same time, because of multi-target detection, the sensor can detect the flow of people; it makes security and attendance digital and integrated, reducing unnecessary Human, material and financial resources, but also improve the efficiency of unified management.

It is worth mentioning that the technical features such as multi-threaded programming, camera calibration and camera involved in the patent application of the present invention should be regarded as prior art. The method can be conventionally selected in the field, and should not be regarded as the invention point of the patent of the present invention, and the patent of the present invention will not be further elaborated in detail.

For those skilled in the art, it is still possible to modify the technical solutions described in the foregoing embodiments, or to perform equivalent replacements for some of the technical features. Any modifications made within the spirit and principles of the present invention, Equivalent replacements, improvements, etc., should all be included in the protection scope of the present invention.

Claims (8)

1. A sensing method of an automatic door stereo machine vision sensor for detecting a motion state of an object so that the automatic door judges whether to open the door, comprising the steps of:

step S1: installing a binocular stereoscopic machine vision sensor comprising a first camera and a second camera on an automatic door, wherein the first camera and the second camera are electrically connected with a controller through data lines;

step S2: parallax comparison is carried out through different image positions of an object imaged on the first camera and the second camera, so that the distance between the center positions of the first camera and the second camera and the object is obtained;

step S3: the binocular stereo machine vision sensor continuously shoots the monitored area at the speed of a plurality of frames per second to acquire the motion state of an object in the monitored area, and the automatic door judges whether to open the door according to the motion state of the object;

the step S3 is specifically implemented as the following steps:

step S3.1: the binocular stereo machine vision sensor detects that the sensing distance is s1 at the time t1, and the abscissa on imaging is x1;

step S3.2: the binocular stereo machine vision sensor detects that the sensing distance is s2 at the time t2, and the abscissa on imaging is x2;

step S3.3: judging whether the offset between X1 and X2 reaches a set threshold value X or not so as to judge the motion state of an object;

step S3.4: obtaining the movement speed v of the object according to the data acquired in the step S3.1 and the step S3.2 and through a formula v= (S2-S1)/(t 2-t 1) so as to judge whether the object continues to move or is stationary;

step S3.5: obtaining time t for an object to reach an automatic door according to the motion speed v and the sensing distance s 2;

step S3.6: setting the door opening time t-deltat and the door closing time t+deltat of the automatic door according to the preset time quantity deltat;

step S3.7: when a plurality of objects move, the earliest door opening time and the latest door closing time are accumulated by the plurality of objects to calculate the door opening time and the door closing time.

2. The sensing method of an automatic door stereo machine vision sensor according to claim 1, wherein step S1 is implemented as the following steps:

step S1.1: the first camera and the second camera are fixedly placed at preset distances in the same direction;

step S1.2: the preset distance between the first camera and the second camera is set according to the monitoring area of the binocular stereo machine vision sensor;

step S1.3: the binocular stereo machine vision sensor performs camera calibration according to a calibration method.

3. The sensing method of an automatic door stereo machine vision sensor according to claim 2, wherein step S2 is implemented as the following steps:

step S2.1: after the camera is calibrated, the binocular stereo machine vision sensor obtains the picture positions of the same object after the first camera and the second camera respectively image;

step S2.2: because of the preset distance between the first camera and the second camera, the same object generates certain parallax at the picture position after the first camera and the second camera respectively image;

step S2.3: the binocular stereo machine vision sensor performs parallax comparison according to the pixel positions imaged by the cameras, matches the calibrated parallax with the distance to obtain the distance between the center positions of the first camera and the second camera and the object, and calibrates the distance as a sensing distance.

4. The method of claim 1, wherein the step S3.3 is implemented as:

step S3.3.1: if the offset between X1 and X2 reaches the set threshold value X, judging that the object moves parallel to the automatic door and does not perform door opening action;

step S3.3.2: if the offset between X1 and X2 does not reach the set threshold value X, it is determined that the object is moving forward or backward or stationary in the direction of the automatic door, and if the object is moving forward, step S3.4 is executed.

5. An automatic door stereo vision sensor for performing the sensing method of an automatic door stereo vision sensor as defined in any one of claims 1-4, mounted to an automatic door, comprising a binocular stereo sensor comprising a first camera and a second camera, wherein:

installing a binocular stereoscopic machine vision sensor comprising a first camera and a second camera on an automatic door, wherein the first camera and the second camera are electrically connected with a controller through data lines;

parallax comparison is carried out through different image positions of an object imaged on the first camera and the second camera, so that the distance between the center positions of the first camera and the second camera and the object is obtained;

the binocular stereo machine vision sensor continuously shoots the monitored area at the speed of a plurality of frames per second to acquire the motion state of an object in the monitored area, and the automatic door judges whether to open the door according to the motion state of the object;

the binocular stereo machine vision sensor detects that the sensing distance is s1 at the time t1, and the abscissa on imaging is x1; the binocular stereo machine vision sensor detects that the sensing distance is s2 at the time t2, and the abscissa on imaging is x2; judging whether the offset between X1 and X2 reaches a set threshold value X or not so as to judge the motion state of an object; obtaining the movement speed v of the object according to the obtained data and through a formula v= (s 2-s 1)/(t 2-t 1) so as to judge whether the object continues to move or is stationary; obtaining time t for an object to reach an automatic door according to the motion speed v and the sensing distance s 2; setting the door opening time t-deltat and the door closing time t+deltat of the automatic door according to the preset time quantity deltat; when a plurality of objects move, the earliest door opening time and the latest door closing time are accumulated by the plurality of objects to calculate the door opening time and the door closing time.

6. The automatic door stereo machine vision sensor of claim 5, wherein the first camera and the second camera are fixedly placed at a preset distance from each other in the same direction; the preset distance between the first camera and the second camera is set according to the monitoring area of the binocular stereo machine vision sensor; the binocular stereo machine vision sensor performs camera calibration according to a calibration method.

7. The sensor of claim 6, wherein after the camera is calibrated, the binocular stereo machine vision sensor obtains the picture position of the same object after the first camera and the second camera are respectively imaged; because of the preset distance between the first camera and the second camera, the same object generates certain parallax at the picture position after the first camera and the second camera respectively image; the binocular stereo machine vision sensor performs parallax comparison according to the pixel positions imaged by the cameras, matches the calibrated parallax with the distance to obtain the distance between the center positions of the first camera and the second camera and the object, and calibrates the distance as a sensing distance.

8. An autostereoscopic machine vision sensor according to claim 5, wherein,

if the offset between X1 and X2 reaches the set threshold value X, judging that the object moves parallel to the automatic door and does not perform door opening action;

if the offset between X1 and X2 does not reach the set threshold value X, the object is judged to move forward or backward or still towards the automatic door.

Images