CN117197880A - Concentration monitoring method and system - Google Patents

Abstract

Concentration monitoring methods and systems relate to the field of automated monitoring technology. The aforementioned concentration monitoring method includes the following steps: 1. Using an image acquisition device to collect image information of the person and equipment to be tested; 2. Micro-behavior recognition and detection; 3. Obtaining the image information of the person to be tested The person's gaze trajectory on the monitor screen is combined with the attention heat map of the monitor screen to compare the similarity between the two; 4. Analyze the movement trend of the gaze trajectory and determine whether it is consistent with prior knowledge; 5. To be tested Detect the eye characteristics of the person to obtain the blink frequency of the person to be tested and determine whether it is normal; 6. Combine the above test results and use weighted calculation to determine whether the person to be tested is in working condition. The above concentration monitoring method can be applied to the detection of personnel distraction in environments such as aviation dispatch and nuclear power safety control, and improves concealment and convenience, while improving detection accuracy.

Description

Concentration monitoring method and system

Technical field

The present invention relates to the technical field of automated monitoring, and in particular to a concentration monitoring method and system.

Background technique

At present, when performing operations such as aviation dispatching and nuclear power safety control, staff usually need to use auxiliary devices such as human-computer interaction output devices (such as monitors) and human-computer interaction input devices (such as keyboards and mice) to perform operations on the host computer (such as a computer). Status viewing and operational control. When working, the concentration of workers is often affected by their emotions, mental state and other reasons. They are prone to distraction, wandering, and daze when they are in a bad mood, fatigue, etc., resulting in poor concentration and affecting the efficiency of operations. Timeliness and accuracy, therefore, how to identify personnel concentration is particularly important.

Chinese patent document CN109878527A discloses a distraction sensing system that uses a contact sensor to collect skin electrode signals to analyze whether a person is distracted. This method requires the person to be detected to wear a device, and lacks concealment and convenience. Chinese patent document CN104706366A discloses a distraction detection method, device and system that collects electrocardiogram data of the person to be detected, but also has the same problems as mentioned above. Chinese patent document CN115246405A discloses a method and system for detecting driver distraction behavior. It collects the characteristics of the driver's eyeballs and converts them into an interval range of visual attention. When the preset normal interval is exceeded, it is judged that the person to be detected is distracted. The detection granularity of this method is too coarse and cannot distinguish whether the driver is driving normally or being distracted. Chinese patent document CN114026611A discloses a method of using heat maps to detect driver attention. It generates scene attention heat maps and driver gaze heat maps respectively. By analyzing the difference between the two heat maps, it is determined whether the driver is distracted. This method can only identify whether the line of sight of the person to be inspected matches the area of interest, but it lacks time series information, and it is difficult to distinguish situations such as when the person to be inspected is in a daze.

The above-mentioned distraction detection methods for car drivers still have many shortcomings, and the car driving environment is quite different from the operating environment such as aviation dispatch and nuclear power safety control. Therefore, they are not suitable for aviation dispatch and nuclear power safety control. Detection of distractions in work environments.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for detecting personnel distraction in working environments such as aviation dispatching and nuclear power safety control.

In order to solve the above problems, the technical solutions adopted by the present invention are as follows:

The concentration monitoring method includes the following steps:

1. Use the image acquisition device to collect image information of the person and equipment to be tested;

2. Take the area of the human-computer interaction input device in the image as the micro-behavior recognition detection area, detect whether the hand of the person to be tested appears in the area, conduct micro-behavior recognition detection on the hand that appears in it, and obtain the result of the micro-behavior recognition detection area. The state of the person's hands to determine whether the hands are active or not;

Specifically, the area where the human-computer interaction input device in the image is located is the micro-behavior recognition detection area, and it is detected whether the hand characteristics of the person to be tested appear in the micro-behavior recognition detection area. If not, go to the next step. If there is, then the hand characteristics of the person to be tested are detected. Micro-behavior recognition detection is carried out on the hand features to determine whether the hand of the person to be tested is active or stationary. If it is active, it is determined that the person to be tested is in a working state. If it is stationary, it goes to the next step;

3. Obtain the line of sight of the person to be tested on the monitor screen, and combine it with the attention heat map on the monitor screen to compare the similarity between the two;

4. Analyze the movement trend of the sight trajectory and determine whether it is consistent with prior knowledge;

5. Detect the eye characteristics of the person to be tested, obtain the blink frequency of the person to be tested, and determine whether it is normal;

6. Based on the above test results and through weighted calculation, determine whether the person to be tested is in working condition.

Among them, in step one, the image acquisition device includes a camera, the image information of the person to be tested includes the image information of eye features, hand features and facial features of the person to be tested, and the image information of the device collected includes human-computer interaction input. Image information for device and monitor screens.

Among them, in step two, the human-computer interaction input device is a keyboard and a mouse, and the steps of determining the micro-behavior recognition detection area include:

a. Use the target detection technology in image processing to detect and determine the regional positions of the keyboard and mouse in the image R _keyboard and R _mouse respectively, where R _keyboard and R _mouse are:

R _keyboard = ((x _left , y _left ), (x _right , y _right ));

R _mouse = ((x _left , y _left ), (x _right , y _right ));

Among them, (x _left , y _left ) represents the coordinate position of the upper left corner of the region R _* , and (x _right , y _right ) represents the coordinate position of the lower right corner of the region R _* .

b. Determine the micro-behavior recognition detection area R _detection through the regional positions of the keyboard and mouse. The area coordinate parameters are:

Among them, in step two, the steps of micro-behavior recognition and detection include:

Detect the key point information on the hand features that appear in the micro-behavior recognition detection area, including finger tips, finger joints, palm center position and wrist joint position, and then use the key points of the hand features in the video sequence The trajectory in the test is used to determine whether the hand of the person to be tested is active or stationary.

Among them, in step three, the steps to obtain the line of sight trajectory include:

a. Extract and analyze the eye characteristics of the person to be tested and obtain the direction of sight;

b. Detect several eigenvalues of the human face and compare them with the eigenvalues of the face in standard postures to obtain the posture relationship between the face and the image acquisition device;

c. Combined with the posture relationship between the face of the person to be tested and the monitor screen, calculate the line of sight trajectory of the person to be tested on the monitor screen:

(1) Through the relationship between the spatial coordinates of the image acquisition device and the plane position of the display screen, the conversion matrix R between the face and the screen is obtained;

(2) Calculate the pose of the face relative to the screen through the transformation matrix R, and calculate the position P _t of the line of sight on the screen in real time through the calculated distance between the face and the screen;

(3) Through accumulation over time, the position sequence (P _i ) of the line of sight on the screen is obtained, where i≥0.

Among them, in step three, the distribution function of the screen attention heat map is Prob _S :

Among them, Bright(i,j) is the brightness value of the screen coordinate i-j position,

Among them, width and height are the resolution of the screen, and (i, j) is the coordinate position of the sampling point on the screen.

Among them, in step three, the steps of detecting the similarity between the gaze trajectory and the screen attention heat map include:

Normalize the gaze trajectory and the screen attention heat map, and sample according to the screen resolution to obtain the distribution function of the gaze trajectory Prob _Eye , and the distribution function of the screen attention heat map is Prob _Screen . The two are obtained through cross entropy calculation. Similarity between them:

Among them, width and height are the resolution of the screen respectively, and (i, j) is the coordinate position of the sampling point on the screen;

Compare the similarity between the calculated gaze trajectory and the screen attention heat map with the preset threshold to determine whether the person being tested is in a working state.

Among them, in step four, the prior knowledge is that the subject's line of sight starts from the middle of the monitor screen and moves from the upper left position to the lower right position; the steps to determine the direction of the line of sight trajectory include:

By calculating the gradient of the gaze trajectory, we can determine the direction in which the gaze moves on the screen:

Among them, Trail _i (x, y) is the coordinate information of the i-th point of the sight trajectory;

pass The positive and negative signs of the two dimensions are used to determine the direction of trajectory movement.

Among them, in step five, the steps of detecting the blink frequency of the person to be tested include:

a. Use image processing technology to extract the face position in the image and segment the eye area;

b. From the eye area, the positions of several key points of the eyes are detected, including the positions of the inner corner of the eye, the outer corner of the eye, the upper eyelid and the lower eyelid;

c. Calculate the ratio of the distance between the upper and lower eyelids to the distance between the inner and outer canthus of the eye, and combine the time information to calculate the blink frequency and the duration of eye closure;

d. The calculation method of blink discrimination parameters is:

Among them, Dis _W is the distance between the inner canthus P ₁ and the outer canthus P ₄ , Dis _H is the distance between the upper eyelid P ₂ and the lower eyelid P ₆ ;

Periodically calculate the blink discrimination parameter Blink to obtain a time-based change trend;

e. Based on the duration of eyes closed, Time _close , determine whether the person being tested is tired from work and make a judgment on the abnormal state of the person being tested:

Among them, threshold is the threshold of the duration of eyes closing.

Another object of the present invention is to provide a concentration monitoring system, which includes a storage unit and a computing unit. The storage unit stores a concentration monitoring program. The concentration monitoring program is run through the computing unit to execute the concentration. Steps in the monitoring method.

The concentration monitoring method provided by the present invention is different from the common contact distraction detection method in the past. It adopts a non-contact detection method, which does not require the person to be tested to wear equipment. It only needs to use non-contact equipment such as cameras and cameras to collect images. Information, concealment and convenience are better, and the status of the person to be tested is determined through various methods such as micro-behavior recognition, gaze trajectory and screen attention heat map detection, blink feature detection, etc., and through weighted calculation (the weight value of each indicator can be determined through experiments), comprehensively analyze the concentration of the person to be tested, and the test results obtained based on this are more accurate than a single judgment method. Compared with the existing car driver distraction detection method, the concentration monitoring method provided by the present invention is more concealed and convenient, and has higher detection accuracy, and can be suitable for operations such as aviation dispatch and nuclear power safety control. Human distraction detection in environments.

Description of the drawings

Figure 1 is a schematic flow chart of concentration monitoring in the embodiment;

Figure 2 is a schematic diagram of eye features in the embodiment;

Figure 3 is a schematic diagram of several feature points of the human face in the embodiment;

Figure 4 is a schematic diagram of facial features in the embodiment;

Figure 5 is a schematic diagram of calculating the line of sight trajectory of the person to be tested's eyes on the screen in the embodiment;

Figure 6 is a heat map of screen attention in the embodiment;

Figure 7 is a schematic diagram of the micro-behavior recognition detection area determined in the embodiment;

Figure 8 is a schematic diagram of the key point positions of the hand in the embodiment;

Figure 9 is a schematic diagram of the trajectory of hand key points in the video sequence during implementation;

Figure 10 is a schematic diagram of the positions of the inner canthus, outer canthus, upper eyelid and lower eyelid of the eye area in the embodiment;

Figure 11 is a schematic diagram of blink state changes in the embodiment, in which (a) is the eyes open, (b) is the eyes closed, and (c) is the change trend of the blink discrimination parameters based on time.

Detailed ways

In order to facilitate those skilled in the art to better understand the improvements of the present invention over the prior art, the present invention will be further described below in conjunction with the accompanying drawings and examples.

In view of the extremely high requirements for the concentration of workers in operating environments such as aviation dispatching and nuclear power safety control, this embodiment provides a concentration monitoring method to monitor the distraction of workers during the operation, as much as possible Eliminate unrest caused by worker distraction to ensure safe operation and operation of equipment. However, many of the existing distraction detection methods for car drivers use contact sensors to detect the driver's distraction. Their concealment and convenience are poor. If they are directly applied to aviation dispatching In operating environments such as nuclear power safety control and nuclear power safety control, contact sensors worn on the body will obviously have a certain impact on workers, and may even cause discomfort and distraction due to wearing these devices. Therefore, contact collection methods are not suitable for Applicable to this operating environment. Although some of the existing distraction detection methods for car drivers use non-contact detection methods, most of these methods still have the problem of low accuracy of judgment due to too coarse detection granularity, and Due to the lack of time series information, it is difficult to distinguish the situation of the person being tested being in a daze. In order to solve these problems, this embodiment provides a non-contact concentration monitoring method and improves the accuracy of detection through various detections such as micro-behavior recognition, gaze trajectory and screen attention heat map detection, blink feature detection, etc. The weighted analysis of the method can more accurately determine whether the person to be tested is distracted, thereby reducing misjudgments and missed judgments.

The invention adopts a non-contact method, using preset cameras and other non-contact sensors, and a distraction detection and analysis system to conduct non-contact detection of the distraction state of office workers in front of the computer without contacting the person being detected. Detection to achieve the purpose of automatically detecting the distraction of office workers when using computers. The concentration monitoring method uses multiple detection sub-functions, and after aggregating the results through weighted calculation (the weight value of each indicator can be determined through experiments), the distraction state of the person to be detected can be analyzed. Distracted office workers (including not only operators of aviation dispatch and nuclear power safety control, but also other office workers who use computers) have behaviors such as static behavior, abnormal sight trajectories, and abnormal blink characteristics, so the detection method includes the following Detection system: 1. Gaze trajectory and screen attention heat map detection; 2. Micro-behavior recognition; 3. Blink feature detection and other parts. The entire detection process is shown in Figure 1. The following is an example of an office worker sitting at a desk and working in front of a computer. See Figures 1 to 11.

1. Collect image information.

It mainly uses image acquisition devices to collect image information of people and equipment to be tested. The image acquisition device includes a camera, and may also include a video camera. The image information collected by the person to be tested includes the image information of the eye features, hand features and facial features of the person to be tested. The image information collected by the device includes a human-computer interaction input device. and image information on the monitor screen. The human-computer interaction input devices mainly refer to keyboards and mice. Of course, they can also be control joysticks, handles and other devices.

2. Micro-behavior recognition detection.

When office workers use computers to work, their behavioral movements are small, involving only keyboard typing and mouse manipulation. Therefore, micro-behavior recognition and detection can be performed on the hands of office workers. The camera can be used to capture the desk area and detect the hands of office workers. state.

Take the area of the human-computer interaction input device in the image as the micro-behavior recognition detection area, and detect whether the hand characteristics of the person to be tested appear in the micro-behavior recognition detection area. If not, go to the next step and perform other detection methods. Then perform micro-behavior recognition detection on the hand features that appear in it, and determine whether the hand of the person to be tested is active or stationary. If it is active, it is determined that the person to be tested is in a working state. If it is stationary, it goes to the next step and performs other methods. detection.

Among them, the steps to determine the micro-behavior recognition detection area include:

a. Use the target detection technology in image processing to detect and determine the regional positions of the keyboard and mouse in the image R _keyboard and R _mouse respectively, where R _keyboard and R _mouse are defined as:

R _keyboard = ((x _left , y _left ), (x _right , y _right ));

R _mouse = ((x _left , y _left ), (x _right , y _right ));

Among them, (x _left , y _left ) represents the coordinate position of the upper left corner of the region R _* , and (x _right , y _right ) represents the coordinate position of the lower right corner of the region R _* .

b. Use the keyboard and mouse to determine the image area R _detection for micro-behavior recognition to be detected. The calculation method of the area coordinate parameters is:

The white rectangular area in Figure 7 is the calculated micro-behavior recognition detection area.

Among them, the steps of micro-behavior recognition detection include:

Detect the key point information on the hand features that appears in the micro-behavior recognition detection area (including finger tips, finger joints, palm center position and wrist joint position), and then based on the key points of the hand features in the video sequence The trajectory is used to determine whether the hand of the person to be tested is active or stationary.

Specifically, it detects whether there are hands in the area and detects the key point information of both hands. The key point information of both hands includes the fingertips and joints of each finger, as well as the center position of the palm and the position of the wrist joint; see Figure 8;

Through the trajectory of the key points of both hands in the video sequence, it can be determined whether the hands are in a static state or whether they are typing on the keyboard or controlling the mouse. If both hands are operating the mouse and keyboard, it means that the person being tested is working and is not distracted. See Figure 9.

3. Detection of the eye trace of the person to be tested on the monitor screen and screen attention heat map.

Gaze track and screen attention heat map detection means to separately obtain the gaze track of the person to be detected on the screen, and combine it with the attention area on the computer screen to determine whether the person to be detected is distinguished by comparing the similarity between the two. Heart.

Among them, the steps to obtain the sight trajectory include:

a. Extract and analyze the eye characteristics of the person to be tested, and calculate the direction of sight through the eye characteristics; you can choose the deep neural network as a tool and fit the two angle parameters of the eyeball through the detected eye pictures: pitch( vertical direction) and yaw (horizontal direction); see Figure 2.

b. Detect several feature values of the face (such as the currently commonly used 68 feature points), and compare them with the face feature values in standard poses to estimate the pose relationship between the face and the camera; see the attachment Figures 3 and 4;

In the real facial image, several unoccluded feature points are selected and the head pose estimation is performed using the PnP method to obtain the pose information of the face relative to the camera imaging plane.

c. Combined with the posture relationship between the face of the person to be tested and the monitor screen, calculate the line of sight trajectory of the person to be tested on the monitor screen, as shown in Figure 5;

(1) Through the relationship between the spatial coordinates of the installed camera and the plane position of the computer screen, the conversion matrix R between the face and the screen can be calculated;

(2) The pose of the face relative to the screen can be calculated through the transformation matrix, and the position of the line of sight on the screen P _t can be calculated in real time through the estimated distance between the face and the screen;

(3) Through accumulation over time, the position sequence (P _i ) of the line of sight on the screen can be obtained, where i≥0, and the length of the sequence is related to the sampling frequency and sampling duration.

Among them, the detection program detects the content of the screen, uses the attention detection network to analyze the content of the screen, and generates a screen attention heat map, which can be seen in Figure 6.

The heat map is the same size as the screen, and the brightness of each position is proportional to the importance of the corresponding content. The brightness information at each position can be converted into a two-dimensional distribution Prob _S. The conversion method is:

Among them, Bright(i,j) is the brightness value of the screen coordinate i-j position. It is easy to know from the calculation results

Among them, width and height are the resolution of the screen respectively, and (i, j) is the coordinate position of the sampling point on the screen.

Then, the fit between the attention heat map and the gaze trajectory is calculated and compared with the set threshold. When the threshold is exceeded, it is considered that the person being detected is working seriously, otherwise there is a possibility of distraction.

The calculation method is to regard the eye's line of sight trajectory on the screen and the screen's attention heat map as a two-dimensional distribution. After normalizing the two distributions respectively, and sampling according to the screen resolution, the distribution function of the eye track Prob _Eye is obtained, and the distribution of the screen attention heat map is Prob _Screen . The similarity between the two can be calculated by crossing Entropy calculation:

Among them, width and height are the resolution of the screen respectively, and (i, j) is the coordinate position of the sampling point on the screen.

4. Detection of the trend and prior knowledge of the line of sight trajectory of the person to be tested’s eyes on the monitor screen.

After detecting the degree of agreement, it is also necessary to determine the directionality of the line of sight trajectory. Due to the particularity of computer operations, the operator's line of sight generally starts from the middle and moves from the upper left to the lower right. Analyze the movement trend of the sight trajectory to see whether it is consistent with prior knowledge.

Assume that the coordinate information of the i-th point of the line of sight trajectory is Trail _i (x, y). The direction of movement of the line of sight on the screen can be determined by calculating the gradient of the line of sight trajectory.

then pass The direction of trajectory movement can be judged by the positive and negative signs of the two dimensions.

5. Blink feature detection and analysis.

Blink features refer to some blink-related information that is analyzed based on the feature point parameter technology after computer vision technology collects the eye feature points of the person to be detected.

1. Use image processing technology to extract the position of the face in the image and segment the eye area;

2. From the eye area, the positions of several key points of the eyes are detected, including the positions of the inner canthus, outer canthus, upper eyelid and lower eyelid; see Figure 10;

3. Calculate the ratio of the distance between the upper and lower eyelids to the distance between the inner and outer canthus of the eye. Because the distance between the left and right canthus remains unchanged, and the distance between the upper and lower eyelids is related to the state of opening and closing the eyes, the blink frequency and the duration of eye closure can be calculated based on the ratio of the distance between the upper and lower eyelids to the distance between the inner and outer canthus, combined with time information;

Assuming that the distance between the inner canthus P ₁ and the outer canthus P ₄ is Dis _W , and the distance between the upper eyelid P ₂ and the lower eyelid P ₆ is Dis _H , then the calculation method of the blink discrimination parameter is:

By periodically calculating the blink discrimination parameter Blink (such as 10Hz) and drawing the time-based change trend, you can obtain a blink state change diagram; see Figure 11.

4. According to the duration of eyes _closed , it can be judged whether the person being monitored is working tiredly and the abnormal state of the person to be detected can be judged.

6. After aggregating the detection results, the distraction state of the person to be detected can be analyzed through weighted calculation.

In addition, assuming that the gaze track detection result is Result _praj , the hand micro-behavior detection result is Result _aetion , and the blink detection result is Result _blink , then the detection result of the entire system is

Result＝α×Result _praj +β×Result _action +(1-α-β)×Result _blmk ;

Among them, 0≤α≤1, 0≤β≤1, 0≤α+β≤1.

The concentration monitoring method provided by the present invention is different from the common contact distraction detection method in the past. It adopts a non-contact detection method, which does not require the person to be tested to wear equipment. It only needs to use non-contact equipment such as cameras and cameras to collect images. Information, concealment and convenience are better, and the status of the person to be tested is determined through various methods such as micro-behavior recognition, gaze trajectory and screen attention heat map detection, blink feature detection, etc., and through weighted calculation (the weight value of each indicator can be determined through experiments), comprehensively analyze the concentration of the person to be tested, and the test results obtained based on this are more accurate than a single judgment method. Compared with the existing car driver distraction detection method, the concentration monitoring method provided by the present invention is more concealed and convenient, and has higher detection accuracy, and can be suitable for operations such as aviation dispatch and nuclear power safety control. Human distraction detection in environments.

It should be noted that the concentration monitoring method of the present invention uses non-contact devices such as cameras to collect required image information, such as collecting images of the person's hands and keyboard and mouse for micro-behavior recognition detection, and for blink detection. Images of the eyes of the person to be tested, images of faces used to detect gaze trajectories, etc., and then a preliminary judgment is made based on whether the hands of the person to be tested appear in the micro-behavior recognition detection area, and then based on the presence of the hand of the person to be tested in the micro-behavior recognition detection area The activity status of the hands in the detection area is further judged, and then further judgment is made by comparing the line of sight trace of the person to be tested on the monitor screen with the screen attention heat map, and then the judgment is made by comparing the line of sight trace with prior knowledge. Further judgment is made, and then further judgment is made based on the blink frequency of the person to be tested. Finally, various test results are collected and weighted calculations are performed to determine the current state of the person to be tested (working state, distracted state). The present invention can avoid many misjudgments and missed judgments by combining multiple detection methods, and effectively improves the accuracy of detection. Among them, in the process of micro-behavior recognition, if the hands are not operating the keyboard and mouse, they may or may not be distracted. If the person's status is directly determined through the traditional single judgment method (for example, whether the person is working or working based on whether the hands are operating the keyboard and mouse), (Mind), there will obviously be misjudgments, and the present invention considers multiple possibilities and makes distraction detection more accurate through multi-level detection and step-by-step multiple optimization.

In addition, this embodiment also provides a concentration monitoring system, which includes a storage unit and an arithmetic unit. The storage unit stores a concentration monitoring program. The concentration monitoring program is run through the arithmetic unit to perform the concentration monitoring method described above. A step of.

The above embodiments are preferred implementation solutions of the present invention. In addition, the present invention can also be implemented in other ways. Any obvious substitutions are within the protection scope of the present invention without departing from the concept of the technical solution.

In order to allow those of ordinary skill in the art to more easily understand the improvements of the present invention over the prior art, some drawings and descriptions of the present invention have been simplified, and for the sake of clarity, some other elements have been omitted in this application document. Those of ordinary skill in the art should realize that these omitted elements may also constitute the content of the present invention.

Claims (10)

1. The concentration monitoring method is characterized by comprising the following steps of:

1. acquiring image information of a person to be detected and equipment by using an image acquisition device;

2. taking the region where the human-computer interaction input equipment is located in the image as a micro-motion recognition detection region, detecting whether the hand of the person to be detected appears in the region, performing micro-motion recognition detection on the hand appearing in the region, obtaining the state of the hand of the person to be detected, and judging whether the hand is moving or not;

3. acquiring a sight line track of a person to be tested on a display screen, and comparing the similarity between the sight line track and the attention thermodynamic diagram of the display screen;

4. analyzing the movement trend of the sight line track, and judging whether the movement trend is consistent with the priori knowledge;

5. detecting eye features of a person to be detected, obtaining blink frequency of the person to be detected, and judging whether the blink frequency is normal or not;

6. and (5) integrating the detection results and judging whether the personnel to be detected is in a working state or not through weighting calculation.

2. The concentration monitoring method of claim 1 wherein: in the first step, the image acquisition device comprises a camera, the acquired image information of the person to be detected comprises the image information of the eye feature, the hand feature and the face feature of the person to be detected, and the acquired image information of the equipment comprises the image information of the man-machine interaction input equipment and the display screen.

3. The concentration monitoring method according to claim 1, wherein in the second step, the human-computer interaction input device includes a keyboard and a mouse, and the step of determining the micro-behavior recognition detection area includes:

a. detecting and determining the region position R of the keyboard and the mouse in the image _keyboard And R is _mouse Wherein R is _keyboard And R is _mouse The method comprises the following steps:

R _keyboard ＝((x _left ，y _left )，(x _right ，y _right ))；

R _mouse ＝((x _left ，y _left )，(x _right ，y _right ))；

wherein, (x) _left ， _yleft ) Representation area R _* Upper left corner coordinate position, (x) _right ，y _right ) Representation area R _* The coordinate position of the lower right corner;

b. determining a micro-behavior recognition detection region R through the region positions of a keyboard and a mouse _detection The region coordinate parameters are:

4. the concentration monitoring method according to claim 1, wherein in the second step, the step of detecting the micro-behavior recognition includes:

and detecting key point information including finger tips, finger joints, palm center positions and wrist joint positions on the hand features in the micro-behavior recognition detection area, and judging whether the hand of the person to be detected is moving or stationary according to the track of the key points of the hand features in the video sequence.

5. The concentration monitoring method according to claim 1, wherein in the step three, the step of acquiring the line-of-sight trajectory includes:

a. extracting and analyzing eye features of a person to be detected to obtain a sight line direction;

b. detecting a plurality of characteristic values of a face, and comparing the characteristic values with the characteristic values of the face under a standard posture to obtain a posture relation between the face and an image acquisition device;

c. and calculating the sight track of the person to be tested on the display screen by combining the pose relation between the face of the person to be tested and the display screen.

6. The concentration monitoring method according to claim 5, wherein in step three, the distribution function of the screen attention thermodynamic diagram is Prob _s ：

Wherein Bright (i, j) is the luminance value at the position of screen coordinates i-j,

where width and height are the resolution of the screen, respectively, (i, j) is the coordinate position of the sampling point on the screen.

7. The method of claim 6, wherein in the third step, the step of detecting the similarity of the gaze track and the screen attention thermodynamic diagram comprises:

normalizing the sight line track and the screen attention thermodynamic diagram, and sampling according to the screen resolution to obtain a distribution function P of the sight line trackrob _Eye The distribution function of the screen attention thermodynamic diagram is Prob _Screen The similarity between the two is obtained through cross entropy calculation:

wherein width and height are the resolution of the screen respectively, (i, j) are the coordinate positions of the sampling points on the screen;

and comparing the similarity of the calculated sight line track and the screen attention thermodynamic diagram with a preset threshold value, and judging whether the tested person is in a working state.

8. The concentration monitoring method according to claim 1, wherein in step four, the priori knowledge is that the line of sight of the person under test starts from the middle of the display screen and moves from the upper left position to the lower right position; the step of judging the direction of the movement of the sight line track comprises the following steps:

judging the moving direction of the sight line on the screen by calculating the gradient of the sight line track:

wherein Trail is _i (x, y) is coordinate information of an ith point of the line-of-sight track;

by passing throughThe sign of the two dimensions in the track is determined.

9. The concentration monitoring method according to claim 1, wherein in the fifth step, the step of detecting the blink frequency of the person to be tested includes:

a. extracting the face position in the image and dividing the eye region;

b. detecting positions of a number of key points of the eye from the eye region, including positions of inner corners of the eye, outer corners of the eye, upper eyelid and lower eyelid;

c. calculating the ratio of the upper eyelid distance to the lower eyelid distance to the inner and outer eyelid distance, and calculating the blink frequency and the duration of eye closure by combining time information;

d. the Blink discrimination parameters Blink are:

wherein Dis _w Dis is the distance between the inner and outer canthus _H Is the distance between the upper eyelid and the lower eyelid;

periodically calculating a Blink discrimination parameter Blink to obtain a time-based variation trend;

e. according to the duration Time of eye closure _close Judging whether the tested person works in fatigue or not and judging the abnormal state of the tested person:

wherein threshold is a threshold.

10. Concentration monitoring system comprising a storage unit and an arithmetic unit, the storage unit storing a concentration monitoring program which is run by the arithmetic unit to perform the steps in the concentration monitoring method according to any one of claims 1-9.