CN116883946B - Real-time detection method, device, equipment and storage medium for abnormal behavior of the elderly - Google Patents

Abstract

The present invention relates to the technical field of real-time detection of abnormal behavior of the elderly. It solves the problem in the existing technology that the abnormal behavior of the elderly cannot be accurately detected in real time and the user's elderly care experience is poor. It provides a method and device for real-time detection of abnormal behavior of the elderly. Equipment and storage media. The method includes: obtaining a real-time video stream in an elderly care scenario, decomposing the real-time video stream into multiple frames of real-time images; performing feature extraction on each of the real-time images, and obtaining a target containing the elderly based on the extracted human body feature information. image; perform motion analysis on each of the target images, and output motion characteristic information of the preset parts of the elderly; detect the preset abnormal behavior of the elderly based on the motion characteristic information, and output the detection results; according to the detection results, when When abnormal behavior of the elderly is identified, a safety reminder is issued to the user. The invention realizes real-time and accurate detection of abnormal behavior of the elderly and improves the user's elderly care experience.

Description

Real-time detection method, device, equipment and storage medium for abnormal behavior of the elderly

Technical field

The invention relates to the technical field of real-time detection of abnormal behavior of the elderly, and in particular to a real-time detection method, device, equipment and storage medium for abnormal behavior of the elderly.

Background technique

As the trend of global population aging continues to increase, how to provide high-quality and effective care services for the elderly, especially how to promptly detect and deal with various abnormal behaviors in their daily lives, has become an important issue of social concern. In recent years, the rapid development of intelligent technology, especially the application of artificial intelligence, machine learning, Internet of Things and other technologies, has provided new possibilities for real-time detection of abnormal behaviors of the elderly. By utilizing these technologies, the accuracy and real-time nature of detection can be improved, thereby better ensuring the life safety of the elderly. However, how to design and implement an effective and accurate real-time detection method for abnormal behavior of the elderly is still an important research topic. Traditional abnormal behavior detection of the elderly mainly relies on manual observation or some simple sensor devices, such as pressure sensors, infrared sensors, etc. These methods have obvious limitations. First of all, manual observation requires a lot of manpower and material resources, and 24-hour uninterrupted monitoring is not possible. Secondly, the elderly may have difficulty concentrating due to cognitive decline, exhibit abnormal wandering behavior, forget their destination or become disoriented, such as cognitive impairment: Alzheimer's disease and other cognitive impairment diseases may cause the elderly to get lost and wander. Because they have difficulty remembering the environment and directions; Depression or anxiety: Emotional problems may cause the elderly to wander intentionally or unintentionally in search of comfort or emotional relief; Changes in living environment: The elderly may become confused and wander due to changes in residence or fewer relatives around them; The elderly may also be unable to stand due to bone diseases (such as fractures), muscle atrophy or arthritis; neurological diseases such as stroke and Parkinson's disease may affect the elderly's balance and standing ability; among them, one of the main symptoms of Parkinson's disease is One is hand tremors, which may make it difficult for the elderly to control hand movements. Although simple sensor devices can monitor around the clock, they can only provide limited information and are difficult to accurately judge the abnormal behavior of the above-mentioned elderly people.

The existing Chinese patent CN116189232A discloses a method and system for detecting abnormal behavior of the elderly in nursing homes based on machine vision. The method includes: obtaining video data captured by a camera; performing video data containing target objects based on different abnormal behavior recognition models. Abnormal behavior detection; wherein, the abnormal behavior recognition model includes an active raising hand for help recognition model, a falling behavior recognition model and a permanent immobility behavior recognition model; for the abnormal behavior of raising a hand to actively ask for help, an active raising hand for help recognition model is used. Detection, the active hand-raising for help recognition model is a target detection algorithm based on YOLOv5; the fall behavior recognition model uses the above-mentioned YOLOV5-CBAM detection model to detect target objects. For the human body's falling and upright states, the shape of its circumscribed rectangular box will be Obviously different, so the height-to-width ratio of the human body detection frame can be judged to determine whether the detection target has fallen; for the abnormal behavior of standing still for a long time, the standing still behavior recognition model is used to detect, based on the above YOLOV5-CBAM detection The model detection results are combined with the Deepsort target tracking model to track existing targets, create new objects for new targets, and delete targets that have not been matched for a long time. The above method only uses the YOLOv5 target detection algorithm for abnormal behavior detection. The performance and accuracy of the target detection algorithm may be affected by some factors, such as target occlusion, illumination changes, or target scale changes, which may lead to inaccuracy in target detection. , thereby affecting the results of abnormal behavior detection.

For this reason, how to accurately detect the abnormal behavior of the elderly in real time and improve the user's elderly care experience is an urgent problem to be solved.

Contents of the invention

In view of this, the present invention provides a real-time detection method, device, equipment and storage medium for the abnormal behavior of the elderly to solve the problem in the existing technology that the abnormal behavior of the elderly cannot be accurately detected in real time and the user's elderly care experience is poor. .

The technical solution adopted by the present invention is:

In a first aspect, the present invention provides a real-time detection method for abnormal behavior of the elderly, which is characterized in that the method includes:

S1: Obtain the real-time video stream in the elderly care scenario, and decompose the real-time video stream into multiple frames of real-time images;

S2: Perform feature extraction on each of the real-time images, and obtain the target image containing the elderly based on the extracted human body feature information;

S3: Perform motion analysis on each of the target images, and output the motion characteristic information of the preset parts of the elderly;

S4: Based on the motion characteristic information, detect the preset abnormal behavior of the elderly and output the detection results;

S5: Based on the detection results, when abnormal behavior of the elderly is identified, a safety reminder is issued to the user.

Preferably, the S2 includes:

S21: Detect each of the images according to the target detection algorithm, and output the upper body position information of the human body;

S22: Based on the upper body position information, use the target classification network to extract features of the upper body area of the human body, and output the upper body feature information of the human body;

S23: Input the upper body feature information into the pre-trained classifier and output the classification result;

S24: According to the classification result, when the number of image frames classified as elderly in each of the real-time images is greater than the preset number of image frames, obtain the target image.

Preferably, the S3 includes:

S31: Use the target matching algorithm to track the old man in the target image;

S32: Input the target image into the pre-trained target detection model, and output the preset part image of the elderly, where the preset part image includes hand and torso images;

S33: Perform optical flow field calculation on the torso image and hand image, and output the average optical flow field of the torso image and the center optical flow field of the hand image as the motion feature information.

Preferably, the S4 includes:

S41: Obtain preset abnormal behaviors of the elderly, wherein the abnormal behaviors of the elderly include at least one of the following behaviors: abnormal wandering of the elderly, inability to stand, and trembling hands of the elderly;

S42: Obtain a preset time interval, and based on the variance of the average optical flow field within the time interval, when it is determined that there is repeated movement of the old man's torso, identify the old man's abnormal wandering behavior;

S43: Identify the old man's inability to stand based on the first average value of the average optical flow field and the second average value of the central optical flow field within the time interval;

S44: Based on the central optical flow field within the time interval, when it is determined that there is weak movement of the old man's hand, identify it as trembling behavior of the old man's hand.

Preferably, the S42 includes:

S421: Obtain the time interval and the preset first optical flow field threshold;

S422: Calculate the average optical flow field within the time interval, and output the variance of the average optical flow field;

S423: When the variance is greater than the first optical flow field threshold, identify the old man's abnormal wandering behavior.

Preferably, the S43 includes:

S431: Obtain the preset second optical flow field threshold and the third optical flow field threshold;

S432: Calculate the average optical flow field and the central optical flow field within the time interval, and output the first average value and the second average value;

S433: When the first average value is less than the second optical flow field threshold and the second average value is greater than the third optical flow field threshold, identify the old man's behavior as being unable to stand.

Preferably, the S44 includes:

S441: Obtain the preset fourth optical flow field threshold;

S442: Perform Fourier transform on the central optical flow field within the time interval, and output the frequency component corresponding to the central optical flow field;

S443: Compare each of the frequency components. When the maximum value of the frequency component is greater than the fourth optical flow field threshold, identify the hand trembling behavior.

In a second aspect, the present invention provides a device for real-time detection of abnormal behavior of the elderly, which device includes:

A real-time image acquisition module, used to acquire a real-time video stream in an elderly care scenario, and decompose the real-time video stream into multiple frames of real-time images;

The elderly identification module is used to extract features from each of the real-time images, and obtain target images containing the elderly based on the extracted human body feature information;

A motion feature extraction module is used to perform motion analysis on each of the target images and output motion feature information of the preset parts of the elderly;

The abnormal behavior detection module of the elderly is used to detect the preset abnormal behavior of the elderly based on the motion characteristic information and output the detection results;

A safety reminder module is used to issue a safety reminder to the user when abnormal behavior of the elderly is identified based on the detection results.

In a third aspect, embodiments of the present invention also provide an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the above embodiments are implemented. The first aspect of the method.

In a fourth aspect, embodiments of the present invention also provide a storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method of the first aspect in the above embodiments is implemented.

To sum up, the beneficial effects of the present invention are as follows:

The invention provides a real-time detection method, device, equipment and storage medium for abnormal behavior of the elderly. The method includes: obtaining a real-time video stream in an elderly care scene, decomposing the real-time video stream into multiple frames of real-time images; Real-time image feature extraction is performed, and a target image containing the elderly is obtained based on the extracted human body feature information; motion analysis is performed on each of the target images to output motion feature information of the preset parts of the elderly; based on the motion feature information, the preset It is designed to detect the abnormal behavior of the elderly and output the detection results; based on the detection results, when the abnormal behavior of the elderly is identified, a safety reminder is issued to the user. This invention analyzes the real-time video stream and detects abnormal behaviors, analyzes the motion characteristic information and abnormal behaviors of the elderly, and obtains individualized understanding of the elderly, thereby accurately detecting the abnormal behaviors of the elderly. Once the abnormal behaviors of the elderly are detected, The system can send safety reminders to users. Timely safety reminders and intervention measures can help reduce the threats to the health and safety of the elderly from accidents and emergencies, and improve the user's elderly care experience.

Description of drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the drawings required to be used in the embodiments of the present invention will be briefly introduced below. For those of ordinary skill in the art, without exerting creative efforts, they can also Other drawings can be obtained based on these drawings, and these are all within the protection scope of the present invention.

Figure 1 is a schematic flow chart of the overall work of the real-time detection method for abnormal behavior of the elderly in Embodiment 1 of the present invention;

Figure 2 is a schematic flowchart of obtaining a target image containing an elderly person in Embodiment 1 of the present invention;

Figure 3 is a schematic flow chart of motion analysis of a target image in Embodiment 1 of the present invention;

Figure 4 is a schematic flowchart of identifying abnormal behavior of the elderly in Embodiment 1 of the present invention;

Figure 5 is a schematic flowchart of identifying abnormal wandering behavior of the elderly in Embodiment 1 of the present invention;

Figure 6 is a schematic flowchart of identifying the behavior of an elderly person unable to stand in Embodiment 1 of the present invention;

Figure 7 is a schematic flowchart of identifying the hand trembling behavior of the elderly in Embodiment 1 of the present invention;

Figure 8 is a structural block diagram of a real-time detection device for abnormal behavior of the elderly in Embodiment 2 of the present invention;

Figure 9 is a schematic structural diagram of an electronic device in Embodiment 3 of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. In the description of the present invention, it should be understood that the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", The orientations or positional relationships indicated by "top", "bottom", "inner", "outside", etc. are based on the orientations or positional relationships shown in the drawings. They are only for the convenience of describing the present application and simplifying the description, and are not indicated or implied. The devices or elements referred to must have a specific orientation, be constructed and operate in a specific orientation and therefore are not to be construed as limitations of the invention. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprising..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element. If there is no conflict, the embodiments of the present invention and various features in the embodiments can be combined with each other, and they are all within the protection scope of the present invention.

Example 1

Please refer to Figure 1. Embodiment 1 of the present invention discloses a real-time detection method for abnormal behavior of the elderly. The method includes:

S1: Obtain the real-time video stream in the elderly care scenario, and decompose the real-time video stream into multiple frames of real-time images;

Specifically, a real-time video stream in an elderly care scenario is obtained. The real-time video stream refers to a color video stream captured by a camera installed at an oblique perspective, and the real-time video stream is decomposed into N frames of real-time images; the camera at an oblique perspective It provides more comprehensive video information from multiple angles and can monitor the status and behavior of the elderly in real time. At the same time, by decomposing the real-time video stream into multiple frames of real-time images, guardians can remotely monitor the elderly's condition. In this way, potential problems can be discovered in time, help and remote guidance can be provided, increasing the sense of security and quality of care for the elderly.

S2: Perform feature extraction on each of the real-time images, and obtain the target image containing the elderly based on the extracted human body feature information;

Specifically, by performing feature extraction on each of the real-time images, useful human body feature information can be extracted from the images through image processing and computer vision technology. Based on the extracted human body feature information, the target image containing the elderly can be obtained, that is, the parts related to the elderly in the image can be extracted. , among which, in the feature extraction stage, image processing and computer vision technology are used to analyze and process each real-time image to extract distinctive human body feature information. The human body feature information at least includes the outline, key points, postures, movement trajectories, expressions, etc. of the human body. By analyzing the human body feature information, target images containing the elderly are identified and extracted. Through feature extraction, focus on the parts of the image related to the elderly, reduce unnecessary background interference, and improve the accuracy and precision of target positioning, thereby better capturing the movements, behaviors, and status changes of the elderly; the extracted human body Feature information is used to detect abnormal behaviors of the elderly. By analyzing the characteristics of the elderly's posture, movement trajectories, etc., abnormal behaviors such as falling, wandering, and sudden departure can be detected. This helps to promptly detect emergencies of the elderly and take appropriate safety measures. . At the same time, by extracting human body feature information, the amount of image data that needs to be processed can be reduced, thereby improving the efficiency of image processing and analysis. Only focusing on the parts related to the elderly can reduce the consumption of computing resources and time, allowing the system to respond more quickly. and processing live video streams.

In an embodiment, please refer to Figure 2, the S2 includes:

S21: Detect each of the images according to the target detection algorithm, and output the upper body position information of the human body;

Specifically, download and prepare the YOLOv8s model and its corresponding weight file, which contains pre-trained weight parameters for the target detection task; use an appropriate deep learning framework (such as PyTorch, TensorFlow) to load the YOLOv8s model and its weights ; For the N frames of images decomposed, process each frame in turn: perform preprocessing operations on each frame of image, such as image scaling, normalization, channel adjustment, etc., to adapt to the input requirements of the YOLOv8s model, which will be preprocessed The image is input into the YOLOv8s model to perform target detection operations. The model will output the position information of the pedestrian target and its bounding box detected in the image, and extract the corresponding upper body area based on the boundary box position information of the pedestrian target. Among them, the specific area range of the upper body can be defined according to the needs, such as from the head to the waist, save the extracted upper body image for subsequent use or display, and repeatedly process the next frame of image until all N frames of images have been traversed.

S22: Based on the upper body position information, use the target classification network to extract features of the upper body area of the human body, and output the upper body feature information of the human body;

Specifically, download and prepare the ResNet model and its corresponding weight files suitable for pedestrian feature extraction. These files contain pre-trained weight parameters for feature extraction tasks, and are loaded using an appropriate deep learning framework (such as PyTorch, TensorFlow). The ResNet model and its weights traverse the pedestrian upper body images obtained in the previous step: for each upper body image frame, the following processing is performed in sequence. Preprocessing operations are performed on each upper body image, such as image scaling, normalization, channel adjustment, etc., to adapt to the input requirements of the ResNet model. Input the preprocessed upper body image into the ResNet model to perform feature extraction operations. The ResNet model will output the feature vector corresponding to the pedestrian's upper body image, and save the extracted pedestrian's upper body feature vector, which can be stored in the form of a matrix or vector. Used for subsequent pedestrian analysis, anomaly detection and other tasks, the next frame of upper body images is repeatedly processed until all pedestrian upper body images have been traversed. Through the above steps, the ResNet network can be used to extract features from the pedestrian upper body images and obtain discriminative Feature vector. These feature vectors can be used for further pedestrian recognition, behavior analysis, anomaly detection and other tasks, thereby improving the understanding and judgment of pedestrian behavior in elderly care scenarios.

S23: Input the upper body feature information into the pre-trained classifier and output the classification result;

Specifically, obtain the already trained SVM classifier for elderly classification, save its relevant parameters and weights, use a machine learning library (such as scikit-learn) to load the already trained SVM classifier model and its weights, and The extracted pedestrian upper body feature vector is used as input data to prepare the feature vector for classification. According to the requirements of the SVM classifier, necessary preprocessing is performed on the feature vector, such as normalization, standardization or other conversion operations, and the preprocessed The feature vector is input into the SVM classifier for classification operation. SVM will output a classification label corresponding to each feature vector, indicating whether the upper body of the pedestrian belongs to an old person. For the upper body of the pedestrian in each image, it is judged according to the classification result of SVM whether it is Belongs to the elderly. Further decisions can be made based on the confidence or probability of the classification label. Based on the classification results, the judgment result of whether the upper body of the pedestrian in each image belongs to an old person can be output, for example, in binary form (elderly/non-elderly) or in the form of a probability value. By using the trained SVM classifier to classify the extracted upper body features of pedestrians, it is judged whether they belong to the elderly. This can realize the identification and classification of the elderly, and provide a basis for subsequent analysis of the elderly's behavior, abnormality detection, etc.

S24: According to the classification result, when the number of image frames classified as elderly in each of the real-time images is greater than the preset number of image frames, obtain the target image.

Specifically, for each frame of image, record the classification result of the SVM classifier, label it as elderly or non-elderly, and calculate the number of times that all image frames are judged to be elderly, that is, the number of image frames judged to be elderly. According to the total number of frames N, Calculate the threshold for image frames exceeding 2/3, that is, multiply N times 2/3, and compare the relationship between the number of judgments made by the old man and the threshold. If the number of elderly judgments is greater than the threshold, that is, more than 2/3 of the image frames are judged to be elderly, then the pedestrian target frame is considered to be an elderly person, the corresponding target image is obtained, and the judgment result of whether the pedestrian target frame is an elderly person is output, which can be expressed in binary form (elderly/non-elderly) or other forms of expression. By comprehensively analyzing the classification results of N frames of images and determining whether the pedestrian target frame is an elderly person based on the condition that more than 2/3 of the image frames are judged to be elderly, the accuracy and stability of the judgment can be improved and the accuracy of the elderly target can be ensured. Accurate identification.

S3: Perform motion analysis on each of the target images, and output the motion characteristic information of the preset parts of the elderly;

Specifically, motion analysis is performed on each of the target images, and motion feature information of the preset parts of the elderly is output; and relevant motion feature information is extracted by analyzing the motion of the preset parts of the elderly in the target image. This characteristic information can help understand the behavior patterns of the elderly and monitor abnormal behaviors, thereby enabling behavioral monitoring and health status assessment of the elderly.

In an embodiment, please refer to Figure 3, the S3 includes:

S31: Use the target matching algorithm to track the old man in the target image;

Specifically, the elderly target in the current frame is detected through a target detection algorithm (such as YOLO, SSD, etc.), and its bounding box position information is obtained. Calculate the IOU between the bounding box in the current frame and the bounding box in the previous frame, measure their overlap, perform target matching based on the IOU calculation results, and adopt the maximum IOU matching strategy, that is, compare the bounding box in the current frame with the previous frame The bounding box with the largest IOU in the frame is matched. According to the target matching results, the tracking information of the elderly target is updated, including the bounding box position, speed, motion status, etc., and tracking state estimation is performed using methods such as Kalman filtering and Kalman-IOU. and update, repeatedly detecting, IOU calculation, and target matching the elderly target in subsequent frames, thereby achieving tracking and updating of the elderly target in the target image. The use of IOU matching for elderly target tracking has the advantages of simplicity, good real-time performance, high robustness, continuous tracking and target correlation, and is suitable for elderly monitoring, behavior analysis, security monitoring and other scenarios.

S32: Input the target image into the pre-trained target detection model, and output the preset part image of the elderly, where the preset part image includes hand and torso images;

Specifically, the YoloV8s target detection algorithm is used to process the image to detect the hands and torso of the old man, where the hands refer to the palms and the torso refers to the body and limbs (except the palms). According to the results of the target detection , extract the old man's head, hands and body torso areas from the image, intercept the corresponding image area by using the target's bounding box information, and output the old man's preset part image, where the preset part image includes the hands and torso head image; the YoloV8s algorithm has high positioning and recognition accuracy, can accurately detect the head, hands and body torso of the elderly, and provides reliable target location information. At the same time, the YoloV8s algorithm has faster processing speed and High computational efficiency, suitable for real-time application scenarios, and can quickly detect different body parts of the elderly in real-time video streams. By detecting the head, hands and body torso, the characteristic information of different body parts of the elderly can be obtained, thereby providing a more comprehensive and diversified analysis basis.

S33: Perform optical flow field calculation on the torso image and hand image, and output the average optical flow field of the torso image and the center optical flow field of the hand image as the motion feature information.

Specifically, an optical flow algorithm (such as the Lucas-Kanade algorithm, Farneback algorithm, etc.) is used to calculate the optical flow vector of each pixel in the torso image. The optical flow vector represents the movement direction and speed of the pixel between adjacent frames. The optical flow vectors of all pixels in the torso image are averaged to obtain the average optical flow vector value of the torso image. This can be achieved by accumulating the x and y components of all optical flow vectors and dividing by the number of pixels; extract the optical flow vector of the area based on the position of the center point area of the hand image. You can choose the center point of the hand as the center and extract the optical flow vector in the area with a certain area as the radius. By calculating the average optical flow vector value of the torso image, the motion information of the overall torso part can be obtained. This helps determine the overall movement status and movement patterns of the elderly, such as whether there are abnormal movements or wandering behaviors. The elderly usually move slowly due to physical aging and functional decline. Since the optical flow method is a method based on pixel displacement calculation, it is very sensitive to slow movements. Even if the elderly move slowly, the optical flow method can effectively Capture these tiny displacements to extract their motion features; the movement of the elderly is often slow, and in many cases real-time monitoring is required to prevent potential accidents. The optical flow method has the ability to process images in real time and can Extract motion features in the video stream in real time and provide timely feedback on the elderly person's action status so that necessary measures can be taken early. Although the movement of the elderly is relatively slow, the optical flow method has a high precision in displacement calculation and can accurately capture the tiny displacements between pixels. This accuracy makes the optical flow method excellent in tracking the movement trajectories and movement changes of the elderly. Helping to provide more accurate behavioral monitoring and analysis results, optical flow method is a technology based on image processing that does not require additional sensor wearing or intervention for the elderly. This non-invasiveness makes optical flow method more effective in the behavior of the elderly. It is easier to be accepted and applied in monitoring and will not cause discomfort or trouble to the elderly; by calculating the optical flow velocity field in the center point area of the hand image, the specific characteristics of the hand movement can be obtained. The movement of the hand limbs may be inconsistent with the movement of the hand area, so by analyzing the optical flow velocity field in the center point area of the hand, the movement information of the hand can be captured more accurately.

S4: Based on the motion characteristic information, detect the preset abnormal behavior of the elderly and output the detection results;

Specifically, based on the motion characteristic information, the preset abnormal behavior of the elderly is detected and the detection results are output; by detecting the preset abnormal behavior of the elderly, possible problems or emergencies can be discovered in advance. Timely early warning can prompt relevant personnel to take appropriate measures to ensure the safety and health of the elderly.

In an embodiment, please refer to Figure 4, the S4 includes:

S41: Obtain preset abnormal behaviors of the elderly, wherein the abnormal behaviors of the elderly include at least one of the following behaviors: abnormal wandering of the elderly, inability to stand, and trembling hands of the elderly;

Specifically, the elderly may have difficulty concentrating, exhibit abnormal wandering behavior, forget their destination, or become disoriented due to cognitive decline, such as cognitive impairment: Alzheimer's disease and other cognitive impairment diseases may cause the elderly to get lost and wander. , because they have difficulty remembering the environment and directions; Depression or anxiety: Emotional problems may cause the elderly to wander intentionally or unintentionally in search of comfort or emotional relief; Changes in living environment: The elderly may become confused and wander due to changes in residence or fewer relatives around them ; The elderly may also be unable to stand due to bone diseases (such as fractures), muscle atrophy or arthritis; neurological diseases such as stroke and Parkinson's disease may affect the elderly's balance and standing ability; among them, the main symptoms of Parkinson's disease One of them is hand tremor, which may make it difficult for the elderly to control hand movements. First, according to the specific application needs and the abnormal behaviors of the elderly that are of concern, a set of definitions and rules for the elderly's abnormal behaviors are preset, such as abnormal wandering of the elderly, inability to stand, and trembling hands of the elderly for the above problems; specifically, according to actual needs, It can also flexibly define and adjust preset abnormal behaviors of the elderly, and customize them according to specific scenarios and characteristics of the elderly group to improve detection accuracy and adaptability.

S42: Obtain a preset time interval, and based on the variance of the average optical flow field within the time interval, when it is determined that there is repeated movement of the old man's torso, identify the old man's abnormal wandering behavior;

In an embodiment, please refer to Figure 5, the S42 includes:

S421: Obtain the time interval and the preset first optical flow field threshold;

S422: Calculate the average optical flow field within the time interval, and output the variance of the average optical flow field;

S423: When the variance is greater than the first optical flow field threshold, identify the old man's abnormal wandering behavior.

Specifically, the optical flow field L1 is used to represent the average optical flow field of the torso image within a preset time interval. The body torso contains most of the core and balance support areas. In the abnormal behavior monitoring of the elderly, attention should be paid to the body torso. Movement characteristics are crucial for preventing falls and assessing posture stability. The movement of the body's trunk is more integrated. The average optical flow field can capture the overall movement trend and direction, helping to judge whether the elderly's actions are stable in one direction. If the variance of L1 exceeds the set threshold within a time window, then we can consider that there may be wandering behavior: define the optical flow field of the body trunk part from time t to t+Δt as L1(t,t+Δt), let Set a first optical flow field threshold Th1. If Var[L1(t,t+Δt)]>Th1, it is judged that there is abnormal wandering behavior of the elderly; among them, Var[] represents the variance, t represents the current time, and Δt represents the setting The time window is determined by the experiment. Th1 is the first optical flow field threshold set. If the position change of the body exceeds this threshold within Δt time, then it is considered that there is abnormal wandering behavior of the elderly; the variance of the optical flow field It is calculated based on real-time video streams and can monitor the behavior of the elderly in real time. By continuously calculating the variance of the optical flow field, the behavior of the elderly can be continuously tracked and monitored, and abnormal wandering behavior can be discovered in time. At the same time, by setting the appropriate The first optical flow field threshold can be used to judge the degree of body position change based on the variance of the optical flow field. When the variance exceeds the threshold, the abnormal wandering behavior of the elderly can be more accurately identified, improving the accuracy and sensitivity of detection. , the specific setting of the first optical flow field threshold and time window size can be adjusted according to the actual situation. Different elderly people, different environments and different needs may have different threshold and time window settings, which can improve the adaptability and adjustability of the algorithm. sex.

S43: Identify the old man's inability to stand based on the first average value of the average optical flow field and the second average value of the central optical flow field within the time interval;

In an embodiment, please refer to Figure 6, the S43 includes:

S431: Obtain the preset second optical flow field threshold and the third optical flow field threshold;

S432: Calculate the average optical flow field and the central optical flow field within the time interval, and output the first average value and the second average value;

S433: When the first average value is less than the second optical flow field threshold and the second average value is greater than the third optical flow field threshold, identify the old man's behavior as being unable to stand.

Specifically, the optical flow field L2 is used to represent the central optical flow field of the hand image. The hand is one of the most flexible and delicate parts of the human body. It performs various subtle movements in the daily life of the elderly, such as pinching and grasping. , writing, etc., using the central optical flow field can more sensitively detect the subtle movements of pixels around the center point of the hand, helping to capture the fine movement characteristics of the hand; in the behavior monitoring of the elderly, there may be overall camera or image movement, Such as lens movement or camera shake, using the central optical flow field can partially remove the influence of these overall movements, focusing on the subtle movements of the hand image itself, reducing unnecessary interference. The old man's inability to stand up means that his body trunk should maintain a relatively small range of motion, that is, L1 will be small. At the same time, he may try to support it with his hands, so L2 will become larger. When Mean[L1(t, t +Δt)]<Th2 and Mean[L2(t,t+Δt)]>Th3, it is judged that the old man cannot stand, where Mean means averaging, t is the current time, and Δt is the set time window, Th2 and Th3 are the set second optical flow field threshold and the third optical flow field threshold. Using the optical flow field to analyze the movement characteristics of the hand image can more accurately identify the elderly person's inability to stand, distinguish it from other abnormal behaviors, and improve the accuracy and reliability of detection; by comprehensively considering both aspects of the optical flow field Two directions (L1 and L2) can better capture the characteristics of the elderly's inability to stand and avoid misjudgments caused by relying solely on a single feature. By setting appropriate second optical flow field thresholds and third optical flow field thresholds, the behavior can be flexibly adjusted according to the actual situation and the adaptability and adjustability of the algorithm can be improved.

S44: Based on the central optical flow field within the time interval, when it is determined that there is weak movement of the old man's hand, identify it as trembling behavior of the old man's hand.

In one embodiment, please refer to Figure 7, the S44 includes:

S441: Obtain the preset fourth optical flow field threshold;

S442: Perform Fourier transform on the central optical flow field within the time interval, and output the frequency component corresponding to the central optical flow field;

S443: Compare each of the frequency components. When the maximum value of the frequency component is greater than the fourth optical flow field threshold, identify the hand trembling behavior.

Specifically, hand tremor in the elderly refers to abnormalities in the hands of the elderly. The hands tremble unconsciously, and their behavior may cause L2 to frequently change within a small range: set a time interval and count this period. Optical flow field inside the hand; calculate the fast Fourier transform of the hand optical flow; when Max[Freq(L2(t,t+Δt))]>Th4, it is judged as possible hand shaking behavior; where, Freq Represents the frequency component after fast Fourier transform, t is the current time, Δt is the set time window, Th4 is the set fourth optical flow field threshold, and Max represents the maximum value. Using changes in the optical flow field and frequency components to judge hand trembling behavior, we can accurately identify abnormalities in the hands of the elderly and promptly detect possible health problems; by performing fast Fourier transform, we can extract the components in the optical flow field of the hands. Frequency information distinguishes hand trembling behavior from other movement behaviors to improve the accuracy and reliability of detection; setting an appropriate fourth optical flow field threshold Th4 can judge hand trembling behavior based on the actual situation to avoid misjudgment Or missed judgments may occur.

S5: Based on the detection results, when abnormal behavior of the elderly is identified, a safety reminder is issued to the user.

Specifically, when an abnormal behavior of an elderly person is recognized, the system can issue a safety reminder to the user in an appropriate manner. For example, notifications are sent to users through mobile applications, text messages, phone calls or alarm systems to remind them to pay attention to the elderly's situation. By monitoring the elderly's behavior in real time and immediately issuing safety reminders when abnormal behavior is identified, ensuring that timely action can be taken. Measures to deal with possible emergencies; by sending safety reminders to users, it can enhance the safety of elderly care, help users pay attention to the health and safety of the elderly in a timely manner, and avoid potential risks and accidents.

Example 2

Please refer to Figure 8. Embodiment 2 of the present invention also provides a real-time detection device for abnormal behavior of the elderly. The device includes:

A real-time image acquisition module, used to acquire a real-time video stream in an elderly care scenario, and decompose the real-time video stream into multiple frames of real-time images;

The elderly identification module is used to extract features from each of the real-time images, and obtain target images containing the elderly based on the extracted human body feature information;

In one embodiment, the elderly identification module includes:

The upper body position acquisition unit is used to detect each of the images according to the target detection algorithm and output the upper body position information of the human body;

An upper body feature extraction unit is used to extract features of the upper body region of the human body using a target classification network based on the upper body position information, and output the upper body feature information of the human body;

An upper body feature classification unit is used to input the upper body feature information into a pre-trained classifier and output a classification result;

The target image acquisition unit is configured to acquire the target image based on the classification result when the number of image frames classified as elderly in each of the real-time images is greater than a preset number of image frames.

A motion feature extraction module is used to perform motion analysis on each of the target images and output motion feature information of the preset parts of the elderly;

In one embodiment, the motion feature extraction module includes:

An old man tracking unit is used to track the old man in the target image using a target matching algorithm;

The preset part image acquisition unit for the elderly is used to input the target image into the pre-trained target detection model and output the preset part image for the elderly, where the preset part image includes hand and torso images;

An optical flow field calculation unit is configured to perform optical flow field calculation on the torso image and hand image, and output the average optical flow field of the torso image and the center optical flow field of the hand image as the motion feature information.

The abnormal behavior detection module of the elderly is used to detect the preset abnormal behavior of the elderly based on the motion characteristic information and output the detection results;

In one embodiment, the elderly abnormal behavior detection module includes:

The abnormal behavior acquisition unit of the elderly is used to obtain preset abnormal behaviors of the elderly, wherein the abnormal behaviors of the elderly include at least one of the following behaviors: abnormal wandering of the elderly, inability of the elderly to stand, and trembling of the hands of the elderly;

An abnormal wandering identification unit for the elderly is used to obtain a preset time interval and, based on the variance of the average optical flow field within the time interval, when it is determined that there is repeated movement of the elderly man's torso, identify the elderly man's abnormal wandering behavior;

The first optical flow field threshold acquisition subunit is used to acquire the time interval and the preset first optical flow field threshold;

A variance calculation subunit, used to calculate the average optical flow field within the time interval and output the variance of the average optical flow field;

The elderly's abnormal wandering detection subunit is used to identify the elderly's abnormal wandering behavior when the variance is greater than the first optical flow field threshold.

An elderly person cannot stand identification unit, configured to identify the behavior of the elderly person being unable to stand based on the first average value of the average optical flow field and the second average value of the central optical flow field within the time interval;

In one embodiment, the elderly unable to stand identification unit includes:

The second optical flow field threshold and the third optical flow field threshold acquisition subunit are used to acquire the preset second optical flow field threshold and the third optical flow field threshold;

The first average value and the second average value calculation subunit are used to calculate the average optical flow field and the central optical flow field within the time interval, and output the first average value and the second average value;

An old man's inability to stand behavior detection subunit is used to identify that the old man is unable to stand when the first average value is less than the second optical flow field threshold and the second average value is greater than the third optical flow field threshold. standing behavior.

The old man's hand trembling identification unit is used to identify the old man's hand trembling behavior when it is determined that there is weak movement in the old man's hand based on the central optical flow field within the said time interval.

In one embodiment, the elderly hand tremor identification unit includes:

The fourth optical flow field threshold acquisition subunit is used to acquire the preset fourth optical flow field threshold;

A frequency component calculation subunit, used to perform Fourier transform on the central optical flow field within the time interval and output the frequency component corresponding to the central optical flow field;

A hand trembling behavior detection subunit is used to compare each of the frequency components, and when the maximum value of the frequency component is greater than the fourth optical flow field threshold, identify the hand trembling behavior.

A safety reminder module is used to issue a safety reminder to the user when abnormal behavior of the elderly is identified based on the detection results.

Specifically, Embodiment 2 of the present invention provides a real-time detection device for abnormal behavior of the elderly. The device includes: a real-time image acquisition module, used to obtain a real-time video stream in an elderly care scenario, and decompose the real-time video stream into multiple frames of real-time video. image; the elderly recognition module is used to extract features of each of the real-time images, and obtain a target image containing the elderly based on the extracted human body feature information; the motion feature extraction module is used to perform motion analysis on each of the target images, Output the motion characteristic information of the preset parts of the elderly; the abnormal behavior detection module of the elderly is used to detect the preset abnormal behavior of the elderly based on the motion characteristic information and output the detection results; the safety reminder module is used to detect the abnormal behavior of the elderly based on the detection results , when an abnormal behavior of the elderly is identified, a safety reminder is issued to the user. This device analyzes the real-time video stream and detects abnormal behaviors, analyzes the movement characteristics information and abnormal behaviors of the elderly, and obtains personalized understanding of the elderly, thereby accurately detecting the abnormal behaviors of the elderly. Once the abnormal behaviors of the elderly are detected, The system can send safety reminders to users. Timely safety reminders and intervention measures can help reduce the threats to the health and safety of the elderly from accidents and emergencies, and improve the user's elderly care experience.

Example 3

In addition, the real-time detection method of abnormal behavior of the elderly in Embodiment 1 of the present invention described with reference to FIG. 1 can be implemented by an electronic device. FIG. 9 shows a schematic diagram of the hardware structure of the electronic device provided in Embodiment 3 of the present invention.

Electronic devices may include a processor and a memory storing computer program instructions.

Specifically, the above-mentioned processor may include a central processing unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits that may be configured to implement embodiments of the present invention.

Memory may include bulk storage for data or instructions. By way of example and not limitation, the memory may include a Hard Disk Drive (HDD), a floppy disk drive, flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive or two or more A combination of these. Storage may include removable or non-removable (or fixed) media, where appropriate. Where appropriate, the memory may be internal or external to the data processing device. In certain embodiments, the memory is non-volatile solid state memory. In certain embodiments, the memory includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically rewritable ROM (EAROM) or flash memory or A combination of two or more of these.

The processor reads and executes the computer program instructions stored in the memory to implement any of the real-time detection methods of abnormal behavior of the elderly in the above embodiments.

In one example, the electronic device may also include a communication interface and bus. Among them, as shown in Figure 9, the processor, memory, and communication interface are connected through the bus and complete communication with each other.

The communication interface is mainly used to implement communication between modules, devices, units and/or equipment in the embodiments of the present invention.

A bus includes hardware, software, or both, that couples the components of the device to each other. By way of example, and not limitation, the bus may include Accelerated Graphics Port (AGP) or other graphics bus, Enhanced Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), HyperTransport (HT) interconnect, Industry Standard Architecture (ISA) Bus, Infinite Bandwidth Interconnect, Low Pin Count (LPC) Bus, Memory Bus, Micro Channel Architecture (MCA) Bus, Peripheral Component Interconnect (PCI) Bus, PCI-Express (PCI-X) Bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, a bus may include one or more buses. Although embodiments of the invention describe and illustrate a particular bus, the invention contemplates any suitable bus or interconnection.

Example 4

In addition, combined with the real-time detection method of abnormal behavior of the elderly in the above-mentioned Embodiment 1, Embodiment 4 of the present invention can also provide a computer-readable storage medium for implementation. Computer program instructions are stored on the computer-readable storage medium; when the computer program instructions are executed by the processor, any of the real-time detection methods of abnormal behavior of the elderly in the above embodiments can be implemented.

To sum up, embodiments of the present invention provide a real-time detection method, device, equipment and storage medium for abnormal behavior of the elderly.

It is to be understood that this invention is not limited to the specific arrangements and processes described above and illustrated in the drawings. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications and additions, or change the order between steps after understanding the spirit of the present invention.

The functional blocks shown in the above structural block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, or the like. When implemented in software, elements of the invention are programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted over a transmission medium or communications link via a data signal carried in a carrier wave. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. Code segments may be downloaded via computer networks such as the Internet, intranets, and the like.

It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps. That is to say, the steps may be performed in the order mentioned in the embodiments, or may be different from the order in the embodiments, or several steps may be performed simultaneously.

The above are only specific implementations of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described systems, modules and units can be referred to the foregoing method embodiments. The corresponding process will not be described again here. It should be understood that the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be covered. within the protection scope of the present invention.

Claims (7)

1. A real-time detection method for abnormal behavior of the elderly, characterized in that the method includes: S1: Obtain the real-time video stream in the elderly care scenario, and decompose the real-time video stream into multiple frames of real-time images; S2: Perform feature extraction on each of the real-time images, and obtain the target image containing the elderly based on the extracted human body feature information; S3: Perform motion analysis on each of the target images, and output motion characteristic information of the preset parts of the elderly; S4: Based on the motion characteristic information, detect the preset abnormal behavior of the elderly and output the detection results; S5: Based on the detection results, when the abnormal behavior of the elderly is identified, a safety reminder is issued to the user; The S3 includes: S31: Use the target matching algorithm to track the old man in the target image; S32: Input the target image into the pre-trained target detection model, and output the preset part image of the elderly, where the preset part image includes hand and torso images; S33: Perform optical flow field calculation on the torso image and hand image, and output the average optical flow field of the torso image and the center optical flow field of the hand image as the motion feature information; The S4 includes: S41: Obtain preset abnormal behaviors of the elderly, wherein the abnormal behaviors of the elderly include at least one of the following behaviors: abnormal wandering of the elderly, inability to stand, and trembling hands of the elderly; S42: Obtain a preset time interval, and based on the variance of the average optical flow field within the time interval, when it is determined that there is repeated movement of the old man's torso, identify the old man's abnormal wandering behavior; S43: Identify the old man's inability to stand based on the first average value of the average optical flow field and the second average value of the central optical flow field within the time interval; S44: Based on the central optical flow field within the time interval, when it is determined that there is weak movement of the old man's hand, identify it as the trembling behavior of the old man's hand; The S44 includes: S441: Obtain the preset fourth optical flow field threshold; S442: Perform Fourier transform on the central optical flow field within the time interval, and output the frequency component corresponding to the central optical flow field; S443: Compare each of the frequency components. When the maximum value of the frequency component is greater than the fourth optical flow field threshold, identify the hand trembling behavior. 2. The real-time detection method of abnormal behavior of the elderly according to claim 1, characterized in that, the S2 includes: S21: Detecting each of the images according to a target detection algorithm, and outputting position information of the upper body of the human body; S22: Based on the upper body position information, use the target classification network to extract features of the upper body area of the human body, and output the upper body feature information of the human body; S23: Input the upper body feature information into the pre-trained classifier and output the classification result; S24: According to the classification result, when the number of image frames classified as elderly in each of the real-time images is greater than the preset number of image frames, obtain the target image. 3. The real-time detection method of abnormal behavior of the elderly according to claim 1, characterized in that, the S42 includes: S421: Obtain the time interval and the preset first optical flow field threshold; S422: Calculate the average optical flow field within the time interval, and output the variance of the average optical flow field; S423: When the variance is greater than the first optical flow field threshold, identify the old man's abnormal wandering behavior. 4. The real-time detection method of abnormal behavior of the elderly according to claim 1, characterized in that, the S43 includes: S431: Obtain the preset second optical flow field threshold and the third optical flow field threshold; S432: Calculate the average optical flow field and the central optical flow field within the time interval, and output the first average value and the second average value; S433: When the first average value is less than the second optical flow field threshold and the second average value is greater than the third optical flow field threshold, identify the old man's behavior as being unable to stand. 5. A real-time detection device for abnormal behavior of the elderly, characterized in that the device includes: A real-time image acquisition module, used to acquire a real-time video stream in an elderly care scenario, and decompose the real-time video stream into multiple frames of real-time images; The elderly identification module is used to extract features from each of the real-time images, and obtain target images containing the elderly based on the extracted human body feature information; A motion feature extraction module is used to perform motion analysis on each of the target images and output motion feature information of the preset parts of the elderly; The abnormal behavior detection module of the elderly is used to detect the preset abnormal behavior of the elderly based on the motion characteristic information and output the detection results; A safety reminder module is used to issue a safety reminder to the user when abnormal behavior of the elderly is identified based on the detection results; Wherein, the motion feature extraction module includes: An old man tracking unit is used to track the old man in the target image using a target matching algorithm; The preset part image acquisition unit for the elderly is used to input the target image into the pre-trained target detection model and output the preset part image for the elderly, where the preset part image includes hand and torso images; An optical flow field calculation unit, configured to perform optical flow field calculation on the torso image and hand image, and output the average optical flow field of the torso image and the center optical flow field of the hand image as the motion feature information; The abnormal behavior detection module for the elderly includes: The abnormal behavior acquisition unit of the elderly is used to obtain preset abnormal behaviors of the elderly, wherein the abnormal behaviors of the elderly include at least one of the following behaviors: abnormal wandering of the elderly, inability of the elderly to stand, and trembling of the hands of the elderly; An abnormal wandering identification unit for the elderly is used to obtain a preset time interval and, based on the variance of the average optical flow field within the time interval, when it is determined that there is repeated movement of the elderly man's torso, identify the elderly man's abnormal wandering behavior; An elderly person cannot stand identification unit, configured to identify the behavior of the elderly person being unable to stand based on the first average value of the average optical flow field and the second average value of the central optical flow field within the time interval; The old man's hand trembling identification unit is used to identify the old man's hand trembling behavior when it is determined that there is weak movement in the old man's hand based on the central optical flow field within the said time interval; The elderly hand tremor recognition unit includes: The fourth optical flow field threshold acquisition subunit is used to obtain the preset fourth optical flow field threshold; A frequency component calculation subunit, used to perform Fourier transform on the central optical flow field within the time interval and output the frequency component corresponding to the central optical flow field; A hand trembling behavior detection subunit is used to compare each of the frequency components, and when the maximum value of the frequency component is greater than the fourth optical flow field threshold, identify the hand trembling behavior. 6. An electronic device, characterized in that it includes: at least one processor, at least one memory and computer program instructions stored in the memory. When the computer program instructions are executed by the processor, the claims are implemented The method described in any one of 1-4. 7. A storage medium with computer program instructions stored thereon, characterized in that when the computer program instructions are executed by a processor, the method according to any one of claims 1-4 is implemented.