CN115953872B - Pedestrian crossing early warning method, device and storage medium based on video monitoring - Google Patents
Pedestrian crossing early warning method, device and storage medium based on video monitoring Download PDFInfo
- Publication number
- CN115953872B CN115953872B CN202211698370.8A CN202211698370A CN115953872B CN 115953872 B CN115953872 B CN 115953872B CN 202211698370 A CN202211698370 A CN 202211698370A CN 115953872 B CN115953872 B CN 115953872B
- Authority
- CN
- China
- Prior art keywords
- warning
- marker
- determining
- pedestrian
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Analysis (AREA)
Abstract
The application provides a pedestrian crossing early warning method, equipment and a storage medium based on video monitoring, which belong to the technical field of image processing, and the method comprises the following steps: acquiring a monitoring video acquired from a monitoring area; decoding from the monitoring video to obtain a single frame image, determining the position information and the warning color of each marker link based on the corresponding pixel point of each marker link in the single frame image, determining a datum line based on the position information of each marker link, and obtaining a warning line by extending the datum line based on the position information and the warning color of each marker link, wherein a region between the warning line and the datum line is used as a warning region; if the pedestrian is determined to enter the warning area, calculating based on the vertical distance and the included angle between the pedestrian and the datum line at a plurality of continuous moments to obtain a risk value of the pedestrian crossing the datum line; if the risk value is determined to be larger than a preset risk threshold value, triggering out-of-range alarm; the pedestrian crossing prediction method and the pedestrian crossing prediction device can accurately predict pedestrian crossing in advance aiming at various application scenes.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a pedestrian crossing early warning method, device and storage medium based on video monitoring.
Background
In the related art, when video images are adopted for cross-line monitoring, specific areas in the video images need to be scribed manually, and when pedestrians cross-line, an alarm is generated. In some application scenes where the monitoring area is changed frequently, for example, when safety production of outside operation is monitored, because the construction area is changed frequently, manual rescheduling is needed when the monitoring area is changed each time, the operation efficiency is low, the line crossing monitoring mode adopted for different application scenes is single, only the condition that pedestrians are crossed is alarmed, and accurate pre-judgment on pedestrian crossing in advance cannot be performed.
Therefore, it is necessary to provide a solution that can accurately pre-judge the pedestrian crossing in advance for various application scenarios.
Disclosure of Invention
The embodiment of the application mainly aims to provide a pedestrian crossing early warning method, equipment and a storage medium based on video monitoring, which can accurately pre-judge pedestrian crossing in advance aiming at various application scenes.
In order to achieve the above object, a first aspect of an embodiment of the present application provides a pedestrian crossing early warning method based on video monitoring, the method including:
acquiring a monitoring video acquired from a monitoring area; wherein, a plurality of marker posts which enclose the construction area are arranged in the monitoring area;
Decoding from the monitoring video to obtain a single-frame image, and carrying out image recognition on the single-frame image by adopting a marker link detection model to obtain corresponding pixel points of each marker link in the single-frame image; the marker post detection model is obtained by training in advance based on an example segmentation model;
Determining the position information and the warning color of each marker rod based on the corresponding pixel point of each marker rod in the single frame image, determining a datum line based on the position information of each marker rod, obtaining a warning line based on the position information and the warning color of each marker rod by extending the datum line, and taking the area between the warning line and the datum line as a warning area; the warning color is used for representing the warning level of the marker post;
Detecting pedestrians in a warning area in the monitoring video, if the pedestrians enter the warning area, determining the vertical distance and the included angle between the pedestrians and the datum line at a plurality of continuous moments, and calculating the risk value of the pedestrians crossing the datum line based on the vertical distance and the included angle between the pedestrians and the datum line at the plurality of continuous moments;
And comparing the risk value of the pedestrian crossing the datum line with a preset risk threshold, and triggering out-of-range alarm if the risk value is determined to be larger than the preset risk threshold.
In some embodiments, the determining the position information and the warning color of each marker post based on the corresponding pixel point of each marker post in the single frame image, determining the reference line based on the position information of each marker post, and obtaining the warning line based on the position information and the warning color of each marker post and extending the reference line includes:
determining bottom corner points of the marker bars based on corresponding pixel points of the marker bars in the single-frame image, and sequentially connecting the bottom corner points of the marker bars to form a datum line;
determining top corner points of the marker bars based on corresponding pixel points of the marker bars in the single-frame image, determining the heights of the marker bars based on the distances between bottom corner points and top corner points of the marker bars, determining the maximum observation distance of the marker bars based on the heights of the marker bars, determining effective acting lines of the marker bars by taking the bottom corner points as origin points and the maximum observation distance as radius, obtaining the effective acting lines of the marker bars, further determining circumscribed lines of adjacent effective acting lines, obtaining a plurality of circumscribed lines, and sequentially connecting the plurality of circumscribed lines to form a circumscribed line;
Determining the warning color of the marker bar based on the color information of the pixel point corresponding to the marker bar, determining a risk coefficient based on the warning color of the marker bar, further determining the vertical distance between the epitaxial line and the datum line, determining a warning distance based on the product of the risk coefficient and the vertical distance, and shrinking the epitaxial line to the distance between the epitaxial line and the datum line to obtain a warning line; the guard line is located between the reference line and the epitaxial line.
In some embodiments, the determining the warning color of the marker post based on the color information of the pixel point corresponding to the marker post, and determining the risk factor based on the warning color of the marker post includes:
Performing color clustering on the pixel points corresponding to each marker link by adopting DBScan algorithm to obtain a plurality of color clusters;
averaging the color values of the pixels in each color cluster to obtain the average color of each color cluster;
After removing the color clusters with the average colors of white and black, taking the average color corresponding to the color cluster with the largest pixel number in the rest color clusters as the main color of the marker link;
Taking the warning color with the highest similarity with the main color as the warning color of the marker post;
Determining a risk coefficient corresponding to the warning color of the marker post; the risk coefficient corresponding to the warning color increases with the warning level.
In some embodiments, the detecting the pedestrian in the alert area in the surveillance video, if it is determined that the pedestrian enters the alert area, determining the vertical distance and the included angle between the pedestrian and the reference line at a plurality of continuous moments, and calculating the risk value of the pedestrian crossing the reference line based on the vertical distance and the included angle between the pedestrian and the reference line at a plurality of continuous moments, including:
sampling the monitoring video to obtain a monitoring image, determining a warning area in the monitoring image, and detecting pedestrians in the warning area;
If the pedestrian is determined to enter the warning area, continuously sampling the monitoring video at set time intervals to obtain multi-frame sampling images, and determining the position of the pedestrian in the continuous multi-frame sampling images;
determining the minimum circumscribed rectangle of the pedestrian in the multi-frame sampling image, and taking the center of the minimum circumscribed rectangle as the mass center of the pedestrian to obtain the mass center of the pedestrian at a plurality of continuous moments;
Respectively determining the vertical distance between the mass center of the pedestrian at a plurality of continuous moments and the datum line, and determining the included angle between the pedestrian and the datum line at a plurality of continuous moments based on the mass center of the pedestrian at a plurality of continuous moments;
when the vertical distance is determined to be smaller than a distance threshold value, calculating to obtain a risk value of the pedestrian at the current moment through the following formula:
Wherein n is the total frame number of the sampling image, d (i) is the vertical distance between the centroid in the sampling image of the i-th frame and the datum line, d (i-1) is the vertical distance between the centroid in the sampling image of the i-th frame and the datum line, θ (i) is the included angle between the tangential direction of the centroid in the sampling image of the i-th frame and the datum line, the value range of θ (i) is [0, 90 ° ], and Risk (n) is the risk value of the pedestrian at the current moment.
In some embodiments, the determining the angle of the pedestrian from the reference line at successive times based on the centroid of the pedestrian at successive times comprises:
Determining the coordinates of the mass center in the ith frame of sampling image and the coordinates of the mass center in the ith-1 frame of sampling image, and calculating to obtain the direction angle of the mass center in the ith frame of sampling image based on the coordinates of the mass center in the ith frame of sampling image and the coordinates of the mass center in the ith-1 frame of sampling image;
arbitrarily selecting two pixel points on the reference line, determining coordinates of the two pixel points on the reference line, and determining a direction angle of the reference line based on the coordinates of the two pixel points;
Determining the included angle between the pedestrian and the datum line at the sampling moment of the ith frame of sampling image by the direction angle of the datum line and the direction angle of the centroid in the ith frame of sampling image;
And taking the included angle between the pedestrian and the datum line at the sampling time of the n frames of sampling images as the included angle between the pedestrian and the datum line at a plurality of continuous time points.
To achieve the above object, a second aspect of the embodiments of the present application proposes an electronic device comprising a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for enabling a connection communication between the processor and the memory, the program, when executed by the processor, implementing the method according to the first aspect.
To achieve the above object, a third aspect of the embodiments of the present application proposes a computer-readable storage medium for computer-readable storage, the storage medium storing one or more programs executable by one or more processors to implement the steps of the method described in the first aspect.
According to the pedestrian crossing early warning method, the pedestrian crossing early warning equipment and the storage medium based on the video monitoring, the datum line is determined based on the position information of each marker link by identifying the locating lever in the single-frame image, and the datum line can be adaptively adjusted according to the changed application scene; the warning line is obtained by extending the datum line based on the position information and the warning color of each marker rod, so that the datum line is reasonably extended according to the height of the warning level, different construction scenes are adapted, the risk identification accuracy is improved, and the pedestrian detection is carried out on the warning area in the monitoring video, so that the image noise interference is avoided, and the operation speed and the monitoring accuracy are improved; and obtaining the risk value of the pedestrian crossing the datum line more comprehensively and accurately based on the vertical distance and the included angle between the pedestrian and the datum line at a plurality of continuous moments. The pedestrian crossing prediction method and the pedestrian crossing prediction device can accurately predict pedestrian crossing in advance aiming at various application scenes.
Drawings
Fig. 1 is a flowchart of a pedestrian crossing early warning method based on video monitoring provided by an embodiment of the application;
fig. 2 is a schematic hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
The embodiment of the application provides a pedestrian crossing early warning method, equipment and a storage medium based on video monitoring, which are specifically described by the following embodiment.
Embodiments of the present application will be further described below with reference to the accompanying drawings.
Fig. 1 is an optional flowchart of a pedestrian crossing early warning method based on video monitoring according to an embodiment of the present application, where the method in fig. 1 includes the following steps:
Step S100, acquiring a monitoring video acquired from a monitoring area; wherein, a plurality of marker posts which enclose the construction area are arranged in the monitoring area;
It should be noted that the same marker post is adopted in the same monitoring area, in an embodiment, the monitoring area where the marker post needs to be set is subjected to video shooting by a camera, a monitoring video is acquired, the picture of the monitoring video includes the marker post, and the monitoring area further includes a peripheral area of the construction area, namely a warning area described below.
Step S200, decoding from the monitoring video to obtain a single-frame image, and carrying out image recognition on the single-frame image by adopting a marker link detection model to obtain corresponding pixel points of each marker link in the single-frame image; the marker post detection model is obtained by training in advance based on an example segmentation model;
In some embodiments, the example segmentation model may be a Mask R-CNN model, PANet model, HTC model, blend Mask model, CENTER MASK model, or the like, and the marker post detection model obtained by training in advance based on the example segmentation model may perform pixel-level image recognition on the marker posts in the single frame image, so as to identify pixel points corresponding to each marker post in the single frame image; for example, a marker post sample image currently required to be trained can be determined from a plurality of collected marker post sample images, a current marker post sample image is obtained, then the current marker post sample image is guided into a preset example segmentation model for training, and the shape characteristics of the marker posts in the marker post sample image are learned and identified, so that a marker post detection model is obtained.
Step S300, determining the position information and the warning color of each marker bar based on the corresponding pixel point of each marker bar in the single frame image, determining a datum line based on the position information of each marker bar, and obtaining a warning line on the datum line in an extending manner based on the position information and the warning color of each marker bar, wherein a region between the warning line and the datum line is used as a warning region; the warning color is used for representing the warning level of the marker post;
In some embodiments, the position information includes a position of each pixel point in the marker post in the single frame image; the datum line is a peripheral line segment obtained by connecting the positions of the marker bars; corresponding warning levels are determined based on warning colors of the marker bars, corresponding extension is carried out on the datum lines according to the warning levels, warning lines are obtained and can be adapted to risks represented by the warning levels, and risk identification accuracy is improved.
Step S400, detecting pedestrians in a warning area in the monitoring video, if the pedestrians enter the warning area, determining the vertical distance and the included angle between the pedestrians and the datum line at a plurality of continuous moments, and calculating the risk value of the pedestrians crossing the datum line based on the vertical distance and the included angle between the pedestrians and the datum line at the plurality of continuous moments;
In the step, after the warning area is determined, the monitoring video is continuously decoded, and as only the pedestrian detection is carried out on the warning area in the monitoring video, the image processing area can be reduced to the warning area, and the unnecessary image processing area is removed, so that the image noise interference is avoided, and the operation speed and the monitoring accuracy are improved. In some embodiments, the method for detecting the pedestrian in the warning area in the surveillance video is a machine learning-based method, for example, a HOG feature-based method, and the like, which can identify an external rectangular frame of the pedestrian and track the pedestrian, so that the position information of the pedestrian at a plurality of continuous moments is determined through pedestrian detection and tracking, and further the vertical distance and the included angle between the pedestrian and the reference line at a plurality of continuous moments are determined; the smaller the vertical distance between the pedestrian and the reference line, the closer the pedestrian is to the reference line, and the greater the risk of crossing the reference line; the greater the angle between the pedestrian and the reference line, the greater the likelihood of walking towards the reference line, and the greater the risk of crossing the reference line; the method combines two factors of the vertical distance and the included angle between the pedestrian and the datum line, so that the risk value of the pedestrian crossing the datum line is obtained more comprehensively and accurately.
Step S500, if the risk value of the pedestrian crossing the datum line is determined to be larger than a set risk threshold value, triggering out-of-range alarm;
In some embodiments, when the safety production monitoring of the out-field operation is performed, after a constructor arrives at a construction area site, the marker post is deployed around the construction area as an endpoint position for setting a datum line; erecting a monitoring camera to aim at a construction area, wherein an image of the monitoring area collected by the monitoring camera covers a marker post and a warning area; the monitoring camera is connected with electronic equipment and transmits a monitoring video acquired from a monitoring area to the electronic equipment; the electronic equipment decodes the monitoring video to obtain a single-frame image, and determines the position information and the warning color of the marker post in the single-frame image; and automatically generating a warning region according to the position information and the warning color of the marker post in the single-frame image. When a pedestrian enters a warning area, detecting and tracking the pedestrian in a monitoring picture, determining a risk value of the pedestrian crossing the datum line, and triggering out-of-range alarm if the risk value of the pedestrian crossing the datum line is determined to be larger than a set risk threshold value.
The application identifies the positioning rods in the single-frame image, determines the datum line based on the position information of each marking rod, improves the original manual marking mode into automatic marking, and extends the datum line to obtain the warning line based on the position information and the warning color of each marking rod, thereby reasonably expanding the datum line according to the height of the warning level, adapting to different construction scenes, improving the risk identification accuracy, avoiding the interference of image noise and improving the operation speed and the monitoring accuracy by detecting pedestrians in the warning area in the monitoring video; based on the vertical distance and the included angle between the pedestrian and the datum line at a plurality of continuous moments, the risk value of the pedestrian crossing the datum line is obtained more comprehensively and accurately; therefore, the pedestrian crossing prediction method and the pedestrian crossing prediction device can accurately predict pedestrian crossing in advance aiming at various application scenes.
In some embodiments, in step S300, the determining, based on the pixel points corresponding to the respective marker bars in the single frame image, the position information and the warning color of the respective marker bars, determining a reference line based on the position information of the respective marker bars, and obtaining a warning line for the reference line based on the position information and the warning color of the respective marker bars includes:
step S310, determining bottom corner points of the marker bars based on corresponding pixel points of the marker bars in the single-frame image, and sequentially connecting the bottom corner points of the marker bars to form a datum line;
In some embodiments, the bottom corner points of the adjacent marker bars in the single frame image are sequentially connected in a clockwise or anticlockwise direction to obtain a reference line in the single frame image, and the area within the reference line is a construction area surrounded by the marker bars in the single frame image.
Step S320, determining top corner points of the marker bars based on corresponding pixel points of the marker bars in the single-frame image, determining the heights of the marker bars based on the distances between bottom corner points and top corner points of the marker bars, determining the maximum observation distance of the marker bars based on the heights of the marker bars, determining effective acting lines of the marker bars by taking the bottom corner points as an origin and the maximum observation distance as a radius, obtaining the effective acting lines of the marker bars, further determining circumscribed lines of adjacent effective acting lines, obtaining a plurality of circumscribed lines, and sequentially connecting the plurality of circumscribed lines to form a circumscribed line;
It should be noted that, the maximum observation distance represents the critical position where the pedestrian should be able to accurately identify the symbol element in the marker post; the maximum observation distance is the product of the height of the marker post and an observation factor, and the observation factor is determined based on the vertical illuminance of the surface of the marker post, the expected visual acuity of pedestrians and the like; those skilled in the art may set in advance that in some embodiments, the observation factor has a value in the range of 2 to 50; the effective action line determined based on the maximum observation distance can reflect the action range of the marker rod on the pedestrian, and the maximum action range of all the marker rods on the pedestrian can be reflected by sequentially connecting a plurality of circumscribed lines to form an extension line, so that the image processing area is preliminarily determined. In some embodiments, the epitaxial line is contained within the monitored region.
Step S330, determining the warning color of the marker bar based on the color information of the pixel point corresponding to the marker bar, determining a risk coefficient based on the warning color of the marker bar, further determining the vertical distance between the epitaxial line and the datum line, determining a warning distance based on the product of the risk coefficient and the vertical distance, and shrinking the epitaxial line to the distance between the epitaxial line and the datum line as the warning distance to obtain a warning line; the guard line is located between the reference line and the epitaxial line.
In some embodiments, the construction area has different operation types, such as stacking materials, overhauling a road surface, working aloft, and the like, different operation types have different risks, and due to the requirement of risk prevention, pedestrians need not be too close to the construction area after seeing the marker bars, a certain warning distance needs to be set.
In some embodiments, in step S330, the determining the warning color of the marker post based on the color information of the pixel point corresponding to the marker post, and determining the risk factor based on the warning color of the marker post includes:
Step S331, performing color clustering on pixel points corresponding to each marker link by adopting DBScan algorithm to obtain a plurality of color clusters;
Step S332, averaging the color values of the pixels in each color cluster to obtain the average color of each color cluster;
step S333, after eliminating the color clusters with the average colors of white and black, taking the average color corresponding to the color cluster with the largest number of pixels in the rest color clusters as the main color of the marker rod;
step S334, taking the warning color with the highest similarity with the main color as the warning color of the marker post;
Step S335, determining a risk coefficient corresponding to the warning color of the marker post; the risk coefficient corresponding to the warning color increases with the warning level.
It should be noted that, in some embodiments, the alert color includes red, yellow and blue, each alert color corresponds to a risk coefficient, the risk coefficients corresponding to the red, yellow and blue of the alert color are sequentially reduced, the value ranges are all between 0 and 1, and different alert colors have different alert levels and respectively play roles of prohibiting, warning, instructing, prompting, and the like; for example, red indicates inaccuracy or refrain from certain actions, yellow indicates that a person is paying attention to a possible danger, and blue indicates that compliance is necessary to force or limit the person's actions. In this embodiment, corresponding risk coefficients are determined based on alarm levels with different colors, and the larger the risk coefficient is, the higher the protection level is.
In some embodiments, in step S400, the detecting the pedestrian in the alert area in the surveillance video, if it is determined that the pedestrian enters the alert area, determining the vertical distance and the included angle between the pedestrian and the reference line at a plurality of continuous moments, and calculating the risk value of the pedestrian crossing the reference line based on the vertical distance and the included angle between the pedestrian and the reference line at a plurality of continuous moments, includes:
Step S410, sampling the monitoring video to obtain a monitoring image, determining a warning area in the monitoring image, and detecting pedestrians in the warning area;
after the warning area is determined, the monitoring video is continuously sampled, and as the pedestrian detection is only carried out on the warning area in the video stream, the object of image processing can be reduced to the warning area, and the operation speed is improved and the monitoring efficiency is improved by reducing the processing range.
Step S420, if it is determined that a pedestrian enters the warning area, continuously sampling the monitoring video at set time intervals to obtain multi-frame sampling images, and determining the position of the pedestrian in the continuous multi-frame sampling images;
Step S430, determining the minimum circumscribed rectangle of the pedestrian in the multi-frame sampling image, and taking the center of the minimum circumscribed rectangle as the mass center of the pedestrian to obtain the mass center of the pedestrian at a plurality of continuous moments;
That is, if it is determined that a pedestrian enters the warning region, the pedestrian is tracked based on the multi-frame sampling images obtained by continuous sampling, the position of the pedestrian is simplified, and the centroid of the pedestrian is used as a processing object, so that the calculation efficiency is improved.
Step S440, respectively determining the vertical distance between the mass center of the pedestrian at a plurality of continuous moments and the datum line, and determining the included angle between the pedestrian and the datum line at a plurality of continuous moments based on the mass center of the pedestrian at a plurality of continuous moments;
step S450, when it is determined that the vertical distance is smaller than the distance threshold, calculating the risk value of the pedestrian at the current moment according to the following formula:
Wherein n is the total frame number of the sampling image, d (i) is the vertical distance between the centroid in the sampling image of the i-th frame and the datum line, d (i-1) is the vertical distance between the centroid in the sampling image of the i-th frame and the datum line, θ (i) is the included angle between the tangential direction of the centroid in the sampling image of the i-th frame and the datum line, the value range of θ (i) is [0, 90 ° ], and Risk (n) is the risk value of the pedestrian at the current moment.
In some embodiments, d (i) has a value in meters ranging from [0, d ]; d is a distance threshold, d is greater than or equal to 30 meters, in one embodiment, d=30 meters, and distances other than 30 meters are considered to be risk-free of crossing the warning line, and risk value calculation is not performed;
It should be noted that, the closer to the guard line and/or the larger the included angle with the guard line, the more obvious the trend of crossing the guard line, the more likely the guard line is crossed, when the distance is close to 0, the edge of the guard line is at the edge of the guard line, the motion trend thereof is very likely to cross the guard line, and the accumulated risk value is increased; the larger the included angle is, the faster the risk value is increased, and when the risk value reaches the risk threshold value, the line-crossing alarm is triggered, so that pedestrians are warned to be far away as soon as possible; the farther the distance from the warning line is, the more obvious the trend of the distance from the warning line is, the more likely the trend of the distance from the warning line is, the more the accumulated risk value is reduced, the more the included angle is, the faster the risk value is reduced, and when the risk value is lower than the risk threshold value, the line crossing alarm is released; when the vertical distance is larger than the distance threshold, the vertical distance is far away from the warning line, and no matter how large the included angle is, the movement trend of the vertical distance cannot cross the warning line and cannot reach the risk threshold; the embodiment comprehensively considers the distance and the included angle of a plurality of past moments to form a behavior trend, accurately predicts the motion trend of pedestrians,
In some embodiments, in step S440, the determining, based on the centroids of the pedestrians at the successive moments, the angles between the pedestrians and the reference line at the successive moments includes:
Step S441, determining the coordinates of the mass center in the ith frame of sampling image and the coordinates of the mass center in the ith-1 frame of sampling image, and calculating the direction angle of the mass center in the ith frame of sampling image based on the coordinates of the mass center in the ith frame of sampling image and the coordinates of the mass center in the ith-1 frame of sampling image;
step S442, arbitrarily selecting two pixel points on the reference line, determining coordinates of the two pixel points on the reference line, and determining a direction angle of the reference line based on the coordinates of the two pixel points;
Step S443, determining the included angle between the pedestrian and the datum line at the sampling moment of the ith frame of sampling image by the direction angle of the datum line and the direction angle of the centroid in the ith frame of sampling image;
Step S444 uses the included angle between the pedestrian and the reference line at the sampling time of the n frames of sampling images as the included angle between the pedestrian and the reference line at a plurality of continuous times.
In some embodiments, the direction angle of the centroid within the i-th frame sample image is calculated as:
Wherein (x i,yi) represents the coordinates of the centroid in the i-th frame sample image, (x i-1,yi-1) represents the coordinates of the centroid in the i-1-th frame sample image, and phi (i) represents the direction angle of the centroid in the i-th frame sample image;
the calculation formula of the direction angle of the datum line is as follows:
wherein (x 1, y 1), (x 2, y 2) represent coordinates of two pixel points on the reference line, and θ (0) represents a direction angle of the reference line.
In some embodiments, a calculation formula of the angle between the pedestrian and the reference line at the sampling time of the i-th frame of the sampled image is:
Wherein θ (i) represents an angle between the pedestrian and the reference line at a sampling timing of the i-th frame sampling image.
The values of phi (i) and theta (0) are all [ -90 degrees, 90 degrees ], and the calculated value range of theta (i) is [0,90 degrees ].
In addition, referring to fig. 2, an embodiment of the present invention also provides an electronic device 10, the electronic device 10 comprising: memory 11, processor 12, and a computer program stored on memory 11 and executable on processor 12.
The processor 12 and the memory 11 may be connected by a bus or other means.
The non-transitory software program and instructions required to implement the pedestrian crossing early warning method based on video monitoring of the above-described embodiment are stored in the memory 11, and when executed by the processor 12, the pedestrian crossing early warning method based on video monitoring of the above-described embodiment is executed.
In addition, an embodiment of the present invention further provides a computer readable storage medium, where computer executable instructions are stored, where the computer executable instructions are executed by a processor or a controller, for example, by one of the processors in the above-mentioned electronic device embodiment, and cause the processor to execute the pedestrian crossing early warning method based on video monitoring in the above-mentioned embodiment.
The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
In some embodiments, the computer readable storage medium may be FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; but may be a variety of devices including one or any combination of the above memories. The computer may be a variety of computing devices including smart terminals and servers.
In some embodiments, the executable instructions may be in the form of programs, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and they may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.
As an example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices located at one site or distributed across multiple sites and interconnected by a communication network.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read-only memory/random-access memory, magnetic disk, optical disk), comprising instructions for causing a multimedia terminal device (which may be a mobile phone, a computer, a television receiver, or a network device, etc.) to perform the method according to the embodiments of the present application.
The above-mentioned embodiments are only preferred embodiments of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can make equivalent substitutions or modifications according to the technical solution and the inventive concept of the present invention within the scope of the present invention disclosed in the present invention patent, and all those skilled in the art belong to the protection scope of the present invention.
Claims (7)
1. The pedestrian crossing early warning method based on video monitoring is characterized by comprising the following steps:
acquiring a monitoring video acquired from a monitoring area; wherein, a plurality of marker posts which enclose the construction area are arranged in the monitoring area;
Decoding from the monitoring video to obtain a single-frame image, and carrying out image recognition on the single-frame image by adopting a marker link detection model to obtain corresponding pixel points of each marker link in the single-frame image; the marker post detection model is obtained by training in advance based on an example segmentation model;
Determining the position information and the warning color of each marker rod based on the corresponding pixel point of each marker rod in the single frame image, determining a datum line based on the position information of each marker rod, obtaining a warning line based on the position information and the warning color of each marker rod by extending the datum line, and taking the area between the warning line and the datum line as a warning area; the warning color is used for representing the warning level of the marker post; the datum line is a peripheral line segment obtained by connecting the positions of the marker posts; the warning line determines corresponding warning levels based on the warning colors of the marker posts, and further carries out corresponding extension on the datum line according to the warning levels to obtain the warning level;
Detecting pedestrians in a warning area in the monitoring video, if the pedestrians enter the warning area, determining the vertical distance and the included angle between the pedestrians and the datum line at a plurality of continuous moments, and calculating the risk value of the pedestrians crossing the datum line based on the vertical distance and the included angle between the pedestrians and the datum line at the plurality of continuous moments;
And comparing the risk value of the pedestrian crossing the datum line with a preset risk threshold, and triggering out-of-range alarm if the risk value is determined to be larger than the preset risk threshold.
2. The method according to claim 1, wherein determining the position information and the warning color of each marker post based on the corresponding pixel point of each marker post in the single frame image, determining the reference line based on the position information of each marker post, and obtaining the warning line based on the position information and the warning color of each marker post and extending the reference line, comprises:
determining bottom corner points of the marker bars based on corresponding pixel points of the marker bars in the single-frame image, and sequentially connecting the bottom corner points of the marker bars to form a datum line;
Determining top corner points of the marker bars based on corresponding pixel points of the marker bars in the single-frame image, determining the heights of the marker bars based on the distances between bottom corner points and top corner points of the marker bars, determining the maximum observation distance of the marker bars based on the heights of the marker bars, determining effective acting lines of the marker bars by taking the bottom corner points as origin points and the maximum observation distance as radius, obtaining the effective acting lines of the marker bars, further determining circumscribed lines of adjacent effective acting lines, obtaining a plurality of circumscribed lines, and sequentially connecting the plurality of circumscribed lines to form a circumscribed line; the extension line can reflect the maximum action range of all the marker bars on pedestrians;
Determining the warning color of the marker bar based on the color information of the pixel point corresponding to the marker bar, determining a risk coefficient based on the warning color of the marker bar, further determining the vertical distance between the epitaxial line and the datum line, determining a warning distance based on the product of the risk coefficient and the vertical distance, and shrinking the epitaxial line to the distance between the epitaxial line and the datum line to obtain a warning line; the guard line is located between the reference line and the epitaxial line.
3. The method of claim 2, wherein the determining the warning color of the marker post based on the color information of the pixel point corresponding to the marker post, and the determining the risk factor based on the warning color of the marker post, comprises:
Performing color clustering on the pixel points corresponding to each marker link by adopting DBScan algorithm to obtain a plurality of color clusters;
averaging the color values of the pixels in each color cluster to obtain the average color of each color cluster;
After removing the color clusters with the average colors of white and black, taking the average color corresponding to the color cluster with the largest pixel number in the rest color clusters as the main color of the marker link;
Taking the warning color with the highest similarity with the main color as the warning color of the marker post;
Determining a risk coefficient corresponding to the warning color of the marker post; the risk coefficient corresponding to the warning color increases with the warning level.
4. A method according to claim 3, wherein the detecting the pedestrian in the warning area in the monitoring video, if it is determined that the pedestrian enters the warning area, determining the vertical distance and the included angle between the pedestrian and the reference line at a plurality of consecutive moments, and calculating the risk value of the pedestrian crossing the reference line based on the vertical distance and the included angle between the pedestrian and the reference line at the plurality of consecutive moments, includes:
sampling the monitoring video to obtain a monitoring image, determining a warning area in the monitoring image, and detecting pedestrians in the warning area;
If the pedestrian is determined to enter the warning area, continuously sampling the monitoring video at set time intervals to obtain multi-frame sampling images, and determining the position of the pedestrian in the continuous multi-frame sampling images;
determining the minimum circumscribed rectangle of the pedestrian in the multi-frame sampling image, and taking the center of the minimum circumscribed rectangle as the mass center of the pedestrian to obtain the mass center of the pedestrian at a plurality of continuous moments;
Respectively determining the vertical distance between the mass center of the pedestrian at a plurality of continuous moments and the datum line, and determining the included angle between the pedestrian and the datum line at a plurality of continuous moments based on the mass center of the pedestrian at a plurality of continuous moments;
when the vertical distance is determined to be smaller than a distance threshold value, calculating to obtain a risk value of the pedestrian at the current moment through the following formula:
；
Wherein n is the total frame number of the sampling image, d (i) is the vertical distance between the centroid in the sampling image of the i-th frame and the datum line, d (i-1) is the vertical distance between the centroid in the sampling image of the i-th frame and the datum line, theta (i) is the included angle between the tangential direction of the centroid in the sampling image of the i-th frame and the datum line, the value range of theta (i) is [0, 90 DEG ], and Risk (n) is the Risk value of the pedestrian at the current moment.
5. The method of claim 4, wherein the determining the angle of the pedestrian from the reference line at successive times based on the centroid of the pedestrian at successive times comprises:
Determining the coordinates of the mass center in the ith frame of sampling image and the coordinates of the mass center in the ith-1 frame of sampling image, and calculating to obtain the direction angle of the mass center in the ith frame of sampling image based on the coordinates of the mass center in the ith frame of sampling image and the coordinates of the mass center in the ith-1 frame of sampling image;
arbitrarily selecting two pixel points on the reference line, determining coordinates of the two pixel points on the reference line, and determining a direction angle of the reference line based on the coordinates of the two pixel points;
Determining the included angle between the pedestrian and the datum line at the sampling moment of the ith frame of sampling image by the direction angle of the datum line and the direction angle of the centroid in the ith frame of sampling image;
And taking the included angle between the pedestrian and the datum line at the sampling time of the n frames of sampling images as the included angle between the pedestrian and the datum line at a plurality of continuous time points.
6. An electronic device comprising a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for enabling a connected communication between the processor and the memory, the program when executed by the processor implementing the steps of the method according to any of claims 1 to 5.
7. A computer readable storage medium for computer readable storage, wherein the storage medium stores one or more programs executable by one or more processors to implement the steps of the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211698370.8A CN115953872B (en) | 2022-12-28 | 2022-12-28 | Pedestrian crossing early warning method, device and storage medium based on video monitoring |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211698370.8A CN115953872B (en) | 2022-12-28 | 2022-12-28 | Pedestrian crossing early warning method, device and storage medium based on video monitoring |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115953872A CN115953872A (en) | 2023-04-11 |
| CN115953872B true CN115953872B (en) | 2024-10-18 |
Family
ID=87296507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211698370.8A Active CN115953872B (en) | 2022-12-28 | 2022-12-28 | Pedestrian crossing early warning method, device and storage medium based on video monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115953872B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117854228A (en) * | 2024-03-07 | 2024-04-09 | 山东大通前沿电子科技股份有限公司 | Security protection equipment job site safety precaution system based on artificial intelligence |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618688A (en) * | 2015-01-19 | 2015-05-13 | 荣科科技股份有限公司 | Visual monitor protection method |
| CN104978751A (en) * | 2015-06-16 | 2015-10-14 | 电子科技大学 | Border-crossing detection method based camera angle |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4990552B2 (en) * | 2006-04-28 | 2012-08-01 | 綜合警備保障株式会社 | Attention position identification system, attention position identification method, and attention position identification program |
| JP5621171B2 (en) * | 2010-09-07 | 2014-11-05 | オプテックス株式会社 | Monitoring device with position detector |
| CN205051806U (en) * | 2015-09-23 | 2016-02-24 | 国网河南方城县供电公司 | Monitoring device that crosses border in electrified region of transformer substation based on IVS |
| CN112929611A (en) * | 2021-02-07 | 2021-06-08 | 重庆惠统智慧科技有限公司 | Campus intelligent security prevention and control system |
| CN113139521B (en) * | 2021-05-17 | 2022-10-11 | 中国大唐集团科学技术研究院有限公司中南电力试验研究院 | Pedestrian boundary crossing monitoring method for electric power monitoring |
| CN113344874B (en) * | 2021-06-04 | 2024-02-09 | 温州大学 | Pedestrian boundary crossing detection method based on Gaussian mixture modeling |
| CN113516818B (en) * | 2021-07-13 | 2023-01-03 | 国网江苏省电力有限公司常州供电分公司 | Transformer substation electronic security fence based on live-action three-dimensional model and monitoring method |
| CN114267011A (en) * | 2021-12-08 | 2022-04-01 | 北京京能电力股份有限公司 | Distribution room live equipment misoperation detection system |
| CN114140832A (en) * | 2022-01-30 | 2022-03-04 | 西安华创马科智能控制系统有限公司 | Method and device for detecting pedestrian boundary crossing risk in well, electronic equipment and storage medium |
| CN114639214B (en) * | 2022-05-23 | 2022-08-12 | 安徽送变电工程有限公司 | Intelligent safety distance early warning system and method for electric power hoisting operation |
-
2022
- 2022-12-28 CN CN202211698370.8A patent/CN115953872B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618688A (en) * | 2015-01-19 | 2015-05-13 | 荣科科技股份有限公司 | Visual monitor protection method |
| CN104978751A (en) * | 2015-06-16 | 2015-10-14 | 电子科技大学 | Border-crossing detection method based camera angle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115953872A (en) | 2023-04-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9652863B2 (en) | Multi-mode video event indexing | |
| CN103366374B (en) | Based on the passageway for fire apparatus obstacle detection method of images match | |
| US8977060B2 (en) | Motion stabilization and detection of articulated objects | |
| CN113284168A (en) | Target tracking method and device, electronic equipment and storage medium | |
| US20120274776A1 (en) | Fault tolerant background modelling | |
| CN112733690B (en) | High-altitude parabolic detection method and device and electronic equipment | |
| CN107437318B (en) | Visible light intelligent recognition algorithm | |
| US20230005176A1 (en) | Throwing position acquisition method and apparatus, computer device and storage medium | |
| WO2012039496A1 (en) | Track estimation device and program | |
| CN110781844B (en) | Security patrol monitoring method and device | |
| CN115953872B (en) | Pedestrian crossing early warning method, device and storage medium based on video monitoring | |
| CN113012383B (en) | Fire detection alarm method, related system, related equipment and storage medium | |
| KR20140095333A (en) | Method and apparratus of tracing object on image | |
| CN115797832A (en) | Abnormal behavior identification method and system in swimming scene and storage medium | |
| KR102155724B1 (en) | Method and system for risk detection of objects in ships using deep neural networks | |
| CN116311084B (en) | Crowd gathering detection method and video monitoring equipment | |
| CN114445769A (en) | Fishing behavior detection method, device and system | |
| CN116778673A (en) | Water area safety monitoring method, system, terminal and storage medium | |
| CN113657286B (en) | Unmanned aerial vehicle-based power transmission line monitoring method and device | |
| CN110718041A (en) | Method, device and system for preventing children from getting lost and storage medium | |
| CN115938069A (en) | Alarm method for dangerous behavior and alarm system for dangerous behavior | |
| CN115731477A (en) | Image recognition method, illicit detection method, terminal device, and storage medium | |
| CN109492548B (en) | Method for obtaining region mask picture based on video analysis | |
| CN113507577A (en) | Target object detection method, device, equipment and storage medium | |
| CN110602446A (en) | Garbage recovery reminding method and system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |