[go: up one dir, main page]

CN114429616B - Box abnormal state identification method based on computer vision - Google Patents

Box abnormal state identification method based on computer vision Download PDF

Info

Publication number
CN114429616B
CN114429616B CN202210094293.9A CN202210094293A CN114429616B CN 114429616 B CN114429616 B CN 114429616B CN 202210094293 A CN202210094293 A CN 202210094293A CN 114429616 B CN114429616 B CN 114429616B
Authority
CN
China
Prior art keywords
video data
pictures
neural network
network model
door
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210094293.9A
Other languages
Chinese (zh)
Other versions
CN114429616A (en
Inventor
戴书球
孙维
孙中光
张建鑫
谭一川
梁帅
韩麟之
文学峰
冉昆鹏
于乐泉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Smart City Science And Technology Research Institute Co ltd
CCTEG Chongqing Research Institute Co Ltd
Original Assignee
Chongqing Smart City Science And Technology Research Institute Co ltd
CCTEG Chongqing Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Smart City Science And Technology Research Institute Co ltd, CCTEG Chongqing Research Institute Co Ltd filed Critical Chongqing Smart City Science And Technology Research Institute Co ltd
Priority to CN202210094293.9A priority Critical patent/CN114429616B/en
Publication of CN114429616A publication Critical patent/CN114429616A/en
Application granted granted Critical
Publication of CN114429616B publication Critical patent/CN114429616B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of video monitoring, and particularly discloses a method for identifying abnormal states of a box body based on computer vision, which comprises the following steps: s1, acquiring video data in a preset format and decoding the video data; s2, framing the decoded video data to generate a plurality of pictures, S3, preprocessing the pictures, classifying and marking the pictures, and constructing a training picture set; the marks comprise door opening and door non-opening; s4, inputting the training picture set into a neural network model for training; s5, judging by using the trained neural network model, and outputting a judging result; and S6, when the judging result is that the door is opened, judging whether the door is opened abnormally or not based on the operation log, and if so, generating alarm information. By adopting the technical scheme of the invention, the abnormal door opening can be automatically identified.

Description

Box abnormal state identification method based on computer vision
Technical Field
The invention relates to the technical field of video monitoring, in particular to a method for identifying abnormal states of a box body based on computer vision.
Background
In the construction of smart cities, part of dangerous areas and important places are not allowed to be accessed by common people, such as box-type transformer stations with high voltage power, microcomputer rooms for deploying important equipment and the like, and in order to avoid personnel safety accidents caused by abnormal door opening of the box or loss of public equipment, the abnormal door opening state of the box needs to be monitored and alarmed in real time.
The common scheme for monitoring abnormal opening of the box body at home and abroad is to use a door magnetic sensor, and the monitoring scheme can only monitor the opening and closing state of the box body door, cannot realize real-time analysis of the opening and closing scene and has great limitation. For example, the box may not have door magnet installation conditions, or door magnet data may not be uploaded to a remote control center; the intelligent identification of the scene can not be carried out, and the normal opening and the abnormal opening can not be identified; the door magnetic element is easy to damage, so that the door of the box body cannot be normally monitored.
In other schemes, video equipment linkage can be carried out, video monitoring equipment is installed simultaneously to continuously monitor the box body for 24 hours, when the door magnetic sensor monitors that the box body is abnormally opened, alarm information is pushed, and then a manager opens the corresponding video monitoring equipment to browse or replay historical video data in real time, but the monitoring method needs manual processing, and is low in working efficiency.
Therefore, there is a need for a method for recognizing abnormal states of a case based on computer vision that can perform automatic recognition.
Disclosure of Invention
The invention provides a method for identifying abnormal states of a box body based on computer vision, which can automatically identify abnormal door opening.
In order to solve the technical problems, the application provides the following technical scheme:
The method for identifying the abnormal state of the box body based on computer vision comprises the following steps:
s1, acquiring video data in a preset format and decoding the video data;
S2, framing the decoded video data to generate a plurality of pictures,
S3, preprocessing the pictures, classifying and marking the pictures, and constructing a training picture set; the marks comprise door opening and door non-opening;
s4, inputting the training picture set into a neural network model for training;
S5, judging by using the trained neural network model, and outputting a judging result;
And S6, when the judging result is that the door is opened, judging whether the door is opened abnormally or not based on the operation log, and if so, generating alarm information.
The basic scheme principle and the beneficial effects are as follows:
Because the video data is subjected to coding compression in order to reduce the volume of the video, each frame in the video data can be restored by decoding the video data, and subsequent framing is facilitated. By framing, a large number of pictures can be obtained, and the sample richness of the training picture set is ensured. The aim of optimizing the sample is achieved by preprocessing the picture, and the training effect of neural network model identification can be improved. After the neural network model is successfully trained, whether the box body is opened or not can be judged in real time. When the door is opened, the operation log is combined, so that whether the door is opened abnormally or not can be accurately judged.
In summary, the scheme can realize automatic identification of abnormal states of the box body.
Further, in the step S2, framing the decoded video data according to a first preset rule; the first preset rule includes an interval of processing frames, a size of a picture, and a storage path of the picture.
By setting the interval of the processing frames, the number of generated pictures can be adjusted.
Further, in the step S3, during preprocessing, the image is converted into a gray level image, and eight-bit plane decomposition is performed on the gray level image, so as to extract one of the bit plane image classification marks.
By converting the information into gray images and performing eight-bit plane decomposition, part of the information can be filtered, and the interference of excessive information on training is avoided.
Further, in the step S5, video data in a preset format is obtained in real time and decoded; and framing the decoded video data according to a second preset rule to generate a plurality of pictures, preprocessing the pictures, and inputting the preprocessed pictures into the trained neural network model.
As compared with training in actual operation, the number of generated pictures can be adjusted according to actual conditions by setting the second preset rule without excessive pictures.
Further, in S6, the time of opening the door is obtained, and it is determined whether the operation log has a door opening plan corresponding to the time recorded, and if not, it is determined that the door is opened abnormally.
The door opening plan is recorded in an operation log during normal inspection or maintenance of the staff. When the operation log has no door opening plan corresponding to the time, normal operation of non-staff is indicated, so that abnormal door opening can be judged.
In step S3, after performing eight-bit plane decomposition on the gray level image, discarding a plurality of bit plane images according to a preset standard, and randomly extracting one bit plane image from the rest bit plane images to perform classification marking.
And discarding a plurality of bit plane images according to a preset standard, so that the bit plane images which do not contain effective information can be removed, and invalid training is avoided.
Further, in S1, the preset format is an h.264 format.
The H.264 format is adopted, so that the compression rate is high, and the volume of video data can be reduced.
Further, in S1, decoding is performed through FFMpeg.
FFmpeg is a set of open source computer programs that can be used to convert digital video and convert it into streams, which contain a rich library of video decoders.
Further, in S4, the neural network model is a convolutional neural network model.
In image recognition, convolutional neural network models have higher accuracy than other neural networks.
Drawings
Fig. 1 is a flowchart of a method for identifying abnormal states of a case based on computer vision according to an embodiment.
Detailed Description
The following is a further detailed description of the embodiments:
Example 1
As shown in fig. 1, the method for identifying abnormal state of a box based on computer vision in this embodiment includes the following steps:
S1, acquiring video data in a preset format and decoding the video data. In this embodiment, the preset format is an h.264 format, and decoding is performed through FFMpeg. In order to reduce the volume of video and reduce the bandwidth occupation during transmission, the video is encoded and compressed to reduce the volume. In the h.264 format, the coding algorithm is intra-frame compression and inter-frame compression, the intra-frame compression is an algorithm for generating an I-frame, the inter-frame compression is an algorithm for generating a B-frame and a P-frame, wherein the I-frame is a key frame, the information recorded by the P-frame is the difference between the frame and a previous key frame (or P-frame), the B-frame is a bidirectional difference frame, and the recorded information is the difference between the frame and a previous frame and a subsequent frame. So that decoding is required to restore each frame in the video data.
S2, framing the decoded video data according to a first preset rule, and generating a plurality of pictures. In this embodiment, the framing is performed by using a segmentation component in OpenCV. In this embodiment, the first preset rule includes an interval of processing frames, a size of a picture, and a storage path of the picture. The interval of frames, i.e. how many frames take a picture, is processed. For example, at a frame rate of 25fps, the process frames are spaced 25 apart, i.e., 1 second to produce 1 picture.
S3, preprocessing the pictures, classifying and marking the pictures, and constructing a training picture set; the indicia include open door and unopened door. And converting the image into a gray image during preprocessing, performing eight-bit plane decomposition on the gray image, and extracting one bit plane image classification mark. A gray scale image, i.e. an 8 bit 256 color image. By extracting each bit of the image separately, one image can be decomposed to form 8 images.
In this embodiment, during preprocessing, after performing eight-bit plane decomposition on the gray level image, a plurality of bit plane images are discarded according to a preset standard, and one of the bit plane images is randomly extracted from the remaining bit plane images to perform classification marking. The predetermined criterion is to discard the bit-plane image that does not contain valid information. For example, the 8 bit-planes formed by decomposition are respectively designated as bit-plane 1, bit-plane 2, bit-plane 3, bit-plane 4 … … and bit-plane 8, and according to experience, bit-plane 1 and bit-plane 2 usually contain effective information enough for training, so that the predetermined criterion is to discard bit-plane 3 to bit-plane 8, and randomly extract bit-plane image 1 or bit-plane image 2 for classification marking.
S4, inputting the training picture set into a neural network model for training; the neural network model is a convolutional neural network model. In this embodiment, the convolutional neural network model is a convolutional neural network model component in OpenCV.
And S5, judging by using the trained neural network model, and outputting a judging result. Specifically, video data in a preset format is obtained in real time and decoded; and framing the decoded video data according to a second preset rule to generate a plurality of pictures, preprocessing the pictures, and inputting the preprocessed pictures into the trained neural network model. In this embodiment, the second preset rule is different from the first preset rule in that the interval of processing frames, that is, the recognition and training, does not need a large number of pictures any more, and by increasing the interval of processing frames, the amount of generated pictures can be reduced.
And S6, when the judging result is that the door is opened, judging whether the door is opened abnormally or not based on the operation log, and if so, generating alarm information. Specifically, the door opening time is acquired, whether a door opening plan corresponding to the time is recorded in the operation log is judged, and if not, abnormal door opening is judged. In this embodiment, before performing operations such as inspection and maintenance, the staff may have a corresponding inspection schedule or maintenance schedule, where the content related to the entering box may be extracted and added to the operation log as a door opening schedule, as a basis for judgment.
Example two
The difference between the present embodiment and the first embodiment is that in S5 of the present embodiment, the face recognition module further determines whether there is a person in the monitoring area; and when a person exists in the monitoring area, carrying out face recognition, judging whether the person records, if not, marking the corresponding video data as high-priority video data, and if so, marking the corresponding video data as low-priority video data. And during framing, the high-priority video data corresponding to the cameras in the preset area are subjected to framing preferentially.
When no personnel exist in the monitoring area, the possibility of abnormal opening of the box door is small, pictures are extracted every preset time, and are input into the neural network model for judgment, instead of judging the pictures in real time, so that the pressure of the neural network model processing can be reduced. The face data of the staff is collected in advance, and the staff can be accurately identified through face recognition. When staff appears in the monitoring area, namely, in the vicinity of the box body, the possibility of maintenance or inspection is high, corresponding video data are marked as low-priority video data, and when the framing module processes busy, the processing is not needed immediately.
Example III
The difference between the present embodiment and the first embodiment is that in S5 in the present embodiment, when no person is in the monitored area, the pictures are extracted every preset time, and the pictures are input into the neural network model after training to perform judgment, and in the present embodiment, 1 picture is extracted.
When personnel exist in the monitoring area, the preset picture number of the input neural network model is determined according to the high-priority video data and the low-priority video data, and the pictures corresponding to the high-priority video data are input into the neural network model to be judged preferentially. The number of the preset pictures corresponding to the video data with high priority is larger than that of the preset pictures corresponding to the video data with low priority. In this embodiment, when the analysis module inputs the pictures into the neural network model, one picture is extracted from the neural network model at a preset number of intervals until the number of the inputted pictures is equal to the preset number of pictures or the judgment result of the neural network model is that the door is opened.
For example, the number of preset pictures corresponding to the high-priority video data is 20, and 5 pictures are extracted from each interval and input into the neural network model.
It is possible to ensure that a sufficient identification judgment is made when a non-worker is present near the box.
The foregoing is merely an embodiment of the present application, the present application is not limited to the field of this embodiment, and the specific structures and features well known in the schemes are not described in any way herein, so that those skilled in the art will know all the prior art in the field before the application date or priority date of the present application, and will have the capability of applying the conventional experimental means before the date, and those skilled in the art may, in light of the present application, complete and implement the present scheme in combination with their own capabilities, and some typical known structures or known methods should not be an obstacle for those skilled in the art to practice the present application. It should be noted that modifications and improvements can be made by those skilled in the art without departing from the structure of the present application, and these should also be considered as the scope of the present application, which does not affect the effect of the implementation of the present application and the utility of the patent. The protection scope of the present application is subject to the content of the claims, and the description of the specific embodiments and the like in the specification can be used for explaining the content of the claims.

Claims (9)

1. The method for identifying the abnormal state of the box body based on computer vision is characterized by comprising the following steps of:
s1, acquiring video data in a preset format and decoding the video data;
S2, framing the decoded video data to generate a plurality of pictures,
S3, preprocessing the pictures, classifying and marking the pictures, and constructing a training picture set; the marks comprise door opening and door non-opening;
s4, inputting the training picture set into a neural network model for training;
S5, judging by using the trained neural network model, and outputting a judging result;
S6, when the judging result is that the door is opened, judging whether the door is opened abnormally or not based on the operation log, and if so, generating alarm information;
S5, judging whether personnel exist in the monitoring area through the face recognition module, and if the personnel do not exist in the monitoring area, extracting pictures at preset intervals and inputting the pictures into the neural network model;
If personnel exist in the monitoring area, face recognition is carried out, whether personnel are recorded or not is judged, if personnel are not recorded, the corresponding video data are marked as high-priority video data, if personnel are recorded, the corresponding video data are marked as low-priority video data, the preset picture number of the video data input to the neural network is determined according to the priority, the preset picture number of the high-priority video data is larger than the preset picture number of the low-priority video data, framing processing is carried out on the high-priority video data and judgment is carried out in the input neural network model,
And if personnel exist in the monitoring area, when the pictures are input into the neural network model, extracting one picture at each preset interval to be input into the neural network model until the number of the input pictures is equal to the preset number of the pictures or the judgment result of the neural network model is that the door is opened.
2. The method for identifying abnormal conditions of a casing based on computer vision according to claim 1, wherein: in the step S2, framing is carried out on the decoded video data according to a first preset rule; the first preset rule includes an interval of processing frames, a size of a picture, and a storage path of the picture.
3. The method for identifying abnormal conditions of a casing based on computer vision according to claim 2, wherein: in the step S3, during preprocessing, the image is converted into a gray level image, eight-bit plane decomposition is carried out on the gray level image, and one bit plane image classification mark is extracted.
4. The method for identifying abnormal conditions of a casing based on computer vision according to claim 3, wherein: in the step S5, video data in a preset format are obtained in real time and decoded; and framing the decoded video data according to a second preset rule to generate a plurality of pictures, preprocessing the pictures, and inputting the preprocessed pictures into the trained neural network model.
5. The method for identifying abnormal conditions of a casing based on computer vision according to claim 4, wherein: in S6, the time of opening the door is obtained, and it is determined whether the operation log has a door opening plan corresponding to the time recorded, and if not, it is determined that the door is opened abnormally.
6. The method for identifying abnormal conditions of a casing based on computer vision according to claim 5, wherein: in the step S3, after the gray level image is subjected to eight-bit plane decomposition, discarding the six-bit plane image according to a preset standard, and randomly extracting one bit plane image from the rest bit plane images to carry out classification marking.
7. The method for identifying abnormal conditions of a casing based on computer vision according to claim 6, wherein: in the step S1, the preset format is H.264 format.
8. The method for identifying abnormal conditions of a casing based on computer vision according to claim 7, wherein: in S1, decoding is performed through FFMpeg.
9. The method for identifying abnormal conditions of a casing based on computer vision according to claim 8, wherein: in the step S4, the neural network model is a convolutional neural network model.
CN202210094293.9A 2022-01-26 2022-01-26 Box abnormal state identification method based on computer vision Active CN114429616B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210094293.9A CN114429616B (en) 2022-01-26 2022-01-26 Box abnormal state identification method based on computer vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210094293.9A CN114429616B (en) 2022-01-26 2022-01-26 Box abnormal state identification method based on computer vision

Publications (2)

Publication Number Publication Date
CN114429616A CN114429616A (en) 2022-05-03
CN114429616B true CN114429616B (en) 2024-07-26

Family

ID=81313746

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210094293.9A Active CN114429616B (en) 2022-01-26 2022-01-26 Box abnormal state identification method based on computer vision

Country Status (1)

Country Link
CN (1) CN114429616B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115116005A (en) * 2022-06-29 2022-09-27 济南银捷实业有限公司 Method for closing cash box, training field unmanned algorithm model and judging logic

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109033993A (en) * 2018-06-29 2018-12-18 南京行者易智能交通科技有限公司 A kind of method and device of image recognition detection switch door
CN113393625A (en) * 2021-05-08 2021-09-14 中电海康集团有限公司 Anti-theft alarm evidence obtaining method and system for intelligent lamp pole

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018000116A1 (en) * 2016-06-26 2018-01-04 康丰生 Face recognition-based safe box
SE543788C2 (en) * 2019-04-04 2021-07-20 Icomera Ab System and method for door error detection
CN110427815B (en) * 2019-06-24 2020-07-10 特斯联(北京)科技有限公司 Video processing method and device for realizing interception of effective contents of entrance guard
CN110647192B (en) * 2019-09-03 2021-05-25 泰安东华合创软件有限公司 Intelligent monitoring and management system for firearms
CN112381066B (en) * 2020-12-10 2023-04-18 杭州西奥电梯有限公司 Abnormal behavior identification method for elevator riding, monitoring system, computer equipment and storage medium
CN112542021A (en) * 2020-12-18 2021-03-23 三门核电有限公司 Normally closed fireproof door intelligent monitoring system and method
CN113158888A (en) * 2021-04-19 2021-07-23 广州咔隆安防科技有限公司 Elevator abnormal video identification method
CN113901895B (en) * 2021-09-18 2022-09-27 武汉未来幻影科技有限公司 Door opening action recognition method and device for vehicle and processing equipment
CN113923418A (en) * 2021-10-28 2022-01-11 北京国基科技股份有限公司 System and method for detecting abnormal opening of box door based on video analysis

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109033993A (en) * 2018-06-29 2018-12-18 南京行者易智能交通科技有限公司 A kind of method and device of image recognition detection switch door
CN113393625A (en) * 2021-05-08 2021-09-14 中电海康集团有限公司 Anti-theft alarm evidence obtaining method and system for intelligent lamp pole

Also Published As

Publication number Publication date
CN114429616A (en) 2022-05-03

Similar Documents

Publication Publication Date Title
CN112001284A (en) Labor service real-name system management system based on artificial intelligence
CN102014278A (en) Intelligent video monitoring method based on voice recognition technology
CN105540377A (en) Internet of things remote elevator monitoring system with human face matching function
CN101317455A (en) Video code stream filtering method and filtering node
CN114429616B (en) Box abnormal state identification method based on computer vision
CN108364374A (en) Face access control device based on deep learning and method
CN206271580U (en) A switch based on image recognition position
CN105430316B (en) Trial video automatic control system and method
CN111800617A (en) Intelligent security system based on Internet of things
CN114078224A (en) A kind of infrastructure on-site monitoring system and method based on 5G network architecture
CN114500950B (en) Box abnormal state detection system and method based on smart city
CN116055690A (en) Method and equipment for processing machine room monitoring video
CN113115003B (en) Hotel daily management safety online monitoring method based on image acquisition and analysis technology and Internet of things
CN117197713A (en) Extraction method based on digital video monitoring system
CN115512304B (en) Image recognition-based subway station security monitoring system
CN113869290B (en) Fire fighting access occupation identification method and device based on artificial intelligence technology
CN114429617B (en) Abnormal recognition result processing method applied to smart city box detection
CN118506273A (en) Risk feature quantity twin method and system based on power monitoring video stream
CN114285970A (en) Intelligent station operation auxiliary linkage method based on video analysis technology
CN113158888A (en) Elevator abnormal video identification method
CN114390260A (en) Hazardous area and important place monitoring platform applied to smart city
CN117201739A (en) Real-time monitoring method, system and device for construction operation and storage medium
CN112669553A (en) Unattended system and method for oil and gas station
CN115880821A (en) Access control management method based on digital twin and AIoT technology
CN112669554A (en) Illegal intrusion detection and driving system and method for oil and gas station

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant