CN116389492A - Video analysis system, method, device, computer readable storage medium - Google Patents

Abstract

The present disclosure provides a video analysis system, method, apparatus, computer readable storage medium. The system comprises: the client is used for initiating a video analysis task to the computing center; the computing center is configured with a container running environment, an event library and a resource scheduler, wherein the event library stores model images corresponding to various video analysis tasks, and the resource scheduler is used for determining task execution nodes from a plurality of edge working nodes and the computing center and sending the video analysis tasks to the task execution nodes; each edge working node is configured with a container running environment, a video docking module and a task execution module, the video docking module is used for acquiring video data of a video analysis task, the task execution module is used for acquiring a target model mirror image corresponding to the video analysis task based on the model mirror image of the event library, and a container is created and run according to the target model mirror image in the container running environment to obtain a video analysis result.

Description

Video analysis system, method, device, computer readable storage medium

technical field

The present disclosure relates to the technical field of video processing, and in particular to a video analysis system, method, device, and computer-readable storage medium.

Background technique

With the development of video processing technology, video analysis systems such as security monitoring, identity recognition, and behavior analysis based on video processing technology have become an important part of digital transformation and business value mining in various industries.

In related technologies, a video analysis system is composed of a cloud server and a network camera. In this video analysis system, the network camera uploads the obtained video data to the cloud server, and the cloud server centrally processes the video data to complete the video analysis task. .

However, uploading the video data to the cloud server for centralized processing takes up a lot of network bandwidth resources and easily causes network congestion.

It should be noted that the information disclosed in the above background section is only for enhancing the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art.

Contents of the invention

The present disclosure provides a video analysis system, method, device, and computer-readable storage medium, at least to a certain extent, overcoming the problem of occupying large network bandwidth resources and easily causing network congestion in related technologies.

Other features and advantages of the present disclosure will become apparent from the following detailed description, or in part, be learned by practice of the present disclosure.

According to a first aspect of the present disclosure, a video analysis system is provided, including: a client, configured to initiate a video analysis task to a computing center; the computing center is configured with a container operating environment, an event library, and a resource scheduler, wherein, The event library stores model images corresponding to various video analysis tasks, and the resource scheduler is used to determine a task execution node from a plurality of edge work nodes and the computing center, and send the video to the task execution node Analyzing tasks; the plurality of edge work nodes, wherein each edge work node is configured with a container operating environment, a video docking module and a task execution module, wherein the video docking module is used to obtain video data of the video analysis task , the task execution module is configured to obtain a target model image corresponding to the video analysis task based on the model image of the event library, and create and run a container according to the target model image in the container running environment to obtain a video Analyze the results.

In an embodiment of the present disclosure, the task execution module is further configured to stop the running of the container when the video analysis result is obtained.

In an embodiment of the present disclosure, each edge working node is further configured with a memory; the task execution module is configured to acquire the Target model mirroring, and storing the target mirroring model in the memory; and, in the case of acquiring the target model mirroring for the nth time, acquiring the target model mirroring from the memory, where n is greater than 1 integer.

In an embodiment of the present disclosure, each edge working node is also configured with a resource detection pointer; the resource detection pointer is used to monitor and report the available operating resource information of the edge working node to the computing center; the computing center A node state storage is also configured; the node state storage is used to store and update the available running resource information of the plurality of edge working nodes; Resource information, determining a task execution node from the plurality of edge working nodes and the computing center.

In an embodiment of the present disclosure, the resource scheduler is configured to determine that the multiple edge working nodes do not meet the conditions for executing the video analysis task according to the available running resource information of the multiple edge working nodes In this case, it is determined that the computing center is the task execution node.

In an embodiment of the present disclosure, the available running resource information includes a CPU usage rate of a central processing unit, memory usage information, a GPU usage rate of a graphics processing unit, and video memory usage information.

In an embodiment of the present disclosure, the task execution module is further configured to upload the video analysis result to the computing center; the computing center is also configured with a streaming media server and a file sharing server; the streaming media The server is configured to store the video data in the video analysis result; the file sharing server is configured to store the image data in the video analysis result.

In an embodiment of the present disclosure, the computing center is further configured with a backup network video recorder; the backup network video recorder is used for storing backup history videos.

In an embodiment of the present disclosure, each edge working node is also configured with a message routing module; the message routing module is configured to send messages related to the video analysis task to other edge working nodes or the computing center .

In an embodiment of the present disclosure, the system further includes: a plurality of network cameras, configured to acquire and send the video data to the task execution node.

According to the second aspect of the present disclosure, a video analysis method is provided, which is applied to an edge working node, including: receiving a video analysis task sent by a computing center; acquiring a target model image corresponding to the video analysis task; acquiring the video analysis task Corresponding video data; creating a container according to the target model image, and running the container to process the video data to obtain a video analysis result.

According to a third aspect of the present disclosure, a video analysis method is provided, which is applied to a computing center, including: receiving a video analysis task sent by a client; obtaining information on available running resources of multiple edge working nodes; and according to the video analysis task and The information about the available operating resources determines a task execution node from the plurality of edge working nodes and the computing center; sends the video analysis task to the task execution node, so that the task execution node executes the task described video analysis tasks.

According to a fourth aspect of the present disclosure, a video analysis device is provided, which is applied to an edge working node, including: a first receiving module, configured to receive a video analysis task sent by a computing center; a first obtaining module, configured to obtain the video The target model image corresponding to the analysis task; the first acquisition module is also used to obtain the video data corresponding to the video analysis task; the processing module is used to create a container according to the target model image, and run the container to the The video data is processed to obtain video analysis results.

According to a fifth aspect of the present disclosure, a video analysis device is provided, which is applied to a computing center, including: a second receiving module, configured to receive a video analysis task sent by a client; a second acquiring module, configured to acquire a plurality of edge tasks Available operating resource information of the node; a determining module, configured to determine a task execution node from the plurality of edge working nodes and the computing center according to the video analysis task and the available operating resource information; a sending module, It is configured to send the video analysis task to the task execution node, so that the task execution node executes the video analysis task.

According to a sixth aspect of the present disclosure, there is provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to Execute any of the video analysis methods described above.

According to a seventh aspect of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the video analysis methods described above is implemented.

According to an eighth aspect of the present disclosure, a computer program product is provided, the computer program product includes a computer program or a computer instruction, the computer program or the computer instruction is loaded and executed by a processor, so that the computer implements any of the above The video analysis method described.

The technical solutions provided by the embodiments of the present disclosure at least include the following beneficial effects:

In the technical solution provided by the embodiments of the present disclosure, after the computing center receives the video analysis task sent by the client, it determines the task execution node from multiple edge working nodes and the computing center through the configured resource scheduler, and sends the task execution node to the task execution node. The node sends the video analysis task. Each edge work node is configured with a container operating environment. After the edge work node receives the video analysis task, the task execution module configured by the edge work node can obtain the corresponding video analysis task according to the model image in the event library of the computing center. The target model image, and create a container in the container operating environment according to the target model image, and process the video data obtained by the video docking module according to the container to obtain the video analysis result. Using edge working nodes to complete video analysis tasks can reduce the network bandwidth resources occupied by uploading video data to a certain extent, and reduce the possibility of network congestion.

In addition, configuring the event library in the computing center and the way that the edge working nodes can obtain the required model mirroring from the event library can avoid the need to perform in the edge working nodes one by one due to different video analysis tasks and the heterogeneity of edge working nodes. The problem of adaptive deployment reduces the cost of deploying video analysis system and subsequent system maintenance.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure. Apparently, the drawings in the following description are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to these drawings without creative efforts.

FIG. 1 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

Figure 2 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

FIG. 3 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

FIG. 4 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

FIG. 7 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

FIG. 8 shows a schematic diagram of a video analysis system in one embodiment of the present disclosure;

FIG. 9 shows a flowchart of a video analysis method in one embodiment of the present disclosure;

FIG. 10 shows a flowchart of a video analysis method in one embodiment of the present disclosure;

Fig. 11 shows a schematic diagram of a video analysis device in an embodiment of the present disclosure;

Fig. 12 shows a schematic diagram of a video analysis device in an embodiment of the present disclosure;

Fig. 13 shows a structural block diagram of an electronic device in an embodiment of the present disclosure.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and/or processor means and/or microcontroller means.

It should be understood that the various steps described in the method implementations of the present disclosure may be executed in different orders, and/or executed in parallel. Additionally, method embodiments may include additional steps and/or omit performing illustrated steps. The scope of the present disclosure is not limited in this respect.

It should be noted that concepts such as "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the sequence of functions performed by these devices, modules or units or interdependence.

It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more" multiple".

FIG. 1 shows a schematic diagram of a video analysis system in an embodiment of the present disclosure. As shown in FIG. 1 , the video analysis system may include: a client 110 , a computing center 120 , and multiple edge working nodes 130 .

Wherein, the client 110 is used for initiating a video analysis task to the computing center 120 and receiving a video analysis result fed back by the computing center 120 . The embodiments of the present disclosure do not limit the specific processing of the video data for the video analysis task. For example, a video analysis task may be an object detection task. For another example, the video analysis task may also be a human skeleton detection task. For another example, the video analysis task may also be an object re-identification task.

In one embodiment, the computing center 120 is configured with an event library 121 , a resource scheduler 122 and a container runtime environment 123 . Wherein, the event library 121 stores model images corresponding to various video analysis tasks. For example, when the video analysis task is a target monitoring task, the model image corresponding to the video analysis task is the image of a model capable of performing target monitoring processing on video data. For another example, when the video analysis task is a human skeleton detection task, the model image corresponding to the video analysis task is the image of a model capable of performing human skeleton detection on video data.

The embodiment of the present disclosure does not limit which model images corresponding to the various video analysis tasks configured in the event library 121 specifically include which model images correspond to the video analysis tasks. For example, the video analysis tasks corresponding to the model images configured in the event library 121 include: video segmentation tasks, human skeleton detection tasks, object detection tasks, object tracking tasks, object re-identification tasks, human behavior detection tasks, and the like.

Wherein, the video data needs to be processed as follows to complete the above video analysis tasks.

Video segmentation task: cut the video at a certain time interval, and output the segmented pictures.

Human skeleton detection task: use the human body key point detection model to perform feature extraction and calibration on the human skeleton in the video, and output the picture of the calibrated skeleton or the extracted skeleton features; among them, the specific model of the human body key point detection model is implemented in this disclosure Examples are not limited. For example, the human body key point detection model applied in the embodiment of the present disclosure is Open pose (a human body posture detection model).

Target detection task: use the target detection model to identify the specified target in the video, use the bounding box to mark the target in the video frame, and output the picture or video with the bounding box. The embodiment of the present disclosure does not limit what kind of model the target detection mold body is. For example, the target detection model applied in the embodiment of the present disclosure is yolov5Lite-s.pt (a target detection model).

Target tracking task: By combining the target detection model and the tracking model to achieve continuous calibration of the same target in consecutive video frames, use the bounding box to mark the target in the video frame, and mark the number of the target, and output the continuous pictures with the bounding box and number or video. The embodiments of the present disclosure do not limit the specific model of the tracking model. For example, the tracking model is a deepsort tracking algorithm model.

Target re-identification task: The features of the target in the video are extracted through the neural network model to match the defined search target, and the output target contains a numbered re-identification picture under the video or picture. The embodiments of the present disclosure do not limit the specific model of the neural network model. For example, the neural network model is ResNet-50 (Residual Network50, a residual network model).

Human behavior detection task: Combine the human skeleton detection model and the action recognition model to extract the texture actions and behaviors of the characters in the video, and output the watermarked pictures of the calibration behavior detection results. The embodiment of the present disclosure does not limit the specific model of the action recognition model. For example, the action recognition model applied to the embodiments of the present disclosure is an R-CNN (Region-Convolutional Neural Networks, regional convolutional neural network) action recognition model.

The resource scheduler 122 is configured to determine a task execution node from a plurality of edge work nodes 130 and the computing center 120, and send a video analysis task to the task execution node. Wherein, the task execution node selected by the resource scheduler 122 may be one of the edge working nodes 130 or the computing center 120 .

In one embodiment, when the computing center 120 is used as a task execution node, the computing center 120 can call the target model image corresponding to the video analysis task in the event library 121, and construct the target model image corresponding to the target model image in the container operating environment 123. The container runs the container to process the video data and obtain the video analysis result.

Each edge work node among the plurality of edge work nodes 130 is configured with a container runtime environment 131 , a video docking module 132 and a task execution module 133 .

Wherein, the video docking module 132 is used for acquiring video data of a video analysis task. The video data of the video analysis task may be real-time video data or pre-saved video data, which is not limited in this embodiment of the present disclosure.

The task execution module 133 is used to obtain the target model image corresponding to the video analysis task based on the model image of the event library 121, and create and run a container according to the target model image in the container running environment 131 to obtain the video analysis result. Wherein, creating and running a container according to the target model image in the container running environment 131 to obtain the video analysis result includes: creating a container according to the target model image in the container running environment 131, and running the container to process the video data to obtain the video analysis result .

In another embodiment, the task execution module 133 is further configured to stop the running of the container when the video analysis result is obtained. After obtaining the processing result of the video analysis task, the method of stopping the corresponding container and releasing the resources occupied by running the container can reduce the waste of resources and improve the utilization rate of resources.

In the technical solution provided by the embodiments of the present disclosure, after the computing center receives the video analysis task sent by the client, it determines the task execution node from multiple edge working nodes and the computing center through the configured resource scheduler, and sends the task execution node to the task execution node. The node sends the video analysis task. Each edge work node is configured with a container operating environment. After the edge work node receives the video analysis task, the task execution module configured by the edge work node can obtain the corresponding video analysis task according to the model image in the event library of the computing center. The target model image, and create a container in the container operating environment according to the target model image, and process the video data obtained by the video docking module according to the container to obtain the video analysis result. Using edge working nodes to complete video analysis tasks can reduce the network bandwidth resources occupied by uploading video data to a certain extent, and reduce the possibility of network congestion.

In addition, configuring the event library in the computing center and the way that the edge working nodes can obtain the required model mirroring from the event library can avoid the need to perform in the edge working nodes one by one due to different video analysis tasks and the heterogeneity of edge working nodes. The problem of adaptive deployment reduces the cost of deploying video analysis system and subsequent system maintenance.

FIG. 2 shows a schematic diagram of a video analysis system in an embodiment of the present disclosure. As shown in FIG. 2 , the video analysis system may include: a client 110 , a computing center 120 , and multiple edge working nodes 130 .

Wherein, the edge working node is configured with a container operating environment 131 , a video docking module 132 and a task execution module 133 , and is also configured with a memory 134 .

The task execution module 133 is used to obtain the target model mirror image from the event library 121 in the case of acquiring the target model mirror image for the first time, and store the target mirror image model in the memory 134; and, acquire the target model mirror image for the nth time In the case of , the target model image is obtained from the memory 134, and n is an integer greater than 1.

By obtaining the corresponding target model image when executing a certain type of video analysis task for the first time, it can avoid manually configuring the target model image in multiple edge working nodes one by one, and realize automatic edge working nodes Configure model mirroring in the middle, which reduces the workload of configuring the entire video analysis system and improves the efficiency of configuring the video analysis system.

After the task execution module 133 obtains the target model image, it creates a container according to the target model image, and runs the container to process the video data, so that the video data can be processed by the model corresponding to the model image, and the video analysis result can be obtained.

FIG. 3 shows a schematic diagram of a video analysis system in an embodiment of the present disclosure. As shown in FIG. 3 , the video analysis system may include: a client 110 , a computing center 120 , and multiple edge working nodes 130 .

In one embodiment, multiple edge working nodes 130 are configured with a container runtime environment 131 , a video docking module 132 , and a task execution module 133 , and are also configured with a resource detection pointer 135 ; the computing center 120 is also configured with a node state storage 124 .

Wherein, the resource detection pointer 135 is used to monitor and report to the computing center 120 the available running resource information of the edge working nodes. The resource detection pointer 135 can monitor the available running resources of the edge working nodes in real time, and generate corresponding available running resource information, and then the resource detecting pointer 135 reports the available running resource information to the computing center 120 .

The node state storage 124 is used to store and update the available running resource information of multiple edge working nodes. After the computing center 120 receives the available operating resource information reported by the edge node, it stores the available operating resource information in the node state memory 124, and the node state memory 124 utilizes the information of the available operating resource reported by the edge working node for the last time to perform work on the edge. The available running resource information reported by the node last time is updated.

The resource scheduler 122 is configured to determine a task execution node from the multiple edge worker nodes 130 and the computing center 120 according to the available running resource information of the multiple edge worker nodes 130 .

Different video analysis tasks require different operating resources, and the operating resources required by the video analysis task can be determined according to the type of the video analysis task. The resource scheduler 122 may determine the task execution node from the multiple edge working nodes 130 and the computing center 120 according to the running resources required by the video analysis task and the available running resource information of the multiple edge working nodes 130 .

The process for the resource scheduler 122 to determine the task execution node may include: sorting the corresponding running resources of multiple edge work nodes according to the running resources required by the video analysis task, and obtaining the sorting result; A task execution node that runs resources required by a video analysis task.

In one embodiment, the resource scheduler 122 is configured to determine that the computing center 120 is the task when it is determined that the multiple edge working nodes 130 do not meet the conditions for executing the video analysis task according to the available running resource information of the multiple edge working nodes 130. Execute the node. That is to say, the computing center 120 executes the video analysis task as a task execution node when none of the multiple edge working nodes 130 meets the conditions for executing the video analysis task.

By using the computing center 120 as a backup task execution node, the stability of the video analysis system can be improved, and multiple edge working nodes are prevented from being insufficient in available operating resources, or the network connection is not smooth, or the video memory of the edge working nodes overflows, etc. , the video analysis system cannot handle the video analysis task.

In an embodiment, the available running resource information of the edge node may include: CPU (Central Processing Unit, central processing unit) occupancy rate, memory occupancy information, GPU (Graphics Processing Unit, graphics processing unit) utilization rate, video memory occupancy information.

For example, when the video analysis task has a high demand for GPU resources, multiple edge work nodes can be sorted according to the GPU utilization and video memory occupation information of multiple edge work nodes, and the task to perform the video analysis task can be determined according to the sorting results Execute the node.

In one embodiment, the relative requirements of CPU resources and GPU resources for video segmentation tasks, human skeleton detection tasks, target detection tasks, target tracking tasks, target re-identification tasks and human behavior detection tasks are shown in Table 1 below.

Table 1

Video Analysis Task CPU resource requirements GPU resource requirements Video Segmentation Task Low - object detection task Low Low object tracking task high - Object Re-identification Task Low middle Human Skeleton Detection Task middle - Human Behavior Detection Task middle high

The resource scheduler 122 automatically assigns the task execution nodes of the video analysis task according to the available operating resource information of the plurality of edge working nodes 130, while taking into account the available operating resources of the edge working nodes and the operating resources required by the video analysis task, It is conducive to improving the resource utilization and business efficiency of edge working nodes.

FIG. 4 shows a schematic diagram of a video analysis system in an embodiment of the present disclosure. As shown in FIG. 4 , the video analysis system may include: a client 110 , a computing center 120 , and multiple edge working nodes 130 .

Wherein, the computing center 120 is not only configured with an event library 121 , a resource scheduler 122 and a container running environment 123 , but also configured with a streaming media server 125 and a file sharing server 126 .

The edge working node serves as a task execution node, and after completing the video analysis task to obtain the video analysis result, the task execution module 133 uploads the video analysis result to the computing center 120 . After receiving the video analysis result, the computing center 120 stores the video data in the video analysis result in the streaming media server 125 ; and stores the image data in the video analysis result in the file sharing server 126 . That is to say, the streaming media server 125 is used to store the video data in the video analysis result; the file sharing server 126 is used to store the image data in the video analysis result.

In one embodiment, after the computing center 120 obtains the video analysis result as a task execution node, the video data in the video analysis result is stored in the streaming media server 125, and the image data in the video analysis result is stored in the file sharing server 126 .

In one embodiment, the file sharing server 126 is also used to assist multiple edge working nodes 130 to realize a unified data plane, so that each business module of different edge working nodes can call the data uploaded by other edge working nodes from the file sharing server 126. image data.

FIG. 5 shows a schematic diagram of a video analysis system in an embodiment of the present disclosure. As shown in FIG. 5 , the video analysis system may include: a client 110 , a computing center 120 , and multiple edge working nodes 130 .

Wherein, the computing center 120 is configured with a backup network video recorder 127 in addition to an event library 121 , a resource scheduler 122 and a container operating environment 123 . The backup network video recorder 127 is used for storing backup history videos. The embodiments of the present disclosure do not limit which videos are included in the backup history video. For example, backing up historical videos includes backing up video data and backing up video-type data in video analysis results.

FIG. 6 shows a schematic diagram of a video analysis system in an embodiment of the present disclosure. As shown in FIG. 6 , the video analysis system may include: a client 110 , a computing center 120 , and multiple edge working nodes 130 .

Wherein, each edge working node among the plurality of edge working nodes 130 is also configured with a message routing module 136; the message routing module 136 is used to send messages related to video analysis tasks to other edge working nodes or the computing center 120. The embodiments of the present disclosure do not limit which messages are specifically included in the messages related to the video analysis task. For example, messages related to video analysis tasks include: service start messages, service stop messages, container running status messages, and messages carrying analysis results.

FIG. 7 shows a schematic diagram of a video analysis system in an embodiment of the present disclosure. As shown in FIG. 7 , the video analysis system may include: a client 110, a computing center 120, a plurality of edge working nodes 130 and a plurality of network cameras 140.

Wherein, each of the plurality of network cameras 140 can obtain video data, and upload the obtained video data to the task execution node. For example, when the task execution node is an edge working node, the network camera can send the acquired video data to the edge working node, and the edge working node receives the video data uploaded by the network camera through the video docking module. For another example, when the task execution node is the computing center 120, the network camera may send the acquired video data to the computing center 120, and the computing center 120 completes the acquisition of the video data after receiving the video data.

In order to facilitate understanding of the technical solutions provided by the embodiments of the present disclosure, the technical solutions provided by the embodiments of the present disclosure will be described below in conjunction with the video analysis systems corresponding to FIGS. 1 to 7 and the video analysis system shown in FIG. 8 .

As shown in FIG. 8 , the video analysis system may include: a client 110 , a computing center 120 , multiple edge working nodes 130 and multiple network cameras 140 .

Wherein, the computing center 120 is configured with an event library 121 , a resource scheduler 122 , a container operating environment 123 , a node state storage 124 , a streaming media server 125 , a file sharing server 126 and a backup network video recorder 127 .

A plurality of edge working nodes 130 are configured with a container runtime environment 131 , a video docking module 132 , a task execution module 133 , a memory 134 , a resource detection pointer 135 and a message routing module 136 .

After the client 110 sends the video analysis task to the computing center 120, the resource scheduler 122 configured by the computing center 120 obtains the available operating resource information of multiple edge nodes 130 from the node state storage 124, and determines the Outgoing task execution node. The available operating resource information of multiple edge nodes 130 is monitored by the respectively configured resource detection pointers 135 and reported to the computing center 120 through the message routing module 136 .

In the case that the task execution node is an edge working node, the resource scheduler 122 configured by the central node 120 sends the video analysis task to the edge working node. After receiving the video analysis task, the edge working node obtains the video data corresponding to the video analysis task through the video docking module 132, and obtains the target model corresponding to the video analysis task from the memory 134 or the event library 121 through the task execution module 133 image, and create a container in the container operating environment 131 according to the target model image, and run the container in the container operating environment 131 to process the video data and obtain the video analysis result. Wherein, video data may be provided by multiple network cameras 140 .

Afterwards, the task execution module 133 will feed back the video analysis result to the computing center 120 through the message routing 136 . The computing center 120 stores the video data in the video analysis result in the streaming server 125, stores the image data in the video analysis result in the file sharing server 126, and stores the backup of the image data in the video analysis result in the backup Network Video Recorder 127.

In the case that the task execution node is the computing center 120, the computing center 120 obtains the video data provided by a plurality of network cameras 140; obtains the target model image corresponding to the video analysis task from the event library 121, and according to the target model image in the container A container is created in the running environment 123, and the container is run in the running environment 123 to process the video data to obtain a video analysis result. Afterwards, the computing center 120 stores the video data in the video analysis results in the streaming server 125, stores the image data in the video analysis results in the file sharing server 126, and stores the image data in the video analysis results. A backup of the video data is stored in the backup network video recorder 127 .

After the computing center 120 obtains the video analysis result, it feeds back the video analysis result to the client 110 .

The client 110, the computing center 120, the plurality of edge working nodes 130 and the plurality of network cameras 140 are connected through a network, and the network may be a wired network or a wireless network.

Optionally, the aforementioned wireless network or wired network uses standard communication technologies and/or protocols. The network is usually the Internet, but can be any network, including but not limited to Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless network, private network, or any combination of virtual private networks). In some embodiments, data exchanged over a network is represented using technologies and/or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), and the like. In addition, conventional methods such as Secure Socket Layer (Secure Socket Layer, SSL), Transport Layer Security (Transport Layer Security, TLS), Virtual Private Network (Virtual Private Network, VPN), Internet Protocol Security (Internet Protocol Security, IPsec) can also be used. Encryption technology to encrypt all or some links. In some other embodiments, customized and/or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.

The client 110 may be various electronic devices including, but not limited to, smartphones, tablet computers, laptop computers, desktop computers, wearable devices, augmented reality devices, virtual reality devices, and the like.

Optionally, the clients of the application programs installed in different clients 110 are the same, or the clients of the same type of application programs based on different operating systems. Based on different terminal platforms, the specific form of the client of the application program may also be different, for example, the client of the application program may be a mobile phone client, a PC client, and the like.

The computing center 120 and multiple edge working nodes 130 can be servers that provide various services, and the servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud services, Basic cloud computing such as cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (Content Delivery Network, content distribution network), and big data and artificial intelligence platform Cloud server for the service.

The exemplary implementation manner will be described in detail below in conjunction with the accompanying drawings and embodiments.

An embodiment of the present disclosure provides a video analysis method, which can be executed by any electronic device with computing and processing capabilities. For example, this electronic device is a fringe work section.

FIG. 9 shows a flowchart of a video analysis method in an embodiment of the present disclosure. As shown in FIG. 9 , the video analysis method provided in the embodiment of the present disclosure includes the following steps S901 to S904.

S901. The edge working node receives the video analysis task sent by the computing center.

S902. The edge working node acquires the target model image corresponding to the video analysis task.

In an embodiment, obtaining the target model image corresponding to the video analysis task by the edge working node may include: in the case of obtaining the target model image for the first time, the edge working node obtains the target model image from the event library of the computing center, And store the target model image in the memory; when obtaining the target model image for the nth time, the edge working node obtains the target model image from the memory, n is an integer greater than 1.

S903. The edge working node acquires video data corresponding to the video analysis task.

Wherein, the video data can be obtained by the network camera in real time and provided to the edge working node; or, the video data can be pre-stored in the network device, and the edge working node can obtain the video data from the network device through the network.

S904. The edge working node creates a container according to the target model image, and runs the container to process the video data to obtain a video analysis result.

Configure the event library in the computing center and the way that the edge working nodes can obtain the required model image from the event library, which can avoid the need to adapt one by one in the edge working nodes due to different video analysis tasks and the heterogeneity of edge working nodes Deployment problems reduce the cost of deploying video analysis systems and subsequent system maintenance.

FIG. 10 shows a flowchart of a video analysis method in an embodiment of the present disclosure. As shown in FIG. 10 , the video analysis method provided in the embodiment of the present disclosure includes the following steps S1001 to S1004.

S1001. The computing center receives a video analysis task sent by a client.

S1002. The computing center acquires information about available running resources of multiple edge working nodes.

S1003. The computing center determines a task execution node from multiple edge working nodes and the computing center according to the video analysis task and available operating resource information.

In one embodiment, the computing center determines the task execution node from the multiple edge working nodes and the computing center according to the video analysis task and the available running resource information, which may include: according to the available running resource information of the multiple edge working nodes, When it is determined that multiple edge working nodes do not meet the conditions for performing the video analysis task, determine the computing center as the task execution node.

By using the computing center as a backup task execution node, the stability of the video analysis system can be improved, and multiple edge working nodes are prevented from running out of available resources, or the network connection is not smooth, or the video memory of the edge working nodes overflows, etc. The video analytics system cannot handle video analytics tasks.

S1004. The computing center sends the video analysis task to the task execution node, so that the task execution node executes the video analysis task.

Based on the same inventive concept, the embodiments of the present disclosure also provide two video analysis devices, as described in the following embodiments. Since the problem-solving principle of this device embodiment is similar to that of the above-mentioned method embodiment, the implementation of this device embodiment can refer to the implementation of the above-mentioned method embodiment, and repeated descriptions will not be repeated.

Fig. 11 shows a schematic diagram of a video analysis device in an embodiment of the present disclosure. As shown in Fig. 11, the device is applied to an edge working node, and includes: a first receiving module 1101, configured to receive a video analysis task sent by a computing center; An acquisition module 1102, used to acquire the target model image corresponding to the video analysis task; the first acquisition module 1102, also used to acquire the video data corresponding to the video analysis task; a processing module 1103, used to create a container according to the target model image, and run The container processes the video data to obtain the video analysis result.

Fig. 12 shows a schematic diagram of a video analysis device in an embodiment of the present disclosure. As shown in Fig. 12, the device is applied to a computing center and includes: a second receiving module 1201, configured to receive a video analysis task sent by a client; The obtaining module 1202 is used to obtain the available operating resource information of multiple edge working nodes; the determining module 1203 is used to determine the task execution node from multiple edge working nodes and computing centers according to the video analysis task and the available operating resource information ; The sending module 1204 is configured to send the video analysis task to the task execution node, so that the task execution node executes the video analysis task.

Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be embodied in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which can be collectively referred to herein as "circuit", "module" or "system".

An electronic device 1300 according to this embodiment of the present disclosure is described below with reference to FIG. 13 . The electronic device 1300 shown in FIG. 13 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 13, electronic device 1300 takes the form of a general-purpose computing device. Components of the electronic device 1300 may include but not limited to: at least one processing unit 1310 , at least one storage unit 1320 , and a bus 1330 connecting different system components (including the storage unit 1320 and the processing unit 1310 ).

Wherein, the storage unit stores program codes, and the program codes can be executed by the processing unit 1310, so that the processing unit 1310 executes various exemplary implementations according to the present disclosure described in the above "Detailed Embodiments" section of this specification. Implementation steps.

The storage unit 1320 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 1321 and/or a cache storage unit 1322 , and may further include a read-only storage unit (ROM) 1323 .

Storage unit 1320 may also include programs/utilities 1324 having a set (at least one) of program modules 1325, such program modules 1325 including but not limited to: an operating system, one or more application programs, other program modules, and program data, Implementations of networked environments may be included in each or some combination of these examples.

Bus 1330 may represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local area using any of a variety of bus structures. bus.

The electronic device 1300 can also communicate with one or more external devices 1340 (such as keyboards, pointing devices, Bluetooth devices, etc.), and can also communicate with one or more devices that enable the user to interact with the electronic device 1300, and/or communicate with Any device (eg, router, modem, etc.) that enables the electronic device 1300 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 1350 . Moreover, the electronic device 1300 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through the network adapter 1360 . As shown in FIG. 13 , the network adapter 1360 communicates with other modules of the electronic device 1300 through the bus 1330 . It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

Through the description of the above implementations, those skilled in the art can easily understand that the example implementations described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of software products, and the software products can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, and the computer-readable storage medium may be a readable signal medium or a readable storage medium. A program product capable of realizing the above-mentioned methods of the present disclosure is stored thereon. In some possible implementation manners, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code, and when the program product is run on a terminal device, the program code is used to make the The terminal device executes the steps according to various exemplary embodiments of the present disclosure described in the above "Detailed Embodiments" section of this specification.

More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

In the present disclosure, a computer-readable storage medium may include a data signal carrying readable program code in baseband or as part of a carrier wave traveling as a data signal. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

Alternatively, program code contained on a computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, cable, optical cable, RF, etc., or any suitable combination of the above.

During specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, and the programming language includes an object-oriented programming language—such as Java, C++, etc., or Includes conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (for example, using an Internet service provider). business to connect via the Internet).

In an exemplary embodiment of the present disclosure, a computer program product is also provided. The computer program product includes a computer program or a computer instruction. The computer program or computer instruction is loaded and executed by a processor, so that the computer implements the above-mentioned "specific The steps according to various exemplary embodiments of the present disclosure are described in the "EMBODIMENTS" section.

It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. Actually, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided to be embodied by a plurality of modules or units.

In addition, although steps of the methods of the present disclosure are depicted in the drawings in a particular order, there is no requirement or implication that the steps must be performed in that particular order, or that all illustrated steps must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution, etc.

Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of software products, and the software products can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

Other embodiments of the present disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. The present disclosure is intended to cover any modification, use or adaptation of the present disclosure. These modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. . The specification and examples are to be considered exemplary only, with the true scope of the disclosure being indicated by the appended claims.

Claims (15)

1. A video analysis system, characterized in that, comprising: The client is used to initiate video analysis tasks to the computing center; The computing center is configured with a container operating environment, an event library, and a resource scheduler, wherein the event library stores model images corresponding to various video analysis tasks, and the resource scheduler is used to generate data from multiple edge working nodes and all determining a task execution node in the computing center, and sending the video analysis task to the task execution node; The plurality of edge work nodes, wherein each edge work node is configured with a container operating environment, a video docking module, and a task execution module, wherein the video docking module is used to obtain video data of the video analysis task, and the The task execution module is used to obtain the target model image corresponding to the video analysis task based on the model image of the event library, and create and run a container according to the target model image in the container running environment to obtain the video analysis result. 2. The system according to claim 1, wherein the task execution module is further configured to stop the operation of the container when the video analysis result is obtained. 3. The system according to claim 1, wherein each edge working node is also configured with a memory; The task execution module is configured to acquire the target model image from the event library when acquiring the target model image for the first time, and store the target image model in the memory; and, In the case of acquiring the target model image for the nth time, acquire the target model image from the memory, where n is an integer greater than 1. 4. The system according to claim 1, wherein each edge working node is also configured with a resource detection pointer; the resource detection pointer is used to monitor and report the available operating resources of the edge working node to the computing center information; The computing center is also configured with a node state memory; the node state memory is used to store and update the available operating resource information of the plurality of edge working nodes; The resource scheduler is configured to determine a task execution node from the plurality of edge working nodes and the computing center according to the available running resource information of the plurality of edge working nodes. 5. The system according to claim 4, wherein the resource scheduler is configured to determine that the plurality of edge working nodes do not meet the requirements for executing the In the case of the condition of the video analysis task, it is determined that the computing center is the task execution node. 6 . The system according to claim 4 or 5 , wherein the available operating resource information includes CPU occupancy rate of the central processing unit, memory occupancy information, graphics processor GPU utilization rate, and video memory occupancy information. 7. The system according to claim 1, wherein the task execution module is further configured to upload the video analysis result to the computing center; The computing center is also configured with a streaming media server and a file sharing server; the streaming media server is used to store video class data in the video analysis results; the file sharing server is used to store the video analysis results Image class data in . 8. The system according to claim 1, wherein the computing center is further equipped with a backup network video recorder; and the backup network video recorder is used to store backup historical videos. 9. The system according to claim 1, wherein each edge working node is also equipped with a message routing module; the message routing module is used to send messages related to the other edge working nodes or the computing center Messages related to video analysis tasks. 10. The system according to claim 1, further comprising: Multiple network cameras are used to acquire and send the video data to the task execution node. 11. A video analysis method, characterized in that it is applied to edge work nodes, comprising: Receive video analysis tasks sent by the computing center; Obtain the target model image corresponding to the video analysis task; Obtain video data corresponding to the video analysis task; A container is created according to the target model image, and the container is run to process the video data to obtain a video analysis result. 12. A video analysis method, characterized in that it is applied to a computing center, comprising: Receive the video analysis task sent by the client; Obtain the available running resource information of multiple edge working nodes; Determining a task execution node from the plurality of edge working nodes and the computing center according to the video analysis task and the available operating resource information; Sending the video analysis task to the task execution node, so that the task execution node executes the video analysis task. 13. A video analysis device, characterized in that it is applied to an edge working node, comprising: The first receiving module is used to receive the video analysis task sent by the computing center; The first obtaining module is used to obtain the image of the target model corresponding to the video analysis task; The first acquiring module is further configured to acquire video data corresponding to the video analysis task; The processing module is configured to create a container according to the target model image, and run the container to process the video data to obtain a video analysis result. 14. A video analysis device, characterized in that it is applied to a computing center, comprising: The second receiving module is used to receive the video analysis task sent by the client; The second obtaining module is used to obtain the available operating resource information of multiple edge working nodes; A determining module, configured to determine a task execution node from the plurality of edge working nodes and the computing center according to the video analysis task and the available operating resource information; A sending module, configured to send the video analysis task to the task execution node, so that the task execution node executes the video analysis task. 15. A computer-readable storage medium, on which a computer program is stored, wherein the computer program implements the video analysis method according to claim 11 or 12 when executed by a processor.

Images