CN114785790A - Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium - Google Patents
Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium Download PDFInfo
- Publication number
- CN114785790A CN114785790A CN202210476425.4A CN202210476425A CN114785790A CN 114785790 A CN114785790 A CN 114785790A CN 202210476425 A CN202210476425 A CN 202210476425A CN 114785790 A CN114785790 A CN 114785790A
- Authority
- CN
- China
- Prior art keywords
- scheduling
- cross
- task
- application
- analyzed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 256
- 238000000034 method Methods 0.000 title claims abstract description 145
- 238000007405 data analysis Methods 0.000 claims abstract description 340
- 230000008859 change Effects 0.000 claims description 64
- 238000004364 calculation method Methods 0.000 claims description 43
- 238000012545 processing Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 12
- 238000010191 image analysis Methods 0.000 abstract description 7
- 238000004422 calculation algorithm Methods 0.000 description 296
- 230000008569 process Effects 0.000 description 38
- 238000012550 audit Methods 0.000 description 17
- 238000004891 communication Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 15
- 238000012544 monitoring process Methods 0.000 description 13
- 230000005540 biological transmission Effects 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 230000002457 bidirectional effect Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000012552 review Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 238000013500 data storage Methods 0.000 description 3
- 230000000977 initiatory effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005021 gait Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013475 authorization Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1014—Server selection for load balancing based on the content of a request
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1031—Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multi Processors (AREA)
Abstract
The application discloses a cross-domain analysis system, a cross-domain resource scheduling method, a cross-domain resource scheduling device and a storage medium, relates to the technical field of video image intelligent analysis, and can achieve cross-domain scheduling of video image analysis resources. The cross-domain analysis system comprises: a first office point and a second office point; the first local point includes: the second local point comprises: a second data analysis device and a second scheduling device; the first scheduling device is used for receiving a cross-domain resource scheduling application of the first data analysis device; the cross-domain resource scheduling application comprises: the task to be analyzed is applied to other local points except the first local point to execute the task to be analyzed; the first scheduling device is also used for forwarding the cross-domain resource scheduling application to the second scheduling device; the second scheduling device is used for forwarding the cross-domain resource scheduling application to the second data analysis device; and the second data analysis device is used for analyzing the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result.
Description
Technical Field
The present application relates to the field of video image intelligent analysis technologies, and in particular, to a cross-domain analysis system, a cross-domain resource scheduling method, an apparatus, and a storage medium.
Background
At present, in the technical field of video image intelligent analysis, a main working mode of an analysis task is to perform resource scheduling in a single video image intelligent analysis system, wherein scheduled video image intelligent analysis computing resources can be deployed in a single node, and multi-node distributed cluster deployment is also supported.
It can be seen that, the traditional video image intelligent analysis system often only completes the scheduling of resources in the local site, and such a working mode may cause the following problems: when the resources in the office of the office point are insufficient, if a new analysis task occurs, the new analysis task cannot be responded in time, and only can be queued for waiting or discarded, which can cause problems of untimely task processing and low processing efficiency. In addition, if a plurality of video image intelligent analysis systems exist at the same time, because each video image intelligent analysis system is isolated from each other, the busy and idle states of each office point are uneven, and the utilization rate of the whole resources of the video image intelligent analysis system is low.
Disclosure of Invention
The application provides a cross-domain analysis system, a cross-domain resource scheduling method, a cross-domain resource scheduling device and a storage medium, which can realize cross-domain (namely cross-local point) scheduling of video image analysis resources.
In a first aspect, the present application provides a cross-domain resource scheduling method, which is applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the second local point comprises: a second data analysis device and a second scheduling device; the method comprises the following steps: a second scheduling device receives a first cross-domain resource scheduling application sent by a first scheduling device; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; and the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device, so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
It can be understood that the cross-domain resource scheduling method provided by the present application is applied to a cross-domain analysis system, where the cross-domain analysis system includes a plurality of local points (one local point may be regarded as one video image intelligent analysis platform), and each local point includes a scheduling device. One office point can receive cross-domain resource scheduling applications (including tasks to be analyzed) sent by scheduling devices of other office points through the scheduling device of the office point, and analyze the tasks to be analyzed according to the cross-domain resource scheduling applications to obtain analysis results. Therefore, the cross-domain resource scheduling method and the cross-domain resource scheduling device can perform cross-domain (cross-local point information interaction is called as cross-domain) scheduling of analysis resources through cross-domain resource scheduling application, realize calculation resource integration, and improve the utilization rate of global calculation resources.
In a possible implementation manner, the cross-domain analysis system further includes: the central scheduling device is respectively connected with the first scheduling device and the second scheduling device, and the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device, and the method comprises the following steps: the second scheduling device receives a first cross-domain resource scheduling application through the central scheduling device, wherein the first cross-domain resource scheduling application is sent to the central scheduling device by the first scheduling device, so that the central scheduling device determines a second local point from other local points according to the first cross-domain resource scheduling application and sends the first cross-domain resource scheduling application to the second scheduling device; and the intra-office resource allowance of the second office point is larger than or equal to the size of the resource needed by the first task to be analyzed.
It can be understood that the cross-domain resource scheduling method provided by the present application is applied to a cross-domain analysis system, and the cross-domain analysis system further includes: a central scheduling device; the central dispatching device is respectively connected with the first dispatching device and the second dispatching device. One office point can send a cross-domain resource scheduling application (including a task to be analyzed) to the central scheduling device through the scheduling device of the office point, so that the central scheduling device determines the office point which can be used for executing the task to be analyzed from other office points according to the cross-domain resource scheduling application, and then the central scheduling device can send the cross-domain resource scheduling application to the scheduling device of the office point, the task to be analyzed is analyzed in the office point, and an analysis result is obtained. Therefore, by using the method provided by the application, on one hand, when a certain local point cannot bear the task to be analyzed, the task to be analyzed can be sent to other idle local points for analysis in time by initiating the cross-domain resource scheduling application, so that the task processing efficiency is improved, and the utilization rate of global computational power resources is improved; on the other hand, the local scheduling device may synchronize the resource information in the local point to the central scheduling device, and the central scheduling device makes a decision (i.e., determines the requested party of the cross-domain resource scheduling application), so that the configuration of the local scheduling device is lighter.
In another possible implementation manner, when the amount of resources required by the first task to be analyzed increases, the second scheduling device receives a first capacity expansion application through the central scheduling device, where the first capacity expansion application is sent by the first scheduling device to the central scheduling device, so that the central scheduling device sends the first capacity expansion application to the second scheduling device; the first capacity expansion application is used for applying for the increased resource quantity of the first task to be analyzed from the second local point.
It can be understood that, according to the cross-domain resource scheduling method provided by the present application, in the case that the amount of resources required by the first task to be analyzed increases, the increased amount of resources of the first task to be analyzed can be applied to the local point where the first task to be analyzed is executed again, so that the processing process of the first task to be analyzed is not interrupted, and the task analysis efficiency is improved.
In another possible implementation manner, the method further includes: the second scheduling device sends an algorithm query application to the second data analysis device, wherein the algorithm query application comprises: the identification of the algorithm required by the first task to be analyzed, so that the second data analysis device queries whether the algorithm required by the first task to be analyzed is included in the second data analysis device or not according to the identification of the algorithm required by the first task to be analyzed; under the condition that the algorithm required by the first task to be analyzed is not included in the second data analysis device, the second scheduling device sends an algorithm downloading application to the second data analysis device; the algorithm downloading application comprises the following steps: and the second data analysis device downloads the algorithm required by the first task to be analyzed according to the download address of the algorithm required by the first task to be analyzed.
It can be understood that, based on the method provided in the present application, before the second data analysis device analyzes the task to be analyzed, the second scheduling device sends an algorithm query application to the second data analysis device to query whether the second data analysis device has the algorithm required by the task to be analyzed, and in a case that the second data analysis device does not have the algorithm required by the task to be analyzed, sends an identifier of the algorithm required by the task to be analyzed to the second data analysis device, so that the second data analysis device downloads the algorithm required by the task to be analyzed according to the identifier of the algorithm required by the task to be analyzed. Therefore, before the task to be analyzed is analyzed, the second data analysis device is ensured to have the related algorithm, and the task to be analyzed can be normally analyzed.
In another possible implementation manner, the method further includes: and the second scheduling device receives the control instruction sent by the first scheduling device and controls the second data analysis device to analyze the state of the first task to be analyzed according to the control instruction.
It can be understood that, based on the method provided by the present application, the processing state of the task to be analyzed, which is analyzed by the second data analysis device, can be obtained in real time by the first scheduling device, and the state is controlled. Therefore, when the upper-layer application modifies the task to be analyzed, corresponding modification operation can be timely carried out through the first scheduling device.
In another possible implementation manner, the method further includes: the second scheduling device receives the algorithm change information sent by the first scheduling device; the algorithm change information comprises an identification of a target algorithm which is changed in the first data analysis device and a download address of the target algorithm; the second scheduling device sends the identification of the target algorithm to the second data analysis device, so that the second data analysis device inquires whether the target algorithm exists in the second data analysis device according to the identification of the target algorithm; and under the condition that the target algorithm does not exist in the second data analysis device, the second scheduling device sends the download address of the target algorithm to the second data analysis device, so that the second data analysis device downloads the target algorithm according to the download address of the target algorithm.
It can be understood that cross-domain resource scheduling can be performed only when the algorithm information in each local point in the cross-domain analysis system is consistent. For example, if there are no resources required by the task to be analyzed in the resource applying party scheduled by the cross-domain resource, the task to be analyzed cannot be processed. Therefore, based on the method provided by the application, algorithm synchronization can be performed regularly through the scheduling device, so that the requested party of the resource can process the task to be analyzed in time in cross-domain resource scheduling.
In another possible implementation manner, the method further includes: the second scheduling device receives the calculation resource change information sent by the first scheduling device; the calculation force resource change information comprises: current remaining calculation power information of the first local point; and the second scheduling device changes the residual calculation power information of the first local point recorded in the second scheduling device into the current residual calculation power information of the first local point according to the calculation power resource change information.
It can be understood that, since the method provided by the present application needs to determine the requested party of the cross-domain resource scheduling application according to the intra-office resource allowance of each office point in the cross-domain analysis system, the intra-office resource allowance of each office point recorded in the scheduling device needs to be periodically updated (for example, computational resource synchronization is periodically performed), so that the requested party of the cross-domain resource scheduling application can be ensured to have the capability of processing the task to be analyzed.
In another possible implementation, in the case where no direct communication is possible between the first local point and the second local point; the receiving, by the second scheduling device, the first cross-domain resource scheduling application sent by the first scheduling device includes: and the second scheduling device receives the first cross-domain resource scheduling application sent by the first scheduling device through the boundary security server.
It can be understood that, based on the method provided by the present application, information can be transmitted through the border security server under the condition that the first local area network and the second local area network cannot be directly communicated with each other (for example, an internal local area network of a certain organization cannot be directly connected to a private video network), so that the method provided by the present application can be applied to richer use scenarios.
In a second aspect, the present application provides a cross-domain resource scheduling method, which is applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point, a second office point, and a third office point; wherein the first local point comprises: the second local point comprises: the third local point comprises a second data analysis device and a second scheduling device: a third scheduling device and a third data analysis device; the method comprises the following steps: the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying for executing a second task to be analyzed at other local points except the second local point; the second scheduling device determines a third local point from other local points according to a second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes a second task to be analyzed; and the intra-office resource allowance of the third office point is larger than or equal to the size of the resource required by the second task to be analyzed.
It can be understood that the cross-domain resource scheduling method provided by the present application is applied to a cross-domain analysis system, where the cross-domain analysis system includes a plurality of local points (one local point may be regarded as one video image intelligent analysis platform), and each local point includes a scheduling device. One office point can send a cross-domain resource scheduling application (including a task to be analyzed) to scheduling devices of other office points through the scheduling device of the office point, so that data analysis devices of other office points analyze the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, by using the method provided by the application, when a certain office point cannot bear the task to be analyzed, the task to be analyzed can be timely sent to other idle office points for analysis by initiating the cross-domain resource scheduling application, so that the task to be analyzed can be timely processed under the condition of ensuring that the normal service of the office point is not interrupted. Therefore, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle times of different local points can be solved, and the calculation resource utilization rate of the universe is improved.
In a possible implementation manner, the second scheduling device sends a second capacity expansion application to the third scheduling device when the amount of resources required by the second task to be analyzed is increased; the second capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point.
It can be understood that, according to the cross-domain resource scheduling method provided by the present application, in the case that the amount of resources required by the second task to be analyzed is increased, the increased amount of resources of the second task to be analyzed can be applied to the local point where the second task to be analyzed is executed again, so that the processing process of the second task to be analyzed is not interrupted, and the task analysis efficiency is improved.
In another possible implementation manner, the cross-domain analysis system further includes: a terminal device; the determining a third office point from other office points according to the second cross-domain resource scheduling application includes: and the second scheduling device sends a second cross-domain resource scheduling application to the terminal equipment, so that the terminal equipment receives the second cross-domain resource scheduling application and receives a third office point indicated by the user as the second cross-domain resource scheduling application.
It can be understood that, based on the method provided by the present application, a cross-domain resource scheduling application may be sent to a terminal device through a scheduling apparatus, and then the terminal device may present the cross-domain resource scheduling application to a user, so that the user may indicate a resource-applied party for the cross-domain resource scheduling application according to the content of the cross-domain resource scheduling application. Therefore, the decision-making right can be provided for the user through the terminal equipment, so that the method provided by the application can be suitable for richer use scenes.
In a third aspect, the present application provides a cross-domain resource scheduling method, which is applied to a cross-domain analysis system, where the cross-domain analysis system includes: a second office point, a third office point and a central scheduling device; wherein the second local point comprises: the third local point comprises a second data analysis device and a second scheduling device: a third scheduling device and a third data analysis device; the central scheduling device is respectively connected with the second scheduling device and the third scheduling device; the method further comprises the following steps: the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying other local points except the second local point to execute a second task to be analyzed; the second scheduling device sends a second cross-domain resource scheduling application to the central scheduling device, so that the central scheduling device determines a third local point from other local points according to the second cross-domain resource scheduling application, and sends the cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes a second task to be analyzed; the intra-office resource allowance of the third office point is larger than or equal to the size of the resource needed by the second task to be analyzed; under the condition that the resource quantity required by the second task to be analyzed is increased, the second scheduling device sends a capacity expansion application to the central scheduling device, so that the central scheduling device sends the capacity expansion application to the third scheduling device; the capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point.
In a fourth aspect, the present application provides a cross-domain resource scheduling method, which is applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point, a second office point, and a fourth office point. Wherein the first local point comprises: the second office point comprises: the fourth office point comprises a second data analysis device and a second scheduling device: a fourth scheduling device; the method further comprises the following steps: the second scheduling device receives a task analysis request sent by the fourth scheduling device, the task analysis request is used for applying for the second scheduling device to execute a task to be analyzed in the second office point in the fourth office point, and the task analysis request includes: address information of a fourth office point and an intra-office resource margin of the fourth office point.
It can be understood that the cross-domain resource scheduling method provided by the present application is applied to a cross-domain analysis system, where the cross-domain analysis system includes multiple local points (one local point may be regarded as one video image intelligent analysis platform), and each local point includes a scheduling device. One office point can send a task analysis request to the scheduling devices of other office points through the scheduling device of the office point so as to request to execute the tasks to be analyzed of the other office points at the office point. Therefore, by using the method provided by the application, on one hand, when the idle resources of a certain local point are more, the task to be analyzed can be actively requested to other local points, so that the task to be analyzed can be processed in time. Therefore, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle states of different local points can be solved, and the global computational resource utilization rate is improved.
In a fifth aspect, the present application provides a cross-domain resource scheduling method, which is applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the second local point comprises: the second data analysis device and the second scheduling device; the method comprises the following steps: a second data analysis device receives a first cross-domain resource scheduling application sent by a second scheduling device; the first cross-domain resource scheduling application comprises: a first task to be analyzed; the first cross-domain resource scheduling application is used for applying other local points except the first local point to execute a first task to be analyzed; the first cross-domain resource scheduling application is initiated by the first data analysis device and is forwarded to the second scheduling device through the first scheduling device; and the second data analysis device analyzes the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
It can be understood that the cross-domain resource scheduling method provided by the present application is applied to a cross-domain analysis system, where the cross-domain analysis system includes a plurality of local points (one local point may be regarded as one video image intelligent analysis platform), and each local point includes a scheduling device. The data analysis device of one office point can receive a cross-domain resource scheduling application (including a task to be analyzed) initiated by the data analysis device of other office points through the scheduling device of the office point, and analyze the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, the cross-domain resource scheduling method and the cross-domain resource scheduling device can perform cross-domain (cross-local point information interaction is called as cross-domain) scheduling of analysis resources through cross-domain resource scheduling application, realize calculation resource integration, and improve the utilization rate of global calculation resources.
In a possible implementation manner, the first task to be analyzed is a task for analyzing data to be analyzed; the first cross-domain resource scheduling application further includes: obtaining an address of data to be analyzed; the method further comprises the following steps: the second data analysis device acquires the data to be analyzed according to the acquisition address of the data to be analyzed; the first cross-domain resource scheduling application further includes: downloading addresses of algorithms required by a first task to be analyzed; the method further comprises the following steps: the second data analysis device directly downloads the algorithm required by the first task to be analyzed from the first data analysis device according to the download address of the algorithm required by the first task to be analyzed; in another possible implementation manner, the analyzing, by the second data analysis device, the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result, including: and the second data analysis device analyzes the first task to be analyzed according to the data to be analyzed and the algorithm required by the first task to be analyzed contained in the algorithm download application, and an analysis result is obtained.
It can be understood that, based on the method provided in the present application, the data to be analyzed (e.g., video data or picture data, etc.) can be directly obtained according to the obtaining address of the data to be analyzed; and acquiring the algorithm required by the data to be analyzed according to the identifier of the algorithm required by the data to be analyzed, namely the second data analysis device can directly acquire the service data without the need of forwarding by a scheduling device, and the scheduling device only needs to be responsible for forwarding the task control data.
Therefore, by adopting the mode of separately forwarding the task control data and the service data, the configuration of the scheduling device is lighter, and meanwhile, the data forwarding efficiency can also be improved.
In another possible implementation manner, the method further includes: the second data analysis device receives an algorithm query application sent by the second scheduling device; the algorithm inquiry application comprises the following steps: identification of an algorithm required by a first task to be analyzed; the second data analysis device inquires whether the second data analysis device comprises the algorithm required by the first task to be analyzed according to the identifier of the algorithm required by the first task to be analyzed; when the algorithm required by the first task to be analyzed is not included in the second data analysis device, the second data analysis device receives an algorithm downloading application sent by the second scheduling device; the algorithm downloading application comprises the following steps: downloading addresses of algorithms required by a first task to be analyzed; and the second data analysis device downloads the algorithm required by the first task to be analyzed according to the download address of the algorithm required by the first task to be analyzed.
In a sixth aspect, the present application provides a scheduling apparatus, which is applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the second office point comprises: a second data analysis device and the scheduling device; the scheduling device includes: a receiving unit, configured to receive a first cross-domain resource scheduling application sent by a first scheduling apparatus; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; and the sending unit is used for sending the first cross-domain resource scheduling application to the second data analysis device so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
In one possible implementation manner, the cross-domain analysis system further includes: the central scheduling device is respectively connected with the first scheduling device and the scheduling device, and the receiving unit is specifically used for receiving a first cross-domain resource scheduling application through the central scheduling device, wherein the first cross-domain resource scheduling application is sent to the central scheduling device by the first scheduling device, so that the central scheduling device determines a second local point from other local points according to the first cross-domain resource scheduling application and sends the first cross-domain resource scheduling application to the scheduling device; and the intra-office resource allowance of the second office point is larger than or equal to the size of the resource required by the first task to be analyzed.
In another possible implementation manner, the receiving unit is further configured to receive, by the central scheduling apparatus, a first capacity expansion application when the amount of resources required by the first task to be analyzed is increased, where the first capacity expansion application is sent by the first scheduling apparatus to the central scheduling apparatus, so that the central scheduling apparatus sends the first capacity expansion application to the second scheduling apparatus; the first capacity expansion application is used for applying for the increased resource quantity of the first task to be analyzed from the second local point.
In another possible implementation manner, the cross-domain analysis system further includes: a third office, the third office comprising: the receiving unit is also used for receiving a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying for executing a second task to be analyzed at other local points except the second local point; the scheduling device further comprises: a determining unit, configured to determine a third office point from the other office points according to the second cross-domain resource scheduling application; the sending unit is further configured to send a second cross-domain resource scheduling application to a third scheduling device, so that the third data analysis device executes a second task to be analyzed; and the intra-office resource allowance of the third office point is larger than or equal to the size of the resource required by the second task to be analyzed.
In another possible implementation manner, the sending unit is further configured to send a second capacity expansion application to a third scheduling apparatus when the amount of resources required by the second task to be analyzed is increased; the second capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point.
In another possible implementation manner, the central scheduling apparatus is further connected to a third scheduling apparatus, and the sending unit is further configured to send a second cross-domain resource scheduling application to the central scheduling apparatus, so that the central scheduling apparatus determines a third office point from other office points according to the second cross-domain resource scheduling application, and sends the cross-domain resource scheduling application to the third scheduling apparatus, so that the third data analysis apparatus executes a second task to be analyzed; the intra-office resource allowance of the third office point is larger than or equal to the size of the resource needed by the second task to be analyzed; the sending unit is further configured to send a second capacity expansion application to the central scheduling device under the condition that the amount of resources required by the second task to be analyzed is increased, so that the central scheduling device sends the second capacity expansion application to the third scheduling device; the second capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point.
In another possible implementation manner, the cross-domain analysis system further includes: a fourth office point, the fourth office point comprising: a fourth scheduling device; a receiving unit; the task analysis request is used for applying for the scheduling device to execute a task to be analyzed in the second office point in the fourth office point, and the task analysis request includes: address information of a fourth office point and an intra-office resource margin of the fourth office point.
In another possible implementation manner, the sending unit is further configured to send an algorithm query application to the second data analysis device, where the algorithm query application includes: the identification of the algorithm required by the first task to be analyzed, so that the second data analysis device queries whether the algorithm required by the first task to be analyzed is included in the second data analysis device or not according to the identification of the algorithm required by the first task to be analyzed; the sending unit is also used for sending an algorithm downloading application to the second data analysis device under the condition that the algorithm required by the first task to be analyzed is not included in the second data analysis device; the algorithm downloading application comprises the following steps: and the second data analysis device downloads the algorithm required by the first task to be analyzed according to the download address of the algorithm required by the first task to be analyzed.
In another possible implementation manner, the receiving unit is further configured to receive a control instruction sent by the first scheduling device, and control, according to the control instruction, the second data analysis device to analyze the state of the first task to be analyzed.
In another possible implementation manner, the receiving unit is further configured to receive algorithm change information sent by the first scheduling apparatus; the algorithm change information comprises an identification of a target algorithm which is changed in the first data analysis device and a download address of the target algorithm; the sending unit is further used for sending the identification of the target algorithm to the second data analysis device so that the second data analysis device can inquire whether the target algorithm exists in the second data analysis device or not according to the identification of the target algorithm; and the sending unit is also used for sending the download address of the target algorithm to the second data analysis device under the condition that the target algorithm does not exist in the second data analysis device, so that the second data analysis device downloads the target algorithm according to the download address of the target algorithm.
In another possible implementation manner, the receiving unit is further configured to receive the calculation power resource change information sent by the first scheduling apparatus; the calculation force resource change information comprises: current remaining calculation power information of the first local point; the scheduling device further comprises: and an information changing unit configured to change the remaining computation power information of the first local point described in the scheduling apparatus to the current remaining computation power information of the first local point, based on the computation power resource change information.
In another possible implementation manner, the cross-domain analysis system further includes: a terminal device; the sending unit is further configured to send a second cross-domain resource scheduling application to the terminal device, so that the terminal device receives the second cross-domain resource scheduling application, and receives a third office point indicated by the user as the second cross-domain resource scheduling application.
In another possible implementation manner, the receiving unit is further configured to receive, by the border security server, the first cross-domain resource scheduling application sent by the first scheduling apparatus when direct communication cannot be performed between the first local point and the second local point.
In a seventh aspect, the present application provides a data analysis apparatus applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the second office point comprises: the data analysis device and the second scheduling device; the data analysis device includes: a receiving unit, configured to receive a first cross-domain resource scheduling application sent by a second scheduling apparatus; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is initiated by the first data analysis device and is forwarded to the second scheduling device through the first scheduling device; and the processing unit is used for analyzing the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
In a possible implementation manner, the first task to be analyzed is a task for analyzing data to be analyzed; the first cross-domain resource scheduling application further includes: obtaining an address of data to be analyzed; the data analysis device further includes: the acquisition unit is used for acquiring the data to be analyzed according to the acquisition address of the data to be analyzed; the first cross-domain resource scheduling application further includes: a downloading address of an algorithm required by a first task to be analyzed; the acquisition unit is also used for directly downloading the algorithm required by the first task to be analyzed from the first data analysis device according to the downloading address of the algorithm required by the first task to be analyzed; and the processing unit is specifically used for downloading the algorithm required by the first task to be analyzed contained in the application according to the data to be analyzed and the algorithm, analyzing the first task to be analyzed and obtaining an analysis result.
In another possible implementation manner, the receiving unit is further configured to receive an algorithm query application sent by the second scheduling apparatus; the algorithm inquiry application comprises the following steps: identification of an algorithm required by a first task to be analyzed; the data analysis device further includes: the query unit is used for querying whether the data analysis device comprises the algorithm required by the first task to be analyzed or not according to the identifier of the algorithm required by the first task to be analyzed; the receiving unit is also used for receiving an algorithm downloading application sent by the second scheduling device when the algorithm required by the first task to be analyzed is not included in the data analysis device; the algorithm downloading application comprises the following steps: a downloading address of an algorithm required by a first task to be analyzed; and the acquisition unit is also used for downloading the algorithm required by the first task to be analyzed according to the download address of the algorithm required by the first task to be analyzed.
In an eighth aspect, the present application provides a local point, comprising: scheduling means and data analysis means; the scheduling device comprises a first memory and a first processor; a first memory coupled to the first processor; the first memory is for storing computer program code, the computer program code comprising computer instructions; wherein the computer instructions, when executed by the first processor, cause the scheduling apparatus to perform a method as provided in the first, second, third or fourth aspect and any of its possible designs; the data analysis device comprises a second memory and a second processor; a second memory coupled to the second processor; the second memory is for storing computer program code, the computer program code comprising computer instructions; wherein the computer instructions, when executed by the second processor, cause the data analysis apparatus to perform the method as provided in the fifth aspect and any one of its possible design forms.
In a ninth aspect, the present application provides a cross-domain analysis system, comprising: a first office point and a second office point; wherein the first local point comprises: the second local point comprises: a second data analysis device and a second scheduling device; the first scheduling device is used for receiving a cross-domain resource scheduling application of the first data analysis device; the cross-domain resource scheduling application comprises the following steps: a task to be analyzed; a cross-domain resource scheduling application for applying for executing the task to be analyzed at other local points except the first local point; the first scheduling device is also used for forwarding the cross-domain resource scheduling application to the second scheduling device; the second scheduling device is used for forwarding the cross-domain resource scheduling application to the second data analysis device; and the second data analysis device is used for analyzing the tasks to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result.
In a possible implementation manner, the cross-domain resource scheduling application further includes: the size of the resource needed by the task to be analyzed; the first scheduling device is further configured to determine a second office point from the other office points according to the cross-domain resource scheduling application.
And the intra-office resource allowance of the second office point is larger than or equal to the size of the resource needed by the task to be analyzed.
In another possible implementation manner, the system further includes: a central scheduling device; the central dispatching device is respectively connected with the first dispatching device and the second dispatching device; the cross-domain resource scheduling application further comprises: the size of the resources required by the task to be analyzed; the first scheduling device is also used for forwarding the cross-domain resource scheduling application to the central scheduling device; the central scheduling device is used for determining a second office point from other office points according to the cross-domain resource scheduling application, and the intra-office resource allowance of the second office point is larger than or equal to the size of the resource needed by the task to be analyzed; and the central scheduling device is also used for sending a cross-domain resource scheduling application to the second scheduling device.
In another possible implementation manner, the first data analysis device is further configured to: when the size of the resource needed by the task to be analyzed is detected to be larger than or equal to a first threshold value, sending a cross-domain resource scheduling application to a first scheduling device; or when detecting that the computing resources in the first local point are in an off-line state, sending a cross-domain resource scheduling application to the first scheduling device.
In another possible implementation manner, the task to be analyzed is a task for analyzing data to be analyzed; the cross-domain resource scheduling application further comprises: obtaining an address of data to be analyzed; the second data analysis device is also used for acquiring an address according to the data to be analyzed and acquiring the data to be analyzed; and/or, the cross-domain resource scheduling application further comprises: downloading addresses of algorithms required by the tasks to be analyzed; the second data analysis device is also used for downloading the algorithm required by the task to be analyzed according to the downloading address of the algorithm required by the task to be analyzed; and the second data analysis device is specifically used for analyzing the task to be analyzed according to the data to be analyzed and the algorithm required by the task to be analyzed to obtain an analysis result.
In another possible implementation manner, the second scheduling device is further configured to send an algorithm query application to the second data analysis device; the algorithm query application is used for enabling the second data analysis device to query whether an algorithm required by a task to be analyzed exists; the second data analysis device is also used for carrying out algorithm query according to the algorithm query application and sending a query result to the second scheduling device; the second scheduling device is also used for sending an algorithm downloading application to the second data analysis device when the query result is that the algorithm required by the task to be analyzed is not contained in the second data analysis device; the algorithm downloading application comprises a downloading address of an algorithm required by the task to be analyzed; and the second data analysis device is specifically used for directly downloading the algorithm required by the task to be analyzed from the first data analysis device according to the download address of the algorithm required by the task to be analyzed included in the algorithm download application.
In another possible implementation manner, the cross-domain resource scheduling application further includes: a storage address of the analysis result; and the second data analysis device is also used for storing the analysis result in the storage space indicated by the analysis result storage address.
In another possible implementation manner, the first scheduling apparatus is further configured to send a control instruction to a second scheduling apparatus; and the second scheduling device is also used for controlling the processing state of the task to be analyzed by the second data analysis device according to the control instruction.
In another possible implementation manner, the first data analysis device is further configured to send algorithm change information to the first scheduling device; the algorithm change information comprises a download address of a changed target algorithm in the first data analysis device and an identification of the target algorithm; the first scheduling device is also used for forwarding the algorithm change information to the second scheduling device; the second scheduling device is also used for sending the identification of the target algorithm and the download address of the target algorithm to the second data analysis device; the second data analysis device is further used for inquiring whether the target algorithm exists in the second data analysis device according to the identification of the target algorithm, and downloading the target algorithm according to the downloading address of the target algorithm under the condition that the target algorithm does not exist in the second data analysis device.
In another possible implementation manner, the first data analysis device is further configured to send computational resource change information to the first scheduling device; the calculation power resource change information comprises: current remaining calculation force information of the first local point; the first scheduling device is also used for forwarding the calculation resource change information to the second scheduling device; and the second scheduling device is further used for changing the residual calculation force information of the first local point recorded in the second scheduling device into the current residual calculation force information of the first local point according to the calculation force resource change information.
In a tenth aspect, the present application provides a computer readable storage medium comprising computer instructions. Wherein the computer instructions, when executed on a computer, cause the computer to perform the method as provided by the first, second, third, fourth or fifth aspect and any of its possible designs.
In an eleventh aspect, the present application provides a computer program product comprising computer instructions. The computer instructions, when executed on a computer, cause the computer to perform the method as provided by the first, second, third, fourth or fifth aspect and any of its possible designs.
For a detailed description of the fifth to eleventh aspects and their various implementations in this application, reference may be made to the detailed description of the first, second, third, fourth aspects and their various implementations. For the beneficial effects of the fifth aspect to the twelfth aspect and the various implementation manners thereof, reference may be made to beneficial effect analysis of the first aspect, the second aspect, the third aspect, the fourth aspect and the various implementation manners thereof, which is not described herein again.
These and other aspects of the present application will be more readily apparent from the following description.
Drawings
Fig. 1 is a schematic structural diagram of a local point according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a cross-domain analysis system provided in the embodiment of the present application;
fig. 5 is a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present application;
fig. 6 is a flowchart illustrating a registration process of a data analysis apparatus according to an embodiment of the present application;
fig. 7 is a flowchart of a registration process of a scheduling apparatus according to an embodiment of the present application;
FIG. 8 is a first flowchart of algorithm synchronization provided in an embodiment of the present application;
FIG. 9 is a second flowchart of algorithm synchronization provided in the embodiments of the present application;
FIG. 10 is a first flowchart of computing resource synchronization according to an embodiment of the present disclosure;
fig. 11 is a second flowchart of computing power resource synchronization according to an embodiment of the present application;
fig. 12 is a flowchart of a cross-domain resource scheduling method according to an embodiment of the present application;
fig. 13 is a first schematic view of a display interface of a terminal device according to an embodiment of the present application;
fig. 14 is a second schematic view of a display interface of a terminal device according to an embodiment of the present application;
fig. 15 is a third schematic view of a display interface of a terminal device according to an embodiment of the present application;
fig. 16 is a first schematic diagram illustrating a data forwarding mode according to an embodiment of the present application;
fig. 17 is a second schematic diagram of a data forwarding mode according to an embodiment of the present application;
fig. 18 is a flowchart of a cross-domain resource scheduling method according to an embodiment of the present application;
fig. 19 is a flowchart of a cross-domain resource scheduling method provided in the embodiment of the present application;
fig. 20 is a flowchart of a cross-domain resource scheduling method according to a fourth embodiment of the present disclosure;
fig. 21 is a flowchart of a cross-domain resource scheduling method according to an embodiment of the present application;
fig. 22 is a schematic structural diagram of a scheduling apparatus according to an embodiment of the present application;
fig. 23 is a schematic structural diagram of a data analysis apparatus according to an embodiment of the present application;
fig. 24 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone.
The terms "first" and "second" and the like in the specification and drawings of the present application are used for distinguishing different objects or for distinguishing different processes for the same object, and are not used for describing a specific order of the objects.
Furthermore, the terms "including" and "having," and any variations thereof, as referred to in the description of the present application, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements recited, but may alternatively include other steps or elements not recited, or may alternatively include other steps or elements inherent to such process, method, article, or apparatus.
It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present relevant concepts in a concrete fashion.
In the description of the present application, the meaning of "a plurality" means two or more unless otherwise specified.
At present, in the technical field of intelligent analysis of video images, a main working mode of an analysis task is to schedule resources in a single intelligent video image analysis system, and the following problems can be caused by such working mode: when the resources in the office of the office are insufficient, if a new analysis task occurs, the new analysis task cannot be responded in time, and can only be queued for waiting or discarded, which can cause problems of untimely task processing and low processing efficiency. In addition, if a plurality of intelligent video image analysis systems exist at the same time, because the intelligent video image analysis systems are isolated from one another, the busy and idle states of the local points are uneven, and the utilization rate of the whole resources of the intelligent video image analysis systems is low.
To solve the problem, an embodiment of the present application provides a cross-domain analysis system, which includes a plurality of office points (one office point may be regarded as an intelligent video image analysis platform), each office point includes a scheduling device, and one office point may initiate a cross-domain resource scheduling application (including a task to be analyzed) to another office point through the scheduling device; and the corresponding other local point can receive the cross-domain resource scheduling application through the scheduling device, and analyze the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, the system provided by the embodiment of the application does not need to modify each local point in a large scale, and can realize cross-domain scheduling of analysis resources (cross-domain information interaction of the local points is called here cross-domain) by introducing the scheduling device on the basis of the original data analysis device, so that the cross-domain scheduling of analysis resources can be realized, the integration of computational resources is realized, and the utilization rate of global computational resources is improved.
In addition, the cross-domain analysis system provided by the embodiment of the application can timely send the task to be analyzed to other idle local points for analysis when a certain local point cannot bear the task to be analyzed, so that the task to be analyzed can be timely processed under the condition that normal service of the local point is not interrupted. Therefore, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle states of different local points can be solved, and the global computational resource utilization rate is improved.
For example, if there are 40 chips in west lake region, 84 chips in coastal river region, and 56 chips in arcade region in Hangzhou city, the resources in each region are not balanced. If an emergency happens in the west lake region, the number of video monitoring points needing to be guaranteed is large, resources are difficult to avoid and insufficient, and if equipment is temporarily purchased in the west lake region, not only is time consumed, but also resource waste is caused after the event processing is completed. Therefore, by using the technical scheme provided by the embodiment of the application, the west lake region local point can initiate a cross-domain resource scheduling application to the coastal river region local point and/or the arch villa region local point, and temporarily borrow resources of the coastal river region local point and/or resources of the arch villa region local point.
Embodiments of the present application will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present application provides a local point (i.e., a video image intelligent analysis platform), including: data analysis device and scheduling device. The data analysis device may be registered in the scheduling device at the present local site.
And the data analysis device is used for receiving an analysis task (such as a video image intelligent analysis task) and distributing corresponding computing resources and algorithms for the analysis task. Meanwhile, the data analysis device is also used for initiating a cross-domain resource scheduling application to the scheduling device of the local site when the local site can not bear the analysis task. Illustratively, the data analysis device may be a video image intelligent analysis system.
And the scheduling device is used for receiving the cross-domain resource scheduling application sent by the data analysis device of the local point, determining an applied party (other local points except the local point) of the cross-domain resource scheduling application, and sending the cross-domain resource scheduling application to the scheduling device of the applied party.
Meanwhile, the scheduling device is also used for receiving a cross-domain resource scheduling application initiated by the data analysis device of other local points except the local point (the cross-domain resource scheduling application is forwarded to the local point by the scheduling devices of other local points or forwarded to the local point by the central scheduling device), and forwarding the cross-domain resource scheduling application to the data analysis device of the local point.
In some embodiments, the scheduling means may be physically one network node; alternatively, the scheduling means may be a logically one component (or functional module). The scheduler may be an AI gateway, for example.
In some embodiments, the scheduling means and the data analysis means may be two separate devices; alternatively, the scheduling means and the first data analysis means may be integrated.
On the basis of the local point as shown in fig. 1, an embodiment of the present application further provides a cross-domain analysis system, where the cross-domain analysis system includes a plurality of local points. The data analysis device of each local point is registered to the dispatching device of the local point, and meanwhile, the dispatching devices of each local point are mutually registered; or, the dispatching device of each local point registers in the central dispatching device to construct a logically uniform large-capacity resource POOL (POOL). Therefore, resource scheduling can be realized among the registered local points, and the purpose of resource sharing is achieved.
It can be understood that the cross-domain resource scheduling referred to in the embodiments of the present application refers to: and performing cross-office resource scheduling, namely performing resource scheduling among different office points.
Optionally, the deployment mode of the cross-domain analysis system provided in the embodiment of the present application includes: a fully symmetric deployment mode and a central deployment mode.
Please refer to fig. 2, which illustrates a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present application. As shown in fig. 2, the cross-domain analysis system is in a fully symmetric deployment mode, and includes: a first local point 100 and a second local point 200. Wherein, the first office point 100 includes: a first data analysis means 110 and a first scheduling means 120; the second office 200 includes: a second data analysis means 210 and a second scheduling means 220.
It can be understood that the fully symmetric deployment mode of the cross-domain analysis system refers to that one scheduling device is deployed in each local point, and the functions of each scheduling device are consistent. For example, the first scheduling device 120 is consistent with the second scheduling device 220.
When the cross-domain analysis system is in a fully-symmetric deployment mode, the scheduling device of each office point is used for receiving a cross-domain resource scheduling application sent by the data analysis device of the office point, determining an applied party (other office points except the office point) of the cross-domain resource scheduling application, and sending the cross-domain resource scheduling application to the scheduling device of the applied party. Meanwhile, the scheduling device of each office point is also used for receiving the cross-domain resource scheduling application sent by the scheduling devices of other office points except the office point, and forwarding the cross-domain resource scheduling application to the data analysis device of the office point.
For example, the first scheduling device 120 is configured to receive the cross-domain resource scheduling application sent by the first data analysis device 110, determine that the requested party of the cross-domain resource scheduling application is the second office point 200, and send the cross-domain resource scheduling application to the second scheduling device 220.
For another example, the second scheduling device 220 is configured to receive the cross-domain resource scheduling application sent by the first scheduling device 120, and forward the cross-domain resource scheduling application to the second data analysis device 210.
In addition, connection is established between the scheduling devices of each local point, and bidirectional transmission of information such as algorithm identification, computing power state, tasks, control commands and the like can be carried out in real time. For example, in fig. 2, the first scheduling apparatus 120 and the second scheduling apparatus 220 may be connected via an IF3a interface (there may be one or more IF3a interfaces, such as IF3 a)1、IF3a2、IF3a3Etc., not shown in fig. 2) connected in a wired or wireless manner to enable information to be communicated in both directions.
Meanwhile, the data analysis device of each local point is connected with the scheduling device of the local point, and bidirectional transmission of information such as algorithm identification, computing power state, tasks, control instructions and the like can be carried out in real time. For example, in fig. 2, the first data analysis device 110 and the first scheduling device 120 may have one or more IF1a interfaces (IF1a interfaces, for example, IF1 a) therebetween1、IF1a2、IF1a3Etc., not shown in fig. 2), connected in a wired or wireless manner, so as to realize bidirectional information transmission; the second data analysis device 210 and the second scheduling device 220 may be connected via an IF2a interface (there may be one or more IF2a interfaces, for example, IF2 a)1、IF2a2、IF2a3Etc., not shown in fig. 2), connected in a wired or wireless manner, to realize bidirectional information transfer.
In some embodiments, in the case that the first local point 100 and the second local point 200 cannot communicate directly (for example, the local area network in a department cannot be connected directly to the private video network), as shown in fig. 3, a border security server 400 may be introduced, so that the first scheduling device 120 and the second scheduling device 220 are connected to the border security server 400 respectively.
Wherein the border security server 400 is used for information transmission between the first scheduling device 120 and the second scheduling device 220.
Alternatively, the border security server 400 may be a File Transfer Protocol (FTP) server. An FTP server is a computer that provides file storage and access services over the internet, and the FTP server provides services in accordance with the FTP protocol.
Specifically, the first scheduling device 120 may upload cross-domain scheduling information that needs to be sent to the second scheduling device 220 to the storage space of the border security server 400, and the second scheduling device 220 may view the cross-domain scheduling information in the storage space of the border security server 400.
Please refer to fig. 4, which illustrates a schematic structural diagram of another cross-domain analysis system provided in an embodiment of the present application. As shown in fig. 4, the cross-domain analysis system is a central deployment mode, which includes: a first local point 100, a second local point 200 and a central scheduler 300. Wherein, the first office point 100 includes: a first data analysis means 110 and a first scheduling means 120; the second office 200 includes: a second data analysis means 210 and a second scheduling means 220.
It can be understood that the central deployment mode of the cross-domain analysis system refers to that a scheduling device is deployed in each office point, and the function of each scheduling device is consistent, and meanwhile, a central scheduling device is deployed outside each office point and is respectively connected with the scheduling device of each office point, and the function of the central scheduling device is inconsistent with the function of the scheduling device deployed in each office point. For example, the first scheduling device 120 and the second scheduling device 220 are identical in function. The central scheduler 300 is logically disposed outside the first local point 100 and the second local point 200, and is connected to the first scheduler 120 and the second scheduler 220, respectively, and the central scheduler 300 does not have the same function as the first scheduler 120 and the second scheduler 220.
When the cross-domain analysis system is in a central deployment mode, the scheduling device of each local point is used for receiving a cross-domain resource scheduling application sent by the data analysis device of the local point and forwarding the cross-domain resource scheduling application to the central scheduling device. And the central scheduling device is used for receiving the cross-domain resource scheduling application sent by the scheduling device of each local point, determining the applied party of the cross-domain resource scheduling application, and sending the cross-domain resource scheduling application to the scheduling device of the applied party. Meanwhile, the scheduling device of each office point is also used for receiving the cross-domain resource scheduling application sent by the central scheduling device and forwarding the cross-domain resource scheduling application to the data analysis device of the office point.
For example, the first scheduling device 120 is configured to receive a cross-domain resource scheduling application sent by the first data analysis device 110, and forward the cross-domain resource scheduling application to the central scheduling device 300. The central scheduling device 300 is configured to receive the cross-domain resource scheduling application sent by the first scheduling device 120, determine that the party to be applied for the cross-domain resource scheduling application is the second office point 200, and send the cross-domain resource scheduling application to the second scheduling device 220.
For another example, the second scheduling device 220 is configured to receive a cross-domain resource scheduling application sent by the central scheduling device 300, and forward the cross-domain resource scheduling application to the second data analysis device 210.
In some embodiments, the central scheduling device is connected with the scheduling devices of the local points, so that the bidirectional transmission of information such as algorithm identification, calculation force state, tasks, control instructions and the like can be performed in real time. For example, in fig. 4, the central scheduler 300 and the first scheduler 120 may have one or more IF3b interfaces (IF3b interfaces, for example, IF3 b) between them1、IF3b2、IF3b3Etc., not shown in fig. 4), connected in a wired or wireless manner; the central scheduler 300 and the second scheduler 220 may be connected via an IF4b interface (there may be one or more IF4b interfaces, for example, IF4 b)1、IF4b2、IF4b3Etc., not shown in fig. 4) connected in a wired or wireless manner.
Meanwhile, the data analysis device of each local point is connected with the scheduling device of the local point, and bidirectional transmission of information such as algorithm identification, computing power state, tasks, control instructions and the like can be carried out in real time. For example, in fig. 4, the first data analysis apparatus 110 and the first scheduling apparatus 120 may have one or more IF1b interfaces (IF1b interfaces, for example, IF1 b) therebetween1、IF1b2、IF1b3Etc., not shown in fig. 4), connected in a wired or wireless manner; the second data analysis device 210 and the second scheduling device 220 may be connected via an IF2b interface (there may be one or more IF2b interfaces, for example, IF2 b)1、IF2b2、IF2b3Etc., not shown in fig. 4), connected in a wired or wireless manner.
In some embodiments, no direct communication is possible between the first local point 100 and the central scheduler 300; and/or, in case that the central scheduling device 300 and the second office point 200 cannot communicate directly, as shown in fig. 5, a border security server 400 may be introduced, such that the first scheduling device 120 and the central scheduling device 300 are connected to the border security server 400 respectively; and/or, the central scheduler 300 and the second scheduler 220 are respectively connected with the border security server 400.
Wherein, the border security server 400 is used for information transmission between the first scheduling device 120 and the central scheduling device 300; and/or, the border security server 400 is used for information transmission between the central scheduler 300 and the second scheduler 220.
Specifically, the first scheduling device 120 may upload cross-domain scheduling information that needs to be sent to the second scheduling device 220 to the storage space of the border security server 400, and the central scheduling device 300 may view the cross-domain scheduling information in the storage space of the border security server 400; and/or, the central scheduling apparatus 300 may upload cross-domain scheduling information that needs to be sent to the second scheduling apparatus 220 to the storage space of the border security server 400, and then the second scheduling apparatus 220 may view the cross-domain scheduling information in the storage space of the border security server 400.
In some embodiments, before performing cross-domain resource scheduling based on the cross-domain analysis system provided in the embodiments of the present application, the following operation procedures need to be performed: a registration process, an algorithm synchronization process and an information synchronization process of computing resources.
1. Registration process
The registration process comprises the following steps: registration of the data analysis device, registration of the scheduling device, and registration of the border security server.
(1) Registration of data analysis devices
The registration of the data analysis device means that each data analysis device needs to be registered in the scheduling device at the local site. For example, a first data analysis device needs to be registered with a first scheduling device, and a second data analysis device needs to be registered with a second scheduling device.
Optionally, when the data analysis apparatus registers to the scheduling apparatus, the content to be filled includes at least one of the following: internet Protocol (IP) addresses, port numbers, etc.
It can be understood that after the registration is completed, the data analysis device and the scheduling device in the local site establish connection communication, and can transmit information such as algorithms, computational resources, tasks, states and the like.
For example, taking the first data analysis device as an example to register in the first scheduling device, as shown in fig. 6, the registration process includes:
sa1, the first scheduling device receives a registration request initiated by the first data analysis device.
Wherein, this registration application includes: the IP address and port number of the first data analysis device.
Sa2, the first scheduling apparatus determines whether the content of the registration request meets the standard.
In some embodiments, in a case that the content of the registration application does not meet the standard, the following step Sa3 is executed; if the content of the registration application conforms to the standard, the following step Sa4 is executed.
Optionally, the first scheduling device may adopt a manual review or automatic review mode to determine whether the content of the registration application meets the standard.
Sa3, the first scheduling device refutes the registration request.
In some embodiments, after the first scheduling device refutes the registration application, the first data analysis device may modify the registration application and re-initiate the registration application until the audit is passed; alternatively, the first data analysis means may cancel the registration request.
Sa4 requests for registration by the first scheduling apparatus.
Thus, the first data analysis device is successfully registered to the first scheduling device.
(2) Registration of scheduling device
The cross-domain analysis systems in different deployment modes and the registration modes of the scheduling devices are different.
For example, when the cross-domain analysis system is in the fully-symmetric deployment mode as shown in fig. 2, the registration of the scheduling apparatus refers to: the scheduling device of each office point needs to be registered in the scheduling devices of other office points except the office point. For example, the first scheduling apparatus needs to be registered with the second scheduling apparatus, and the second scheduling apparatus needs to be registered with the first scheduling apparatus.
For example, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the registration of the scheduling apparatus refers to: the scheduler at each office needs to register with the central scheduler. For example, a first scheduler needs to register with a central scheduler, and a second scheduler needs to register with the central scheduler.
Optionally, when the scheduling device of each office point performs registration, the content to be filled includes at least one of the following: the name of the local point, the Internet Protocol (IP) address, the port number, etc.
It can be understood that after the registration is completed, connection communication is established between the dispatching devices of the respective office points (or between the central dispatching devices and the respective office points), and information such as algorithm identification, computing power status, tasks, control instructions, etc. can be transferred.
For example, taking the first scheduling apparatus as registering in the second scheduling apparatus, as shown in fig. 7, the registration process includes:
sb1, the second scheduling apparatus receives a registration application initiated by the first scheduling apparatus.
Wherein, this registration application includes: the name of the first office point, the IP address, port number and Kafka address of the first scheduling device.
Sb2, the second scheduling apparatus determines whether the content of the registration application meets the standard.
In some embodiments, in a case that the content of the registration application does not meet the standard, the following step Sa3 is performed; if the contents of the registration application meet the standard, the following step Sa4 is executed.
Optionally, the second scheduling device may adopt a manual review or automatic review mode to determine whether the content of the registration application meets the standard.
Sb3, the second scheduler refutes the registration application.
In some embodiments, after the second scheduling device refutes the registration application, the first scheduling device may modify the registration application and re-initiate the registration application until the audit is passed; alternatively, the first scheduling apparatus may cancel the registration request.
Sb4, the second scheduling apparatus requests for registration.
Thus, the first scheduling device is successfully registered to the second scheduling device.
In some embodiments, the process of registering in the second scheduling apparatus by the first scheduling apparatus is the same as the process of registering in the second scheduling apparatus by the first scheduling apparatus, and is not described herein again.
In another embodiment, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the first scheduling device and the second scheduling device need to be registered in the central scheduling device, the registration process is the same as the registration process shown in fig. 7, and only the execution main body in fig. 7 needs to be replaced by the central scheduling device from the second scheduling device, which is not described herein again.
(3) Registration of border security servers
In the case where direct communication between two local points is not possible, a border security server is introduced, and at this time, the border security server acts as an intermediary, enabling indirect communication between the two local points. Therefore, the border security server needs to register in the scheduling devices of the two local points, respectively. Thus, the border security server can realize information transmission with the dispatching devices of the two local points through registration.
For example, in a case where direct communication cannot be performed between the first local point and the second local point, a border security server is introduced, and the border security server needs to be registered in the first scheduling apparatus and the second scheduling apparatus, respectively.
Specifically, the first scheduling device adds registration information of the border security server, so that the border security server is registered in the first scheduling device. The second scheduling device adds the registration information of the boundary security server so that the boundary security server is registered in the second scheduling device. The registration information includes: IP address, port, identification information, authentication information (e.g., account number, password, etc.), address information (e.g., Kafka address, mapped Kafka address, storage address, and mapped storage address), and the like of the border security server.
Meanwhile, the first scheduling device or the second scheduling device creates a shared folder in a storage space of the boundary security server for storing cross-domain scheduling information. Meanwhile, the first scheduling device or the second scheduling device may further generate a first subfolder and a second subfolder in the shared folder, where the first subfolder is used to store the algorithm information, and the second subfolder is used to store the task information.
For example, in the case where direct communication cannot be performed between the central scheduling device and the second central office, if the border security server is introduced, the border security server needs to be registered in the central scheduling device and the second scheduling device, respectively. The registration process of the border security server in the central scheduling device is the same as that of the border security server in the first scheduling device in the above example.
2. Algorithm synchronization flow
Algorithm synchronization means that algorithm information of each local point needs to be kept consistent. That is, the scheduling device at each office point needs to synchronize (or broadcast) the algorithm change information of the changed target algorithm in the data analysis device at the office point to the scheduling devices at other office points in real time, so that the scheduling devices at other office points perform algorithm synchronization according to the algorithm change information.
It can be understood that, because the algorithm packet is large, if the algorithm packet is downloaded only when in use, the timeliness of the task is affected, and the progress of task analysis is delayed. For example, if the requested party of the cross-domain resource scheduling application does not have the algorithm required by the task to be analyzed, the task to be analyzed cannot be processed. Therefore, before cross-domain resource scheduling, an algorithm synchronization process needs to be performed to ensure that algorithm information of each local point is kept consistent.
The algorithm synchronization process of the cross-domain analysis system in different deployment modes is different.
For example, when the cross-domain analysis system is in the fully symmetric deployment mode as shown in fig. 2, as shown in fig. 8, the algorithm synchronization process may include:
sc1, the first scheduling device receives the algorithm change information transmitted by the first data analysis device.
The algorithm change information comprises an identification of a target algorithm which is changed in the first data analysis device and a download address of the target algorithm. Illustratively, the target algorithm may be an own algorithm (not downloaded from other local points) added in the first data analysis device.
For example, when an own algorithm (not an algorithm downloaded from another office point) is newly added to the first data analysis device, the first data analysis device transmits algorithm change information to the first scheduling device.
The Sc2 and the second scheduling apparatus receive the algorithm change information transmitted by the first scheduling apparatus.
And the Sc3 and the second scheduling device send the download address of the target algorithm to the second data analysis device.
In some embodiments, prior to step Sc3, the method further comprises: the second scheduling device sends an algorithm query instruction to the second data analysis device, wherein the algorithm query instruction comprises: and the algorithm query instruction is used for enabling the second data analysis device to query whether the target algorithm is included in the second data analysis device or not according to the identification of the target algorithm.
In the case where the target algorithm is not included in the second data analysis device, step Sc3 is executed.
And the Sc4 and the second data analysis device download the target algorithm according to the download address of the target algorithm.
In some embodiments, in the case that the first local point and the second local point cannot directly communicate with each other, the algorithm synchronization process of the cross-domain analysis system in the fully symmetric deployment mode may be implemented as follows:
the first scheduling device uploads the algorithm change information and the algorithm package of the changed target algorithm in the first data analysis device to a first subfolder (used for storing algorithm information) in a shared folder of the boundary security server; meanwhile, the first scheduling device adds a summary file in the first subfolder, wherein the summary file is used for quickly viewing the information of the target algorithm, and the summary file comprises: the algorithm package name of the target algorithm, the algorithm manufacturer, the algorithm version, the algorithm identifier, the chip type, the file size and the like.
The second scheduling device may check the summary file of the first subfolder in the shared folder of the border security server at regular time, and when a newly added target algorithm is found, send an algorithm query instruction to the second data analysis device, so that the second data analysis device queries whether the target algorithm is included.
In the case where the target algorithm is not included in the second data analysis apparatus, the second scheduling apparatus transmits a download address of the target algorithm to the second data analysis apparatus. And the downloading address of the target algorithm is the address of a first subfolder in a shared folder of the boundary security server.
And the second data analysis device downloads the target algorithm in the boundary security gateway according to the download address of the target algorithm.
In some embodiments, after the second data analysis device completes downloading the target algorithm, the boundary security gateway may delete the algorithm information of the target algorithm stored in the first subfolder of the shared folder, reduce space occupation, and improve a storage space utilization rate.
Illustratively, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, as shown in fig. 9, the algorithm synchronization process includes:
sd1, the first scheduling device receives the algorithm change information sent by the first data analysis device.
Sd2, the central scheduling device receives the algorithm change information sent by the first scheduling device.
Sd3 and the second scheduling device receives the algorithm change information sent by the central scheduling device.
Sd4, the second scheduling means sends the download address of the target algorithm to the second data analysis means.
In some embodiments, prior to step Sd4, the method further comprises: the second scheduling device sends an algorithm query instruction to the second data analysis device, wherein the algorithm query instruction comprises: and the algorithm query instruction is used for enabling the second data analysis device to query whether the target algorithm is included in the second data analysis device or not according to the identification of the target algorithm.
In case the target algorithm is not included in the second data analysis means, step Sd4 is performed.
Sd5, and the second data analysis device downloads the target algorithm according to the identification of the target algorithm.
In some embodiments, in the case that the central scheduling device cannot directly communicate with the second local point, the algorithm synchronization process in the cross-domain analysis system-centric deployment mode may be implemented as follows:
the method comprises the steps that a first scheduling device receives algorithm change information sent by a first data analysis device and an algorithm packet of a target algorithm changed in the first data analysis device; and the central scheduling device receives the algorithm change information and the algorithm packet of the target algorithm sent by the first scheduling device.
The central scheduling device uploads the algorithm change information and the algorithm package of the target algorithm to a first subfolder (used for storing algorithm information) in a shared folder of the boundary security server; meanwhile, the central scheduling device adds the summary file in the first subfolder.
The second scheduling device may check the summary file of the first subfolder of the shared folder of the border security server at regular time, and when a newly added target algorithm is found, send an algorithm query instruction to the second data analysis device, so that the second data analysis device queries whether the target algorithm is included.
In a case where the target algorithm is not included in the second data analysis apparatus, the second scheduling apparatus transmits a download address of the target algorithm to the second data analysis apparatus, so that the second data analysis apparatus downloads the target algorithm in the boundary security gateway according to the download address of the target algorithm. And the downloading address of the target algorithm is the address of a first subfolder in a shared folder of the boundary security server.
In some embodiments, after the second data analysis device completes downloading the target algorithm, the boundary security gateway may delete the algorithm information of the target algorithm stored in the first subfolder of the shared folder, reduce space occupation, and improve a storage space utilization rate.
3. Information synchronization process of computing power resource
The information synchronization process of the computing resources means that the scheduling device of each office point needs to synchronize (or broadcast) the basic information of the computing resources of the office point to the scheduling devices of other office points in real time.
Wherein, the basic information of the computing power resource comprises: chip type, total calculation power scale, calculation power resource allowance and the like.
It can be understood that, since the scheduling device can determine which office point can be the requested party of the cross-domain resource scheduling application under the condition of knowing the computation resource of each office point in the cross-domain analysis system, the scheduling device needs to perform the information synchronization process of the computation resource in real time.
In the cross-domain analysis system in different deployment modes, the information synchronization process of computing resources is different.
For example, when the cross-domain analysis system is in the fully symmetric deployment mode as shown in fig. 2, as shown in fig. 10, the information synchronization process of the computing resource may include:
se1, the first scheduling device receives the computing power resource change information transmitted by the first data analysis device.
Wherein, calculating the resource change information of the power includes: current remaining computation force information for the first local point.
For example, the first data analysis device transmits the calculation resource change information to the first scheduling device when detecting that the calculation resource is changed.
Se2 and the second scheduling device receive the computing power resource change information sent by the first scheduling device.
Se3, the second scheduling device changes the remaining computation power information of the first office point recorded in the second scheduling device into the current remaining computation power information of the first office point according to the computation power resource change information.
In some embodiments, in the case that the first local point and the second local point cannot directly communicate with each other, the first scheduling device of the first local point may upload the computing resource change information (the computing resource change information includes current remaining computing resource information of the first local point) to a shared folder of the border security server; correspondingly, the second scheduling device of the second office point may periodically check the shared folder of the border security server, and when detecting the new computing power resource change information, the second scheduling device changes the remaining computing power information of the first office point recorded in the second scheduling device into the current remaining computing power information of the first office point according to the computing power resource change information.
For example, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, as shown in fig. 11, the information synchronization process of the computational resource may include:
sf1, the first scheduling device receives the computing power resource change information sent by the first data analysis device.
Sf2, the central scheduling device receives the computing resource change information sent by the first scheduling device.
Sf3 and the second scheduling device receive the computing resource change information sent by the central scheduling device.
Sf4, the second scheduling device changes the residual calculation force information of the first local point recorded in the second scheduling device into the current residual calculation force information of the first local point according to the calculation force resource change information.
In some embodiments, in the case that the central scheduling device cannot directly communicate with the second local point, the central scheduling device uploads the computing resource change information to a shared folder of the border security server after receiving the computing resource change information (the computing resource change information includes current remaining computing resource information of the first local point) sent by the first scheduling device; correspondingly, the second scheduling device of the second office point may periodically check the shared folder of the border security server, and when detecting the new computing power resource change information, the second scheduling device changes the remaining computing power information of the first office point recorded in the second scheduling device into the current remaining computing power information of the first office point according to the computing power resource change information.
The cross-domain resource scheduling method provided by the embodiment of the present application is explained in detail below.
Specifically, the cross-domain resource scheduling method provided in the embodiment of the present application is applied to a cross-domain analysis system, and the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the second local point comprises: a second data analysis device and a second scheduling device.
Please refer to fig. 12, which is a flowchart illustrating a cross-domain resource scheduling method according to an embodiment of the present invention. As shown in fig. 12, the method may include S101-S104.
S101, a first scheduling device receives a first cross-domain resource scheduling application sent by a first data analysis device.
Wherein the first cross-domain resource scheduling application comprises: the first task to be analyzed. The first cross-domain resource scheduling application is used for applying for executing a first task to be analyzed at other local points (for example, a second local point) except the first local point. For example, the first task to be analyzed may include: a gait recognition task (recognizing gait information in a target video/picture), a human body recognition task (recognizing human body in the target video/picture), a face recognition task (analyzing face information in the target video/picture), a vehicle recognition task (recognizing vehicle in the target video/picture), and the like.
In addition, the first cross-domain resource scheduling application may further include at least one of: the information of the application party of the first task to be analyzed, the size of the resource required by the first task to be analyzed, an acquisition address of data to be analyzed (for example, if the data to be analyzed is video data, the acquisition address is a video streaming IP address, and if the data to be analyzed is picture data, the acquisition address is a URL address of a picture), information of an algorithm required by the first task to be analyzed (including an identifier of an algorithm required by the task to be analyzed, a download address of the algorithm required by the first task to be analyzed, and the like), an analysis result storage address, an application reason, and the like.
In some embodiments, there are two scheduling schemes when cross-domain resource scheduling is performed: firstly, borrowing analysis capability from a requested party of resources by taking the number of video paths as a unit, namely, the amount of resources requested by a first cross-domain resource scheduling application is the number of video paths needing to be analyzed. For example, if the first task to be analyzed needs to analyze 10 videos in common, the resource amount applied for the first cross-domain resource scheduling is the resource needed for analyzing 10 videos. And borrowing the GPU cards from the resource requested party by taking the chip as a unit, namely, the resource amount requested by the first cross-domain resource scheduling application is the number and the type of the needed GPU cards. For example, if the first task to be analyzed needs three GPU cards, the resource amount of the first cross-domain resource scheduling application is three GPU cards.
The difference between the two scheduling schemes is shown in table 1:
TABLE 1
It can be understood that, in the embodiment of the present application, a scheduling scheme in cross-domain resource scheduling is not specifically limited, and a user may select the scheduling scheme according to an actual use situation.
For example, if the first task to be analyzed is to analyze a vehicle in the target video. The first cross-domain resource scheduling application is: the first task to be analyzed: analyzing vehicles in the target video; the first task applicant information to be analyzed: the identification of the first local point and the address information of the first local point; the size of the resources required by the first task to be analyzed: analyzing the maximum amount of the calculation resources required by the target video, for example, the first task to be analyzed needs to analyze 10 paths of GPU calculation resources of the target video; storage address of data to be analyzed: the video of the target video is taken as the IP address; algorithm information required by a first task to be analyzed: a video structuring algorithm; the storage address of the analysis result is: analyzing the storage addresses of the vehicle attribute, the vehicle characteristic model and the vehicle picture; the application reason is as follows: the intra-office resource allowance of the first office point does not meet the size of the resource needed by the task to be analyzed.
In some embodiments, each local point is connected to a fixed video monitoring point, for example, the first local point is connected to the first video monitoring point, and the first local point is configured to analyze a video data analysis task corresponding to the first video monitoring point; and the second local point is connected with the second video monitoring point and is used for analyzing the video data analysis task corresponding to the second video monitoring point.
The first cross-domain resource scheduling application may be used to apply for analyzing a video data analysis task corresponding to the second video monitoring point in the second local point; or, the first cross-domain resource scheduling application may also be used to apply for analyzing a video data analysis task corresponding to a third-party video monitoring point (other video monitoring points except the first video monitoring point and the second video monitoring point) in the second office point.
Exemplarily, if the first office point is a west lake scheduling platform in a west lake region and the second office point is a coastal river scheduling platform in a coastal river region, the first cross-region resource scheduling application is initiated by the west lake scheduling platform and is used for applying for analyzing video monitoring point locations in the coastal river region from the coastal river scheduling platform; or the first cross-domain resource scheduling application is initiated by the West lake scheduling platform and is used for applying for analyzing the video monitoring point location of the archway area (third party) to the coastal river scheduling platform.
As a possible implementation manner, when detecting that the size of the resource required by the task to be analyzed is greater than or equal to the first threshold, the first data analysis device generates a first cross-domain resource scheduling application, and sends the first cross-domain resource scheduling application to the first scheduling device.
Exemplarily, if the intra-local resource allowance of the first local point is a GPU computational power resource capable of analyzing 5 paths of videos; if the first threshold is 5, the first data analysis device generates a first cross-domain resource scheduling application according to the first task to be analyzed when detecting that the size of the resource required by the first task to be analyzed is the GPU computational power resource for analyzing the 8-channel video, and sends the first cross-domain resource scheduling application to the first scheduling device.
As another possible implementation manner, when detecting that the computational resource in the first local point is in an offline state, the first data analysis device generates a first cross-domain resource scheduling application, and sends the first cross-domain resource scheduling application to the first scheduling device.
Illustratively, when an executing node 1 in a first office point is in an offline state, a first data analysis device generates a first cross-domain resource scheduling application according to a first task to be analyzed when detecting that the executing node required by the first task to be analyzed includes the executing node 1, and sends the first cross-domain resource scheduling application to a first scheduling device.
In some embodiments, the first scheduling apparatus records resource headroom conditions of all local points in the cross-domain analysis system. For example, the first scheduling apparatus records the resource headroom of the second office. The first scheduling device may determine the second office point from the other office points according to the first cross-domain resource scheduling application, and forward the first cross-domain resource scheduling application to the second scheduling device.
And the intra-office resource allowance of the second office point is larger than or equal to the size of the resource needed by the first task to be analyzed.
It is understood that the first local point is a resource applying party, and the second local point is a resource applying party. When the intra-office resource allowance of the second office point is larger than or equal to the size of the resource needed by the first task to be analyzed, the second office point can be used as a requested party of the resource, the first cross-domain resource scheduling application is forwarded to the second scheduling device, and at this moment, the resource amount requested by the first cross-domain resource scheduling application is the size of the resource needed by the task to be analyzed.
In other embodiments, when the intra-office resource margins of all office points in the cross-domain analysis system do not satisfy (i.e., are smaller than) the size of the resource required by the first task to be analyzed, the first scheduling device determines the second office point (at this time, a plurality of second office points) according to the size of the resource required by the first task to be analyzed and the preset application rule.
As a possible implementation manner, the preset application rule may be a proportional approximation policy, and the first scheduling device is specifically configured to determine, according to the size of the resource required by the first task to be analyzed and the resource headroom conditions of all office points in the cross-domain analysis system, second office points (at this time, a plurality of second office points are provided) according to the proportional approximation policy, and determine, according to the resource headroom of each second office point, the size of the resource amount applied in the first cross-domain resource scheduling application sent to the office point.
The proportion approximation strategy is to determine an applied party applying for cross-domain resource scheduling and the size of the resource amount applied to each applied party according to the proportion of the resource allowance of each local point in the cross-domain analysis system to the total amount of the resource allowances of all local points in the cross-domain analysis system.
Illustratively, assume that the cross-domain analysis system includes: office point 1, office point 2, office point 3, and office point 4. If the local point 1 is the resource applicant, and the size of the resource required by the task to be analyzed is the GPU computing power resource capable of analyzing 10 paths of videos. The resource allowance of the local point 2 is a GPU computational power resource capable of analyzing 2 paths of videos, the resource allowance of the local point 3 is a GPU computational power resource capable of analyzing 4 paths of videos, and the resource allowance of the local point 4 is a GPU computational power resource capable of analyzing 6 paths of videos, and then according to a proportional approximation strategy, second local points are determined to be respectively: office point 2, office point 3, and office point 4; wherein, the cross-domain resource scheduling application sent to the office point 2 has the following resource amount: 10 × (2/(2+4+ 6)); sending a cross-domain resource scheduling application to the office point 3, wherein the applied resource amount is as follows: 10 × (4/(2+4+ 6)); sending a cross-domain resource scheduling application to the office point 4, wherein the applied resource amount is as follows: 10 × (6/(2+4+ 6)).
As another possible implementation manner, the preset application rule may be a sequential allocation policy, and the first scheduling device is specifically configured to determine, according to the size of the resource required by the first task to be analyzed and the resource headroom conditions of all office points in the cross-domain analysis system, second office points (at this time, a plurality of second office points) according to the sequential allocation policy, and determine, according to the resource headroom of each second office point, the size of the resource amount applied by the first cross-domain resource scheduling application sent to the office point.
In the sequential allocation strategy, in order to sort the resource margins of each office point in the cross-domain analysis system from large to small, a cross-domain resource scheduling application is sent to a first office point (i.e. the office point with the largest resource margin) first, then the cross-domain resource scheduling application is sent to a second office point, and the cross-domain resource scheduling applications are sent in sequence in this way until the resources required by the target task are allocated completely.
Illustratively, assume that the cross-domain analysis system includes: office point 1, office point 2, office point 3, and office point 4. If the local point 1 is the resource applicant and the resource required by the target task is the GPU computational power resource capable of analyzing 10 paths of video. The resource allowance of the local point 2 is a GPU computational power resource capable of analyzing 2 paths of videos, the resource allowance of the local point 3 is a GPU computational power resource capable of analyzing 4 paths of videos, the resource allowance of the local point 4 is a GPU computational power resource capable of analyzing 6 paths of videos, and then according to a sequential distribution principle, second local points are determined to be respectively: office point 4 and office point 3; firstly, sending a cross-domain resource scheduling application to a local point 4, wherein the applied resource amount is a GPU computing resource for analyzing 6 paths of videos, and then sending the cross-domain resource scheduling application to a local point 3, wherein the applied resource amount is the GPU computing resource for analyzing 4 paths of videos.
It can be understood that, in the cross-domain resource scheduling method provided in the embodiment of the present application, the scheduling device may determine that the requested party for the cross-domain resource scheduling application is one office point or multiple office points according to the size of the resource needed by the task to be analyzed and the intra-office resource margins of all office points in the system. Meanwhile, under the condition that the applied party of the cross-domain resource scheduling application is a plurality of local points, the method also supports the determination of a plurality of applied parties according to a plurality of preset application rules (including a proportional approximation strategy or a sequential distribution strategy). Therefore, the scheduling device can flexibly process the cross-domain resource scheduling application, so that the resource scheduling is more reasonable, and the resource utilization rate is improved.
As another possible implementation manner, the first scheduling device may feed back the intra-office resource margins of all office points in the cross-domain analysis system to the user, and the user manually selects one or more second office points (i.e., the requested party of the cross-domain resource scheduling application).
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: and the terminal equipment can interact with the user and receive the indication of the user. In this way, the first scheduling device may send the first cross-domain resource scheduling application to the terminal device, and the terminal device displays the first cross-domain resource scheduling application to the user. Further, the user may indicate the requested party of the cross-domain resource scheduling application.
For example, as shown in fig. 13 (a), the terminal device may display a cross-domain resource scheduling application interface on a display screen, so that a user may obtain the content of the first cross-domain resource scheduling application; further, as shown in (b) in fig. 13, the user may apply for the first cross-domain resource scheduling application to indicate the applied party of the resource in the interface indicating the applied party of the resource, for example, the applied party of the first cross-domain resource scheduling application may be: a second local point.
Correspondingly, the terminal device may further send the requested party of the first cross-domain resource scheduling application indicated by the user to the first scheduling apparatus, so that the first scheduling apparatus sends the first cross-domain resource scheduling application to the scheduling apparatus of the resource requested party indicated by the user.
It can be understood that, in the embodiment of the present application, a cross-domain resource scheduling application may be sent to a terminal device by a scheduling apparatus, and then, the terminal device may display the cross-domain resource scheduling application to a user, so that the user may indicate a resource-applied party for the cross-domain resource scheduling application according to the content of the cross-domain resource scheduling application. Therefore, the decision-making right can be provided for the user through the terminal equipment, so that the method provided by the application can be suitable for richer use scenes.
In some embodiments, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the first scheduling device sends the first cross-domain resource scheduling application to the central scheduling device, so that the central scheduling device determines a second office point from other office points according to the first cross-domain resource scheduling application, and sends the first cross-domain resource scheduling application to the second scheduling device.
The central scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system.
Optionally, the manner in which the central scheduling device determines the second local point is the same as the manner in which the first scheduling device determines the second local point.
In some embodiments, when the amount of resources required by the first task to be analyzed increases, the first scheduling apparatus may directly send a first capacity expansion application to the second scheduling apparatus; or the first scheduling device may send the first capacity expansion application to the second scheduling device through the central scheduling device.
The first capacity expansion application is used for applying the increased resource amount of the first task to be analyzed to the second local point.
S102, the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device.
In some embodiments, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the second scheduling device receives the first cross-domain resource scheduling application through the central scheduling device.
In some embodiments, the second scheduling apparatus determines whether the amount of resources that can be shared externally in the second office point satisfies the amount of resources applied by the first cross-domain resource scheduling application sent by the first scheduling apparatus.
And when the resource quantity which can be shared externally in the second office point meets the resource quantity applied by the first cross-domain resource scheduling application sent by the first scheduling device, the second scheduling device checks the first cross-domain resource scheduling application sent by the first scheduling device.
It is understood that the amount of resources that can be shared externally in the second office point may be the same as or different from the amount of resources remaining in the second office point. For example, if the intra-office resource margin of the second office point is 20, the second office point may set the amount of resources that can be shared externally to 10, and use the remaining amount of resources for processing the task inside the second office point.
Optionally, the second scheduling device may perform auditing on the first cross-domain resource scheduling application in an automatic auditing manner; or, a manual auditing mode is adopted to audit the first cross-domain resource scheduling application sent by the first scheduling device.
In some embodiments, the second scheduling device may receive the verification code sent by the first scheduling device, determine whether the identity of the first local point meets the specification (to avoid malicious attack) according to the verification code, and check the first cross-domain resource scheduling application when the first local point meets the specification.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: and the terminal equipment can interact with the user and receive the indication of the user. The terminal device comprises a display screen and can display the first cross-domain resource scheduling application.
In this way, the above-mentioned manner of manual auditing is adopted to audit the first cross-domain resource scheduling application sent by the first scheduling device, which can be implemented as follows: the first scheduling device sends a first cross-domain resource scheduling application to the terminal equipment, and the terminal equipment displays the first cross-domain resource scheduling application on an auditing interface of the cross-domain resource scheduling application, so that a user audits the first cross-domain resource scheduling application.
For example, as shown in fig. 14, an audit interface of a cross-domain resource scheduling application may include: the method comprises the steps of cross-domain resource scheduling application to be audited, cross-domain resource scheduling application which is audited to pass, and cross-domain resource scheduling application which is audited not to pass. The user can click on any window of the cross-domain resource scheduling application and view the detailed content of the cross-domain resource scheduling application.
For example, the second scheduling apparatus audits the first cross-domain resource scheduling application sent by the first scheduling apparatus, including: and auditing information of the applicant of the first task to be analyzed, auditing algorithm information required by the first task to be analyzed, auditing application reason and the like.
And under the condition that the first cross-domain resource scheduling application sent by the first scheduling device is not approved, the second scheduling device rejects the first cross-domain resource scheduling application sent by the first scheduling device.
And under the condition that the first cross-domain resource scheduling application sent by the first scheduling device passes the audit, the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device.
In some embodiments, after the first cross-domain resource scheduling application sent by the first scheduling device is approved, the second scheduling device may further modify a resource amount applied by the first cross-domain resource scheduling application sent by the first scheduling device, for example, in a case that the resource amount applied by the first cross-domain resource scheduling application sent by the first scheduling device is a GPU computational power resource for analyzing 5-way videos, the resource amount approved by the second scheduling device may be a GPU computational power resource for analyzing 2-way videos, that is, the resource amount applied by the first scheduling device is modified into a GPU computational power resource for analyzing 2-way videos.
It can be understood that, after the second scheduling apparatus receives the cross-domain resource scheduling application, the second scheduling apparatus further checks the cross-domain resource scheduling application at the perspective of the resource applicant, and determines whether the application meets the requirement of the second office point. On one hand, the second scheduling device needs to confirm whether the resources in the local point meet the size of the resources needed by the task to be analyzed; on the other hand, the second scheduling device needs to confirm whether the cross-domain resource scheduling application conforms to the relevant rules of the local point. And processing the task to be analyzed in the cross-domain resource scheduling application under the condition that the audit is passed. Therefore, through further auditing, the problems in the cross-domain resource scheduling application can be found in one step before the task to be analyzed is analyzed, the task analysis efficiency is improved, and the cross-domain analysis system can perform cross-domain resource scheduling orderly.
In some embodiments, in the case that the first local point and the second local point cannot directly communicate with each other, the step S102 may be implemented as: and the second scheduling device receives the first cross-domain resource scheduling application sent by the first scheduling device through the boundary security server.
Specifically, the first scheduling device may generate a lightweight data exchange format (JSON) file from the first cross-domain resource scheduling application, upload the JSON file to a second subfolder (used for storing task information) of the shared folder of the border security server, and name the JSON file as the name of the first task to be analyzed.
Accordingly, the second scheduling device may periodically check a second subfolder of the shared folder of the border security server, and when detecting that the first task to be analyzed is newly added, the second scheduling device generates a new task (the same as the first task to be analyzed) inside the second office point according to the content of the first task to be analyzed stored in the second subfolder, and executes the new task inside the second office point.
S103, the second data analysis device receives the first cross-domain resource scheduling application sent by the second scheduling device.
In some embodiments, when the first cross-domain resource scheduling application passes the audit, the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device, and the second data analysis device receives the first cross-domain resource scheduling application.
And S104, the second data analysis device analyzes the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
The first task to be analyzed is a task for analyzing data to be analyzed.
In some embodiments, the first cross-domain resource scheduling application further comprises: obtaining an address of data to be analyzed; the method further comprises: and the second data analysis device acquires the data to be analyzed according to the acquisition address of the data to be analyzed.
In some embodiments, in the case that the first local point and the second local point cannot communicate directly, the first scheduling device further needs to store the data to be analyzed and/or the acquisition address of the data to be analyzed in the second subfolder of the shared folder of the border security server.
Illustratively, if the data to be analyzed is real-time video data, the first scheduling device stores the video point location information and the stream fetching address of the real-time video data in a second subfolder of the shared folder of the border security server in a JSON file format. Correspondingly, the second scheduling device may obtain the video point location information and the stream fetching address of the real-time video data in a second subfolder of the shared folder of the border security server, and then the second data analysis device obtains the real-time video data according to the video point location information and the stream fetching address of the real-time video data.
For example, if the data to be analyzed is video data, the first scheduling device stores the video data in a JSON file format in the second subfolder of the shared folder of the border security server, and the data analyzing device may download the video data in the second subfolder of the shared folder of the border security server.
If the first scheduling device detects that no corresponding video data exists locally, the first scheduling device writes task abnormal information of 'video data does not exist' in the second subfolder of the shared folder of the boundary security server, and then the first scheduling device downloads videos from the local, stores the videos as offline video data in the second subfolder of the shared folder of the boundary security server, deletes previous tasks, and generates new tasks.
For another example, if the data to be analyzed is offline video data, the first scheduling device stores the offline video data in a second subfolder of the shared folder of the boundary security server in a JSON file format, and the data analyzing device may download the offline video data in the second subfolder of the shared folder of the boundary security server.
In some embodiments, after the second data analysis device acquires the data to be analyzed, the boundary security server may delete the data file to be analyzed in the second subfolder of the shared folder, so as to reduce space occupation and improve storage space utilization.
In some embodiments, if the second data analysis device cannot acquire the data to be analyzed, that is, if there is a video stream fetching abnormality, the stream fetching abnormality result is sent to the second scheduling device, and the second scheduling device forwards the stream fetching abnormality result to the first scheduling device, so that the first scheduling device acquires the stream fetching abnormality result. Wherein, get and flow the abnormal result to include: and the error code returned when the video data is obtained according to the obtaining address of the data to be analyzed for the first time and the code rate of the video data. The first scheduling device further analyzes whether the flow taking bottleneck of the data acquisition server reaches an upper limit (for example, the flow taking bottleneck of a single data acquisition server can be 600M) according to the flow taking abnormal result, and if the flow taking abnormal result reaches the upper limit, the data acquisition server is prompted to be added; if the data to be analyzed is real-time video data, the task to be analyzed is issued again; and if the data to be analyzed is video data or offline video data, re-issuing the task to be analyzed after re-acquiring the stream address.
In some embodiments, the first cross-domain resource scheduling application further comprises: the method comprises the steps that an identifier of an algorithm needed by a first task to be analyzed and a download address of the algorithm needed by the first task to be analyzed are obtained; the method further comprises: the second data analysis device inquires whether the algorithm exists in the second data analysis device according to the identification of the algorithm needed by the first task to be analyzed, and under the condition that the algorithm does not exist in the second data analysis device, the second data analysis device directly downloads the algorithm needed by the first task to be analyzed from the first data analysis device according to the download address of the algorithm needed by the first task to be analyzed.
Illustratively, the second scheduling apparatus sends an algorithm query application to the second data analysis apparatus; the algorithm query application comprises an identifier of an algorithm required by a first task to be analyzed; the algorithm query application is used for enabling the second data analysis device to query whether an algorithm required by a task to be analyzed exists; the second data analysis device queries the algorithm according to the algorithm query application and sends a query result to the second scheduling device; when the query result is that the algorithm required by the task to be analyzed does not exist in the second data analysis device, the second scheduling device sends an algorithm downloading application to the second data analysis device; the algorithm downloading application comprises a downloading address of an algorithm required by the task to be analyzed; and the second data analysis device directly downloads the algorithm required by the task to be analyzed from the first data analysis device according to the download address of the algorithm required by the task to be analyzed.
In some embodiments, in the case where direct communication is not possible between the first local point and the second local point, the first scheduler stores an algorithm package of algorithms required for the task to be analyzed in a second subfolder of the shared folder of the border security server. Further, in a case where the algorithm does not exist in the second data analysis device, the second data analysis device may download an algorithm package of an algorithm required for a task to be analyzed in the second subfolder of the shared folder of the border security server.
In some embodiments, if the second local point does not have the right to use the algorithm required by the first task to be analyzed, the second local point cannot execute the first task to be analyzed. Therefore, when the second office point does not have the right of use of the algorithm required by the first task to be analyzed, the second office point needs to acquire the right of use of the algorithm required by the first task to be analyzed.
For example, the second office point may apply for a short-term trial right and import an authorization file to obtain the right to use the algorithm required by the first task to be analyzed.
Thus, the step S104 may be implemented as: and the second data analysis device analyzes the first task to be analyzed according to the data to be analyzed and the algorithm required by the first task to be analyzed to obtain an analysis result.
In some embodiments, the first cross-domain resource scheduling application further comprises: a storage address of the analysis result; after obtaining the analysis result from the step S104, the method further includes: the second data analysis means stores the analysis result in the storage space indicated by the storage address of the analysis result (when storing the analysis result, the second data analysis means stores the analysis result directly from the result storage address without using the scheduling means).
In some embodiments, the second data analysis device stores the analysis result in a storage space indicated by a storage address of the analysis result by the Kafka system. Kafka is a high-throughput distributed publish-subscribe messaging system that can process large amounts of data in real-time to meet various demand scenarios. The Kafka system consists primarily of two messaging modes: a point-to-point delivery mode and a publish-subscribe delivery mode. The message delivery mode is not particularly limited in the embodiments of the present application.
In some embodiments, in a case where direct communication cannot be performed between the first local point and the second local point, the second data analysis apparatus may store the analysis result in a storage space indicated by a storage address of the analysis result through a video image transmission server (e.g., a view library gateway).
For example, the second data analysis means may transmit the analysis result and the analysis result storage address to the video image transmission server, and the video image transmission server stores the analysis result in the storage space indicated by the analysis result storage address.
It can be understood that, in the cross-domain resource scheduling method provided in the embodiment of the present application, the second data analysis device may directly obtain data to be analyzed (for example, video data or picture data) according to the obtaining address of the data to be analyzed; under the condition that the algorithm required by the data to be analyzed does not exist in the second data analysis device, directly acquiring the algorithm required by the data to be analyzed according to the download address of the algorithm required by the data to be analyzed; and storing the analysis result in a corresponding position according to the analysis result storage address. That is, the second data analysis device can directly acquire the service data or directly store the analysis result data without the need of forwarding by the scheduling device. The scheduling device only needs to be responsible for forwarding the task control data. In this way, the configuration of the scheduling device is made more lightweight, and the data transfer efficiency can be improved.
In some embodiments, in the step S104, the method further includes: and the second scheduling device receives the control instruction sent by the first scheduling device and forwards the control instruction to the second data analysis device, so that the second data analysis device controls the analysis of the state of the first task to be analyzed according to the control instruction.
For example, a user at the first local point may send a control instruction through the first scheduling device, forward the control instruction through the second scheduling device, and send the control instruction to an analysis task in the second data analysis device, so as to perform the following control operations on the processing state of the analysis task: start, modify, delete, or stop, etc.
For another example, in the process of executing the second task to be analyzed by the second data analysis device, if the first local point learns that the intra-office resource allowance of the second local point is smaller than the size of the resource required by the first task to be analyzed, the first local point may further replace the local point, and send a local point replacement message to the second scheduling device through the first scheduling device.
For another example, if the first local point learns that the first task to be analyzed is in a waiting state or an abnormal state in the second local point for a long time, the first scheduling device may send an exception troubleshooting instruction to the second scheduling device, so that the second local point may troubleshoot an exception cause; alternatively, the first scheduling device may send a priority execution instruction to the second scheduling device, so that the second office point stops the low-priority task and preferentially executes the first task to be analyzed.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: and the terminal equipment can interact with the user and receive the indication of the user. The terminal equipment comprises a display screen which can display the execution state of the first cross-domain resource scheduling application, so that a user can know the execution state of the first cross-domain resource scheduling application.
For example, as shown in fig. 15, a task execution state interface may be displayed on a display screen of the terminal device, and the task execution state interface may include: the name of each task to be analyzed, the task execution state, the resources required by the task and other information.
In some embodiments, in a case where direct communication cannot be performed between the first local point and the second local point, the second scheduling device may transmit, to the first scheduling device, the state in which the second data analysis device analyzes the first task to be analyzed through the boundary security server; the first scheduling device can transmit the control instruction to the second scheduling device through the boundary security server, and the control instruction is forwarded to the second data analysis device by the second scheduling device, so that the second data analysis device controls the analysis of the state of the first task to be analyzed according to the control instruction.
Illustratively, the second scheduling means generates a JSON file by identifying the first task to be analyzed and analyzing the state of the first task to be analyzed by the second data analysis means, stores the JSON file in the second subfolder of the shared folder of the border security server, and names the JSON file as the name and the execution state of the first task to be analyzed.
Accordingly, the first scheduler may view the progress of the first task to be analyzed in a second subfolder of the shared folder of the border security server.
In some embodiments, after the first scheduling device obtains the progress of the first task to be analyzed, the boundary security server may delete the task progress file in the second subfolder of the shared folder, so as to reduce space occupation and improve the storage space utilization rate.
For example, the first scheduler may generate a JSON file for the control instruction of the first task to be analyzed, store the JSON file in a second subfolder of the shared folder of the border security server, and name the JSON folder as the name of the first task to be analyzed and the timestamp of the control instruction.
Correspondingly, the second scheduling device may periodically check the second subfolder of the shared folder of the border security server, and when a new control command is detected, send the control command to the second data analysis device, so that the second data analysis device controls the state of analyzing the first task to be analyzed according to the control command.
In some embodiments, after the second scheduling device obtains the control instruction, the border security server may delete the control instruction file in the second subfolder of the shared folder, so as to reduce space occupation and improve storage space utilization.
It can be understood that the first scheduling device can acquire the processing state of the task to be analyzed, which is analyzed by the second data analysis device, in real time, and control the state. Therefore, when the upper-layer application modifies the task to be analyzed, the first scheduling device can timely perform corresponding modification operation.
It can be understood that, the cross-domain resource scheduling method provided in the embodiment of the present application is based on a cross-domain analysis system, so that each office point can initiate a cross-domain resource scheduling application (including a task to be analyzed) to another office point through a scheduling device; and the corresponding other local point can receive the cross-domain resource scheduling application through the scheduling device, and analyze the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, large-scale transformation of each local point is not needed, information can be circulated among the local points only by introducing the scheduling device on the basis of the original data analysis device, cross-domain scheduling of analysis resources (cross-domain information interaction of the local points is called here) can be realized, computational resource integration is realized, and the utilization rate of global computational resources is improved.
In addition, the cross-domain resource scheduling method provided by the application can be used for sending the task to be analyzed to other idle local points in time for analysis when a certain local point cannot bear the task to be analyzed, so that the task to be analyzed can be processed in time under the condition that normal service of the local point is not interrupted. Therefore, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle states of different local points can be solved, and the global computational resource utilization rate is improved.
It can be seen that, in the cross-domain resource scheduling method provided in the embodiment of the present application, the forwarding modes of the task control data (for example, a cross-domain resource scheduling application, a control instruction, algorithm change information, or calculation resource change information, etc.) and the service data (for example, data to be analyzed, an algorithm required by a task to be analyzed, an analysis result, etc.) are separated, where the scheduling device forwards the task control data, and the data processing module of the data analysis device directly obtains or stores the service data. Thus, the performance requirement on the scheduling device is low, and the configuration of the scheduling device is lighter.
In addition, the embodiment of the application also provides a mode of task control data and service data forwarding in a unified manner. For ease of understanding, two data transfer methods will be described below by taking fig. 16 and 17 as examples.
For example, fig. 16 provides a schematic diagram of a manner of unified forwarding of task control data and service data. As shown in fig. 16, after receiving a cross-domain resource scheduling application sent by a first data analysis device, a first scheduling device obtains data to be analyzed according to an address of the data to be analyzed in the cross-domain resource scheduling application, and forwards the cross-domain resource scheduling application (task control data) and the data to be analyzed (service data) to a second scheduling device in an integrated manner; after the audit is passed, the second scheduling device integrates the cross-domain resource scheduling application (task control data) and the data to be analyzed (service data) and forwards the cross-domain resource scheduling application and the data to be analyzed (service data) to the second data analysis device; and a second data analysis device for storing the analysis result in the storage system after the analysis result is obtained, and further storing the analysis result in a result storage place by the storage system.
It can be understood that, the above manner of forwarding the task control data and the service data in a unified manner has a higher requirement on the performance of the scheduling device, the scheduling device needs to have a larger storage space and more processing resources, and the forwarded data amount is larger, which may affect other modules inside the office.
For example, fig. 17 provides a schematic diagram of a manner in which task control data and service data are separately forwarded. As shown in fig. 17, after receiving the cross-domain resource scheduling application sent by the first data analysis device, the first scheduling device selects the second office as the requested party of the resource, and further sends the cross-domain resource scheduling application to the second scheduling device, and after the audit is passed, the second scheduling device sends the cross-domain resource scheduling application to the second data analysis device; after receiving the cross-domain resource scheduling application, the second data analysis device directly acquires data to be analyzed from a data storage place according to a data storage address to be analyzed, directly acquires an algorithm required by a task to be analyzed according to the address of the algorithm required by the task to be analyzed (under the condition that the algorithm required by the task to be analyzed does not exist in the second data analysis device), further analyzes the target task according to the algorithm required by the task to be analyzed, obtains an analysis result, and directly stores the analysis result to a result storage place.
It can be understood that, the above-mentioned manner of separately forwarding the task control data and the service data has a lower requirement on the performance of the scheduling device, so that the scheduling device is only responsible for the flow of the task control data and not responsible for the flow of the service data (in the field of image video processing, the bytes of the service data are large). Thus, the configuration of the scheduling device is more lightweight, and the burden of each local point is not increased. In some embodiments of the present application, a method of separately forwarding task control data and service data is adopted.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a third office, the third office comprising: a third scheduling device and a third data analysis device. When the cross-domain analysis system is in the fully symmetric deployment mode as shown in fig. 2, as shown in fig. 18, the method further includes steps S201 to S205.
S201, the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device.
And the second cross-domain resource scheduling application comprises a second task to be analyzed. A second cross-domain resource scheduling application for applying to execute a second task to be analyzed at other local points except the second local point;
as a possible implementation manner, when detecting that the size of the resource required by the second task to be analyzed is greater than or equal to the first threshold, the second data analysis device generates a second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device.
As another possible implementation manner, when detecting that the computation resource in the second local point is in an offline state, the second data analysis device generates a second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device.
S202, the second scheduling device determines a third office point from other office points according to the second cross-domain resource scheduling application.
And the intra-office resource allowance of the third office point is larger than or equal to the size of the resource required by the second task to be analyzed.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a terminal device, which may receive an indication of a user. Thus, the step S202 can be realized as the following steps:
and step g1, the second scheduling device sends a second cross-domain resource scheduling application to the terminal equipment.
Step g2, the terminal device receives the second cross-domain resource scheduling application, and receives the third office point indicated by the user as the second cross-domain resource scheduling application.
And g3, the terminal device sends the identifier of the third office point to the second scheduling device.
S203, the second scheduling device forwards the second cross-domain resource scheduling application to a third scheduling device.
And S204, the third scheduling device forwards the second cross-domain resource scheduling application to a third data analysis device.
And S205, the third data analysis device analyzes the second task to be analyzed according to the second cross-domain resource scheduling application to obtain an analysis result.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a third office point, the third office point comprising: a third scheduling device and a third data analysis device. When the cross-domain analysis system is in the central deployment mode as shown in fig. 4, as shown in fig. 19, the method further includes steps S301 to S306.
S301, the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device.
And the second cross-domain resource scheduling application comprises a second task to be analyzed. A second cross-domain resource scheduling application for applying to execute a second task to be analyzed at other local points except the second local point;
as a possible implementation manner, when detecting that the size of the resource required by the second task to be analyzed is greater than or equal to the first threshold, the second data analysis device generates a second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device.
As another possible implementation manner, when detecting that the computation resource in the second local point is in an offline state, the second data analysis device generates a second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device.
S302, the second scheduling device sends a second cross-domain resource scheduling application to the central scheduling device.
S303, the central scheduling device determines a third local point from other local points according to the second cross-domain resource scheduling application.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a terminal device, which may receive an indication of a user. Thus, the step S303 can be realized as the following steps:
and h1, the central scheduling device sends a second cross-domain resource scheduling application to the terminal equipment.
And h2, the terminal equipment receives the second cross-domain resource scheduling application and receives a third local point indicated by the user as the second cross-domain resource scheduling application.
And h3, the terminal equipment sends the identifier of the third local point to the central dispatching device.
And the intra-office resource allowance of the third office point is larger than or equal to the size of the resource required by the second task to be analyzed.
S304, the central scheduling device sends a second cross-domain resource scheduling application to a third scheduling device.
S305, the third scheduling device forwards the second cross-domain resource scheduling application to a third data analysis device.
And S306, the third data analysis device analyzes the second task to be analyzed according to the second cross-domain resource scheduling application to obtain an analysis result.
It can be understood that the second office point may also be used as an application party for the cross-domain resource scheduling application, and the second scheduling device or the central scheduling device determines a third office point according to the second cross-domain resource scheduling application, sends the second cross-domain resource scheduling application to the third office point, and executes the second task to be analyzed by the third office point. That is, each local point provided by the embodiment of the present application can be used as an applying party or an applied party for cross-domain resource scheduling, thereby truly realizing cross-domain scheduling of resources and effectively improving the resource utilization rate of the universe.
In some embodiments, in the process of analyzing the second task to be analyzed by the third data analysis device, if the second data analysis device receives an indication that the amount of resources required by the second task to be analyzed is increased, the second data analysis device may generate the second capacity expansion application according to the increased amount of resources of the second task to be analyzed. Specifically, as shown in fig. 20, the method may be implemented as the following steps:
s401, the second data analysis device sends a second capacity expansion application to the second scheduling device.
The second capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point.
Optionally, the second capacity expansion application includes an increased resource amount of the second task to be analyzed.
S402, the second dispatching device sends a second capacity expansion application to the third dispatching device.
As a possible implementation manner, when the cross-domain analysis system is in the fully symmetric deployment mode as shown in fig. 2, the second scheduling device directly sends the second capacity expansion application to the third scheduling device.
As another possible implementation manner, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the second scheduling device sends the second capacity expansion application to the central scheduling device, and the central scheduling device forwards the second capacity expansion application to the third scheduling device.
And S403, the third scheduling device checks the second capacity expansion application.
In some embodiments, the third scheduling device reviewing the second capacity expansion application may include: the third scheduling device judges whether the intra-office resource allowance of the third office point meets the resource amount applied by the second capacity expansion application (namely the increased resource amount of the second task to be analyzed); if yes, the third scheduling device determines that the second capacity expansion application passes the audit; and if not, the third scheduling device determines that the second capacity expansion application is not approved and rejects the second capacity expansion application.
And S404, under the condition that the second capacity expansion application passes the audit, the third scheduling device sends the second capacity expansion application to the third data analysis device.
Correspondingly, after receiving the second capacity expansion application, the third data analysis device allocates the resource amount applied by the second capacity expansion application (i.e., the increased resource amount of the second task to be analyzed) to the second task to be analyzed.
It can be understood that, according to the cross-domain resource scheduling method provided by the present application, in the case that the amount of resources required by the second task to be analyzed is increased, the increased amount of resources of the second task to be analyzed can be applied to the local point where the second task to be analyzed is executed again, so that the processing process of the second task to be analyzed is not interrupted, and the task analysis efficiency is improved.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a fourth office point, the fourth office point comprising: and a fourth scheduling device. As shown in fig. 21, the method further includes steps S501 to S503.
S501, the fourth scheduling device sends a task analysis request to the second scheduling device.
The task analysis request is used for applying for the second scheduling device to execute a task to be analyzed in the second office point in the fourth office point.
Optionally, the task analysis request includes: address information of a fourth office point, intra-office resource headroom of the fourth office point, and authentication information.
And S502, the second scheduling device checks the task analysis request.
In some embodiments, the second scheduler reviewing the task analysis request comprises: the second scheduling device verifies the validity of the fourth local point (avoids malicious network attack) according to the verification information, and determines that the task analysis request passes the verification under the condition that the fourth local point is legal; and/or the second scheduling device judges whether the intra-office resource allowance of the fourth office point meets the size of the resource required by the task to be analyzed in the second office point, and if so, the task analysis request is determined to pass the audit.
In some embodiments, the second scheduler rejects the task analysis request in case the task analysis request fails the audit.
S503, under the condition that the task analysis request passes the audit, the second scheduling device sends a third cross-domain resource scheduling application to the fourth scheduling device.
The third cross-domain resource scheduling application comprises a third task to be analyzed; the third cross-domain resource scheduling application is used for applying for executing the third task to be analyzed in a fourth local point.
It can be understood that, in the cross-domain resource scheduling method provided in this embodiment of the present application, a local point may initiate a task analysis request to a scheduling device of another local point through the scheduling device of the local point, so as to request that a task to be analyzed of another local point be executed at the local point. Therefore, by using the method provided by the application, on one hand, when the idle resources of a certain office point are more, the task to be analyzed can be actively requested to other office points, so that the task to be analyzed can be processed in time. Therefore, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle states of different local points can be solved, and the global computational resource utilization rate is improved.
The embodiment of the application provides a scheduling device, which is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the second office point comprises: a second data analysis device and the scheduling device.
As shown in fig. 22, the scheduling apparatus 600 includes: a receiving unit 601 and a transmitting unit 602. In some embodiments, the scheduling apparatus 600 further includes: a determination unit 603 and an information change unit 604.
A receiving unit 601, configured to receive a first cross-domain resource scheduling application sent by a first scheduling apparatus; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device.
A sending unit 602, configured to send the first cross-domain resource scheduling application to the second data analysis device, so that the second data analysis device analyzes the first task to be analyzed, and obtains an analysis result.
In a possible implementation manner, the cross-domain analysis system further includes: the central scheduling device is respectively connected with the first scheduling device and the scheduling device 600; the receiving unit 601 is specifically configured to receive a first cross-domain resource scheduling application through a central scheduling apparatus, where the first cross-domain resource scheduling application is sent by the first scheduling apparatus to the central scheduling apparatus, so that the central scheduling apparatus determines a second office point from other office points according to the first cross-domain resource scheduling application, and sends the first cross-domain resource scheduling application to the scheduling apparatus; and the intra-office resource allowance of the second office point is larger than or equal to the size of the resource required by the first task to be analyzed.
In another possible implementation manner, the receiving unit 601 is further configured to receive, by the central scheduling apparatus, a first capacity expansion application when the amount of resources required by the first task to be analyzed is increased, where the first capacity expansion application is sent by the first scheduling apparatus to the central scheduling apparatus, so that the central scheduling apparatus sends the first capacity expansion application to the scheduling apparatus 600; the first capacity expansion application is used for applying for the increased resource quantity of the first task to be analyzed from the second local point.
In another possible implementation manner, the cross-domain analysis system further includes: a third office point, the third office point comprising: a third scheduling device, a third data analysis device, and a receiving unit 601, further configured to receive a second cross-domain resource scheduling application sent by a second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying other local points except the second local point to execute a second task to be analyzed; a determining unit 603, configured to determine a third office point from other office points according to the second cross-domain resource scheduling application; a sending unit 602, further configured to send a second cross-domain resource scheduling application to a third scheduling apparatus, so that the third data analysis apparatus executes a second task to be analyzed; and the intra-office resource allowance of the third office point is larger than or equal to the size of the resource required by the second task to be analyzed.
In another possible implementation manner, the sending unit 602 is further configured to send a second capacity expansion application to a third scheduling apparatus when the amount of resources required by the second task to be analyzed is increased; the second capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point.
In another possible implementation manner, the cross-domain analysis system further includes: a fourth office point, the fourth office point comprising: a fourth scheduling device; a receiving unit 601; the system is further configured to receive a task analysis request sent by a fourth scheduling device, where the task analysis request is used to apply the scheduling device to execute a task to be analyzed in a second office point in a fourth office point, and the task analysis request includes: address information of a fourth office point and an intra-office resource margin of the fourth office point.
In another possible implementation manner, the sending unit 602 is further configured to send an algorithm query application to the second data analysis device, where the algorithm query application includes: the identification of the algorithm required by the first task to be analyzed, so that the second data analysis device inquires whether the algorithm required by the first task to be analyzed is included in the second data analysis device or not according to the identification of the algorithm required by the first task to be analyzed; a sending unit 602, further configured to send an algorithm download application to the second data analysis apparatus when the second data analysis apparatus does not include the algorithm required by the first task to be analyzed; the algorithm downloading application comprises the following steps: and the second data analysis device downloads the algorithm required by the first task to be analyzed according to the download address of the algorithm required by the first task to be analyzed.
In another possible implementation manner, the receiving unit 601 is further configured to receive a control instruction sent by the first scheduling apparatus, and control the second data analysis apparatus to analyze the state of the first task to be analyzed according to the control instruction.
In another possible implementation manner, the receiving unit 601 is further configured to receive algorithm change information sent by the first scheduling apparatus; the algorithm change information comprises an identification of a target algorithm which is changed in the first data analysis device and a download address of the target algorithm; a sending unit 602, configured to send an identifier of the target algorithm to the second data analysis apparatus, so that the second data analysis apparatus queries whether the target algorithm exists in the second data analysis apparatus according to the identifier of the target algorithm; the sending unit 602 is further configured to send, to the second data analysis device, the download address of the target algorithm when the target algorithm does not exist in the second data analysis device, so that the second data analysis device downloads the target algorithm according to the download address of the target algorithm.
In another possible implementation manner, the receiving unit 601 is further configured to receive computing power resource change information sent by the first scheduling apparatus; the calculation force resource change information comprises: current remaining calculation power information of the first local point; an information changing unit 604 for changing the remaining computation power information of the first local point described in the scheduling apparatus to the current remaining computation power information of the first local point, based on the computation power resource change information.
In another possible implementation manner, the cross-domain analysis system further includes: a terminal device; the sending unit 601 is further configured to send a second cross-domain resource scheduling application to the terminal device, so that the terminal device receives the second cross-domain resource scheduling application, and receives a third office point indicated by the user as the second cross-domain resource scheduling application.
In another possible implementation manner, the receiving unit 601 is further configured to receive, by the border security server, the first cross-domain resource scheduling application sent by the first scheduling apparatus when the first local point and the second local point cannot directly communicate with each other.
Of course, the scheduling apparatus 600 provided in the embodiment of the present application includes, but is not limited to, the above modules.
The embodiment of the application provides a data analysis device, which is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the second office point comprises: the data analysis device and the second scheduling device.
As shown in fig. 23, the data analysis device 700 includes: a receiving unit 701 and a processing unit 702. In some embodiments, the data analysis apparatus 700 further comprises: an obtaining unit 703 and an inquiring unit 704.
A receiving unit 701, configured to receive a first cross-domain resource scheduling application sent by a second scheduling apparatus; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device through the first scheduling device.
The processing unit 702 is configured to analyze the first task to be analyzed according to the first cross-domain resource scheduling application, so as to obtain an analysis result.
In a possible implementation manner, the first task to be analyzed is a task for analyzing data to be analyzed; the first cross-domain resource scheduling application further includes: obtaining an address of data to be analyzed; an obtaining unit 703, configured to obtain data to be analyzed according to an obtaining address of the data to be analyzed; the first cross-domain resource scheduling application further includes: downloading addresses of algorithms required by a first task to be analyzed; the obtaining unit 703 is further configured to directly download the algorithm required by the first task to be analyzed from the first data analysis device according to the download address of the algorithm required by the first task to be analyzed; the processing unit 702 is specifically configured to analyze the first task to be analyzed according to the data to be analyzed and the algorithm required by the first task to be analyzed included in the algorithm download application, and obtain an analysis result.
In another possible implementation manner, the receiving unit 701 is further configured to receive an algorithm query application sent by the second scheduling apparatus; the algorithm inquiry application comprises the following steps: identification of an algorithm required by a first task to be analyzed; a query unit 704, configured to query, according to an identifier of an algorithm required by a first task to be analyzed, whether the data analysis apparatus includes the algorithm required by the first task to be analyzed; the receiving unit 701 is further configured to receive an algorithm download application sent by a second scheduling device when the data analysis device does not include the algorithm required by the first task to be analyzed; the algorithm downloading application comprises the following steps: downloading addresses of algorithms required by a first task to be analyzed; and the acquisition unit is also used for downloading the algorithm required by the first task to be analyzed according to the downloading address of the algorithm required by the first task to be analyzed.
Of course, the data analysis apparatus 700 provided in the embodiments of the present application includes, but is not limited to, the modules described above.
In the case of implementing the functions of the integrated modules in the form of hardware, another embodiment of the present application provides a structural schematic diagram of an electronic device. Optionally, the electronic device may be a scheduling apparatus; alternatively, the electronic device may be a data analysis apparatus.
As shown in fig. 24, the electronic apparatus 800 includes: a processor 802, a communication interface 803, and a bus 804. Optionally, the electronic device 800 may further include a memory 801.
The processor 802 may be any means that implements or executes the various illustrative logical blocks, modules, and circuits described in connection with the present disclosure. The processor 802 may be a central processing unit, general-purpose processor, digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 802 may also be a combination of computing functions, e.g., comprising one or more microprocessors in conjunction with one or more DSPs and microprocessors, or the like.
A communication interface 803 for connecting with other devices through a communication network. The communication network may be an ethernet network, a wireless access network, a Wireless Local Area Network (WLAN), or the like.
The memory 801 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that may store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
As one possible implementation, the memory 801 may exist independently of the processor 802, and the memory 801 may be coupled to the processor 802 via the bus 804 for storing instructions or program code. The processor 802 can implement the cross-domain resource scheduling method provided by the embodiment of the present application when calling and executing the instructions or program codes stored in the memory 801.
In another possible implementation, the memory 801 may also be integrated with the processor 802.
The bus 804 may be an Extended Industry Standard Architecture (EISA) bus or the like. The bus 804 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 24, but this does not mean only one bus or one type of bus.
Through the description of the foregoing embodiments, it will be clear to those skilled in the art that, for convenience and simplicity of description, only the division of the functional modules is illustrated, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the electronic device is divided into different functional modules, so as to complete all or part of the functions described above.
Another embodiment of the present application also provides a computer-readable storage medium having computer instructions stored therein. When the computer instructions are run on a computer, the computer is caused to execute the steps in the flow of the cross-domain resource scheduling method shown in the above method embodiment.
Another embodiment of the present application also provides a computer program product comprising computer instructions. When the computer instructions are run on a computer, the computer is caused to execute each step in the flow of the cross-domain resource scheduling method shown in the above method embodiment.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented using a software program, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The processes or functions according to the embodiments of the present application are generated in whole or in part when the computer-executable instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer-readable storage media can be any available media that can be accessed by a computer or data storage device including one or more available media integrated servers, data centers, and the like. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
The foregoing is only illustrative of the present application. Those skilled in the art should appreciate that changes and substitutions can be made in the embodiments provided herein without departing from the scope of the present disclosure.
Claims (17)
1. A cross-domain resource scheduling method is applied to a cross-domain analysis system, and the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the second local point comprises a first data analysis device and a first scheduling device: the second data analysis device and the second scheduling device; the method comprises the following steps:
the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; wherein the first cross-domain resource scheduling application is an application initiated by the first data analysis device;
and the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device, so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
2. The method of claim 1, wherein the cross-domain analysis system further comprises: the central scheduling device is respectively connected with the first scheduling device and the second scheduling device, and the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device, and the method comprises the following steps:
the second scheduling device receives the first cross-domain resource scheduling application through the central scheduling device, wherein the first cross-domain resource scheduling application is sent to the central scheduling device by the first scheduling device, so that the central scheduling device determines the second office point from the other office points according to the first cross-domain resource scheduling application and sends the first cross-domain resource scheduling application to the second scheduling device; and the intra-office resource allowance of the second office point is larger than or equal to the size of the resource required by the first task to be analyzed.
3. The method of claim 2, further comprising:
under the condition that the resource quantity required by the first task to be analyzed is increased, the second scheduling device receives a first capacity expansion application through the central scheduling device, wherein the first capacity expansion application is sent to the central scheduling device by the first scheduling device, so that the central scheduling device sends the first capacity expansion application to the second scheduling device; the first capacity expansion application is used for applying the increased resource amount of the first task to be analyzed to the second local point.
4. The method of claim 1, wherein the cross-domain analysis system further comprises: a third office point, the third office point comprising: a third scheduling device and a third data analysis device, the method further comprising:
the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying for executing the second task to be analyzed at other local points except the second local point;
the second scheduling device determines the third local point from the other local points according to the second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes the second task to be analyzed; and the intra-office resource allowance of the third office point is larger than or equal to the size of the resource required by the second task to be analyzed.
5. The method of claim 4, further comprising:
under the condition that the resource quantity required by the second task to be analyzed is increased, the second scheduling device sends a second capacity expansion application to the third scheduling device; the second capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point.
6. The method of claim 1, wherein the cross-domain analysis system further comprises: a fourth office point, the fourth office point comprising: a fourth scheduling apparatus, the method further comprising:
the second scheduling device receives a task analysis request sent by the fourth scheduling device, where the task analysis request is used to apply the second scheduling device to execute a task to be analyzed in the second office point in the fourth office point, and the task analysis request includes: address information of the fourth office point and an intra-office resource margin of the fourth office point.
7. The method according to any one of claims 1 to 6, further comprising:
the second scheduling device receives the computing resource change information sent by the first scheduling device; the calculation force resource change information comprises: the current residual calculation force information of the first local point;
and the second scheduling device changes the residual calculation power information of the first local point recorded in the second scheduling device into the current residual calculation power information of the first local point according to the calculation power resource change information.
8. The method of claim 4 or 5, wherein the cross-domain analysis system further comprises: a terminal device;
determining the third office point from the other office points according to the second cross-domain resource scheduling application includes:
and the second scheduling device sends the second cross-domain resource scheduling application to the terminal equipment, so that the terminal equipment receives the second cross-domain resource scheduling application and receives a third local point indicated by the user as the second cross-domain resource scheduling application.
9. A cross-domain resource scheduling method is applied to a cross-domain analysis system, and the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the second local point comprises a first data analysis device and a first scheduling device: the second data analysis device and the second scheduling device; the method comprises the following steps:
the second data analysis device receives a first cross-domain resource scheduling application sent by the second scheduling device; the first cross-domain resource scheduling application comprises: a first task to be analyzed; the first cross-domain resource scheduling application is used for applying other local points except the first local point to execute the first task to be analyzed; wherein the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device via the first scheduling device;
and the second data analysis device analyzes the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
10. A scheduling device, applied to a cross-domain analysis system, the cross-domain analysis system comprising: a first office point and a second office point; wherein the first local point comprises: the second office point comprises a first data analysis device and a first scheduling device, and the second office point comprises: the second data analysis device and the scheduling device; the scheduling device comprises:
a receiving unit, configured to receive a first cross-domain resource scheduling application sent by the first scheduling apparatus; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; wherein the first cross-domain resource scheduling application is an application initiated by the first data analysis device;
a sending unit, configured to send the first cross-domain resource scheduling application to the second data analysis device, so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
11. The scheduler of claim 10,
the cross-domain analysis system further comprises: the central scheduling device is respectively connected to the first scheduling device and the scheduling device, and the receiving unit is specifically configured to receive the first cross-domain resource scheduling application through the central scheduling device, where the first cross-domain resource scheduling application is sent by the first scheduling device to the central scheduling device, so that the central scheduling device determines the second office point from the other office points according to the first cross-domain resource scheduling application, and sends the first cross-domain resource scheduling application to the scheduling device; the intra-office resource allowance of the second office point is larger than or equal to the size of the resource needed by the first task to be analyzed;
the receiving unit is further configured to receive, by the central scheduling apparatus, a first capacity expansion application when the amount of resources required by the first task to be analyzed is increased, where the first capacity expansion application is sent by the first scheduling apparatus to the central scheduling apparatus, so that the central scheduling apparatus sends the first capacity expansion application to the second scheduling apparatus; the first capacity expansion application is used for applying for the increased resource amount of the first task to be analyzed from the second local point;
the cross-domain analysis system further comprises: a third office point, the third office point comprising: the receiving unit is further configured to receive a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying for executing the second task to be analyzed at other local points except the second local point; the scheduling apparatus further includes: a determining unit, configured to determine the third office point from the other office points according to the second cross-domain resource scheduling application; the sending unit is further configured to send the second cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes the second task to be analyzed; the intra-office resource allowance of the third office point is larger than or equal to the size of the resource required by the second task to be analyzed;
the sending unit is further configured to send a second capacity expansion application to the third scheduling device when the amount of resources required by the second task to be analyzed is increased; the second capacity expansion application is used for applying for the increased resource amount of the second task to be analyzed from the third local point;
the cross-domain analysis system further comprises: a fourth office point, the fourth office point comprising: a fourth scheduling device; the receiving unit; and is further configured to receive a task analysis request sent by the fourth scheduling device, where the task analysis request is used to apply for the scheduling device to execute a task to be analyzed in the second office point in the fourth office point, and the task analysis request includes: the address information of the fourth office point and the intra-office resource allowance of the fourth office point;
the receiving unit is further configured to receive the computing resource change information sent by the first scheduling apparatus; the calculation force resource change information comprises: current remaining calculation power information of the first local point; the scheduling apparatus further includes: an information changing unit configured to change, according to the calculation power resource change information, the remaining calculation power information of the first office point described in the scheduling apparatus to the current remaining calculation power information of the first office point;
the cross-domain analysis system further comprises: a terminal device; the sending unit is further configured to send the second cross-domain resource scheduling application to the terminal device, so that the terminal device receives the second cross-domain resource scheduling application, and receives a third office point indicated by the user as the second cross-domain resource scheduling application.
12. A data analysis device applied to a cross-domain analysis system, the cross-domain analysis system comprising: a first office point and a second office point; wherein the first local point comprises: the second local point comprises a first data analysis device and a first scheduling device: the data analysis device and the second scheduling device; the data analysis device includes:
a receiving unit, configured to receive a first cross-domain resource scheduling application sent by the second scheduling apparatus; the first cross-domain resource scheduling application comprises: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; wherein the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device via the first scheduling device;
and the processing unit is used for analyzing the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
13. A office point, comprising: scheduling means and data analysis means;
the scheduling device comprises a first memory and a first processor; the first memory and the first processor are coupled; the first memory for storing computer program code, the computer program code comprising computer instructions;
wherein the computer instructions, when executed by the first processor, cause the scheduling apparatus to perform the cross-domain resource scheduling method of any one of claims 1 to 8;
the data analysis device comprises a second memory and a second processor; the second memory and the second processor are coupled; the second memory for storing computer program code, the computer program code comprising computer instructions;
wherein the computer instructions, when executed by the second processor, cause the data analysis apparatus to perform the cross-domain resource scheduling method of claim 9.
14. A cross-domain analysis system, comprising: a first office point and a second office point; wherein the first local point comprises: the second office point comprises a first data analysis device and a first scheduling device, and the second office point comprises: a second data analysis device and a second scheduling device;
the first scheduling device is configured to receive a cross-domain resource scheduling application of the first data analysis device; the cross-domain resource scheduling application comprises: a task to be analyzed; the cross-domain resource scheduling application is used for applying for executing the task to be analyzed at other local points except the first local point;
the first scheduling device is further configured to forward the cross-domain resource scheduling application to the second scheduling device.
15. The system of claim 14, wherein the cross-domain resource scheduling application further comprises: the size of the resources required by the task to be analyzed;
the first scheduling device is further configured to determine the second office point from the other office points according to the cross-domain resource scheduling application; and the intra-office resource allowance of the second office point is larger than or equal to the size of the resource required by the task to be analyzed.
16. The system of claim 14, further comprising: a central scheduling device; the central scheduling device is respectively connected with the first scheduling device and the second scheduling device; the cross-domain resource scheduling application further comprises: the size of the resource required by the task to be analyzed;
the first scheduling device is further configured to forward the cross-domain resource scheduling application to the central scheduling device;
the central scheduling device is configured to determine the second office point from the other office points according to the cross-domain resource scheduling application, where an intra-office resource margin of the second office point is greater than or equal to a size of a resource required by the task to be analyzed;
the central scheduling device is further configured to send the cross-domain resource scheduling application to the second scheduling device.
17. A computer-readable storage medium, wherein the computer-readable storage medium comprises computer instructions;
wherein the computer instructions, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210476425.4A CN114785790B (en) | 2022-04-29 | 2022-04-29 | Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210476425.4A CN114785790B (en) | 2022-04-29 | 2022-04-29 | Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114785790A true CN114785790A (en) | 2022-07-22 |
| CN114785790B CN114785790B (en) | 2024-03-08 |
Family
ID=82436006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210476425.4A Active CN114785790B (en) | 2022-04-29 | 2022-04-29 | Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114785790B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115794373A (en) * | 2022-10-20 | 2023-03-14 | 浪潮通信信息系统有限公司 | Calculation force resource hierarchical scheduling method, system, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207814A (en) * | 2012-12-27 | 2013-07-17 | 北京仿真中心 | Decentralized cross cluster resource management and task scheduling system and scheduling method |
| CN104461740A (en) * | 2014-12-12 | 2015-03-25 | 国家电网公司 | Cross-domain colony computing resource gathering and distributing method |
| CN106293950A (en) * | 2016-08-23 | 2017-01-04 | 成都卡莱博尔信息技术股份有限公司 | A kind of resource optimization management method towards group system |
| CN107066332A (en) * | 2017-01-25 | 2017-08-18 | 广东神马搜索科技有限公司 | Distributed system and its dispatching method and dispatching device |
| CN112134802A (en) * | 2020-09-23 | 2020-12-25 | 杭州雾联科技有限公司 | Edge computing power resource scheduling method and system based on terminal triggering |
| US20210311785A1 (en) * | 2020-04-01 | 2021-10-07 | Bank Of America Corporation | System and methods for generation and analysis of real-time resource requests |
-
2022
- 2022-04-29 CN CN202210476425.4A patent/CN114785790B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207814A (en) * | 2012-12-27 | 2013-07-17 | 北京仿真中心 | Decentralized cross cluster resource management and task scheduling system and scheduling method |
| CN104461740A (en) * | 2014-12-12 | 2015-03-25 | 国家电网公司 | Cross-domain colony computing resource gathering and distributing method |
| CN106293950A (en) * | 2016-08-23 | 2017-01-04 | 成都卡莱博尔信息技术股份有限公司 | A kind of resource optimization management method towards group system |
| CN107066332A (en) * | 2017-01-25 | 2017-08-18 | 广东神马搜索科技有限公司 | Distributed system and its dispatching method and dispatching device |
| US20210311785A1 (en) * | 2020-04-01 | 2021-10-07 | Bank Of America Corporation | System and methods for generation and analysis of real-time resource requests |
| CN112134802A (en) * | 2020-09-23 | 2020-12-25 | 杭州雾联科技有限公司 | Edge computing power resource scheduling method and system based on terminal triggering |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115794373A (en) * | 2022-10-20 | 2023-03-14 | 浪潮通信信息系统有限公司 | Calculation force resource hierarchical scheduling method, system, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114785790B (en) | 2024-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11099906B2 (en) | Handling tenant requests in a system that uses hardware acceleration components | |
| US11010198B2 (en) | Data processing system having a hardware acceleration plane and a software plane | |
| US10296392B2 (en) | Implementing a multi-component service using plural hardware acceleration components | |
| CN100489790C (en) | Processing management device, computer system, distributed processing method | |
| US20160308649A1 (en) | Providing Services in a System having a Hardware Acceleration Plane and a Software Plane | |
| CN114827286B (en) | Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium | |
| CN114785790A (en) | Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium | |
| WO2023035777A1 (en) | Network configuration method, proxy component, controller, electronic device and storage medium | |
| CN115460659B (en) | Wireless communication data analysis system for bandwidth adjustment | |
| US11909841B2 (en) | System, apparatus and method for adaptive peer-to-peer communication with edge platform | |
| CN114237914A (en) | Management scheduling system based on hybrid cloud | |
| CN114064317A (en) | Node calling method and related device in distributed system | |
| CN120631662B (en) | Data transmission method, device, storage medium and computer program product | |
| CN113316195B (en) | Traffic scheduling method and device | |
| Chen et al. | User satisfaction aware edge server utilization enhancement in mobile edge computing | |
| KR20230119937A (en) | Container-based resource automatic expansion method considering traffic locality, and resource auto-expanding device | |
| CN121125646A (en) | Link resource allocation methods, devices, computer equipment, storage media and software products | |
| HK40058041B (en) | Traffic scheduling method and device | |
| CN120223653A (en) | Message order preservation method, device and network equipment | |
| CN119155299A (en) | Trusted interoperation system based on cloud edge optimization intelligent algorithm | |
| CN117061522A (en) | Edge computing resource scheduling method, device and system in converter station | |
| CN117439955A (en) | Flow control method and related device for container | |
| WO2018001057A1 (en) | Message forwarding control method and device, and broadband access system | |
| JPH06103202A (en) | Computer network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |
