CN113806051B

CN113806051B - Task management method and device of computing equipment, storage medium and computing equipment

Info

Publication number: CN113806051B
Application number: CN202111109241.6A
Authority: CN
Inventors: 许红涛; 吴保杰; 陈海洋; 武璇; 肖睿; 黄灿; 王洪涛
Original assignee: Haitong Securities Co ltd
Current assignee: Haitong Securities Co ltd
Priority date: 2021-09-22
Filing date: 2021-09-22
Publication date: 2022-08-19
Anticipated expiration: 2041-09-22
Also published as: CN113806051A

Abstract

A task management method and device of a computing device, a storage medium and the computing device are provided, and the method comprises the following steps: acquiring a task dependency matrix, wherein the task dependency matrix records the interdependence relation of tasks executable by the computing equipment; acquiring an execution state matrix, wherein the execution state matrix records the execution states of tasks executable by the computer equipment, and the execution states at least comprise execution completion and non-execution completion; and judging whether each task meets the executed condition or not according to the task dependency matrix and the execution state matrix, wherein when all other tasks on which the target task depends are executed and finished, the target task meets the executed condition. By the method, whether the plurality of tasks in the computing equipment meet the executed conditions can be judged at one time, forward accurate task management is achieved, computing resources are saved, and task management efficiency is improved.

Description

Task management method and device of computing equipment, storage medium and computing equipment

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for task management of a computing device, a storage medium, and a computing device.

Background

With the development of computer devices, computing devices (e.g., computers, servers, etc. having computing capabilities) may need to manage a large number of tasks. A task may refer to a program that the computing device needs to run, data that needs to be processed, a process that needs to be executed, and so on. Because there may be dependency relationships between different tasks, when the computing device manages multiple tasks, it is necessary to determine whether each task satisfies the executed condition in real time, and then the subsequent task scheduling and task execution can be continued.

Particularly, when each enterprise builds a data platform, the data platform may carry hundreds of thousands of or even millions of task schedules along with business development, complex dependency relationships exist among tasks, and complex dependency relationships such as continuity, correlation and the like also exist inside a task group formed by a plurality of tasks. How to manage the tasks and the dependency relationship among the tasks becomes a problem to be solved urgently.

The dependency between tasks can be represented as: the executed condition is satisfied for a task only after one or more preceding instances of the task are executed, and the task depends on one or more preceding tasks.

In the prior art, a task management scheme during task scheduling is provided, that is, dependency judgment is performed from back to front based on a polling principle, and since hundreds of tasks depend on one task, tasks meeting executed conditions may need to be found only after a plurality of times of polling of each task, and then the tasks meeting the executed conditions are scheduled. According to the scheme, a large number of tasks which do not meet the dependency relationship are judged uselessly, and resources of computing equipment are occupied. In addition, a large number of tasks satisfying the dependency relationship need to be scheduled after a certain time delay due to the limitation of the polling queue mechanism, and the larger the polling task amount is, the longer the delay time is.

In summary, in the existing task management scheme, when determining the task meeting the executed condition, a large amount of redundant judgment needs to be performed, which wastes computing resources and has low task management efficiency.

Disclosure of Invention

The technical problem solved by the invention is how to provide a task management method of a computing device, so as to save computing resources and improve task management efficiency.

To solve the foregoing technical problem, an embodiment of the present invention provides a task management method for a computing device, where the method includes: acquiring a task dependency matrix, wherein the task dependency matrix records the interdependence relation of tasks executable by the computing equipment; acquiring an execution state matrix, wherein the execution state matrix records the execution state of a task executable by the computer equipment, and the execution state at least comprises execution completion and non-execution completion; and judging whether each task meets the executed condition or not according to the task dependency matrix and the execution state matrix, wherein when all other tasks on which the target task depends are executed and finished, the target task meets the executed condition.

Optionally, before the obtaining the task dependency matrix, the method further includes: constructing the task dependency matrix according to the dependency relationship among a plurality of tasks, wherein the rows of the task dependency matrix correspond to the tasks one by one, and the columns of the task dependency matrix correspond to the tasks one by one; for the task dependency matrix, when a first task depends on a second task, a row corresponding to the first task and a column corresponding to the second task commonly indicate that element values are first numerical values; when a first task does not depend on a second task, the element value indicated by the row corresponding to the first task and the column corresponding to the second task is a second numerical value.

Optionally, before the obtaining the execution state matrix, the method further includes: the execution state matrix is constructed according to the execution state of each task in a plurality of tasks, the rows of the execution state matrix correspond to the tasks one by one, the columns of the execution state matrix correspond to the execution time points one by one, and each task comprises one or more execution time points; for the execution state matrix, when a target task is executed, a row corresponding to the target task and a column corresponding to the target execution time point indicate a common element value as a third numerical value; and when the target task is not executed and completed, the element value indicated by the row corresponding to the target task and the column corresponding to the target execution time point is a fourth numerical value.

Optionally, the determining, according to the task dependency matrix and the execution state matrix, whether each task satisfies an executed condition includes: obtaining the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; acquiring column vectors corresponding to the target execution time points from the dependency satisfaction degree matrix to obtain satisfaction degree vectors of each task at the target execution time points; and comparing the satisfaction degree vector of each task at the target execution time point with the dependency degree vector of each task to judge whether each task meets the executed condition.

Optionally, the determining whether each task satisfies the executed condition according to the dependency relationship and the execution state of each task includes: multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; obtaining the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; constructing a dependency matrix according to the dependency vectors, wherein rows of the dependency matrix correspond to tasks one by one, columns of the dependency matrix correspond to the execution time points one by one, and each element of the dependency matrix is the dependency vector of the task corresponding to the row of the element at the execution time point corresponding to the column of the element; and comparing the dependency matrix with the dependency satisfaction matrix to judge whether each task meets the executed condition at each execution time point.

Optionally, all or part of the tasks executable by the computing device are in a waiting queue, and the method further includes: and adding the tasks meeting the executed conditions in the waiting queue into a queue to be executed, and taking out and executing the tasks in the queue to be executed according to the batches.

Optionally, the method further includes: determining preposed examples of a plurality of tasks, wherein each preposed example has a preset corresponding relation with one or more tasks, and one or more examples are selected from the preposed examples of each task as trigger examples; and executing the preposed instance, and adding the task corresponding to the preposed instance into the waiting queue if the execution is successful and the preposed instance is a trigger instance.

An embodiment of the present invention further provides a task management device for a computing device, including: the task dependency matrix acquisition module is used for acquiring a task dependency matrix, and the task dependency matrix records the interdependence relation of tasks executable by the computing equipment; an execution state matrix obtaining module, configured to obtain an execution state matrix, where the execution state matrix records an execution state of a task that can be executed by the computer device, and the execution state at least includes an execution completion state and an unexecuted completion state; and the judging module is used for judging whether each task meets the executed condition or not according to the task dependency matrix and the execution state matrix, wherein when all other tasks on which the target task depends are executed and finished, the target task meets the executed condition.

An embodiment of the present invention further provides a storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps of any one of the task management methods of the computing device.

The embodiment of the present invention further provides a computing device, which includes a memory and a processor, where the memory stores a computer program capable of running on the processor, and the processor executes the steps of any task management method of the computing device when running the computer program.

Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:

compared with the prior art, the embodiment of the invention provides the task management method of the computing equipment, which monitors the interdependence relation of the tasks executable by the computing equipment and the execution state of the tasks executable by the computing equipment by constructing the task dependency matrix and the execution state matrix, so that whether each task meets the executed condition or not can be judged. Therefore, whether the task meets the executed condition can be determined by judging each task once only under the condition that the dependency meets the requirement (namely the dependent task is executed and completed), the forward accurate management task is realized, the computing resources can be effectively saved, and the task management efficiency is improved.

Furthermore, the computing equipment monitors whether the dependency relationship between the tasks changes in real time, updates the task dependency matrix in time, and ensures the accuracy of the task dependency matrix, so that each task can be accurately managed.

Furthermore, the computing equipment monitors whether the execution state among tasks changes or not in real time, updates the execution state matrix in time, and ensures the accuracy of the execution state matrix, thereby ensuring the accurate management of each task. The execution state matrix can manage a task including one or more execution time points, and can accurately acquire the execution state of the task each time when the same task is executed for multiple times.

Furthermore, in the task scheduling process, the computing device adds the tasks meeting the executed conditions into the to-be-executed queue from the waiting queue, redundant computation caused by a polling principle can be avoided, and only the tasks meeting the executed conditions can enter the to-be-executed queue at a preset time point, so that the task transfer efficiency is improved, and the transfer efficiency is not reduced along with the increase of the number of the tasks.

Drawings

Fig. 1 is a flowchart of a task management method of a computing device according to an embodiment of the present invention.

FIG. 2 is a schematic flow chart diagram illustrating one embodiment of step S103 in FIG. 1;

FIG. 3 is a schematic flow chart of another embodiment of step S103 in FIG. 1;

FIG. 4 is a schematic structural diagram of a task management apparatus of a computing device according to an embodiment of the present invention;

fig. 5 is a schematic diagram of an overall architecture to which a task scheduling method is applied in an embodiment of the present invention.

Detailed Description

As background art, in the existing task management scheme, when determining a task satisfying an executed condition, a large amount of redundancy judgment needs to be performed, which wastes computing resources and reduces efficiency of task management.

In order to solve the above problem, embodiments of the present invention provide a task management method for a computing device, which monitors interdependence relationships between tasks executable by the computing device and an execution state of the tasks executable by the computing device by constructing a task dependency matrix and an execution state matrix, so as to determine whether each task satisfies an executed condition. Therefore, whether the task meets the executed condition can be determined by judging each task once only under the condition that the dependency meets the requirement (namely the dependent task is executed and completed), the forward accurate management task is realized, the computing resources can be effectively saved, and the task management efficiency is improved.

In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, several embodiments accompanied with figures are described in detail below.

Example 1: referring to fig. 1, fig. 1 is a flowchart of a task management method of a computing device in an embodiment of the present invention, where the method may be executed by a terminal or a server, where the terminal may be a computer, a mobile phone, a smart watch, and the like, and the server may refer to a server cluster or a cloud platform. The tasks mentioned in the embodiment of the invention refer to the minimum execution unit for management and scheduling, and each task is used for completing the specific logic of a certain function. For example, a task may refer to a program that the computing device needs to run, data that needs to be processed, a process that needs to be executed, and so on.

The task management method of the computing device illustrated in fig. 1 may include the steps of:

step S101: and acquiring a task dependency matrix, wherein the task dependency matrix records the interdependence relation of the tasks which can be executed by the computing equipment.

The task dependency matrix is a pre-established matrix, each row in the matrix corresponds to a task executable by the computing device, and each column in the matrix also corresponds to a task executable by the computing device.

In one embodiment, the dependencies between n (where n is a natural number) tasks (denoted as task 1, task 2, …, task n-1, task n) that a computing device can execute are seen in Table 1 below.

TABLE 1

	Task 1	Task 2	...	Task n-1	Task n
						Task 1	Whether or not	Is that	...	Is that	Whether or not
Task 2	Whether or not	Whether or not	...	Is that	Whether or not
						...	...	...	…	...	...
Task n-1	Whether or not	Whether or not	...	Whether or not	Is that
						Task n	Is that	Is that	...	Whether or not	Whether or not

In table 1, the element "yes" indicates that there is a dependency between two tasks, and the element "no" indicates that there is no dependency between two tasks. For example: task 1 is dependent on task 2, then the row corresponding to task 1 and the column corresponding to task 2 collectively indicate an element of "yes". A task cannot rely on itself, e.g., the row corresponding to task 1 is given the element "no" in common with the column corresponding to task 1. And two tasks cannot depend circularly, that is, the task i can depend on the task j, otherwise, the task j does not depend on the task i, and at this time, the values of i and j are any natural number from 1 to n, i is not equal to j.

The dependency relationship between n tasks in table 1 can be represented by a task dependency matrix, for example, each row in table 1 is taken as each row of the matrix, each column in table 1 is taken as each column of the matrix, yes in table 1 is replaced by "1", and no in table 1 is replaced by "0".

At this time, the task dependency matrix D may be expressed as:

in another embodiment, each element D1 in the task dependency matrix D1 _ij (i and j are as defined above) may represent the degree of dependency between different tasks, said degree of dependency having a value in the range of 0 ≦ D1 _ij ≤1。D1 _ij 0 indicates that task i is completely independent of task j, D1 _ij D1, indicating that task i must depend on the execution of task j _ij 0.5 indicates that task i depends partially on task j. Further description of the rows, columns and elements in the task dependency matrix D1 can be found in the above-mentioned taskThe matrix D is relied upon and will not be described in detail here.

It should be noted that the task dependency matrices D and D1 are only two examples of task dependency matrices, and all matrices that can be used to represent the interdependence relationships between tasks executed by respective computing devices are the scope of protection of the task dependency matrices in the embodiments of the present invention.

Step S102: acquiring an execution state matrix, wherein the execution state matrix records the execution state of a task executable by the computer equipment, and the execution state at least comprises execution completion and non-execution completion.

Optionally, the execution state matrix includes n rows corresponding to the n tasks, and the execution state matrix further includes a column, where at this time, an element indicated by the row corresponding to each task is an execution state of the task at a preset time point. The preset time point may be determined as needed, for example, if the preset time point is the current time, each element in the execution state matrix represents an execution state of each task at the current time.

Optionally, after determining the dependency relationship between one task and another relationship, if the dependency relationship is not changed, the task dependency matrix does not need to be updated, so that the task dependency matrix may be used when the task management method is executed for multiple times, and the preset time points for each time may be different.

Optionally, the execution state of each task in the execution state matrix needs to be updated in real time, and the execution state of each task at each preset time point needs to be acquired to update the execution state matrix.

Optionally, an element "0" indicates that the task corresponding to the row is not executed and completed, and an element "1" indicates that the task corresponding to the row is executed and completed. For example, the execution state matrix T may be represented as:

at this time, the execution state of the task 1 at the preset time point is "1", the execution state of the task 2 at the preset time point is "0", …, the execution state of the task n-1 at the preset time point is "1", and the execution state of the task n at the preset time point is "0".

Optionally, the execution state of each task may also include partial execution, for example, the element "0.5" indicates that the task corresponding to the row has been executed by 50%.

It should be noted that the execution state matrix includes, but is not limited to, the above examples, and any matrix that can be used to represent the execution state of the task executed by each computing device is the scope of protection of the task dependency matrix in the embodiment of the present invention.

It should be noted that there is no sequential execution order between step S101 and step S102, and the task dependency matrix may be obtained first and then the execution state matrix may be obtained, or the execution state matrix may be obtained first and then the task dependency matrix may be obtained, or both steps may be executed simultaneously.

Step S103: and judging whether each task meets the executed condition or not according to the task dependency matrix and the execution state matrix, wherein when all other tasks which are depended by the target task are executed, the target task meets the executed condition.

A target task refers to any task that a computing device can perform. And judging whether all other tasks depending on the target task are executed completely or not through matrix operation, if so, enabling the target task to meet the executed condition, and otherwise, enabling the target task not to meet the executed condition. Thus, whether each target task satisfies the executed condition at a preset time point can be determined, and the task satisfying the executed condition is picked out to be determined by the computing device to execute.

When the dependency relationship is judged from back to front based on the conventional polling principle, the condition of the task which is judged from back to the dependency relationship has an information blind area, so that only a black hole detection type scheduling mode with no difference judgment can be carried out. Compared with the existing scheme for managing tasks based on the polling principle, the embodiment of fig. 1 provides a task management method based on matrix calculation, which can determine whether each task meets the executed condition by only performing one judgment under the condition that the dependency of each task meets the requirements (namely, the dependent task is executed and completed), so that the forward accurate task management is realized, the calculation resources can be effectively saved, and the task management efficiency is improved.

In an embodiment, before the task dependency matrix is obtained in step S101 in fig. 1, the method may further include: constructing the task dependency matrix according to the dependency relationship among a plurality of tasks, wherein the rows of the task dependency matrix correspond to the tasks one by one, and the columns of the task dependency matrix correspond to the tasks one by one; for the task dependency matrix, when a first task depends on a second task, a row corresponding to the first task and a column corresponding to the second task commonly indicate that element values are first numerical values; when the first task is not dependent on the second task, the element value jointly indicated by the row corresponding to the first task and the column corresponding to the second task is a second numerical value.

In one embodiment, the task dependency matrix is a matrix D as described above, where the first value is "1" and the second value is "0". The values of the first numerical value and the second numerical value include, but are not limited to, this example, and may be changed into other numerical values according to the calculation requirements.

In this embodiment, a computing device executing the method illustrated in fig. 1 may construct the task dependency matrix according to a dependency relationship between a plurality of tasks managed by the computing device. The "number" in the embodiment of the present invention means one or more.

Specifically, when the computing device acquires a certain task from another terminal or creates a task, rows and columns corresponding to the task are newly added to the task dependency matrix, and the interdependency between the task and other tasks is added to the task dependency matrix. When the task managed by the computing device is cancelled, the task dependency matrix can be correspondingly deleted. Optionally, when the computing device detects that the task or the dependency relationship between the tasks in the task dependency matrix changes, the computing device updates the task dependency matrix according to the changed task or the dependency relationship between the tasks.

At the moment, the computing equipment monitors whether the dependency relationship among the tasks changes in real time, updates the task dependency matrix in time, and ensures the accuracy of the task dependency matrix, so that each task can be accurately managed.

In an embodiment, before the acquiring the execution state matrix in step S102 in fig. 1, the method may further include: the execution state matrix is constructed according to the execution state of each task in a plurality of tasks, the rows of the execution state matrix correspond to the tasks one by one, the columns of the execution state matrix correspond to the execution time points one by one, and each task comprises one or more execution time points; for the execution state matrix, when a target task is executed, a row corresponding to the target task and a column corresponding to the target execution time point indicate a third numerical value together; and when the target task is not executed and completed, the element value indicated by the row corresponding to the target task and the column corresponding to the target execution time point is a fourth numerical value.

Wherein the target execution time point is any one execution time point.

Optionally, the execution state matrix may also be constructed by the computing device when acquiring or generating each task, and optionally, when each execution time point of each task is executed, the computing device acquires an execution result (executed or not executed, etc.) of the task, and updates the execution state matrix. In addition, automatic updates to the execution state matrix may also be automatically triggered when tasks managed by the computing device change (e.g., add or subtract tasks).

Each task comprises one or more execution time points, and when a certain task comprises one execution time point, the task is executed only once at the execution time point. In one embodiment, the execution state matrix is represented as the matrix T described above. At this time, the third numerical value is "1" and the fourth numerical value is "0".

For example, task 1 includes 1 execution time point: today 15: 30. The task is executed at 15:30 today, the execution time of the task with small calculation amount can be ignored, and if the current time point is 15:30 and later, the computing device can determine the execution state of the task at the current time point as being executed and completed. In addition, if the amount of computation required for executing a task is large, the computing device may predict the execution time of the task to obtain the execution state of the task at the current time point. As another example, the computing device updates the state of a task in the execution state matrix after each execution of the task.

When a certain task includes multiple execution time points, the task needs to be executed multiple times only at its multiple execution time points. For example, a set periodic task needs to be executed once per cycle. At this time, there may be execution states corresponding to t execution time points (respectively, time 1 and …, and time t and t are positive integers greater than or equal to 2) in the execution state matrix to indicate whether the task is executed at each execution time point. The execution states of the tasks at the current time point are shown in table 2, and the t execution time points are arranged according to the execution sequence.

TABLE 2

	Time 1	...	Time t
				Task 1	Complete the process	...	Complete the process
Task 2	Complete the process	...	Unfinished
				...	...	...	...
Task n-1	Complete the process	...	Unfinished
				Task n	Unfinished	...	Unfinished

The execution state matrix T may now be converted to T1:

for each column of the matrix T1, for an execution time point, the other descriptions of the matrix T1 are the same as the matrix T and are not repeated here.

For the matrix T1, since the execution of each task is continuous in time, that is, the execution time point after the execution time point before the execution time point after the execution time point before the execution time point is not allowed to be completed, the vectors in the same row in the matrix T1 can only consist of 0 to T consecutive 1 and T to 0 consecutive 0, and the intersection of 0 and 1 does not occur.

It should be noted that the values of the third and fourth values include, but are not limited to, this example, and may be changed into other values according to the calculation requirement.

In this embodiment, the computing device monitors whether the execution state between tasks changes in real time, updates the execution state matrix in time, and ensures the accuracy of the execution state matrix, thereby ensuring accurate management of each task. The execution state matrix can manage a task including one or more execution time points, and can accurately acquire the execution state of the task each time when the same task is executed for multiple times.

In an embodiment, referring to fig. 2, the step S103 in fig. 1 of determining whether each task satisfies the executed condition according to the task dependency matrix and the execution state matrix may include the following steps S201 to S204, where:

step S201, obtaining the sum of elements in each row from the task dependency matrix to obtain a dependency vector of each task.

For example, continuing to refer to the task dependency matrix D in the above example, the sum of all elements on the ith row of the matrix D is the number of dependent tasks that need to be satisfied by the task i, the number of dependent tasks that need to be satisfied by each of the n tasks can be represented by the matrix P, and each row of the matrix P corresponds to each row in the task dependency matrix D one to one. The elements on each row in the matrix P represent the number of dependent tasks that the task corresponding to the row needs to satisfy. Only one column vector, i.e. the dependency vector, is present in the matrix P.

And step S202, multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction matrix.

For the task dependency matrix D and the execution state matrix T as in the above example, the dependency satisfaction matrix M is represented as:

each row in the matrix M represents each task, and the element corresponding to each row is the number of dependent tasks that the task corresponding to the row satisfies at a preset time point (e.g., a current time point).

For the task dependency matrix D and the execution state matrix T1 in the above example, the dependency satisfaction matrix M1 is represented as:

step S203, obtaining the column vector corresponding to the target execution time point from the dependency satisfaction degree matrix, and obtaining the satisfaction degree vector of each task at the target execution time point.

For the matrix M, it only includes a column vector of one execution time point (i.e. the target execution time point), which is a satisfaction degree vector of n tasks at the target execution time point.

For the matrix M1, which includes the column vectors corresponding to the t execution time points, t satisfaction degree vectors can be obtained by taking the distribution of each execution time point as the target execution time point.

It should be noted that there is no sequential execution order between step S201 and step S202, step S201 may be executed first and then step S202 is executed, or step S202 may be executed first and then step S201 is executed, or both steps may be executed simultaneously.

And step S204, comparing the satisfaction degree vector of each task at the target execution time point with the dependency degree vector of each task to judge whether each task meets the executed condition.

Optionally, comparing the satisfaction degree vector of each task at the target execution time point with the dependency degree vector of each task, which may include: and calculating the difference between the satisfaction degree vector of each task at the target execution time point and the dependency degree vector of each task.

For example, for matrix M, the difference is the difference between matrix P and matrix M subtracted:

in the matrix obtained by P-M, if an element corresponding to a certain row is 0, it indicates that all tasks that the task corresponding to the row depends on at a preset time point (e.g., a current time point) have been executed, that is, the task corresponding to the row satisfies an executed condition. If the element corresponding to a certain row is not 0, it indicates that the tasks that the task corresponding to the row depends on at the preset time point are not all executed, that is, the task corresponding to the row does not meet the executed condition. The matrix of P-M above, task 1, task 2, and task n all satisfy the executed condition.

For the matrix M1, the matrix operation of P-M may be performed with each column vector in M1 as the matrix M, respectively, to determine whether each task of the target execution time point satisfies the executed condition.

It should be noted that steps S1031 to S1034 in fig. 2 provide a matrix operation manner, according to which the computing device can automatically determine whether each task at each execution time point satisfies the executed condition.

In an embodiment, referring to fig. 3, the determining, according to the dependency relationship and the execution state of each task in step S103 in fig. 1, whether each task satisfies the executed condition may further include:

and S301, multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction matrix. Step S301 is described with reference to step S201 in fig. 2, and is not described herein again. The dependency satisfaction matrix may be matrix M or matrix M1.

Step S302, the sum of elements of each row is obtained from the task dependency matrix to obtain the dependency vector of each task. The dependency vector is exemplified by the column vector in the matrix P.

Step S303, a dependency matrix is constructed according to the dependency vectors, rows of the dependency matrix correspond to tasks one by one, columns of the dependency matrix correspond to the execution time points one by one, and each element of the dependency matrix is the dependency vector of the execution time point corresponding to the task corresponding to the row of the element in the column of the element.

It should be noted that there is no sequential execution order between step S301 and step S302, step S301 may be executed first and then step S302 is executed, or step S302 is executed first and then step S301 is executed, or both steps may be executed simultaneously.

For the above dependency vectors (column vectors in matrix P), considering that the dependency vectors P of multiple execution time points are all the same for the current time point, the dependency matrix E of t execution time points is a matrix composed of t identical dependency vectors, and the matrix E is represented as:

step S304, comparing the dependency matrix with the dependency satisfaction matrix to judge whether each task meets the executed condition at each execution time point.

Comparing the dependency matrix and the dependency satisfaction matrix may include: calculating the difference between the dependency matrix and the dependency satisfaction matrix, taking the dependency matrix E and the dependency satisfaction matrix M1 as an example, and expressing the matrix R of the difference between the dependency matrix E and the dependency satisfaction matrix M1 as follows:

in the matrix R, an element of 0 indicates that the execution time point corresponding to the column in which the task corresponding to the row of the element is located satisfies the executed condition, otherwise indicates that the executed condition is not satisfied.

The rest of the description in fig. 3 refers to the explanation in fig. 2 and is not described here in detail.

Example 2: all or part of the tasks executable by the computing device are in a waiting queue, and the task management method of the computing device can further comprise the following steps: and adding the tasks meeting the executed conditions in the waiting queue into a queue to be executed, and taking out the tasks in the queue to be executed according to the batches for execution.

Wherein the wait queue is used to manage all or part of the tasks on the computing device side. The queue to be executed is used for managing the tasks meeting the executed conditions on the computing device side, the computing device determines the tasks meeting the executed conditions in the waiting queue according to the mode, and the tasks are stored in the queue to be executed and are ready to be executed.

Optionally, the waiting queue and the queue to be executed may be both a first-in first-out queue, a first-in last-out queue, or other existing queues.

Optionally, each time one or more tasks are added to the to-be-executed queue from the waiting queue, the tasks added to the to-be-executed queue are cleared in the waiting queue to release the storage space of the waiting queue.

Optionally, after all the tasks related to the current batch are executed, the execution of the tasks of the next batch can be started, so as to ensure the overall integrity of the tasks of the same dispatching batch.

Optionally, the number of tasks executed in each batch may be determined according to the computing capability and the memory size of the computing device, for example, if the number of tasks currently allowed to be executed by the computing device is 100, the number of tasks included in the next batch is 100.

In a specific implementation, the computing device determining the number of tasks per batch may include: acquiring the number of tasks allowed to be executed currently, and recording the number as the current executable number; comparing the current executable number with the number of the current management tasks in the queue to be executed; if the current executable number is less than or equal to the number of the current management tasks in the queue to be executed, taking the current executable number as the number of the tasks executed in the batch; and if the current executable number is larger than the number of the current management tasks in the queue to be executed, taking the number of the current management tasks in the queue to be executed as the number of the tasks executed in the batch.

Example 3: the task management method of the computing device may further include: determining preposed examples of a plurality of tasks, wherein each preposed example has a preset corresponding relation with one or more tasks, and one or more examples are selected from the preposed examples of each task as trigger examples; and executing the preposed instance, and adding the task corresponding to the preposed instance into the waiting queue if the execution is successful and the preposed instance is a trigger instance. That is, the tasks satisfying the trigger condition are stored in the waiting queue, not all tasks managed by the computing device.

Wherein an instance has the same meaning as a task. The present invention is by way of example only to distinguish from tasks in the wait queue. The front example is a front job responsible for evoking the scheduling of the task corresponding to the front example.

For a single task, the task corresponds to one or more preposed instances, and one or more of the preposed instances are selected as trigger instances. In one embodiment, there is one and only one trigger instance in the preamble instance. After the trigger examples in the front-end examples are executed successfully, because each front-end example and the corresponding task have the preset mapping relation, the corresponding task is put into the waiting queue based on the fact that the trigger examples are executed completely.

Therefore, before the computing equipment schedules the tasks, the computing equipment does not need to define the execution sequence of all the preposed instances of each task, and can complete the scheduling and execution of each task only by knowing the corresponding relation between the preposed instances and the tasks and the execution state of the preposed instances (especially trigger instances), so that the various and complex scheduling requirements can be met, and the task scheduling accuracy is improved. The present embodiment may include a preset relationship table, where the relationship table describes a correspondence between the preceding instance and one or more tasks.

Optionally, the pre-instance may include a dependent instance, the trigger instance, and a related instance. The dependent instance means that the execution of the tasks of the same batch can be started only after all the dependent instances are executed. The trigger instance is an instance responsible for evoking corresponding task scheduling, for example, after the trigger instance is executed, a step of putting a task corresponding to the preceding instance into a waiting queue may be triggered. The related instances are instances where there are dependencies of execution with other instances of the dispatched batch.

Therefore, the embodiment of the invention can automatically determine whether the tasks can be scheduled or not and meet the executed conditions through the relationship among the tasks and the instances, can effectively ensure the ordered scheduling and execution of the tasks, and can not generate the chaos of the scheduling and execution of the tasks.

The trigger instance may be any one of the preceding instances, and preferably, the preceding instance with the highest probability of being executed last in the preceding instances is used as the trigger instance, so that timeliness of task scheduling can be improved. Specifically, before determining the front instances of the plurality of tasks, the method further comprises: determining the probability of the last execution of each preposed instance in all the preposed instances corresponding to each task; and determining the preposed instance with the maximum probability as the trigger instance of the task. For example, the probability that the last of the various preceding instances was executed may be determined according to an algorithm. And taking the front example with the maximum probability of being executed as a trigger example, if the trigger example is the last executed task in the front examples, and after the execution of the trigger example is completed, putting the task corresponding to the front example into a waiting queue, adding the task into the queue to be executed if all the front examples of the task are executed, thereby reducing the waiting time of the task in the waiting queue and improving the task scheduling efficiency.

Further, if there is a task T, it has n leading instances T ₁ ……T _n The following formula is used to determine the preamble instance T _j Probability P (T | T) of last being performed _j ) It is expressed by formula (1):

wherein j is more than or equal to 1 and less than or equal to n, M is a statistical period, W (T) _i ) Is the weight value of the ith execution example of the task T in the statistical period M, W (T) _ji ) Is a leading example T _j The weight value of the i-th execution instance within the statistical period M,

is the time that the task T is added into the waiting queue in the ith execution instance and the preposed instance T _j Is measured by a normalized value of the difference between completion times. As will be appreciated by those skilled in the art, the time at which a task T is added to the wait queue and the leading instance T _j The smaller the normalized value of the difference between completion times of (A) is, the leading instance T _j The greater the probability of triggering a task being task T.

Optionally, before determining the front instances of the plurality of tasks, a step of publishing the tasks and/or rolling back may be further included. The release task mainly comprises the following steps:

backing up scripts and databases before upgrading; issuing a script program to a corresponding executing client, such as an agent server; checking after upgrading, including task dependency checking, task trigger checking, whether a script corresponding to the task is consistent with metadata and the like; and synchronizing the latest script to the corresponding latest version library catalogue.

The rollback task mainly comprises the following steps: backing back metadata and scripts according to the backup; and (4) checking after rollback, wherein the checking comprises task dependency relation checking, task trigger relation checking, whether a script corresponding to the task is consistent with the metadata and the like, and if an error exists during the checking, performing task rollback.

The automatic task issuing and returning can guarantee the normalization of the task issuing and improve the task issuing efficiency.

Further, if any of the instances of the prefix fails to execute, the instances of the prefix that failed to execute are added to the exception queue. The failure to execute any of the preceding instances includes, but is not limited to, the following exceptions: the task execution is interrupted, the task execution result is not checked, the task is not started up until the specified time, and the like.

Further, each pre-instance has a priority, and in a specific implementation, the pre-instance with the execution failure can be read, and the warning information is sent out according to the priority of the pre-instance with the execution failure.

It should be noted that the priority may be preset or determined in real time through calculation.

Further, the following formula may be used to calculate the preamble instance T _j Priority P (T) _j ) It is expressed by formula (2):

wherein N is the leading instance T _j The number of corresponding ending tasks B; p (T) _j |B _i ) For representing the preceding instance T _j For it finishes task B _i An influence factor of (A), W (B) _i ) For representing the preceding instance T _j Corresponding end task B _i The weight value of (2). The impact factor may take any value between 0 and 1. As will be understood by those skilled in the art, the ending task is a final task corresponding to the preceding instance, and the ending task is determined by a service to which the preceding instance belongs, such as a supervision service, a transaction service, and the like. And when the ending task is completed, completing the business corresponding to the preposed instance.

In a specific implementation, each leading instance has a unique corresponding task identifier, each type of abnormal condition has a unique corresponding error identifier, and adding the leading instance into the abnormal queue may include adding the task identifier into the abnormal queue, and may further include adding the error identifier corresponding to the abnormal condition of the leading instance that fails to be executed into the abnormal queue.

Further, if the priority of the pre-instances is preset, the method may further include reading task identifiers and/or error identifiers of the pre-instances in the exception queue, searching a preset priority corresponding to the task identifiers in a preset priority list according to the task identifiers, where the priority list may reflect a mapping relationship between the pre-instances and the preset priority, and sending out warning information according to the preset priority corresponding to the task identifiers.

In a specific implementation, the warning information may be sent by one or more of the following methods: a text display reminder, a sound reminder, an icon flashing reminder, etc. It can be understood that the prepositive instances with different priorities can correspond to the warning information in different modes, for example, the priority can be divided into a first level, a second level, a third level and a fourth level, the larger the number of the levels is, the higher the priority is, for the tasks with the third level and the fourth level of the priority, the warning information can be sent out by selecting a voice reminding mode, and for the tasks with the first level and the second level of the priority, the warning information can be sent out by selecting a text display reminding mode.

It can be understood that, in a specific implementation manner of the embodiment of the present invention, when a leading instance that fails to be executed joins an exception queue, the leading instance that newly joins the exception queue that fails to be executed is read, and warning information is sent; in another embodiment, a time period may be set, and the leading instance in the exception queue is read periodically to send out an alert message.

Further, the method may further include searching for an abnormal reason and/or a solution corresponding to the error identifier in a preset abnormal list according to the error identifier, where the preset abnormal list may include a plurality of error identifiers and abnormal reasons and/or solutions having a mapping relationship with each error identifier.

Further, the alert information may include one or more of: the task identification, the task name corresponding to the task identification, the error identification, the abnormal reason corresponding to the error identification and the solution corresponding to the error identification.

Further, in the embodiment of the present invention, the task scheduling method may further include recording the warning information, and automatically counting and analyzing abnormal conditions according to a period.

In embodiment 3, as the number of tasks in the waiting queue increases and the complexity of the dependency relationship between the tasks increases, more and more tasks need to be determined and accumulated in the waiting queue, but the reason is that the triggered execution of the tasks is a "unguided" manner, and the dependency relationship between the tasks is not considered, so that a large number of tasks that do not satisfy the condition exist in the waiting queue. At this time, if the tasks in the waiting queue are judged based on the polling principle, only the tasks meeting the executed conditions are removed from the waiting queue and added into the waiting queue. This may cause the inactive tasks to exist in the waiting queue for a long time, and each round of determination requires repeatedly performing the determination actions of the inactive tasks, which consumes a large amount of computation, causes the polling period to be seriously lengthened, and greatly increases the task scheduling and execution efficiency.

In order to solve this problem, the method in embodiment 1 is used to determine that the task meeting the executed condition is added to the queue to be executed from the waiting queue, so as to avoid redundant computation caused by the polling principle, and only the task meeting the executed condition can enter the queue to be executed at a preset time point, thereby improving the task circulation efficiency, and not reducing the circulation efficiency with the increase of the number of tasks.

In an embodiment, the solutions of embodiments 1 to 3 may be applied to a task scheduling scenario, for example, a scenario in which task scheduling is performed based on secondary development of open source software or by using commercial software. The task scheduling refers to timely and accurate execution of a series of batch tasks according to the batch arrangement sequence. Typical task scheduling systems are Elastic-joba, Spring Batch, TaskCtl, Airflow, etc. The inventor finds that, because the current task scheduling method based on open source software adopts a scheduling method based on a task flow, such as oozie, azkaban and the like, the logical relationship among tasks in the task flow needs to be defined. The method has high coupling among different task flows, when the number of tasks and the task relation are increased in a complex way, it is very difficult to accurately and clearly define the task flows, and if the task flows cannot be accurately defined, errors are easy to occur in the scheduling process. In addition, the scheduling method based on open source software only provides a task scheduling function of a core, multiple capabilities of auxiliary task development, task relation organization, task state maintenance, task monitoring, task management, service monitoring, log downloading and the like are relatively deficient, and the workload of secondary development is still huge. In addition, although some software (such as business software) is mature and stable, when the task relationship is complex, it is difficult to meet various complex or personalized scheduling requirements in time.

Therefore, the secondary development based on the open source software is mainly suitable for small-scale application with simple and clear business logic. Although some software is mature and stable, on one hand, the software cannot be completely and autonomously controlled, and once problems occur, emergency is difficult; on the other hand, the relation among batch jobs (namely, tasks) of the data warehouse is complex, the processing special requirements of the batch tasks are more, and mature software can meet various requirements in time. Therefore, in terms of autonomous controllable and quick response to service demands, the scheduling system of the autonomous development core is a preferred choice when the data platform has high requirements on data processing.

Because the existing task scheduling has the problems, the task management method of the embodiment of the invention can be used for selecting the task meeting the executed condition from a large number of tasks and scheduling the tasks. Therefore, complex relationships among a large number of tasks in batch operation of the data warehouse can be accurately managed, and the service demand response speed is improved. The task scheduling needs to depend on the corresponding relation between the preposed instance and the task, is not limited by the definition of the task flow, and can support various scheduling mechanisms including but not limited to discrete scheduling, quasi-real-time scheduling, interactive scheduling and the like.

Embodiment 4, please refer to fig. 4, where fig. 4 is a schematic structural diagram of a task management device of a computing device according to an embodiment of the present invention, and the task management device 40 of the computing device may include:

a task dependency matrix obtaining module 401, configured to obtain a task dependency matrix, where the task dependency matrix records an interdependence relationship between tasks that can be executed by the computing device;

an execution state matrix obtaining module 402, configured to obtain an execution state matrix, where the execution state matrix records an execution state of a task that can be executed by the computer device, and the execution state at least includes an execution completion state and an unexecuted completion state;

a determining module 403, configured to determine whether each task satisfies the executed condition according to the task dependency matrix and the execution state matrix, where when all other tasks that a target task depends on are executed and completed, the target task satisfies the executed condition.

Optionally, before obtaining the task dependency matrix, the task management apparatus 40 of the computing device may further include: the task dependency matrix building module is used for building the task dependency matrix according to the dependency relationship among a plurality of tasks, the rows of the task dependency matrix correspond to the tasks one by one, and the columns of the task dependency matrix correspond to the tasks one by one; for the task dependency matrix, when a first task depends on a second task, a row corresponding to the first task and a column corresponding to the second task commonly indicate that element values are first numerical values; when a first task does not depend on a second task, the element value indicated by the row corresponding to the first task and the column corresponding to the second task is a second numerical value.

Optionally, before obtaining the execution state matrix, the task management apparatus 40 of the computing device may further include: the execution state matrix construction module is used for constructing an execution state matrix according to the execution state of each task in a plurality of tasks, the rows of the execution state matrix correspond to the tasks one by one, the columns of the execution state matrix correspond to the execution time points one by one, and each task comprises one or more execution time points; for the execution state matrix, when a target task is executed, a row corresponding to the target task and a column corresponding to the target execution time point indicate a common element value as a third numerical value; and when the target task is not executed and completed, the element value indicated by the row corresponding to the target task and the column corresponding to the target execution time point is a fourth numerical value.

Optionally, the determining module 403 may include: the dependency vector acquiring unit is used for acquiring the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; the dependency satisfaction degree matrix acquisition unit is used for multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; the satisfaction degree vector acquisition unit is used for acquiring column vectors corresponding to the target execution time points from the dependency satisfaction degree matrix to obtain satisfaction degree vectors of each task at the target execution time points; and the first judgment unit is used for comparing the satisfaction degree vector of each task at the target execution time point with the dependency degree vector of each task to judge whether each task meets the executed condition.

Alternatively, the determining module 403 may further include: the dependency satisfaction degree matrix obtaining unit is used for multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; the dependency vector acquiring unit is used for acquiring the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; a dependency matrix construction unit, configured to construct a dependency matrix according to the dependency vector, where rows of the dependency matrix correspond to tasks one to one, columns of the dependency matrix correspond to the execution time points one to one, and each element of the dependency matrix is a dependency vector of an execution time point, corresponding to a task in a row where the element is located, in a column where the element is located; and the second judging unit is used for comparing the dependency matrix with the dependency satisfaction matrix to judge whether each task meets the executed condition at each execution time point.

Optionally, all or part of the tasks executable by the computing device are in the waiting queue, and the task management apparatus 40 of the computing device may further include: and the queue management module is used for adding the tasks meeting the executed conditions in the waiting queue into the queue to be executed, and taking out and executing the tasks in the queue to be executed according to the batches.

Optionally, the task management apparatus 40 of the computing device may further include: the device comprises a preposed instance determining module, a task selecting module and a task selecting module, wherein the preposed instance determining module is used for determining preposed instances of a plurality of tasks, each preposed instance has a preset corresponding relation with one or more tasks, and one or more instances are selected from the preposed instances of each task as trigger instances; and the task determining module is used for executing the preposed instance, and adding the task corresponding to the preposed instance into the waiting queue if the execution is successful and the preposed instance is a trigger instance.

Those skilled in the art understand that the task scheduling apparatus in this embodiment can be used to implement the task management method of the computing device in the embodiments shown in fig. 1 to fig. 3.

Further, an embodiment of the present invention further discloses a storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the technical solution of the task management method of the computing device in the embodiments shown in fig. 1 to fig. 3. Preferably, the storage medium may include a computer-readable storage medium such as a non-volatile (non-volatile) memory or a non-transitory (non-transient) memory. The storage medium may include ROM, RAM, magnetic or optical disks, or the like. Further, an embodiment of the present invention further discloses a terminal, which includes a memory and a processor, where the memory stores a computer program capable of running on the processor, and the processor executes the technical solution of the task management method of the computing device in the embodiments shown in fig. 1 to 3 when running the computer program. The terminal includes, but is not limited to, a mobile phone, a computer, a tablet computer and other terminal devices.

It should be noted that, in the embodiment of the present application, the task scheduling method may adopt a micro-service architecture. The micro-service architecture divides a single application program into a group of small services, each micro-service is only concerned with completing one task, but the services can be coordinated and matched with each other, each service runs in an independent process, and the services are communicated with each other by adopting a lightweight communication mechanism.

Those skilled in the art will appreciate that the existing micro services usually communicate with each other by means of Remote Procedure Call (RPC), and this communication is synchronous, for example, when the micro service a wants to communicate with the micro service B, a communication connection must be established first, and then the addressing problem needs to be solved, that is, how the micro service a is connected to the server to which the micro service B belongs and a specific port, when the RPC communication is performed after the addressing is completed, the transmitted data needs to be serialized. The communication mode of the RPC can only support synchronous communication, cannot be decoupled, and has poor performance.

The micro-services in the embodiment of the present invention interact with each other by using a message queue mechanism, so as to support each task (or refer to an instance) executed by the micro-service. The message queue may include: a wait queue, a pending execution queue, an execution queue, etc., but is not so limited. Specifically, the task scheduling method in the embodiment of the present invention is based on a micro service architecture, each micro service in the micro service architecture is implemented by a message queue in a task scheduling process, and each micro service is decoupled. In the embodiment of the invention, the execution sequence of all the preposed instances of the task does not need to be defined, and the task is also decoupled, so that not only are the micro services decoupled, but also the tasks are decoupled, the various complex scheduling requirements can be met, and the performance of the system is greatly improved.

Those skilled in the art can appreciate that the micro-service architecture has the advantages of controllable complexity, independent deployment, flexible technology model selection, good fault tolerance and good expansion performance.

Specifically, the micro-service architecture decomposes services, and avoids endless accumulation of original complexity. Each microservice is dedicated to a single function and the individual microservice boundaries are clearly represented by well-defined interfaces. Due to the small size and low complexity, each microservice can be completely controlled by a small-scale development team, and high maintainability and development efficiency are easy to maintain.

Each microservice is built around a specific business, and each microservice can have an independent running process and thus can be deployed independently to a production environment, a production-like environment, and the like. When a micro service is changed, the whole application does not need to be compiled and deployed.

Under the micro-service architecture, technology type selection is decentralized, each micro-service can freely select the most suitable technology stack according to the self requirement, the risk of upgrading the technology stack is low, and even one micro-service can be completely reconstructed.

Under the micro-service architecture, because the fault is isolated in a single service, other services can realize the fault tolerance of the application layer through mechanisms such as retry, stable degradation and the like, and the condition that the fault is diffused in the process to form global unavailability of the application in the traditional architecture of a single process can be avoided.

Under the micro-service architecture, because the micro-service architecture has flexibility, each micro-service can be independently expanded according to actual requirements, and then the difference of different components of the application in the expansion requirements is solved, so that the whole application can be copied to different nodes.

The task scheduling device adopting the task scheduling method can also relate to micro-services with multiple functions, and a large amount of complex and interactive event processing and state conversion are involved in the micro-services. By the task scheduling method, each micro-service is stateless, the services are communicated through the message queue, the method has the characteristics of asynchronism, concurrency and the like, is more efficient, and can avoid a unified and centralized service management mechanism.

Referring to fig. 5, fig. 5 is a schematic diagram illustrating an overall architecture to which a task management method according to an embodiment of the present invention is applicable.

As will be understood by those skilled in the art, the construction, extraction, transformation and loading (ETL) of a data platform is indifferent, and as business develops, the data platform carries thousands or even hundreds of thousands of ETL task schedules, and these tasks have various forms and certain dependency relationships with each other. How to enable a large number of ETL tasks to efficiently and accurately complete scheduling without problems, even under the condition that errors occur in task scheduling or execution, the tasks can complete self-recovery, and meanwhile, error alarm and complete log query are executed, so that the method becomes a key for improving the development efficiency and the service capability of a data platform. The scheme of the embodiment of the invention provides a self-research architecture based on microservice.

Specifically, the embodiment of the present invention adopts a micro-service architecture, and can implement functions of task management 501, task publishing 502, task querying 503, task monitoring 505, system configuration 505, and the like. The embodiment of the present invention may further provide an open system Interface, that is, an Application Programming Interface (API) for providing Representational State Transfer (REST) externally, so as to facilitate docking of a peripheral system.

Specifically, the task management 501 may refer to managing a task having a right, including adding, deleting, modifying, resuming running, and the like; supporting priority management of an instance (or called a task), automatically calculating a key path according to a final service object, and setting an instance priority matched with a data service level according to the key path; automatic retry after execution failure is supported, and the automatic retry times, retry intervals and the like can be set; a periodic task can be temporarily prohibited or started, a batch executed by the task after being prohibited stops at the current prohibited task, and the task is automatically continuously executed according to a preset sequence after being started; and performing concurrency management, and setting the concurrency of task execution according to multiple dimensions such as the execution client, the task group, the task type and the like.

The task issuing 502 refers to supporting automatic task template uploading, checking and issuing, and solves the problems that when a task is on line, a traditional scheduling system needs to manually configure basic information of the task, dependency relationship of the task and the like, and is complex and prone to errors.

The task query 503 is to support the retrieval of the task plan and the execution flow of the current day, and support the retrieval of the periodic task information, including task overview, historical execution flow, execution log, change record, dependency tree query, and the like.

The task monitoring 505 can be implemented based on a status monitoring service 515, an exception handling service 516, and a host monitoring service 517, so as to perform comprehensive operation and maintenance monitoring management.

It should be further noted that the task scheduling method disclosed in the embodiment of the present invention may further support user permission management (not shown), where users in different roles have different operation permissions, may also support multi-tenant isolation, support a platform scheduling mode of distributed management and unified scheduling, support user audits in various granularities, and meet audit requirements.

The micro service adopted in the micro service architecture of the embodiment of the invention can comprise: the task instance trigger event monitoring service 511, the interface trigger service 512, the task control service 513, the task issuing service 514, the status monitoring service 515, the exception handling service 516, and the host monitoring service 517, but are not limited thereto.

The task instance trigger event listening service 511 is configured to listen whether a trigger condition of a task is satisfied, and if so, put a corresponding load ready message into a task instance ready message queue (not shown). Task instance trigger time listening services 511 may include daily batch task listening, multi-batch task listening, interactive task listening, and the like.

The interface trigger service 512 is configured to listen to a task instance ready message queue (not shown), take a corresponding number of tasks from the task instance ready message queue (not shown) according to priorities of the tasks, and invoke a task execution service (not shown) to execute the tasks.

The task control service 513 is configured to listen to the waiting queue 521, determine whether the task in the waiting queue 521 meets the executed condition, and if so, delete the task from the waiting queue 521 and add the task to the queue to be executed 522.

The task issuing service 514 is configured to monitor the queue to be executed 522, determine the number of currently executable tasks according to the current concurrency condition, and invoke a task execution service (not shown) to execute the tasks.

The status monitoring service 515 is used to monitor exception queues and to alert for execution failures, etc.

The exception handling services 516 are responsible for exception operations such as exception re-fetch, re-run, skip, etc. of tasks. The exception handling service 516 may perform exception analysis on the task that fails to be executed, and specifically may include providing an exception reason for each exception early warning, supporting online exception classification and analysis registration after the fact, automatically counting exception analysis conditions according to a period, performing classification coding on the exceptions, and formulating an exception phenomenon, an exception reason, a suggested solution, and the like for each type of exception; the method can also be used for abnormal log management, checking and downloading the historical task log which fails to execute, and quickly analyzing and positioning problems.

The host monitoring service 517 is configured to monitor whether each executing client is normal, and automatically recover or alarm according to a predetermined policy.

Therefore, the embodiment of the present invention can implement comprehensive operation and maintenance monitoring management through the state monitoring service 515, the exception handling service 516, and the host monitoring service 517.

It should be noted that, in the micro-service architecture disclosed in the embodiment of the present invention, each micro-service is responsible for a clear function, has a low degree of mutual coupling, and has good extensibility, stability and fault tolerance, and can implement function modularization, implement high cohesion inside different micro-services, and loosely couple services.

Those skilled in the art will appreciate that a plurality of microservices involve a large number of complex and interactive time processing and state transitions within them, which in turn place extremely high demands on real-time performance and efficiency, and therefore require an efficient communication mechanism. The micro-services in the embodiment of the invention interact by adopting a message queue mechanism, are used for supporting each task executed by the micro-services, have the characteristics of asynchronism, concurrency and the like, are more efficient, and have no stateful core service. The message queue may include: waiting queue 521, pending execution queue 522, execution queue 523, exception queue 524, and delay queue 525.

The waiting queue 521 is used for storing tasks meeting the trigger condition, that is, storing corresponding tasks after the trigger instance is executed; the queue to be executed 522 is used for storing tasks meeting executed conditions; the execution queue 523 is used for executing the tasks in the queue to be executed 522 according to the concurrency; the exception queue 524 is used for storing tasks that fail to execute; the delay queue 525 is used to store tasks that are delayed for execution. For more contents of the wait queue 521, the queue to be executed 522, the execution queue 523, and the exception queue 524, reference may be made to the related descriptions of embodiments 1 to 3, which are not repeated herein.

Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected by one skilled in the art without departing from the spirit and scope of the invention, as defined in the appended claims.

Claims

1. A method of task management for a computing device, the method comprising:

acquiring a task dependency matrix, wherein the task dependency matrix records the interdependence relation of tasks executable by the computing equipment;

acquiring an execution state matrix, wherein the execution state matrix records the execution state of a task executable by the computing equipment, and the execution state at least comprises execution completion and non-execution completion;

judging whether each task meets the executed condition or not according to the task dependency matrix and the execution state matrix, wherein when all other tasks which are depended by the target task are executed, the target task meets the executed condition;

executing the execution state matrix according to the task dependency matrix and the execution state matrix by adopting a first mode or a second mode, and judging whether each task meets the executed condition;

the first mode includes: obtaining the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; acquiring column vectors corresponding to the target execution time points from the dependency satisfaction degree matrix to obtain satisfaction degree vectors of each task at the target execution time points; comparing the satisfaction degree vector of each task at the target execution time point with the dependency degree vector of each task to judge whether each task meets the executed condition;

the second mode includes: multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; obtaining the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; constructing a dependency matrix according to the dependency vectors, wherein rows of the dependency matrix correspond to tasks one by one, columns of the dependency matrix correspond to the execution time points one by one, and each element of the dependency matrix is the dependency vector of the task corresponding to the row of the element at the execution time point corresponding to the column of the element; and comparing the dependency matrix with the dependency satisfaction matrix to judge whether each task meets the executed condition at each execution time point.

2. The method of claim 1, wherein prior to obtaining the task dependency matrix, further comprising:

constructing the task dependency matrix according to the dependency relationship among a plurality of tasks, wherein the rows of the task dependency matrix correspond to the tasks one by one, and the columns of the task dependency matrix correspond to the tasks one by one;

for the task dependency matrix, when a first task depends on a second task, a row corresponding to the first task and a column corresponding to the second task commonly indicate that element values are first numerical values; when a first task does not depend on a second task, the element value indicated by the row corresponding to the first task and the column corresponding to the second task is a second numerical value.

3. The method according to claim 1 or 2, wherein before the obtaining the execution state matrix, further comprising:

constructing an execution state matrix according to the execution state of each task in a plurality of tasks, wherein the rows of the execution state matrix correspond to the tasks one by one, the columns of the execution state matrix correspond to the execution time points one by one, and each task comprises one or more execution time points;

for the execution state matrix, when a target task is executed, a row corresponding to the target task and a column corresponding to the target execution time point indicate a common element value as a third numerical value;

and when the target task is not executed and completed, the element value indicated by the row corresponding to the target task and the column corresponding to the target execution time point is a fourth numerical value.

4. The method of claim 1, wherein all or a portion of the tasks executable by the computing device are in a waiting queue, the method further comprising:

and adding the tasks meeting the executed conditions in the waiting queue into a queue to be executed, and taking out and executing the tasks in the queue to be executed according to the batches.

5. The method of claim 4, further comprising:

determining preposed examples of a plurality of tasks, wherein each preposed example has a preset corresponding relation with one or more tasks, and one or more examples are selected from the preposed examples of each task as trigger examples;

and executing the preposed instance, and if the execution is successful and the preposed instance is a trigger instance, adding the task corresponding to the preposed instance into the waiting queue.

6. A task management apparatus of a computing device, comprising:

the task dependency matrix acquisition module is used for acquiring a task dependency matrix, and the task dependency matrix records the interdependence relation of tasks executable by the computing equipment;

an execution state matrix obtaining module, configured to obtain an execution state matrix, where the execution state matrix records an execution state of a task that can be executed by the computing device, and the execution state at least includes an execution completion state and an unexecuted completion state;

the judging module is used for judging whether each task meets the executed condition or not according to the task dependency matrix and the execution state matrix, wherein when all other tasks which are depended by the target task are executed, the target task meets the executed condition;

the judging module comprises: the dependency vector acquisition unit is used for acquiring the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; the dependency satisfaction degree matrix obtaining unit is used for multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; the satisfaction degree vector acquisition unit is used for acquiring column vectors corresponding to the target execution time points from the dependency satisfaction degree matrix to obtain satisfaction degree vectors of each task at the target execution time points; the first judgment unit is used for comparing the satisfaction degree vector of each task at the target execution time point with the dependency degree vector of each task to judge whether each task meets the executed condition; or

The judging module comprises: the dependency satisfaction degree matrix acquisition unit is used for multiplying the task dependency matrix and the execution state matrix to obtain a dependency satisfaction degree matrix; the dependency vector acquisition unit is used for acquiring the sum of elements of each row from the task dependency matrix to obtain a dependency vector of each task; a dependency matrix construction unit, configured to construct a dependency matrix according to the dependency vectors, where rows of the dependency matrix correspond to tasks one to one, columns of the dependency matrix correspond to the execution time points one to one, and each element of the dependency matrix is a dependency vector of a task corresponding to a row of the element at an execution time point corresponding to a column of the element; and the second judging unit is used for comparing the dependency matrix with the dependency satisfaction matrix to judge whether each task meets the executed condition at each execution time point.

7. A storage medium having a computer program stored thereon, the computer program, when being executed by a processor, performing the steps of the method according to any one of claims 1 to 5.

8. A computing device comprising a memory and a processor, the memory having stored thereon a computer program operable on the processor, wherein the processor, when executing the computer program, performs the steps of the method of any of claims 1 to 5.