CN109408204A

CN109408204A - A kind of method for scheduling task and device of distributed task scheduling system

Info

Publication number: CN109408204A
Application number: CN201710694826.6A
Authority: CN
Inventors: 翁忠杰; 曾文秋
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2017-08-15
Filing date: 2017-08-15
Publication date: 2019-03-01

Abstract

The application provides the method for scheduling task and device of a kind of distributed task scheduling system；Wherein, method for scheduling task includes: to check the task instances that task instances to be run are relied on when the task instances wait run node；Wherein, the task instances that the task instances to be run are relied on include: the task instances of previous periodic duty；If the task instances that the task instances to be run are relied on output, the operation task instances to be run.At least one embodiment of the application can be realized complicated scheduling dependence.

Description

Task scheduling method and device of distributed task system

Technical Field

The present invention relates to the field of data processing, and in particular, to a task scheduling method and apparatus for a distributed task system.

Background

The task scheduler of a data production workflow, which is common in the industry today, is typically used to run a set of jobs and processes in a specific order within a workflow. These workflow schedulers can support Cron syntax scheduling, but the scenario is generally simpler.

The most common workflow which is operated by day is defined according to the dependency relationship in the workflow, and each node generates one task instance for operation every day; the nodes are referred to as task nodes in the workflow. For example, in fig. 1, a node a and a node B run a task instance a and a task instance B, respectively, at a zero point every day, where the task instance B depends on the task instance a.

In more complex cases, a workflow may need to run regularly many times a day, that is, supporting Cron syntax scheduling in hours.

Under the scene, according to the definition of the workflow, a task instance is operated for each task node in the workflow at a specified timing time, the workflows executed at different time points are mutually independent, and the tasks in the workflow cannot depend on the workflow and cannot influence each other. For example, in fig. 2, node a and node b each run a task instance at 0, 8, and 16 points of each day; node a runs three task instances a1, a2, A3 per day; three task instances B1, B2 and B3 which are operated by the node B every day depend on A1, A2 and A3 respectively.

The current big data workflow system can realize the timing scheduling function of the workflow, but has great limitation on the dependence scheduling.

Disclosure of Invention

The application provides a task scheduling method and device of a distributed task system, which can realize complex scheduling dependency relationship.

The technical scheme is as follows.

A task scheduling method of a distributed task system comprises the following steps:

when the task instance of the node to be operated, checking the task instance on which the task instance to be operated depends; wherein, the task instance on which the task instance to be run depends includes: a task instance running in a previous cycle;

and if the task instance on which the task instance to be run depends is produced, running the task instance to be run.

Wherein the task instance running in the previous cycle may include one or more of the following:

a task instance of the node running in the previous period;

task instances of other nodes running in the previous cycle.

If the task instance to be run is the first task instance of the node in the current system cycle, the task instance of the node running in the previous cycle may refer to: in the previous system period, the last task instance of the node is obtained;

if the task instance to be run is not the first task instance of the node in the current system cycle, the task instance of the node running in the previous cycle may refer to: in the same system period, the task instance of the node is operated in the previous task period;

wherein one of the system cycles may comprise one or more task cycles; for a node, a task instance is run in a task cycle.

Wherein the task instances of the other nodes running in the previous period may include one or more of the following:

task instances of other nodes running in a previous task period of the same system period;

one or more task instances of other nodes operating in a previous system cycle;

If the task instance on which the task instance to be executed depends includes the task instance of the node that is executed in the previous cycle, the task instance of the other node that is executed in the previous cycle may refer to: task instances of other nodes running in the previous system cycle.

When the task instances on which the task instances of other nodes depend include task instances of other nodes running in a previous cycle, the one or more task instances of other nodes running in a previous system cycle may include: the last task instance of the other nodes in the previous system cycle;

when the task instances depended on by the task instances of other nodes do not include the task instances of other nodes running in the previous cycle, the one or more task instances of other nodes running in the previous system cycle may include: task instances of other nodes in the previous system cycle.

The node may be a child node, and the other nodes may be parent nodes;

or,

the node may be a parent node, and the other nodes may be child nodes.

A task scheduling device of a distributed task system, comprising: a processor and a memory;

the memory is used for storing a program for scheduling tasks; when the program for task scheduling is read and executed by the processor, the following operations are performed:

a task instance of the node running in the previous period;

task instances of other nodes running in the previous cycle.

one or more task instances of other nodes operating in a previous system cycle;

The node may be a child node, and the other nodes may be parent nodes;

or,

the node may be a parent node, and the other nodes may be child nodes.

A task scheduling device of a distributed task system, comprising:

the checking module is used for checking the task instance which is depended by the task instance to be operated when the task instance of the node to be operated; wherein, the task instance on which the task instance to be run depends includes: a task instance running in a previous cycle;

and the operation module is used for operating the task instance to be operated when the task instance on which the task instance to be operated depends is produced.

a task instance of the node running in the previous period;

task instances of other nodes running in the previous cycle.

Wherein the task instance of the second node running in the previous cycle may include one or more of the following:

one or more task instances of other nodes operating in a previous system cycle;

The node may be a child node, and the other nodes may be parent nodes;

or,

the node may be a parent node, and the other nodes may be child nodes.

At least one embodiment of the application can realize cross-cycle dependence and can meet the requirements of different service scenes; in one implementation, the objects that depend on across cycles may be other nodes and the node, or any one of them; in one implementation, one system cycle may include one or more task cycles, and cross-cycle dependencies on other nodes may include one or more of: cross-system cycle dependence and cross-task cycle dependence; in one implementation, the nodes with cross-cycle dependency relationship can be parent and child nodes; the implementation modes are used independently or combined with each other, so that various complex task scheduling dependency relationships in big data production can be met, and a flexible blood relationship is provided for data services.

Of course, it is not necessary for any product to achieve all of the above-described advantages at the same time for the practice of the present application.

Drawings

FIG. 1 is a schematic diagram of the dependencies of tasks running on a daily basis;

FIG. 2 is a schematic diagram of the dependencies of tasks that run multiple times a day;

FIG. 3 is a flowchart of a task scheduling method according to the first embodiment;

FIG. 4 is a schematic diagram of a system architecture employing the task scheduling method of the first embodiment;

5(a) -5 (c) are schematic diagrams of the dependency relationships of the three scenarios of example one, respectively;

FIGS. 6 to 10 are schematic diagrams of the dependency relationships of the five scenarios of example two, respectively;

FIGS. 11 to 14 are schematic diagrams of the dependency relationships of the four scenarios of example three, respectively;

FIGS. 15 to 20 are schematic diagrams illustrating the dependency relationships of five scenarios included in the fourth first case;

FIGS. 21 to 32 are schematic diagrams illustrating the dependency relationships of twelve scenarios included in the fourth second case of the example, respectively;

FIGS. 33-42 are schematic diagrams illustrating the dependency relationships of ten scenarios involved in the fourth and third cases, respectively;

fig. 43 is a flowchart of the task scheduling device according to the third embodiment.

Detailed Description

The technical solutions of the present application will be described in more detail below with reference to the accompanying drawings and embodiments.

It should be noted that, if not conflicted, the embodiments and the features of the embodiments can be combined with each other and are within the scope of protection of the present application. Additionally, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.

In one configuration, a computing device that performs task scheduling may include one or more processors (CPUs), input/output interfaces, network interfaces, and memory (memories).

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium. The memory may include one or more modules.

Computer-readable media include both non-transitory and non-transitory, removable and non-removable storage media that can implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

For a node, in a system period (such as but not limited to one day), one task instance (as in the case of fig. 1) may be run, or multiple task instances (as in the case of fig. 2) may be run; in this document, a case where one system cycle runs one task instance is regarded as that one system cycle only includes one task cycle; the case that a system cycle runs a plurality of task instances is considered as a system cycle comprising a plurality of task cycles.

Herein, for a node, a task instance is run in each task period (or called run period); when the starting time of the task period (which can also be regarded as the timing time of the node or the task instance) arrives, whether the dependency relationship is satisfied or not can be checked, and one task instance is operated when the dependency relationship is satisfied.

Wherein, the dependency satisfaction may mean that one or more task instances on which the task instance to be run depends have been spawned.

Herein, the meaning of dependent may refer to: one or more task instances as dependents need to be produced before running as a task instance of a depended party.

The meaning of the output of the task instance may mean that the task instance is normally completed, for example, the data obtained after the task instance runs is complete, the data quality meets the inspection standard, and the like.

If the task instance of the first node depends on the task instance of the second node, the first node can also be said to depend on the second node

In other words, a first node depends on a second node, i.e. at least one task instance run by the first node, and depends on at least one task instance run by the second node. For another example, a first node depends on a second node across cycles, i.e., at least one task instance that the first node runs, on a task instance that the second node runs in a previous cycle.

A task instance may be generated at the beginning of each system cycle and the generated task instance may be run at the beginning of the task cycle.

The system period may refer to an update period of the workflow system, for example, assuming that the version of the workflow system is updated every day, the length of the system period is "one day".

One or more of the above task periods may be included in one system period; the number of task cycles in a system cycle is not necessarily the same for different nodes.

For example, assuming that the system cycle length is one day, the node a has three task cycles in one day, and the timing times are 0 point, 8 point, and 16 point, respectively, it is checked whether the dependency relationship is satisfied when the node a reaches 0 point, 8 point, and 16 point in one day, and if so, the corresponding task instance of the node a is executed.

The length of the task period can be called as a step length, and the step length can also be regarded as the time length between the starting time of two adjacent task periods in one system period; for example, when the step length is 8 hours, the starting time of one task cycle is 0 point, the starting time of the next task cycle is 8 points, and the starting time of the next task cycle is 16 points.

The interval of the task period can refer to a system period and at least comprises a time interval of the starting time of each task period; for example, if the starting time of the first task period in one system period of the node a is 0, and the starting time of the first task period in one system period of the node B is 6, the task periods of the node a and the node B have different intervals.

The first and second nodes are used for distinguishing different nodes and are not limited; when one node depends on other nodes across cycles, the node can be seen as a first node in the text; when depended on by other nodes across cycles, it can be seen as the second node in the text.

An embodiment of a method for scheduling tasks in a distributed task system, as shown in fig. 3, includes steps S110 to S120:

s110, when the task instance of the node to be operated is executed, checking the task instance on which the task instance to be operated depends; wherein, the task instance on which the task instance to be run depends includes: a task instance running in a previous cycle;

and S120, if the task instance depended by the task instance to be operated is produced, operating the task instance to be operated.

The embodiment can realize cross-cycle dependence and can meet the requirements of different service scenes.

In this embodiment, in order to implement the dependency on the task instance running in the previous cycle, the unique identifier of the task instance may be extended, for example, the instance identifier of the task instance includes an identifier of a system cycle to which the task instance belongs, an identifier of a task cycle, and an identifier of a node, and a task instance may be uniquely determined by the instance identifier; the identification of a task period may be omitted if there is only one task instance in a system period.

In this way, when the task instance on which the task instance to be executed depends is checked, the checking may be performed according to the instance identifier, for example, when the list for recording the task instance that has been spawned includes the instance identifiers of the task instances on which the task instance to be executed depends, it may be considered that the task instance on which the task instance to be executed depends has been spawned.

In this embodiment, a distributed task system may refer, but is not limited to, a workflow system, and may be used, but is not limited to, for big data production, or other tasks.

In this embodiment, the task instance of the node to be operated may refer to, but is not limited to, the arrival of the timing time of the node or the start of a task period, for example, a day is assumed to be a system period; wherein:

in a scenario of running a workflow by day, such as in fig. 1, a node a is to run a task instance at the beginning of a new day (i.e., when reaching zero); if the node a is taken as the node in the step S110, when the new zero point of the day is reached, the task instance of the node a to be operated is the time;

in the scenario of timing scheduling, a node runs a task instance at one of a plurality of specified timing times, for example, node a in fig. 2 runs a task instance at 0, 8, and 16 points of a day, where 0, 8, and 16 points are the specified timing times; if the node a is taken as the node in the step S110, the arrival time of the node a at 0 point, 8 points and 16 points in the day is the time of the task instance of the node a to be operated.

Besides the two cases listed above, the method can also include other cases of task instances of nodes to be executed, such as forcing to trigger task instances of running nodes, and the like.

In this embodiment, the task instances on which the task instances to be executed depend may include, in addition to the task instances executed in the previous cycle, task instances of other nodes in the current task cycle or the current system cycle. Namely: the node in step S110 may be a child node of other nodes.

In this embodiment, the node in step S110 may also be a parent node of another node, that is, the task instance of the node may be depended by the task instances of other nodes in the current task cycle or the current system cycle.

Hereinafter, the node in step S110 is referred to as a first node, and the other nodes are referred to as second nodes.

A system architecture adopting the task scheduling method of the embodiment is shown in fig. 4, and includes:

a plurality of executors, at least one scheduler;

the scheduler can be used for allocating the task instances meeting the operation conditions to the executors for operation and recording the produced task instances; wherein, achieving the operating condition may refer to: the timing time arrives (such as 0 in the case of fig. 1, 0, 8, 16 in the case of fig. 2) and the dependent task instance has spawned.

The scheduler may generate a rule for a configuration file storing each task instance, and generate a configuration file according to the generation rule when a new system cycle starts, where the configuration file includes task instances on which each task instance depends.

Each task instance can be uniquely determined by an instance identifier, and the instance identifier can include: the system period identifier, the task period identifier and the node identifier. The identification of a task period may be omitted if there is only one task instance in a system period.

The generation rule may be preset and may be changed as needed.

The generation rule may include a dependency relationship, and in configuration files generated in different system periods, although the instance identifications of the dependent task instances are different, the dependency relationship is the same; for example, assuming that a system cycle is one day, task instance a1 of node a depends on task instance B3 of node B on the previous day, and is embodied in the configuration file of 2014.12.31, where the task instance is identified as (2014.12.31+1+ a) and the dependent instance is identified as (2014.12.30+3+ B); the configuration file represented in 2015.1.1 is a task instance with an instance identifier of (2015.1.1+1+ a), and a task instance with a dependent instance identifier of (2014.12.31+3+ b); it can be seen that the dependencies expressed in the two configuration files are the same, only the identification of the system cycle in the instance identification is different.

In one implementation, the task instance run in the previous cycle may include one or more of the following: the task instance of the first node running in the previous period and the task instance of the second node running in the previous period.

In the implementation mode, the first node depends on the first node or other nodes in a cross-cycle manner, so that a more complex scheduling dependency relationship can be realized, and the requirements of different scenes are met.

In this implementation, the task instance on which the task instance to be run depends is the task instance of the first node running in the previous cycle, and may be referred to as first node self-dependency;

the case that the task instance on which the task instance to be run depends is the task instance of the second node running in the previous cycle can be referred to as that the first node depends on the second node across cycles.

For convenience of description, the task instance executed by the first node is referred to as a first task instance, and the task instance executed by the second node is referred to as a second task instance.

In this implementation, there may be one or more second nodes.

In this implementation, the self-dependence may include the following two cases:

if the task instance to be run is the first task instance of the first node in the current system cycle, the task instance of the first node that is run in the previous task cycle may refer to: a task instance running in the last task period of the first node in the previous system period (i.e., the last task instance of the first node in the previous system period);

if the first task instance to be run is not the first task instance of the first node in the current system cycle, the task instance of the first node that was run in the previous task cycle may refer to: and in the same system period, the task instance of the first node is operated in the previous task period.

Wherein one of said system cycles comprises one or more task cycles; for a node, a task instance is run in a task cycle.

In this implementation, the task instance of the second node running in the previous cycle may include one or more of the following:

a task instance of the second node running in a previous task period of the same system period;

one or more task instances of the second node operating at a previous system cycle.

The number of task cycles of the first node and the second node in one system cycle may be the same or different.

In this implementation, the task instance to be run depends on the task instance running in the previous cycle (i.e., cross-cycle dependency), and may include the following seven dependency relationships:

(dependency relationship 1) the first node only depends on itself, that is, the task instance to be run depends on the first task instance running in the previous period;

(dependency 2) the first node only depends on the second node across system cycles, i.e. the task instance to be run depends on one or more second task instances running in the previous system cycle;

(dependency relationship 3) the first node only depends on the second node across task cycles, that is, the task instance to be run depends on a second task instance running in the previous task cycle; in this case, the first task instance in the system cycle does not have the second task instance of the previous task cycle;

(dependency 4) includes dependencies 1, 2;

(dependency 5) includes dependencies 1, 3;

(dependency 6) includes dependencies 2, 3;

(dependency 7) includes dependencies 1, 2, and 3.

When the dependencies 1 and 2 coexist (i.e., in the dependency 4 or 7), only the first task instance in the system cycle may depend on the second task instance in the previous system cycle, and the remaining first task instances may not depend on the second task instance in the previous system cycle.

Wherein, if the second node is also self-dependent, in the dependency relationship 2, the first task instance only depends on the last second task instance of the previous system cycle.

The first node and the second node can be a father node and a child node respectively, or a child node and a father node respectively, that is, a non-cross-period dependency relationship exists; at this time, the dependency relationship between the parent and child nodes and the above 7 cross-cycle dependency relationships may coexist.

Wherein, the dependency relationship 3 generally exists in a system period under the condition that the first task instance and the second task instance are in one-to-one correspondence; the task instance of the previous task cycle may refer to: a task instance before the corresponding task instance; for example, in fig. 2, with node a as the first node and node B as the second node, the task instance of the previous task cycle is B1 for task instance a 2.

In an embodiment of this implementation, if the task instance on which the task instance to be executed depends includes a task instance of the first node that is executed in a previous cycle, the task instance of the second node that is executed in the previous cycle may refer to: a task instance of the second node running in a previous system cycle.

That is, in the case that a first node is self-dependent and depends on a second node across system cycles, other first task instances in one system cycle, except for the first running first task instance, may not need to depend on the second task instance in the previous system cycle, i.e.: it may be sufficient that only the first task instance running first in a system cycle depends on the second task instance of the previous system cycle.

In other embodiments, if the first node is not self-dependent, when the first node depends on the second node across system cycles, each first task instance in one system cycle may depend on the second task instance in the previous system cycle, and the second task instance set as a part of the first task instances in one system cycle depending on the second task instance in the previous system cycle is not excluded.

In other embodiments, the case of repeated dependency is not excluded, for example, when the first node is self-dependent and depends on the second node across system cycles, the second task instance of the previous system cycle may still be dependent on multiple first task instances.

In one implementation, the second node may also be self-dependent, i.e.: the task instances on which the second task instance to be run depends include: a second task instance running in a previous cycle.

Similar to the case of the first node being self-dependent, if the second task instance to be executed is the first one to be executed in the current system cycle, the second task instance executed in the previous task cycle may refer to: a second task instance running in the last task period of the second node in the previous system period;

if the second task instance to be run is not the second task instance running first in the current system cycle, the second task instance running in the previous task cycle may refer to: and in the same system period, a second task instance is operated in the previous task period of the second node.

In one embodiment of this implementation, when the task instance on which the task instance of the second node depends includes a task instance of the second node running in a previous system cycle, the one or more task instances of the second node running in the previous system cycle include: the last task instance of the second node in the previous system cycle;

when the task instance on which the task instance of the second node depends does not include the task instance of the second node running in the previous cycle, the one or more task instances of the second node running in the previous system cycle include: task instances of the second node in the previous system cycle.

That is, in the case of the second node being self-dependent, if the first node depends on the second node across system cycles, it may only need to depend on the last running second task instance in the previous system cycle. Under the condition that the second node is not self-dependent, if the first node depends on the second node across the system period, the first node can depend on each second task instance in the previous system period.

In other embodiments, in the case that the first node depends on the second node across system cycles and the second node does not depend on itself, the first task instance may depend on only a part of the second task instance of the previous system cycle.

In other embodiments, the case of repeated dependency is not excluded, for example, when the first node depends on the second node across system cycles, and the second node is independent, the first task instance may still depend on a plurality of second task instances of the previous system cycle.

In one implementation, the first node may be a child node and the second node may be a parent node.

In this implementation, the expression of parent and child nodes means that the first node depends on the second node in the same system period and corresponding task period; to distinguish from cross-cycle dependencies, the dependencies that form parent and child node relationships may be referred to as regular dependencies.

In one implementation, the first node may be a parent node and the second node may be a child node.

In this implementation, the expression of parent and child nodes means that the second node depends on the first node in the same system cycle and corresponding task cycle; to distinguish from cross-cycle dependencies, the dependencies that form parent and child node relationships may be referred to as regular dependencies.

In the two implementation manners, when the parent node has only one task period in one system period, in the conventional dependence, each sub-task instance of the child node in one system period can depend on the parent task instance of the parent node;

when the father node has a plurality of task periods in one system period, in the conventional dependence, each subtask instance of the child node in one system period can depend on the father task instance with the smallest difference between the timing time of the father task instance and the timing time of the subtask instance; if there are two parent task instances with the same timing time difference as the child task instance, the parent task instance with the timing time before the current task instance is depended on.

In the two implementation manners, the task instance of the second node in the previous task cycle may refer to: a second task instance preceding the regular second task instance; wherein the regular second task instance refers to a second task instance having regular dependency with the first task instance to be run.

For example, in fig. 2, it is assumed that the first node is a node a, the second node is a child node b, and there are three task periods in one system period (i.e., one day in fig. 2); the second node has three task instances B1, B2, B3 in one system cycle, and the first node has three task instances a1, a2, A3 in the system cycle. Wherein, B1 depends on A1 and B2 of the system cycle depend on A2 and B3 of the system cycle depend on A3 of the system cycle, and the dependence is conventional. When the first task instance to be run is a2, the second task instance with regular dependencies is B2, then for a2, the task instance of the second node running in the previous task cycle is B1; by analogy, for a3, the task instance of the second node running in the previous task cycle is B2.

For another example, assume that the first node is node b, the second node is child node a, and there are three task periods in a system period; the second node runs three task instances A1, A2 and A3 in one system period, and the first node runs three task instances B1, B2 and B3 in the system period. Wherein, B1 depends on A1 and B2 of the system cycle depend on A2 and B3 of the system cycle depend on A3 of the system cycle, and the dependence is conventional. When the first task instance to be run is B2, the second task instance with regular dependencies is a2, then for B2, the task instance of the second node running in the previous task cycle is a 1; by analogy, for B3, the task instance of the second node running in the previous task cycle is A2.

In the two implementation modes, if the relationship of the task periods of the father node and the son node in one system period is one-to-many and the son node is self-dependent, part of conventional dependence can be omitted; if the relationship of the task period of the parent node and the child node in one system period is many-to-one, and the parent node is self-dependent, part of the conventional dependence can be omitted.

In other implementation manners, there may be no parent-child relationship between the first node and the second node.

Four examples are used below to illustrate some of the dependencies that may occur between two nodes. In the following four examples, it is assumed that the two nodes are a child node and a parent node, respectively; the task instance of the child node is referred to as a child task instance, and the task instance of the parent node is referred to as a parent task instance.

In fig. 5 to 42 for showing various dependencies included in the following four examples, a node a is a parent node, and a node b is a child node; solid arrows between nodes or task instances represent co-periodic dependencies, and dashed arrows represent cross-periodic (including at least one of cross-system period, cross-task period) dependencies on other nodes or self-dependencies; the arrow points to the relying party, the origin of which is the depended party.

In the following four examples, the system cycle is "one day", and the description will be divided into four examples according to the difference in the lengths of the task cycles of the parent node and the child node.

When the task period is equal to one day, it may be referred to as a day task, and the nodes for a day task may be referred to as day nodes or day nodes. When the task period is more than one day, such as one week or one month, the task period can be called a week task or a month task, and the nodes corresponding to the week task or the month task can be called week nodes or month nodes; the weekly task or the monthly task can be regarded as a task which runs one task instance every day, and the dependency relationship can refer to the dependency relationship of the daily task. When the task period is less than one day, the task period may be called as an hour task or a minute task according to the length of the task period, and the node corresponding to the task is called as an hour node or a minute node.

The following description is made according to different lengths of the task periods of the parent node and the child node.

Example one

The task periods of the parent-child nodes are all more than or equal to 1 day, namely the tasks corresponding to the parent-child nodes can be daily tasks, weekly tasks or monthly tasks; the following three scenarios may be included:

(1) the parent nodes and the child nodes are all day nodes, and the child nodes depend on the parent nodes in a period-crossing mode.

In this scenario, the subtask instance depends not only on the parent task instance of the current day, but also on the parent task instance of the previous day. As shown in FIG. 5(a), the subtask instance B at 0 of node B at 2015.1.1 depends not only on the parent task instance A of node a of the current day, but also on the parent task instance A of node a at 2014.12.31; if the parent task instance A of the previous day is not produced, the child task instance B of the current day cannot be run.

(2) The parent node and the child node are all day nodes, and the parent node depends on the child nodes in a period-crossing mode.

In this scenario, the parent task instance depends on the yield of the child task instance on the previous day. As shown in FIG. 5(B), the parent task instance A at 0 of node a at 2015.1.1 depends on the child task instance B at 2014.12.31 of node B; if the sub-task instance B of the previous day is not produced, the parent task instance A of the current day cannot be run.

(3) The parent and child nodes are all day nodes and the parent node is self-dependent.

The parent task instance depends on the yield of the parent task instance on the previous day. As shown in FIG. 5(c), parent task instance A, when node a is 0 at 2015.1.1, depends on parent task instance A, when node a is 2014.12.31; and if the parent task instance A of the previous day is not produced, the parent task instance A of the current day cannot run.

In the first example, the node a and the node b have only one task instance in one system cycle (i.e. in one day), and the cross-cycle dependence on other nodes means that the cross-system cycle dependence on other nodes is performed.

Example two

The parent node is a node with a task period of more than or equal to 1 day, the child node is a node with a task period of less than 1 day, namely, the task corresponding to the parent node can be a day, week or month task, and the task corresponding to the child node can be an hour or minute task; the following five scenarios may be included:

(1) and no cross-cycle dependency exists between the parent node and the child node.

The scene is simple, a plurality of task instances can be generated by the subtask of the day according to the Cron expression, but the subtask instance of each different task period in the day depends on the same father task instance of the day, and if the father task instance of a certain day is not produced, all the subtask instances of the day cannot run; after the parent task instance A of a certain day is produced, the child task instances B1, B2 and B3 of the day run at set time. As shown in FIG. 6, three subtask instances B1, B2, B3 of node B at 2014.12.31 all depend on the parent task instance A of node a at 2014.12.31, and three subtask instances B1, B2, B3 of node B at 2015.1.1 all depend on the parent task instance A of node a at 2015.1.1.

(2) Parent node with cross-period dependency of child nodes

And if the dependent parent task instance is produced after the timing time of the child task instance, the child task instance exceeding the timing time can be immediately operated after the dependent parent task instance is produced. As shown in FIG. 7, the child task instances B1, B2, B3 of node B at 2015.1.1 depend not only on the parent task instance A of node a on the current day, but also on the parent task instance A of node a at 2014.12.31.

(3) Child nodes depend on parent nodes across cycles and child nodes are self-dependent

In this scenario, the subtask instance depends on the output of the subtask instance in the previous task period (including that each subtask instance except the first subtask instance in one day depends on the output of the previous subtask instance, and the first subtask instance in one day depends on the output of the last subtask instance in one day), and the first task instance of the child node in one day further depends on not only the output of the parent task instance in the previous day but also the output of the parent task instance in the current day. As shown in fig. 8, subtask instance B2 depends on subtask instance B1 for the current day, and subtask instance B3 depends on subtask instance B2 for the current day; the subtask instance B1 for node B at 2015.1.1 depends on the parent task instance A for node a at 2014.12.31, the parent task instance A for node a at 2015.1.1, and the subtask instance B3 for node B at 2014.12.31.

In this scenario, since the child node is self-dependent, and the child task instance B1 already depends on the parent task instance a in the same system cycle, the dependency between the child task instances B2, B3 and the parent task instance a in the same system cycle may be omitted.

(4) Parent node cross-cycle dependent child node

In the scene, the parent task is produced according to the subtask instances of all task periods in the previous day, and if the subtask instance of any task period in the previous day is not produced, the parent task instance cannot run. As shown in FIG. 9, the parent task instance A at 0 of node a at 2015.1.1 depends on the child task instances B1, B2, and B3 of node B at 2014.12.31.

(5) Parent node relies on child node across cycles and child node relies on itself

In the scene, the subtask instance depends on the subtask instance of the previous task period, and the first subtask in one day is also produced depending on the current father task instance; in addition, the parent task instance also depends on the last subtask instance in the previous day, and the first subtask instance in the day runs on the premise that the last subtask instance in the previous day yields and the parent task in the current day also yields. As shown in fig. 10, subtask instance B2 depends on subtask instance B1 for the current day, and subtask instance B3 depends on subtask instance B2 for the current day; parent task instance A, node a at 0 of 2015.1.1, depends on child task instance B3, node B at 2014.12.31; the subtask instance B1 of node B at 2015.1.1 depends on the subtask instance B3 of node B at 2014.12.31 and the parent task instance A of node a at 2015.1.1.

In the second example, in one system cycle (i.e. one day), the conventional dependency between the task instance of the node a and the task instance of the node b is a one-to-many relationship, and the cross-cycle dependency on other nodes means that the cross-system cycle depends on other nodes.

Example III

The father node is a node with a period of less than 1 day, the child node is a node with a period of more than or equal to 1 day, namely the task instance of the father node can be an hour or minute task, and the task instance of the child node can be a day, week or month task; the following four scenarios may be included.

(1) Neither parent nor child node is dependent across cycles

Under the scene, the subtask instance depends on the output of all the father task instances on the day, the subtask instance can run only under the premise that all the father task instances on the day are output, and the workflows in two days have no mutual dependency relationship. As shown in FIG. 11, node B at 2014.12.31, dependent node a at 2014.12.31 parent task instance A1, A2, A3; node B at 2015.1, the child task instance B, dependent on node a at 2015.1.1, parent task instances A1, A2, A3.

(2) Parent node cross-cycle dependent child node

Under the scene, all the parent task instances in one day depend on the output of the sub-task instances in the previous day, and the output of the sub-task instances in the previous day can ensure that the parent task instances in each task period are executed according to the specified time. As shown in FIG. 12, the parent task instance A1, A2, A3 of node a at 2015.1.1 all depend on the child task instance B of node B at 2014.12.31.

In this scenario, the dependency of the subtask instance B on the parent task instance A3 may be omitted.

(3) Parent node relies on child nodes across cycles and parent node relies on itself

In this scenario, the parent task instance of each task period depends on the parent task instance of the previous task period, and the first parent task instance in a day depends on the child task instance of the previous day. And the child node instance depends on the parent task instance running in the corresponding period of the parent node forward according to the time-nearest principle. If the subtask instance runs 12 hours a day at regular time, and the father task instance runs 0-23 points a day at intervals of 8 hours, the subtask instance depends on the task instance of the father node at 8 points a day at regular time. As shown in fig. 13, parent task instance a2 depends on the current day's parent task instance a1, and parent task instance A3 depends on the current day's parent task instance a 2; parent task instance A1 at 2015.1.1 for node a, parent task instance A3 at 2014.12.31 for dependent node a, and child task instance B at 2014.12.31 for node B; child task instance B depends on the current day's parent task instance A2.

(4) Child nodes depend on father nodes in a period-crossing mode and father nodes are independent

Under the scene, the task instances of each task period of the father node all depend on the father task instance of the previous task period; besides depending on the latest time principle, the sub-task instance depends on the task instance in the corresponding period of the parent node of the current day forward, and also depends on the task instance output in the last task period of the parent node of the previous day. As shown in fig. 14, parent task instance a2 depends on the current day's parent task instance a1, parent task instance A3 depends on the current day's parent task instance a 2; parent task instance A1, node a at 2015.1.1, parent task instance A3, dependent node a at 2014.12.31; child task instance B depends on the current day's parent task instance A2, and the previous day's parent task instance A3.

In the third example, in one system cycle (i.e. one day), the conventional dependency between the task instance of the node a and the task instance of the node b is a many-to-one relationship, and the cross-cycle dependency on other nodes means that the cross-system cycle depends on other nodes.

Example four

The parent-child nodes are all nodes with a period of <1 day, that is, the parent-child nodes can be tasks of hours or minutes, and the scene is complicated and is subdivided into the following three cases (a), (B) and (C) for explanation:

(A) the period step length of the parent node and the child node is the same and the time interval is the same;

(B) the period step length of the parent node and the child node is the same but the time interval is different;

(C) the period step of the parent node and the period step of the child node are different, but the time intervals are the same.

The following description will be made in order of cases.

(A) The step length of the task period of the parent-child node is the same and the time interval is the same; the following six scenarios may be included.

(1) Child nodes depend on parent nodes across cycles.

In this scenario, the child task instance of each task period of the child node depends on the parent task instance of the same task period on the same day, the parent task of the previous task period on the same day, and the parent tasks of all task periods in the previous day (i.e., the previous system period). As shown in FIG. 15, node B's child task instance B1 at 2015.1.1 depends on node a's parent task instance A1 for the current day, and node a's parent task instances A1, A2, A3 at 2014.12.31; the child task instance B2 of node B at 2015.1.1 depends on the parent task instances A1 and A2 of node a on the current day, and the parent task instances A1, A2 and A3 of node a at 2014.12.31; the child task instance B3 of node B at 2015.1.1 depends on the parent task instances A2 and A3 of node a for the current day, and the parent task instances A1, A2, A3 of node a at 2014.12.31.

(2) Parent node cross-cycle dependent child node

Under the scene, the task instance of each task period of the child node is produced by relying on the parent task instance of the same task period on the same day, and the task instance of each task period of the parent node is produced by relying on not only the child task instance of the previous task period on the same day but also the task instances of all task periods of the child node on the previous day (namely, the previous system period). As shown in fig. 16, the child task instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the parent task instances a1, a2, A3 of node a at 2015.1.1 all depend on the child task instances B1, B2, B3 of node B at 2014.12.31, and the parent task instances a2, A3 of node a at 2015.1.1 depend on the child task instances B1 and B2 of node B at 2015.1.1, respectively.

(3) The father node depends on the child node in a period-crossing way and is self-dependent

In this scenario, the task instance of each task period of the parent node depends on the output of the parent task instance of the previous task period on the same day and the output of the child task instance of the previous task period on the same day, and the first task period of the day depends on the output of the task instance of the last task period of the parent node on the previous day and the outputs of all periods of the child nodes on the previous day. As shown in fig. 17, the subtask instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the parent task instance A2 depends on the parent task instance A1 of the current day, and the parent task instance A3 depends on the parent task instance A2 of the current day; parent task instance A1, node a at 2015.1.1, parent task instance A3, dependent node a at 2014.12.31; the parent task instance A1 of node a at 2015.1.1 depends on the child task instances B1, B2 and B3 of node B at 2014.12.31, and the parent task instances A2 and A3 of node a at 2015.1.1 depend on the child task instances B1 and B2 of node B at 2015.1.1, respectively.

(4) Parent nodes depend on children across periods and child nodes are self-dependent

Under the scene, each task period of the child nodes depends on the task instance output of the previous task period; the task instance of each task period of the father node is produced according to the task instance of the previous task period of the current child node, and the task instance of each task period of the father node is produced according to the task instance of the last task period of the previous day of the child node. As shown in fig. 18, the child task instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the subtask instance B2 depends on the subtask instance B1 of the current day, and the subtask instance B3 depends on the subtask instance B2 of the current day; subtask instance B1 when node B is 0 at 2015.1.1 depends on subtask instance B3 when node B is 2014.12.31; the parent task instance A1, A2 and A3 of the node a at 2015.1.1 all depend on the child task instance B3 of the node B at 2014.12.31; parent task instances a2 and A3 depend on node B's current day's child task instances B1 and B2, respectively.

(5) Parent nodes depend on child nodes in a period-crossing mode and have self-dependence

In this scenario, the task instance of each task period of the parent node depends on the parent task instance and the child task instance of the previous task period of the current day, and the task instance of the first task period of the parent node depends on the parent task instance and the child task instance of the last task period of the previous day; the task instance of each task period of the child node depends on the parent task instance of the same task period and the child task instance of the task period before the current day. As shown in fig. 19, the child task instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the parent task instance A2 depends on the parent task instance A1 of the current day, and the parent task instance A3 depends on the parent task instance A2 of the current day; parent task instance A1 of node a at 2015.1.1 depends on parent task instance A3 of node a at 2014.12.31; the subtask instance B2 depends on the subtask instance B1 of the current day, and the subtask instance B3 depends on the subtask instance B2 of the current day; the subtask instance B1 for node B at 0 of 2015.1.1 depends on the subtask instance B3 for node B at 2014.12.31. Parent task instance A1 at node a at 2015.1.1 depends on child task instance B3 at 2014.12.31 at node B; parent task instances a2 and A3 depend on node B's child task instances B1 and B2, respectively, on the current day.

(6) The child nodes depend on the parent node in a period-crossing manner and the parent node and the child nodes all have self-dependence

In this scenario, the task instance of each task period of the parent node depends on the parent task instance of the previous task period of the current day, and the task instance of the first task period of the parent node depends on the task instance of the last task period of the previous day of the parent node; each task cycle of a child node depends on a parent task instance of the same task cycle, a parent task instance of a task cycle immediately before the current day, and a child task instance of a task cycle immediately before the current day. As shown in fig. 20, the subtask instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the parent task instance A2 depends on the parent task instance A1 of the current day, and the parent task instance A3 depends on the parent task instance A2 of the current day; parent task instance A1, node a at 2015.1.1, parent task instance A3, dependent node a at 2014.12.31; the subtask instance B2 depends on the subtask instance B1 of the current day, and the subtask instance B3 depends on the subtask instance B2 of the current day; the subtask instance B1 for node B at 0 of 2015.1.1 depends on the subtask instance B3 for node B at 2014.12.31. Node B's child task instance B1 at 2015.1.1 depends on node a's parent task instance A3 at 2014.12.31; the child task instances B2 and B3 depend on the parent task instances A1 and A2, respectively, of node a on the current day.

In the example (a), in one system cycle (i.e., one day), the conventional dependencies between the task instances of the node a and the task instances of the node b are in a one-to-one correspondence relationship, and the cross-cycle dependency on other nodes may include not only cross-system cycle dependency on other nodes but also cross-task cycle dependency on other nodes.

(B) The step length of the task period of the parent-child node is the same but the time interval is different; the following three cases can be included, and there are 12 scenarios:

(1) the following three scenarios can be included when the parent node and the child node have the same number of task cycles each day, but the timing time of the parent node is later than that of the child node:

a. none of the parent and child nodes has cross-cycle dependency

The task instance of each task period of the child node depends on the parent task instance of the corresponding task period of the current day backwards according to the time-nearest principle, the child task instance cannot run according to the timing time because the parent task instance is later than the timing time of the child task instance, and the child task instance can run immediately after the parent task instance is produced. As shown in FIG. 21, the child task instances B1, B2, B3 of node B depend on the parent task instances A1, A2, A3 of node a for the current day, respectively.

b. Child nodes depend on father nodes in a period-crossing mode and father nodes are independent

The task instance of each task period of the child node depends on the parent task instance of the corresponding task period of the current day backwards according to the time nearest principle, and also depends on the task instance of the last task period of the parent node of the previous day. As shown in fig. 22, the subtask instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the parent task instance A2 depends on the parent task instance A1 of the current day, and the parent task instance A3 depends on the parent task instance A2 of the current day; parent task instance A1 of node a at 2015.1.1 depends on parent task instance A3 of node a at 2014.12.31; the subtask instances B1, B2, B3 of node B at 2015.1.1 all depend on the parent task instance A3 of node a at 2014.12.31.

c. Parent node relies on child node across cycles and child node relies on itself

The task instance of any task period of the parent node depends on the task instance of the last task period of the child node on the previous day. As shown in fig. 23, the subtask instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the subtask instance B2 depends on the subtask instance B1 of the current day, and the subtask instance B3 depends on the subtask instance B2 of the current day; subtask instance B1 of node B at 2015.1.1 depends on subtask instance B3 of node B at 2014.12.31; the parent task instance A1, A2, A3 of node a at 2015.1.1 all depend on the child task instance B3 of node B at 2014.12.31.

(2) The following three scenarios can be included when the parent and child nodes have the same number of task cycles each day, but the timing time of the parent node is earlier than that of the child node:

a. none of the parent and child nodes has cross-cycle dependency

And the task instance of each task period of the child node depends on the parent task instance of the corresponding task period of the current day forward according to the time-most principle, and after the parent task instance is produced, the child task instance runs according to the specified timing time. As shown in FIG. 24, the child task instances B1, B2, B3 of node B depend on the parent task instances A1, A2, A3 of node a for the current day, respectively.

Besides depending on the parent task instance of the corresponding task period on the current day forward according to the time-nearest principle, the task instance of each task period of the child node also depends on the task instance of the last task period of the parent node on the previous day. As shown in fig. 25, the subtask instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the parent task instance A2 depends on the parent task instance A1 of the current day, and the parent task instance A3 depends on the parent task instance A2 of the current day; parent task instance A1 of node a at 2015.1.1 depends on parent task instance A3 of node a at 2014.12.31; the subtask instances B1, B2, B3 of node B at 2015.1.1 all depend on the parent task instance A3 of node a at 2014.12.31.

And each task period task of the parent node depends on the task instance of the last task period of the child node on the previous day. As shown in fig. 26, the subtask instances B1, B2, B3 of node B depend on the parent task instances a1, a2, A3 of node a for the current day, respectively; the subtask instance B2 depends on the subtask instance B1 of the current day, and the subtask instance B3 depends on the subtask instance B2 of the current day; subtask instance B1 of node B at 2015.1.1 depends on subtask instance B3 of node B at 2014.12.31; the parent task instance A1, A2, A3 of node a at 2015.1.1 all depend on the child task instance B3 of node B at 2014.12.31.

(3) The number of task cycles of the parent-child nodes is different every day, but the starting scheduling time of the parent-child nodes is the same (the deadline is different), and the following 6 scenes can be included:

a. the number of task cycles of the father node is small, and the father node and the son node do not have cross-cycle dependence

The number of the sub task instances equal to the number of the task cycles of the parent node is one-to-one dependent on the parent task instance, and the extra sub task instances depend on the task instance of the last task cycle of the parent node. As shown in FIG. 27, the subtask instance B1 of node B depends on the current day's parent task instance A1 of node a, and both subtask instances B2 and B3 of node B depend on the current day's parent task instance A2 of node a.

b. The number of task cycles of the father node is small, the child nodes depend on the father node in a cross-cycle mode and the child nodes are independent

The number of the sub task instances equal to the number of the task cycles of the father node is one-to-one dependent on the father task instance, and the extra sub task instances depend on the task instance of the last task cycle of the father node; the sub task instance of the next task period depends on the sub task instance of the previous task period, and in addition, the task instance of the first task period of the child node depends on the task instances of all the task periods of the parent node of the previous day. As shown in FIG. 28, the subtask instance B1 of node B depends on the parent task instance A1 of node a for the current day, and both the subtask instances B2 and B3 of node B depend on the parent task instance A2 of node a for the current day; the subtask instance B2 depends on the subtask instance B1 of the current day, and the subtask instance B3 depends on the subtask instance B2 of the current day; the subtask instance B1 of node B at 2015.1.1 depends on the subtask instance B3 of node B at 2014.12.31 and the parent task instances A1 and A2 of node a at 2014.12.31.

c. The number of task cycles of the father node is small, the father node depends on the child node across cycles, and the child node depends on itself

The number of the sub task instances equal to the number of the task cycles of the father node is one-to-one dependent on the father task instance, and the extra sub task instances depend on the task instance of the last task cycle of the father node; the subtask of the latter task period depends on the subtask instance of the former task period, and in addition, the task instance of any task period of the parent node depends on the task instance of the last task period of the previous day. As shown in FIG. 29, the subtask instance B1 of node B depends on the parent task instance A1 of node a for the current day, and both the subtask instances B2 and B3 of node B depend on the parent task instance A2 of node a for the current day; the subtask instance B2 depends on the subtask instance B1 of the current day, and the subtask instance B3 depends on the subtask instance B2 of the current day; the subtask instance B1 at node B at 2015.1.1, the parent task instance A1 at 2015.1.1, and A2 of node a all depend on the subtask instance B3 at 2014.12.31 of node B.

d. The parent node has more task cycles, and the parent and child nodes have no cross-cycle dependence

Parent and child task instances with equal task cycles are one-to-one dependent, and the excess parent task instances have no corresponding child task instance dependence. As shown in FIG. 30, the child task instances B1, B2 of node B depend on the parent task instances A1, A2 of node a for the current day, respectively.

e. The parent node has more task cycles, the child node depends on the parent node across cycles and the parent node is self-dependent

The parent-child task instances with the same task period number depend on one by one, the parent task instance of the next task period depends on the parent task instance of the previous task period, and the task instances of all task periods of the child nodes depend on the task instance of the last task period of the parent node of the previous day. As shown in FIG. 31, the subtask instances B1, B2 of node B depend on the parent task instances A1, A2 of node a for the current day, respectively; the parent task instance A2 depends on the parent task instance A1 of the current day, and the parent task instance A3 depends on the parent task instance A2 of the current day; the parent task instance A1 at node a at 2015.1.1, the child task instance B1 and B2 at node B at 2015.1.1 all depend on the parent task instance A3 at node a at 2014.12.31.

f. The number of task cycles of the father node is large, the father node depends on the child nodes across cycles and the father node is self-dependent

Parent-child task instances with equal task cycle numbers are one-to-one dependent; the parent task instance of the latter task period depends on the parent task instance of the former task period, and in addition, the task instance of the first task period of the parent node depends on the task instances of all the task periods of the child nodes on the previous day. As shown in FIG. 32, the subtask instances B1, B2 of node B depend on the parent task instances A1, A2 of node a for the current day, respectively; the parent task instance A2 depends on the parent task instance A1 of the current day, and the parent task instance A3 depends on the parent task instance A2 of the current day; the parent task instance A1 of node a at 2015.1.1 depends on the parent task instance A3 of node a at 2014.12.31 and the child task instances B1 and B2 of node B at 2014.12.31.

In example four (B), the cross-cycle dependence on other nodes means that other nodes are depended on in a cross-system cycle, but in practical application, it is not excluded that the cross-cycle dependence on other nodes can also include cross-task cycle dependence on other nodes.

(C) The task period step lengths of the parent nodes and the child nodes are different but the time intervals are the same; the following two cases can be included, for a total of 10 scenarios.

(1) When the step size of the task period of the parent node is larger than that of the task period of the child node, the following five scenarios can be included:

a. none of the parent and child nodes has cross-cycle dependency

In this scenario, the task instance of each task period of the child node depends on the parent task instance with the upper timing time less than or equal to the self timing time and the smallest time difference. As shown in FIG. 33, the subtask instances B1, B2 of node B depend on the current day's parent task instance A1 of node a, and the subtask instances B3, B4 of node B depend on the current day's parent task instance A2 of node a.

b. Parent node with cross-period dependency of child nodes

In this scenario, the task instance of each task period of the child node depends on all the parent task instances of the previous day except the parent task instance whose timing time of the day is less than or equal to the self timing time and whose time difference is the smallest. As shown in FIG. 34, the subtask instances B1, B2 of node B depend on the parent task instance A1 of node a for the current day, and the subtask instances B3, B4 of node B depend on the parent task instance A2 of node a for the current day; the subtask instances B1, B2, B3, B4 of node B at 2015.1.1 all depend on the parent task instances A1 and A2 of node a at 2014.12.31.

c. Parent node cross-cycle dependent child node

In such a scenario, the task instance of each task period of the child node depends on the parent task instance with the timing time of the day less than or equal to the self timing time and the smallest time difference; any periodic task of a parent node depends on all child task instances of the previous day. As shown in fig. 35, the subtask instances B1, B2 of node B depend on the parent task instance a1 of node a for the current day, and the subtask instances B3, B4 of node B depend on the parent task instance a2 of node a for the current day; the parent task instance A1 and A2 of node a at 2015.1.1 are dependent on the child task instances B1, B2, B3, B4 of node B at 2014.12.31.

d. Parent nodes depend on child nodes in a period-crossing mode and the parent nodes and the child nodes are all self-dependent

In this scenario, the task instance of each task period of the child node depends on the parent task instance with the timing time less than or equal to the self timing time and the minimum time difference, and the child task instance of the previous task period, and the task instance of each task period of the parent node depends on the parent task instance of the previous task period; the parent task instance for the first task period of a day is also dependent on the child task instance for the last task period of the previous day. As shown in FIG. 36, the subtask instances B1, B2 of node B depend on the current day's parent task instance A1 of node a, and the subtask instances B3, B4 of node B depend on the current day's parent task instance A2 of node a; the parent task instance A2 depends on the parent task instance A1 of the current day, the parent task instance A1 of the node a at 2015.1.1 depends on the parent task instance A2 of the node a at 2014.12.31; the subtask instances B2, B3, B4 depend on the subtask instances B1, B2, B3 of the current day, the subtask instance B1 of node B at 2015.1.1, and the parent task instance A1 of node a at 2015.1.1 depend on the subtask instance B4 of node B at 2014.12.31, respectively.

e. Children depend on parent across cycles and parents and children are all self-dependent

Under the scene, the task of the first period of the father node depends on the output of the task of the last period of the child node on the previous day. As shown in fig. 37, the subtask instances B1, B2 of node B depend on the parent task instance a1 of node a for the current day, and the subtask instances B3, B4 of node B depend on the parent task instance a2 of node a for the current day; the parent task instance A2 depends on the parent task instance A1 of the current day, the parent task instance A1 of the node a at 2015.1.1 depends on the parent task instance A2 of the node a at 2014.12.31; the subtask instances B2, B3, B4 depend on the subtask instances B1, B2, B3 of the current day, respectively, the subtask instance B1 of node B at 2015.1.1 depends on the subtask instance B4 of node B at 2014.12.31, and the parent task instance A2 of node a at 2014.12.31.

(2) When the step size of the task period of the parent node is less than the step size of the task period of the child node, the following five scenarios may be included:

a. none of the parent and child nodes has cross-cycle dependency

In such a scenario, the task instance of each task period of the child node depends on one or more parent task instances, the timing time of which is less than or equal to the self timing time, and the child task instances of other task periods do not depend on the parent task instances; and the parent task instance with the timing time later than the latest timing time of the child node does not depend on the child task instance. As shown in FIG. 38, the subtask instance B1 of node B depends on the parent task instance A1 of node a for the current day, and the subtask instance B2 of node B depends on the parent task instances A2 and A3 of node a for the current day; parent task instance A4 has no child task instance dependencies.

b. Parent node cross-cycle dependent child node

In such a scenario, the task instance of each task period of the child node depends on one or more parent task instances, the timing time of which is less than or equal to the self timing time, and the child task instances of other task periods do not depend on the parent task instances; the parent task instance with the timing time later than the latest timing time of the child node does not depend on the child task instance; the task instance of any task period of the parent node depends on all the child task instances of the previous day. As shown in FIG. 39, the subtask instance B1 of node B depends on the parent task instance A1 of node a for the current day, and the subtask instance B2 of node B depends on the parent task instances A2 and A3 of node a for the current day; parent task instance A4 has no child task instance dependencies; the parent task instances A1, A2, A3, A4 of node a at 2015.1.1 all depend on the child task instances B1 and B2 of node B at 2014.12.31.

c. Parent node with cross-period dependency of child nodes

In this scenario, the task instance of each task period of the child node may depend on all parent task instances of the previous day, except that the timing time is less than or equal to the self timing time, and one or more parent task instances on which the child task instances of other task periods depend are not available. As shown in FIG. 40, the subtask instance B1 of node B depends on the parent task instance A1 of node a for the current day, and the subtask instance B2 of node B depends on the parent task instances A2 and A3 of node a for the current day; parent task instance A4 has no child task instance dependencies; node B's child task instance B1 and B2 at 2015.1.1 both depend on node a's parent task instance A1, A2, A3, and A4 at 2014.12.31.

In such a scenario, the task instance of each task period of the child node depends on one or more parent task instances, the timing time of which is less than or equal to the self timing time, and the child task instances of other task periods do not depend on the parent task instances; the parent task instance with the timing time later than the latest timing time of the child node does not depend on the child task instance; in addition, the task instance of each task period of the parent node depends on the parent task instance of the previous task period, and the task instance of each task period of the child node depends on the child task instance of the previous task period; the task instance of the first task period of the parent node in a day also depends on the task instance of the last task period of the previous day of the child node. As shown in FIG. 41, the subtask instance B1 of node B depends on the parent task instance A1 of node a for the current day, and the subtask instance B2 of node B depends on the parent task instances A2 and A3 of node a for the current day; parent task instance A4 has no child task instance dependencies; the parent task instances A2, A3 and A4 are respectively dependent on the parent task instances A1, A2 and A3 of the current day; parent task instance A1 of node a at 2015.1.1 depends on parent task instance A4 of node a at 2014.12.31; the subtask instance B2 depends on the subtask instance B1 for the current day, the subtask instance B1 for node B at 2015.1.1, and the parent task instance A1 for node a at 2015.1.1 all depend on the subtask instance B2 for node B at 2014.12.31.

e. The child nodes depend on the parent node in a period-crossing manner and the parent node and the child node are all self-dependent

The task of each period of the child node depends on the output of the child task of the previous period, and the task of the first period of the child node also depends on the output of the last period of the previous day of the parent node. As shown in FIG. 42, the subtask instance B1 of node B depends on the parent task instance A1 of node a for the current day, and the subtask instance B2 of node B depends on the parent task instances A2 and A3 of node a for the current day; parent task instance A4 has no child task instance dependencies; the parent task instances A2, A3 and A4 are respectively dependent on the parent task instances A1, A2 and A3 of the current day; parent task instance A1 of node a at 2015.1.1 depends on parent task instance A4 of node a at 2014.12.31; subtask instance B2 depends on subtask instance B1 for the current day, subtask instance B1 for node B at 2015.1.1, subtask instance B2 for node B at 2014.12.31, and parent task instance A4 for node a at 2014.12.31.

In the example four (C), the cross-cycle dependence on other nodes means that other nodes are depended on in a cross-system cycle, but when the method is actually applied, it is not excluded that the cross-cycle dependence on other nodes can also include the cross-task cycle dependence on other nodes.

The above four examples only list the dependencies in some common scenarios, and the dependencies in other scenarios can be designed by themselves with reference to the description of the present embodiment.

The four examples can adapt to different service scene requirements, complex task scheduling dependency relations such as same-period node dependency, cross-period node dependency, inter-hour (minute) node dependency and the like can be realized in the scheduling system, and the dependency relations can be simultaneously selected and mutually influenced, so that the requirement of data production can be met, and a flexible consanguinity relation is provided for data services.

An embodiment two, a task scheduling apparatus of a distributed task system, includes: a processor and a memory;

when a task instance of a node to be run (hereinafter, the node is referred to as a first node, and other nodes are referred to as second nodes), checking a task instance on which the task instance to be run depends; wherein, the task instance on which the task instance to be run depends includes: a task instance running in a previous cycle;

In one implementation, the task instance running in the previous cycle may include one or more of the following:

a task instance of the first node running in a previous cycle;

a task instance of the second node running in a previous cycle.

In this implementation, if the task instance to be run is the first task instance of the first node in the current system cycle, the task instance of the first node running in the previous cycle may refer to: in the previous system period, the first node is the last task instance;

if the task instance to be run is not the first task instance of the first node in the current system cycle, the task instance of the first node running in the previous cycle may refer to: and in the same system period, the task instance of the first node is operated in the previous task period.

If the task instance on which the task instance to be executed depends includes a task instance of the first node that is executed in a previous cycle, the task instance of the second node that is executed in the previous cycle may refer to: a task instance of the second node running in a previous system cycle.

When the task instance on which the task instance of the second node depends includes a task instance of the second node running in a previous cycle, the one or more task instances of the second node running in the previous system cycle may include: the last task instance of the second node in the previous system cycle;

when the task instance on which the task instance of the second node depends does not include the task instance of the second node running in the previous cycle, the one or more task instances of the second node running in the previous system cycle may include: task instances of the second node in the previous system cycle.

In this implementation, the first node may be a child node, and the second node may be a parent node;

alternatively, the first node may be a parent node and the second node may be a child node.

In this embodiment, when the program for performing task scheduling is read and executed by the processor, the operations performed correspond to steps S110 to S120 in the first embodiment, and other implementation details can be referred to in the first embodiment.

In a third embodiment, a task scheduling apparatus of a distributed task system, as shown in fig. 43, includes:

a checking module 31, configured to, when a task instance of a node to be executed (hereinafter, the node is referred to as a first node, and other nodes are referred to as second nodes), check a task instance on which the task instance to be executed depends; wherein, the task instance on which the task instance to be run depends includes: a task instance running in a previous cycle;

and the running module 32 is used for running the task instance to be run when the task instance on which the task instance to be run depends is produced.

a task instance that the first node runs in a previous cycle;

and the task instance which is run by the second node in the previous period.

If the task instance on which the task instance to be executed depends includes a task instance in which the first node executes in a previous cycle, the task instance in which the second node executes in the previous cycle may refer to only when the task instance to be executed is the first task instance in which the first node executes in the current system cycle: and the second node is used for executing the task instance in the previous system period.

In this embodiment, when the checking module and the running module are read and executed by the processor, the operations performed by the checking module and the running module respectively correspond to steps S110 to S120 of the first embodiment, and other implementation details can be referred to in the first embodiment.

It will be understood by those skilled in the art that all or part of the steps of the above methods may be implemented by instructing the relevant hardware through a program, and the program may be stored in a computer readable storage medium, such as a read-only memory, a magnetic or optical disk, and the like. Alternatively, all or part of the steps of the above embodiments may be implemented using one or more integrated circuits. Accordingly, each module/unit in the above embodiments may be implemented in the form of hardware, and may also be implemented in the form of a software functional module. The present application is not limited to any specific form of hardware or software combination.

There are, of course, many other embodiments of the invention that can be devised without departing from the spirit and scope thereof, and it will be apparent to those skilled in the art that various changes and modifications can be made herein without departing from the spirit and scope of the invention.

Claims

1. A task scheduling method of a distributed task system comprises the following steps:

2. The task scheduling method of claim 1, wherein the task instance running in the previous cycle comprises one or more of:

a task instance of the node running in the previous period;

task instances of other nodes running in the previous cycle.

3. The task scheduling method of claim 2, wherein:

if the task instance to be run is the first task instance of the node in the current system cycle, the task instance of the node running in the previous cycle refers to: in the previous system period, the last task instance of the node is obtained;

if the task instance to be run is not the first task instance of the node in the current system cycle, the task instance of the node running in the previous cycle refers to: in the same system period, the task instance of the node is operated in the previous task period;

4. A task scheduling method according to claim 2 wherein the task instances of the other nodes running in the previous cycle include one or more of:

one or more task instances of other nodes operating in a previous system cycle;

5. The task scheduling method of claim 4, wherein:

if the task instance on which the task instance to be run depends includes the task instance of the node running in the previous cycle, the task instance of the other node running in the previous cycle refers to: task instances of other nodes running in the previous system cycle.

6. The task scheduling method of claim 4, wherein:

when the task instances on which the task instances of other nodes depend include task instances of other nodes running in a previous cycle, the one or more task instances of other nodes running in the previous system cycle include: the last task instance of the other nodes in the previous system cycle;

when the task instances depended on by the task instances of other nodes do not include the task instances of other nodes running in the previous cycle, the one or more task instances of other nodes running in the previous system cycle comprise: task instances of other nodes in the previous system cycle.

7. The task scheduling method of claim 2, wherein:

the node is a child node, and the other nodes are father nodes;

or,

the node is a father node, and the other nodes are child nodes.

8. A task scheduling device of a distributed task system, comprising: a processor and a memory;

the method is characterized in that:

9. The task scheduler of claim 8, wherein the task instance that was run in the previous cycle comprises one or more of:

a task instance of the node running in the previous period;

task instances of other nodes running in the previous cycle.

10. A task scheduler according to claim 9, wherein the task instances of other nodes running in the previous cycle include one or more of:

one or more task instances of other nodes operating in a previous system cycle;

11. The task scheduler of claim 9, wherein:

the node is a child node, and the other nodes are father nodes;

or,

the node is a father node, and the other nodes are child nodes.

12. A task scheduling apparatus of a distributed task system, comprising:

13. The task scheduler of claim 12, wherein the task instance that was run in the previous cycle comprises one or more of:

a task instance of the node running in the previous period;

task instances of other nodes running in the previous cycle.

14. The task scheduler of claim 13, wherein the task instance of the second node running in the previous cycle comprises one or more of:

one or more task instances of other nodes operating in a previous system cycle;

15. The task scheduler of claim 13, wherein:

the node is a child node, and the other nodes are father nodes;

or,

the node is a father node, and the other nodes are child nodes.