[go: up one dir, main page]

CN111324428A - Task allocation method, device, equipment and computer readable storage medium - Google Patents

Task allocation method, device, equipment and computer readable storage medium Download PDF

Info

Publication number
CN111324428A
CN111324428A CN201910895064.5A CN201910895064A CN111324428A CN 111324428 A CN111324428 A CN 111324428A CN 201910895064 A CN201910895064 A CN 201910895064A CN 111324428 A CN111324428 A CN 111324428A
Authority
CN
China
Prior art keywords
buffer pool
nodes
task
injection
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910895064.5A
Other languages
Chinese (zh)
Other versions
CN111324428B (en
Inventor
杜斌
赵健
吴承超
陶荣杰
高丛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision System Technology Co Ltd
Original Assignee
Hangzhou Hikvision System Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision System Technology Co Ltd filed Critical Hangzhou Hikvision System Technology Co Ltd
Priority to CN201910895064.5A priority Critical patent/CN111324428B/en
Publication of CN111324428A publication Critical patent/CN111324428A/en
Application granted granted Critical
Publication of CN111324428B publication Critical patent/CN111324428B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Factory Administration (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The embodiment of the application provides a task allocation method, a device, equipment and a computer readable storage medium, relates to the technical field of computers, and can improve task allocation efficiency. The task allocation method comprises the following steps: acquiring the number of task nodes in a buffer pool; determining whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, performing one-time traversal injection on the buffer pool, and allocating the tasks injected into the buffer pool to an actuator, wherein the performing one-time traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting the tasks in the task pool into the empty nodes to change the empty nodes into task nodes; and when the task execution result of the executor is obtained, the corresponding node of the executed task in the buffer pool is changed into a null node. The scheme of the application is mainly used for task scheduling.

Description

Task allocation method, device, equipment and computer readable storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for task allocation.
Background
The task system based on the network environment comprises an executor for executing tasks by using resources and a task pool for storing the tasks, wherein the task pool is provided with a fixed storage space for storing the tasks, the tasks are periodically injected into the task pool and issued to the executor to be executed by the executor, however, the task distribution efficiency in the current task distribution mode is lower.
Disclosure of Invention
The application provides a task allocation method, a device, equipment and a computer readable storage medium, which can improve task allocation efficiency.
In a first aspect, an embodiment of the present application provides a task allocation method, including:
acquiring the number of task nodes in a buffer pool;
determining whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, performing one-time traversal injection on the buffer pool, and allocating the tasks injected into the buffer pool to an actuator, wherein the performing one-time traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting the tasks in the task pool into the empty nodes to change the empty nodes into task nodes;
and when the task execution result of the executor is obtained, the corresponding node of the executed task in the buffer pool is changed into a null node.
Optionally, the method further includes:
periodically acquiring the number of times of traversing injection on the buffer pool within a first preset time;
and determining whether the number of times of traversing injection to the buffer pool in the first preset time meets a preset condition, if so, adjusting the number of nodes of the buffer pool.
Optionally, the injection threshold is positively correlated with the number of nodes of the buffer pool.
Optionally, the determining whether the number of times of traversing injection to the buffer pool within the first preset duration meets a preset condition, if so, the process of adjusting the number of nodes of the buffer pool includes:
determining whether the number of times of traversing injection to the buffer pool within the first preset time is greater than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
and determining whether the number of times of traversing injection to the buffer pool within the first preset time is less than a second preset time, if so, reducing the number of nodes of the buffer pool.
Optionally, the first preset number of times is positively correlated with the number of nodes in the buffer pool, and the second preset number of times is positively correlated with the number of nodes in the buffer pool.
Optionally, the change amplitude of the number of nodes of the buffer pool at each time is positively correlated with the value before the change of the number of nodes of the buffer pool at the time.
Optionally, the process of periodically obtaining the number of times of traversal injection performed on the buffer pool within a first preset time includes:
periodically acquiring the number of times of traversing injection to the buffer pool within a first preset time and the number of reference injection times, wherein the number of reference injection times is positively correlated with the number of nodes of the buffer pool;
obtaining the first preset times and the second preset times according to the reference injection times, wherein the first preset times are greater than the reference injection times and are positively correlated with the reference injection times, and the second preset times are less than the reference injection times and are positively correlated with the reference injection times;
after the number of the nodes of the buffer pool is increased every time, the ratio of the number of the nodes of the buffer pool after the current increase to the number of the nodes of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of the nodes of the buffer pool is reduced each time, the ratio of the number of the nodes of the buffer pool after the current reduction to the number of the nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
Optionally, the process of periodically obtaining the number of times of traversal injection performed on the buffer pool within a first preset time includes:
periodically acquiring the number N of traversal injection to the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time;
determining N, m whether p has no change within a second preset time, if yes, resetting the buffer pool;
the resetting the buffer pool comprises:
canceling the task allocation relation between tasks corresponding to all task nodes in the buffer pool and an actuator, and injecting the tasks corresponding to all the task nodes in the buffer pool into the task pool to enable the task nodes to become empty nodes so as to enable the buffer pool to be emptied;
and injecting the tasks in the task pool into the emptied buffer pool, changing all empty nodes in the buffer pool into task nodes, and distributing the tasks injected into the buffer pool to an actuator.
On the other hand, an embodiment of the present application further provides a task allocation apparatus, including:
the first acquisition module is used for acquiring the number of task nodes in the buffer pool;
an injection module, configured to determine whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, perform a traversal injection on the buffer pool, and allocate a task injected into the buffer pool to an actuator, where performing a traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting the tasks in the task pool into the empty nodes to change the empty nodes into task nodes;
and the execution result processing module is used for changing the corresponding node of the executed task in the buffer pool into a null node when the task execution result of the executor is obtained.
Optionally, the apparatus further comprises:
the second acquisition module is used for periodically acquiring the times of traversing injection on the buffer pool within a first preset time length;
and the node number adjusting module is used for determining whether the number of times of traversing injection of the buffer pool in the first preset time meets a preset condition, and if so, adjusting the node number of the buffer pool.
Optionally, the injection threshold is positively correlated with the number of nodes of the buffer pool.
Optionally, the node number adjusting module is specifically configured to:
determining whether the number of times of traversing injection to the buffer pool within the first preset time is greater than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
and determining whether the number of times of traversing injection to the buffer pool within the first preset time is less than a second preset time, if so, reducing the number of nodes of the buffer pool.
Optionally, the first preset number of times is positively correlated with the number of nodes in the buffer pool, and the second preset number of times is positively correlated with the number of nodes in the buffer pool.
Optionally, the change amplitude of the number of nodes of the buffer pool at each time is positively correlated with the value before the change of the number of nodes of the buffer pool at the time.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the number of times of traversing injection to the buffer pool within a first preset time and the number of reference injection times, wherein the number of reference injection times is positively correlated with the number of nodes of the buffer pool;
obtaining the first preset times and the second preset times according to the reference injection times, wherein the first preset times are greater than the reference injection times and are positively correlated with the reference injection times, and the second preset times are less than the reference injection times and are positively correlated with the reference injection times;
after the number of the nodes of the buffer pool is increased every time, the ratio of the number of the nodes of the buffer pool after the current increase to the number of the nodes of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of the nodes of the buffer pool is reduced each time, the ratio of the number of the nodes of the buffer pool after the current reduction to the number of the nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the number N of traversal injection to the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time;
determining N, m whether p has no change within a second preset time, if yes, resetting the buffer pool;
the resetting the buffer pool comprises:
canceling the task allocation relation between tasks corresponding to all task nodes in the buffer pool and an actuator, and injecting the tasks corresponding to all the task nodes in the buffer pool into the task pool to enable the task nodes to become empty nodes so as to enable the buffer pool to be emptied;
and injecting the tasks in the task pool into the emptied buffer pool, changing all empty nodes in the buffer pool into task nodes, and distributing the tasks injected into the buffer pool to an actuator.
On the other hand, an embodiment of the present application further provides a task allocation apparatus, including:
a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the method described above.
On the other hand, the embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the above method.
According to the task allocation method, the device, the equipment and the computer readable storage medium in the embodiment of the application, the buffer pool is arranged between the task pool and the actuator, the tasks in the task pool are injected into the buffer pool according to the judgment basis of the corresponding task quantity in the buffer pool, the tasks in the buffer pool are allocated to the corresponding actuator, decoupling between the actuator and the task pool is realized in a buffering mode, the task allocation process of the actuator cannot directly influence the task pool, and the task scheduling system can still keep high task allocation and execution efficiency on the condition of uneven resource distribution.
Drawings
Fig. 1 is a schematic view of an application scenario in an embodiment of the present application;
FIG. 2 is a schematic flowchart of a task allocation method according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of another task allocation method according to an embodiment of the present application;
FIG. 4 is a schematic flow chart illustrating some steps of another task allocation method according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of a refinement step of step 201 in FIG. 4;
FIG. 6 is a schematic flow chart of another refinement step of step 201 in FIG. 4;
FIG. 7 is a schematic flow chart of another refinement step of step 201 in FIG. 4;
FIG. 8 is a block diagram illustrating a task allocation apparatus according to an embodiment of the present application;
FIG. 9 is a block diagram of another task assigning apparatus according to an embodiment of the present application;
fig. 10 is a block diagram of a task allocation apparatus in an embodiment of the present application.
Detailed Description
The terminology used in the description of the embodiments section of the present application is for the purpose of describing particular embodiments of the present application only and is not intended to be limiting of the present application.
Before introducing the embodiments of the present application, a process of the embodiments of the present application provided by the inventor is first described, and the inventor finds that, in the prior art, a task pool has a fixed storage space for storing tasks, and the tasks are periodically injected into the task pool and issued to an executor, and since resources utilized by the executor are not uniformly distributed in time and space dimensions, task allocation efficiency is low. For example, if a server executing a task executes the task at night, since network transmission resources are sufficient at night, the time required for the server executing the task at night to execute the task is shorter than that required for the server executing the task at night in the daytime, which may cause that both parties receiving the task and allocating the task frequently access the task pool, thereby causing a competitive situation and resulting in low overall task allocation efficiency. In order to improve the above problems, the inventors provide embodiments of the present application, which are described below.
The embodiment of the application can be applied to a task scheduling system. Fig. 1 is a schematic view of an application scenario in an embodiment of the present application. The task scheduling system shown in fig. 1 mainly includes a task allocation device 11, an executor set 12, a buffer pool 13, and a task pool 14, where the task allocation device 11 and the executor set 12 may be servers or other network devices in a network environment, the executor set 12 includes at least one executor 120, the executor 120 is used for specifically executing tasks and feeding back task execution results, for example, the executor set 12 is a server, the executor 120 is a process, the executor 120 executes a task by using a network, resources used for executing the task include network bandwidth resources, and may also include other necessary resources, the resources are not uniformly distributed in the time and space dimensions, the task execution performance of the executor set 12 is related to the resources available, for example, the executors of the executor set 12 are executed more efficiently at idle times such as nights, and the executors of the executor set 12 are executed more efficiently as they are closer to the target location of the data transmitted by the task. The buffer pool 13 is used to store tasks to be executed, the buffer pool 13 may adaptively generate and correspond to the executor 120 according to the executor 120, so that the executor 120 and the buffer pool 13 have a one-to-one correspondence relationship, it may be understood that, in other implementable embodiments, one buffer pool may correspond to multiple executors, or one buffer pool may correspond to multiple executor sets, in fig. 1 and the description of the following embodiments, only one buffer pool 13 corresponds to one executor 120 as an example, the tasks in the buffer pool 13 are executed by the corresponding executor 120, the task pool 14 is used to store the tasks to be allocated, the buffer pool 13 and the task pool 14 may specifically be data storage structures in the server, the buffer pool 13 stores the tasks in the form of nodes, each node corresponds to one task, the larger the number of the nodes is, the larger the capacity of the buffer pool 13 is, the greater the number of tasks that can be stored.
An embodiment of the present application provides a task allocation method, where a task may be a task executed by using a network, for example, a video image detection task, and an execution main body of the method may be a task allocation device 11 in a task scheduling system, as shown in fig. 2, fig. 2 is a schematic flow diagram of a task allocation method in an embodiment of the present application, and the method may correspond to a task allocation method corresponding to one buffer pool 13 in fig. 1, and the method includes:
step 101, acquiring the number of task nodes in a buffer pool 13;
step 102, determining whether the number of task nodes in the buffer pool 13 is smaller than an injection threshold, if so, entering step 103, otherwise, entering step 101 again, and continuously monitoring the number of task nodes in the buffer pool 13;
103, performing once traversal injection on the buffer pool 13, distributing the task injected into the buffer pool 13 to the executor 120, and then entering step 104 and entering step 101; wherein, performing one-time traversal injection on the buffer pool 13 comprises: and traversing each node in the buffer pool 13, and injecting the tasks in the task pool 14 into the empty nodes so as to change the empty nodes into task nodes.
Specifically, after completing one-time traversal injection of the buffer pool 13 in step 103, step 101 and step 104 may be directly entered, that is, the number of task nodes in the buffer pool 13 and the task execution result are monitored in real time. In a specific traversal injection process, all nodes in the buffer pool 13 are traversed, if a node is found to be a task node, the buffer pool 13 is skipped to be continuously traversed, and if a node is found to be an empty node, a new task is injected from the task pool 14 to the node, so that the node becomes a task node until all nodes in the buffer pool 13 are task nodes, that is, one traversal injection is completed. It should be noted that, if the tasks in the task pool 14 are not enough to fill the buffer pool 13, the tasks in the task pool 14 are all injected, and then the traversal injection is completed. The task pool 14 is configured to store all to-be-allocated tasks received by the task scheduling system, after the tasks in the task pool 14 are injected into the buffer pool 13, the tasks are allocated to the executor 120, and when the tasks in the task pool 14 are allocated to the executor 120, the location information of the tasks is sent to the executor 120, for example, pooln. If a node in the buffer pool 13 is injected with a task, the node becomes a task node, and if a node in the buffer pool 13 does not store information related to the task, the node becomes an empty node. A process of injecting a task into the buffer pool 13, i.e., a process of changing a hollow node into a task node in the buffer pool 13.
Step 104, when the task execution result of the executor 120 is obtained, changing the corresponding node of the executed task in the buffer pool 13 into an empty node, and then entering step 101 to continuously monitor the number of task nodes in the buffer pool 13;
specifically, after the executor 120 executes the completed task, a task execution result is generated, which indicates that the task has been executed and is not required to be distributed or executed, and since the task execution result includes the location information of the task, the task distribution device 11 may find the corresponding node according to the location information and change the task node into a null node, so as to store other tasks.
The above steps indicate that when the number of task nodes in the buffer pool 13 drops below the injection threshold, a new task is supplemented from the task pool 14 and stored in the buffer pool 13 as a task to be executed by the executor 120, and the injection threshold may be a preset fixed value or a preset adjustable value, for example, for a certain buffer pool in which the number of nodes is 100, the injection threshold is set to be 50, the number of task nodes currently acquired in the buffer pool is 49 and less than 50, that is, the number of task nodes in the buffer pool reaches the injection condition, at this time, the buffer pool 13 is subjected to one traversal injection, after the traversal injection, the number of task nodes in the buffer pool 13 becomes 100, in the process that the task in the buffer pool 13 is gradually executed, the number of empty nodes gradually increases, the number of task nodes gradually decreases, until it is determined again that the number of task nodes is less than the injection threshold, and performing next traversal injection, and so on, because the buffer pool 13 is additionally arranged, the tasks in the task pool 14 are firstly injected into the buffer pool 13, and then the tasks are allocated to the corresponding executors 120 by the buffer pool 13, so that even if the time for the executors 120 to execute the tasks is short, the task receiving of the task pool 14 or the task allocation to other executors 120 is not directly influenced.
It should be noted that the execution order relationship among the above steps is only an example, and the execution order among the steps is not limited in the embodiment of the present application, for example, step 104 is executed after the execution result of the task is fed back by the actuator 120, and may be executed in a gap of the execution of other steps or in synchronization with other steps.
According to the task allocation method in the embodiment of the application, the buffer pool is arranged between the task pool and the actuator, the tasks in the task pool are injected into the buffer pool according to the judgment basis of the number of the corresponding tasks in the buffer pool, the tasks in the buffer pool are allocated to the corresponding actuator, decoupling between the actuator and the task pool is realized in a buffering mode, the task pool is not directly influenced in the task allocation process of the actuator, and the task scheduling system is enabled to keep high task allocation and execution efficiency integrally under the condition of uneven resource distribution.
Optionally, as shown in fig. 3, fig. 3 is a schematic flowchart of another task allocation method in an embodiment of the present application, where the task allocation method further includes:
step 201, periodically acquiring the number of times of traversing injection to the buffer pool 13 within a first preset time period, and then entering step 202;
step 202, determining whether the number of times of traversing injection to the buffer pool 13 in a first preset time period meets a preset condition, if so, entering step 203, and if not, entering step 101;
step 203, adjust the node number of the buffer pool 13, and then may proceed to step 101.
Specifically, steps 201 and 202 may be periodically executed to periodically determine whether the number of nodes in the buffer pool 13 needs to be adjusted, where the buffer pool 13 in the above steps is a certain buffer pool 13 in the task allocation system of fig. 1, that is, it is determined whether one buffer pool 13 meets the requirement for adjusting the number of nodes. In step 201, for example, the number of times of performing traversal injection on the buffer pool 13 in x seconds is obtained every x seconds, that is, every x seconds is a period, the number of times of performing traversal injection on the buffer pool 13 in the previous period is periodically obtained, the logic of traversal injection is the same as that in the above embodiment, when the number of task nodes in the buffer pool 13 is reduced to a preset value, the buffer pool 13 is filled, and the greater the number of times of performing traversal injection in x seconds is, the higher the execution efficiency of the executor 120 corresponding to the buffer pool 13 is indicated, and conversely, the smaller the number of times of performing traversal injection in x seconds is, the lower the execution efficiency of the executor 120 corresponding to the buffer pool 13 is indicated, so that the number of nodes in the buffer pool 13 can be dynamically adjusted according to the number of times of performing traversal injection on the buffer pool 13 in the first preset duration to improve the matching degree between the number of nodes in the buffer pool 13 and the execution efficiency of the executor 120, thereby further improving task allocation efficiency. The preset condition may be set as required, for example, a threshold y is set, when the number of times of performing traversal injection in x seconds is greater than y, the number of nodes in the buffer pool 13 is increased, and when the number of times of performing traversal injection in x seconds is less than y, the number of nodes in the buffer pool 13 is decreased.
Alternatively, in a task allocation method such as that shown in fig. 3, the injection threshold is positively correlated with the number of nodes in the buffer pool 13.
Specifically, for example, when the number of nodes in the buffer pool 13 is 100, the corresponding injection threshold is 70, that is, when the number of task nodes in the buffer pool 13 drops below 70, new tasks are supplemented from the task pool 14 and injected into the buffer pool 13; since the number of nodes in the buffer pool 13 is dynamically adjusted, when the number of nodes in the buffer pool 13 becomes 400, the injection threshold may become 300, that is, when the number of task nodes in the buffer pool 13 drops below 300, new tasks are supplemented from the task pool 14 and injected into the buffer pool 13; and so on. The larger the number of nodes in the buffer pool 13 is, the larger the corresponding injection threshold value is; the smaller the number of nodes in the buffer pool 13, the smaller the corresponding injection threshold. In this way, the number of task nodes stored in buffer pool 13 that matches the number of nodes in buffer pool 13 can be maintained without performing injection operations too frequently. The injection threshold may be proportional to the number of nodes in buffer pool 13, for example, the injection threshold is set to 50% of the number of nodes in buffer pool 13, and no matter how the number of nodes in buffer pool 13 changes, the traversal injection is performed as long as the number of task nodes therein drops below half of the number of nodes. It will be appreciated that in other realizable embodiments, the injection threshold may also be a fixed preset value, for example, the number of nodes of the buffer pool 13 is divided into three 80, 90 and 100 stages, and is dynamically adjusted in these three stages, but the injection threshold is 70 regardless of the value of the number of nodes of the buffer pool 13, i.e., the traversal injection is performed as long as the number of task nodes in the buffer pool 13 is less than 70.
Optionally, as shown in fig. 4, fig. 4 is a schematic flowchart of a part of steps in another task allocation method in this embodiment, where in the step 202, it is determined whether the number of times of performing traversal injection on the buffer pool 13 in the first preset time period satisfies a preset condition, and if so, the process of entering the step 203 and adjusting the number of nodes in the buffer pool 13 includes:
step 2021, determining whether the number of times of traversing injection to the buffer pool 13 within the first preset time duration is greater than a first preset number of times, if so, entering step 2031, and if not, entering step 2022;
step 2031, increase the number of nodes in the buffer pool 13, and then go to step 101;
step 2022, determining whether the number of times of traversing injection to the buffer pool 13 within the first preset time period is less than a second preset number of times, if yes, entering step 2032, and if not, entering step 101;
and step 2032, reducing the number of the nodes in the buffer pool.
Specifically, when the buffer pools 13 have the same number of nodes, the second preset number is less than or equal to the first preset number, for example, the number of nodes in a certain buffer pool 13 may be divided into three stages, 50, 100, and 200, the first preset duration is 10 minutes, the first preset number is 15, the second preset number is 10, and the number of times of traversal injection on the buffer pool 13 in the latest 10 minutes is obtained every 10 minutes. At the 10 th minute, the number of nodes in the buffer pool 13 is 100, the obtained number of traversal injection is 20, which exceeds the first preset number 15, so that the step 2031 is performed to increase the number of nodes in the buffer pool 13 to 200, and then the step 101 is performed to continuously monitor the number of task nodes in the buffer pool 13, and the step 102 is performed to determine whether to perform injection, it should be noted that, steps 102, 103, and 104 are not illustrated in fig. 4, and the specific processes of these steps may be the same as the process illustrated in fig. 3; in the 20 th minute, the number of traversal times acquired in step 201 is 19, although the number exceeds the first preset time 15, the number of nodes is unchanged because the number of nodes in the buffer pool 13 has reached the maximum number of nodes 200, and then step 101 is entered to continuously monitor the number of nodes in the buffer pool 13 and perform injection when the number falls to the injection threshold; in the 30 th minute, the number of traversal obtained in step 201 is 5, which is less than the second preset number 10, so step 2032 is entered, the number of nodes in the buffer pool 13 is reduced to 100, and then step 101 is entered; in the 40 th minute, the number of traversal obtained in step 201 is 6, which is less than the second preset number of times 10, so step 2032 is entered, the number of nodes in the buffer pool 13 is reduced to 50, and then step 101 is entered; by analogy, when the number of times of traversing injection to the buffer pool 13 in the first preset duration is large, it indicates that the execution efficiency of the corresponding actuator 120 is high, so that the number of nodes in the buffer pool 13 is increased, and it can be avoided that frequent injection reduces the allocation efficiency of the buffer pool 13 and the task pool 14, and when the number of times of traversing injection to the buffer pool 13 in the first preset duration is small, it indicates that the execution efficiency of the corresponding actuator 120 is low, so that the number of nodes in the buffer pool 13 is reduced, and a node space with low use efficiency can be released, so that the released node can be used for storing other data, and the utilization rate of the storage space is improved. Since the executor 120 will continue to execute tasks, so that the tasks in the buffer pool 13 are gradually reduced, the process may enter step 101 after the other steps are completed, to determine whether the injection condition is satisfied, and ensure that the tasks can be injected into the buffer pool 13 in time. Therefore, the number of nodes in the buffer pool 13 is dynamically adjusted according to the number of times of traversal injection performed on the buffer pool 13 within the first preset time duration, so that the matching degree between the number of nodes in the buffer pool 13 and the execution efficiency of the executor 120 can be improved, and the task allocation efficiency can be improved. In other realizable embodiments, for example, the number of nodes in a certain buffer pool 13 defaults to 100, the number of times of traversal injection for the buffer pool 13 in the last 5 minutes is obtained every 10 minutes, if the number of nodes is more than 20, the number of nodes in the buffer pool 13 is increased by 20, and if the number of nodes is less than 10, the number of nodes in the buffer pool 13 is decreased by 10.
It should be noted that, in the embodiment of the present application, the execution order of each step is not limited, for example, in addition to the step execution shown in fig. 4, in other realizable implementations, step 2022 may be executed first in each cycle to determine whether to reduce the number of nodes in the buffer pool 13, and if not, step 2021 is executed again to determine whether to increase the number of nodes in the buffer pool 13; it is understood that step 101 may be entered to determine whether to go through the injection in each cycle, and then step 2021 and step 2022 are executed to determine whether to adjust the number of nodes in the buffer pool 13.
Optionally, the first preset number of times is positively correlated with the number of nodes in the buffer pool 13, and the second preset number of times is positively correlated with the number of nodes in the buffer pool 13.
Specifically, for example, the number of nodes in a certain buffer pool 13 can be divided into three grades 50, 100 and 200, and the first preset time period is 10 minutes. When the number of nodes in the buffer pool 13 is 50, the first preset number of times is 10, and the second preset number of times is 5; when the number of the nodes in the buffer pool 13 is 100, the first preset number is 15, and the second preset number is 10; when the number of nodes in the buffer pool 13 is 200, the first preset number is 20, and the second preset number is 15. The number of times the buffer pool 13 has been injected in the last 10 minutes is acquired every 10 minutes. At the 10 th minute, the number of nodes in the buffer pool 13 is 100, the obtained number of times of traversal injection is 20, and exceeds the first preset number of times 15 at the gear, so that the step 2031 is performed to increase the number of nodes in the buffer pool 13 to 200, the step 101 is performed to obtain the number of task nodes in the buffer pool 13, and the step 102 is performed to determine whether to perform injection; in the 20 th minute, the obtained traversal number is 19, and the number of the nodes is between the first preset number 20 and the second preset number 15 in the gear, so that the number of the nodes is unchanged, then the method enters the step 101, the number of the nodes in the buffer pool 13 is continuously monitored, and the injection is performed when the number of the nodes drops to the injection threshold; in the 30 th minute, the obtained traversal number is 5, which is smaller than the second preset number 15 in the gear, so that the step 2032 is performed, the number of nodes in the buffer pool 13 is reduced to 100, and then the step 101 is performed; by analogy, after the number of the nodes of the buffer pool 13 is adjusted, the first preset times and the second preset times are dynamically adjusted, and the first preset times and the second preset times are positively correlated to the number of the nodes of the buffer pool 13, so that the condition for judging whether the number of the nodes of the buffer pool 13 is adjusted at each time can be adapted to the number of the nodes of the current buffer pool 13.
Alternatively, the magnitude of the change in the number of nodes per buffer pool 13 is positively correlated with the value before the change in the number of nodes of the buffer pool 13.
Specifically, for example, the number of nodes in a certain buffer pool 13 may be divided into five stages, i.e., 25, 50, 100, 200, and 400, and the number of nodes in the buffer pool 13 is adjusted step by step, e.g., the number of nodes in the current buffer pool 13 is 25, it is determined that the number of times of traversal injection performed on the buffer pool 13 within a first preset time period is greater than a first preset number of times, the number of nodes in the buffer pool 13 is increased to 50, and the change amplitude of the number of nodes is 25; determining that the number of times of traversing injection to the buffer pool 13 in the first preset time duration is greater than the first preset number of times, increasing the number of nodes of the buffer pool 13 to 100, wherein the change amplitude of the number of nodes is 50, and the value before the change of the number of nodes of the buffer pool 13 is 50; and determining that the number of times of traversing injection to the buffer pool 13 in the first preset time duration is greater than the first preset number of times next time, increasing the number of nodes of the buffer pool 13 to 200, wherein the change amplitude of the number of nodes is 100, and the value before the change of the number of nodes of the buffer pool 13 is 100. When the number of nodes in the buffer pool 13 is adjusted each time, the change amplitude of the number of nodes is positively correlated with the number of nodes before the change, that is, when the number of nodes in the buffer pool 13 is large, the number of nodes is increased or decreased to avoid frequent adjustment, and when the number of nodes in the buffer pool 13 is small, the number of nodes is increased or decreased to avoid over-adjustment.
Optionally, with reference to fig. 4 and fig. 5, fig. 5 is a schematic flowchart of a refinement step of step 201 in fig. 4, where in step 201, periodically obtaining the number of times of performing traversal injection on the buffer pool 13 within the first preset time period includes:
step 2011, periodically acquiring the number of times of traversal injection performed on the buffer pool 13 within a first preset time period and a reference injection number, where the reference injection number is positively correlated with the number of nodes of the buffer pool 13, for example, when the number m of nodes of the buffer pool 13 is 100, the corresponding reference injection number n is 10, when the number m of nodes of the buffer pool 13 is 200, the corresponding reference injection number n is 20, and when the number m of nodes of the buffer pool 13 is 400, the corresponding reference injection number n is 40;
2012, obtaining a first preset number of times and a second preset number of times according to the reference injection number of times, where the first preset number of times is greater than the reference injection number of times and is positively correlated with the reference injection number of times, and the second preset number of times is less than the reference injection number of times and is positively correlated with the reference injection number of times, for example, the first preset number of times n1(1+ r) × n, r > 0, r representing the multiple of the node number increase, a second preset number of times n2In the following description, R is 1 and R is 0.5, and when n is 10, n is1＝20，n2When n is 20, n is 51＝40，n2＝10；
After step 2031 is executed each time, the number of nodes in the buffer pool 13 is increased, and the ratio of the number of nodes in the buffer pool 13 after the current increase to the number of nodes in the buffer pool 13 before the current increase is equal to the ratio of the first preset number of times to the reference injection number of times;
specifically, for example, the number of nodes of the buffer pool 13 is increased to m ═ 1+ r) × m ', m' is the number of nodes of the buffer pool 13 before the increase, m is the number of nodes of the buffer pool 13 after the increase, and the ratio of the number of nodes of the buffer pool 13 after the increase to the number of nodes of the buffer pool 13 before the increase is
Figure BDA0002209938040000151
n1(1+ r) × n, the ratio of the first preset number of times to the reference injection number of times being
Figure BDA0002209938040000152
Namely, it is
Figure BDA0002209938040000153
For example, when the number of nodes in the buffer pool 13 is 100, n is 10, n1＝20，n2The number N of times of obtaining the traversal injection to the buffer pool 13 within the first preset duration is 44 > N ═ 51That is, step 2031 is executed to increase the number of nodes in the buffer pool 13, and increase the number of nodes in the buffer pool 13 to m 2 × 100 to 200, in which the reference injection frequency is changed with the number of nodes in the buffer pool 13 to n (1+ r) × n ', n' is the reference injection frequency before the change of the number of nodes in the buffer pool 13, n is the reference injection frequency after the change of the number of nodes in the buffer pool 13, that is, n 2 × 10 is 20, and the first preset frequency n is changed with the change of the reference injection frequency n1And a second predetermined number n of times2Are all changed, n1＝40，n2Then, the next acquisition of N44 > N1That is, step 2031 is executed to increase the number of nodes in the buffer pool 13, and increase the number of nodes in the buffer pool 13 to m 2 × 200 400, at which time, the reference injection frequency is changed to n 2 × 20 40, n is changed to n 35201＝80，n220. It should be noted that, if slowThe number of nodes in the flushing pool 13 has reached the maximum number even if N > N is satisfied1Step 2031 is also not performed to prevent the buffer pool 13 from over-occupying resources.
After reducing the number of nodes of the buffer pool 13 each time step 2032 is performed, the ratio of the number of nodes of the buffer pool 13 after the current reduction to the number of nodes of the buffer pool 13 before the current reduction is equal to the ratio of the second preset number of times to the reference injection number of times.
Specifically, for example, the number of nodes of the buffer pool 13 is reduced to m ═ 1-R × m ″, where m ″ is the number of nodes of the buffer pool 13 before reduction, m is the number of nodes of the buffer pool 13 after reduction, and the ratio of the number of nodes of the buffer pool 13 after reduction to the number of nodes of the buffer pool 13 before reduction is
Figure BDA0002209938040000154
n2(1-R) × n, the ratio of the second preset number of times to the reference injection number of times being
Figure BDA0002209938040000155
Namely, it is
Figure BDA0002209938040000156
For example, when the number of nodes in the buffer pool 13 is 400, n is 40, n1＝80，n220, the number N of times of performing traversal injection on the buffer pool 13 within the obtained first preset time duration is 8 < N2That is, step 2032 is executed to decrease the number of nodes in the buffer pool 13 and decrease the number of nodes in the buffer pool 13 to m 0.5 × 400 200, where the number of reference injections varies with the number of nodes in the buffer pool 13 and is changed to n (1-R) × n ', where n' is the number of reference injections before the number of nodes in the buffer pool 13 varies, n is the number of reference injections after the number of nodes in the buffer pool 13 varies, that is, n is 0.5 × 40 20, and n is the number of reference injections after the number of nodes in the buffer pool 13 varies, that is, n is 0.5 ×1＝40，n210, then, the next time N is obtained 8 < N2That is, step 2032 is executed to decrease the number of nodes in the buffer pool 13 to m 0.5 × 200 100, and at this time, the number of reference injections is changed to n 0.5 × 20 10, n is changed to n 0.5 × 10, and n is changed to n1＝20，n25. It should be noted that if the number of nodes in the buffer pool 13 has reached the minimum number, even if N < N is satisfied2Step 2032 is also not performed any more to prevent the number of nodes in the buffer pool 13 from being too small.
The reference injection frequency N may be a preset empirical value or a preset test value, and is used to represent an injection frequency of the buffer pool 13 under a normal condition, that is, an injection frequency standard value, and if the number N of times of traversing injection performed on the buffer pool 13 within a first preset duration is greater than N, it indicates that the injection frequency of the current buffer pool 13 is relatively fast, so that each time when the actual injection frequency value is greater than 1+ r times of the injection frequency standard value, the number of nodes of the buffer pool 13 is increased to 1+ r times of the original number, so that the matching degree between the number of nodes of the buffer pool 13 and the injection frequency is closer to each other, but does not change too much; similarly, if N < N, it means that the injection frequency of the current buffer pool 13 is slow, and therefore, each time when the actual injection frequency value is smaller than the standard injection frequency value by 1-R times, the number of nodes in the buffer pool 13 is reduced to 1-R times, so that the matching degree between the number of nodes in the buffer pool 13 and the injection frequency is closer, but does not change too much.
In addition, in other realizable embodiments, the value obtained after the change of the number of nodes in each buffer pool 13 may be positively correlated with the number N of times of performing traversal injection on the buffer pool within the first preset duration, for example, in a standard case, the number of times of performing traversal injection on the buffer pool 13 with the number of nodes of 100 is 10 within the first preset duration, the number of nodes in the current buffer pool 13 is 100, at this time, it is obtained that the number of times of performing traversal injection on the buffer pool 13 within the first preset duration is 30, which is greater than the first preset number, and since the actual value of the injection frequency is 3 times of the standard value of the injection frequency, the number of nodes in the buffer pool 13 may be increased to 3 times of the standard frequency, that is, increased from 100 to 300; similarly, the number of nodes in the current buffer pool 13 is 100, and at this time, it is acquired that the number of times of performing traversal injection on the buffer pool 13 within the first preset time duration is 50, which is greater than the first preset number of times, and since the actual value of the injection frequency is 5 times of the standard value of the injection frequency, the number of nodes in the buffer pool 13 may be increased to 5 times of the standard frequency, that is, increased from 100 to 500.
In the process of reducing the number of nodes in the buffer pool 13 in step 2032, if the change value of the number of nodes in the buffer pool 13 is less than or equal to the number of empty nodes in the buffer pool 13, all the task nodes in the buffer pool 13 are reserved, at least part of the empty nodes in the buffer pool 13 are removed, and if the change value of the number of nodes in the buffer pool 13 is greater than the number of empty nodes in the buffer pool 13, all the empty nodes and part of the task nodes in the buffer pool 13 are removed, the task corresponding to the removed task node in the buffer pool 13 is injected into the task pool 14, and the task allocation relationship between the task corresponding to the removed task node in the buffer pool 13 and the executor 120 is cancelled. For example, before the number of nodes in the buffer pool 13 is reduced from 100 to 50, if there are 40 task nodes and 60 empty nodes in 100 nodes, it is only necessary to directly remove 50 of the 60 empty nodes; if there are 60 task nodes and 40 empty nodes in 100 nodes, first removing 40 empty nodes, then selecting 10 task nodes from the 60 task nodes, injecting the tasks in the 10 task nodes into the task pool 14, and simultaneously revoking the task allocation relationship between the tasks corresponding to the previous 10 task nodes and the executor 120, so that the executor 120 no longer stores information related to the tasks, and the tasks no longer serve as to-be-executed tasks to be processed by the executor 120. The tasks injected into the task pool 14 may be distributed again according to the logic of the task pool 14 itself, so as to avoid the loss of the tasks and ensure that the removed tasks in the buffer pool 13 will be executed in the subsequent process.
Optionally, with reference to fig. 4 and fig. 6, fig. 6 is a schematic flowchart of another refinement step of step 201 in fig. 4, where the step 201, the process of periodically obtaining the number of times of performing traversal injection on the buffer pool 13 within the first preset time period includes:
step 2011, periodically acquiring the number N of traversal injection to the buffer pool, the number m of nodes in the buffer pool 13 and the number p of task nodes within a first preset time period, and then entering step 2012;
step 2012, determining whether N, m and p are unchanged within a second preset time period, if yes, going to step 2013, and if not, going to step 2021;
step 2013, reset buffer pool 13, then may proceed to step 101.
Step 2013, resetting the buffer pool 13 includes:
canceling the task allocation relationship between the tasks corresponding to all the task nodes in the buffer pool 13 and the executor 120, and injecting the tasks corresponding to all the task nodes in the buffer pool 13 into the task pool 14 to make the task nodes become empty nodes, so that the buffer pool 13 is emptied;
and injecting the tasks in the task pool 14 into the emptied buffer pool 13, changing all empty nodes in the buffer pool 13 into task nodes, and distributing the tasks injected into the buffer pool 13 to the executor 120.
Specifically, after the parameters N, m and p of the buffer pool 13 are periodically acquired, a reset determination is first performed, for example, if each parameter in the buffer pool 13 has not changed for a long time, it indicates that an exception has occurred in task allocation, execution, or node processing in the buffer pool 13, and in order to ensure task processing efficiency, the buffer pool 13 may be reset to return to an initial state. In the process of resetting, the tasks in the buffer pool 13 are returned to the task pool 14, the process of returning the tasks includes canceling the task allocation relationship between the tasks in the buffer pool 13 and the executor 120, and injecting the tasks in the buffer pool 13 into the task pool 14, so as to avoid the loss of the tasks in the resetting process, ensure that the tasks in the buffer pool 13 can be executed in the subsequent process, and then re-inject the tasks in the task pool 14 into the buffer pool 13, so as to recover the task allocation process of the buffer pool 13.
In addition, as shown in fig. 4 and fig. 7, fig. 7 is a schematic flowchart of another detailed step of step 201 in fig. 4, where the step 201, the process of periodically obtaining the number of times of performing traversal injection on the buffer pool 13 within the first preset time period includes:
step 2011, periodically acquiring the number N of traversal injection to the buffer pool within a first preset time period, the number N of reference injection, the number m of nodes in the buffer pool 13 and the number p of task nodes, and then entering step 2012;
step 2012, determining whether N, m and p are unchanged within a second preset time period, if yes, going to step 2013, and if not, going to step 2014;
step 2013, resetting the buffer pool 13, and then entering step 101, wherein the process of resetting the buffer pool 13 is the same as the process described above, and is not described again here;
step 2014, obtaining a first preset number and a second preset number according to the reference injection number n, where the first preset number is greater than the reference injection number and is positively correlated with the reference injection number, and the second preset number is less than the reference injection number and is positively correlated with the reference injection number.
The specific process and principle of step 2014 are the same as those of step 2012 in fig. 5, and are not described herein again.
Optionally, the buffer pool has a Key-Value binary tree structure, where Key is used to store an index of a node in the buffer pool, Value is used to store an address, if the address points to the task information, the node is a task node, and if the address points to null, the node is a null node. A Key-Value binary tree structure is used as a data storage structure of a buffer pool, a binary tree which accords with search performance can be constructed, so that the task position can be quickly positioned, the completed task can be cleaned, and meanwhile, the efficient injection of the task by traversing the binary tree is not influenced.
It should be noted that the above-mentioned task allocation method is only an allocation method corresponding to one buffer pool 13 in the task scheduling system shown in fig. 1, and task allocation can be performed by the same method for each buffer pool 13 in the system.
It is to be understood that some or all of the steps or operations in the above-described embodiments are merely examples, and other operations or variations of various operations may be performed by the embodiments of the present application. Further, the various steps may be performed in a different order presented in the above-described embodiments, and it is possible that not all of the operations in the above-described embodiments are performed.
An embodiment of the present application further provides a task allocation apparatus, as shown in fig. 8, where fig. 8 is a block diagram of a structure of the task allocation apparatus in the embodiment of the present application, where the apparatus includes: a first obtaining module 1, configured to obtain the number of task nodes in a buffer pool; the injection module 2 is configured to determine whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, perform traversal injection on the buffer pool once, and allocate a task injected into the buffer pool to the actuator, where performing traversal injection on the buffer pool once includes: traversing each node in the buffer pool, and injecting the tasks in the task pool into the empty nodes to change the empty nodes into task nodes; and the execution result processing module 3 is configured to, when the task execution result of the executor is obtained, change the corresponding node of the executed task in the buffer pool into an empty node.
The task allocation apparatus may apply the task allocation method in the above embodiment, and the specific working process and principle thereof may be the same as those described in the above embodiment, and are not described herein again.
According to the task allocation device in the embodiment of the application, the buffer pool is arranged between the task pool and the actuator, the tasks in the task pool are injected into the buffer pool according to the judgment basis of the number of the corresponding tasks in the buffer pool, the tasks in the buffer pool are allocated to the corresponding actuator, decoupling between the actuator and the task pool is realized in a buffering mode, the task pool is not directly influenced in the task allocation process of the actuator, and therefore the task scheduling system can still keep high task allocation and execution efficiency on the whole under the condition of uneven resource distribution.
Optionally, as shown in fig. 9, fig. 9 is a block diagram of another task allocation apparatus in an embodiment of the present application, where the apparatus further includes: the second obtaining module 4 is configured to periodically obtain the number of times of traversal injection performed on the buffer pool within a first preset duration; and the node number adjusting module 5 is configured to determine whether the number of times of traversing injection to the buffer pool within the first preset time period meets a preset condition, and if so, adjust the node number of the buffer pool.
Optionally, the injection threshold is positively correlated to the number of nodes of the buffer pool.
Optionally, the node number adjusting module is specifically configured to: determining whether the number of times of traversing injection to the buffer pool within a first preset time length is greater than a first preset number of times, if so, increasing the number of nodes of the buffer pool; and determining whether the number of times of traversing injection to the buffer pool within the first preset time is less than a second preset time, if so, reducing the number of nodes of the buffer pool.
Optionally, the first preset number of times is positively correlated with the number of nodes in the buffer pool, and the second preset number of times is positively correlated with the number of nodes in the buffer pool.
Optionally, the magnitude of the change of the number of nodes in each buffer pool is positively correlated with the value before the change of the number of nodes in the buffer pool.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the number of times of traversing injection to the buffer pool within a first preset time and the number of reference injection times, wherein the number of reference injection times is positively correlated with the number of nodes of the buffer pool;
obtaining a first preset number and a second preset number according to the reference injection number, wherein the first preset number is greater than the reference injection number and is positively correlated with the reference injection number, and the second preset number is less than the reference injection number and is positively correlated with the reference injection number;
after the number of the nodes of the buffer pool is increased every time, the ratio of the number of the nodes of the buffer pool after the current increase to the number of the nodes of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of the nodes of the buffer pool is reduced each time, the ratio of the number of the nodes of the buffer pool after the current reduction to the number of the nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
Optionally, the second obtaining module is specifically configured to:
periodically acquiring the number N of traversal injection to the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time;
determining N, m whether p has no change within a second preset time, if yes, resetting the buffer pool;
resetting the buffer pool includes:
canceling the task allocation relation between tasks corresponding to all task nodes in the buffer pool and the actuator, and injecting the tasks corresponding to all the task nodes in the buffer pool into the task pool to change the task nodes into empty nodes so as to empty the buffer pool;
and injecting the tasks in the task pool into the emptied buffer pool, so that all the empty nodes in the buffer pool become task nodes, and distributing the tasks injected into the buffer pool to the executors.
It should be understood that the division of the modules of the apparatuses shown in fig. 8 to 9 is merely a logical division, and the actual implementation may be wholly or partially integrated into one physical entity or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling by the processing element in software, and part of the modules can be realized in the form of hardware. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when some of the above modules are implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling programs.
An embodiment of the present application further provides a task allocation device, as shown in fig. 10, where fig. 10 is a block diagram of a structure of a task allocation device in an embodiment of the present application, and the task allocation device includes: a processor 610 and a memory 620, the memory 620 being configured to store at least one instruction that is loaded and executed by the processor 610 to implement the method in the above-described embodiments. The number of the processors 610 may be one or more, and in fig. 8, one processor 610 is taken as an example, the processor 610 and the memory 620 may be connected by a bus or other means, and in fig. 10, the connection by a bus is taken as an example.
The memory 620, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the transmission methods in the embodiments of the present application. The processor 610 executes various functional applications and data processing by executing non-transitory software programs, instructions and modules stored in the memory 620, i.e., implements the methods in any of the method embodiments described above.
The memory 620 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; and necessary data, etc. Further, the memory 620 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device.
The task allocation device may be, for example, a server, and is applied to a task scheduling system as shown in fig. 1, as the task allocation apparatus 11 therein, where the executor set 12 may be, for example, different servers, the multiple executors 120 in the same executor set 12 may be, for example, different processes configured in the servers, and the buffer pool 13 and the task pool 14 may be formed by data storage areas in the task allocation apparatus 11 or other servers.
Embodiments of the present application further provide a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the method described in the above embodiments.
In addition, the present application also provides a computer program product, which includes a computer program, when the computer program product runs on a computer, the computer is caused to execute the method described in the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions described in accordance with the present application are generated, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid state disk), among others.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (18)

1. A task allocation method, comprising:
acquiring the number of task nodes in a buffer pool;
determining whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, performing one-time traversal injection on the buffer pool, and allocating the tasks injected into the buffer pool to an actuator, wherein the performing one-time traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting the tasks in the task pool into the empty nodes to change the empty nodes into task nodes;
and when the task execution result of the executor is obtained, the corresponding node of the executed task in the buffer pool is changed into a null node.
2. The method of claim 1, further comprising:
periodically acquiring the number of times of traversing injection on the buffer pool within a first preset time;
and determining whether the number of times of traversing injection to the buffer pool in the first preset time meets a preset condition, if so, adjusting the number of nodes of the buffer pool.
3. The method of claim 2,
the injection threshold is positively correlated with the number of nodes of the buffer pool.
4. The method of claim 2,
the determining whether the number of times of traversing injection to the buffer pool in the first preset time period meets a preset condition, if so, the process of adjusting the number of nodes of the buffer pool comprises:
determining whether the number of times of traversing injection to the buffer pool within the first preset time is greater than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
and determining whether the number of times of traversing injection to the buffer pool within the first preset time is less than a second preset time, if so, reducing the number of nodes of the buffer pool.
5. The method of claim 4,
the first preset times are positively correlated with the number of the nodes in the buffer pool, and the second preset times are positively correlated with the number of the nodes in the buffer pool.
6. The method of claim 4,
the change amplitude of the number of the nodes of the buffer pool at each time is positively correlated with the value before the change of the number of the nodes of the buffer pool at the time.
7. The method of claim 4,
the process of periodically acquiring the number of times of traversing injection of the buffer pool within a first preset time comprises the following steps:
periodically acquiring the number of times of traversing injection to the buffer pool within a first preset time and the number of reference injection times, wherein the number of reference injection times is positively correlated with the number of nodes of the buffer pool;
obtaining the first preset times and the second preset times according to the reference injection times, wherein the first preset times are greater than the reference injection times and are positively correlated with the reference injection times, and the second preset times are less than the reference injection times and are positively correlated with the reference injection times;
after the number of the nodes of the buffer pool is increased every time, the ratio of the number of the nodes of the buffer pool after the current increase to the number of the nodes of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of the nodes of the buffer pool is reduced each time, the ratio of the number of the nodes of the buffer pool after the current reduction to the number of the nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
8. The method of claim 2,
the process of periodically acquiring the number of times of traversing injection of the buffer pool within a first preset time comprises the following steps:
periodically acquiring the number N of traversal injection to the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time;
determining N, m whether p has no change within a second preset time, if yes, resetting the buffer pool;
the resetting the buffer pool comprises:
canceling the task allocation relation between tasks corresponding to all task nodes in the buffer pool and an actuator, and injecting the tasks corresponding to all the task nodes in the buffer pool into the task pool to enable the task nodes to become empty nodes so as to enable the buffer pool to be emptied;
and injecting the tasks in the task pool into the emptied buffer pool, changing all empty nodes in the buffer pool into task nodes, and distributing the tasks injected into the buffer pool to an actuator.
9. A task assigning apparatus, comprising:
the first acquisition module is used for acquiring the number of task nodes in the buffer pool;
an injection module, configured to determine whether the number of task nodes in the buffer pool is smaller than an injection threshold, if so, perform a traversal injection on the buffer pool, and allocate a task injected into the buffer pool to an actuator, where performing a traversal injection on the buffer pool includes: traversing each node in the buffer pool, and injecting the tasks in the task pool into the empty nodes to change the empty nodes into task nodes;
and the execution result processing module is used for changing the corresponding node of the executed task in the buffer pool into a null node when the task execution result of the executor is obtained.
10. The apparatus of claim 9, further comprising:
the second acquisition module is used for periodically acquiring the times of traversing injection on the buffer pool within a first preset time length;
and the node number adjusting module is used for determining whether the number of times of traversing injection of the buffer pool in the first preset time meets a preset condition, and if so, adjusting the node number of the buffer pool.
11. The apparatus of claim 10,
the injection threshold is positively correlated with the number of nodes of the buffer pool.
12. The apparatus of claim 10,
the node number adjusting module is specifically configured to:
determining whether the number of times of traversing injection to the buffer pool within the first preset time is greater than a first preset number of times, if so, increasing the number of nodes of the buffer pool;
and determining whether the number of times of traversing injection to the buffer pool within the first preset time is less than a second preset time, if so, reducing the number of nodes of the buffer pool.
13. The apparatus of claim 12,
the first preset times are positively correlated with the number of the nodes in the buffer pool, and the second preset times are positively correlated with the number of the nodes in the buffer pool.
14. The apparatus of claim 12,
the change amplitude of the number of the nodes of the buffer pool at each time is positively correlated with the value before the change of the number of the nodes of the buffer pool at the time.
15. The apparatus of claim 12,
the second obtaining module is specifically configured to:
periodically acquiring the number of times of traversing injection to the buffer pool within a first preset time and the number of reference injection times, wherein the number of reference injection times is positively correlated with the number of nodes of the buffer pool;
obtaining the first preset times and the second preset times according to the reference injection times, wherein the first preset times are greater than the reference injection times and are positively correlated with the reference injection times, and the second preset times are less than the reference injection times and are positively correlated with the reference injection times;
after the number of the nodes of the buffer pool is increased every time, the ratio of the number of the nodes of the buffer pool after the current increase to the number of the nodes of the buffer pool before the current increase is equal to the ratio of the first preset times to the reference injection times;
after the number of the nodes of the buffer pool is reduced each time, the ratio of the number of the nodes of the buffer pool after the current reduction to the number of the nodes of the buffer pool before the current reduction is equal to the ratio of the second preset times to the reference injection times.
16. The apparatus of claim 10,
the second obtaining module is specifically configured to:
periodically acquiring the number N of traversal injection to the buffer pool, the number m of nodes in the buffer pool and the number p of task nodes in a first preset time;
determining N, m whether p has no change within a second preset time, if yes, resetting the buffer pool;
the resetting the buffer pool comprises:
canceling the task allocation relation between tasks corresponding to all task nodes in the buffer pool and an actuator, and injecting the tasks corresponding to all the task nodes in the buffer pool into the task pool to enable the task nodes to become empty nodes so as to enable the buffer pool to be emptied;
and injecting the tasks in the task pool into the emptied buffer pool, changing all empty nodes in the buffer pool into task nodes, and distributing the tasks injected into the buffer pool to an actuator.
17. A task assigning apparatus, comprising:
a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the method of any one of claims 1 to 8.
18. A computer-readable storage medium, in which a computer program is stored which, when run on a computer, causes the computer to carry out the method according to any one of claims 1 to 8.
CN201910895064.5A 2019-09-20 2019-09-20 Task allocation method, device, equipment and computer readable storage medium Active CN111324428B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910895064.5A CN111324428B (en) 2019-09-20 2019-09-20 Task allocation method, device, equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910895064.5A CN111324428B (en) 2019-09-20 2019-09-20 Task allocation method, device, equipment and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN111324428A true CN111324428A (en) 2020-06-23
CN111324428B CN111324428B (en) 2023-08-22

Family

ID=71166863

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910895064.5A Active CN111324428B (en) 2019-09-20 2019-09-20 Task allocation method, device, equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN111324428B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5781801A (en) * 1995-12-20 1998-07-14 Emc Corporation Method and apparatus for receive buffer management in multi-sender communication systems
US5784698A (en) * 1995-12-05 1998-07-21 International Business Machines Corporation Dynamic memory allocation that enalbes efficient use of buffer pool memory segments
WO2002019168A1 (en) * 2000-08-25 2002-03-07 Hayes Scott R Heuristic automated method for ideal bufferpool tuning in a computer database
CN103605567A (en) * 2013-10-29 2014-02-26 河海大学 Cloud computing task scheduling method facing real-time demand change
CN104580396A (en) * 2014-12-19 2015-04-29 华为技术有限公司 Task scheduling method, node and system
WO2017113278A1 (en) * 2015-12-31 2017-07-06 华为技术有限公司 Data processing method, apparatus and system
CN107153573A (en) * 2016-03-02 2017-09-12 阿里巴巴集团控股有限公司 Distributed task scheduling treating method and apparatus
CN108510150A (en) * 2018-02-01 2018-09-07 东华大学 A kind of spinning CPS and its real-time task processing method based on edge calculations
WO2018219480A1 (en) * 2017-05-29 2018-12-06 Barcelona Supercomputing Center - Centro Nacional De Supercomputación Managing task dependency

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5784698A (en) * 1995-12-05 1998-07-21 International Business Machines Corporation Dynamic memory allocation that enalbes efficient use of buffer pool memory segments
US5781801A (en) * 1995-12-20 1998-07-14 Emc Corporation Method and apparatus for receive buffer management in multi-sender communication systems
WO2002019168A1 (en) * 2000-08-25 2002-03-07 Hayes Scott R Heuristic automated method for ideal bufferpool tuning in a computer database
CN103605567A (en) * 2013-10-29 2014-02-26 河海大学 Cloud computing task scheduling method facing real-time demand change
CN104580396A (en) * 2014-12-19 2015-04-29 华为技术有限公司 Task scheduling method, node and system
WO2017113278A1 (en) * 2015-12-31 2017-07-06 华为技术有限公司 Data processing method, apparatus and system
CN107153573A (en) * 2016-03-02 2017-09-12 阿里巴巴集团控股有限公司 Distributed task scheduling treating method and apparatus
WO2018219480A1 (en) * 2017-05-29 2018-12-06 Barcelona Supercomputing Center - Centro Nacional De Supercomputación Managing task dependency
CN108510150A (en) * 2018-02-01 2018-09-07 东华大学 A kind of spinning CPS and its real-time task processing method based on edge calculations

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
蒋维成;李兰英;刘华春;柳军;: "云计算中任务分配研究", 信息技术, no. 09 *

Also Published As

Publication number Publication date
CN111324428B (en) 2023-08-22

Similar Documents

Publication Publication Date Title
CN107800768B (en) Open platform control method and system
US11030009B2 (en) Systems and methods for automatically scaling compute resources based on demand
US8799913B2 (en) Computing system, method and computer-readable medium for managing a processing of tasks
US20150295970A1 (en) Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system
US11150999B2 (en) Method, device, and computer program product for scheduling backup jobs
CN111158892B (en) Task queue generating method, device and equipment
EP3958122A1 (en) Memory management method, apparatus, and system
CN116225669B (en) Task execution method and device, storage medium and electronic equipment
CN111857992B (en) Method and device for allocating linear resources in Radosgw module
CN107026879B (en) Data caching method and background application system
US20180322075A1 (en) Method for processing client requests in a cluster system, a method and an apparatus for processing i/o according to the client requests
US9152457B2 (en) Processing request management
CN109889406B (en) Method, apparatus, device and storage medium for managing network connection
CN112799606A (en) IO request scheduling method and device
CN103475520B (en) Service processing control method and device in distribution network
CN115617497B (en) Thread processing method, scheduling component, monitoring component, server and storage medium
CN108153583B (en) Task allocation method and device and real-time computing framework system
US9386087B2 (en) Workload placement in a computer system
CN116737345A (en) Distributed task processing system and method, device, storage medium and equipment
US9740618B2 (en) Memory nest efficiency with cache demand generation
CN111339132B (en) Data query method and database proxy
US10630602B1 (en) Resource allocation using restore credits
JP2020024636A (en) Scheduling device, scheduling system, scheduling method and program
CN111324428A (en) Task allocation method, device, equipment and computer readable storage medium
CN112395063B (en) Dynamic multithreading scheduling method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant