CN111782366B - A distributed task scheduling method and device - Google Patents
A distributed task scheduling method and device Download PDFInfo
- Publication number
- CN111782366B CN111782366B CN202010615362.7A CN202010615362A CN111782366B CN 111782366 B CN111782366 B CN 111782366B CN 202010615362 A CN202010615362 A CN 202010615362A CN 111782366 B CN111782366 B CN 111782366B
- Authority
- CN
- China
- Prior art keywords
- task
- batch
- client
- task scheduling
- scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000000977 initiatory effect Effects 0.000 claims 1
- 230000001934 delay Effects 0.000 abstract 1
- 238000004891 communication Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 14
- 238000012545 processing Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 6
- 239000000872 buffer Substances 0.000 description 5
- 230000002035 prolonged effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
一种分布式任务调度方法及装置，所述方法包括：接收客户端发送的开始任务调度的报文请求；根据所述报文请求中的任务批次号或根据所述任务批次号及当前批次任务执行状态执行任务调度操作。本申请能够让分布式任务调度系统在对两个相邻作业任务进行调度处理时，调度更加紧凑，缩短等待时延。
A distributed task scheduling method and device, the method comprising: receiving a message request for starting task scheduling sent by a client; performing a task scheduling operation according to a task batch number in the message request or according to the task batch number and the current batch task execution status. The application enables a distributed task scheduling system to schedule two adjacent job tasks more compactly and shorten waiting delays.
Description
Technical Field
The application relates to a computer application technology, in particular to a distributed task scheduling method and a distributed task scheduling device.
Background
The distributed task scheduling system is widely favored by virtue of the advantages of strong resource sharing, high calculation speed, high reliability, convenient and quick application and the like, can realize remote communication and data transmission among different nodes, can coordinate system resources comprehensively, performs distributed processing on the task submitted by the user, and provides great convenience for information exchange among people and completion of various calculation task.
However, in many cases, when two adjacent job tasks are scheduled, the scheduling is not compact and the waiting time is prolonged.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a distributed task scheduling method and a distributed task scheduling device, which can solve the problems of incompact scheduling and large waiting time delay when a distributed task scheduling system performs scheduling processing on two adjacent job tasks.
In order to solve the technical problems, the application provides the following technical scheme:
in a first aspect, the present application provides a distributed task scheduling method, including:
receiving a task scheduling message starting request sent by a client;
and executing task scheduling operation according to the task batch number in the task scheduling starting message request or according to the task batch number and the current batch task execution state.
Further, the task scheduling operation is executed according to the task lot number in the start task scheduling message request or according to the task lot number and the current lot task execution state, including:
judging whether the task batch number is 1;
And if so, executing task scheduling operation and sending a task executing notification to the client.
Further, the task scheduling operation is executed according to the task lot number in the start task scheduling message request or according to the task lot number and the current lot task execution state, and the method further includes:
if the task batch number is not 1;
judging whether task scheduling operation of a client capable of executing the current batch task is performed according to the current batch task execution state;
And if so, executing the task scheduling operation and sending a task executing notification to the client.
Further, before sending the task execution notification to the client, the method further includes:
judging whether an executing node exists in the current task batch or not;
if not, creating an execution node for the current task batch and executing task scheduling operation.
Further, the distributed task scheduling method further includes:
receiving a task scheduling message finishing request sent by a client;
And modifying the task execution state of the corresponding node to be completed according to the task batch number in the task ending scheduling message request.
Further, the task scheduling operation of the client for determining whether the current batch task can be executed according to the current batch task execution state includes:
and judging whether the task scheduling operation of the client side of the current batch task can be executed or not according to the batch number in the task scheduling end message request of the client side corresponding to the last batch task.
In a second aspect, a distributed task scheduling device includes:
a start message receiving unit, configured to receive a request for starting task scheduling message sent by a client;
And the dispatching unit is used for executing task dispatching operation according to the task batch number in the starting task dispatching message request or according to the task batch number and the current batch task execution state.
Further, the scheduling unit includes:
the batch judgment module is used for judging whether the task batch number is 1;
and the execution module is used for executing task scheduling operation corresponding to the task batch with the batch number of 1 and sending a task executing notification to the client.
Further, the scheduling unit further includes:
The state judging module is used for judging whether task scheduling operation of the client side of the current batch task with the task batch number not being 1 can be executed according to the execution state of the current batch task;
The execution module is further configured to execute the task scheduling operation, and send a task execution notification to the client.
Further, the distributed task scheduling device further includes:
The dispatching unit is also used for creating the execution node for the current task batch without the execution node and executing task dispatching operation.
Further, the distributed task scheduling device further includes:
the ending message receiving unit is used for receiving a task ending scheduling message request sent by the client;
and the state modifying unit is used for modifying the task execution state of the corresponding node to be completed according to the task batch number in the task ending scheduling message request.
Further, the execution module includes:
The judging module is used for judging whether the task scheduling operation of the client side of the current batch task can be executed or not according to the batch number in the task scheduling end message request of the client side corresponding to the last batch task.
In a third aspect, the present application provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the distributed task scheduling method when executing the program.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the distributed task scheduling method.
According to the technical scheme, the distributed task scheduling method and the distributed task scheduling device provided by the application have the advantages that the client initiates the task execution request to the server, and according to the task batch number corresponding to the task execution request and the task execution state set by the server, the distributed task scheduling system can schedule two adjacent job tasks more compactly and shorten the waiting time delay.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a distributed task scheduling method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a task scheduling operation performed in a distributed task scheduling method according to an embodiment of the present application;
FIG. 3 is a second flow chart illustrating a task scheduling operation performed in the distributed task scheduling method according to the embodiment of the present application;
FIG. 4 is a second general flow chart of a distributed task scheduling method according to an embodiment of the present application;
FIG. 5 is a third general flow chart of a distributed task scheduling method according to an embodiment of the present application;
FIG. 6 is one of the overall block diagrams of a distributed task scheduler in an embodiment of the present application;
FIG. 7 is a block diagram of a dispatch unit in an embodiment of the present application;
FIG. 8 is a second block diagram of a dispatch unit in an embodiment of the present application;
FIG. 9 is a second general block diagram of a distributed task scheduler in accordance with an embodiment of the present application;
FIG. 10 is a third general block diagram of a distributed task scheduler according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of an electronic device in an embodiment of the application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The scheme of the application can be applied to the scene that the distributed task scheduling system comprises a plurality of clients and a zookeeper scheduling server. The client is used to perform tasks that reach the client under the remote system, which may be divided into multiple batches. Because the programs loaded in the clients are different, the functions of the clients are different, so that the task batch and the clients have a binding relationship, that is, the task of one batch is fixedly executed by a plurality of clients, the task of the other batch is fixedly executed by another plurality of clients, and the tasks of the previous batch and the task of the next batch have a sequence, that is, the task of the next batch must be executed after the task of the previous batch is executed. Based on the above scenario, the zookeeper dispatch server needs to complete the dispatch of a plurality of clients in a lot and the dispatch of each client between lots.
In order to solve the problems of non-compact scheduling and prolonged waiting time when a distributed task scheduling system performs scheduling processing on two adjacent job tasks, the application provides a distributed task scheduling method, which, referring to fig. 1, comprises the following steps:
s101, receiving a message request for starting task scheduling sent by a client;
It can be understood that, in the above application scenario, when the client receives a task from a task lot corresponding to the client issued by the remote system, the client needs to first send a request for starting task scheduling message to the server, so as to inform the server that the client needs to execute the task. The start task scheduling message request is named StartSchedule. When the client initiates a task execution starting request to the server, the server calls StartScheduleHandle a processor for processing.
S102, executing task scheduling operation according to the task batch number in the task scheduling starting message request or according to the task batch number and the current batch task execution state.
It can be understood that the server side first needs to determine whether the task in the request for starting task scheduling message is the first task, and the determination basis is the task batch number carried in the request for starting task scheduling message. If the task is the first batch, the server side can immediately dispatch the first batch to be executed on the client side, and the first batch of the task does not exist in the first batch and can be executed without waiting for the execution result of the first batch. If the task is not the first batch task, the execution state of the current batch task needs to be continuously judged. Because the execution sequence exists between the tasks of the current batch and the tasks of the next batch, the task of the next batch can be executed only after the task of the current batch is executed, and therefore, before the client corresponding to the next batch is scheduled to execute the task of the next batch, whether the execution state of the task of the current batch is finished or not must be judged. The task scheduling operation is an operation of scheduling the task.
As can be seen from the above description, in the distributed task scheduling method provided by the present application, a server receives a task starting scheduling message request sent by a client, and performs task scheduling operation according to a task batch number in the task starting scheduling message request or according to the task batch number and a current batch task execution state, so that a distributed task scheduling system can schedule two adjacent job tasks more compactly, and reduce waiting time delay.
In one embodiment, in order to enable the distributed task scheduling system to schedule two adjacent job tasks, the scheduling is more compact, the waiting time delay is shortened, and the server side is set by adding a ServerCnxnFactory subclass to the zookeeper server side, and establishing connection between the client side and the server side based on SofaBolt.
Firstly, the server starts a thread, which builds a monitor on the port ClientPort responsible for the connection of the client, and monitors whether the client initiates a connection request. Then, the adaptation layer for processing the connection event and disconnection event of the client is registered at the server, that is, when the client requests to establish a connection with the server, the server receives the CONNECT event, and when the client requests to disconnect from the server, the server receives the CLOSE event. For the CONNECT event, the server side needs to buffer the connection information of the client side so as to identify the client side, and for the CLOSE event, the server side CLOSEs the client side connection and cleans up the relevant non-persistent data written in the server side by the client side. Finally, a client task schedule start processor StartScheduleHandle and an end processor EndScheduleHandle are registered on the server side. After the client establishes connection with the server, the client can send two types of messages to the server, namely a task scheduling starting message and a task scheduling ending message, and the message types are distinguished by message heads sent by the client.
After the client establishes connection with the server, when the server receives a task starting scheduling message request sent by the client, the server prepares to call StartScheduleHandle a processor to process the task starting scheduling message request. Before that, however, the server first needs to determine whether the server has cached the connection of the client that sent the message.
If not, rejecting the client corresponding to the message to initiate a task execution request;
If the task batch number is cached, judging whether a request message corresponding to a task execution request initiated by the client carries a task batch number of the client participating in task scheduling, and if the task batch number which is not carried or carried is not a number, rejecting the client corresponding to the message to initiate the task execution request;
And if the task batch number carried in the request message corresponding to the task execution request initiated by the client is digital, allowing the client to execute the task of the corresponding batch. Before executing the task, whether the task to be executed initiated by the client is a first task is judged according to the task batch number, if yes, the task can be executed, otherwise, whether the task of the previous batch corresponding to the task to be executed initiated by the client is already executed is judged, if the task of the previous batch corresponding to the task to be executed is already executed, that is, the current batch task execution state dispatched by the server is completed, the server can dispatch the client corresponding to the task batch to execute the task, otherwise, the dispatching is refused.
In another embodiment, in order to solve the problems that when the distributed task scheduling system performs scheduling processing on two adjacent job tasks, scheduling is not compact and waiting time is prolonged, the reconstructed zookeeper server side and the client side can establish network connection based on SofaBolt. After the network connection is established, different clients agree with each other to write the catalog and format of the execution node into the zookeeper server. These directories and formats will play a role in the subsequent dispatch process and can be used to determine if a batch of tasks has started to execute and completed.
The form of these directories and formats is, for example:
The path of the execution node established by the first batch of clients executing tasks is/rwdd/task/p 0001;
the path of the execution node established by the second batch of clients executing tasks is/rwdd/task/p 0002.
After the above arrangement is completed, the execution of the first batch task and the second batch task will be described as an example. The execution principle of other batches is similar, and will not be described again. In an embodiment, first, N clients corresponding to the first batch of tasks request the zookeeper server to create an execution node for the N clients in a competitive and exclusive manner according to the agreed directory and format, and the server may execute the task for the only one client for which the creation of the execution node is successful. For other N-1 clients, the competition fails, that is, when another N-1 clients attempt to create the execution node to the server, it is found that the execution node of the corresponding batch already exists, that is, the task of the task batch is being executed, and repeated execution is not needed, that is, the execution node is not needed to be created, that is, the execution node is not created.
Then, when the M clients responsible for executing the second batch task want to execute the second batch task, first, the executing node created by the client corresponding to the current batch task (the first batch task) to the zookeeper server needs to be monitored at the zookeeper server, so as to know whether the task execution state of the current batch task is completed.
After the client corresponding to the first batch of tasks finishes executing the tasks, the M client corresponding to the second batch of tasks requests the zookeeper server to create an execution node for the client according to the appointed catalogue and format, and the server only dispatches the client which creates the execution node successfully, so that the client can execute the second batch of tasks, and the dispatching process is similar to the dispatching process of N clients when the first batch of tasks are executed, and is not repeated.
In order to execute task scheduling operations of first batch tasks in the distributed task scheduling system, in an embodiment of the distributed task scheduling method provided by the present application, referring to fig. 2, the task scheduling operations are executed according to the task lot number in the start task scheduling message request or according to the task lot number and the current batch task execution state, including:
S201, judging whether the task batch number is 1;
It can be understood that, because the task batch number is carried in the task scheduling request, the server side can determine whether the task batch number is 1, that is, whether the batch task is the first batch task.
And S202, if yes, executing task scheduling operation and sending a task executing notification to the client.
It can be understood that if the task lot number is 1, it indicates that the task lot is a first task to be executed, i.e. a non-preface lot, and at this time, the server may schedule one of the plurality of clients corresponding to the first task to execute the first task. During scheduling, the server side sends a notification of executing the task to one of the plurality of clients.
As can be seen from the above description, the distributed task scheduling method provided by the present application can determine whether the task batch number is 1, and know whether the batch task is the first batch task to be executed, so as to create an execution node for the client corresponding to the batch task, and finally complete the task.
In order to execute task scheduling operations of non-first-lot tasks in the distributed task scheduling system, referring to fig. 3, in an embodiment of the distributed task scheduling method provided by the present application, the executing task scheduling operations according to the task lot number in the start task scheduling message request or according to the task lot number and the current lot task execution state further includes:
S301, if the task batch number is not 1;
it can be understood that, because the task batch number is carried in the task scheduling request, the server side can determine whether the task batch number is 1, that is, whether the batch task is a non-first batch task.
S302, judging whether task scheduling operation of a client side of the current batch task can be executed or not according to the execution state of the current batch task;
it will be appreciated that if the lot number of the task being executed is not 1, it is indicated that the lot exists in the leading lot, and thus execution may begin after the leading lot execution has been waited for. In order to determine whether the task of the previous lot of the lot to be executed, that is, the current lot, is executed, it is necessary to check whether the task execution state of the execution node corresponding to the current task lot is set to be the completed state.
And S303, if yes, executing the task scheduling operation and sending a task executing notification to the client.
It can be understood that if the execution status of the previous lot corresponding to the task lot to be executed is completed, the server side may schedule one of several servers corresponding to the lot to execute the task of the lot.
As can be seen from the above description, the distributed task scheduling method provided by the present application can determine whether the task batch number is not 1, and obtain whether the batch task is not the first batch task to be executed, and then, in combination with the task execution status of the current batch at the server, create an execution node for the client corresponding to the task, thereby finally completing the task.
In order to execute tasks of a client in a distributed task scheduling system, referring to fig. 4, the distributed task scheduling method provided by the present application further includes, before sending a task execution notification to the client:
s401, judging whether an execution node exists in the current task batch or not;
It can be understood that taking the second batch of tasks as an example, it is assumed that there are L clients corresponding to the second batch of tasks, and all the L clients want to execute the second batch of tasks, so that all the competing request server ends create execution nodes for the second batch of tasks, but according to the sequence, only the execution node of one client can be created successfully. Therefore, before the L clients prepare to create nodes to the server, it is required to determine whether the execution node exists in the second batch of tasks, that is, whether the execution node/rwdd/task/p 0002 corresponding to the second batch of tasks exists. If so, it indicates that the second batch of tasks is already being performed by the client without repeated execution. The execution principle of other batches is similar, and will not be described again.
S402, if not, creating an execution node for the current task batch and executing task scheduling operation.
It will be appreciated that, taking the second batch task as an example, if the/rwdd/task/p 0002 node does not exist yet, an execution node may be created for the client corresponding to the current task batch (second task batch) and a task scheduling operation may be performed, that is, one of the L clients may execute the task of the second batch. The execution principle of other batches is similar, and will not be described again.
As can be seen from the above description, the distributed task scheduling method provided by the present application can implement rapid notification scheduling of tasks of clients in a distributed task scheduling system by determining whether the current task lot has an execution node, thereby reducing latency.
Referring to fig. 5, in order to end the client to execute the completed task, the distributed task scheduling method further includes:
s501, receiving a task scheduling message finishing request sent by a client;
It can be understood that when the task execution of the task lot corresponding to the client is completed, a message request for ending task scheduling needs to be sent to the server, and the server receives the message, so as to know that the task of the task lot corresponding to the client has been completed, and can schedule the task of a new task lot. The end task scheduling message request is named EndSchedule. When the message type received by the server is EndSchedule, the server schedules EndScheduleHandle the processor to process.
S502, modifying the task execution state of the corresponding node to be completed according to the task batch number in the task ending scheduling message request.
It can be understood that the server may end execution of the task scheduling operation according to the task lot number carried in the end task scheduling message request. If the task execution of the client corresponding to the current batch of tasks is completed, the client corresponding to the current batch of tasks initiates a request for modifying the task execution state to completed, the server side makes modification according to the request, and all clients of the execution node can be monitored to quickly receive the notification.
The monitoring process refers to that in order to know whether the task of the previous batch is completed or not, a monitoring mechanism needs to be established at the server, that is, the server is actively informed of which batch (i.e., the batch number of the previous task batch) the corresponding task executing node needs to monitor by itself, when the task of the previous batch is completed, the task executing state corresponding to the task of the previous batch is modified to be completed, and the corresponding clients of the task batch to be executed can accordingly send a request for starting task scheduling message to the server.
As can be seen from the above description, the distributed task scheduling method provided by the present application uses the task end scheduling message request and combines the task lot number in the task end scheduling message request and the current lot task execution state to end the task execution scheduling operation, so that the server can start scheduling new lot tasks.
In order to determine whether the task scheduling operation of the client of the current batch task can be executed, the determining whether the task scheduling operation of the client of the current batch task can be executed according to the execution state of the current batch task includes:
and judging whether the task scheduling operation of the client side of the current batch task can be executed or not according to the batch number in the task scheduling end message request of the client side corresponding to the last batch task.
It can be understood that, still taking the second batch task as an example, since only one client of the L clients corresponding to the second batch task can be successfully created, when another L-1 clients prepare to create nodes at the server, it can be found that the executing node corresponding to the second batch task already exists, that is, the second batch task is already executed, and at this time, the server can return a message to inform that the task corresponding to the task batch of the L-1 clients does not need to be processed. The execution principle of other batches is similar, and will not be described again.
In addition, taking the second batch task as an example, before dispatching the second batch task, it is further required to check whether the first batch task is already executed, if the path/rwdd/task/p 0001 corresponding to the execution node corresponding to the task batch with the task batch number of 1 does not exist, it is considered that the first batch task has not yet started to be executed, and at this time, a return message is sent to inform the client corresponding to the second batch task of waiting. The execution principle of other batches is similar, and will not be described again.
If the path/rwdd/task/p 0001 corresponding to the execution node corresponding to the first batch of tasks exists, but the set task execution state is incomplete, monitoring the/rwdd/task/p 0001 node is established for the client corresponding to the second batch of tasks, and when the task execution state marked in the/rwdd/task/p 0001 node is modified to be completed by the server, the L clients corresponding to the second batch of tasks can obtain the notification from the server to inform the server that the first batch of tasks have been completed, and a task scheduling starting message can be sent to request the server to create the execution node of the second batch of tasks for the client.
As can be seen from the above description, the distributed task scheduling method provided by the present application can modify the task execution state of the corresponding execution node to be completed according to the batch number in the task scheduling end message request of the client corresponding to the previous batch task, and determine whether the task scheduling operation of the client of the current batch task can be executed.
Based on the same inventive concept, the embodiment of the present application also provides a distributed task scheduling device, which can be used to implement the method described in the above embodiment, as described in the following embodiment. Because the principle of the distributed task scheduling device for solving the problem is similar to that of the distributed task scheduling method, the implementation of the distributed task scheduling device can be referred to the implementation of the determination method based on the software performance reference, and the repetition is not repeated. As used below, the term "unit" or "module" may be a combination of software and/or hardware that implements the intended function. While the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
In order to solve the problems of non-compact scheduling and prolonged waiting time when a distributed task scheduling system schedules two adjacent job tasks from a hardware level, the application provides an embodiment of an electronic device with all or part of contents in a distributed task scheduling method, wherein the electronic device specifically comprises the following contents:
Referring to fig. 6, the present application provides a distributed task scheduling device, which includes a start message receiving unit 601 and a scheduling unit 602:
a start message receiving unit 601, configured to receive a start task scheduling message request sent by a client;
the dispatching unit 602 is configured to execute task dispatching operations according to the task lot numbers in the start task dispatching message request or according to the task lot numbers and the current lot task execution status.
Referring to fig. 7, the dispatching unit includes a batch judgment module 701 and an execution module 702:
A batch judgment module 701, configured to judge whether the task batch number is 1;
and the execution module 702 is configured to execute a task scheduling operation corresponding to the task lot with the lot number of 1, and send a task execution notification to the client.
Referring to fig. 8, the scheduling unit further includes a status judging module 801 and an executing module 702:
a state judgment module 801, configured to judge whether a task scheduling operation of a client of a current batch task with a task batch number different from 1 can be executed according to a current batch task execution state;
the execution module 702 is further configured to execute the task scheduling operation, and send a task execution notification to the client.
Referring to fig. 9, the distributed task scheduling device further includes:
a node determining unit 901, configured to determine whether an execution node exists in a current task batch;
the scheduling unit 602 is further configured to create an execution node for the current task lot without the execution node and perform task scheduling operations.
Referring to fig. 10, the distributed task scheduling device further includes an end message receiving unit 1001 and a state modifying unit 1002:
An end message receiving unit 1001, configured to receive an end task scheduling message request sent by a client;
The state modifying unit 1002 is configured to modify the task execution state of the corresponding node to be completed according to the task lot number in the task ending scheduling message request.
The system comprises a processor (processor), a memory (memory), a communication interface (Communications Interface) and a bus, wherein the processor, the memory and the communication interface are used for completing communication among the processors, the memory and the communication interface through the bus, the communication interface is used for realizing information transmission among a distributed task scheduling device, a core service system, a user terminal, related equipment such as a related database and the like, and the logic controller can be a desktop computer, a tablet personal computer, a mobile terminal and the like. In this embodiment, the logic controller may refer to an embodiment of the distributed task scheduling method and an embodiment of the distributed task scheduling device in the embodiment, and the contents thereof are incorporated herein, and are not repeated here.
It is understood that the user terminal may include a smart phone, a tablet electronic device, a network set top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Wherein, intelligent wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..
In practical applications, part of the distributed task scheduling method may be performed on the electronic device side as described above, or all operations may be performed in the client device. Specifically, the selection may be made according to the processing capability of the client device, and restrictions of the use scenario of the user. The application is not limited in this regard. If all operations are performed in the client device, the client device may further include a processor.
The client device may have a communication module (i.e. a communication unit) and may be connected to a remote server in a communication manner, so as to implement data transmission with the server. The server may include a server on the side of the task scheduling center, and in other implementations may include a server of an intermediate platform, such as a server of a third party server platform having a communication link with the task scheduling center server. The server may include a single computer device, a server cluster formed by a plurality of servers, or a server structure of a distributed device.
Fig. 11 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 11, the electronic device 9600 can include a central processor 9100 and a memory 9140, the memory 9140 being coupled to the central processor 9100. It is noted that this fig. 11 is exemplary, and that other types of structures may be used in addition to or in place of the structures to implement telecommunications functions or other functions.
In one embodiment, the distributed task scheduling method functionality may be integrated into the central processor 9100. The central processor 9100 may be configured to perform the following control:
s101, receiving a message request for starting task scheduling sent by a client;
S102, executing task scheduling operation according to the task batch number in the start message request or according to the task batch number and the current batch task execution state.
As can be seen from the above description, in the electronic device provided by the embodiment of the present application, a task execution request is initiated to a server through a client, and according to a task batch number corresponding to the task execution request and a task execution state set by the server, a distributed task scheduling system can schedule two adjacent job tasks more compactly, so as to shorten a waiting time delay.
In another embodiment, the distributed task scheduling device may be configured separately from the central processor 9100, for example, the distributed task scheduling device may be configured as a chip connected to the central processor 9100, and the distributed task scheduling method function is implemented by control of the central processor.
As shown in fig. 11, the electronic device 9600 may further include a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 does not necessarily include all the components shown in fig. 11, and furthermore, the electronic device 9600 may include components not shown in fig. 11, to which reference is made in the related art.
As shown in fig. 11, the central processor 9100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, which central processor 9100 receives inputs and controls the operation of the various components of the electronic device 9600.
The memory 9140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information about failure may be stored, and a program for executing the information may be stored. And the central processor 9100 can execute the program stored in the memory 9140 to realize information storage or processing, and the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, but not limited to, an LCD display.
The memory 9140 may be a solid state memory such as Read Only Memory (ROM), random Access Memory (RAM), SIM card, etc. But also a memory which holds information even when powered down, can be selectively erased and provided with further data, an example of which is sometimes referred to as EPROM or the like. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 storing application programs and function programs or a flow for executing operations of the electronic device 9600 by the central processor 9100.
The memory 9140 may also include a data store 9143, the data store 9143 for storing data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, address book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. A communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, as in the case of conventional mobile communication terminals.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, etc., may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and to receive audio input from the microphone 9132 to implement usual telecommunications functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100 so that sound can be recorded locally through the microphone 9132 and sound stored locally can be played through the speaker 9131.
The embodiment of the present application further provides a computer readable storage medium capable of implementing all the steps in the distributed task scheduling method in which the execution subject is a server or a client in the above embodiment, the computer readable storage medium storing a computer program, which when executed by a processor implements all the steps in the distributed task scheduling method in which the execution subject is a server or a client in the above embodiment, for example, the processor implements the following steps when executing the computer program:
s101, receiving a message request for starting task scheduling sent by a client;
S102, executing task scheduling operation according to the task batch number in the start message request or according to the task batch number and the current batch task execution state.
As can be seen from the above description, in the electronic device provided by the embodiment of the present application, a task execution request is initiated to a server through a client, and according to a task batch number corresponding to the task execution request and a task execution state set by the server, a distributed task scheduling system can schedule two adjacent job tasks more compactly, so as to shorten a waiting time delay.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the principles and embodiments of the present invention have been described in detail in the foregoing application of the principles and embodiments of the present invention, the above examples are provided for the purpose of aiding in the understanding of the principles and concepts of the present invention and may be varied in many ways by those of ordinary skill in the art in light of the teachings of the present invention, and the above descriptions should not be construed as limiting the invention.
Claims (12)
1. A distributed task scheduling method, comprising:
receiving a task scheduling message starting request sent by a client;
The distributed task scheduling method further comprises the steps of receiving a task scheduling ending message request sent by a client, modifying the task execution state of a corresponding node to be completed according to the task batch number in the task scheduling ending message request, and specifically, if the task execution of the client corresponding to the current batch task is completed, initiating a request for modifying the task execution state to be completed by the client corresponding to the current batch task, modifying the task execution state by the server according to the request, and monitoring all clients of an execution node to receive notification;
The method comprises the steps that a zookeeper server establishes SofaBolt-based network connection with a client, different clients agree with each other to write a catalog and a format of an execution node into the zookeeper server after the network connection is established, and the zookeeper server is used for completing dispatching of a plurality of clients in a batch and dispatching of all clients among the batches.
2. The method according to claim 1, wherein the executing task scheduling operation according to the task lot number and the current lot task execution state in the start task scheduling message request includes:
judging whether the task batch number is 1;
And if so, executing task scheduling operation and sending a task executing notification to the client.
3. The method according to claim 2, wherein the task scheduling operation is performed according to the task lot number and the current lot task execution state in the start task scheduling message request, further comprising:
if the task batch number is not 1;
judging whether task scheduling operation of a client capable of executing the current batch task is performed according to the current batch task execution state;
And if so, executing the task scheduling operation and sending a task executing notification to the client.
4. A distributed task scheduling method according to claim 2 or 3, further comprising, before sending a notification of execution of a task to the client:
judging whether an executing node exists in the current task batch or not;
if not, creating an execution node for the current task batch and executing task scheduling operation.
5. The distributed task scheduling method according to claim 1, wherein the task scheduling operation of the client that determines whether the current batch task can be performed according to the current batch task execution state includes:
and judging whether the task scheduling operation of the client side of the current batch task can be executed or not according to the batch number in the task scheduling end message request of the client side corresponding to the last batch task.
6. A distributed task scheduling device, comprising:
a start message receiving unit, configured to receive a request for starting task scheduling message sent by a client;
The dispatching unit is used for executing task dispatching operation according to the task batch number in the task dispatching starting message request and the current batch task execution state; the distributed task scheduling device further comprises an end message receiving unit, a state modifying unit and a state monitoring unit, wherein the end message receiving unit is used for receiving an end task scheduling message request sent by a client, and the state modifying unit is used for modifying the task execution state of a corresponding node to be completed according to a task batch number in the end task scheduling message request;
The method comprises the steps that a zookeeper server establishes SofaBolt-based network connection with a client, different clients agree with each other to write a catalog and a format of an execution node into the zookeeper server after the network connection is established, and the zookeeper server is used for completing dispatching of a plurality of clients in a batch and dispatching of all clients among the batches.
7. A distributed task scheduling apparatus according to claim 6, wherein the scheduling unit includes:
the batch judgment module is used for judging whether the task batch number is 1;
and the execution module is used for executing task scheduling operation corresponding to the task batch with the batch number of 1 and sending a task executing notification to the client.
8. A distributed task scheduling apparatus according to claim 7, wherein the scheduling unit further comprises:
The state judging module is used for judging whether task scheduling operation of the client side of the current batch task with the task batch number not being 1 can be executed according to the execution state of the current batch task;
The execution module is further configured to execute the task scheduling operation, and send a task execution notification to the client.
9. A distributed task scheduling apparatus according to claim 7 or 8, further comprising:
The node judging unit is used for judging whether an executing node exists in the current task batch or not;
The scheduling unit is also used for creating an execution node for the current task batch without the execution node and executing task scheduling operation.
10. A distributed task scheduling apparatus according to claim 7, wherein the execution module includes:
The judging module is used for judging whether the task scheduling operation of the client side of the current batch task can be executed or not according to the batch number in the task scheduling end message request of the client side corresponding to the last batch task.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the distributed task scheduling method of any one of claims 1 to 5 when the program is executed by the processor.
12. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the distributed task scheduling method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010615362.7A CN111782366B (en) | 2020-06-30 | 2020-06-30 | A distributed task scheduling method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010615362.7A CN111782366B (en) | 2020-06-30 | 2020-06-30 | A distributed task scheduling method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111782366A CN111782366A (en) | 2020-10-16 |
| CN111782366B true CN111782366B (en) | 2024-12-06 |
Family
ID=72760916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010615362.7A Active CN111782366B (en) | 2020-06-30 | 2020-06-30 | A distributed task scheduling method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111782366B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112965814B (en) * | 2021-02-09 | 2024-07-05 | 中国工商银行股份有限公司 | Multi-task scheduling method and system |
| CN113949601B (en) * | 2021-11-12 | 2023-04-28 | 杭州和利时自动化有限公司 | Inter-controller station communication method, device and computer readable storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109656706A (en) * | 2018-12-25 | 2019-04-19 | 江苏满运软件科技有限公司 | Distributed task dispatching method, system, equipment and medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9552229B2 (en) * | 2015-05-14 | 2017-01-24 | Atlassian Pty Ltd | Systems and methods for task scheduling |
| CN105677462A (en) * | 2015-12-30 | 2016-06-15 | 生迪光电科技股份有限公司 | Distributed task system based on internet of things and business processing method |
| CN107168789B (en) * | 2016-03-08 | 2021-05-11 | 创新先进技术有限公司 | Multitask serial scheduling method and device |
-
2020
- 2020-06-30 CN CN202010615362.7A patent/CN111782366B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109656706A (en) * | 2018-12-25 | 2019-04-19 | 江苏满运软件科技有限公司 | Distributed task dispatching method, system, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111782366A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111031058A (en) | Websocket-based distributed server cluster interaction method and device | |
| CN111147357B (en) | Use of digital assistant in communication | |
| CN109586929B (en) | Conference content transmission method and device, electronic equipment and storage medium | |
| US20130311191A1 (en) | Method, device, and system for voice approval | |
| US11758087B2 (en) | Multimedia conference data processing method and apparatus, and electronic device | |
| CN111782366B (en) | A distributed task scheduling method and device | |
| CN110764881A (en) | Distributed system background retry method and device | |
| CN111445331A (en) | Transaction matching method and device | |
| CN112689012A (en) | Cross-network proxy communication method and device | |
| CN113138812B (en) | Spacecraft task scheduling method and device | |
| US10453160B2 (en) | Embeddable communications software module | |
| CN111767558B (en) | Data access monitoring method, device and system | |
| CN111367561B (en) | Remote development method and device for software program | |
| CN112947880A (en) | Screen projection method and device and electronic equipment | |
| CN112291216A (en) | Communication method and device and electronic equipment | |
| CN114710496B (en) | Multi-node load balancing method and device | |
| CN111818293B (en) | Communication method and device and electronic equipment | |
| CN115103053A (en) | Call establishment method, device, terminal, system and readable storage medium | |
| CN113626086A (en) | Multi-core processor starting method and device based on shared starting space | |
| CN111355853A (en) | Call center data processing method and device | |
| CN113342501B (en) | System fault processing method and device | |
| KR100690852B1 (en) | Resource usage control method and mobile terminal | |
| CN114911583B (en) | Information processing method, device and distributed system | |
| JP7661371B2 (en) | Communication method, device and electronic device | |
| CN113163154B (en) | Information processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |