CN102681881B - Across machine dispatching method and system thereof - Google Patents

Abstract

The present invention provides a cross-machine scheduling system, which is characterized in that it includes a master node and a slave node, wherein the master node is configured to send to the slave node according to the slave node information, predefined process control information and slave node status Call request; the slave node is configured to execute the call request and return the call result to the master node.

Description

Cross-machine scheduling method and system

technical field

The invention relates to program execution of a computer system, in particular to a cross-machine scheduling method and system thereof.

Background technique

UNIX is a multi-user, multi-tasking operating system. At the same time, multiple processes can be executed at the same time. A process is an execution of a system program or application program in memory, and is an execution program currently running by the operating system. There is no one-to-one correspondence between processes and programs, and one program can be executed as multiple processes. In UNIX systems, process scheduling is implemented through system calls. The process scheduling of the operating system is mainly to realize the scheduling of each process in time slices according to the priority, so as to share resources (mainly CPU). The flow scheduling of a process means that multiple processes are executed according to a predefined flow.

The UNIX operating system does not directly provide a control function for calling a process according to a predefined flow. The way to directly realize the predefined process scheduling under the operating system is generally to write command scripts (shells), but this command script method has the following disadvantages: since the command scripts are written for process process scheduling, each shell corresponds to a A scheduling process, so once the process changes, the calling script needs to be modified. Moreover, for execution steps with a certain logical relationship, this logical relationship between the steps cannot be controlled.

The patent application with the application number 200610028504.X discloses a process scheduling method for application processes, which solves the problem that the single process scheduling method implemented by using command scripts in the Unix (Linux) environment cannot satisfy various application processes according to the predefined process Scheduling issues. This method sets up the process scheduling service program, including: reading predefined service process configuration information; creating an output message queue and an input message queue corresponding to the service process configuration information; reading single-step predefined process control information one by one, and executing Write the input message queue of the corresponding service when calling the request, and read the output message queue when the call returns.

However, as the characteristics of networked and heterogeneous deployment of computer system operations become more and more obvious, the above methods or OLTP-oriented middleware systems are difficult to adapt to complex distributed applications. For example, the scheduling method is more complicated in the case of multi-machine deployment. Some applications are machine-independent; some applications are host- and location-sensitive and require mandatory execution on all accessible machines or need to run on specified machines. Furthermore, the batch processing system requires high performance, that is, the full use of hardware resources throughout the process until the end of all process tasks, and a large amount of calculation, long process running time, strong high availability, and even a certain disaster recovery function. When a process fails or even some physical machines fail, try not to affect the execution of the process.

In order to achieve the above purpose, a batch application-oriented cross-machine scheduling system suitable for Unix/Linux systems is needed.

Contents of the invention

One or more objectives of the present invention are achieved through the following technical solutions,

A method for cross-machine scheduling, characterized in that it comprises the following steps:

(a) receiving a predefined process execution request at the host node,

(b) Read predefined process configuration information at the host node,

(c) read the slave node information at the master node, and load the slave node list,

(d) Read the predefined flow control information at the master node, and send a call request to the slave node according to the predefined flow control information,

(e) Execute the call request at the slave node,

(f) return the calling result at the slave node to the master node.

Preferably, the slave node information includes information about the processing capability of the slave node, and the predefined flow control information includes a calling method of the predefined flow.

Preferably, before sending the invocation request to the slave node, a query operation is further included, wherein the query operation includes: a step of sending query parameters from the master node to the slave node, and returning the slave node to the master node by the slave node Steps for node status.

Preferably, when the predefined flow control information indicates that the calling method of the predefined flow is single-machine synchronous calling, query the first slave node in the slave node list, and, when the first slave node has an idle process, send the A slave node sends a call request, and when the first slave node has no idle process, the process fails.

Preferably, when the predefined process control information indicates that the calling method of the predefined process is multi-machine synchronous calling, query the first slave node in the slave node list, and, when the first slave node has an idle process, send The first slave node sends a call request. When the first slave node has no idle process, query the second slave node in the slave node list. When the second slave node has an idle process, send the call request to the second slave node. Send a call request.

Preferably, when the predefined process control information indicates that the calling method of the predefined process is a multi-machine forced synchronous call, query the specified slave node in the list of slave nodes, and when the specified slave nodes all have idle processes, send The specified slave node sends the call request, otherwise, the process fails. After the call is successful, the call result is synchronously returned to the master node at the specified slave node.

Preferably, when the predefined process control information indicates that the calling method of the predefined process is single machine asynchronous calling, query the first slave node in the slave node list, and send the call request according to the number of idle processes of the first slave node. After the call is successful, the call result is asynchronously returned to the master node at the first slave node.

Preferably, after the master node receives a call result from the first slave node, it sends a new call request to the first slave node.

Preferably, when the predefined flow control information indicates that the calling method of the predefined flow is multi-machine asynchronous calling, query the specified slave node in the slave node list, and send the call request according to the number of idle processes of the specified slave node. After the call is successful, the call result is asynchronously returned to the master node at the specified slave node.

Preferably, the master node sends a new call request to the designated slave node after receiving the call result from the designated slave node.

A cross-machine scheduling system, characterized in that it includes a master node and a slave node, wherein the master node is configured to send a call request to the slave node according to slave node information, predefined flow control information and slave node status; The slave node is configured to execute the call request and return the call result to the master node.

Preferably, the respective slave node information is also stored in the slave node, and the respective slave node information is consistent with the slave node information stored in the master node.

Preferably, the master node acquires the state of the slave node by initiating a query connection session to the slave node, wherein the call request is directly sent to the slave node through the query connection session.

Preferably, the master node and one or more slave nodes are co-deployed on the same physical machine.

In the present invention, by setting up a master-slave logical server in computers distributed on the network and coordinating with the message queue mechanism, multiple processes in a distributed environment are deployed across computers according to a predefined process, scheduled across hosts, and support synchronous calls ( Prohibit multi-machine/allow multi-machine/force multi-machine), asynchronous call (prohibit multi-machine/allow multi-machine), so as to meet various calling processes. In the scheduling of multiple processes, the present invention also adopts a message queue control mechanism, and for execution steps with logical relationships, the logical control between steps can be realized through combined calls of single process/multiple processes and synchronous/asynchronous.

In the present invention, the host node can obtain process information in real time during scheduling, so as to maximize resource utilization as much as possible. When a process or a slave node fails, the transfer or activation of the slave node can be realized by simply modifying the configuration file. By separating the scheduling function of the master from the management function of the slave, the coupling degree between the system components is low, and the location of the process scheduling is transparent to the slave node, which can be freely specified, which greatly increases the flexibility of the system.

Description of drawings

Those skilled in the art will understand various aspects of the present invention more clearly after reading the detailed description of the present invention with reference to the accompanying drawings. It should be understood by those skilled in the art that these drawings are only used to describe the technical solution of the present invention in conjunction with the specific implementation, and are not intended to limit the protection scope of the present invention. in,

Fig. 1 is a schematic diagram of a cross-machine scheduling system according to an embodiment of the present invention.

Fig. 2 is a flowchart of a cross-machine scheduling method according to an embodiment of the present invention.

Detailed ways

In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

Fig. 1 is a schematic diagram of a cross-machine scheduling system according to an embodiment of the present invention. As shown in the figure, the logical modules that implement the process scheduling service are logically deployed in two levels. The first level is the master node (the first logical server), as the executor of process scheduling; the second level is the slave node (the second logical server), which is used to manage the process of each application program and distribute call request. The master node and the slave node can be deployed on different physical machines as logical service modules, and the master node can also be deployed together with one or more slave nodes on one physical machine. The master node and the slave node can communicate through the socket, and the master node can also provide predefined process execution services to the external system through the socket.

Fig. 1 also shows predefined process configuration information, slave node information and predefined process control information with a dotted line box. Wherein, the predefined process configuration information indicates the resource configuration required to execute the predefined process, for example, the name of the application program or the name of the specific instruction sequence to be executed, and the configuration of the processes of the application program or the specific instruction sequence. The slave node information includes information on the processing capabilities of the slave nodes. Here, the processing capability information may indicate which application programs or specific instruction sequences the corresponding slave node is suitable for executing, as well as the maximum number of configured processes of the slave node. It can be understood that the above processing capability information may also include resource configurations of corresponding slave nodes, such as CPU resources, memory resources, and the like. In addition, the slave node information may also include information about which slave nodes can execute various application programs or specific instruction sequences, for example, the application programs or specific instruction sequences may be associated with the corresponding slave node identifiers . The predefined process control information consists of one or more ordered process control commands, which may include the calling method of the predefined process, calling parameters, application program identifier or specific instruction sequence identifier. Here, the calling methods of the predefined process may include single-machine synchronous calling, multi-machine synchronous calling, multi-machine forced synchronous calling, single-machine asynchronous calling and multi-machine asynchronous calling. Here, a synchronous call means that only one call can be made at the same time, and the next call can only be made after the call returns; an asynchronous call means that after a call is made, the call can be made again without waiting for the return, which is equivalent to the same time. Make multiple calls. The implementation of these calling methods will be described in detail below. The predefined flow configuration information in the master node, the slave node information and the predefined flow control information can be preset and stored in one or more configuration files.

In one embodiment of the present invention, it is also possible to store respective slave node information at each slave node, and to synchronize the storage files of the slave node information through the Internet or a local area network so that the respective slave node information is stored in the master node. The node information of each slave machine is consistent. In this way, by making the master node and the corresponding slave node maintain the same slave node information, the system can share consistent configuration information globally. When a slave node fails, the transfer of the node can be easily realized by changing the information of the slave node.

Fig. 2 is a flowchart of a cross-machine scheduling method according to an embodiment of the present invention. As shown in the figure, in step 201, a predefined process execution request is received at a host node. In step 202, the configuration information of the predefined process is read at the host node to obtain requirements about the predefined process to be executed. In step 203, the information of the slave nodes is read at the master node, the processing capability of the slave nodes is analyzed, and the relevant slave nodes are loaded into the slave node list. In step 204, the predefined process control information is read at the master node, and each time a process control command is executed, a call request is sent to the slave node according to the information of the process control command. At this time, in step 208, the call request is executed at the slave node, and the call result is returned in step 209. The host node receives the call return in step 210, thereby completing the call of a flow control command. Thereafter, the host node invokes the next flow control command according to the flow control command sequence. Here, in order to complete the above steps, as an example, a control module, a return queue, and a coordination module may be set on the host node. Among them, the control module can be used as the execution module of process scheduling; the return queue is used to store the completion message of the call request, and the control module reads the completion message (process call return) from the return queue to obtain the completion status of the application program; the coordination module is used for Receive the process call return from each slave node and put it into the return queue. As an example, a management module, a cooperative module, one or more input queues and one output queue can be set on the slave node. Among them, the management module is used to save and manage the state of each queue and process; the coordination module is used to transfer the return message of each process from the output queue to the host node. Multiple input queues can be process queues of different applications, or multiple input queues can be configured for an application for concurrent processing of processes to improve single-step processing performance.

As shown in FIG. 2 , before sending the calling request to the slave node, a step 205 of querying the state of the slave node and a step 206 of returning the state of the slave node may also be included. Wherein, the master node sends query parameters to the slave nodes, and the slave nodes return the status of the slave nodes to the master node according to the query parameters. Here, the query parameter may be an identifier such as an application program name or a specific instruction sequence that can indicate a task that the slave should complete. The status of the slave node may be information reflecting the status of the process, such as the number of idle processes of the queried slave node corresponding to the query parameters, or any information reflecting the current resource usage status of the slave node.

As mentioned above, the predefined process control information includes the calling method of the predefined process. When the predefined process control information indicates that the calling method of the predefined process is a single-machine synchronous call, query a slave node in the slave node list, and when the slave node has an idle process, send a call request to the slave node, Otherwise, the process fails.

When the predefined process control information indicates that the calling method of the predefined process is multi-machine synchronous calling, query a slave node in the slave node list, and send a call request to the slave node when the slave node has an idle process ; When the slave node has no idle process, query another slave node in the slave node list until the first node with an idle process is found to call. If none of the slave nodes in the slave node list have idle processes, the process fails.

When the predefined process control information indicates that the calling method of the predefined process is multi-machine forced synchronous call, query the specified slave node in the slave node list, and when the specified slave nodes have idle processes, send the specified slave node The machine node sends the call request, otherwise, the process fails. After the call is successful, the call result is synchronously returned to the master node at the specified slave node. The host node will receive and return after all the slave node calls are completed.

When the predefined process control information indicates that the calling method of the predefined process is single-machine asynchronous calling, query a slave node in the slave node list, and send a call request according to the number of idle processes of the slave node. After the call is successful, the call result is asynchronously returned to the master node at the slave node. Here, every time the host node receives a call return, it sends a new call request to the node that sent the return.

When the predefined flow control information indicates that the calling method of the predefined flow is multi-machine asynchronous calling, query the specified slave node in the list of slave nodes, and send a call request according to the number of idle processes of the specified slave node. After the call is successful, the call result is asynchronously returned to the master node at the specified slave node. Here, every time the master node receives a call return from a specified slave node, it sends a new call request to the node that sent the return.

In addition, the master node can send a call request to the slave node in the connection session of the query operation, which can further ensure the accuracy of the status information of the slave node.

Those of ordinary skill in the art can understand that all or part of the steps in the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include ROM, RAM, magnetic disk or optical disk, etc. . Those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and equivalent technologies, the present invention also intends to include these modifications and variations.

Hereinbefore, specific embodiments of the present invention have been described with reference to the accompanying drawings. However, those skilled in the art can understand that without departing from the spirit and scope of the present invention, various changes and substitutions can be made to the specific embodiments of the present invention. These changes and substitutions all fall within the scope defined by the claims of the present invention.

Claims (16)

1. A cross-machine scheduling method, characterized in that, comprising the following steps: (a) receiving a predefined process execution request at the host node, (b) Read predefined process configuration information at the host node, (c) read the slave node information at the master node, and load the slave node list, (d) Read the predefined flow control information at the master node, and send a call request to the slave node according to the predefined flow control information and the slave node information, (e) Execute the call request at the slave node, (f) return the call result to the host node at the slave node, Wherein, the slave node information includes information about the processing capability of the slave node, and the processing capability information indicates which application programs or specific instruction sequences the corresponding slave node is suitable for executing, and the predefined flow control information is composed of one or more It consists of ordered process control commands, which include the calling method of the predefined process, calling parameters, application program identification or specific instruction sequence identification. 2. The method of claim 1, wherein, In step (d), Before sending the call request to the slave node, it further includes a query operation, wherein, The query operations include: the step of sending query parameters by the master node to the slave node, and, The step of returning the slave node status from the slave node to the master node. 3. The method of claim 2, wherein, In step (d), Send a call request to a slave node in the connection session of the query operation. 4. The method of claim 2, wherein, In step (d), Send a call request to the slave node in the newly created connection session. 5. The method of claim 2, wherein, The slave node status is the number of idle processes. 6. The method of claim 2, wherein, In step (d), When the predefined flow control information indicates that the calling method of the predefined flow is single-machine synchronous calling, query the first slave node in the slave node list, and, When the first slave node has an idle process, send a call request to the first slave node, When the first slave node has no idle processes, the flow fails. 7. The method of claim 2, wherein, In step (d), When the predefined process control information indicates that the calling method of the predefined process is multi-machine synchronous calling, query the first slave node in the slave node list, and, When the first slave node has an idle process, send a call request to the first slave node, When the first slave node has no idle process, query the second slave node in the slave node list, When the second slave node has an idle process, a call request is sent to the second slave node. 8. The method of claim 2, wherein, In step (d), When the predefined process control information indicates that the calling method of the predefined process is multi-machine forced synchronous calling, query the specified slave node in the slave node list, When the specified slave node has an idle process, send a call request to the specified slave node, otherwise, the process fails; In step (f), Synchronously return the call result to the master node at the specified slave node. 9. The method of claim 2, wherein, In step (d), When the predefined flow control information indicates that the calling method of the predefined flow is a single-machine asynchronous call, query the first slave node in the list of slave nodes, and send a call request according to the number of idle processes of the first slave node; In step (f), The call result is asynchronously returned to the master node at the first slave node. 10. The method of claim 9, wherein, After the master node receives a call result from the first slave node, it sends a new call request to the first slave node that sent the return. 11. The method of claim 2, wherein, In step (d), When the predefined flow control information indicates that the calling method of the predefined flow is multi-machine asynchronous calling, query the specified slave node in the list of slave nodes, and send a call request according to the number of idle processes of the specified slave node; In step (f), Asynchronously returns the result of the call at the specified slave node to the master node. 12. The method of claim 11, wherein, After the host node receives the call result from the specified slave node, it sends a new call request to the specified slave node that sent the return. 13. A cross-machine scheduling system, characterized in that it includes a host node and a slave node, wherein, The master node is configured to send a call request to the slave node according to the slave node information, the predefined process control information and the slave node state; The slave node is configured to execute the call request and return the call result to the master node, Wherein, the slave node information includes information about the processing capability of the slave node, and the processing capability information indicates which application programs or specific instruction sequences the corresponding slave node is suitable for executing, and the predefined flow control information is composed of one or more It consists of ordered process control commands, which include the calling method of the predefined process, calling parameters, application program identification or specific instruction sequence identification. 14. The cross-machine scheduling system according to claim 13, characterized in that, The respective slave node information is also stored in the slave nodes, and the respective slave node information is consistent with the slave node information stored in the master node. 15. The cross-machine scheduling system according to claim 13, characterized in that, The master node obtains the state of the slave node by initiating a query connection session to the slave node, wherein the call request is directly sent to the slave node through the query connection session. 16. The cross-machine scheduling system according to claim 13, wherein the master node and one or more slave nodes are co-deployed on the same physical machine.