JPH0612392A - Computer resource distribution method and system - Google Patents

Abstract

(57) [Summary] [Purpose] To improve the efficiency of a computer system by distributing multiple processes with different processing amounts to multiple computers. [Structure] When the processing amount of a process to be distributed is known, the scheduler 13 in the client 31b allocates the processes having a large processing amount to the computers 33 to 36 having a small cumulative processing amount in order, and The processing amount is added to update the integrated processing amount, and each processing is sequentially assigned to the computer. Further, in order to allocate an available computer to the client, each computer adds the reliability of the state data to the state data such as the utilization rate and sends it to the server (domain resource manager DM) for distributed control of the computer, The server allocates the available computers 33 to 36 to the clients based on these data. The agent AGT of each available computer sends a resource use token to the client to allow exclusive use of computer resources.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a computer resource distribution method and system, and more particularly to a computer resource distribution method and system for distributing processing to a plurality of computers connected to a network.

[0002]

2. Description of the Related Art LAN (Local Area Network)
There is a computer system in which a large number of computers are connected to a network such as the above and the processing is distributed among the computers or files are distributed and managed. In such a computer system, when assigning a process to each computer, it has been taken into consideration that the amount of the process actually assigned or the amount of processing time (CPU time) actually required was hardly taken into consideration. However, these processing amounts or processing times are individually designated by the judgment of the operator or the like. Also, when distributing the processing to multiple computers,
CPU speed (processing speed) and CPU utilization rate (C) of each computer
It has not been done to allocate processing in consideration of (PU load). Furthermore, when multiple processes with different throughput per unit time are executed in parallel in a pipeline manner, when each process is distributed to computers and executed in parallel without considering the difference in throughput per unit time. Was the most. Therefore, a plurality of processes having different processing amounts or processing speeds can be efficiently executed in parallel on a plurality of computers, and further, the plurality of processes can be dynamically transferred to the plurality of computers in consideration of the CPU speed and the CPU utilization rate of the computers. There is a demand for a distributed processing method of computer resources for distributed and parallel processing.

By the way, in order to dynamically distribute a plurality of processes to a plurality of computers in consideration of the CPU state such as a CPU utilization rate, a computer (called a server) which controls the process distribution must be notified of the CPU state of each computer. I have to. As a method of notifying such state data, there are a method of shortening the communication interval and sending frequently, a method of lengthening the communication interval and sending, frequently sending state data when changing the state, and slowly sending state data when the state is stable. However, according to the first method, although accurate state data can be maintained, the number of communications increases, the overhead of the reception process and the occupation rate of the communication path increase, and the communication efficiency of other data decreases. There's a problem. Further, according to the second method, the number of communications is reduced, but there is a problem that a rapid state change cannot be detected. Further, according to the third method, the number of communications decreases when the state is stable, but when the state changes drastically, there is a problem that the number of communications increases as in the first method.

In order to dynamically execute a plurality of processes by distributing them to a plurality of computers in consideration of the CPU state,
The state of the CPU changes dynamically, including host down, the types of computers connected are diverse,
It is necessary to consider that there is a communication delay in the CPU status notification. Because when the CPU state changes,
This is because it is not possible to secure a certain amount of resources in advance, and it becomes difficult to collect state information that changes from moment to moment, and if a process is assigned, the state will change and the end of the process will be delayed. Also, if there are various types of computers,
This is because there is a large variation in the performance of the CPU, and it becomes difficult to collect the state information for each computer, so that correct process allocation cannot be performed and the end of the allocated process is delayed. Furthermore, due to the delay in status notification,
This is because multiple allocation of processing may occur, and in order to completely eliminate this, it is necessary to synchronize all CPUs, which complicates the configuration and causes cost increase.

[0005]

As described above, in the conventional computer system, the processing cannot be uniformly assigned to each computer, the throughput of the entire system is lowered, and the processing result cannot be obtained quickly. there were. In addition, in the conventional computer system, when trying to maintain the state of the computer accurately, the number of communications increases, the overhead for receiving processing increases, the computer cannot process efficiently, and the communication path is occupied. There is a problem that the rate increases and the communication efficiency of other data decreases. Furthermore, in the conventional computer system, it is difficult to collect accurate status information for each computer, and there is a problem that processing cannot be allocated correctly.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a computer resource distribution method and system which can distribute a plurality of processes having different processing amounts or processing speeds to a plurality of computers and efficiently execute them in parallel. Another object of the present invention is to provide a computer resource distribution method and system capable of performing parallel processing by distributing a plurality of processes to each computer in consideration of the CPU speed and CPU utilization rate of the computer. Still another object of the present invention is to store a processing amount and the like obtained by actually executing a process as history information and distribute the process to a plurality of computers by using the history information. And to provide a system. Another object of the present invention is to provide a computer resource distribution system that manages the state of each computer resource accurately with a small amount of communication and distributes the processing to a plurality of computers based on the state. Still another object of the present invention is to provide a computer resource distribution system capable of efficiently allocating a plurality of computer resources whose states change, to a plurality of other computers that have requested resources.

[0007]

1 and 2 are explanatory views of the principle of the present invention. In FIG. 1, 11 is an execution processing group,
Reference numeral 13 is a process allocation device (scheduler) for allocating a plurality of processes to a computer, and 30 is a computer (CPU) group. 11a, 11b, 11c ... Processes to be executed, 3
1-36 are computers (CPU). In FIG. 2, 3
1 to 36 are a plurality of computers (host A to host F) connected to the network, 31a to 36a are local resource managers (Local Resource Managers: LMs) provided in the respective computers and monitoring the operating state of the computers, 31b, 3
2b is a client (CLIENT: CLT) that requests a computer that can be used for distributed processing, and distributes and executes the process using the assigned computer, and 32c collects and holds the operating state of each computer, and A domain resource manager (Domain Resource Manager) that determines the computers that can be used in response to computer resource use requests from the client CLT.
anager: DM), 31d to 36d are provided in each computer, and send a message (token) permitting exclusive use of computer resources to the client CLT instructed by the domain resource manager DM to exclude the own computer resource from the client. It is a resource use permission mechanism (called an agent) that is used for specific purposes.

[0008]

A scheduler 13 (see FIG. 1) for processing distribution is provided in at least one computer, and the processing to be distributed (11
a to 11 g), the scheduler allocates the processes to the computers (31 to 36) with the smallest cumulative processing amount (initial value is 0) in order from the largest processing amount, and the processing amount of the allocated processing. Is added to the integrated processing amount to update the integrated processing amount of the computer, and thereafter, the processes are sequentially assigned to the computers by the same method. By doing so, the processing amount of each computer can be made uniform, the processing end time of each computer can be made approximately the same time, the efficiency of the computer system can be improved, and the processing result can be obtained quickly.

A computer that can be used for distributed processing is determined as follows (see FIG. 2 (a)). The local resource managers LM31a-36a of the computers 31-36 are CPUs.
The domain resource manager DM is added to the state data indicating the usage rate and the like, by adding the data indicating the reliability of the state data.
32c. DM 32c is the client 31b
When a request for allocating computers that can be used for distributed processing is received from (corresponding to the scheduler 13 in FIG. 1), one or more computers that can be used are determined in consideration of the utilization rate and reliability of each computer. For example, a computer having a low utilization rate and a high reliability is sequentially determined as an available computer.
The client 31b distributes the processing to the available computers by the method described in FIG. The local resource managers LM 31a to 36a of the respective computers increase the reliability (stability, effective time) when the degree of state change is small, decrease the reliability when the degree of state change is large, and
When the reliability is high, the transmission interval of the state data is lengthened, and when the reliability is low, the transmission interval of the state data is shortened.

With this arrangement, the domain resource manager DM can manage the state of each computer resource accurately with a small amount of communication, and can efficiently allocate the computer resource. In this case, the local resource managers LM 31a to 36a of the respective computers add the hardware information of the computers to the state data and send the domain data to the domain resource manager DM 32c, and the domain resource manager DM sends the operating state, reliability, hardware The client 31b can use one or more computers based on the wear information. In this way, computers can be assigned according to the capacity of the computers, uniform distributed processing can be performed, and system efficiency can be improved.

The following processing is performed so that the client 31b can actually request the available computers 33 to 36 to perform processing. Domain resource manager DM
When the computer 32c determines the computers 33 to 36 that can be used in response to the computer resource utilization request from the client 31b as described above, it instructs the agents 33d to 36d of each computer to accept the processing request from the client (see FIG. a)). By this instruction, each agent 33d
36d is a client 3 who wishes to use computer resources
A connection is established with 1b (FIG. 2 (b)), and then a resource use token is transmitted to exclusively allocate computer resources to the client. The resource use token has a resource state, an available resource amount, a resource stability, and a resource use time limit, and the client 31b distributes the processing to a plurality of computers by the method of FIG. 1 in consideration of these pieces of information.
The agent is a plurality of clients 31b, 32b.
If the client wants to use the computer resource, a connection is established with each client, and a resource use token is sent to only one predetermined client 31b to exclusively use the computer resource for the client 31b. When the client finishes using the computer resource, a resource use token is sent to the next client 32b to exclusively use the computer resource. By doing so, the plurality of computer resources whose states change can be efficiently allocated to the other plurality of computers that have requested the resources.

[0012]

【Example】

(a) Schematic overall configuration of the present invention FIG. 3 is an overall configuration diagram of the present invention. 11 is a processing group to be executed, 30 is a communication network such as a LAN (Local Area network), and 31 to 36 are connected to the network. A plurality of computers (host A to host F). In each computer, 31a to 36a are local resource managers (LMs) that monitor the operating status of the computers, 31b and 32b request computers that can be used for distributed processing, and the processing 11 is distributed to the allocated computers and executed. A client (CLIENT: CLT) 32c that collects and holds the states such as the utilization rate of each computer and also determines a computer that can be used in response to a computer resource utilization request from the client CLT (domain resource manager DM). ), 31d to 36d send a message (referred to as a resource use token) for permitting exclusive use of computer resources to the client CLT instructed by the DM 32c to allow the client to exclusively use its own computer resources. Agent).

General Process Distribution of Overall Operation The client 31b must obtain a use permission from a remote computer in order to execute the process 11 by distributing it to a remote computer. The control for obtaining the usage permission will be described later.
Now, it is assumed that the use is permitted by the plurality of computers 33 to 36. Scheduler 13 in client 31b
When the processing amounts of the processes 11a to 11n to be dispersed are known, the processes are assigned to the computers (33 to 36) with the smallest integrated processing amount (initial value is 0) in order from the largest processing amount. When the scheduler 13 allocates a predetermined process to the computer, the scheduler 13 updates the integrated processing amount of the computers 31 to 36 by adding the processing amount of the allocated process to the integrated processing amount. After that,
All processes 11a to 11n are sequentially assigned to the computers by the same method.

By doing so, the processing amount of each computer can be made uniform, the processing end time of each computer can be made substantially the same time, the efficiency of the computer system can be improved, and the processing result can be obtained quickly. . In this case, if the processing amount obtained by actually executing each process is accumulated as history information and the processing amount of each process is obtained by referring to the history information, the correct processing amount can be obtained. Therefore, the uniformity of the throughput of each computer can be further increased, and the efficiency of the computer system can be improved. Also, the amount of processing is C of the computer
If the value obtained by dividing by the PU speed is added to the integrated processing amount and the integrated processing amount is updated and the processing is distributed based on this integrated processing amount, uniform processing is performed in consideration of the difference in the processing speed of each computer. The distributed processing can be performed, and the efficiency of the system can be further improved. In this case, if the CPU speed is corrected based on the load on the computer, the uniformity of processing can be further increased.

Determination of Available Computers (FIGS. 4 to 6) The computers that can be used by the client for distributed processing are determined as follows. The local resource managers LM31a to 36a of the respective computers 31 to 36 monitor their own CPU status (for example, CPU utilization rate), add data indicating the reliability to the status data, and send it to the domain resource manager DM32c ( (Fig. 4). Then, the domain resource manager DM 32c becomes the client 31.
When a computer use request is received from the computer b via the local resource manager LM31a (see FIG. 5), it is available 1 considering the utilization rate and reliability of each computer 31-36.
Determine the above computer. For example, the computers 33 to 36 having low utilization rates and high reliability are sequentially determined as available computers. As a result, the client 31b distributes the processing to the available computers 33 to 36 by the method described in FIG.

As the reliability, for example, stability and effective time of state data can be considered. The local resource managers LM31a to 36a of the respective computers increase the reliability (stability, effective time) when the degree of state change is small, and decrease the reliability when the degree of state change is large. When the reliability is high, the transmission interval of the state data is lengthened, and when the reliability is low, the transmission interval of the state data is shortened. By doing so, the domain resource manager DM 32c can manage the state of each computer resource accurately with a small amount of communication, and can efficiently allocate the computer resource. In addition, each of the local resource managers LM31a to 36a uses the hardware information (CP) of the computer.
(U speed etc.) is added to the state data and transmitted to the domain resource manager DM32c, and the domain resource manager DM32a can use one or more computers for the client CLT based on the operating state, reliability and hardware information of each computer. To In this way, computers can be assigned according to the capacity of the computers, uniform distributed processing can be performed, and system efficiency can be improved.

Computer resource use permission client 31b is a domain resource manager DM
In order to be able to actually request the processing to the computers 33 to 36 determined by the computer 32c, it is necessary to obtain usage permission from each computer, and the following control is performed.
When the domain resource manager DM 32c determines the computers 33 to 36 that can be used in response to the computer resource use request from the client 31b as described above, the agents 33d to 36d of the respective computers perform processing from the client 31b via the local resource manager LM. Instruct to accept the request (Fig. 6). By this instruction,
Each of the agents 33d to 36d establishes a connection with the client 31b desiring to use the computer resource (see FIG. 7), and thereafter, the resource use token (Token).
To exclusively allocate computer resources to the client. The resource use token has a resource state, an available resource amount, a resource stability, and a resource use time limit, and the client 31b considers these pieces of information and performs processing on a plurality of computers by the method described in FIG. Spread. By doing so, the plurality of computer resources whose states change can be efficiently allocated to the other plurality of computers that have requested the resources.

When a plurality of clients 31b and 32b desire to use computer resources, the agents 33d to 36d set up a connection with each client, and at the same time, one predetermined client 31b.
The resource use token is sent only to the client 31
b to exclusively use the computer resource, and when the client finishes using the computer resource, the next client 32
A resource use token is sent to b to exclusively use computer resources.

(B) Distributed control of processing FIG. 8 is a block diagram of an embodiment of distributed processing according to the present invention. 11 is an execution processing group, 13 is a processing allocation device (scheduler) for allocating a plurality of processings to a computer, A computer group 30 is included in the client 31b of FIG.
In the execution processing group 11, 11a to 11g are processing executed by a computer, and in the computer group 30, 31 to 36 are computers (referred to as CPUs or hosts) connected to a network such as a LAN (not shown). In the scheduler 13, 13b is a process 11a to be actually executed.
A history information management unit that manages history information related to the processing amount of 11g, 13c is an execution processing table that stores data regarding the processing procedure of the processing 11a to 11g and the processing amount, 13e.
Is information about the computer to which the process is assigned, eg, CPU
The processing allocation computer table that stores the speed (processing speed) Vcpu and the CPU load (utilization rate) η, 13f is a processing allocation rule table that stores the processing procedure when the processing is actually allocated to each computer, and 13g is the information of each table. A processing allocation unit that allocates the processing to be executed by each of the computers 31 to 36 based on the above, 13h is a computer execution processing table that stores the correspondence between the computers and processing created by the processing allocation unit, and 13i is stored in the computer execution processing table 13h. According to the contents described above, it is a process activation / execution monitoring unit that allocates a process to each computer and monitors the end of process execution.

The process activation / execution monitoring unit 13i determines the time (process time) from the start to the end of the process, the process amount (the number of processing steps) notified from the computer, or the process amount per unit time (= process amount). / Processing time)
After the processing is completed, the history information management unit 13 uses these as history information.
It has a function of notifying b and storing it in association with the processing name. The execution processing table 13c stores data relating to processing procedures, processing amounts, etc. based on processing information and history information. The process allocation unit 13g uses the execution algorithm table 13c and the process allocation computer table 1 according to the allocation algorithm instructed by the process allocation rule table 13f.
A process is assigned to each computer based on the content of 3e (processing amount, CPU speed, load factor, etc.). The following methods can be considered as the allocation method. That is, when the processing amount of each process 11a, 11b, 11c, ... Is known, the respective processes are arranged in descending order of processing amount and stored in association with the computers 31, 32, 33. The process is assigned to the computer with the smallest accumulated processing amount (initial value is 0). Then, add the processing amount of the processing assigned to the integrated processing amount to update the integrated processing amount of the computer,
After that, the respective processes are sequentially assigned in the same manner to the computers, and the obtained computer-process correspondences are stored in the computer execution process table 13h. According to this method
Each process can be assigned so that the processing amount of each computer becomes uniform, and the end times of the processes can be made approximately the same or as close as possible. The number of processing steps or the processing time can be used as the processing amount.

When the processing amount per unit time (= processing amount / processing time) of each processing is known in advance by measurement or calculation, each processing 11a, 11b, 11c, ... The processing amount is arranged in descending order, and the processing is assigned to the computer having the smallest accumulated processing amount (initial value is 0) stored in association with the computers 31, 32, 33, .... Then, the integrated processing amount of the computer is updated by adding the processing amount of the allocated process per unit time to the integrated processing amount, and thereafter, each process is sequentially allocated in the same manner and allocated to the computer. According to this method, the CPU load (utilization rate) of each computer can be made uniform. The utilization rate η is given by the following equation η = (Tu + To, where Tu is the time the user (application program) is using the computer during a certain time T and To is the time the computer OS is using ) · 100 / T (%) (a) The utilization rate η is notified to the scheduler 13 as computer information from the OS (operation system) of each computer every predetermined time.

When processing a file, each processing 1
Computers having the smallest accumulated file amount (initial value is 0) stored in association with the computers 31, 32, 33, ... by arranging 1a, 11b, 11c, ... In order of file size (size). Assign processing to. Then, the accumulated file amount of the computer is updated by adding the file amount of the allocated process to the accumulated file amount, and thereafter, each process is sequentially allocated in the same manner to the computer. This method uses the size of the file as a measure of the processing amount, paying attention to the correspondence between the processing amount and the file size.

First, the allocation is performed by another method to execute the processing. Then, the processing time or processing amount (the number of processing steps) or the processing amount per unit time (= processing amount / processing time) of each processing 11a, 11b, 11c, ... -The execution monitoring unit 13i monitors the information, and after the processing is completed, notifies the history information management unit 13b of these as history information and stores the history information in association with the processing name. After that, when the process allocation becomes necessary again, the processing amount of each process is obtained using the history information to create the execution process table 13c, and the process allocation unit 13g uses the method or the method based on this process amount. Each process is assigned to the computers 31, 32, 33, .... According to this method, since the actual throughput can be known, it is possible to further improve the uniformity of the throughput of each computer or the uniformity of the utilization rate of each computer.

When the processing speed of each computer of the computer system is different, each processing 1 is performed in the same manner as the above-mentioned methods (1) to (3).
1a, 11b, 11c, ... Are arranged in order from the processing amount or the processing amount per unit time or the file amount is large, and thereafter, the integration is stored corresponding to the computers 31, 32, 33. The process is assigned to the computer with the smallest value (initial value is 0). Then, the processing amount or the processing amount or file amount per unit time is calculated by C of the computer.
A value divided by the PU speed (processing speed) Vcpu is added to the integrated value to update the integrated value, and thereafter, the same allocation is sequentially performed to allocate each process to the computer. According to this method, uniform distributed processing can be performed in consideration of the difference in processing speed between computers. The CPU speed is previously notified from each computer to the scheduler 13 as computer information.

In the above method, the CPU speed Vcpu of the computer is corrected based on the load of the computer, and the integrated value is updated using the effective CPU speed Veff obtained by the correction. According to this method, CP is calculated based on the load of the computer.
Since the U speed can be corrected, the uniformity of processing can be further improved as compared with the above method. The effective CPU speed can be calculated by the following formula: Veff = (100−η) · Vcpu / 100 (b). The scheduler 13 can calculate the effective CPU speed Veff from the equation (b).

Which of the above-mentioned methods (1) to (3) is used to allocate the processing is as follows, for example. That is,
(1) Use the method in the processing of files, (2)
If you know the throughput in advance, use the method of
(3) Use the method in the case of pipeline processing using multiple computers, (4) Use the method in the case of processing that has been executed once, and (5) Consist of computers with different speeds. In the system, use the method of, and (6) Use the method of when the computer is executing other processing in parallel. In this way, efficient processing distribution, that is, scheduling processing can be realized.

Embodiment When Applied to Compile Processing FIG. 9 is a block diagram of a computer system for distributed processing of compile processing. 10 is a network such as a LAN, 31-
Reference numeral 35 denotes a computer (hereinafter, referred to as host 1 to host 5) that is connected to a network and operates on UNIX (hereinafter referred to as host 1 to host 5). The CPU speed of hosts 1 to 3 is 100% (normalized to 1) CPU speed doubled to 200%
(Normalized to 2). Among the hosts, one host 4 is equipped with a scheduler 13 that performs process allocation (scheduling). It is assumed that the computers 31 to 35 have already been permitted to be used for distributed processing.

FIG. 10 is a block diagram of a compiling device when the compiling process is distributed and processed by the computer system of FIG. This compiling device generates some compiling commands from a file in which the compiling procedure of a program is written, and processes according to each compiling command (processing procedure) are distributed to hosts on a plurality of networks and executed in parallel. It is intended to improve the speed of the compilation process. In the figure, 31 to 35 are computers (host 1 to host 5), 13 is a scheduler, 21 is a procedure file in which a program compilation procedure is written, and 22 is the processing amount or processing time of the processing executed so far or per unit time. The processing procedure execution history file that stores the processing amount as the history information, 23 is computer information (distributed processing host information) including the CPU speed Vcpu and load factor η input to the scheduler from the computer (host), and 24 is a file to be processed. It is processing information having a name and a file size.
The scheduler 13 refers to the processing procedure file 21, the processing procedure execution history file 22, and the distributed processing host information 23, assigns the processing according to the compile command to the hosts 1 to 5, and activates the compile processing in parallel for each host.

FIG. 11 is a block diagram of the scheduler 13. The same parts as those in FIG. 8 are designated by the same reference numerals. In the figure,
13a is a procedure file analysis unit that analyzes the contents of the procedure file 21 to generate a compile command, and outputs the relationship between the process such as the compile command and the file and the file size by referring to the processing information 24, 13b.
Is a history information management unit that reads out information regarding the processing amount of the processing to be actually executed from the processing procedure execution history file 22 and manages it as an execution history table, and outputs the processing amount or processing time. An execution processing table that stores data regarding processing procedures and the amount of processing, 13d is a host information management unit that manages distributed processing host information (CPU speed, CPU utilization rate) 23 sent from each host 1 to 5, and 13e is Host 1 that distributes processing
About 5 CPU speed (processing speed) Vcpu and current C
A distributed host table (processing allocation computer table 13e in FIG. 8) that stores host information such as PU load (utilization rate) η
And is updated by the host information management unit 13d. Reference numeral 13f is a process allocation rule table, and reference numeral 13j is a resource request unit for securing computer resources that can be used for distributed processing, which will be described later.

Reference numeral 13g refers to the information in each of the tables 13c and 13e, and a processing allocation unit that allocates processing to each host 1 to 5 according to a predetermined allocation algorithm. The host / execution process mapping table to be stored (corresponding to the computer execution process table 13h in FIG. 8), 13i is in accordance with the contents stored in the mapping table 13h.
A process activation / execution monitoring unit that allocates a process to each computer and monitors the end of process execution. Start this process
The execution monitoring unit 13i displays the time (processing time) from the start to the end of the processing, the processing amount (the number of processing steps) notified from the computer, or the processing amount per unit time (=
The processing amount / processing time) is monitored, and after completion of the processing, these are notified to the history information management unit 13b as history information. The history information management unit 13b stores the notified history information and
It is registered in the processing procedure execution history file 22.

FIG. 12 is a block diagram of the execution processing table 13c, which is a sequence number column 13c-indicating an overall reference number.
1, start processing column (compile command showing the processing procedure) 1
3c-2, execution history column 13c-3, file size column 13c-4, execution mode column 13c-5, and process ID column 13c-6. Among them, the execution mode represents the type of execution such as compilation processing and pipeline processing. When the execution mode is pipeline processing, it indicates that pipeline processing is performed. Further, the history information stores a processing amount per unit time (speed of data processing), processing time, and the like. Since the history information is a value converted into execution on a certain one machine (computer), when the CPU speeds are different, the history information is converted and used according to the speed information. The accumulated information includes a list of processing tables and a history information portion, and stores the best value (state at that time), the worst value, the average, etc. in the execution history.

The distributed processing method of the present invention will be described below with reference to an actual compilation example. Step 1: First, the compilation procedure file 21 and the processing information 24 are input. The procedure file 21 has the following contents, and the processing information 24 has the file size of the source files xc, yc, zc to be compiled, and the file size of the source file xc is 100K.
B, the file size of the source file yc is 200K
B, the source file zc has a file size of 150 KB
Suppose

Procedure file SRCS: xc yc zc (1) OPT: -c (2) all: xo yo zo (3) ld zo xo yo -o a.out (4) xo: cc $ (OPT) xc (5 ) yo: cc $ (OPT) yc (6) zo: cpp zc ｜ cc1 -0 ｜ as> zo (7)

In the procedure file 21, (1) indicates the abbreviation of the source file.
Indicates that there are three files, xc yc zc.
(2) specifies a compile option, "-c"
Tells that the assembler should be executed and the compile process should be terminated when the .o file (intermediate object file) is created. (3) is the source file x.
Specifies the name of the intermediate object file generated from c yc zc, and each intermediate object file is xo yo zo. In (4), "ld" is a loader command, which instructs the creation of the executable file a.out using some object files shown after "ld". Note that "-o" specifies the output file name, and the files specified by the "ld" argument are combined in the specified order. (5) is the intermediate object xo
The source file xc
To compile to create an intermediate object xo. Incidentally, "cc" indicates a C language compile command, $ (OPT) is a variable part, and is replaced by the option value "-c" specified in (2). (6) is the intermediate object yo
The source file yc
Compiles to create the intermediate object yo. (7) indicates the procedure for creating the intermediate object zo. It preprocesses the source file zc, passes the processing result to the next compilation body cc1 connected by the pipe "Pipe", and the compilation body cc1 Indicates that compile processing is performed to create an assembler code file, the processing result is passed to the next assembler connected by a pipe "|", and assembler as creates an intermediate object file zo from the assembler code file. .

Step 2: When the procedure file 21 and the processing information 24 are input, the procedure file analysis section 13a first analyzes the contents of the procedure file and compiles the following compile command cc -c xc (8) cc- c yc (9) cpp zc ｜ cc1 -0 ｜ as> zo (10) ld zo xo yo -o a.out (11) is generated. Here, the compile command (11) is (8), (9),
It is executed after the processing of (10) is completed.

From the file dependency description, (8), (9), (1
Since the command (0) can be executed in parallel, the command, file size, and execution mode for these three are specified in the start processing column 13c-2, file size column 13c-4, and execution mode column 13c- of the execution processing table 13c. Fill in 5 (see Figure 13). In this case, since (10) is connected by a pipe, in other words, pipeline processing is performed, the command of each stage delimited by the pipe "|" is separated and the start processing column 13c- Fill in the 2nd class. Then,
Regarding the process of (11) executed after the processes of (8), (9), and (10) are completed, the command and the execution mode are entered in the startup process column 13c-2 and the execution mode column 13c-5. In addition, the file size column 13c-4
Contains each command (8) specified by the processing information 24,
The sizes of the files xc, yc, and zc to be processed in (9) and (10) are entered, and "self" and "pipe (g
-n) "or" gang "is entered. Run mode" s
elf "means that it can be run independently, run mode" p
"ipe" means pipeline processing, g is a group, n is the execution order of pipes, the same processing of group g is executed at the same time. Execution mode "gang" ends the processing of the sequence number in parentheses. It means to execute afterwards.
The “execution history” is the history information management unit 13 as described later.
Obtained by inquiring b. If this history information does not exist, scheduling of process allocation is performed based on the file size. P.ID indicates the process ID when the corresponding process is activated.

Step 3: Next, the procedure file analysis unit 13a requests the history information management unit 13b for execution history information. The history information management unit 13b reads the execution history corresponding to the requested “processing” from the processing procedure execution history file 22, creates the execution history table shown in FIG. 14, and passes it to the procedure file analysis unit 13a. In addition, sequence numbers 1, 2,
There is no execution history for the processing of No. 6, and for the pipeline processing of sequence numbers 3, 4, and 6, the processing capacity ratio (processing amount per unit time) is 10 in advance by measurement or the like.
It shall be obtained as 0, 20, 40. When the processing time is used as the execution history, T: 60 (execution time 60
Second), an execution history table is created. The procedure file analysis unit 13a refers to the input execution history table and embeds each execution history information in the corresponding execution history column 13c-3 of the execution processing table 13c. As a result, the execution processing table 13c shown in FIG. 15 is created.

Step 4: In parallel with the above, the host information management section 13d periodically updates the contents of the distributed host table 13e, and the distributed host table 13e has the contents shown in FIG. 16 at the current time. . It should be noted that the distributed host table 13e is stored in the host NO. Column 13e-1, host name column 13e-2, user column 13e-3, CPU utilization column 13e-4, CPU
It has a load column 13e-5 and a CPU speed column 13e-6. FIG.
Then, it is assumed that the host 4 has a CPU usage rate of 30% and the other hosts have a CPU usage rate of 0%.

Step 5: When the execution process table 13c is created, the process allocation unit 13g allocates each process to the host using the data of the execution process table 13c and the distributed host table 13e. Regarding the pipeline processing of (10), the processing is distributed for each stage. First,
The size of the file is used as the amount of processing, and each process (the process of sequence numbers 1 to 6) is sorted in descending order of the amount of processing (step 5a). As a result, the processes are rearranged as shown in FIG.

Next, the process allocation unit 13g refers to the built-in host and the integrated processing amount correspondence table (FIG. 18), finds the host with the smallest integrated processing amount, and executes the process with the largest processing amount (Seq 2). assign. In the initial stage, since the integrated processing amount of all hosts is 0, allocation is not performed based on the integrated processing amount, and the CPU speed is the fastest and the CPU
Assign to a host with a light load. In other words, the effective CPU speed of equation (b) is calculated for each host, and the most effective CPU speed is calculated.
Assign processing Seq 2 to the host with higher speed. In this case, the effective CPU speed of host 1 to host 5 is "1",
Since it becomes "1", "1", "1.4", "2", it is assigned to the host 5 (step 5b).

Thereafter, the integrated processing amount of the host 5 is updated. That is, the file 200 KB is converted into the effective CPU speed 2
The value 100 KB divided by is the current integrated processing amount (initial value is 0)
Is added to the sum and is entered in the integrated throughput table as the integrated throughput of the host 5 (FIG. 18B, step 5c). Then, it is judged whether the allocation is completed (step 5d), and if not completed, the next largest process (Seq 4) is similarly allocated to the predetermined host. That is, of the hosts 1-4
The processing (Seq 4) is assigned to the host 4 having the highest effective CPU speed, and the integrated processing amount is updated (FIG. 18 (c)).
Thereafter, similarly, the respective processes of Seq 5, Seq 3, and Seq 1 are assigned to the host 2, host 1, and host 3, respectively, and the integrated processing amount is updated, as shown in FIG. 18 (d).

Finally, the process of Seq 6 is assigned to the host with the smallest cumulative processing amount (the host with the high effective CPU speed when there are two or more). In this case, the accumulated processing amount of the host 1, the host 3, and the host 5 is the smallest, but since the effective CPU speed of the host 5 is faster, the process of Seq 6 is assigned to the host 5, and the assignment process is ended. Step 6: When the process allocation is completed by the above step 5, the host / execution process mapping table 13h showing the correspondence between the host and the process shown in FIG. 19 is created.

Step 7: When the creation of the host / execution process mapping table 13h is completed, the process activation / execution monitoring unit 13i instructs the corresponding hosts to execute the processes of Seq 1 to Seq 5 in parallel. Of these, Seq 3, Seq 4,
Seq 5 is connected and executed by a pipe in order.
Further, the process activation / execution monitoring unit 13i instructs the host 5 to execute the process Seq 6 after the completion of all the processes Seq 1 to Seq 5.

Step 8: Process start / execution monitoring unit 13i
Monitors the time required for actual processing (processing time). That is, the elapsed time from the time when the host is instructed to start processing to the time when the host notifies the end of processing is monitored,
The elapsed time is defined as the processing time. Then, the processing time is multiplied by the effective CPU speed to calculate the effective history, and the effective history is notified to the history information management unit 13b. For example, 10 seconds to execute Seq 1, 11 seconds to execute Seq 2, and 9 seconds to execute Seq 4.
If it takes 2 seconds, the effective CPU speed of the host 3 that executes Seq 1 is 1, so 10 seconds (T: 10)
This is the effective history of the process. Also, since the effective CPU speed of the host 5 that executes Seq 2 is 2, it is 22 (= 11 × 2).
Let seconds be the effective history of Seq 2 processing. Furthermore, since the effective CPU speed of host 4 executing Seq 4 is 1.4, 1
The effective history of Seq 4 processing is 2.6 (= 1.4 × 9) seconds. Also, Seq 3 to Seq 5 are pipeline processing,
Since the processing speed P has a relationship of 5: 1: 2, the execution history of Seq 3 is 2.52 (= 12.6 / 5) seconds, and Seq 5
Execution history is 6.3 seconds.

Step 9: If the notified execution history is registered in the processing procedure execution history file 22 by the history information management unit 13b, the compilation distribution processing ends. FIG. 20 shows a newly created execution history table. Thereafter, the processing is arranged based on the processing time T regardless of the file size, and is sequentially assigned to the host. FIG. 21 is a flow chart of the entire compilation distribution processing described above, and the step numbers given to the respective blocks correspond to the step numbers in the sentence. In the distributed processing of the above compilation, the processing allocation was performed according to the algorithm of, but the distributed processing can be performed by appropriately adopting the following algorithms.

(B) State Collection Control Local Resource Manager LM / Configuration FIG. 22 is a configuration diagram of the local resource manager LM that monitors the state of the computer and sends the state data to the domain resource manager DM (see FIG. 3). 41 is the state of the host, that is, resources (CPU, memory, disk)
It is a resource information acquisition unit that periodically acquires the utilization rate of the resource from the host. Hereinafter, the host state will be described focusing on the CPU utilization rate η. Reference numeral 42 is a load state detection unit that compares the acquired current utilization rate with the past utilization rate, for example, the average value of the past 10 times or the utilization rate of the previous one. A state history management unit 43 stores a resource state accumulation buffer 43a that stores the utilization rate η and other data input from the load state detection unit 42 in association with the day of the week and time, and statistics for performing statistical processing of each data. Processing unit 43b and statistical data file 44
The file input / output unit 43c for inputting / outputting

FIG. 23 shows the resource status accumulation buffer 43a.
As shown in (a), the usage rate (η) 43a-3 and the probability of the computer going down (P
d) 43a-4, computer stability (Sa) 43a-5, remaining service time (Ts) 43a-6, short-term and long-term forecast information 43a-7, 43a-8,
State data such as the valid time (Te) 43a-9 of the data is stored. State data other than the utilization rate is calculated and stored by the control unit 45 described later. As shown in FIG. 23B, the resource status accumulation buffer 43a also stores host hardware information, that is, CPU speed, CPU type, installed memory amount, and the like. The statistical processing unit 43b uses the average value, the sum, and the data in each column of the resource status accumulation buffer.
Values such as the number of samples and standard deviation are calculated, and these statistical data and each state data are put together and statistical data file 44
As a disk. Reference numeral 45 denotes a control unit, which calculates the state data such as the stability Sa of the computer, the data valid time Te, and the prediction information based on the comparison result by the load state detection unit 42 and controls the transmission of the state data. The stability Sa of the computer and the effective time Te of the data are calculated as follows, for example.

The stability Sa has an initial value of 50, and the data valid time Te has an initial value of 2 seconds. Also, the stability is 0-100
, And the effective time takes a value of 2 to 300 seconds. By calculation, the value exceeding these ranges is the minimum value,
It is changed to the maximum value. If the variation of the utilization rate is within 1%, 10 is added to the current stability (Sa + 10 → Sa), and 5 is added to the current valid time (Te + 5 → Te). When the fluctuation is 1 to 2%, the load of the computer is stable, and therefore increases by Sa + 5 → Sa and Te + 2 → Te. If the fluctuation is 2 to 5%, the stability and the effective time are not changed. When the fluctuation is 5 to 7%, the load of the computer is a little unstable, so the stability and the effective time are changed from Sa-2 to Sa, T.
It decreases with e-1 → Te. When the fluctuation is 7 to 10%, the load on the computer is unstable, so the stability and the data valid time are set to the initial values. When the fluctuation is 10 to 20%, Sa-20 → Sa and the effective time is set to the initial value. When the fluctuation is 20% or more, 0 → Sa is set, the effective time is set to the initial value, and the state data is instructed to be transmitted to the domain resource manager DM. When the fluctuation of 20% or more continues for several times, the transmission of the state data to the domain resource manager DM is stopped,
Instruct the domain resource manager DM to send a message to suspend data transmission. The stability and the effective time can be changed depending on whether or not the variation value of the utilization rate is a certain value or more.

The probability Pd of the host going down is calculated from the statistical information. For example, it can be obtained by obtaining the number of times the host is down at that time and dividing by the measured number of days. The remaining service time Ts is the time until the probability Pd of going down from the current time to 10% or more, and is calculated using past history information. The short-term or long-term prediction information is a predicted value of the usage rate in a short time period (for example, within 5 minutes) or a long time period (for example, within 30 minutes) from the current time, and is calculated from the history information. Reference numeral 46 is a transmission information buffer that holds state data and the like to be transferred to the domain resource manager DM according to an instruction from the control unit 45. The data to be transmitted is, as shown in FIG. Sa, probability P of host going down
d, remaining service time Ts, short-term and long-term forecast information,
Data valid time (actually the calculated valid time plus 1 second to absorb the transfer delay) Te, state ID,
This is hardware information. In this case, the utilization rate η is the state data of the computer, and the stability, the valid time of the data, the probability of downtime, and the remaining service time are data indicating the reliability of the state data.

Reference numeral 47 is a message creating section for creating a message (hereinafter also referred to as a packet) to be sent to the domain resource manager DM, 48 is a communication control section for exchanging messages with the domain resource manager DM, agent AGT, client CLT, and 49 is Packet-driven unit (Pa
cket Driven unit). Upon receiving the message via the communication control unit 48, the packet driven unit 49 analyzes the message as shown in FIG. 25 (step 49a). If the message is a request to the agent, a message packet is assembled and sent to the agent (steps 49b and 49b '), and if it is a request to the domain resource manager DM, a message packet is assembled and sent to the DM (step 49b, 49b'). 49
c, 49c '), if it is a request to the client, a message packet is assembled and transmitted to the client (steps 49d, 49d').

Process Flow of Control Unit FIG. 26 is a process flow diagram of the control unit 45. Control unit 45
When the utilization rate and the fluctuation data are received from the load state detection unit 42 (step 45a), the state change is checked (step 45b). When the fluctuation of the utilization rate is small, the stability S
a and the effective time Te of the data are increased (step 45
c) If the variation of the utilization rate is large, the stability Sa and the effective time Te of the data are decreased (step 45d).
The initial value of the effective time Te is 2 seconds, which is the first state data transfer time T ₀ . Then, the probability of the host going down, the remaining service time, etc. are calculated (step 45).
e), it is checked whether the state data transfer time has come (step 45f), and if not, it is checked whether the probability of the host going down is 10% or more (step 45g), and if 10% or less, the process returns to the beginning and the following steps are performed. Repeat the process. on the other hand,
If the state data transfer time has come, or if the probability that the host is down is 10% or more, the control unit 45
Is the state data (utilization rate) at that time, stability, effective time T
Send e, etc. to the domain resource manager DM (step 45h). Then, the state data transfer time T ₀ is set as the effective time at that time (Te → To, step 45i), and the process is returned to the beginning and the subsequent processes are repeated.

If the state change is a sudden change of 20% or more in step 45b, it is checked whether the rapid change is continuous several times (step 45j), and if it is not continuous, 0 → S.
At the same time, the effective time Te is set to the initial value (step 45k), state data and the like are transmitted to the domain resource manager DM, and To = Te is set to the beginning and the subsequent processing is repeated. When the rapid change of 20% or more in a row occurs, the transmission of the state data to the domain resource manager DM is stopped, and a message to the effect that the data transmission is interrupted due to the sudden change is issued to the domain resource manager DM.
To (step 45m). In this case, the message to suspend the data transmission is sent at the initial value interval of the effective time,
When the sudden change state disappears, the processes after step 45a are restarted.

When a special state occurs, the state data is not transmitted. For example, if the CPU usage rate becomes a set value, for example 80% or more, and the processing cannot be executed remotely for another computer, the status data is not transmitted. In this case, the domain resource manager DM directly inquires the computer of the status as necessary. Further, even when the computer is already remotely executing the process requested by the client, the state data is not transmitted, and the state data is transmitted after the remote process is completed. Further, when the status data is transmitted during remote execution, the value obtained by subtracting the remote processing usage rate from the overall usage rate is taken as the true usage rate. By doing this, it is possible to correctly predict the utilization rate when the remote execution process is completed. As described above, the number of pieces of transmission data per unit time decreases for a computer whose state is stable. In addition, the domain resource manager DM that collects the state data uses the stability Sa of the host, the effective time Te of the data, and the prediction information that are sent together with the state data. The system allows you to grasp the status of the computer.

Domain Resource Manager DM / Configuration FIG. 27 collects and manages the status data of each computer,
It is a block diagram of the domain resource manager DM which determines an available computer with respect to the computer use request of a client. 51 is a local resource manager L of each computer
A host state storage unit that stores the host state data sent from M, 52 is a host state management unit that manages the contents of the host state storage unit, and 53 is an agent AGT (FIG. 3).
It is a resource use information storage unit that stores information sent from the client indicating the resource use status of the client. The resource usage information includes a client host name, a used resource name, a usage amount, a usage time, and the like. Reference numeral 54 is a resource use management unit, 55 is a resource allocation scheduler that determines a computer that can be used by a client in consideration of a host state and client resource use information in response to a computer use request from a client, and 56 is a local resource manager L of each computer.
A communication control unit 57 for exchanging messages with M is a packet driven unit. As a received message,
A host status message MS1 (see FIG. 24) from the local resource manager LM, a resource utilization information message MS2 from the agent AGT, and a client CLT.
From the resource use request message MS3. As shown in FIG. 28A, the resource use request message MS3 includes a destination, a resource allocation request, a resource type (for example, CPU),
It includes the required quantity (number of CPUs), host name, and connection port name.

Process of Domain Resource Manager DM FIG. 29 is an overall process flow diagram of the domain resource manager DM. When the packet driven unit 57 receives a message from the local resource manager LM via the communication control unit 56, it analyzes the message.
If the received message is the host status message MS1,
The packet driven unit 57 passes the message to the host status management unit 52, and the host status management unit 52 stores the host status data in the host status storage unit 51 (step 50a). In addition, the received message is the agent AGT.
If the resource usage information message is MS2, the packet driven unit 57 passes the message to the resource usage management unit 54, and the resource usage management unit 54 registers the resource usage information for each client in the resource usage information storage unit 53 ( Step 50b). Further, if the received message is the computer resource use request message MS3 from the client CLT, the message is allocated to the resource allocation scheduler 55.
Pass to. The resource allocation scheduler 55 refers to the resource use request message MS3, the host status, and the resource use information of the client to determine an available computer, and sends a connection request message MS4 to the agent AGT of the computer. As shown in FIG. 28B, the connection request message MS4 includes a destination, an AGT connection request, a resource type, a host name, and a port number.

Processing of Host Status Management Unit FIG. 30 is a processing flow chart of the host status management unit 52.
This is an example focusing on one host state. When the host status management unit 52 receives the host status message MS1 from the packet driven unit 57, the host status storage unit 5 stores the host status data (see FIG. 24) in association with the host name.
1 (steps 52a and 52b). Then, the valid time Te of the data contained in the host status data is reduced every predetermined time (step 52c), and it is checked whether the valid time Te has become 0 (step 52d). Te =
If it is not 0, the process returns to step 52a, and if the host status message is not received, the effective time is reduced. If a new host status message is received before Te = 0, the old host status data is updated with the new host status data received and the above processing is repeated. If the computer is operating normally, before Te = 0, the
New host status data is sent from the resource manager LM. On the other hand, if the effective time Te = 0,
Assuming that the computer is down, the computer is excluded from the distributed processing targets until new host status data is sent. That is, even if a client requests to use a computer, the computer is not assigned (step 5).
2e).

Process of Resource Allocation Scheduler FIG. 31 is a process flow chart of the resource allocation scheduler 55. When the resource allocation scheduler 55 receives the computer resource use request message MS3 from the packet driven unit 57, the resource allocation scheduler 55 finds the computer with the smallest utilization rate and sets it as the available computer (steps 55a, 55b,
55d). In this case, if two or more computers are obtained (step 55c), the available computer is determined in consideration of the stability, the CPU speed, the probability of the host going down, the predicted value of the future utilization rate, and the remaining service time ( Step 55e). For example, an evaluation function having these as variables is prepared, and the computer having the largest function value is determined as the available computer. The function is determined such that the larger the stability, the faster the CPU speed, the smaller the probability of down, the smaller the predicted utilization rate in the future, and the longer the remaining service time, the larger the function value.
Then, it is checked whether the number of computers requested by the client has been determined (step 55f). If not, the process returns to step 55b and the same processing is performed. Once the requested number of computers has been determined, send a connection request message MS4 to the agents of the available computers,
The computer resource allocation processing ends. When the usage requests are simultaneously issued from a plurality of clients, the computer allocation schedule is determined based on the resource usage information for each client, and the computer resources are allocated to each client.

(D) Generation control of resource utilization request message from client Configuration of client FIG. 32 is a configuration diagram of the client. The same parts as those in FIG. 11 are designated by the same reference numerals. 13a is a procedure file 2
The procedure file analysis unit 13c analyzes the contents of No. 1 to generate a processing command, and outputs the relationship between the processing such as the processing command and the file and the file size by referring to the processing information 24. A processing table for storing data related to the amount, 13e is a host information storage unit for storing host information (utilization rate, stability, hardware information, etc.) included in the resource use token transmitted from the available computer, and 13g is available A process allocation unit that allocates a process to a computer, 13j is a resource request unit that creates and sends a resource use request message (see FIG. 28 (a)) based on a process command, and 13m is a communication control unit.
It has a connection disconnection detection mechanism 13n that detects that the connection established with the agent has been disconnected. 13p is a resource use information storage unit, and as shown in FIG. 33, the name of the host used for processing distribution,
The port number, the used resource name, the used amount, the used time, the statistical data of the used amount, etc. are stored in association with the agent. 13q is a packet driven unit that analyzes a message received from the local resource manager LM or the agent AGT.

The disconnection detecting mechanism 13n can detect the disconnection by the following method. That is,
The connection disconnection is detected because an error occurred when writing to the corresponding port. Alternatively, the KEEPALIVE (UNIX) option is set when the connection is established, and the disconnection is detected by the interrupt generated from the OS when the connection is disconnected. Alternatively, the disconnection of the connection is detected by a method of detecting the disconnection of the connection by receiving a message of disconnection sent before the computer goes down.

The operation procedure file analysis unit 13a of the client analyzes the contents of the procedure file 21 to generate a processing command, and at the same time, the processing information 24
The relationship between the processing command and the file and the size of the file are output by referring to the above, and are stored in the processing table 13c.
The resource requesting unit 13j determines the amount of resources (the number of computers) required for distributed processing based on the processing procedure stored in the processing table.
To generate a resource use request message MS3 and send it to the domain resource manager DM via the local resource manager LM (see FIG. 27). The domain resource manager DM determines the available computer as described above, and sends the connection request message MS4 to the agent of the available computer. The computer agent AGT that has received the connection request message establishes a connection with the client CLT as described later, and thereafter sends a resource use token that permits exclusive use of computer resources to the client CLT. This resource use token is token I as shown in FIG.
D, resource status (CPU usage rate), available resource amount,
It includes resource stability, resource usage time limit, hardware information (CPU speed), token valid time, and the like.

When the packet driven unit 13q receives the resource use token from the agent AGT, the host information contained in the token is stored in the host information storage unit 13e.
And the resource utilization information stored in association with the agent AGT is updated. Further, the packet driven unit 13q instructs the process allocation unit 13g to execute the process allocation control. As a result, the processing allocation unit 13g distributes the processing to the available computers by the distributed control described in section (a). Packet driven unit 1
Upon receiving the resource use end message from the agent AGT, 3q deletes the host information corresponding to the agent AGT from the host information storage unit 13e. When the connection disconnection detection mechanism 13n detects the connection disconnection, the packet driven unit 13q regards the connection disconnection as computer down or forced use refusal, and disconnects the agent AG.
The host information corresponding to T is deleted from the host information storage unit 13e. After that, the resource allocation unit 13g distributes the processing excluding the computer, and the resource request unit 13j reacquires the resource as needed.

(E) Exclusive allocation control of computer CLT is the domain resource manager DM
In order to be able to actually request processing to the available computers determined by
It is necessary to obtain a permission to use more. Agent Configuration FIG. 35 is a configuration diagram of the agent AGT. 61 is the state of computer resources (CPU usage rate), the amount of available resources,
A resource utilization state storage unit for storing the stability of resources and the like, and a resource utilization information storage unit 62 for storing data indicating the resource utilization state for each client. As shown in FIG. 36, the resource usage information for each client is the host name of the client,
It includes connection port number, user, resource name, usage amount, usage time, statistical data of usage amount. Reference numeral 63 is a resource usage status management unit that registers the resource usage status (host status) of the local computer sent from the local resource manager LM in the storage unit 61, and 64 is resource allocation control to a plurality of clients, connection establishment control, resource A resource scheduler that executes transmission control of usage tokens (see FIG. 34), 65 is a resource usage token storage unit, 66 is a communication control unit, 67 is a resource usage control unit that performs token check and execution control of processing commands from the client, Reference numeral 68 is a packet driven unit, and 69 is a connection control unit for controlling connection / disconnection of the connection with the client CLT.

The token-based permission control domain resource manager DM determines the available computers in response to the resource use request from the client CLT as described above, and sends the connection request message to the agent AGT of each available computer (see FIG. 28 (b). ) See) is sent. When the packet driven unit 68 of each agent AGT receives the message via the communication control unit 66, it sends it to the resource scheduler 64 and the connection control unit 69. As a result, the connection control unit 69 uses TCP in UNIX.
According to the / IP protocol, a connection is set up with the client using the host name and port number specified in the connection request message. Next, the resource scheduler 64 determines a client to which the resource is allocated in consideration of the resource utilization state (state of the own computer) and the resource utilization state of the client, creates a resource utilization token, and sends it to the client. At this time, the resource scheduler 64
The content of the resource use token is stored in the resource use token storage unit 65 in association with the token ID. Further, the resource scheduler 64 updates the contents of the resource utilization information storage unit 62 of the client.

When there are a plurality of clients, the resource scheduler 64 considers the resource utilization status of the clients and allocates the resources to the clients in order of the resource utilization amount and the utilization time in a short order, or in the round-robin sequence. The resource use token is sent so that the allocation or the use is alternately permitted. By doing this, even if multiple clients make resource requests at the same time,
The agent AGT can make the other client wait by sending the resource use token to only one client. When the client CLT receives the resource use token, the process is distributed to the available computers and the resource use message (process request message) MS5 having the token ID and the process command string is sent to the agent AGT as shown in FIG. When the packet driven unit 68 receives the resource use message MS5, it transfers the message to the resource use control unit 67. When the message is input, the resource use control unit 67 reads the current resource use state from the resource use state storage unit 61, reads the content of the resource use token from the token storage unit 65 using the token ID as a keyword, and compares the two. To do. When the resource status changes considerably when the resource use message MS5 is received compared to when the token is sent, the resource scheduler 64 uses the client CL.
Notify T to cancel the resource use message and send a new resource use token. Alternatively, the resource scheduler 64 sends a message notifying the processing delay to the client CLT and delays its use until the computer resources become available.

On the other hand, if the resource state has not changed, the resource use control unit 67 extracts and executes the processing command string and sends the execution result to the client. In this case, the connection control unit 69 establishes a connection with the client CLT in order to output the execution result of the command sequence and the like. The resource use control unit 67 monitors whether the resource use message MS5 is received within the valid time of the token included in the resource use token. If the resource use message MS5 is not received within the valid time, the transmitted resource use token is checked. Resource scheduler 64 to invalidate and send the token to the next client
Instruct. The resource use control unit 67 disconnects the resource scheduler when the usage time designated by the token has elapsed, when the client has not used the computer for a predetermined time, and when a token return message has been received from the client. 64 is instructed to send the resource use token to the next client.

The other control packet driven unit 68 notifies the resource scheduler 64 when there is a request from the domain resource manager DM to notify the resource utilization status of the client, and the resource scheduler 64 notifies the domain resource manager DM.
Send a client resource usage status message to.
When the packet driven unit 68 receives a resource usage status message from its own local resource unit LM, it notifies the resource usage status management unit 63 and the storage unit 61.
Update the contents of.

Modified Example In the above, the case where the domain resource manager DM decides the available computer in response to the request from the client CLT, the domain resource manager DM is made to have the function of the client and the state data of each computer In addition to collecting and managing, it is also possible to configure each computer to execute the processing in a distributed manner. Also, the above is the computer C
Although the description has been made focusing on the PU utilization rate, it can be applied to the distributed use of the memory, the disk, and the like. Although the present invention has been described above with reference to the embodiments, the present invention can be variously modified according to the gist of the present invention described in the claims, and the present invention does not exclude these.

[0068]

As described above, according to the present invention, a scheduler for processing distribution is provided in at least one computer, and when the processing amount of the processing to be distributed is known, the scheduler integrates the processing in descending order of the processing amount. The computer allocates a small amount (initial value is 0) to the computer and updates the integrated processing amount of the computer by adding the allocated processing amount to the integrated processing amount. Since the allocation is made, the processing amount of each computer can be made uniform, and the processing end time of each computer can be made almost the same time. That is, the efficiency of the computer system can be improved and the processing result can be obtained quickly. Further, according to the present invention,
Since the processing amount obtained by actually executing each process is stored as history information and the processing amount of each process is obtained by referring to the history information, the correct processing amount can be obtained. The uniformity of the throughput of each computer can be further increased, and the efficiency of the computer system can be improved.

Further, according to the present invention, since the processing amount is set to the processing amount per unit time, the CPU utilization rate can be made uniform as a whole, and a plurality of processings having different processing speeds can be efficiently parallelized by a plurality of computers. Can be executed. Further, according to the present invention, since the processing amount is expressed by the file size to be processed, the processing time, etc., a predetermined value can be selected and used as the processing amount in the distributed processing. Further, according to the present invention, a value obtained by dividing the processing amount by the CPU speed of the computer is added to the integrated processing amount to update the integrated processing amount, and the processing is distributed based on the integrated processing amount. Therefore, evenly distributed processing can be performed in consideration of the difference in processing speed of each computer, and the efficiency of the system can be further improved. Further, in this case, since the CPU speed is configured to be corrected based on the load of the computer, the uniformity of processing can be further increased. Further, according to the present invention, when processing is distributed to a plurality of computers connected to a network, or when one or more computers are assigned to a client that is a request source of computer resources, each computer is assigned a resource utilization rate or the like. The data indicating the reliability of the state data is added to the state data of and the data is transmitted to the computer (server) that performs the distributed processing of computer resources, and the server distributes the processing to the computers based on these data, or the client. Since it is configured to enable the use of a predetermined computer by using reliability, it is possible to manage the state of each computer resource accurately by using reliability, and it is possible to allocate computer resources efficiently. Become.

Further, according to the present invention, each computer increases the reliability when the degree of the state change is small, decreases the reliability when the degree of the state change is large, and reduces the reliability. Is configured so that the transmission interval of the state data is long when it is large, and the transmission interval of the state data is short when the reliability is small, it is possible to manage the state of each computer resource accurately with less communication, and it is efficient. It is possible to allocate various computer resources. Further, according to the present invention, the hardware information of the computer is added to the state data and transmitted to the server, and the server distributes the processing to the computer resources based on the operating state of each computer, the reliability, and the hardware information, Alternatively, since a predetermined computer is assigned to the client, distributed processing according to the capacity of the computer becomes possible, and the efficiency of the system can be improved.

Further, according to the present invention, the computer is provided with a resource use permission mechanism (agent) for exclusively transmitting a token for making the computer resource available, and the agent communicates with a predetermined client who desires to use the computer resource. The connection is set to the client, and after that, the resource utilization token is transmitted to exclusively allocate the computer resource to the client, so that the plurality of computer resources are sequentially allocated to the plurality of clients without being multi-allocated. , Can be efficiently allocated. Further, according to the present invention, the agent compares the state of the computer resource when the processing request message is received from the client with the state of the computer resource when the token is sent out, and if the state change occurs, requests the processing. Since the configuration is such that the message is not accepted, it is possible to efficiently allocate a plurality of computer resources whose states change to a plurality of clients. Further, according to the present invention, when the client detects disconnection, it is configured to perform the distributed processing excluding the disconnected computer resource, so that it is possible to cope with the computer resource being down or unilateral use refusal due to the computer resource. can do.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the first principle of the present invention.

FIG. 2 is a diagram illustrating a second principle of the present invention.

FIG. 3 is an overall configuration diagram of the present invention.

FIG. 4 is an explanatory diagram of state collection of a computer.

FIG. 5 is an explanatory diagram of a computer use request from a client.

FIG. 6 is an explanatory diagram of a computer resource use permission (No. 1).

FIG. 7 is an explanatory diagram of a computer resource use permission (No. 2).

FIG. 8 is a block diagram of an embodiment of the present invention (distributed processing).

FIG. 9 is a configuration diagram of a computer system.

FIG. 10 is a configuration diagram of a compiling device.

FIG. 11 is a configuration diagram of a scheduler.

FIG. 12 is a configuration diagram of an execution processing table.

FIG. 13 is a first explanatory diagram of a process execution table in a compile process.

FIG. 14 is an explanatory diagram of an execution history table.

FIG. 15 is a second explanatory diagram of a process execution table in the compile process.

FIG. 16 is an explanatory diagram of a distributed host table.

FIG. 17 is an explanatory diagram of sorting.

FIG. 18 is an explanatory diagram of an integrated treatment amount table.

FIG. 19 is an explanatory diagram of a host / execution process mapping table.

FIG. 20 is an explanatory diagram of an execution history table after execution of processing.

FIG. 21 is a flow chart of distributed processing of compilation.

FIG. 22 is a configuration diagram of a local resource manager LM.

FIG. 23 is an explanatory diagram of contents of a resource status accumulation buffer.

FIG. 24 is an explanatory diagram of transmission data to the domain resource manager DM.

FIG. 25 is an operation explanatory diagram of the packet driven unit.

FIG. 26 is a processing flowchart of a local resource manager LM.

FIG. 27 is a configuration diagram of a domain resource manager DM.

FIG. 28 is an explanatory diagram of a message.

FIG. 29 is an overall processing flow diagram of a domain resource manager.

FIG. 30 is a processing flowchart of a host status management unit.

FIG. 31 is a process flow diagram of a resource allocation scheduler.

FIG. 32 is a configuration diagram of a client.

FIG. 33 is an explanatory diagram of resource usage information.

FIG. 34 is a description of the contents of a resource use token.

FIG. 35 is a configuration diagram of an agent.

FIG. 36 is an explanatory diagram of resource usage information for each client.

FIG. 37 is an explanatory diagram of a resource use message.

[Explanation of symbols]

 11 ... Execution process group 13 ... Process allocation device (scheduler) 30 ... Computer (CPU) group 31-36 ... Computers 31a-36a ... Local resource manager 31b, 32b ... Client 32c ... Domain resource manager 31d ~ 36d ... Resource usage permission mechanism (agent)

Claims (27)

[Claims] 1. A computer resource distribution method for distributing processing to a plurality of computers connected to a network, wherein a scheduler for processing distribution is provided in at least one computer, and when the processing amount of the processing to be distributed is known, the scheduler Is assigned to a computer with a small cumulative processing amount in order from the largest processing amount, and the cumulative processing amount of the computer is updated by adding the processing amount of the processing assigned to the cumulative processing amount, and each processing is sequentially transferred to the computer. A computer resource distribution method characterized by allocating. 2. The process amount obtained by actually executing the process is accumulated as history information, and the process amount of each process is obtained using the history information. Computer resource distribution method. 3. The computer resource distribution method according to claim 1, wherein the processing amount is a processing time, a processing amount per unit time, the number of files, or a file size. 4. The integrated processing amount is updated by adding a value obtained by dividing the processed amount by the CPU speed of the computer to the integrated processing amount, and the processing is distributed based on the integrated processing amount. The computer resource distribution method according to claim 1 or 2. 5. The computer resource distribution method according to claim 6, wherein the CPU speed of the computer is corrected based on the utilization rate of the computer. 6. The computer resource distribution method according to claim 1, wherein if the accumulated processing amounts are the same, the processing is assigned to the computer having a high CPU speed. 7. A computer resource distribution method in which processing is distributed to a plurality of computers connected to a network, or one or more computer resources are allocated to a client who is a requester of the computer resources. The data indicating the degree of reliability of the state data is added to the state data indicating the state of and the data is transmitted to the computer (server) that performs the distributed processing of computer resources, and the server distributes the processing to the computers based on these data. Alternatively, a computer resource distribution method characterized by allocating a predetermined computer to a client. 8. The computer resource according to claim 7, wherein the reliability is stability indicating the degree of state fluctuation of the computer resource, valid time of data, remaining service time of the computer resource, or prediction information of future state fluctuation. Dispersion method. 9. Each computer increases the reliability when the degree of state change is small, decreases the reliability when the degree of state change is large, and when the degree of state change is large, state data 8. The computer resource distribution method according to claim 7, wherein the transmission interval of the state data is shortened, and the transmission interval of the state data when the reliability is low is shortened. 10. The computer resource distribution method according to claim 9, wherein the server decreases the reliability as time passes after the status data is received. 11. The computer resource distribution method according to claim 7, wherein a computer whose resource utilization rate exceeds a set value or a computer which is executing distributed processing does not transmit the status data to the server. 12. The computer resource distribution method according to claim 9, wherein each computer predicts a remaining service time until the computer goes down or a future state change based on a history of past state data. 13. The computer resource distribution method according to claim 7, wherein each computer transfers the utilization rate obtained by subtracting the utilization rate related to the distributed processing from the overall utilization rate to the server as status data. 14. Hardware information of a computer is added to status data and transmitted to a server, and the server distributes processing to computer resources based on the status, reliability, and hardware information of each computer, or predetermined by a client. 8. The computer resource distribution method according to claim 7, wherein the computer resources are allocated. 15. A computer assigned to a client transmits status data of the computer to the client, and the client refers to the status data and distributes processing to a plurality of computers. Computer resource distribution method. 16. In a computer resource distribution method for distributing processing to a plurality of computers connected to a network, a resource use permission unit (agent) for transmitting a token for exclusively using computer resources is provided in the computer, and the agent is A computer resource characterized by setting a connection with a predetermined client who desires to use the computer resource, and then transmitting a resource use token to the client to permit the client to exclusively use the computer resource. Dispersion method. 17. When a plurality of clients desire to use a computer resource, the agent sets up a connection with each client and sends a resource use token to one predetermined client to send to the client. 17. The computer resource distribution according to claim 16, wherein the computer resource is exclusively used, and when the use of the computer resource by the client is completed, a resource use token is sent to the next client to exclusively use the computer resource. Method. 18. The computer resource distribution method according to claim 16, wherein the client distributes the processing to the computers allocated by the plurality of agents. 19. The resource usage token includes a resource status,
17. The method according to claim 16, wherein at least one item of available resource amount, resource stability, and resource use time limit is added, and the client distributes the processing to a plurality of computers in consideration of the added item. Computer resource distribution method. 20. The agent holds the resource status included in the resource use token, the status of the computer resource when the processing request message for distributed processing is received from the client, and the held computer when the token is sent out. 19. The computer resource distribution method according to claim 18, wherein the resource status is compared, and if the status change occurs, the processing request message is not accepted. 21. The computer resource distribution method according to claim 20, wherein the agent notifies the client of a processing delay when a state change has occurred. 22. The computer resource distribution according to claim 20, wherein the agent cancels the processing request message from the client and sends a new resource utilization token to the client when the status has changed. Method. 23. The client is provided with a mechanism for detecting disconnection of a connection with an agent, and when the disconnection is detected, the processing is distributed excluding computer resources corresponding to the agent. 18. The computer resource distribution method described in 18. 24. In a computer resource distribution system that distributes processing to a plurality of computers connected to a network, a client that distributes processing to a plurality of available computers, and the status of each computer are collected and A server that allocates available computers to clients based on the state of each computer when a computer use request is made, and a resource that is provided in each computer, monitors the state of the computer, and sends state data to the server. It is equipped with a manager and a resource use permission unit (agent) that is provided in each computer and sends a token that exclusively uses computer resources to the client. The computer agent assigned to the client sends the token to the client, and then , The client is a computer resource that distributes the processing to each computer Distributed processing system. 25. The computer resource distributed processing system according to claim 24, wherein the resource manager transmits data indicating the reliability together with the state data, and the server allocates the computer to the client based on these data. 26. When the processing amount of the processing to be distributed is known, the client allocates the processing amount from the largest processing amount to the computer having the smallest cumulative processing amount, and adds the processing amount of the processing allocated to the cumulative processing amount. A computer resource distribution system characterized in that the integrated processing amount of the computer is updated by doing so, and each processing is sequentially assigned to the computer. 27. The agent of the computer assigned to the client sends the status data and the hardware information of the computer to the client, and the client refers to the status data and the hardware information to perform the processing on a plurality of computers. 27. The computer resource distribution system according to claim 26, wherein