JP2017162266A - Parallel processor, parallel processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To execute inter-process communication processing executed in a parallel program and calculation processing in parallel.SOLUTION: A parallel processor (100) includes an inter-process communication unit (101) capable of controlling inter-process communication including at least data transfer processing executed among a plurality of processes in a parallel program, and a multithread controller (102) for progressing the inter-process communication by a first thread from when synchronous processing is performed in the first thread among a plurality of threads generated during a practical procedure of the parallel program until the first thread and one or more second threads that are the other thread among the plurality of threads are synchronized.SELECTED DRAWING: Figure 4A

Description

  The present invention relates to a parallel processing device or the like that executes interprocess communication executed between a plurality of processes and other arithmetic processing in parallel.

  In the field of HPC (High Performance Computing), a parallel program that executes parallel processing using a plurality of computing nodes (arithmetic processing devices or the like) is used. For example, such a parallel program executes a plurality of processes in parallel by individually allocating processes to a plurality of computing nodes (hereinafter may be referred to as “calculation nodes”). Each process performs implemented processing by transmitting and receiving data to and from each other. In addition, when each computing node includes an arithmetic device (for example, a multi-core CPU (Central Processing Unit)) that can process a plurality of threads in parallel, each process executes parallel processing by using a plurality of threads. be able to.

  Techniques related to parallel processing using a plurality of threads are disclosed in the following patent documents. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique aimed at executing thread switching processing at high speed in a multi-thread processor. The apparatus disclosed in Patent Document 1 shortens the time required for switching threads (particularly, the time required for fetching instructions) by preparing in advance instructions for threads in a dormant state. Patent Document 2 discloses a technique related to a barrier synchronization method for the purpose of improving the operating rate of a processor. In the method disclosed in Patent Document 2, when a predetermined number of processors reach the barrier synchronization point, a job in which barrier synchronization processing is performed until all processors reach the barrier synchronization point. Run another job.

  Further, technologies relating to the synchronization processing of a plurality of threads are disclosed in the following patent documents. Patent Document 3 discloses a technique related to a synchronous processing method that can be applied in common to a case in which a process in which the absolute amount of processing is determined is parallelized and a case in which a process in which the absolute amount of processing is not determined is parallelized. Is disclosed. Patent Document 4 discloses a technique of setting a hierarchical group for a plurality of threads processed in parallel in a plurality of processors and executing a barrier synchronization process for each group.

Japanese Patent Laid-Open No. 10-283203 JP 11-31148 A JP 2011-134145 A JP 2006-259821 A

  As a method for managing the progress of inter-process communication in a parallel program, a “communication thread method” and a “calculation thread method” are known. The communication thread method is a method in which a communication thread independent of a calculation thread exclusively manages the progress of inter-process communication. The calculation thread method is a method in which a calculation thread itself executed in an arithmetic processing unit (such as an arithmetic device) manages the progress of interprocess communication. In the case of the communication thread method, since the communication thread is independent, the calculation process and the progress of the communication process can be executed in parallel. On the other hand, the communication thread method has the following problems. That is, in the case of the communication thread method, the calculation resources (processing capacity of the calculation processing unit) that can be used by the calculation thread are reduced by the communication thread. In addition, when the communication thread and the calculation thread share the arithmetic processing unit, thread switching (switching) occurs, and overhead such as context switching occurs. In HPC, in many cases, it is desirable to allocate calculation resources to calculation processing as much as possible, and a calculation thread method is often used. Since the calculation thread method does not use a communication thread, a reduction in computing resources and a context switch between the communication thread and the calculation thread, which are problems in the communication thread method, do not occur.

  However, in the calculation thread method, if the process related to the interprocess communication is not described during the calculation process in the parallel program, the process related to the data transfer between the processes is not executed. In this case, the timing at which data transfer processing between processes is executed may be delayed. This causes a problem that data transfer processing and calculation processing in a plurality of processes may not be executed in parallel.

  On the other hand, the technique disclosed in Patent Document 1 is a technique aimed at speeding up the switching of threads, and the technique disclosed in Patent Document 2 is an operation rate of a computing device by switching threads. This technology aims to improve That is, since all of these technologies are premised on thread switching, the overhead itself such as context switching cannot be completely eliminated. In addition, none of these techniques can solve the above-described problems related to the calculation thread method. The techniques disclosed in Patent Document 3 and Patent Document 4 are both techniques for implementing the synchronization processing itself among a plurality of threads, and are not techniques that can solve the above-described problems related to the calculation thread method.

  The present invention has been made in view of the above circumstances. That is, the main object of the present invention is to provide a parallel processing device and the like capable of executing inter-process communication processing and calculation processing in parallel using calculation resources consumed in the process of calculation processing in a parallel program. One of them.

  In order to achieve the above object, a parallel processing device according to one aspect of the present invention is an interprocess communication unit capable of controlling interprocess communication including at least data transfer processing, which is executed between a plurality of processes in a parallel program. Among the plurality of threads generated in the execution process of the parallel program, the first thread and the other thread among the plurality of threads after the synchronization process is executed in the first thread. A multi-thread control unit that advances the inter-process communication by the first thread until one or more second threads are synchronized.

  The parallel processing method according to one aspect of the present invention controls inter-process communication including at least data transfer processing executed between a plurality of processes in the parallel program according to execution of the parallel program. Among the plurality of threads generated in the execution process of the parallel program, after the synchronization process is executed in the first thread, the first thread and one or more other threads among the plurality of threads Until the second thread is synchronized, the inter-process communication proceeds by the first thread.

  The object is also achieved by a parallel processing device having the above configuration, a computer program that implements the parallel processing method by a computer, a computer-readable recording medium in which the computer program is stored, and the like.

  According to the present invention, it is possible to execute inter-process communication processing and calculation processing in parallel by using computation resources consumed in the process of calculation processing in a parallel program.

FIG. 1 is an explanatory diagram illustrating the process of inter-process communication processing by the butterfly method. FIG. 2 is an explanatory diagram showing a specific example of a parallel processing program. FIG. 3 is an explanatory diagram illustrating an example of a timeline chart when the parallel processing program illustrated in FIG. 2 is executed. FIG. 4A is a block diagram illustrating a functional configuration of the parallel processing device according to the first embodiment of the present invention. FIG. 4B is a block diagram illustrating a functional configuration of the parallel processing device according to the second embodiment of the present invention. FIG. 5A is an explanatory diagram (pseudo code) illustrating an algorithm of interprocess communication executed in the parallel processing device according to the second embodiment of this invention. FIG. 5B is a flowchart illustrating the processing contents of the inter-process communication algorithm executed in the parallel processing device according to the second embodiment of the present invention. FIG. 6A is an explanatory diagram illustrating an example of data recorded in the communication processing recording unit according to the second embodiment of the present invention. FIG. 6B is an explanatory diagram illustrating an example of data recorded in the synchronization waiting thread recording unit according to the second embodiment of this invention. FIG. 7 is a flowchart illustrating the operation of the communication start processing unit in the second embodiment of the present invention. FIG. 8A is a flowchart illustrating an example of the operation of the communication processing management unit in the second exemplary embodiment of the present invention. FIG. 8B is a flowchart illustrating another example of the operation of the communication processing management unit in the second exemplary embodiment of the present invention. FIG. 9A is a flowchart illustrating an example of the operation of the barrier synchronization processing unit in the second embodiment of the present invention. FIG. 9B is a flowchart illustrating another example of the operation of the barrier synchronization processing unit in the second exemplary embodiment of the present invention. FIG. 10 is a flowchart illustrating the operation of the communication completion processing unit in the second embodiment of the present invention. FIG. 11 is an explanatory diagram illustrating an example of a timeline chart when a parallel processing program is executed in the parallel processing device according to the second exemplary embodiment of the present invention. FIG. 12 is a diagram illustrating a configuration of a hardware device capable of realizing the components of the parallel processing device in each embodiment of the present invention.

  Prior to describing the embodiment of the present invention, technical considerations and the like regarding the present invention will be described in more detail.

  In programming in the HPC field, inter-process communication technology such as MPI (Message Passing Interface) is used for inter-process communication of parallel programs. In the following description, as a specific example, it is assumed that inter-process communication is performed using MPI, but the present embodiment is not limited to this. In the following, inter-process communication using MPI may be referred to as “MPI communication”.

  The MPI communication process includes a data transfer request issue process, a data transfer process, and a data transfer completion process. The data transfer request issuance process is a process in which the CPU core requests an IO (Input / Output) device to start communication. The data transfer process is a process in which the IO device transfers data. The data transfer completion process is a process in which the CPU core confirms or waits for the completion of the data transfer of the IO device. Here, the CPU core is an arithmetic processing unit capable of executing processing related to one or more threads, for example. For example, one CPU may include one or more CPU cores. For example, one CPU may be capable of processing a thread having a number of CPU cores or more in parallel.

  The MPI communication method can be classified into blocking communication and non-blocking communication. In the case of blocking communication, for example, after executing a data transfer request issuance process, the CPU core performs a data transfer completion process and waits for the data transfer process by the IO device to be completed. Thereafter, the CPU core executes arithmetic processing other than inter-process communication.

  In the case of non-blocking communication, the CPU core executes a calculation process without executing a data transfer completion process after executing a data transfer request issuance process. Thereafter, the CPU core executes a data transfer completion process at an appropriate timing. In the case of non-blocking communication, since the calculation process by the CPU core and the data transfer process by the IO device are executed in parallel, the execution time of the entire program is shortened.

  In addition, the MPI communication method can be classified into one-to-one communication and collective communication from the viewpoint of the number of communication objects. One-to-one communication is MPI communication that transmits or receives data between two processes. Collective communication is MPI communication that performs data collection, distribution, exchange, reduction operation, and the like in two or more process groups (a group composed of a plurality of processes). Collective communication may be realized, for example, by combining one-to-one communication.

  In MPI, for example, a non-blocking collective communication function called MPI_Iallreduce function is provided. This function is a function that causes a certain process group to perform a reduction operation and returns the result to all processes. The reduction operation is an operation for obtaining an operation result set including a smaller number of elements than a specific number by performing a specific operation on a data set including a specific number of elements. Specifically, the reduction operation is an operation for obtaining one output data from a plurality of input data, for example. This will be described using a specific example. Assume that a parallel program is executed using eight processes, and each process has a value of “0” to “7” one by one. In such a parallel program, when a reduction operation for calculating the sum using the MPI_Iallreduce function is executed, 23 (the result of the sum from “0” to “7”) is returned to all eight processes. As a simple communication algorithm capable of realizing the MPI_Iallreuce function, a method of performing a reduction operation after each process transmits / receives data to / from all other processes can be considered. The number of communications in this case is on the order of “O (N ^ 2) (N: number of processes)”. That is, since the number of communication increases in proportion to the square of the number of processes, when the number of processes increases, the processing load required for inter-process communication increases.

  In general, more efficient communication algorithms are used to speed up interprocess communication. One such algorithm is the butterfly method. The butterfly method divides communication into stages. In other words, in the butterfly method, a plurality of stages (rounds) of communication processing are executed. In each round, each process transmits / receives an intermediate result calculated up to the previous round to / from another predetermined process.

  FIG. 1 shows transition of values held by the process “0” when the butterfly method is employed. Process “0” initially has the value “0”. In round 1, the process “0” receives the value “4” from the process “4”. The process “0” calculates (adds in this case) the received value with the value “0” held by itself and obtains the value “4”. In round 2, the process “0” receives the value “8” from the process “2”, calculates the value “4” held by itself, and obtains the value “12”. Here, the value “8” received from the process “2” is an intermediate result calculated by the process 2 until round 1. Finally, the process “0” receives the value “11” from the process “1” in round 3 and calculates the value “12” held by itself to obtain the value “23”. The value “11” is an intermediate result calculated by the process “1” up to round 2. The value “23” finally obtained is the result of the MPI_Iallreduce function.

  In the case of the butterfly method, each process transmits and receives an intermediate result of an operation, so that the number of communication is reduced to the order of O (N × log (N)) (N: number of processes). However, since the rose fly method transmits and receives a result in the middle of an operation, the next round cannot be started unless the previous round is completed. That is, a communication algorithm such as the butterfly method reduces the number of times of communication by transmitting and receiving intermediate results of arithmetic processing, but this causes an order relationship in communication. For this reason, each process needs to repeat communication processing in order according to the order relation. Hereinafter, execution of communication processing in accordance with the order relationship may be referred to as “progress of communication processing” (or “progress of inter-process communication processing”). The progress of the communication process is performed using, for example, either a communication thread method or a calculation thread method.

  Hereinafter, an overview of the progress of communication processing when the communication thread method is adopted will be described. When the communication thread method is adopted, each process executes the progress of communication processing using a communication thread independent of the calculation thread. The calculation thread executes an MPI communication start function that is one of MPI library functions described in the parallel program, and requests communication processing from the communication thread. Then, the calculation thread starts calculation processing other than communication processing. In parallel with the calculation processing in the calculation thread, the communication thread and the IO device repeatedly execute communication processing such as data transfer request issue processing, data transfer processing, data transfer completion processing, and next data transfer request issue processing. To do. After executing the calculation process, the calculation thread executes the MPI communication completion function described in the parallel program, and waits for the completion of the data transfer completion process by the communication thread.

  In the communication thread method, since the communication thread is independent of the calculation thread, the calculation process and the progress of the communication process are executed in parallel. On the other hand, in the case of the communication thread method, there is a problem that the calculation capability of the CPU core that can use the calculation thread is reduced by the communication thread. Further, when a communication thread and a calculation thread share a CPU core, there is a problem that an overhead such as a context switch occurs when switching threads. For example, it may be possible to alleviate the above-described problem related to the communication thread main method by speeding up thread switching, but the effect is limited because the context switch itself is not lost.

  In HPC, it is desirable to allocate as much processing capacity of the CPU core as possible to calculation processing, and a calculation thread method is often used. The outline of the calculation thread method will be described below. The calculation thread method is a method in which a calculation thread itself executed by a CPU core advances communication processing. The calculation thread executes an MPI communication start function described in the parallel program. Then, the calculation thread executes a data transfer request issuance process and starts the calculation process. The IO device that has received the request executes data transfer processing in parallel with the calculation processing of the calculation thread. For example, the calculation thread executes an MPI function (API defined in MPI) described in the parallel program in the middle of the calculation process, and in the process, completes the data transfer completion process requested earlier and the next data. Transfer request issuance processing is executed. After the calculation process, the calculation thread executes the MPI communication completion function described in the parallel program, and waits for the completion of the data transfer completion process.

  Since the calculation thread method does not use a communication thread, the problem that occurs when the communication thread method is adopted can be solved. In other words, when the calculation thread method is adopted, there is no problem that the calculation capability of the CPU core that can use the calculation thread is reduced by the communication thread. In addition, since there is no need to switch between a calculation thread and a communication thread, no context switch occurs due to thread switching.

  However, in the calculation thread method, if the execution of the MPI function is not described during the calculation process, the MPI function is not executed. That is, when a developer or the like creates a parallel program and the MPI function is not described during the calculation process, the MPI function is not executed during the calculation process. In this case, during the calculation process, the data transfer completion process and the subsequent data transfer request issue process are not executed. As a result, the data transfer completion process and the subsequent data transfer request issuance process are delayed until, for example, the MPI communication completion function is executed, and the subsequent data transfer process and the calculation process are paralleled. May not be executed.

  The above phenomenon will be described with reference to specific examples shown in FIGS. FIG. 2 is an example of the program code of the parallel program. The calculation processing in “para1 ()” and “para2 ()” is executed in a multi-thread using one or more CPU cores in the calculation node. That is, it is assumed that the parallel program is a multithread program. Barrier synchronization is executed between threads when “para1 ()” is completed and when “para2 ()” is completed.

  Inter-process communication is started by the MPI_Illreduce function before the calculation process in “para1 ()” and “para2 ()” is executed, and the completion of the process communication is waited by the MPI_Wait function after the calculation process is executed. Such inter-process communication is executed between processes executed in different calculation nodes. That is, the parallel program is assumed to be a multi-process program in which processes are executed in parallel on a plurality of calculation nodes.

  Here, it is assumed that the MPI function call is not implemented in the middle of the implementation code of “para1 ()” and “para2 ()”. That is, since the MPI function call is not described in the middle of “para1 ()” and “para2 ()”, the MPI function is not executed.

  FIG. 3 is a timeline chart when the program code of FIG. 2 is executed using four CPU cores (“Core 0” to “Core 3” in FIG. 3). “Para1 ()” and “para2 ()” are executed by four threads using four CPU cores. It is assumed that the MPI_Iallreduce function is composed of 3 rounds. In this case, when the MPI_Iallreduce function is executed, processing for issuing a round 1 data transfer request is executed. Thereafter, calculation processing of “para1 ()” and “para2 ()” is started. The data transfer process of round 1 is executed in parallel with the calculation process of “para1 ()” and “para2 ()”. On the other hand, since the MPI function is not executed during the calculation process of “para1 ()” and “para2 ()”, the round 1 data transfer completion process is not executed until the MPI_Wait function is executed. Thus, the round 2 data transfer request issuance process is not executed. As a result, the round 2 and round 3 data transfer processing is delayed until the calculation processing of “para1 ()” and “para2 ()”, and is not executed in parallel with the calculation processing.

  On the other hand, the parallel processing device in each of the following embodiments according to the present invention provides a technique capable of solving the above-described problems. As an example, the parallel processing device in each of the following embodiments can advance interprocess communication when thread synchronization processing is executed in a parallel program. By using the computation resources of the CPU core for the thread synchronization waiting time, it is possible to execute the communication process and the calculation process in parallel to reduce the execution time of the entire program.

<First Embodiment>
Hereinafter, a first embodiment, which is a basic embodiment of the present invention, will be described with reference to the drawings.

  As illustrated in FIG. 4A, the parallel processing device 100 in the present embodiment includes an inter-process communication unit 101 and a multi-thread control unit 102. These components constituting the parallel processing apparatus 100 are communicably connected using an appropriate communication method.

  The parallel processing device 100 in the present embodiment may be an information processing device (calculation node) that can execute a parallel program. The parallel processing device 100 includes, for example, at least an arithmetic processing device (for example, a multi-core CPU) that can process a plurality of threads in parallel and a memory. The parallel processing apparatus 100 may include a plurality of arithmetic processing units. For example, the arithmetic processing device may include a plurality of arithmetic processing units (for example, CPU cores) capable of executing arithmetic processing. Such an arithmetic processing unit may be capable of executing one or more threads in parallel, for example. Note that the memory may hold a parallel program and data necessary for executing the parallel program.

  The parallel program may include one or more processes including processes that can be executed in parallel by one or more threads. Specifically, when a parallel program is executed, one or more processes are generated. When a plurality of processes are generated, each process may be assigned to any of the arithmetic processing devices, for example. It should be noted that the processes generated in the execution process of the parallel program may be assigned to a plurality of arithmetic processing devices included in one parallel processing device 100, or may be assigned to a plurality of parallel processing devices 100. Each process may execute the processing implemented in the parallel program using the assigned arithmetic processing unit.

  The parallel program may generate one or more threads in the execution process (specifically, in the execution process of each process). When a plurality of threads are generated, they can be executed in parallel in the arithmetic processing unit.

  The inter-process communication unit 101 controls inter-process communication including at least data transfer processing executed between a plurality of processes in the parallel program. Specifically, the inter-process communication unit 101 includes, for example, a process executed in a certain arithmetic processing device (described as a first arithmetic processing device) and another arithmetic processing device (described as a second arithmetic processing device). It is possible to control the progress of inter-process communication related to data transmission / reception between processes executed in In this case, the first arithmetic processing device and the second arithmetic processing device may exist in the same calculation node or may exist in different calculation nodes.

  The inter-process communication unit 101 may process, for example, a data transfer request issuance process (data transfer start process), a data transfer process, a data transfer completion process, and the like among a plurality of processes. For example, the inter-process communication unit 101 may advance the data transfer process in the inter-process communication as described above according to a certain order relation.

  The inter-process communication unit 101 described above may execute inter-process communication processing using MPI. Further, the inter-process communication unit 101 may be realized by extending MPI, for example. The inter-process communication unit 101 is not limited to MPI, and may be realized by using or extending other appropriate inter-process communication technology.

  When a parallel program is executed in the parallel processing apparatus 100, the multi-thread control unit 102 controls processing related to inter-process communication by threads generated by the parallel program. For example, the multi-thread control unit 102 may execute the first thread and another thread (described as a second thread) after a synchronization process is executed in a certain thread (described as a first thread). Until the synchronization, inter-process communication proceeds by the first thread. For example, the multi-thread control unit 102 may execute processing related to inter-process communication by using the inter-process communication unit 101 (using the function provided by the inter-process communication unit 101) using the first thread. Good.

  For example, when the synchronization processing (for example, barrier synchronization processing) is executed in the first thread, the multi-thread control unit 102 determines whether all other second threads that have been executed have executed the synchronization processing. You may confirm (whether or not you are waiting for synchronization). If there is another second thread that does not execute the synchronization process, the first thread uses the function of the inter-process communication unit 101 until the second thread executes the synchronization process. The first thread may be controlled to execute the inter-communication process.

  The multi-thread control unit 102 described above may execute parallel processing between threads using, for example, OpenMP. The multi-thread control unit 102 may be realized by extending OpenMP, for example. The multi-thread control unit 102 is not limited to OpenMP, and may be realized by using or extending other appropriate parallel processing technology.

  The parallel processing device 100 configured as described above proceeds with inter-process communication using arithmetic resources of the arithmetic processing device while the first thread waits for synchronization of the second thread executed in parallel. It is possible. This shortens the execution time of the parallel program.

  Further, the parallel processing apparatus 100 proceeds with the inter-process communication process in the middle of the calculation process even when the process request communication process is not explicitly implemented in the middle of the calculation process executed by the first thread. can do. This is because, when the first thread executes the synchronization process, the first thread is controlled to execute the inter-process communication process. Thereby, the parallel processing apparatus 100 can execute the inter-process communication process and the calculation process in parallel without explicitly describing the inter-process communication process in the middle of the calculation process of the parallel processing program. The parallel processing apparatus 100 can execute interprocess communication processing and calculation processing in parallel without generating an independent communication thread for interprocess communication. As described above, according to the parallel processing device 100 of the present embodiment, the inter-process communication process and the calculation process can be executed in parallel using the operation resources consumed in the process of the calculation process in the parallel program.

<Second Embodiment>
A second embodiment of the present invention based on the first embodiment will be described in detail with reference to the drawings.

[Constitution]
FIG. 4B is a block diagram illustrating a functional configuration of the parallel processing device 400 according to this embodiment.

  The present invention is roughly composed of an inter-process communication unit 200 and a multi-thread control unit 300.

  The inter-process communication unit 200 includes a communication start processing unit 201, a communication completion processing unit 202, a communication processing management unit 203, and a communication processing recording unit 204.

  The inter-process communication unit 200 executes processing related to inter-process communication between a plurality of processes. The communication start processing unit 201 provides a function for starting inter-process communication processing. The communication completion processing unit 202 provides a function for terminating the interprocess communication. The communication processing management unit 203 provides a function for controlling data transfer processing in inter-process communication. The communication processing recording unit 204 holds information related to interprocess communication.

  The multi-thread control unit 300 includes a barrier synchronization processing unit 301 and a synchronization waiting thread recording unit 302. The barrier synchronization processing unit 301 provides a function of executing synchronization (barrier synchronization) processing between a plurality of threads. The synchronization waiting thread recording unit 302 holds information regarding threads that are in a synchronization waiting state. The synchronization waiting thread recording unit 302 may hold information indicating the number of threads waiting for synchronization (or the number of threads not waiting for synchronization).

  FIG. 5A is an example of a program (pseudo code) representing a communication algorithm capable of realizing the communication process executed by the communication process management unit 203. Hereinafter, the communication algorithm represented by the program illustrated in FIG. 5A is simply referred to as “communication algorithm”. FIG. 5B is a flowchart showing the communication process executed by the communication process management unit 203, and shows the process of the communication algorithm.

  In the communication algorithm, the data transfer request issuance process and the data transfer completion process are repeated for the required number of rounds. In the specific example shown in FIG. 5A, the number of rounds is “N” (N is a natural number of 1 or more). In the communication algorithm shown in FIG. 5A, at least one of a data transfer request issuance process and a data transfer completion process is executed in the “step 1” to “step N” rounds.

  In the data transfer completion process, completion confirmation as to whether or not the data transfer process has been completed is executed. If the data transfer has not been completed, the completion process may be interrupted without waiting for completion. In this case, step (“step 1” to “step N”) in FIG. 5A indicates, for example, a position (round) at which the process is resumed. When the process is interrupted, the communication algorithm may return (provide) a step (round) at the time of resuming the process to the communication algorithm execution source (communication algorithm caller). When the communication algorithm is completed, information indicating that the data transfer process is completed may be returned (provided) to the execution source of the communication algorithm. In the specific example shown in FIG. 5A, information indicating “algorithm completion” is used as information indicating that the data transfer processing has been completed. Since the communication algorithm does not wait for the completion of the data transfer process, it can be executed in a short time.

  An outline of the bond of the communication algorithm will be described with reference to the flowchart illustrated in FIG. 5B. In the following description, as an example, it is assumed that the communication processing management unit 203 executes a communication algorithm.

  When the communication processing management unit 203 executes the communication algorithm, the number of rounds of data transfer processing is designated. When the data transfer process is started, for example, “1” may be set as the number of rounds.

  As a result of confirming the number of rounds in the data transfer process (step S501), when the number of rounds is “1”, the communication algorithm executes a process for issuing a data transfer request in the first round (first round) (step S502). ). “2” is set as the number of rounds (step S503). The communication algorithm returns (provides) a step (number of rounds) at the time of resuming the processing to the execution source of the communication algorithm (step S513).

  As a result of confirming the number of rounds in the data transfer process (step S501), when the number of rounds is “i” (i is an integer not less than 2 and less than N), the communication algorithm is the data transfer in the “i−1” -th round. A completion process is executed (step S504).

  If the data transfer of the “i−1” -th round has not been completed (NO in step S505), the communication algorithm sets step (round) when restarting the process to “i” (step S506). In other words, a step for restarting the process is set so that the process is restarted from the “i” -th round. In this case, the communication algorithm does not wait for completion of the data transfer of the “i−1” -th round, and proceeds to step S513.

  When the data transfer for the “i−1” -th round is completed (YES in step S505), the communication algorithm issues a data transfer request for the “i” -th round (step S507). The communication algorithm sets “i + 1” as a step (round) when the process is resumed (step S508). That is, a step for resuming the process is set so that the process is resumed from the “i + 1” -th round (next round).

  After executing the process of step S506 or step S508, the communication algorithm returns (provides) a step (number of rounds) when the process is resumed to the execution source of the communication algorithm (step S513).

  As a result of confirming the number of rounds in the data transfer process (step S501), when the number of rounds is “N” (the last round), the communication algorithm executes a data transfer completion process in the “M” -th round ( Step S509). The “M” th round represents, for example, the previous round of the “N” th round.

  If the data transfer of the “M” -th round has not been completed (NO in step S510), the communication algorithm sets “N” as the step (round) when the process is resumed (step S511). In this case, the communication algorithm does not wait for completion of the data transfer of the “M” -th round, and proceeds to step S513.

  When the data transfer of the “M” -th round is completed (YES in step S510), the communication algorithm sends information indicating the completion of the data transfer process to the execution source of the communication algorithm (for example, “algorithm completion” in FIG. 5A). Is provided (step S512).

  FIG. 6A is an explanatory diagram illustrating an example of data recorded in the communication processing recording unit 204.

  The communication ID 601 indicates an identifier (ID: Identifier) that can identify inter-process communication. In the communication ID 601, for example, data representing an identifier that can specify inter-process communication may be set.

  A communication algorithm 602 indicates a communication algorithm applied to inter-process communication. As a communication algorithm applied to the inter-process communication, a communication algorithm that performs communication processing in one or more rounds (stages) may be appropriately selected. For example, the communication algorithm may be a butterfly method. The communication algorithm may be a method in which each process transmits / receives data to / from all other processes. Such a communication algorithm may be appropriately selected for each collective communication function in MPI, for example. For example, data that can specify a communication algorithm may be set in the communication algorithm 602. In the communication algorithm 602, various algorithms applied to inter-process communication can be set.

  The resume position 603 indicates a step (number of rounds) when the communication algorithm is resumed, or the completion of the communication algorithm. For example, data representing step (number of rounds) or data representing completion of the communication algorithm may be set in the resume position 603. The initial value of the resume position 603 may be set to “step 1”, for example. The setting of the initial value represents, for example, that the first data transfer request issuance process is executed in the communication algorithm shown in FIG. 5A. When “algorithm completion” is set at the resume position 603, for example, it indicates that the communication algorithm is completed (data transfer processing is completed). Hereinafter, the combination of the communication ID 601, the communication algorithm 602, and the resume position 603 recorded in the communication processing recording unit 204 may be simply referred to as “entry”.

  FIG. 6B is an explanatory diagram illustrating an example of data recorded in the synchronization waiting thread recording unit 302. The synchronization waiting thread recording unit 302 holds a counter 604 that represents the number of threads that are not in a synchronization waiting state among one or more threads that execute certain parallel processing. For example, when parallel processing is executed by one or more threads, the total number of threads that execute the parallel processing is set in the counter 604. As one specific example, when parallel processing of a thread is implemented using OMP, the total number of threads that execute the parallel processing is displayed in the counter 604 at the timing when parallel processing is started, such as “#pragma om parallel”. May be set.

  Each thread gradually decreases (decrements) the value of the counter 604 when the synchronization processing is executed (that is, when the thread enters a synchronization waiting state). As one specific example, when parallel processing is implemented using OMP, each thread decrements the counter 604 at the start of execution of barrier synchronization processing such as “#pragma om barrier”.

  Each thread is in a synchronization waiting state until the counter 604 reaches “0”. Each thread may continue to wait in a loop process until the counter 604 becomes “0”, for example. In this case, each thread may exit the loop when the counter 604 reaches “0”, and the value of the counter 604 may be initialized with the total number of threads that execute parallel processing again.

  The parallel processing device 400 configured as described above may be able to realize the same processing as the parallel processing device 100 in the first embodiment. The inter-process communication unit 200 in the present embodiment may be able to realize the same processing as the inter-process communication unit 101 in the first embodiment. The multi-thread control unit 300 according to the present embodiment may be able to realize the same processing as the multi-thread control unit 102 according to the first embodiment.

[Operation]
Hereinafter, the operation of the parallel processing device 400 in the present embodiment will be described. FIG. 7 is a flowchart showing an example of the operation of the communication start processing unit 201. FIG. 8A is a flowchart showing an example of the operation of the communication processing management unit 203. FIG. 9A is a flowchart illustrating an example of the operation of the barrier synchronization processing unit 301. FIG. 10 is a flowchart showing an example of the operation of the communication completion processing unit 202.

  The parallel processing device 400 in the present embodiment executes a parallel program as in the first embodiment. Such a parallel program may include one or more processes including processes that can be executed in parallel by one or more threads. That is, the parallel processing program executed in the parallel processing device 400 may be a program in which multi-process / multi-thread processing is implemented.

  When such a parallel program is executed, for example, a plurality of processes may be generated and executed in parallel. In this case, each process may be executed by the same parallel processing device 400 or may be executed in parallel by a plurality of parallel processing devices 400. The plurality of processes execute processing implemented as a parallel program by executing inter-process communication as necessary. At this time, in the process of executing a certain process, a plurality of threads may be generated and the plurality of threads may be executed in parallel. When parallel processing by a plurality of threads starts, the total number of threads that execute the parallel processing may be set in the counter 604 of the synchronization waiting thread recording unit 302.

  The operation when the parallel program starts interprocess communication will be described below. In the following description, it is assumed that the parallel processing program executes various processes using functions provided by the communication start processing unit 201, the communication completion processing unit 202, the communication processing management unit 203, and the like. For example, when the communication start processing unit 201, the communication completion processing unit 202, and the communication processing management unit 203 are realized as a computer program, the parallel processing program calls the computer program (for example, a library or an execution file) ( Can be executed).

  For example, the communication start processing unit 201 may provide a function of recording information related to inter-process communication in the communication processing recording unit 204. Specifically, for example, the communication start processing unit 201 may set appropriate information for the communication ID 601, the communication algorithm 602, and the restart position 603 in the communication processing recording unit 204 for a certain inter-process communication. For example, the communication start processing unit 201 may provide a function of starting the inter-process communication by notifying the communication processing management unit 203 to execute the communication processing.

  When the parallel program starts inter-process communication, the communication start processing unit 201 records information related to the inter-process communication in the communication processing recording unit 204 (step S701). In this case, for example, a thread generated in the parallel program may use the communication start processing unit 201 to record information related to inter-process communication in the communication processing recording unit 204. Such processing can be executed exclusively between threads. In this case, the communication start processing unit 201 may set the initial value “step 1” as the restart position 603 as information indicating the restart position.

  The communication start processing unit 201 starts communication processing (step S702). For example, the communication start processing unit 201 may control the communication processing management unit 203 to start processing related to inter-process communication (inter-process communication processing).

  As a specific example, in the case of the pseudo code as shown in FIG. 2, the parallel program may execute the above-described process for starting the inter-process communication when the MPI_Iallreduce function is executed, for example. Also, when “#pragma om parallel” (parallel processing start processing by a plurality of threads) is executed, the total number of threads that execute parallel processing is set in the counter 604 of the synchronization waiting thread recording unit 302.

  Hereinafter, the inter-process communication process will be specifically described. A parallel processing program (in particular, a thread generated by the parallel program) may execute inter-process communication processing using a function provided by the communication processing management unit 203 described below. The processes in steps S801 to S805 described below can be executed exclusively between threads.

  The communication process management unit 203 determines whether there is an unprocessed communication process (step S801). Specifically, the communication processing management unit 203 refers to the communication processing recording unit 204, and when information (entry) regarding inter-process communication is registered in the communication processing recording unit 204, there is unprocessed inter-process communication. May be determined.

  If the result of determination in step S801 is that there is no unprocessed interprocess communication (NO in step S801), the communication process management unit 203 may end the process.

  If there is unprocessed interprocess communication as a result of the determination in step S801 (YES in step S801), the communication processing management unit 203 sets the entry (communication ID 601, communication algorithm 602, and restart) set in the communication processing recording unit 204. Position 603) is acquired (step S802).

  The communication processing management unit 203 confirms the restart position 603 related to the communication ID 601 acquired in step S802. When data representing “algorithm completion” is set at the resume position 603 (NO in step S803), the communication process management unit 203 continues the process from step S801.

  When data other than “algorithm completion” is set at the resume position 603 (YES in step S803), the communication processing management unit 203 executes a communication algorithm related to inter-process communication identified by the communication ID 601 (step S804). ). Specifically, the communication process management unit 203 checks the data set at the resume position 603 for each communication ID 601 set in the communication process recording unit 204. If the data set at the resume position 603 is not “algorithm complete”, the communication processing management unit 203 executes the communication algorithm for the interprocess communication identified by the data set in the communication ID 601. At this time, the communication processing management unit 203 provides data (representing step) set at the resume position 603 to the communication algorithm.

  The communication processing management unit 203 updates the resume position 603 using the data returned (provided) from the communication algorithm (step S805). As described above, data representing a step (number of rounds) of inter-process communication may be provided from the communication algorithm. From the communication algorithm, data indicating that the inter-process communication is completed (for example, data indicating “algorithm completion”) may be provided.

  Hereinafter, processing when the parallel program executes thread barrier synchronization processing will be described. A parallel processing program (specifically, a thread generated in the parallel program) may execute thread barrier synchronization processing using the function of the barrier synchronization processing unit 301 described below. For example, when the barrier synchronization processing unit 301 is realized as a computer program, a thread generated in the parallel processing program can call (execute) the computer program.

  For example, the barrier synchronization processing unit 301 may provide a function of acquiring a thread state, a function of setting a thread state, and a function of deleting a thread state. For example, the barrier synchronization processing unit 301 may provide a function of synchronizing threads (function of waiting for thread synchronization). For example, the barrier synchronization processing unit 301 may provide a function for causing the communication processing management unit 203 to execute communication processing.

  When the parallel program executes thread barrier synchronization processing, the barrier synchronization processing unit 301 records in the synchronization waiting thread recording unit 302 that the thread is in a synchronization waiting state (step S901). For example, a certain thread may use the barrier synchronization processing unit 301 to decrement ("1" gradually) the value of the counter 604 recorded in the synchronization waiting thread recording unit 302. As one specific example, when thread parallel processing is implemented using OMP, the thread barrier synchronization processing may be executed when a code such as “#pragma om barrier” is executed. Good.

  The thread that has entered the synchronization waiting state acquires the state of another thread (step S902). Such a thread may acquire the state of another thread using the barrier synchronization processing unit 301. For example, the thread may refer to the value 604 recorded in the synchronization waiting thread recording unit 302 and determine the state of another thread based on the value. Such a reference process can be executed exclusively, for example. Hereinafter, a thread that is in a synchronization waiting state may be referred to as a “synchronization waiting thread” (corresponding to a first thread).

  A case will be described below in which it is determined from the state of another thread acquired in step S902 that there is a thread that is not in a synchronization waiting state among threads participating in synchronization related to certain parallel processing (NO in step S903). . For example, when the value of 604 recorded in the synchronization waiting thread recording unit 302 is not “0”, the synchronization waiting thread may determine that a thread that is not in the synchronization waiting state remains for a certain parallel process.

  Here, the thread participating in the synchronization may be, for example, a thread that waits for the completion of the process by another thread after the process in the own thread is completed in a certain parallel process. For example, when one or more threads participate in synchronization, they may wait for execution of processing by other threads at a certain synchronization point (timing).

  In the case of NO in step S903, the synchronization waiting thread confirms whether or not its own thread is the thread that first entered the synchronization waiting state (step S904). For example, the thread may determine whether or not the own thread is a thread that is initially waiting for synchronization based on the value of 604 recorded in the thread recording unit 302. As an example, if the value of 604 recorded in the thread recording unit 302 is not decremented by another thread, the synchronization waiting thread may determine that the own thread is the thread that first waited for synchronization.

  The case where the synchronization waiting thread is the thread that first entered the synchronization waiting state (YES in step S904) will be described below. The synchronization waiting thread repeatedly executes communication processing (for example, steps S801 to S805 described above) by the communication processing management unit 203 until another thread enters a synchronization waiting state (step S905). The synchronization waiting thread may control the communication processing management unit 203 to execute the communication processing using the barrier synchronization processing unit 301. The synchronization waiting thread may repeatedly execute the communication process by the communication process management unit 203 at time intervals that can be appropriately set.

  If the synchronization waiting thread is not the thread that first entered the synchronization waiting state (NO in step S904), the synchronization waiting thread continues processing from step S902 and waits until another thread enters the synchronization waiting state.

  A case where the synchronization waiting thread confirms that there is no other thread that is not in the synchronization waiting state for a certain parallel processing (YES in step S903) will be described. In this case, the synchronization waiting thread ends the synchronization waiting process related to a certain parallel process. Then, the synchronization waiting thread waits for another thread to finish the synchronization waiting process related to a certain parallel process (step S906).

  The synchronization process in step S906 is a process in which each thread executing a certain parallel process waits until it is confirmed that another thread has entered a synchronization wait state. As a result, for example, it is possible to prevent another thread from continuing processing before a certain thread enters the synchronization waiting state and changing the counter 604 recorded in the synchronization waiting thread recording unit 302.

  If a thread is a thread that has been in a synchronization waiting state first (YES in step S907), the thread initializes the state relating to all threads that have participated in synchronization recorded in the synchronization waiting thread recording unit 302. (Step S908). For example, such a thread may execute a process of reinitializing the counter recorded in the synchronization waiting thread recording unit 302 with the total number of threads executing parallel processing by using the barrier synchronization processing unit 301.

  For convenience of explanation, FIG. 9A shows a mode in which a thread that is initially in a synchronization waiting state executes step S908, but the present embodiment is not limited to this. Step S908 may be executed by any thread other than the thread that first entered the synchronization wait state.

  If NO in step S907, or if the process in step S908 is executed, the synchronization waiting thread may wait for another thread again (step S909). Thereby, for example, it is possible to prevent a thread from continuing processing before the counter 604 recorded in the synchronization waiting thread recording unit 302 is initialized.

  The operation when the parallel program completes the interprocess communication will be described below. In the following description, it is assumed that the parallel processing program executes various processes using functions provided by the communication completion processing unit 202 and the like. For example, when the communication completion processing unit 202 is realized as a computer program (for example, a library or an execution file), the parallel processing program can call (execute) the computer program.

  When the parallel program completes the inter-process communication, the communication completion processing unit 202 refers to the communication processing recording unit 204 and acquires data (resumption position 603 in FIG. 6A) indicating the resumption position of the inter-process communication related to the parallel program. (Step S1001).

  The communication completion processing unit 202 determines whether or not the interprocess communication has been completed, using data representing the restart position of the interprocess communication. Specifically, the communication completion processing unit 202 determines whether or not “algorithm completion” is set in the data indicating the restart position of the interprocess communication. (Step S1002).

  As described above, when the inter-process communication (data transfer process) is completed, “algorithm complete” is set at the restart position of the inter-process communication (for example, 603 in FIG. 6). When inter-process communication (data transfer processing) has not been completed, for example, the number of steps (number of rounds) is set as the restart position of inter-process communication.

  When the restart position of the interprocess communication is not “algorithm completion” (NO in step S1002), the interprocess communication regarding the parallel program is not completed. In this case, the communication completion processing unit 202 executes inter-process communication processing by the communication processing management unit 203 (step S1003), and continues processing from step S1001. That is, the communication completion processing unit 202 repeatedly executes the interprocess communication processing by the communication processing management unit 203 until “algorithm completion” is set at the restart position of the interprocess communication. The communication completion processing unit 202, for example, the communication processing management unit 203 may be controlled to execute the inter-process communication processing.

  When the restart position of the interprocess communication is “algorithm complete” (YES in step S1002), the interprocess communication regarding the parallel program is completed. In this case, the communication completion processing unit 202 deletes the entry related to the interprocess communication recorded in the communication processing management unit 203 (step S1004). The process in step S1004 can be executed exclusively between threads.

  As one specific example, the parallel program may execute the above-described process of starting the interprocess communication when the MPI_Wait function is executed.

[effect]
For example, when a thread generated in a parallel program simply executes a synchronization process, the thread may be in a synchronization wait state waiting for the synchronization of another thread. In this case, the CPU core does not execute arithmetic processing related to the thread in the synchronization waiting state. That is, when the thread enters a synchronization waiting state, the CPU core's computing resources are consumed without being effectively utilized.

  On the other hand, in the parallel processing device 400 configured as described above, the threads of the parallel program can execute interprocess communication while waiting for synchronization of other threads after executing the synchronization processing. Yes (for example, steps S902 to S905).

  That is, the parallel processing device 400 can proceed with the inter-process communication process by using the computation resource of the CPU core that has been consumed while waiting for thread synchronization. Thereby, it is possible to proceed with the inter-process communication process without reducing the calculation resources of the CPU core assigned to the calculation process.

  In the parallel processing device 400 configured as described above, it is not necessary to generate an independent communication thread. Further, in the parallel processing device 400, the user does not have to explicitly describe the process related to the interprocess communication process in the middle of the calculation process in the parallel program, for example. This is because, when a thread enters a synchronization waiting state, the inter-process communication unit 200 and the multi-thread control unit 300 execute a process that advances inter-process communication. This makes it possible to execute data transfer processing and calculation processing in inter-process communication in parallel.

  The above effects will be described with reference to the explanatory drawings. FIG. 11 is an explanatory diagram for explaining the effect when, for example, the program illustrated in FIG. 2 is executed in the parallel processing device 400 of this embodiment.

  Compared to the chart illustrated in FIG. 3, in the chart of FIG. 11, the inter-process communication is performed during the barrier synchronization processing of the thread executed after the calculation processing of “para1 ()” and “para2 ()”. Processing is executed. As a result, the parallel processing device 400 can sequentially proceed with the following inter-process communication processing using the computation resources of the CPU core in which the threads are in a synchronization waiting state (waiting for thread synchronization). That is, the parallel processing device 400 specifically includes a round 1 data transfer completion process (S1101), a round 2 data transfer request issuance process (S1102), a round 2 data transfer completion process (S1103), Round 3 data transfer request issuance processing (S1104) can be executed in sequence. As a result, the data transfer process of round 2 and round 3 is executed in parallel with the calculation process, so that the execution time of the entire parallel program is shortened.

  As described above, according to the present embodiment, the inter-process communication process and the calculation process can be executed in parallel using the operation resources consumed in the process of the calculation process in the parallel program.

  Hereinafter, modifications and the like related to the above embodiments will be described.

  In the above-described embodiment, the processes in steps S904 to S905 are executed by the thread that first enters the synchronization wait state. As a result, the inter-process communication process can proceed at the earliest possible timing after the generation of the synchronization waiting thread.

  As a modification of the present embodiment, the communication process (including steps S801 to S805 in FIG. 8A) in step S905 in FIG. 9A may be executed by an appropriate thread other than the thread that first entered the synchronization wait state. Good. For example, the parallel processing device 400 may hold data (flag) indicating the presence / absence of a thread executing a communication process. In this case, for example, when a thread enters a synchronization waiting state, the thread checks the flag. If there is no other thread executing the communication process, the thread itself may execute the communication process. Specifically, such a modification can be realized as follows. Hereinafter, a description will be given with reference to FIGS. 8B and 9B.

  In this modification, for example, information indicating whether or not a communication process related to the communication ID is being executed is set in the communication process recording unit 204 in association with each communication ID (601 in FIG. 6A). Hereinafter, such information is referred to as “processing flag”. The in-process flag assumes that each thread is exclusively accessed from, for example. That is, it is assumed that only one thread can refer to, update, or delete the in-process flag at a certain time.

  In step S701 in FIG. 7, the communication start processing unit 201 initializes a processing flag relating to a certain inter-process communication. Specifically, the communication start processing unit 201 may set information (for example, “OFF” or the like) indicating that the communication process is not being executed in the processing flag.

  When a certain thread executes the synchronization process, the thread has, for example, the barrier synchronization processing unit 301 and executes steps S901 to S905 in FIG. 9B. At this time, the thread does not need to execute whether or not the self thread is the thread that first entered the synchronization waiting state (step S904 in FIG. 9A).

  The thread executes step S801 of FIG. 8B using, for example, the communication processing management unit 203.

  When there is an unprocessed inter-process communication process (YES in step S801), the thread confirms a processing flag relating to the inter-process communication process (step S806 in FIG. 8B). Specifically, the thread checks the in-process flag associated with the communication ID related to unprocessed interprocess communication in the communication processing recording unit 204.

  When the in-process flag is OFF, there is no other thread executing the communication process. In this case (YES in step S806), the thread sets information (for example, “ON” or the like) indicating that the communication process is being executed in the in-process flag (step S807 in FIG. 8B). Here, the confirmation and update of the processing flag in step S806 can be executed exclusively (atomically).

  After step S807, the thread continues processing from step S802. The processing from step S802 to step S805 may be the same as in the above-described embodiment.

  In step S803, NO (that is, when the communication algorithm is completed), or after the processing in step S805 is executed, the thread sets “OFF” in the processing flag (step S808).

  In step S806, if the processing flag is not OFF (ON), there is a possibility that another thread is executing the communication process. In this case, the thread continues processing from step S801.

  Through the processing as described above, the communication processing illustrated in FIG. 8B is not limited to a specific thread (for example, a thread that is first in a synchronization waiting state), and is executed by any one of a plurality of threads. obtain.

  In the above embodiment, until synchronization is established (for example, until all threads participating in synchronization are in a synchronization waiting state), the barrier synchronization processing unit 301 executes processing related to inter-process communication using the communication processing management unit 203 To do. As a result, the barrier synchronization processing unit 301 may require a little more time for barrier synchronization than when the barrier synchronization processing unit 301 does not execute processing related to inter-process communication. When priority is given to the execution time of barrier synchronization, a reference time until synchronization is established may be appropriately determined, and the barrier synchronization processing unit 301 may advance inter-process communication processing within a range that does not exceed the reference time. The reference time may be, for example, a time that has elapsed since the first synchronization process was executed by a certain thread. For example, the reference time may be determined so as not to exceed the time required for establishing synchronization by recording the time required until establishment of previous synchronization and the time required for progress of communication processing. Alternatively, the reference time may be experimentally determined, for example, when testing a parallel program. The method for determining the reference time is not limited to the above, and other appropriate methods may be adopted.

  In the description of the above embodiment, the barrier synchronization processing unit 301 advances the inter-process communication process when the thread executes the barrier synchronization process. The present invention is not limited to the above, and the above-described embodiment can also be applied to a function (for example, a lock acquisition function) that causes thread waiting other than barrier synchronization.

  In each of the above embodiments, the present invention has been described as an example in which the present invention is applied to a parallel processing device (100, 400). In each said embodiment, a parallel processing method can be implemented by operating a parallel processing apparatus (100,400), for example. The method of executing the parallel processing method is not limited to the above, and is executed by another device (for example, an information processing device such as a computer) capable of executing the same operation or processing as the parallel processing device (100, 400). Is also possible.

<Configuration of hardware and software program (computer program)>
Hereinafter, a hardware configuration capable of realizing each of the above-described embodiments will be described.

  In the following description, the parallel processing devices (100, 400) described in the above embodiments are collectively referred to simply as “parallel processing devices”. In addition, each component of the parallel processing device may be simply referred to as “component of the parallel processing device”.

  The parallel processing device described in each of the above embodiments may be configured by one or a plurality of dedicated hardware devices. In that case, each component shown in each of the above figures (FIGS. 4A and 4B) is realized by using hardware (an integrated circuit or a storage device on which processing logic is mounted) that is partially or entirely integrated. Also good.

  When the parallel processing device is realized by dedicated hardware, the components of the parallel processing device may be realized by, for example, a circuit configuration capable of providing each function. Such a circuit configuration includes, for example, an integrated circuit such as SoC (System on a Chip), a chip set realized using the integrated circuit, and the like. In this case, the data held by the components of the parallel processing device is, for example, a RAM (Random Access Memory) area integrated as SoC, a flash memory area, or a storage device (such as a semiconductor storage device) connected to the SoC. May be stored. In this case, a well-known communication network may be adopted as a communication line for connecting each component of the parallel processing device. Further, the communication line connecting each component may be connected between each component by peer-to-peer.

  Further, the parallel processing device described above may be configured by general-purpose hardware as exemplified in FIG. 12 and various software programs (computer programs) executed by the hardware. In this case, the parallel processing device may be configured by a combination of an appropriate number of general-purpose hardware devices 1200 and software programs.

  An arithmetic device 1201 in FIG. 12 is an arithmetic processing device such as a general-purpose CPU (Central Processing Unit) or a microprocessor. The arithmetic device 1201 may read various software programs stored in a nonvolatile storage device 1203, which will be described later, into the storage device 1202, and execute processing according to the software programs. For example, the functions of the components of the parallel processing device in each of the above embodiments may be realized using a software program executed by the arithmetic device 1201.

  The arithmetic device 1201 may be, for example, a multi-core CPU including a plurality of CPU cores. The arithmetic device 1201 may be a multi-thread CPU in which one CPU core can execute a plurality of threads. The hardware device 1200 may include a plurality of arithmetic devices 1201.

  The parallel programs described in the above embodiments can execute processes in parallel using a plurality of arithmetic devices 1201. In this case, the parallel program may be executed by a plurality of arithmetic devices 1201 included in one hardware device 1200. The parallel program may be executed by one or more arithmetic devices 1201 included in the plurality of hardware devices 1200. The threads generated by the parallel program are executed in parallel using, for example, a CPU core in the arithmetic device 1201.

  The storage device 1202 is a memory device such as a RAM that can be referred to from the arithmetic device 1201, and stores software programs, various data, and the like. Note that the storage device 1202 may be a volatile memory device.

  The nonvolatile storage device 1203 is a nonvolatile storage device such as a magnetic disk drive or a semiconductor storage device using flash memory. The nonvolatile storage device 1203 can store various software programs, data, and the like. For example, the parallel program may be stored in the nonvolatile storage device 1203.

  The drive device 1204 is, for example, a device that processes reading and writing of data with respect to a recording medium 1205 described later.

  The recording medium 1205 is an arbitrary recording medium capable of recording data, such as an optical disk, a magneto-optical disk, and a semiconductor flash memory. The parallel program may be recorded on the recording medium 1205.

  The input / output interface 1206 is a device that controls input / output with an external device.

  The parallel processing device according to the present invention described with the above-described embodiments as an example, or a component thereof, for example, can implement the functions described in the above-described embodiments with respect to the hardware device 1200 illustrated in FIG. It may be realized by supplying a software program.

  More specifically, for example, the present invention may be realized by the arithmetic device 1201 executing a software program supplied to the hardware device 1200. In this case, an operating system running on the hardware device 1200, middleware such as database management software, network software, and virtual environment infrastructure may execute a part of each process.

  In each embodiment described above, each unit illustrated in each of the above drawings can be realized as a software module, which is a function (processing) unit of a software program executed by the hardware described above. However, the division of each software module shown in these drawings is a configuration for convenience of explanation, and various configurations can be assumed for implementation.

  When the respective components of the parallel processing device illustrated in FIGS. 4A and 4B are realized as software modules, for example, these software modules may be stored in the nonvolatile storage device 1203. These software modules may be read out to the storage device 1202 when the arithmetic device 1201 executes each process.

  In addition, these software modules may be configured to transmit various data to each other by an appropriate method such as shared memory or inter-process communication. With such a configuration, these software modules are connected so as to communicate with each other.

  Further, the software program may be recorded on the recording medium 1205. In this case, the software program may be configured to be stored in the nonvolatile storage device 1203 through the drive device 1204 as appropriate at the time of shipment or operation of the parallel processing device.

  In the above case, the method of supplying various software programs to the hardware device 1200 is carried out in the device using an appropriate jig in the manufacturing stage before shipment or the maintenance stage after shipment. An installation method may be adopted. As a method for supplying various software programs, a general procedure may be adopted at present, such as a method of downloading from the outside via a communication line such as the Internet.

  In such a case, the present invention can be considered to be configured by a code that constitutes the software program or a computer-readable recording medium on which the code is recorded. In this case, the recording medium is not limited to a medium independent of the hardware device 1200 but includes a recording medium in which a software program transmitted via a LAN, the Internet, or the like is downloaded and stored or temporarily stored.

  Further, the components of the parallel processing device described above are configured by a virtualized environment in which the hardware device 1200 illustrated in FIG. 12 is virtualized and various software programs (computer programs) executed in the virtualized environment. May be. In this case, the components of the hardware device 1200 illustrated in FIG. 12 are provided as virtual devices in the virtual environment. In this case as well, the present invention can be realized with the same configuration as when the hardware device 1200 illustrated in FIG. 12 is configured as a physical device.

  In the above, this invention was demonstrated as an example applied to exemplary embodiment mentioned above. However, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to such embodiments. In such a case, new embodiments to which such changes or improvements are added can also be included in the technical scope of the present invention. Embodiments combining the above-described embodiments are also included in the technical scope of the present invention. Furthermore, embodiments combining the above-described embodiments and new embodiments obtained by changing or improving the above-described embodiments may also be included in the technical scope of the present invention. This is clear from the matters described in the claims.

DESCRIPTION OF SYMBOLS 100 Parallel processing apparatus 101 Interprocess communication part 102 Multithread control part 200 Interprocess communication part 201 Communication start process part 202 Communication completion process part 203 Communication process management part 204 Communication process recording part 300 Multithread control part 301 Barrier synchronous process part 302 Synchronization waiting thread recording unit 1201 arithmetic device 1202 storage device 1203 nonvolatile storage device 1204 drive device 1205 recording medium 1206 input / output interface

Claims (10)

An interprocess communication unit capable of controlling interprocess communication including at least data transfer processing, which is executed between a plurality of processes in a parallel program;
Among the plurality of threads generated in the execution process of the parallel program, the first thread and the other of the plurality of threads after the synchronization processing is executed in the first thread. A parallel processing device comprising: a multi-thread control unit that advances the inter-process communication by the first thread until one or more second threads are synchronized. The multi-thread controller is
When synchronization processing is executed in the first thread and the first thread enters a synchronization waiting state, whether or not each of the one or more second threads is in a synchronization waiting state is determined. And when any one or more of the second threads is not in a synchronization waiting state, the first thread executes a process related to the inter-process communication using a function provided by the inter-process communication unit. The parallel processing apparatus according to claim 1. The first thread is a thread that is first in a synchronization waiting state among the plurality of generated threads,
The multi-thread control unit performs the inter-process communication by the first thread between the time when the first thread is in a synchronization waiting state and the time when all the second threads are in a synchronization waiting state. The parallel processing apparatus according to claim 2, wherein a process that advances the interprocess communication is executed using a function provided by a unit. The inter-process communication includes one or more stages of the data transfer process having an order relationship,
The inter-process communication unit can control the progress of the inter-process communication by advancing the stage of the data transfer process according to the order relationship,
4. The parallel processing according to claim 2, wherein the multi-thread control unit executes a process of progressing the inter-process communication by using the function provided by the inter-process communication unit by the first thread. 5. apparatus. The inter-process communication includes, for each stage of the inter-process communication, at least one of a completion process that completes the data transfer of the previous stage and a process that executes the data transfer in each stage,
The inter-process communication unit confirms whether or not the data transfer process of the previous stage is completed in the completion process,
When the data transfer process of the previous stage is completed, the data transfer process in the stage is executed and the inter-process communication stage is set to the next stage,
If the data transfer process in the previous stage is not completed, the completion process is interrupted without waiting for the completion of the data transfer process in the previous stage, and the data transfer process is completed. The parallel processing device according to claim 4, wherein the parallel processing device provides information indicating a stage of the inter-process communication that is not performed. The inter-process communication unit is
A communication processing recording unit that holds information that can identify the inter-process communication and information that represents a stage of the inter-process communication;
When the completion process is interrupted without waiting for the completion of the data transfer, information indicating the stage of the inter-process communication in which the data transfer process is not completed is set in the communication process recording unit,
When processing related to the inter-process communication is executed by the first thread, using the information indicating the stage of the inter-process communication set in the communication processing recording unit, from the stage represented by the information, The parallel processing device according to claim 5, wherein the processing for performing interprocess communication is executed. The multi-thread control unit is configured such that the first thread enters a synchronization wait state after all the second threads enter a synchronization wait state after the first thread enters a synchronization wait state. 3. The process for advancing the inter-process communication is executed by the first thread using a function provided by the inter-process communication unit within a range that does not exceed a reference time. Item 4. The parallel processing device according to Item 3. One or more circuit elements including one or more arithmetic units;
A storage device that provides a storage area that can be referred to from the circuit element,
The circuit element is:
A function for controlling inter-process communication including at least data transfer processing executed between a plurality of processes in a parallel program;
Among the plurality of threads generated in the execution process of the parallel program, the first thread and the other of the plurality of threads after the synchronization processing is executed in the first thread. A parallel processing device having a configuration capable of executing a function for executing processing related to the inter-process communication until one or more second threads are synchronized. Controlling inter-process communication including at least data transfer processing executed between a plurality of processes in the parallel program in response to execution of the parallel program;
Among the plurality of threads generated in the execution process of the parallel program, the first thread is the other thread among the first thread and the plurality of threads after the synchronization process is executed in the first thread. A parallel processing method in which the inter-process communication proceeds by the first thread before the above second thread is synchronized. In a computer equipped with an arithmetic processing unit capable of processing a plurality of threads in parallel,
A process for controlling inter-process communication including at least a data transfer process, which is executed between a plurality of processes in the parallel program according to the execution of the parallel program;
Among the plurality of threads generated in the execution process of the parallel program, the first thread is the other thread among the first thread and the plurality of threads after the synchronization process is executed in the first thread. A computer program for executing the process of progressing the inter-process communication by the first thread until the above second thread is synchronized.

Images