CN103744643A - Method and device for structuring a plurality of nodes parallel under multithreaded program - Google Patents

Abstract

The invention discloses a method and a device for structuring a plurality of nodes parallel under a multithreaded program. The method comprises the steps of creating a master process and a slave process in a master function of an original program, and obtaining information which represents the master process and the slave process respectively; performing an operation of retaining a thread of the original program in the master function of the master process, developing thread and binding a thread function; deleting an original code in a hotspot thread function of the master process and adding a code which communicates with a subprocess; and deleting content of master function of the original program in the master function of the subprocess and adding a hotspot thread function calculating part of the original program and a part which communicates with the master process. According to the method and the device, the thread communication is added under a multithreaded version program framework, so that parallel structuring of a plurality of nodes under a multithreaded framework is realized, scalability and performance of the program are improved and a calculation resource among the nodes is fully utilized.

Description

Method and device for multi-node parallel architecture under multi-thread program

technical field

The invention relates to a computer software optimization technology, in particular to a method and a device of a multi-node parallel architecture under a multi-thread program.

Background technique

In a broad sense, parallel computing refers to the simultaneous execution of multiple computing tasks in one program, which is usually used in occasions with extremely high performance requirements, such as scientific computing fields such as weather forecasting and oil exploration. Parallel computing can make full use of CPU computing resources, so it is more and more widely used.

There are usually two ways to implement parallel computing: multi-process and multi-thread. Multi-threaded programs can only be used for parallel computing on a single node (a node refers to a computing device in the network). The advantage lies in the simple communication between threads and low thread overhead. Multi-process programs can be used for multi-node parallel computing, but the communication between nodes needs to be realized through the network, which has a large overhead.

Usually, the design method combining the two is that the framework adopts a multi-process method, and the internal process uses a multi-thread method to make full use of multi-nodes and CPU computing resources in the nodes.

Multi-process programs are usually developed using the Message Passing Interface (MPI, Message Passing Interface). MPI is a general-purpose industrial-grade inter-process communication interface, which can easily communicate between processes through message passing, and provides a convenient cluster multi-process operation mode.

Multi-threaded programs can use a variety of methods for inter-thread parallel computing, such as: OpenMP, pThread, etc. OpenMP simply modifies the source program in the form of compilation instruction statements, and can use the multi-thread library for automatic parallelism, hiding many details. The bottom layer of the OpenMP library is implemented by calling pThread. The pThread library is the underlying thread library under Linux, and there is also a corresponding thread application program interface (API, Application Program Interface) under Windows. Using the pThread library can manually control the thread operation mode, providing more fine-grained control.

However, if the original computing program is a multi-threaded program, in order to make full use of multi-node computing resources, it must be transformed into a multi-process program. The current method is to substantially rewrite the code and change the multi-threaded program framework to a multi-process program framework, which is time-consuming and laborious. At present, there is no method to quickly transform a multi-threaded program into a multi-threaded program framework, and the calculation part adopts a multi-process architecture to make full use of the computing resources between nodes.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for multi-node parallel architecture under the multi-thread program, which can quickly transform the multi-thread program into a multi-process computing architecture under the multi-thread framework.

In order to solve the above-mentioned technical problems, the present invention provides a method of multi-node parallel architecture under multi-thread program, including:

Create master and slave processes in the main function of the original program, and obtain information representing the master and slave processes respectively;

In the main function of the main process, the thread development of the original program and the operation of the thread binding thread function are retained; in the hot thread function of the main process, the original calculation code is deleted, and the code for communicating with the sub-process is added; in the main process of the sub-process In the function, the content of the main function of the original program is deleted, and the hot thread function calculation part and the communication part with the main process of the original program are added.

Further, before the method step, it also includes:

According to the original program, the hot thread function is measured, and through analysis, it is determined that the hot thread function can be optimized into a multi-process parallel architecture.

Further, create master and slave processes in the main function of the original program, and obtain information representing the master and slave processes respectively; specifically include:

In the main function, use the message passing interface initialization function to create the master and slave processes, and obtain the corresponding process numbers through the message passing interface to obtain the process number function, wherein the process number 0 represents the master process, and the remaining non-zero process numbers represent the slave processes.

Further, delete the original calculation code in the hot thread function of the main process, and add the code for communicating with the child process, specifically including:

When entering the hot thread function of the main process, the main process sends task-related information including task length and task data to the sub-process; when all information is sent, the main process enters a waiting state and waits for the sub-process to return the calculation result.

Further, delete the main function content of the original program in the main function of the sub-process, and add the hot thread function calculation part and the communication part with the main process of the original program, specifically including:

The child process waits for the main process to send the task length information, and allocates memory space according to the task length;

The sub-process waits to receive the task data information sent by the main process, and stores the received task data information into the allocated memory space; uses the sub-function or corresponding code called by the calculation part in the hot thread function of the original program to process the received task data Calculation; the calculation is completed and the calculation result is sent to the main process; repeat this step until the end of the program.

In order to solve the above technical problems, the present invention provides a multi-node parallel architecture device under a multi-threaded program, including a main function module and a main process computing module connected to each other, wherein:

The main function module is used to initialize the process parameters, run the corresponding branches for the main process and the slave process, receive the calculation data transmitted by the main process through multiple sub-processes and perform calculations, and send the calculation results to the main process;

The calculation module of the main process is used to transfer the data to be calculated to the sub-processes and receive the calculation results of the sub-processes.

Further, the main function module runs the process initialization function through the process creation module, initializes process parameters, including creating threads and binding thread functions; judges whether the process belongs to the main process or the slave process through the main function judgment module, and runs corresponding branches for different processes ; Through the sub-process calculation module, multiple sub-processes receive the calculation data transmitted by the main process, and respectively calculate the received data by calling the core calculation module, and send the calculation results to the main process respectively.

The invention realizes multi-node parallel architecture under the multi-thread framework by adding process communication under the existing multi-thread version program framework, improves program expansibility and performance, and thus fully utilizes inter-node computing resources.

Description of drawings

Fig. 1 is the flow chart of the method embodiment of multi-node parallel architecture under the multi-thread program of the present invention;

Fig. 2 is the device structure schematic diagram that existing original computing program is formed;

FIG. 3 is a schematic structural diagram of an embodiment of a device in which the device shown in FIG. 2 is transformed into a multi-node parallel architecture under a multi-threaded program of the present invention.

Detailed ways

The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and preferred embodiments. It should be understood that the following examples are only used to illustrate and explain the present invention, but not to limit the technical solution of the present invention.

As shown in Figure 1, it is the process flow of the method embodiment of the multi-node parallel architecture under the multi-thread program of the present invention, including the following steps:

Step S110, measure the hot thread function according to the original program, and determine through analysis that the hot thread function can be optimized into a multi-process parallel architecture;

Wherein, there may be one or more hotspot thread functions to be measured.

Step S120, creating master and slave processes in the main function of the original program, and obtaining information representing the master and slave processes respectively;

Among them, take MPI as an example to establish a process, that is, use the MPI initialization function in the main function to create the master and slave processes, and obtain the process numbers representing the master and slave processes respectively, that is, obtain the corresponding process numbers through the MPI function to obtain the process number, where the process Number 0 means the master process, and other non-zero process numbers mean slave processes. Therefore, the master process and the slave process can be determined by judging the process number, and different situations can be handled in different branches.

It should be noted that the way to establish a process is related to the way used by programmers. For example, taking MPI as an example, the master and slave processes use the same source code. For programmers, the process creation is completed in the initialization function; but the actual process creation operation is completed when the program starts. Since the present invention is aimed at programmers, the programmer’s perspective is adopted here, and the initialization function is considered as the operation of creating a process, that is, each process has a main function, and the corresponding initialization function will be created by executing the initialization function in the main function. process, but all processes use the same source code and perform different operations according to the process number.

Step S130, retaining the operations of thread development and thread binding thread function of the original program in the main function of the main process;

Step S140, delete the original calculation code in the hot thread function of the main process, and increase the code for communicating with the child process;

Among them, when entering the hot thread function of the main process, the main process sends task-related information such as task length and task data to the sub-process; when all information is sent, the main process enters a waiting state and waits for the sub-process to return the calculation result.

Step S150, delete the main function content of the original program in the main function of the sub-process, and add the hot thread function calculation part and the communication part with the main process of the original program.

Among them, the specific steps are:

The child process waits for the main process to send task length information, and allocates memory space according to the task length;

The child process waits to receive the task data information sent by the main process, and stores the received task data information into the allocated memory space; uses the sub-function or corresponding code called by the calculation part of the hot thread function of the original program to calculate the received task data ;Calculation ends and the calculation result is sent to the main process; repeat this step until the program ends.

Aiming at the above-mentioned method embodiment, the present invention also provides a device embodiment of a multi-node parallel architecture under a multi-thread program correspondingly, and its structure is formed by improving the composition structure of an existing original computing program.

As shown in Figure 2, it is a device structure composed of an existing typical multi-thread computing program, wherein:

The main function module 210 includes creating a thread module 220, creating a thread and binding a thread function by calling the creating thread module 220, and computing module 230;

Create a thread module 220, which is used to create a thread and bind a thread function;

Calculation module 230, including hotspot calculation function for realizing parallel calculation, is bound with thread as thread function;

In fact, whether it is a hotspot computing function or not, it can be bound to a thread to become a thread function. Here, the hotspot calculation function of the calculation module 230 is bound as a thread function.

It should be noted that there may be multiple thread functions, which are bound to different threads. In the following embodiments, only one thread function is modified, so other functions are not listed.

The core calculation module 240 is included in the calculation module 230, and is generally a loop body, in the form of completing a calculation work through code segments or sub-functions.

There is no sequence dependency between calling the core computing module 240 each time. That is, when calling the core computing module 240 multiple times, there is no sequence restriction.

The present invention constitutes the structure of the device embodiment shown in FIG. 3 by improving the device structure shown in FIG. 2 , including a main function module 310 and a main process calculation module 330 connected to each other, wherein:

The main function module 310 is used to initialize the process parameters, run corresponding branches for the main process and the slave process, receive and calculate the data involved in the calculation transmitted by the main process through multiple sub-processes, and send the calculation result to the main process;

The calculation module 330 of the main process is used to transmit the data to be calculated to the sub-processes and receive the calculation results of the sub-processes.

In the above device embodiment,

The main function module 310 runs the process initialization function through the process creation module 220, initializes process parameters, including creating threads and binding thread functions; through the main function judging module 320, it is judged that the process belongs to the main process or a slave process, and corresponding branches are run for different processes ; The sub-process calculation module 340 enables multiple sub-processes to receive the calculation data transmitted by the main process, and calculate the received data by calling the core calculation module 240, and send the calculation results to the main process respectively.

For the above-mentioned method and device embodiment, a corresponding application example is given below, and the pseudocode of the original program is as follows:

By applying the above-mentioned device embodiment of the present invention, the multi-node parallelization improvement of the above-mentioned original program is carried out by using the MPI programming interface.

Firstly, test and analyze the program, and know that tFunc1 takes up the most program running time, and the calcFunc function has no dependencies between multiple runs, and can be executed in parallel. The program running time occupied by tFunc2 is less, so it will not be rewritten as a process.

Then rewrite the main function (main), add the initialization process functions MPI_init and MPI_Finalize functions, corresponding to the initialization and exit of the MPI library, respectively.

After the MPI_init function, use the MPI function to obtain the rank number of the process (that is, the process number). Each process will obtain a different rank number. By judging whether the rank number is 0, determine the master-slave process (0 is the master process, non-0 as the slave process); and execute the corresponding master-slave process branch.

If the rank number is 0, copy the codes of creating threads in the main function of the original program——Create_Threads(tFunc1) and Create_Threads(tFunc2) to the main process branch of rank=0.

If the rank number is not 0, it is a child process that needs to perform calculation tasks. First use the MPI function to receive the task length. The task length is related to the number of sub-processes, that is, part of the loop number N in the tFunc1 function. For example, the number of sub-processes is K, and the task length M may be N/K. Then open up memory according to the number of M. Then receive task data from the main process into the newly opened memory. Use the calculation function calcFunc() in the original program to calculate, and at this time, the number of outer loops is not N but M. After calculation, use the MPI function to transfer the data back to the main process.

The main process needs to execute tFunc1 in the original program. Here, the tFunc1 function is modified as follows:

First calculate the task allocation method, such as M=N/K. There are many ways to calculate M. This example takes M=N/K as an example. Cycle through the number of slave nodes K, and use the MPI function to distribute the task number and task data to the slave nodes. Loop again K times, use MPI function to receive data from slave nodes and summarize.

tFunc2 remains unchanged.

When the program starts, use the MPI program startup mode to run in multi-process mode.

Finally, the pseudocode of the new program is as follows:

Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by general-purpose computing devices, and they can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Moreover, they can be implemented with program codes executable by a computing device, so that they can be stored in a storage device and executed by a computing device, or they can be made into individual integrated circuit modules, or a plurality of modules or steps in them Made into a single integrated circuit module to achieve. As such, the present invention is not limited to any specific combination of hardware and software.

Although the embodiments disclosed in the present invention are as above, the described content is only an embodiment adopted for the convenience of understanding the present invention, and is not intended to limit the present invention. Anyone skilled in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed by the present invention, but the patent protection scope of the present invention, The scope defined by the appended claims must still prevail.

Claims (7)

1. A method for multi-node parallel architecture under a multi-threaded program, comprising: Create master and slave processes in the main function of the original program, and obtain information representing the master and slave processes respectively; In the main function of the main process, the thread development of the original program and the operation of the thread binding thread function are retained; in the hot thread function of the main process, the original calculation code is deleted, and the code for communicating with the sub-process is added; in the main process of the sub-process In the function, the content of the main function of the original program is deleted, and the hot thread function calculation part and the communication part with the main process of the original program are added. 2. according to the described method of claim 1, it is characterized in that, also comprise before this method step: The hot thread function is measured according to the original program, and it is determined through analysis that the hot thread function can be optimized into a multi-process parallel architecture. 3. according to the described method of claim 2, it is characterized in that, in the main function of original program, create master, slave process, and obtain the information representing respectively master, slave process; Specifically comprise: In the main function, use the message passing interface initialization function to create the master and slave processes, and obtain the corresponding process numbers through the message passing interface to obtain the process number function, wherein the process number 0 represents the master process, and the remaining non-zero process numbers represent the slave processes. 4. according to the described method of claim 3, it is characterized in that, described in the hot spot thread function of main process, delete original calculation code, increase the code that communicates with child process, specifically comprise: When entering the hot thread function of the main process, the main process sends to the child process the task-related information including task length and task data; when all information is sent, the main process enters a waiting state and waits for the child process to return the calculation result. 5. according to the described method of claim 3, it is characterized in that, in the main function of described subprocess, delete the main function content of original program, increase the hot spot thread function calculation part of original program and communicate with main process part, specifically comprise : The child process waits for the main process to send the information of the task length, and allocates memory space according to the task length; The sub-process waits to receive the information of the task data sent by the main process, and stores the received task data information into the allocated memory space; use the sub-function or corresponding code called by the calculation part in the original program hotspot thread function to receive The task data is calculated; the calculation is completed and the calculation result is sent to the main process; repeat this step until the end of the program. 6. A device with multi-node parallel architecture under a multi-threaded program, characterized in that it includes a main function module and a main process computing module connected to each other, wherein: The main function module is used to initialize the process parameters, run the corresponding branches for the main process and the slave process, receive the calculation data transmitted by the main process through multiple sub-processes and perform calculations, and send the calculation results to the main process; The calculation module of the main process is used to pass the data to be calculated to the sub-processes and receive the calculation results of the sub-processes. 7. The device according to claim 6, characterized in that, The main function module runs the process initialization function through the process creation module, initializes process parameters, including creating threads and binding thread functions; judges that the process belongs to the main process or the slave process through the main function judgment module, and runs corresponding branches for different processes; Through the sub-process calculation module, multiple sub-processes receive the calculation data transmitted by the main process, respectively calculate the received data by calling the core calculation module, and send the calculation results to the main process respectively.

Images