CN118449990B - Calling method and responding method of GPU (graphics processing Unit) equipment - Google Patents
Calling method and responding method of GPU (graphics processing Unit) equipment Download PDFInfo
- Publication number
- CN118449990B CN118449990B CN202410874517.7A CN202410874517A CN118449990B CN 118449990 B CN118449990 B CN 118449990B CN 202410874517 A CN202410874517 A CN 202410874517A CN 118449990 B CN118449990 B CN 118449990B
- Authority
- CN
- China
- Prior art keywords
- host
- gpu
- service module
- equipment
- gpu equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/133—Protocols for remote procedure calls [RPC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/1863—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast comprising mechanisms for improved reliability, e.g. status reports
- H04L12/1868—Measures taken after transmission, e.g. acknowledgments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/164—Adaptation or special uses of UDP protocol
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a calling method and a responding method of GPU equipment, which are applied to the technical field of GPU calling, and the method comprises the following steps: sending a broadcast message for enumerating the GPU equipment to a local network where the first host is located through the first UDP service module; receiving a response message of the second host; determining available target GPU devices from the GPU device list; sending a request message for opening the target GPU equipment to a second host where the target GPU equipment is located; starting RPC service of the first host; and sending a call request to a second RPC module of the second host. Therefore, the first host machine realizes the function of using the GPU equipment in the local network at the GPU driving layer, and the method for using the GPU equipment in the local network by the GPU application program is simplified. And moreover, the GPU device is automatically identified in the local network in the GPU driving layer, so that a GPU application program is not required to be configured, and the use mode and the scene limitation are not required.
Description
Technical Field
The invention relates to the technical field of GPU (graphics processing Unit) call, in particular to a call method and a response method of GPU equipment.
Background
In a local network environment, we often need to use a GPU device or multiple GPU devices in the local network to complete our rendering or computing tasks, and currently existing technical solutions are:
1. Using remote operations to use GPU devices on a host connected to the host in a local network by using remote management tools such as SSH or VNC; 2. using, by a third party GPU application or framework, a GPU device in a local network environment; 3. the GPU application is containerized by a Docker and the GPU device in the local network is used by the Docker.
However, the inventors found in the course of studying the related art that the use of the above-described conventional scheme 1 is cumbersome and only the GPU device on a single host can be used at the same time. The above-mentioned existing scheme 2 needs software familiar with professional fields, and the use of the software in the professional fields limits the use mode and the scene, which is not flexible enough. The above existing scheme 3 also needs to develop a set of supporting software tools for the dock to use the GPU device, which is complicated to use and complex to configure, and also needs the GPU device to support the dock.
Disclosure of Invention
The invention aims at least solving the technical problems existing in the prior art, and therefore, the first aspect of the invention provides a method for calling GPU equipment, the method is applied to a first host, a first UDP service module and a first RPC service module are installed in a GPU driving layer of the first host, and the method comprises the following steps:
Sending a broadcast message for enumerating GPU equipment to a local network where the first host is located through the first UDP service module;
receiving a response message of a second host located in the local network through the first UDP service module, wherein the response message comprises a GPU equipment list on the second host; the second host is a host provided with GPU equipment and a second UDP service module and a second RPC service module in the GPU driving layer;
Determining available target GPU equipment from the GPU equipment list through the first UDP service module according to a task to be executed;
Sending a request message for opening the target GPU equipment to a second host computer where the target GPU equipment is located through the first UDP service module, so that a GPU driver of the target GPU equipment opens the target GPU equipment and RPC service of the second RPC service module is started;
Receiving an opening success message sent by the second host through the first UDP service module, and responding to the opening success message, opening RPC service of the first host;
and sending a call request to a second RPC module of the second host through a first RPC service module of the first host, so that the second host executes the task to be executed according to the call request.
Optionally, before sending, by the first UDP service module, a broadcast message for querying the GPU device to the local network where the first host is located, the method further includes:
Invoking a GPU enumeration interface to acquire all GPU equipment information of the first host;
after receiving the response message of each second host located in the local network, the method further comprises:
Combining all the GPU equipment information of the first host and the GPU equipment information on the second host to obtain a GPU equipment list;
Determining available target GPU equipment according to tasks to be executed and GPU equipment information in the GPU equipment list;
and if the target GPU equipment comprises the GPU equipment of the first host, loading a GPU interface of the GPU equipment of the first host.
Optionally, the determining, by the first UDP service module, the available target GPU device from the GPU device list according to the task to be performed includes:
Inquiring attribute information of GPU equipment in the equipment list; the attribute information comprises functions supported by the GPU equipment;
determining a target function required by the task to be executed;
And determining the GPU equipment comprising the target function as available target GPU equipment.
Optionally, the sending, by the first RPC service module of the first host, a call request to the second RPC module of the second host includes:
and sending a call request to an interface under a Stub object in a second RPC module of the second host through a first RPC service module of the first host.
The second aspect of the present invention proposes a response method of a GPU device, applied to a second host, where the second host includes a GPU device, and a second UDP service module and a second RPC service module are installed in a GPU driving layer of the second host, and the method includes:
receiving a broadcast message which enumerates the GPU equipment and is sent by the first host through the second UDP service module; the first host and the second host are located in the same network;
Responding to the broadcast message, and sending a response message to the first host through the second UDP service module, wherein the response message comprises GPU equipment information on the second host;
Receiving a request message for opening the target GPU equipment sent by the first host through the second UDP service module, opening the target GPU equipment according to the request message, and starting RPC service of the second RPC service module; the target GPU equipment is GPU equipment which accords with the task type of the task to be executed on the second host;
sending an opening success message to the first host through the second UDP service module so that the first host can open RPC service of the first host according to the opening success message;
And receiving a call request sent by the first host through the second RPC service module, and executing the task to be executed according to the call request.
Optionally, the receiving, by the second RPC service module, the call request sent by the first host includes:
and receiving a call request sent by the first host through an interface under a Stub object in the second RPC service module.
The third aspect of the present invention provides a first host, in which a GPU driver layer of the first host is installed with a first UDP service module and a first RPC service module, where the first UDP service module is configured to:
Sending a broadcast message for enumerating GPU equipment to a local network where the first host is located;
Receiving a response message of a second host located in the local network, wherein the response message comprises a GPU equipment list on the second host; the second host is a host provided with GPU equipment and a second UDP service module and a second RPC service module in the GPU driving layer;
Determining available target GPU equipment from the GPU equipment list through the first UDP service module according to a task to be executed;
Sending a request message for opening the target GPU equipment to a second host where the target GPU equipment is located, so that a GPU driver of the target GPU equipment opens the target GPU equipment and RPC service of the second RPC service module is started;
receiving an opening success message sent by the second host, and responding to the opening success message, and opening the RPC service of the first host;
the first RPC service module is configured to:
And sending a call request to a second RPC module of the second host, so that the second host executes the task to be executed according to the call request.
The fourth aspect of the present invention proposes a second host, where the second host includes a GPU device, and a first UDP service module and a first RPC service module are installed in a GPU driver layer of the second host, where the first UDP service module is configured to:
receiving a broadcast message which enumerates GPU equipment and is sent by a first host; the first host and the second host are located in the same network;
Responding to the broadcast message, and sending a response message to the first host, wherein the response message comprises GPU equipment information on the second host;
Receiving a request message for opening a target GPU device sent by the first host, opening the target GPU device according to the request message, and starting the RPC service of the second RPC service module; the target GPU equipment is GPU equipment which accords with the task type of the task to be executed on the second host;
sending an opening success message to the first host, so that the first host opens RPC service of the first host according to the opening success message;
the first RPC service module is configured to:
And receiving a call request sent by the first host, and executing the task to be executed according to the call request.
A fifth aspect of the present invention proposes an electronic device comprising a processor and a memory, wherein at least one instruction, at least one program, a set of codes or a set of instructions is stored in the memory, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded and executed by the processor to implement a method for invoking a GPU device according to the first aspect, and/or a method for responding to a GPU device according to the second aspect.
A sixth aspect of the present invention proposes a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes or a set of instructions, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded and executed by a processor to implement a method for invoking a GPU device according to the first aspect and/or a method for responding to a GPU device according to the second aspect.
The embodiment of the invention has the following beneficial effects:
According to the method for calling the GPU equipment, which is provided by the embodiment of the invention, the broadcast message for enumerating the GPU equipment is sent to the local network where the first host is located through the first UDP service module; receiving a response message of a second host located in the local network through the first UDP service module, wherein the response message comprises a GPU equipment list on the second host; the second host is a host provided with GPU equipment and a second UDP service module and a second RPC service module in the GPU driving layer; determining available target GPU equipment from the GPU equipment list through the first UDP service module according to a task to be executed; sending a request message for opening the target GPU equipment to a second host computer where the target GPU equipment is located through the first UDP service module, so that a GPU driver of the target GPU equipment opens the target GPU equipment and RPC service of the second RPC service module is started; receiving an opening success message sent by the second host, and responding to the opening success message, and opening the RPC service of the first host; and sending a call request to a second RPC module of the second host through a first RPC service module of the first host, so that the second host executes the task to be executed according to the call request. Therefore, the first host machine realizes the function of using the GPU equipment in the local network at the GPU driving layer, so that the GPU application program can directly use the GPU equipment in the local network through the GPU interface, and the method for using the GPU equipment in the local network by the GPU application program is simplified. In addition, the method for automatically identifying the GPU equipment in the local network is realized in the GPU driving layer, any manual configuration is not needed, any change is not needed to be made to a GPU interface, the GPU application program can be used without distinction by the local GPU equipment and the GPU equipment in the local network, and the GPU application program is not needed to be configured, and the use mode and the scene limitation are not needed. In addition, the function of using the GPU equipment in the local network is realized in the GPU driving layer, so that all the GPU equipment in the local network can be automatically constructed into GPU clusters, and the GPU clusters can be conveniently used by utilizing the existing local network.
Drawings
FIG. 1 is a flowchart of a method for calling a GPU device according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a calling system of a GPU device according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a call procedure of a GPU device according to an embodiment of the present invention;
fig. 4 is a data flow diagram of a method for calling a GPU device according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating steps of a method for responding to a GPU device according to an embodiment of the present invention;
FIG. 6 is a block diagram of a first host according to an embodiment of the present invention;
Fig. 7 is a block diagram of a second host according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the embodiments of the present disclosure, unless otherwise indicated, the meaning of "a plurality" is two or more. In addition, the use of "based on" or "according to" is intended to be open and inclusive in that a process, step, calculation, or other action "based on" or "according to" one or more of the stated conditions or values may in practice be based on additional conditions or beyond the stated values.
In a local network environment, we often need to use a GPU device or multiple GPU devices in the local network to complete our rendering or computing tasks, and currently existing technical solutions are:
1. using remote operations to use GPU devices on a host connected to the host in a local network by using remote management tools such as SSH or VNC;
2. using, by a third party GPU application or framework, a GPU device in a local network environment;
3. the GPU application is containerized by a Docker and the GPU device in the local network is used by the Docker.
However, the inventor has found in the process of studying the related art that in the above-mentioned prior art scheme 1, it is necessary to install a remote management tool such as SSH or VNC first, then connect the remote host through the tool, copy the application program rendered or calculated by the GPU to the remote host, and then execute the application program. If GPUs of different configurations are installed in the local network, the configuration of the GPU devices on different hosts needs to be checked manually, and then an appropriate GPU device is selected for use. It can be seen that this approach is cumbersome to use and that it can only use GPU devices on a single host at the same time.
The existing scheme 2 described above uses third party GPU applications or frameworks such as Blender and TensorFlow, etc. When the method is used, corresponding software is required to be installed on all hosts of a local network, GPU equipment nodes are required to be configured, and after the configuration is completed, the software is started. This allows the GPU device in the local network to be used in computing or rendering using third party software or frameworks. This approach requires software familiar with these specialized fields, with steep learning curves. In addition, the application of software in the professional fields limits the application mode and scene, the application is not flexible enough, and each third party software or framework needs to realize the access function of the GPU equipment in the local network, thus adding burden to the third party software or framework.
In the above existing scheme 3, a Docker containerized deployment and distributed framework needs to be used to construct a Docker mirror image and configure Docker nodes, and a set of supporting software tools for Docker to use GPU equipment needs to be developed. It can be seen that the use is complicated, the configuration is complex, and the support of the GPU equipment to the Docker is also required.
The proposal provides a new method for using the GPU equipment in the local network, namely, the function of using the GPU equipment in the local network is realized in a GPU driving layer, all the GPU equipment in the local network is automatically identified in the GPU driving, and a GPU application program can use the local GPU equipment and the GPU equipment in the local network indiscriminately through a GPU interface.
Fig. 1 is a flowchart of a method for calling a GPU device according to an embodiment of the present invention. The method is applied to a first host, a first UDP service module and a first RPC service module are installed in a GPU driving layer of the first host, and the method comprises the following steps:
step 101, a broadcast message for enumerating GPU equipment is sent to a local network where the first host is located through the first UDP service module.
GPU applications use GPU devices through GPU interfaces, which have industry interface specifications such as OpenGL, vulkan and OpenCL, and some manufacturers' own interfaces. The GPU application uses the GPU device through the GPU interface, and the functional implementation of the GPU interface comes from the GPU driver, which is a bridge between the GPU application and the GPU hardware device, and the driver of the corresponding GPU device must be installed to use the GPU hardware device.
Under the condition that the GPU interface is unchanged, two modules are added in the GPU driving layer: a UDP (User Datagram Protocol ) service module and an RPC (Remote Procedure Call ) service module by which GPU applications can use any GPU device in the local network directly using the GPU interface.
The first UDP service module is responsible for automatic identification of GPU equipment in the local network, other loading and initializing services and the like. The first RPC service module is mainly responsible for remote call of the GPU interface in the local network.
Fig. 2 is a schematic structural diagram of a call system of a GPU device according to an embodiment of the present invention.
As shown in fig. 2, the host 1, the host 2 and the host 3 are three hosts in the local network, GPU devices are installed in the three hosts, and GPU application programs are installed on the host 1. The GPU application invokes the GPU interface to implement its own functions using the GPU device. The functions of the GPU interface are implemented in a GPU driver that implements the functions of the interface by invoking GPU device hardware. The GPU drives of the three hosts comprise UDP service modules and RPC service modules. Both the UDP service module and the RPC service module may communicate with the ethernet network.
The first UDP service module uses the UDP protocol to bind the IP address on the local and the assigned port number, so that the first UDP service module may monitor the UDP message on the port.
Fig. 3 is a schematic diagram of a call process of a GPU device according to an embodiment of the present invention.
As shown in fig. 3, the UDP service module binding IP address of three hosts is 0.0.0.0, indicating that the UDP service module is bound to the host. And, the UDP service module binds the port number 6000 specified by the local, so that the UDP service module may monitor the UDP message on the local port 6000.
Further, 255.255.255.255:6000 denotes sending a broadcast message to the local network on port 6000. The content of the broadcast message is to enumerate GPU devices, meaning that the host that receives the broadcast message needs to enumerate GPU devices on the host to host 1.
Step 102, receiving, by the first UDP service module, a response message of a second host located in the local network, where the response message includes a GPU device list on the second host; the second host is a host provided with GPU equipment and a second UDP service module and a second RPC service module in the GPU driving layer.
After receiving the broadcast message for enumerating the GPU device sent by the first host, each host in the local network responds to the broadcast message, and sends a response message to the first host through the first UDP service module, wherein the response message includes the GPU device information on the host.
The second host is one of hosts in which the GPU equipment is installed in the local network and the second UDP service module and the second RPC service module are installed in the GPU driving layer. The second host sends a response message to the first host through the second UDP service module, wherein the response message comprises GPU equipment information on the host.
The GPU device information specifically includes a list of GPU devices on the second host.
As shown in fig. 3, when the GPU application on the host 1 enumerates the GPU device through the GPU interface, the GPU driver uses the UDP service module to send a UDP broadcast message for querying the GPU device in a broadcast manner after enumerating the GPU device of the host, and at this time, the UDP service module in the GPU driver on other hosts (such as the host 2 and the host 3) in the local network will receive the broadcast message. After receiving the broadcast message, each host enumerates its own GPU device information, and then returns the enumerated GPU device information to the requesting host, i.e. host 1. The GPU driver on host 1 will get a list of all GPU devices in the local network.
And step 103, according to the task to be executed, determining available target GPU equipment from the GPU equipment list through the first UDP service module.
And the GPU application program of the first host queries the attribute of each GPU device according to the returned all GPU device lists, and selects the available target GPU device according to the attribute of each GPU device and the matching condition of the task to be executed.
In one possible implementation, step 103 includes:
Step 1031, inquiring attribute information of the GPU equipment in the equipment list; the attribute information comprises functions supported by the GPU equipment;
step 1032, determining a target function required by the task to be executed;
step 1033, determining the GPU device comprising the target functionality as an available target GPU device.
In steps 1031-1033, first, by querying attribute information of GPU devices, functions supported by the respective GPU devices are determined. For example, the supported functions include rendering, computing, and the like. Then, determining a function required by the task to be executed of the first host, wherein the function required by the task to be executed is rendering.
Finally, the GPU device comprising the target function is determined as the target GPU device. For example, GPU device a on the second host, GPU device B on the third host, which includes rendering functionality, are determined to be target GPU devices.
Step 104, sending, by the first UDP service module, a request message for opening the target GPU device to a second host where the target GPU device is located, so that a GPU driver of the target GPU device opens the target GPU device, and opens an RPC service of the second RPC service module.
If the first host inquires that the GPU equipment on the second host is available, a request message for opening the GPU equipment is sent to the second host, and after the GPU driver of the second host receives the request message, the GPU equipment is opened, and the RPC server side service of the corresponding GPU equipment is started.
Step 105, receiving an opening success message sent by the second host, and responding to the opening success message, opening the RPC service of the first host.
The second host returns the opened result information to the first host. And the first host analyzes the result information, and if the result information is found to be the opening success message, the RPC client service of the first host is opened.
And step 106, sending a call request to a second RPC module of the second host through a first RPC service module of the first host, so that the second host executes the task to be executed according to the call request.
The first host performs the task by sending a call request to the second host, invoking a remote GPU device on the second host like the native GPU interface.
After the remote GPU equipment is used, the first host computer calls the GPU interface on the second host computer to perform cleaning work.
In one possible implementation, step 106 includes:
and sending a call request to an interface under a Stub object in a second RPC module of the second host through a first RPC service module of the first host.
Specifically, all GPU interfaces of each GPU device are loaded under one Stub object, through which GPU interfaces of the corresponding GPU device can be invoked.
In one possible implementation manner, before sending, by the first UDP service module, a broadcast message for querying the GPU device to the local network where the first host is located, the method further includes:
Step 107, calling a GPU enumeration interface to obtain all GPU equipment information of the first host;
after receiving the response message of each second host located in the local network, the method further comprises:
Step 108, combining all the GPU equipment information of the first host and the GPU equipment information on the second host to obtain a GPU equipment list;
Step 109, determining available target GPU equipment according to the task to be executed and GPU equipment information in the GPU equipment list;
Step 110, if the target GPU device includes the GPU device of the first host, loading the GPU interface of the GPU device of the first host.
In the embodiment of the invention, when the upper GPU application program of the first host enumerates the GPU equipment through the GPU interface, the GPU driver enumerates the local GPU equipment first, and after the local GPU equipment is enumerated, the first UDP service module is used for sending UDP broadcast messages for inquiring the GPU equipment in a broadcast mode.
After receiving the response messages of the second hosts in the local network, the first host combines all GPU equipment information of the first host with GPU equipment information on the second host to obtain a GPU equipment list. The first host queries available target GPU equipment from the GPU equipment list, and if the target GPU equipment comprises the GPU equipment of the first host, the first host opens the own GPU equipment to execute related tasks.
Fig. 4 is a data flow diagram of a method for calling a GPU device according to an embodiment of the present invention.
As shown in fig. 4, there are three hosts in the home network: host 1, host 2 and host 3, each host is installed with a GPU device and a GPU driver of the present disclosure, and the GPU application on host 1 needs to use multiple GPU devices to complete the rendering task, and the general procedure is as shown in the figure:
s1: the GPU application program calls some interfaces initialized by the GPU to finish some initialized operations.
S2: the GPU application program calls a GPU enumeration interface to enumerate all GPU devices, when the GPU driver enumerates the GPU devices, all the GPU devices of the local machine are enumerated, then broadcast messages for inquiring the GPU devices are sent, all the GPU devices in the local network are inquired, after receiving the broadcast messages for inquiring the GPU devices, UDP monitoring modules of a host 2 and a host 3 inquire the GPU devices on the local machines, then the inquired GPU device information is returned to the host 1, and after the host 1 completes enumeration of the GPU devices, a GPU device information list is returned to the GPU application program.
S3: and the GPU application program queries the attribute of each GPU device according to the returned all GPU device list.
S4: and selecting proper GPU equipment according to the attribute of each GPU equipment.
S5: assuming that all the queried 3 GPU devices are available, all the GPU devices are opened, all the GPU interfaces of the GPU devices are loaded when the GPU devices are opened, if the GPU devices are the GPU devices of the host 1, all the GPU interfaces on the host 1 are directly loaded, if the GPU devices are the GPU devices on the host 2 and the host 3, a request message for opening the GPU devices is sent to a host address of the GPU devices, after receiving the request message, a GPU driver of a destination host opens the GPU devices and starts an RPC server service of the corresponding GPU devices, then the opened result is returned to the host 1, if the host 1 judges that the opened result is successful, the corresponding RPC client service is opened, all the GPU interfaces of each GPU device are loaded under one Stub object, and the GPU interfaces of the corresponding GPU devices can be called through the Stub object.
S6: the GPU application program uses interfaces of different GPU devices according to own business logic, if the GPU device is a remote GPU device in a local network, the GPU driver program can be automatically called to the remote GPU device, and the GPU application program only needs to be called like a local GPU interface.
S7: after the use, the GPU interface is called to carry out cleaning work.
The method and the device realize the function of using the GPU equipment in the local network in the GPU driving layer, so that the GPU application program can directly use the GPU equipment in the local network through the GPU interface without realizing the function of using the GPU equipment in the local network, the method of using the GPU equipment in the local network by the GPU application program is simplified, and the GPU application program can be more focused on the realization of own business.
In addition, the method for automatically identifying the GPU equipment in the local network is realized in the GPU driving layer, any manual configuration is not needed, and any change is not needed to be made to a GPU interface, so that the GPU application program can use the local GPU equipment and the GPU equipment in the local network indiscriminately.
In addition, the function of using the GPU equipment in the local network is realized in the GPU driving layer, so that all the GPU equipment in the local network can be automatically constructed into GPU clusters, and the GPU clusters can be conveniently used by utilizing the existing local network.
To sum up, in the embodiment of the present invention, a broadcast message for enumerating GPU devices is sent to a local network where the first host is located through the first UDP service module; receiving a response message of a second host located in the local network through the first UDP service module, wherein the response message comprises a GPU equipment list on the second host; the second host is a host provided with GPU equipment and a second UDP service module and a second RPC service module in the GPU driving layer; determining available target GPU equipment from the GPU equipment list through the first UDP service module according to a task to be executed; sending a request message for opening the target GPU equipment to a second host computer where the target GPU equipment is located through the first UDP service module, so that a GPU driver of the target GPU equipment opens the target GPU equipment and RPC service of the second RPC service module is started; receiving an opening success message sent by the second host, and responding to the opening success message, and opening the RPC service of the first host; and sending a call request to a second RPC module of the second host through a first RPC service module of the first host, so that the second host executes the task to be executed according to the call request. Therefore, the first host machine realizes the function of using the GPU equipment in the local network at the GPU driving layer, so that the GPU application program can directly use the GPU equipment in the local network through the GPU interface, and the method for using the GPU equipment in the local network by the GPU application program is simplified. In addition, the method for automatically identifying the GPU equipment in the local network is realized in the GPU driving layer, any manual configuration is not needed, any change is not needed to be made to a GPU interface, the GPU application program can be used without distinction by the local GPU equipment and the GPU equipment in the local network, and the GPU application program is not needed to be configured, and the use mode and the scene limitation are not needed. In addition, the function of using the GPU equipment in the local network is realized in the GPU driving layer, so that all the GPU equipment in the local network can be automatically constructed into GPU clusters, and the GPU clusters can be conveniently used by utilizing the existing local network.
Fig. 5 is a flowchart of steps of a response method of a GPU device according to an embodiment of the present invention.
As shown in fig. 5, the method is applied to a second host, the second host includes a GPU device, and a second UDP service module and a second RPC service module are installed in a GPU driver layer of the second host, and the method includes:
Step 201, receiving a broadcast message for enumerating the GPU device sent by the first host through the second UDP service module; the first host and the second host are located in the same network;
Step 202, responding to the broadcast message, and sending a response message to the first host through the second UDP service module, wherein the response message comprises GPU equipment information on the second host;
Step 203, receiving, by the second UDP service module, a request message for opening the target GPU device sent by the first host, opening the target GPU device according to the request message, and opening an RPC service of the second RPC service module; the target GPU equipment is GPU equipment which accords with the task type of the task to be executed on the second host;
step 204, sending an opening success message to the first host through the second UDP service module, so that the first host opens the RPC service of the first host according to the opening success message;
step 205, receiving, by the second RPC service module, a call request sent by the first host, and executing the task to be executed according to the call request.
In steps 201-205, after receiving a broadcast message for enumerating GPU devices sent by a first host, each host in the local network sends a response message to the first host through a second UDP service module, where the response message includes GPU device information on the host.
The second host is one of hosts in which the GPU equipment is installed in the local network and the second UDP service module and the second RPC service module are installed in the GPU driving layer. The second host sends a response message to the first host through the second UDP service module, wherein the response message comprises GPU equipment information on the host.
The GPU device information specifically includes a list of GPU devices on the second host.
If the first host inquires that the GPU equipment on the second host is available through the attribute information of the GPU equipment list, a request message for opening the GPU equipment is sent to the second host, and after the request message is received, the GPU driver of the second host opens the GPU equipment and starts the RPC server side service of the corresponding GPU equipment.
The second host returns an opening success message to the first host. And the first host analyzes the result information, and if the result information is found to be the opening success message, the RPC client service of the first host is opened.
The second host receives the call request sent by the first host, and the second RPC service module of the second host executes the task according to the call request.
In one possible implementation, step 205 includes: and receiving a call request sent by the first host through an interface under a Stub object in the second RPC service module.
All GPU interfaces of each GPU device are loaded under one Stub object, and the corresponding GPU interfaces of the GPU devices can be called through the Stub object. The GPU device of the second host is no exception, and receives the call request through an interface under the Stub object.
In summary, in the embodiment of the present invention, a broadcast message for enumerating GPU devices sent by a first host is received through the second UDP service module; the first host and the second host are located in the same network; responding to the broadcast message, and sending a response message to the first host through the second UDP service module, wherein the response message comprises GPU equipment information on the second host; receiving a request message for opening the target GPU equipment sent by the first host through the second UDP service module, opening the target GPU equipment according to the request message, and starting RPC service of the second RPC service module; the target GPU equipment is GPU equipment which accords with the task type of the task to be executed on the second host; sending an opening success message to the first host through the second UDP service module so that the first host can open RPC service of the first host according to the opening success message; and receiving a call request sent by the first host through the second RPC service module, and executing the task to be executed according to the call request. Therefore, the first host machine realizes the function of using the GPU equipment in the local network at the GPU driving layer, so that the GPU application program can directly use the GPU equipment in the local network through the GPU interface, and the method for using the GPU equipment in the local network by the GPU application program is simplified. In addition, the method for automatically identifying the GPU equipment in the local network is realized in the GPU driving layer, any manual configuration is not needed, and any change is not needed to be made to a GPU interface, so that the GPU application program can use the local GPU equipment and the GPU equipment in the local network indiscriminately. In addition, the function of using the GPU equipment in the local network is realized in the GPU driving layer, so that all the GPU equipment in the local network can be automatically constructed into GPU clusters, and the GPU clusters can be conveniently used by utilizing the existing local network.
Fig. 6 is a block diagram of a first host according to an embodiment of the present invention. As shown in fig. 6, a first UDP service module 301 and a first RPC service module 302 are installed in the GPU driver layer of the first host, where the first UDP service module 301 is configured to:
Sending a broadcast message for enumerating GPU equipment to a local network where the first host is located;
Receiving a response message of a second host located in the local network, wherein the response message comprises a GPU equipment list on the second host; the second host is a host provided with GPU equipment and a second UDP service module and a second RPC service module in the GPU driving layer;
Determining available target GPU equipment from the GPU equipment list through the first UDP service module according to a task to be executed;
Sending a request message for opening the target GPU equipment to a second host where the target GPU equipment is located, so that a GPU driver of the target GPU equipment opens the target GPU equipment and RPC service of the second RPC service module is started;
receiving an opening success message sent by the second host, and responding to the opening success message, and opening the RPC service of the first host;
the first RPC service module 302 is configured to:
And sending a call request to a second RPC module of the second host, so that the second host executes the task to be executed according to the call request.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Fig. 7 is a block diagram of a second host according to an embodiment of the present invention. As shown in fig. 7, a second UDP service module 401 and a second RPC service module 402 are installed in the GPU driver layer of the second host, and the second UDP service module 401 is configured to:
receiving a broadcast message which enumerates GPU equipment and is sent by a first host; the first host and the second host are located in the same network;
Responding to the broadcast message, and sending a response message to the first host, wherein the response message comprises GPU equipment information on the second host;
Receiving a request message for opening a target GPU device sent by the first host, opening the target GPU device according to the request message, and starting the RPC service of the second RPC service module; the target GPU equipment is GPU equipment which accords with the task type of the task to be executed on the second host;
sending an opening success message to the first host, so that the first host opens RPC service of the first host according to the opening success message;
The second RPC service module 402 is configured to:
And receiving a call request sent by the first host, and executing the task to be executed according to the call request.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. The method for calling the GPU equipment is characterized by being applied to a first host, wherein a first UDP service module and a first RPC service module are installed in a GPU driving layer of the first host, and the method comprises the following steps:
Sending a broadcast message for enumerating GPU equipment to a local network where the first host is located through the first UDP service module;
Receiving a response message of a second host located in the local network through the first UDP service module, wherein the response message comprises a GPU equipment list on the second host; the second host is a host provided with GPU equipment and a first UDP service module and a first RPC service module in a GPU driving layer;
determining available target GPU equipment from the GPU equipment list through the first UDP service module according to a task to be executed; the available target GPU equipment is GPU equipment comprising target functions required by the task to be executed;
Sending a request message for opening the target GPU equipment to a second host computer where the target GPU equipment is located through the first UDP service module, so that a GPU driver of the target GPU equipment opens the target GPU equipment and RPC service of the first RPC service module is started;
Receiving an opening success message sent by the second host through the first UDP service module, and responding to the opening success message, opening RPC service of the first host;
and sending a call request to the RPC module of the second host through the first RPC service module of the first host so that the second host executes the task to be executed according to the call request.
2. The method according to claim 1, further comprising, prior to sending, by the first UDP service module, a broadcast message querying a GPU device to a local network on which the first host is located:
Invoking a GPU enumeration interface to acquire all GPU equipment information of the first host;
after receiving the response message of each second host located in the local network, the method further comprises:
Combining all the GPU equipment information of the first host and the GPU equipment information on the second host to obtain a GPU equipment list;
Determining available target GPU equipment according to tasks to be executed and GPU equipment information in the GPU equipment list;
and if the target GPU equipment comprises the GPU equipment of the first host, loading a GPU interface of the GPU equipment of the first host.
3. The method according to claim 1, wherein the determining, by the first UDP service module, available target GPU devices from the list of GPU devices according to the task to be performed, comprises:
Inquiring attribute information of GPU equipment in the equipment list; the attribute information comprises functions supported by the GPU equipment;
determining a target function required by the task to be executed;
And determining the GPU equipment comprising the target function as available target GPU equipment.
4. The method of claim 1, wherein the sending, by the first RPC service module of the first host, a call request to the RPC module of the second host comprises:
And sending a call request to an interface under a Stub object in an RPC module of the second host through a first RPC service module of the first host.
5. A response method of a GPU device, applied to a second host, where the second host includes a GPU device, and a second UDP service module and a second RPC service module are installed in a GPU driver layer of the second host, and the method includes:
receiving a broadcast message which enumerates the GPU equipment and is sent by the first host through the second UDP service module; the first host and the second host are located in the same network;
Responding to the broadcast message, and sending a response message to the first host through the second UDP service module, wherein the response message comprises GPU equipment information on the second host;
Receiving a request message for opening the target GPU equipment sent by the first host through the second UDP service module, opening the target GPU equipment according to the request message, and starting RPC service of the second RPC service module; the target GPU equipment is GPU equipment which accords with the task type of the task to be executed on the second host;
sending an opening success message to the first host through the second UDP service module so that the first host can open RPC service of the first host according to the opening success message;
And receiving a call request sent by the first host through the second RPC service module, and executing the task to be executed according to the call request.
6. The method of claim 5, wherein the receiving, by the second RPC service module, the call request sent by the first host comprises:
and receiving a call request sent by the first host through an interface under a Stub object in the second RPC service module.
7. The first host is characterized in that a first UDP service module and a first RPC service module are installed in a GPU driving layer of the first host, and the first UDP service module is used for:
Sending a broadcast message for enumerating GPU equipment to a local network where the first host is located;
Receiving a response message of a second host located in the local network, wherein the response message comprises a GPU equipment list on the second host; the second host is a host provided with GPU equipment and a first UDP service module and a first RPC service module in a GPU driving layer;
determining available target GPU equipment from the GPU equipment list through the first UDP service module according to a task to be executed; the available target GPU equipment is GPU equipment comprising target functions required by the task to be executed;
Sending a request message for opening the target GPU equipment to a second host where the target GPU equipment is located, so that a GPU driver of the target GPU equipment opens the target GPU equipment and opens RPC service of the first RPC service module;
receiving an opening success message sent by the second host, and responding to the opening success message, and opening the RPC service of the first host;
the first RPC service module is configured to:
and sending a call request to an RPC module of the second host, so that the second host executes the task to be executed according to the call request.
8. The second host is characterized in that the second host comprises GPU equipment, a second UDP service module and a second RPC service module are installed in a GPU driving layer of the second host, and the second UDP service module is used for:
receiving a broadcast message which enumerates GPU equipment and is sent by a first host; the first host and the second host are located in the same network;
Responding to the broadcast message, and sending a response message to the first host, wherein the response message comprises GPU equipment information on the second host;
Receiving a request message for opening a target GPU device sent by the first host, opening the target GPU device according to the request message, and starting the RPC service of the second RPC service module; the target GPU equipment is GPU equipment which accords with the task type of the task to be executed on the second host;
sending an opening success message to the first host, so that the first host opens RPC service of the first host according to the opening success message;
the second RPC service module is configured to:
And receiving a call request sent by the first host, and executing the task to be executed according to the call request.
9. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement a method of invoking a GPU device as in any of claims 1-4, and/or a method of responding to a GPU device as in any of claims 5-6.
10. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement a method of invoking a GPU device as claimed in any of claims 1-4, and/or a method of responding to a GPU device as claimed in any of claims 5-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410874517.7A CN118449990B (en) | 2024-07-02 | 2024-07-02 | Calling method and responding method of GPU (graphics processing Unit) equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410874517.7A CN118449990B (en) | 2024-07-02 | 2024-07-02 | Calling method and responding method of GPU (graphics processing Unit) equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118449990A CN118449990A (en) | 2024-08-06 |
| CN118449990B true CN118449990B (en) | 2024-09-17 |
Family
ID=92312567
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410874517.7A Active CN118449990B (en) | 2024-07-02 | 2024-07-02 | Calling method and responding method of GPU (graphics processing Unit) equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118449990B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111913715A (en) * | 2020-07-30 | 2020-11-10 | 上海数策软件股份有限公司 | Micro-service based machine learning automation process management and optimization system and method |
| CN114026834A (en) * | 2019-04-30 | 2022-02-08 | 英特尔公司 | Multi-entity resource, security, and service management in edge computing deployments |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110389827B (en) * | 2018-04-20 | 2023-05-12 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for optimization in a distributed system |
| CN116680035A (en) * | 2023-04-24 | 2023-09-01 | 浪潮通信技术有限公司 | A method and device for realizing kubernetes container remote scheduling using GPU |
| CN117056041A (en) * | 2023-07-18 | 2023-11-14 | 中国科学院计算技术研究所 | Fine granularity resource pool scheduling method and system for GPU remote call |
-
2024
- 2024-07-02 CN CN202410874517.7A patent/CN118449990B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114026834A (en) * | 2019-04-30 | 2022-02-08 | 英特尔公司 | Multi-entity resource, security, and service management in edge computing deployments |
| CN111913715A (en) * | 2020-07-30 | 2020-11-10 | 上海数策软件股份有限公司 | Micro-service based machine learning automation process management and optimization system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118449990A (en) | 2024-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7058954B1 (en) | System and method for facilitating dynamic loading of stub information to enable a program operating in one address space to invoke processing of a remote method or procedure in another space | |
| US6470342B1 (en) | Process of maintaining a distributed map of transaction identifiers and using hashing to access these maps | |
| US7007077B1 (en) | Remote maintenance method of an information processing apparatus | |
| US9736017B2 (en) | Server-side protocol configuration of accessing clients | |
| CN112199621B (en) | Method and device for quickly generating poster and computer equipment | |
| JP2013522769A (en) | Virtual machine image update service | |
| CN112333017A (en) | Service configuration method, device, equipment and storage medium | |
| CN106686038A (en) | Method and device for calling cloud desktop | |
| GB2366168A (en) | Controlling access to network interface cards | |
| US6625664B2 (en) | Automation system to access functionality of hardware components with each hardware component having system connection unit with function objects representing real functionality of components | |
| CN118449990B (en) | Calling method and responding method of GPU (graphics processing Unit) equipment | |
| CN111740972B (en) | Method, device, equipment and storage medium for updating communication protocol stack information | |
| US6823358B1 (en) | Enabling multiple client access to a process-based system or program from a single java virtual machine | |
| CN112672435A (en) | Communication method, communication apparatus, computer device, and storage medium | |
| US11489925B2 (en) | Information interaction methods, apparatuses, devices, and systems and storage media | |
| CN115242873A (en) | PCB equipment message response method and device, computer equipment and storage medium | |
| CN116527633B (en) | Methods, devices, processing equipment, and storage media for handling pod IP addresses | |
| US6618765B1 (en) | Method of modifying a protocol between distributed objects | |
| CN120821587B (en) | Methods, devices, storage media, and program products for remotely invoking acceleration devices | |
| CN116566969B (en) | Protocol downloading method, device, equipment and storage medium | |
| EP4601248A1 (en) | Method and apparatus for communication, and storage medium | |
| CN120710973A (en) | Message forwarding method, device, electronic device and storage medium | |
| CN114281684A (en) | mock data processing method, system, mock platform and readable storage medium | |
| US20240202049A1 (en) | Gateway apparatus and method for dynamically applying api settings | |
| JP4512142B2 (en) | Communication program test method and communication program test system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: Unit 802, No. 63 Chengyi North Street, Software Park Phase III, Torch High tech Zone, Xiamen City, Fujian Province 361006 Patentee after: Xintong Semiconductor Technology (Xiamen) Co.,Ltd. Country or region after: China Address before: Room 301, Building D, Yeda Science and Technology Park, No. 300 Changjiang Road, Yantai Area, China (Shandong) Pilot Free Trade Zone, Yantai City, Shandong Province Patentee before: Xi'an Xintong Semiconductor Technology Co.,Ltd. Country or region before: China |