CN117593170A - A video memory allocation method, device, equipment and readable storage medium - Google Patents

Abstract

The invention discloses a video memory distribution method, a device, equipment and a readable storage medium in the technical field of computer application, wherein the method comprises the following steps: acquiring a task of a video memory to be allocated; acquiring a container process identifier of each process in a container, and inquiring a host process identifier of each process in the container according to a process identifier relation table; based on the host process identifier, obtaining the memory consumption of each process in the container; superposing the video memory consumption of all processes in the container to obtain the total video memory consumption of the container; and determining the residual video memory usable by the container by using the total video memory consumption and the video memory quota of the container, and distributing the video memory to the task based on the residual video memory. The invention has the technical effects that: the accurate container video memory consumption can be obtained, the authority of a host is not granted to a user, and the multi-card multi-task scene can be effectively managed through the process identification relation table, so that the video memory allocation can be effectively and orderly carried out.

Description

A video memory allocation method, device, equipment and readable storage medium

Technical field

The present invention relates to the field of computer application technology, and in particular to a memory allocation method, device, equipment and readable storage medium.

Background technique

In the computing architecture of CPU+GPU, the GPU (graphics processing unit, graphics processor) mainly carries computing-intensive task loads to accelerate calculations. In the CPU+GPU computing architecture, multiple tasks running on the same GPU will compete for GPU resources. If the remaining resources are not enough to run the task, the task will fail and exit. That is, when the lack of effective and orderly management of GPU resources leads to chaos and disorder when multiple tasks share the GPU, the task failure rate increases.

Currently, k8s (Kubernetes, an application that manages cluster resources) or a container management system are usually used to manage GPU resources, and the technology of pre-allocated video memory is used to solve the problem. The specific method is to hijack the cuda driver (unified computing device architecture driver; cuda: Compute Unified Device Architecture, a unified computing device architecture, is used to run parallel computing on general computing devices (GPU); driver, driver)'s video memory allocation API determines whether the free video memory in the container meets the size of the video memory to be allocated during video memory allocation, thereby performing Video memory allocation.

However, when counting the video memory usage of GPU processes in a container, the size of the implicitly allocated video memory is often omitted, which will cause the actual use of video memory by the process in the container to exceed the quota and affect task execution; in the case of multiple cards and multiple tasks, the management of the container The video memory of each GPU card becomes more complicated, especially when the user sets the CUDA_VISIBLE_DEVICES parameter when running a GPU task. The task will change the GPU index, causing confusion in the logic of controlling the video memory; the nvidia-smi command that comes with the nvidia driver is in the container. GPU process information cannot be displayed. In order to view the running status of their own tasks, users need to log in to the host and run this command. They also need to distinguish among all tasks which tasks are their own containers. When there are many shared containers and GPU tasks on the GPU card, It is very troublesome to check your own tasks, and you need to give ordinary users host permissions, which will cause ordinary users to see each other's tasks, which poses a security risk.

To sum up, how to effectively solve problems such as video memory allocation is a technical problem that those skilled in the art urgently need to solve.

Contents of the invention

The object of the present invention is to provide a video memory allocation method, device, equipment and readable storage medium that can obtain accurate container video memory usage, avoid granting host permissions to users, and effectively manage multiple cards and multiple tasks through a process identification relationship table. scenarios, so that efficient and orderly allocation of video memory can be achieved.

In order to solve the above technical problems, the present invention provides the following technical solutions:

A video memory allocation method, including:

Get the task to be allocated video memory;

Obtain the container process identifier of each process in the container, and query the host process identifier of each process in the container according to the process identifier relationship table;

Based on the host process identifier, obtain the video memory usage of each process in the container;

Add the video memory usage of all processes in the container to obtain the total video memory usage of the container;

The remaining video memory of the container is determined using the total amount of video memory and the video memory quota of the container, and video memory is allocated to the task based on the remaining video memory.

Preferably, obtaining the process identification relationship table includes:

Obtain the container process identifier of each process in the container;

Call the open function to open the mounted custom character device;

Call the calling interface function of the custom character device and enter parameters to obtain a return result; wherein the parameters are predefined instructions and the container process identifier of each process in the container, and the return result is the container process identifier of each process. Host process identifier;

Write the container process identifier and the host process identifier of the same process into the process identification relationship table in pairs.

Preferably, obtaining the container process identifier of each process in the container includes:

Obtain the GPU card information in the container;

Use the GPU card information to obtain the container process identifiers of all processes running on the GPU card.

Preferably, based on the host process identifier, obtaining the video memory usage of each process in the container includes:

Obtain the GPU card information in the container;

Using the GPU card information, obtain the video memory information of all processes running on the GPU card;

Find the video memory usage corresponding to the host process identifier from the video memory information.

Preferably, it also includes:

Use the GPU process display tool to read the process identification relationship table;

Obtain the video memory quota;

Obtain the number of GPUs and GPU unique identifiers in the container;

Based on the number of GPUs and the unique GPU identifier, cyclically obtain each process information corresponding to each GPU in the container;

Match the process identification relationship table with the GPU process information returned by calling the management library function, and record the video memory usage of the host process identification;

Based on the process identification relationship table, parameter replacement processing is performed on the result returned by the computer running information viewing function;

After completing the parameter replacement, the video memory usage information of the container is obtained; wherein the video memory usage information includes the video memory usage of each process, the video memory quota of the container, and the remaining capacity of the container;

Based on the process identification relationship table, parameter replacement processing is performed on the result returned by the computer running information viewing function, including:

Based on the process identification relationship table, replace the host process identification in the returned result with the corresponding container process identification;

Replace the amount of video memory in the container in the return result with the sum of the video memory used by all processes in the container;

Replace the total amount of video memory in the returned result with the video memory quota.

Preferably, the task of obtaining the video memory to be allocated includes:

Hijack the video memory allocation function and obtain the task.

Preferably, allocating video memory to the task based on the remaining video memory includes:

Analyze the task and obtain the requested video memory size;

Determine whether the remaining video memory is greater than or equal to the video memory size;

If so, call the video memory allocation function to allocate video memory to the task;

If not, it is determined that the memory has overflowed and the task returns with an error.

A video memory allocation device, including:

The task acquisition unit is used to obtain the tasks to be allocated to the video memory;

An identification conversion unit is used to obtain the container process identifier of each process in the container, and query the host process identifier of each process in the container according to the process identification relationship table;

A video memory usage acquisition unit, configured to obtain the video memory usage of each process in the container based on the host process identifier; superimpose the video memory usage of all processes in the container to obtain the total video memory usage of the container; using the The total amount of video memory used and the video memory quota of the container determine the remaining video memory of the container;

A video memory allocation unit, configured to allocate video memory to the task based on the remaining video memory.

An electronic device including:

Memory, used to store computer programs;

A processor, configured to implement the steps of the above video memory allocation method when executing the computer program.

A readable storage medium. A computer program is stored on the readable storage medium. When the computer program is executed by a processor, the steps of the above video memory allocation method are implemented.

Apply the method provided by the embodiment of the present invention to obtain the task to be allocated video memory; obtain the container process identifier of each process in the container, and query the host process identifier of each process in the container according to the process identifier relationship table; based on the host process identifier character to obtain the video memory usage of each process in the container; superimpose the video memory usage of all processes in the container to obtain the total video memory usage of the container; use the total video memory usage and the container's video memory quota to determine the remaining video memory that can be used by the container, and compare the remaining video memory based on the remaining video memory The task performs video memory allocation.

After obtaining the task to be allocated video memory, first obtain the container process identifier of each process in the container, and then find out the host process identifier of each process based on the identifier relationship table. The video memory usage of each process can be obtained by using the host process identifier. By accumulating the video memory usage of all processes in the container, the actual total video memory usage in the container can be obtained. Based on the total video memory usage and the video memory quota, the remaining video memory that can be used by the container can be determined. Based on the remaining video memory, video memory can be allocated to the task.

Technical effect: Obtain the video memory usage of each process by obtaining the host process identifier of all processes, and superimpose the video memory usage of all processes to obtain the accurate container video memory usage; in the process of obtaining the video memory usage, there is no need for the user to log in to the host to view it. There is no need to grant host permissions to users to avoid security risks; multi-card and multi-task scenarios can be effectively managed through the process identification relationship table. In other words, the present invention can realize effective and orderly allocation of video memory.

Correspondingly, embodiments of the present invention also provide a video memory allocation device, equipment and a readable storage medium corresponding to the above video memory allocation method, which have the above technical effects and will not be described again here.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings needed to be used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings in the following description are only for illustration purposes. For some embodiments of the invention, those of ordinary skill in the art can also obtain other drawings based on these drawings without exerting creative efforts.

Figure 1 is an implementation flow chart of a video memory allocation method in an embodiment of the present invention;

Figure 2 is a schematic diagram of a host + GPU software system;

Figure 3 is a schematic diagram of a host + GPU software system in an embodiment of the present invention;

Figure 4 is a schematic timing diagram of a specific implementation of a video memory allocation method in an embodiment of the present invention;

Figure 5 is a schematic structural diagram of a video memory allocation device in an embodiment of the present invention;

Figure 6 is a schematic structural diagram of an electronic device in an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Please refer to Figure 1. Figure 1 is a flow chart of a video memory allocation method in an embodiment of the present invention. This method can be applied to containers in a GPU+CPU computing architecture. The method includes the following steps:

S101. Obtain the task of the video memory to be allocated.

When the user submits a task that requires the use of the GPU, it can be determined that the task to be allocated video memory is obtained. For example, when the user submits a task that requires the use of the GPU to the container, it is determined that the task to be allocated video memory is obtained.

In a specific implementation manner of the present invention, obtaining the task of the video memory to be allocated includes: hijacking the video memory allocation function and obtaining the task. In other words, the task can be obtained by hijacking the video memory allocation function. For specific instructions on how to implement function hijacking, you can refer to the relevant hijacking methods and will not go into details here.

S102. Obtain the container process identifier of each process in the container, and query the host process identifier of each process in the container according to the process identifier relationship table.

Among them, the container is the container that currently receives the task to be allocated video memory. In the GPU+CPU computing architecture, when a container receives a task that requires graphics memory task allocation, it can obtain the container process identifier of each of its own processes, and then compare the process identifier relationship table to find the location of these processes in the host. Host process identifier.

It should be noted that for the same process, there is a unique identifier in the container, that is, the container process identifier, represented by pid1, and there is a unique identifier in the host, that is, the host process identifier, represented by pid2 (that is, host pid )express.

In the process identification relationship table, the container process identifier and host process identifier of each process can be stored in advance.

By querying the process identification relationship table, you can quickly obtain the host process identifier of each process in the container. Compared with the way the container communicates with the host, it is more convenient and faster to clarify the host process identifier of each process.

Specifically, the process identifier relationship table may be stored after the container communicates with the host after obtaining the host process identifier. You can also develop a custom character device to suspend the custom character device. By calling the corresponding interface, you can obtain the host process identifier for storage.

In a specific implementation manner of the present invention, obtaining the process identification relationship table includes:

Get the container process identifier of each process in the container;

Call the open function to open the mounted custom character device;

Call the calling interface function of the custom character device and enter the parameters to get the return result; where the parameters are the predefined instructions and the container process identifier of each process in the container, and the return result is the host process identifier of each process;

Write the container process identifier and the host process identifier of the same process into the process identification relationship table in pairs.

For the convenience of description, the above steps are combined for description below.

Generally, because the namespace isolation mechanism (linux pid namespace) is used in the container or pod (the smallest running unit for K8s to manage resources), the process (application) in the container cannot be seen in the host pid2 (pid, processID, process identifier) )). Based on this, in the embodiment of the present invention, a character device can be customized. The main function of the customized character device is to return the pid2 of the process on the host according to the input instruction and the GPU process pid1 in the container. Obtaining the host process identifier based on a custom character device is simple to implement and use, and can be easily integrated into the memory allocation control function library. Compared with the real-time communication method between the host and the container, it is easy to deploy and has good performance.

Among them, obtain the container process identifier of each process in the container, including:

Get the GPU card information in the container;

Use the GPU card information to obtain the container process identifiers of all processes running on the GPU card.

Specifically, the GPU card information can be obtained through the cuda context (the context of the unified computing device architecture), cuda uuid and GPUuuid, and then based on the GPU card information, the container process identifiers of all processes running on the GPU card can be obtained.

In actual applications, when a new task is created in the container, the process corresponding relationship of the new task has not been established in the relationship table at this time, so it is necessary to call a custom character device to create this relationship and update the relationship table. When the task process in the container ends, the information in that relationship table related to this process needs to be deleted. In this way, it can be ensured that the relationship information in the relationship table corresponds to the current process in real time.

S103. Based on the host process identifier, obtain the video memory usage of each process in the container.

After obtaining the host process identifier, the actual video memory usage of the corresponding process can be obtained based on the host process identifier. In other words, based on the host process identifier, the video memory usage of all processes in the container can be obtained.

In a specific implementation manner of the present invention, based on the host process identifier, the video memory usage of each process in the container is obtained, including:

Get the GPU card information in the container;

Use the GPU card information to obtain the video memory information of all processes running on the GPU card;

Find the video memory usage corresponding to the host process identifier from the video memory information.

Among them, the video memory information can be specifically how much video memory is used. For example, the process uses 10GB of video memory.

Specifically, when a container uses one or more GPU cards, the GPU card information can be obtained first, and then, based on the GPU card information, the video memory information of all processes running on the GPU card can be obtained. Then, find the video memory usage corresponding to the host process identifier from the video memory information. In other words, the video memory usage can be obtained through the relationship between the container and the GPU and the pid2 of the process.

For example, assume that there are two application processes in container A, and the pid1 in the container are 2 and 3 respectively; the corresponding pid2 on the host are 14456 and 14457 respectively. This relationship table can be recorded in the following two-dimensional table:

2 14456 3 14457

Since Nvml runs on the host, you can only get the video memory information used by 14456 and 14457. In container A, to get how much video memory has been used by the process of container A, you can first query the process through pid1=2 pid2=14456, and then obtain the video memory value used by 14456 through nvml (that is, the amount of video memory used). Pid1=3 queries the video memory value used by its process, so that you can get the total video memory used by all processes in container A.

S104. Add the video memory usage of all processes in the container to obtain the total video memory usage of the container.

The superposition of the video memory usage of all processes in the container is the total amount of video memory actually used by the container.

It should be noted that since the video memory usage of all processes in the container is superimposed, the superposition result is the total video memory usage of the actual container, rather than the GPU process video memory usage in the container that ignores the size of the implicitly allocated video memory. quantity.

S105. Determine the remaining video memory of the container using the total amount of video memory and the video memory quota of the container, and allocate video memory to the task based on the remaining video memory.

Among them, the video memory quota can be the size of the video memory allocated to the container when it is created. For example, when the video memory quota of the container is 8G, the total number of processes in the container can use up to 8G of video memory.

After obtaining the total video memory usage, the remaining video memory that can be used by the container can be determined by taking the difference between the total video memory usage and the container's video memory quota.

After knowing the remaining video memory of the container, you can allocate video memory to the task based on the remaining video memory.

In a specific implementation of the present invention, allocating video memory to tasks based on remaining video memory includes:

Analyze the task and obtain the requested memory size;

Determine whether the remaining video memory is greater than or equal to the video memory size;

If so, call the video memory allocation function to allocate video memory to the task;

If not, it is determined that the memory has overflowed and the task returns with an error.

That is, first determine the size of the video memory requested by the task. Then, compare the remaining video memory and the size of the video memory. If the remaining video memory is greater than or equal to the video memory size, it means that the current remaining video memory meets the allocation. At this time, the video memory allocation function can be called to allocate video memory to the task; if the remaining video memory is less than the video memory size, then It indicates that the current remaining video memory cannot meet the allocation, and the memory overflow can be determined, and OOM (out of memory, memory (video memory) overflow) can be returned.

Apply the method provided by the embodiment of the present invention to obtain the task to be allocated video memory; obtain the container process identifier of each process in the container, and query the host process identifier of each process in the container according to the process identifier relationship table; based on the host process identifier character to obtain the video memory usage of each process in the container; superimpose the video memory usage of all processes in the container to obtain the total video memory usage of the container; use the total video memory usage and the container's video memory quota to determine the remaining video memory that can be used by the container, and compare the remaining video memory based on the remaining video memory The task performs video memory allocation.

After obtaining the task to be allocated video memory, first obtain the container process identifier of each process in the container, and then find out the host process identifier of each process based on the identifier relationship table. The video memory usage of each process can be obtained by using the host process identifier. By accumulating the video memory usage of all processes in the container, the actual total video memory usage in the container can be obtained. Based on the total video memory usage and the video memory quota, the remaining video memory of the container can be determined. Based on the remaining video memory, video memory can be allocated to the task.

Technical effect: Obtain the video memory usage of each process by obtaining the host process identifier of all processes, and superimpose the video memory usage of all processes to obtain the accurate container video memory usage; in the process of obtaining the video memory usage, there is no need for the user to log in to the host to view it. There is no need to grant host permissions to users to avoid security risks; multi-card and multi-task scenarios can be effectively managed through the process identification relationship table. In other words, the present invention can realize effective and orderly allocation of video memory.

It should be noted that, based on the above embodiments, the embodiments of the present invention also provide corresponding improvement solutions. In the preferred/improved embodiments, the same steps or corresponding steps as in the above-mentioned embodiments may be referred to each other, and the corresponding beneficial effects may also be referred to each other. They will not be described one by one in the preferred/improved embodiments herein.

In a specific implementation manner of the present invention, the user can also be provided with a function of viewing the usage of container memory. The specific implementation process includes:

Use the GPU process display tool to read the process identification relationship table;

Get the video memory quota;

Get the number of GPUs and GPU unique identifier in the container;

Based on the number of GPUs and the unique identifier of the GPU, obtain the process information corresponding to each GPU in the container in a loop;

Match the process identification relationship table with the GPU process information returned by calling the management library function, and record the video memory usage of the host process identification;

Based on the process identification relationship table, perform parameter replacement processing on the results returned by the computer running information viewing function;

After completing the parameter replacement, the video memory usage information of the container is obtained; among which, the video memory usage information includes the video memory usage of each process, the video memory quota of the container, and the remaining capacity of the container;

Based on the process identification relationship table, parameter replacement processing is performed on the result returned by the computer running information viewing function, including:

Based on the process identification relationship table, replace the host process identification in the returned result with the corresponding container process identification;

Replace the amount of video memory in the container in the returned result with the sum of the video memory used by all processes in the container;

Replace the total amount of video memory in the returned result with the video memory quota.

For ease of description, the above steps are combined for description below.

In this embodiment, by analyzing the implementation logic and output format of nvidia-smi, an in-container GPU process display tool (inais-smi) can be customized, which reads the process identification relationship table of the GPU process in the storage module. The process pid2 returned by the nvml function is replaced with the corresponding pid1 in the container. At the same time, the used video memory in the container is replaced with the sum of the video memory used by all GPU processes in the container. The total video memory is replaced with the quota of the video memory in the container, which can realize the control of the video memory in the container. The statistics of GPU process information allows users to see their own GPU process information in the container, avoiding authorizing user host rights. Moreover, this method does not rely on changes in the nvidia-smi command, and is more stable than directly hijacking the nvidia-smi command. Output information can be customized.

Among them, nvml: NVIDIA Management Library, NVIDIA management library, is used to manage NVIDIA GPU devices. This article is also called the management library function.

In order to facilitate those skilled in the art to better understand and implement the video memory allocation method provided by the embodiments of the present invention, the video memory allocation method will be described in detail below with reference to specific application scenarios as examples.

Please refer to Figures 2 and 3. Figure 2 is a schematic diagram of a host + GPU software system; Figure 3 is a schematic diagram of a host + GPU software system in an embodiment of the present invention. That is, Figure 2 is a related host + GPU software system, and Figure 3 is a host + GPU software system that applies the video memory allocation method provided by the embodiment of the present invention. That is to say, the host+GPU software system that applies the video memory allocation method provided by the embodiment of the present invention has the following new modules compared to the host+GPU software system shown in Figure 2:

Among them, the custom character device module uses a namespace isolation mechanism in the container or pod, so the pid2 of the process cannot be seen in the container. The main function of the custom character device module is to return the pid2 of the process on the host based on the input instructions and the GPU process pid1 in the container.

About the isolation mechanism: The process identifier (pid) namespace isolation mechanism is used in Linux systems. Therefore, when an application/process is started in a container, the pid1 of the application can be seen in the container, for example, 1, but the pid2 of the application on the host (host) cannot be seen in the container.

GPU process pid storage module in the container. In the container, by calling the interface of the custom character device module, you can obtain the pid relationship of the GPU process and save the relationship table to the storage module. The GPU process pid storage module in the container is very important to the container. All GPU processes within it can be read and written, but cannot be accessed between different containers. Therefore, the container can be stored in shared memory or files. The storage module can avoid multiple parsing of the container GPU process.

Among them, the pid relationship is a relationship table that stores the process pid2 on the host corresponding to the pid1 of the application process in the container. That is, the pid relationship table can be specifically a relationship table. For example, the process pid1 in the container is 1, and the corresponding process pid2 on the host is 14456. This relationship table can be specifically: 1->14456. When used, the pid2 corresponding to pid1 can be easily obtained by looking up the table, and there is no need to do it separately every time. parse.

The main function of the GPU process memory control module in the container is to control the video memory of all GPU processes in the container to not exceed the container's video memory quota. Specifically, it hijacks the video memory allocation function in the cuda driver. When calling the video memory allocation function, first use the cuda context, cuda uuid and GPU uuid to obtain the GPU card to which video memory is to be allocated. The GPU information obtained by this method is accurate and is suitable for applications where there are files in the container. In the case of multiple GPU cards, the corresponding function of nvml is then used to obtain the video memory information of all running tasks on the GPU card. Finally, the process identification relationship table in the storage module is used, combined with all task information of the GPU card, to obtain the GPU process in the container. Video memory information; by adding up the video memory usage of all GPU processes in the container, you can accurately obtain the GPU usage in the container. This video memory includes implicitly allocated video memory such as the cuda context of each GPU process, and is combined with the container video memory quota and the video memory to be allocated. size to determine whether the remaining video memory is satisfied. If it is satisfied, the real cudadriver function will be called to complete the allocation. If it is not satisfied, OOM will be returned.

GPU process statistics module in the container. By analyzing the implementation logic and output format of nvidia-smi, we customize a GPU process display tool in the container, namely inais-smi, which reads the pid relationship table of the GPU process in the storage module (process identification relationship table, also referred to in this article as (Relationship table), replace the process pid2 returned by the nvml function with the corresponding pid1 in the container, and replace the used video memory in the container with the sum of the video memory used by all GPU processes in the container, and replace the total video memory with the quota of video memory in the container , realizes the statistics of GPU process information in the container, so that users can see their own GPU process information in the container, avoiding authorizing user host permissions. This method does not rely on changes in the nvidia-smi command, and is better than directly hijacking nvidia-smi The command is more stable and the output information can be customized.

During specific implementation, you can follow the following steps for deployment and implementation.

First, customize the character device module, that is, write the character device module processing logic, mainly define the unlocked_ioctl (custom character device call interface function) interface, and implement the logic of obtaining the host pid in the container through the kernel function. When deploying the resource management platform, you can compile and generate the corresponding ko file (linux kernel module file, such as vcuda_dev.ko) according to the specific system kernel version module, and use the insmod command (linux kernel module file installation command) to install the device module .

Mount the vcuda_dev character device. Specifically, when the user creates a container or POD through the resource management platform, the character device is mounted into the corresponding container. The corresponding docker parameter is --device(docker command, for example: docker run- it--device/dev/vcuda_dev<container image>).

Mount the GPU process statistics tool in the container. It can analyze the implementation logic and output format of nvidia-smi, customize a GPU process display tool in a container, and compile and generate executable files, such as inais-smi. When users create a container or POD through the resource management platform, they mount the inais-smi executable file to the /usr/bin (a directory address within the container) directory, and then the user can execute the inais-smi command in any directory within the container. .

Mount the video memory allocation control function library and set priority loading logic. Redefine the video memory allocation related functions in the cuda driver, add the video memory control logic at the beginning of the video memory allocation function, and compile the entire video memory allocation control logic into a so dynamic library, such as libvcuda.so (the name of the video memory allocation control logic dynamic library); Users create a container or POD through the resource management platform, mount it into the container, and use parameters such as ld.so.prelod or LD_PRELOAD (linux internal command) to set the priority loading logic of the video memory allocation control function library.

Please refer to FIG. 4 , which is a schematic timing diagram of a specific implementation of a video memory allocation method in an embodiment of the present invention. Task startup, acquisition and storage of GPU process pid table. When the user starts a GPU task in a container or POD, due to the priority loading logic of the video memory allocation control function library, the custom GPU process pid acquisition logic will be triggered; first, the open function will be called to open the mounted vcuda_dev character device , secondly call the ioctl function, the parameters are the pre-defined instructions and the pid of the GPU process in the container, the returned result is the host pid of the process, and finally the pid relationship table of the GPU process is stored in the shared module.

GPU process memory control in the container: When the GPU task runs and calls the GPU memory allocation function, because the priority loading logic of libvcuda.so is set, the memory allocation function in libvcuda.so will be called first, thus triggering the memory control logic.

That is to say, in the present invention, the custom character device module can be used to obtain the host pid of the GPU process by calling the custom ioctl instruction in the container, which is simple and efficient; by hijacking the video memory allocation function of the cuda driver, using cuda The conversion relationship between context, cuda uuid and GPU uuid solves the problem of memory control of multiple GPU cards and multi-GPU tasks in the container; and based on the GPU process pid relationship table in the container storage module, by hijacking the output of the nvml function and the host and container The conversion strategy of inter-process pid solves the problem that users cannot obtain statistical information of each GPU process in the container.

In addition, the container GPU process pid relationship table storage and filtering module; this storage module can be read and written to each GPU process in the container. Specifically, shared memory, files, etc. can be used. The storage unit stores the pid of the GPU process in the container. The relational table avoids multiple parsing of the GPU process in the container; that is, the filtering module compares the obtained pid information and deletes the pid information corresponding to the processes that are not running in the container. In other words, the filtering module can remove invalid pid information and ensure that the information in the storage unit is consistent with the actually running program. For example, you can determine whether the host pid in the container is located in nvml_table. If it exists, the GPU information of the corresponding process in nvml_table will be retained. If it does not exist, it means that the task/process has ended and it will be deleted from the storage module.

When implementing the GPU card identification strategy when allocating video memory, you first need to determine the GPU card for this video memory allocation. When there are multiple GPU cards in the container, the task may run on multiple GPU cards; in another case, if If the user starts the task and sets the CUDA_VISIBLE_DEVICES parameter, the simple GPU index may fail. By applying the methods and strategies provided by the embodiments of the present invention, the cuda context of the current video memory allocation is obtained, and then the cuda context is used to obtain the cuda uuid. Finally, the conversion relationship between the cuda uuid and the GPU uuid is used to obtain the GPUuuid of the video memory allocation. , according to the GPU uuid, the nvml function can be called to obtain more GPU card information.

Video memory usage statistics policy. When allocating video memory, you need to know the usage information of the video memory on the corresponding GPU card in the container. Since it is impossible to know the size of the implicitly allocated video memory such as the CUDA context, but ignoring this part of the video memory can easily cause the container quota to exceed the quota, the method and strategy provided by the embodiment of the present invention first obtain the GPU through the GPU card identification strategy when controlling the video memory allocation. uuid, then call the nvmlDeviceGetHandleByUUID function to obtain the nvidia device, and finally call the nvmlDeviceGetComputeRunningProcesses function (a function to view process information) through the nvidia device to obtain the GPU process information of all users on the corresponding GPU card. This information is maintained by the nvidia driver, and the video memory size is very accurate. Then combined with the GPU process pid relationship table in the container storage module, by matching the host pids of the two, the total amount of video memory used by all GPU processes in the container can be accurately obtained.

Container GPU process statistics and refresh strategy, record the GPU information returned by the nvmlDeviceGetComputeRunningProcesses function as nvml_table; obtain the GPU process pid relationship table from the storage module in the container, determine whether the host pid in the container is located in nvml_table, if it exists, keep the corresponding one in nvml_table If the GPU information of the process does not exist, it means that the task has ended, delete it from the storage module, and finally write it back to the storage module for refresh.

That is to say, the methods and strategies provided by the embodiments of the present invention can use the custom character device module to obtain the host pid of the GPU process in the container, and use the relationship between cuda context, cuda uuid and GPU uuid when allocating video memory. The information of the real GPU card was obtained, which solved the problem of multi-GPU card tasks and CUDA_VISIBLE_DEVICES special variables in the container; at the same time, the pid relationship table and nvml function in the storage module were used to obtain the video memory occupation information of the corresponding GPU card in the container, realizing Precise control of the video memory in the container; finally, by hijacking the output of the nvml function and the process pid conversion strategy between the host and the container, the problem of being unable to count and display GPU process information in the container is solved.

Corresponding to the above method embodiments, embodiments of the present invention also provide a video memory allocation device. The video memory allocation device described below and the video memory allocation method described above can be referenced correspondingly.

As shown in Figure 5, the device includes the following modules:

Task acquisition unit 101, used to acquire tasks to be allocated video memory;

The identification conversion unit 102 is used to obtain the container process identifier of each process in the container, and query the host process identifier of each process in the container according to the process identification relationship table;

The video memory usage acquisition unit 103 is used to obtain the video memory usage of each process in the container based on the host process identifier; superimpose the video memory usage of all processes in the container to obtain the total video memory usage of the container; determine the total video memory usage and the video memory quota of the container. The remaining video memory of the container;

The video memory allocation unit 104 is used to allocate video memory to tasks based on remaining video memory.

Apply the device provided by the embodiment of the present invention to obtain the task to be allocated video memory; obtain the container process identifier of each process in the container, and query the host process identifier of each process in the container according to the process identifier relationship table; based on the host process identifier character to obtain the video memory usage of each process in the container; superimpose the video memory usage of all processes in the container to obtain the total video memory usage of the container; use the total video memory usage and the container's video memory quota to determine the remaining video memory that can be used by the container, and compare the remaining video memory based on the remaining video memory The task performs video memory allocation.

After obtaining the task to be allocated video memory, first obtain the container process identifier of each process in the container, and then find out the host process identifier of each process based on the identifier relationship table. The video memory usage of each process can be obtained by using the host process identifier. By accumulating the video memory usage of all processes in the container, the actual total video memory usage in the container can be obtained. Based on the total video memory usage and the video memory quota, the remaining video memory of the container can be determined. Based on the remaining video memory, video memory can be allocated to the task.

Technical effect: Obtain the video memory usage of each process by obtaining the host process identifier of all processes, and superimpose the video memory usage of all processes to obtain the accurate container video memory usage; in the process of obtaining the video memory usage, there is no need for the user to log in to the host to view it. There is no need to grant host permissions to users to avoid security risks; multi-card and multi-task scenarios can be effectively managed through the pid relationship table. In other words, the present invention can realize effective and orderly allocation of video memory.

In a specific implementation of the present invention, the table creation unit is used to obtain the process identification relationship table, including: obtaining the container process identifier of each process in the container; calling the open function to open the mounted custom character device; calling Define the calling interface function of the character device, and enter the parameters to get the return result; among them, the parameters are the predefined instructions and the container process identifier of each process in the container, and the return result is the host process identifier of each process; the same process's The container process identifier and the host process identifier are written into the process identification relationship table in pairs.

In a specific implementation of the present invention, the table creation unit is specifically used to obtain the GPU card information in the container;

Use the GPU card information to obtain the container process identifiers of all processes running on the GPU card.

In a specific implementation of the present invention, the video memory usage acquisition unit is used to acquire GPU card information in the container;

Use the GPU card information to obtain the video memory information of all processes running on the GPU card;

Find the video memory usage corresponding to the host process identifier from the video memory information.

In a specific implementation of the present invention, the video memory viewing unit is used to use the GPU process display tool to read the process identification relationship table;

Get the video memory quota;

Get the number of GPUs and GPU unique identifier in the container;

Based on the number of GPUs and the unique identifier of the GPU, obtain the process information corresponding to each GPU in the container in a loop;

Match the process identification relationship table with the GPU process information returned by calling the management library function, and record the video memory usage of the host process identification;

Based on the process identification relationship table, perform parameter replacement processing on the results returned by the computer running information viewing function;

After completing the parameter replacement, the video memory usage information of the container is obtained; among which, the video memory usage information includes the video memory usage of each process, the video memory quota of the container, and the remaining capacity of the container;

Based on the process identification relationship table, parameter replacement processing is performed on the result returned by the computer running information viewing function, including:

Based on the process identification relationship table, replace the host process identification in the returned result with the corresponding container process identification;

Replace the amount of video memory in the container in the returned result with the sum of the video memory used by all processes in the container;

Replace the total amount of video memory in the returned result with the video memory quota.

In a specific implementation manner of the present invention, the task acquisition unit is specifically used to hijack the video memory allocation function and acquire the task.

In a specific implementation of the present invention, the video memory allocation unit is specifically used to parse the task and obtain the requested video memory size to be allocated;

Determine whether the remaining video memory is greater than or equal to the video memory size;

If so, call the video memory allocation function to allocate video memory to the task;

If not, it is determined that the memory has overflowed and the task returns with an error.

Corresponding to the above method embodiments, embodiments of the present invention also provide an electronic device. The electronic device described below and the video memory allocation method described above may be mutually referenced.

As shown in Figure 6, the electronic device includes:

Memory 332 for storing computer programs;

The processor 322 is configured to implement the steps of the video memory allocation method of the above method embodiment when executing the computer program.

In a specific implementation, the electronic device may include a processor and a memory, and the electronic device may be connected to a GPU (such as a plug-in GPU).

In another specific implementation, the electronic device may include a processor, a memory and a GPU (not shown in Figure 6). Specifically, please refer to Figure 7. Figure 7 is a specific structural schematic diagram of an electronic device provided in this embodiment. The electronic device may vary greatly due to different configurations or performance, and may include one or more processors ( central processing units (CPU) 322 (eg, one or more processors), memory 332 and GPU (not shown in FIG. 7 ). The memory 332 stores one or more computer programs 342 or data 344 . Among them, the memory 332 may be short-term storage or persistent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute a series of instruction operations in the memory 332 on the electronic device 301 .

Electronic device 301 may also include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input/output interfaces 358 , and/or, one or more operating systems 341 .

The steps in the video memory allocation method described above can be implemented by the structure of the electronic device.

Corresponding to the above method embodiments, embodiments of the present invention also provide a readable storage medium. The readable storage medium described below and the video memory allocation method described above may be mutually referenced.

A readable storage medium. A computer program is stored on the readable storage medium. When the computer program is executed by a processor, the steps of the video memory allocation method of the above method embodiment are implemented.

The readable storage medium can specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk that can store program codes. readable storage media.

Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on its differences from other embodiments. The same or similar parts between various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For relevant details, please refer to the description in the method section.

Those skilled in the art may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of both. In order to clearly illustrate the hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described according to functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may implement the described functionality using different methods for each specific application, but such implementations should not be considered to be beyond the scope of the present invention.

The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of both. Software modules may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or anywhere in the field of technology. any other known form of storage media.

Finally, it should be noted that in this article, relationships such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or any such actual relationship or sequence between operations. Furthermore, the terms include, include, or any variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or It also includes elements inherent to the process, method, article or equipment.

This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and the core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the present invention There will be changes in the specific implementation and scope of application of the ideas. In summary, the content of this description should not be understood as limiting the present invention.

Claims (10)

1. A video memory allocation method is characterized by comprising the following steps:

Acquiring a task of a video memory to be allocated;

acquiring a container process identifier of each process in a container, and inquiring a host process identifier of each process in the container according to a process identifier relation table;

based on the host process identifier, obtaining the video memory consumption of each process in the container;

superposing the video memory consumption of all processes in the container to obtain the total video memory consumption of the container;

and determining the residual video memory of the container by using the total video memory consumption and the video memory quota of the container, and performing video memory allocation on the task based on the residual video memory.

2. The method of claim 1, wherein obtaining the process identification relationship table comprises:

acquiring a container process identifier of each process in the container;

calling an opening function to open the mounted custom character equipment;

calling the calling interface function of the custom character equipment, and inputting parameters to obtain a return result; the parameter is a predefined instruction and a container process identifier of each process in the container, and the returned result is a host process identifier of each process;

and writing the container process identifier and the host process identifier of the same process into the process identification relation table in pairs.

3. The method of claim 2, wherein the obtaining a container process identifier for each process within the container comprises:

acquiring GPU card information in the container;

and acquiring the container process identifiers of all processes running on the GPU card by utilizing the GPU card information.

4. The method of claim 1, wherein obtaining the memory usage of each process in the container based on the host process identifier comprises:

acquiring GPU card information in the container;

acquiring the video memory information of all processes running on the GPU card by utilizing the GPU card information;

and finding out the video memory quantity corresponding to the host process identifier from the video memory information.

5. The method as recited in claim 1, further comprising:

reading the process identification relation table by using a GPU process display tool;

acquiring the video memory quota;

acquiring the quantity of the GPUs and the unique identifiers of the GPUs in the container;

based on the number of GPUs and the unique identifier of the GPU, circularly acquiring each process information corresponding to each GPU in the container;

matching the process identification relation table with GPU process information returned by calling a management library function, and recording the memory consumption of a host process identification;

Based on the process identification relation table, parameter replacement processing is carried out on a return result of the computer running information checking function;

after parameter replacement is completed, obtaining the video memory use information of the container; the video memory usage information comprises the video memory consumption of each process, the video memory quota of the container and the residual capacity of the container;

based on the process identification relation table, parameter replacement processing is carried out on a return result of the computer running information checking function, and the method comprises the following steps:

replacing the host process identifier in the returned result with a corresponding container process identifier based on the process identifier relation table;

replacing the memory consumption in the container in the returned result with the sum of the memory consumption of all processes in the container;

and replacing the total amount of the video memory in the returned result with the video memory quota.

6. The method of claim 1, wherein the task of obtaining the memory to be allocated comprises:

hijacking a video memory allocation function to acquire the task.

7. The method of any of claims 1 to 6, wherein allocating the task based on the remaining memory comprises:

Analyzing the task to obtain the size of the video memory which is requested to be allocated;

judging whether the residual video memory is larger than or equal to the video memory size;

if yes, calling a video memory allocation function, and allocating video memory to the task;

if not, determining that the memory overflows, and reporting the task by mistake and returning.

8. A memory allocation apparatus, comprising:

the task acquisition unit is used for acquiring tasks of the video memory to be allocated;

the device comprises an identification conversion unit, a storage unit and a storage unit, wherein the identification conversion unit is used for acquiring a container process identifier of each process in a container and inquiring a host process identifier of each process in the container according to a process identification relation table;

the memory amount obtaining unit is used for obtaining the memory amount of each process in the container based on the host process identifier; superposing the video memory consumption of all processes in the container to obtain the total video memory consumption of the container; determining the residual video memory of the container by using the total video memory consumption and the video memory quota of the container;

and the video memory distribution unit is used for distributing the video memory to the task based on the residual video memory.

9. An electronic device, comprising:

a memory for storing a computer program;

A processor for implementing the steps of the memory allocation method according to any one of claims 1 to 7 when executing the computer program.

10. A readable storage medium, wherein a computer program is stored on the readable storage medium, the computer program implementing the steps of the video memory allocation method according to any one of claims 1 to 7 when executed by a processor.