CN113553195A - A method, apparatus, device and readable medium for sharing memory pool resources - Google Patents
A method, apparatus, device and readable medium for sharing memory pool resources Download PDFInfo
- Publication number
- CN113553195A CN113553195A CN202111108280.4A CN202111108280A CN113553195A CN 113553195 A CN113553195 A CN 113553195A CN 202111108280 A CN202111108280 A CN 202111108280A CN 113553195 A CN113553195 A CN 113553195A
- Authority
- CN
- China
- Prior art keywords
- pool
- resources
- sub
- pools
- parent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5011—Pool
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明提供了一种内存池资源共用的方法、装置、设备及可读介质，该方法包括：创建全局唯一父池，并使用每个CPU的物理内存分别创建子池，并将每个子池加入到父池中；响应于子池的资源不足，资源不足的子池向父池发出资源转移的申请；响应于父池接收到申请，父池查询每个子池的剩余资源；响应于查询到其他子池的剩余资源数满足申请中的要求，父池将其他子池的剩余资源转移给资源不足的子池使用。通过使用本发明的方案，能够避免在CPU间资源使用不均衡时的资源浪费，提升数据处理能力，大幅提升系统整体性能。
The present invention provides a method, device, device and readable medium for sharing memory pool resources. The method includes: creating a globally unique parent pool, using the physical memory of each CPU to create sub-pools respectively, and adding each sub-pool to the to the parent pool; in response to insufficient resources in the child pool, the child pool with insufficient resources sends a resource transfer application to the parent pool; in response to the parent pool receiving the application, the parent pool queries the remaining resources of each child pool; in response to the query to other The remaining resources of the sub-pool meet the requirements in the application, and the parent pool transfers the remaining resources of other sub-pools to the sub-pools with insufficient resources for use. By using the solution of the present invention, it is possible to avoid resource waste when resources are used unbalanced among CPUs, improve data processing capability, and greatly improve the overall performance of the system.
Description
Technical Field
The field relates to the field of computers, and more particularly to a method, an apparatus, a device and a readable medium for sharing memory pool resources.
Background
In the NUMA architecture, the memory has the attribution of the CPU, the speed of the CPU accessing the memory of the CPU is high, and the speed of the CPU accessing the memories of other CPUs is much slower than that of the CPU accessing the memory of the CPU. Based on the above-mentioned problem of speed imbalance of CPU accessing memory, the current memory technology generally adopts the following scheme: that is, each CPU has a corresponding task pool and memory pool, and the memory pool uses the CPU's own physical memory.
All tasks are firstly put into a task pool corresponding to the CPU to wait for execution, and the tasks in the task pool can only use the own memory pool and cannot use other memory pools. The scheme makes full use of the characteristic that the CPU has high speed of accessing the memory of the CPU, and can greatly improve the performance. However, problems are inevitably introduced. The CPU may originally use all the memories in the physical layer, however, after the memory pool technology is used to isolate the access of the CPU to the memory, the CPU cannot use the memories to which other CPUs belong, and a serious resource waste may occur in a scene of task imbalance.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a device, and a readable medium for sharing memory pool resources, which can avoid resource waste when resource usage between CPUs is unbalanced, improve data processing capability, and greatly improve overall performance of a system.
In view of the above, an aspect of the embodiments of the present invention provides a method for sharing memory pool resources, including the following steps:
creating a globally unique parent pool, creating a child pool by using a physical memory of each CPU, and adding each child pool into the parent pool;
responding to the resource shortage of the child pool, and sending a resource transfer application from the child pool with the resource shortage to the parent pool;
in response to the parent pool receiving the application, the parent pool queries the remaining resources of each child pool;
and responding to the query that the residual resource number of other sub-pools meets the requirement in the application, and transferring the residual resources of other sub-pools to the sub-pools with insufficient resources by the parent pool for use.
According to an embodiment of the present invention, creating a globally unique parent pool, and creating child pools using a physical memory of each CPU, respectively, and adding each child pool to the parent pool includes:
respectively creating a sub-pool by using a physical memory of each CPU;
converting the sub-pool of each physical memory into a linear logic space through a map mechanism of a linux kernel;
cutting each linear logic space according to a preset granularity size to form a page pool;
allocating page numbers to each page pool and setting a resource pointer for each page pool;
and adding the page number and the resource pointer of each page pool into the parent pool.
According to an embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the parent pool inquires that the child pool with the most residual resources meets the requirement in the application, and sending the page number of the residual page pool in the child pool with the most residual resources and the pointer of the residual resources to the child pool with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using the corresponding resources according to the position of the pointer.
According to an embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the total number of the remaining resources of other sub-pools meets the requirement in the application, and respectively sending the page numbers of the remaining page pools in the sub-pools with the remaining resources and the pointers of the remaining resources to the sub-pools with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using corresponding resources according to the positions of the pointers.
According to an embodiment of the present invention, further comprising:
and responding to the successful disk-dropping of the sub-pool with insufficient resources by using the data in the residual resources of other sub-pools, and transferring the residual resources back to other sub-pools.
According to one embodiment of the invention, the predetermined granularity level is a 4K level.
According to one embodiment of the present invention, in response to the parent pool receiving the request, the parent pool querying the remaining resources of each child pool comprises:
in response to the parent pool receiving the application, the parent pool queries the number of unused page pools in each child pool;
and adding all the resources of the queried page pool number.
In another aspect of the embodiments of the present invention, there is also provided an apparatus for sharing memory pool resources, where the apparatus includes:
the creating module is configured to create a globally unique parent pool, create child pools by using the physical memory of each CPU, and add each child pool into the parent pool;
the application module is configured to respond to the resource shortage of the child pool, and the child pool with the resource shortage sends an application for resource transfer to the parent pool;
the query module is configured to respond to the parent pool receiving the application, and the parent pool queries the residual resources of each child pool;
and the transfer module is configured to respond to the condition that the number of the residual resources of the other sub-pools meets the requirements in the application, and the parent pool transfers the residual resources of the other sub-pools to the sub-pools with insufficient resources for use.
In another aspect of an embodiment of the present invention, there is also provided a computer apparatus including:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of any of the methods described above.
In another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any one of the above-mentioned methods.
The invention has the following beneficial technical effects: in the method for sharing the memory pool resources provided by the embodiment of the invention, the globally unique parent pool is created, the physical memory of each CPU is used for respectively creating the child pools, and each child pool is added into the parent pool; responding to the resource shortage of the child pool, and sending a resource transfer application to the parent pool by the child pool with the resource shortage; in response to the parent pool receiving the application, the parent pool queries the remaining resources of each child pool; in response to the fact that the number of the remaining resources of the other child pools meets the requirements in the application, the parent pool transfers the remaining resources of the other child pools to the child pools with insufficient resources for use, resource waste caused by resource usage imbalance among CPUs can be avoided, data processing capacity is improved, and overall performance of the system is greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram of a method for sharing memory pool resources according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating an apparatus for sharing memory pool resources according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a computer device according to one embodiment of the present invention;
fig. 4 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
In view of the foregoing, a first aspect of the embodiments of the present invention provides an embodiment of a method for sharing memory pool resources. Fig. 1 shows a schematic flow diagram of the method.
As shown in fig. 1, the method may include the steps of:
s1 creates a globally unique parent pool, and creates child pools using the physical memory of each CPU, respectively, and adds each child pool to the parent pool.
Each CPU has its own physical memory, and the physical memory of each CPU is used to create its own sub-pool, each sub-pool of physical memory is converted into a linear logical space by the map mechanism of the linux kernel, then cutting each linear logic space according to a preset granularity size to form a page pool, namely, each CPU creates a sub-pool which is divided into a plurality of page pools with preset sizes, then allocates page numbers to each page pool and sets resource pointers for each page pool, adds the page numbers and the resource pointers of each page pool to a parent pool, the parent pool is a set of a plurality of child pools, the parent pool can acquire resource information in each child pool, the parent pool can acquire which page pool in which child pool has the free or busy resources through the number of each child pool and the number and resource pointer of the page pool in each child pool. When the other sub-pools have insufficient resources to process data or tasks, the parent pool can query whether the other sub-pools have enough resources to help the sub-pools with insufficient resources to process data. An array corresponding to the child pool can be established in the parent pool, the information of the child pool, the number of the page pool corresponding to the child pool and the resource pointer are added into the array, and the parent pool marks various states of the child pool through the array.
S2 responds to the resource shortage of the sub-pool, the sub-pool with the resource shortage sends a resource transfer application to the parent pool.
When the child pool executes tasks or processes data and finds that resources in the child pool to which the child pool belongs are insufficient, the child pool initiates an application for resource transfer to the parent pool, wherein the resource transfer is not really transfer, but processes the data in the child pool by using the resources in other child pools.
S3 in response to the parent pool receiving the request, the parent pool queries each child pool for remaining resources.
After the parent pool receives the application of the child pool, the parent pool queries the number of unused page pools in each child pool, the amount of the resources left in each child pool can be calculated according to the number of the page pools and the size of each page pool, and then the total number of the resources left in other child pools can be obtained by adding the resources of the number of the queried page pools.
S4, in response to the query that the remaining resource number of other sub-pools meets the requirement in the application, the parent pool transfers the remaining resources of other sub-pools to the sub-pool with insufficient resources for use.
The application has the required resource number, if the total number of the remaining resources in other sub-pools calculated by a parent pool is less than the required resource number, no resource transfer is performed, if the total number of the remaining resources in other sub-pools calculated by the parent pool is greater than the required resource number, whether the remaining resources in one sub-pool with the most remaining resources are greater than the required resource number is calculated, if the remaining resources in one sub-pool with the most remaining resources are greater than the required resource number, the remaining resources in one sub-pool with the most remaining resources are transferred to the sub-pool which sends the application for use, the page number of the remaining page pool with the most remaining resources and the pointer of the remaining resources are sent to the sub-pool which sends the application for use, the sub-pool processes the data to be processed by using the corresponding resources according to the position of the pointer, and the page number and the pointer of the transferred page pool in the parent pool need to be correspondingly marked, the resources need to be released after the transferred resources are used up, with corresponding marking in the parent pool. In some embodiments, if the total number of the remaining resources in the other child pools calculated by the parent pool is greater than the number of the required resources, the remaining resources in all the child pools having the remaining resources are transferred to the child pool issuing the application according to the proportion of the remaining resources for use, the page numbers of the remaining page pools having the remaining resources and the pointers of the remaining resources are respectively sent to the child pool issuing the application, the child pool processes the data to be processed by using the corresponding resources according to the positions of the pointers, the page numbers and the pointers of the transferred page pools need to be correspondingly marked in the parent pool, the resources need to be released after the transferred resources are used up, and the corresponding marks are made in the parent pool at the same time.
By the technical scheme, resource waste caused by unbalanced resource use among CPUs can be avoided, data processing capacity is improved, and overall performance of the system is greatly improved.
In a preferred embodiment of the present invention, creating a globally unique parent pool, and creating child pools using the physical memory of each CPU, respectively, and adding each child pool to the parent pool comprises:
respectively creating a sub-pool by using a physical memory of each CPU;
converting the sub-pool of each physical memory into a linear logic space through a map mechanism of a linux kernel;
cutting each linear logic space according to a preset granularity size to form a page pool;
allocating page numbers to each page pool and setting a resource pointer for each page pool;
and adding the page number and the resource pointer of each page pool into the parent pool. Each CPU has its own physical memory, and the physical memory of each CPU is used to create its own sub-pool, each sub-pool of physical memory is converted into a linear logical space by the map mechanism of the linux kernel, then cutting each linear logic space according to a preset granularity size to form a page pool, i.e., the sub-pool created by each CPU is divided into several page pools of a preset size, which can be set on demand, preferably 4KB, then allocating page numbers to each page pool and setting resource pointers for each page pool, adding the page numbers and the resource pointers of each page pool into a parent pool, the parent pool is a set of a plurality of child pools, the parent pool can acquire resource information in each child pool, the parent pool can acquire which page pool in which child pool has the free or busy resources through the number of each child pool and the number and resource pointer of the page pool in each child pool. When the other sub-pools have insufficient resources to process data or tasks, the parent pool can query whether the other sub-pools have enough resources to help the sub-pools with insufficient resources to process data. An array corresponding to the child pool can be established in the parent pool, the information of the child pool, the number of the page pool corresponding to the child pool and the resource pointer are added into the array, and the parent pool marks various states of the child pool through the array.
In a preferred embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the parent pool inquires that the child pool with the most residual resources meets the requirement in the application, and sending the page number of the residual page pool in the child pool with the most residual resources and the pointer of the residual resources to the child pool with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using the corresponding resources according to the position of the pointer.
In a preferred embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the total number of the remaining resources of other sub-pools meets the requirement in the application, and respectively sending the page numbers of the remaining page pools in the sub-pools with the remaining resources and the pointers of the remaining resources to the sub-pools with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using corresponding resources according to the positions of the pointers. And if the total number of the residual resources in the other sub-pools calculated by the parent pool is greater than the required number of the resources, transferring the residual resources in all the sub-pools with the residual resources to the sub-pool which sends the application according to the proportion of the residual resources for use.
In a preferred embodiment of the present invention, the method further comprises:
and in response to the sub-pool with insufficient resources successfully using the data in the remaining resources of the other sub-pool, transferring the remaining resources of the other sub-pool to the other sub-pool.
In a preferred embodiment of the present invention, the predetermined particle size is 4K size.
In a preferred embodiment of the present invention, in response to the parent pool receiving the request, the parent pool querying the remaining resources of each child pool comprises:
in response to the parent pool receiving the application, the parent pool queries the number of unused page pools in each child pool;
and adding all the resources of the queried page pool number.
By the technical scheme, resource waste caused by unbalanced resource use among CPUs can be avoided, data processing capacity is improved, and overall performance of the system is greatly improved.
It should be noted that, as will be understood by those skilled in the art, all or part of the processes in the methods of the above embodiments may be implemented by instructing relevant hardware through a computer program, and the above programs may be stored in a computer-readable storage medium, and when executed, the programs may include the processes of the embodiments of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the method disclosed according to an embodiment of the present invention may also be implemented as a computer program executed by a CPU, and the computer program may be stored in a computer-readable storage medium. The computer program, when executed by the CPU, performs the above-described functions defined in the method disclosed in the embodiments of the present invention.
In view of the above, according to a second aspect of the embodiments of the present invention, there is provided an apparatus for sharing memory pool resources, as shown in fig. 2, the apparatus 200 includes:
the creating module 201, the creating module 201 is configured to create a globally unique parent pool, create child pools by using the physical memory of each CPU, and add each child pool to the parent pool;
an application module 202, wherein the application module 202 is configured to respond to the resource shortage of the sub-pool, and send an application for resource transfer from the sub-pool with the resource shortage to the parent pool;
the query module 203, the query module 203 configured to respond to the parent pool receiving the application, the parent pool querying the remaining resources of each child pool;
and the transfer module 204, wherein the transfer module 204 is configured to respond to the query that the number of the remaining resources of the other sub-pools meets the requirement in the application, and the parent pool transfers the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device. Fig. 3 is a schematic diagram of an embodiment of a computer device provided by the present invention. As shown in fig. 3, an embodiment of the present invention includes the following means: at least one processor S21; and a memory S22, the memory S22 storing computer instructions S23 executable on the processor, the instructions when executed by the processor implementing the method of:
creating a globally unique parent pool, creating a child pool by using a physical memory of each CPU, and adding each child pool into the parent pool;
responding to the resource shortage of the child pool, and sending a resource transfer application from the child pool with the resource shortage to the parent pool;
in response to the parent pool receiving the application, the parent pool queries the remaining resources of each child pool;
and responding to the query that the residual resource number of other sub-pools meets the requirement in the application, and transferring the residual resources of other sub-pools to the sub-pools with insufficient resources by the parent pool for use.
In a preferred embodiment of the present invention, creating a globally unique parent pool, and creating child pools using the physical memory of each CPU, respectively, and adding each child pool to the parent pool comprises:
respectively creating a sub-pool by using a physical memory of each CPU;
converting the sub-pool of each physical memory into a linear logic space through a map mechanism of a linux kernel;
cutting each linear logic space according to a preset granularity size to form a page pool;
allocating page numbers to each page pool and setting a resource pointer for each page pool;
and adding the page number and the resource pointer of each page pool into the parent pool.
In a preferred embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the parent pool inquires that the child pool with the most residual resources meets the requirement in the application, and sending the page number of the residual page pool in the child pool with the most residual resources and the pointer of the residual resources to the child pool with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using the corresponding resources according to the position of the pointer.
In a preferred embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the total number of the remaining resources of other sub-pools meets the requirement in the application, and respectively sending the page numbers of the remaining page pools in the sub-pools with the remaining resources and the pointers of the remaining resources to the sub-pools with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using corresponding resources according to the positions of the pointers.
In a preferred embodiment of the present invention, the method further comprises:
and in response to the sub-pool with insufficient resources successfully using the data in the remaining resources of the other sub-pool, transferring the remaining resources of the other sub-pool to the other sub-pool.
In a preferred embodiment of the present invention, the predetermined particle size is 4K size.
In a preferred embodiment of the present invention, in response to the parent pool receiving the request, the parent pool querying the remaining resources of each child pool comprises:
in response to the parent pool receiving the application, the parent pool queries the number of unused page pools in each child pool;
and adding all the resources of the queried page pool number.
In view of the above object, a fourth aspect of the embodiments of the present invention proposes a computer-readable storage medium. FIG. 4 is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. As shown in fig. 4, the computer readable storage medium S31 stores a computer program S32 that when executed by a processor performs the method of:
creating a globally unique parent pool, creating a child pool by using a physical memory of each CPU, and adding each child pool into the parent pool;
responding to the resource shortage of the child pool, and sending a resource transfer application from the child pool with the resource shortage to the parent pool;
in response to the parent pool receiving the application, the parent pool queries the remaining resources of each child pool;
and responding to the query that the residual resource number of other sub-pools meets the requirement in the application, and transferring the residual resources of other sub-pools to the sub-pools with insufficient resources by the parent pool for use.
In a preferred embodiment of the present invention, creating a globally unique parent pool, and creating child pools using the physical memory of each CPU, respectively, and adding each child pool to the parent pool comprises:
respectively creating a sub-pool by using a physical memory of each CPU;
converting the sub-pool of each physical memory into a linear logic space through a map mechanism of a linux kernel;
cutting each linear logic space according to a preset granularity size to form a page pool;
allocating page numbers to each page pool and setting a resource pointer for each page pool;
and adding the page number and the resource pointer of each page pool into the parent pool.
In a preferred embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the parent pool inquires that the child pool with the most residual resources meets the requirement in the application, and sending the page number of the residual page pool in the child pool with the most residual resources and the pointer of the residual resources to the child pool with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using the corresponding resources according to the position of the pointer.
In a preferred embodiment of the present invention, in response to querying that the number of the remaining resources of the other sub-pools meets the requirement in the application, the transferring, by the parent pool, the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use includes:
responding to the request that the total number of the remaining resources of other sub-pools meets the requirement in the application, and respectively sending the page numbers of the remaining page pools in the sub-pools with the remaining resources and the pointers of the remaining resources to the sub-pools with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using corresponding resources according to the positions of the pointers.
In a preferred embodiment of the present invention, the method further comprises:
and in response to the sub-pool with insufficient resources successfully using the data in the remaining resources of the other sub-pool, transferring the remaining resources of the other sub-pool to the other sub-pool.
In a preferred embodiment of the present invention, the predetermined particle size is 4K size.
In a preferred embodiment of the present invention, in response to the parent pool receiving the request, the parent pool querying the remaining resources of each child pool comprises:
in response to the parent pool receiving the application, the parent pool queries the number of unused page pools in each child pool;
and adding all the resources of the queried page pool number.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method for sharing memory pool resources is characterized by comprising the following steps:
creating a globally unique parent pool, creating a child pool by using a physical memory of each CPU, and adding each child pool into the parent pool;
responding to the resource shortage of the sub-pool, and sending a resource transfer application to the parent pool by the sub-pool with the resource shortage;
in response to the parent pool receiving the application, the parent pool querying the remaining resources of each child pool;
and in response to the query that the number of the remaining resources of the other sub-pools meets the requirement in the application, the parent pool transfers the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use.
2. The method of claim 1, wherein creating a globally unique parent pool and creating a child pool using physical memory of each CPU, respectively, and adding each child pool to the parent pool comprises:
respectively creating a sub-pool by using a physical memory of each CPU;
converting the sub-pool of each physical memory into a linear logic space through a map mechanism of a linux kernel;
cutting each linear logic space according to a preset granularity size to form a page pool;
allocating page numbers to each page pool and setting a resource pointer for each page pool;
and adding the page number and the resource pointer of each page pool into the parent pool.
3. The method of claim 2, wherein in response to querying that the number of remaining resources of other child pools meets the requirements in the application, the parent pool transferring the remaining resources of other child pools to the sub-pool with insufficient resources for use comprises:
responding to the parent pool inquiring that the sub-pool with the most residual resources meets the requirement in the application, and sending the page number of the residual page pool in the sub-pool with the most residual resources and the pointer of the residual resources to the sub-pool with insufficient resources;
and the sub-pool with insufficient resources processes the data to be processed by using the corresponding resources according to the position of the pointer.
4. The method of claim 2, wherein in response to querying that the number of remaining resources of other child pools meets the requirements in the application, the parent pool transferring the remaining resources of other child pools to the sub-pool with insufficient resources for use comprises:
responding to the request that the total number of the remaining resources of other sub-pools meets the requirement in the application, and respectively sending the page numbers of the remaining page pools in the sub-pools with the remaining resources and the pointers of the remaining resources to the sub-pools with insufficient resources;
and the sub-pool with insufficient resources respectively processes the data to be processed by using corresponding resources according to the positions of the pointers.
5. The method of claim 1, further comprising:
and responding to the successful disk-dropping of the sub-pool with insufficient resources by using the data in the residual resources of other sub-pools, and transferring the residual resources back to other sub-pools.
6. The method of claim 2, wherein the predetermined particle size is 4K size.
7. The method of claim 2, wherein in response to the parent pool receiving the request, the parent pool querying each child pool for remaining resources comprises:
in response to the parent pool receiving the application, the parent pool querying the number of unused page pools in each child pool;
and adding all the resources of the queried page pool number.
8. An apparatus for sharing memory pool resources, the apparatus comprising:
the creating module is configured to create a globally unique parent pool, create child pools by using physical memories of all CPUs (central processing units), and add all the child pools to the parent pool;
an application module configured to issue an application for resource transfer to the parent pool in response to a shortage of resources in a child pool;
a query module configured to query the parent pool for remaining resources of each child pool in response to the parent pool receiving the application;
and the transfer module is configured to respond to the inquiry that the number of the remaining resources of the other sub-pools meets the requirement in the application, and the parent pool transfers the remaining resources of the other sub-pools to the sub-pools with insufficient resources for use.
9. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111108280.4A CN113553195A (en) | 2021-09-22 | 2021-09-22 | A method, apparatus, device and readable medium for sharing memory pool resources |
| PCT/CN2022/078325 WO2023045250A1 (en) | 2021-09-22 | 2022-02-28 | Memory pool resource sharing method and apparatus, and device and readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111108280.4A CN113553195A (en) | 2021-09-22 | 2021-09-22 | A method, apparatus, device and readable medium for sharing memory pool resources |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113553195A true CN113553195A (en) | 2021-10-26 |
Family
ID=78134571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111108280.4A Pending CN113553195A (en) | 2021-09-22 | 2021-09-22 | A method, apparatus, device and readable medium for sharing memory pool resources |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113553195A (en) |
| WO (1) | WO2023045250A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023045250A1 (en) * | 2021-09-22 | 2023-03-30 | 苏州浪潮智能科技有限公司 | Memory pool resource sharing method and apparatus, and device and readable medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130173880A1 (en) * | 2011-12-28 | 2013-07-04 | International Business Machines Corporation | Dedicated large page memory pools |
| CN103823709A (en) * | 2014-02-28 | 2014-05-28 | 华为技术有限公司 | Virtualized cluster system, resource distributing method and management node |
| US9904474B2 (en) * | 2014-09-18 | 2018-02-27 | Fujitsu Limited | Control device and storage system |
| CN108519917A (en) * | 2018-02-24 | 2018-09-11 | 国家计算机网络与信息安全管理中心 | A resource pool allocation method and device |
| US10298512B2 (en) * | 2015-06-26 | 2019-05-21 | Vmware, Inc. | System and method for performing resource allocation for a host computer cluster |
| CN110162395A (en) * | 2018-02-12 | 2019-08-23 | 杭州宏杉科技股份有限公司 | A kind of method and device of Memory Allocation |
| CN110780994A (en) * | 2019-09-25 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Method, equipment and medium for regulating and controlling memory |
| CN112181637A (en) * | 2020-09-10 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Method and device for allocating memory resources |
| CN112269659A (en) * | 2020-10-23 | 2021-01-26 | 苏州浪潮智能科技有限公司 | Resource management method, system, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111858034B (en) * | 2020-06-29 | 2023-01-06 | 苏州浪潮智能科技有限公司 | Resource management method, system, device and medium |
| CN113553195A (en) * | 2021-09-22 | 2021-10-26 | 苏州浪潮智能科技有限公司 | A method, apparatus, device and readable medium for sharing memory pool resources |
-
2021
- 2021-09-22 CN CN202111108280.4A patent/CN113553195A/en active Pending
-
2022
- 2022-02-28 WO PCT/CN2022/078325 patent/WO2023045250A1/en active Application Filing
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130173880A1 (en) * | 2011-12-28 | 2013-07-04 | International Business Machines Corporation | Dedicated large page memory pools |
| CN103823709A (en) * | 2014-02-28 | 2014-05-28 | 华为技术有限公司 | Virtualized cluster system, resource distributing method and management node |
| US9904474B2 (en) * | 2014-09-18 | 2018-02-27 | Fujitsu Limited | Control device and storage system |
| US10298512B2 (en) * | 2015-06-26 | 2019-05-21 | Vmware, Inc. | System and method for performing resource allocation for a host computer cluster |
| CN110162395A (en) * | 2018-02-12 | 2019-08-23 | 杭州宏杉科技股份有限公司 | A kind of method and device of Memory Allocation |
| CN108519917A (en) * | 2018-02-24 | 2018-09-11 | 国家计算机网络与信息安全管理中心 | A resource pool allocation method and device |
| CN110780994A (en) * | 2019-09-25 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Method, equipment and medium for regulating and controlling memory |
| CN112181637A (en) * | 2020-09-10 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Method and device for allocating memory resources |
| CN112269659A (en) * | 2020-10-23 | 2021-01-26 | 苏州浪潮智能科技有限公司 | Resource management method, system, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| N CHANDRAKALA 等: "Improved Efficient Dynamic Resource Optimization for Cloud Computing", 《INTERNATIONAL JOURNAL OF P2P NETWORK TRENDS AND TECHNOLOGY》 * |
| 胡晨光 等: "Oracle数据库共享池空间管理调优", 《现代电子技术》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023045250A1 (en) * | 2021-09-22 | 2023-03-30 | 苏州浪潮智能科技有限公司 | Memory pool resource sharing method and apparatus, and device and readable medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023045250A1 (en) | 2023-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11711420B2 (en) | Automated management of resource attributes across network-based services | |
| CN113672175B (en) | Distributed object storage method, device and equipment and computer storage medium | |
| JP6542909B2 (en) | File operation method and apparatus | |
| US10356150B1 (en) | Automated repartitioning of streaming data | |
| US10057187B1 (en) | Dynamic resource creation to connect client resources in a distributed system | |
| CN107423301B (en) | Data processing method, related equipment and storage system | |
| CN106095589A (en) | Partition allocation method, device and system | |
| CN103744719A (en) | Lock management method, lock management system, lock management system configuration method and lock management system configuration device | |
| CN111857539A (en) | Method, apparatus and computer program product for managing a storage system | |
| CN117785758B (en) | CXL module, controller, task processing method, medium and system | |
| US20170364442A1 (en) | Method for accessing data visitor directory in multi-core system and device | |
| US10812408B1 (en) | Preventing concentrated selection of resource hosts for placing resources | |
| CN111984204B (en) | A data reading and writing method, device, electronic equipment, and storage medium | |
| CN113569508A (en) | Database model construction method and device for data indexing and access based on ID | |
| CN110162395B (en) | Memory allocation method and device | |
| CN113553195A (en) | A method, apparatus, device and readable medium for sharing memory pool resources | |
| CN113190528B (en) | Parallel distributed big data architecture construction method and system | |
| CN111475099A (en) | Data storage method, device and equipment | |
| CN111475424A (en) | Method, apparatus and computer program product for managing a storage system | |
| CN107102898B (en) | Memory management and data structure construction method and device based on NUMA (non Uniform memory Access) architecture | |
| CN112839071B (en) | Training system, training data access method and device, electronic equipment and medium | |
| CN117743202A (en) | Memory management method, device, equipment and machine-readable storage medium | |
| CN116755617A (en) | Method and device for writing data into storage cluster, storage medium and electronic equipment | |
| CN108139980B (en) | Method for merging memory pages and memory merging function | |
| US10200301B1 (en) | Logical control groups for distributed system resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211026 |