CN118467172A - Host memory management method, device, storage medium and server - Google Patents
Host memory management method, device, storage medium and server Download PDFInfo
- Publication number
- CN118467172A CN118467172A CN202410706077.4A CN202410706077A CN118467172A CN 118467172 A CN118467172 A CN 118467172A CN 202410706077 A CN202410706077 A CN 202410706077A CN 118467172 A CN118467172 A CN 118467172A
- Authority
- CN
- China
- Prior art keywords
- memory
- sub
- virtual machine
- host
- machine process
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000015654 memory Effects 0.000 title claims abstract description 295
- 238000007726 management method Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 165
- 230000008569 process Effects 0.000 claims abstract description 150
- 238000013507 mapping Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012360 testing method Methods 0.000 claims description 7
- 239000002699 waste material Substances 0.000 abstract description 5
- 238000013500 data storage Methods 0.000 abstract description 3
- 238000013468 resource allocation Methods 0.000 description 9
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000010485 coping Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5022—Mechanisms to release resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45583—Memory management, e.g. access or allocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45587—Isolation or security of virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention discloses a host memory management method, a device, a storage medium and a server, and relates to the field of data storage. And creating independent sub-memories and exchange storage spaces for each virtual machine process, creating independent virtual control groups for each virtual machine process, and triggering the virtual control groups to adjust resources according to the received allocation instructions and the current memory use conditions. In the invention, by providing independent memory resources and virtual control groups for each virtual machine process, the problem that a uniform resource management mode is adopted for all virtual machine processes is avoided, so that each virtual machine process can use the memory according to own requirements without being influenced by other virtual machine processes, and the resource can be dynamically adjusted to avoid resource competition and waste, thereby improving the utilization rate of the whole memory resources.
Description
Technical Field
The present invention relates to the field of data storage, and in particular, to a method and apparatus for managing a host memory, a storage medium, and a server.
Background
In modern computing environments, virtualization technology enables a single physical machine to run multiple isolated virtual machines (VMs, virtual Manufacturing), each having its own operating system and application programs, as if running on separate physical hardware. Memory management, however, is a key challenge in virtualized environments. In a server virtualization environment, the number of virtual machines may far exceed the total memory of a physical server, so that effective allocation and management of memory resources are realized, and stable operation and performance of the virtual machines are ensured, which is one of main problems faced by a virtualization technology.
Host memory superscalar techniques have evolved that allow the total amount of memory allocated by a virtual machine to exceed the actual memory capacity of a physical server. At present, the memory management mainly depends on a control group and a swap file to limit the resource use of all virtual machine processes, lacks fine granularity control on the memory used by a single virtual machine process, cannot carefully meet the memory requirement of the single virtual machine process, and easily has memory resource competition among a plurality of virtual machine processes.
Disclosure of Invention
The invention aims to provide a host memory management method, a device, a storage medium and a server, which provide independent memory resources and virtual control groups for each virtual machine process, avoid the problem that a uniform resource management mode is adopted for all virtual machine processes, enable each virtual machine process to use memory according to own requirements, avoid being influenced by other virtual machine processes, and dynamically adjust resources to avoid resource competition and waste and improve the overall memory resource utilization rate.
In a first aspect, the present invention provides a host memory management method, applied to a processor in a server, where the server further includes a disk and a host memory, the host memory management method includes:
Creating a plurality of virtual machine processes, and distributing at least one sub-memory and at least one exchange storage space for each virtual machine process, wherein the host memory comprises a plurality of sub-memories, and the disk comprises a plurality of exchange storage spaces;
Creating a virtual control group corresponding to each virtual machine process, wherein the virtual control group is used for managing the resource use of the virtual machine process corresponding to each virtual machine process, and the resources of the virtual machine process at least comprise the sub-memory and the exchange storage space;
and triggering the virtual control group to adjust the resources of the virtual machine process according to the received allocation instruction, the size of the sub-memory, the size of the exchange storage space and the data stored in the sub-memory when the virtual machine process is operated.
In one embodiment, adjusting the resources of the virtual machine process according to the received allocation instruction, the size of the sub-memory, the size of the swap memory space, and the data stored in the sub-memory includes:
Determining whether preset data stored in the sub-memory is migrated to the exchange storage space according to a current actually used sub-memory of the virtual machine process, a pressure state of the host memory, the size of the sub-memory and the exchange storage space corresponding to the allocation instruction;
If yes, the preset data stored in the sub-memory corresponding to the virtual machine process are migrated to the exchange memory space.
In one embodiment, before determining whether to migrate the preset data stored in the sub-memory to the swap storage space according to the current actually used sub-memory of the virtual machine process corresponding to the allocation instruction, the pressure state of the host memory, the size of the sub-memory, and the swap storage space, the method further includes:
acquiring a first current residual memory of the host and a first reserved memory of the host;
Calculating the ratio of the first current residual memory to the first reserved memory or the ratio of the first reserved memory to the first current residual memory;
and determining the pressure grade of the host memory according to the ratio, wherein the pressure grade is used for representing the pressure state of the host memory.
In one embodiment, after determining the pressure level of the host memory according to the ratio, the method further comprises:
determining a corresponding preset time period according to the pressure level;
And every preset time period, entering a step of determining whether preset data stored in the sub-memory is migrated to the exchange memory space according to the current actually used memory of the virtual machine process corresponding to the allocation instruction, the pressure state of the host memory, the size of the sub-memory and the exchange memory space.
In one embodiment, determining whether to migrate the preset data stored in the sub-memory to the swap storage space according to the current actually used memory of the virtual machine process, the pressure state of the host memory, the size of the sub-memory, and the swap storage space corresponding to the allocation instruction includes:
Acquiring a current actually used sub-memory of the virtual machine process and an actually allocated sub-memory allocated for the virtual machine process;
Acquiring a second reserved memory of the virtual machine process, wherein the actually allocated sub-memory comprises a used memory and the second reserved memory, and the current actually used sub-memory is a subset of the used memory;
Acquiring average use memory of the virtual machine process in a preset test time period;
Comparing the sizes of the current actually used sub-memory, the average used memory and the actually allocated sub-memory;
if the current actual used sub-memory is less than the average used memory and less than the actual allocated sub-memory, determining to migrate preset data stored in the sub-memory to the exchange storage space;
The preset data are data stored in the sub-memory for a time exceeding a preset residence time, the data size of the preset data is determined to be a target data size, and the target data size is the minimum value of the first data size and the second data size;
The first data amount is the difference between the actually allocated sub-memory and the average used memory, and the second data amount is the difference between the actually allocated sub-memory and the second reserved memory.
In one embodiment, after migrating the preset data stored in the sub-memory corresponding to the virtual machine process to the exchange storage space, the method further includes:
marking a sub-memory area for storing the preset data as available;
and updating a preset mapping table, wherein the preset mapping table stores the corresponding relation between preset data in the sub-memory and the address of the exchange storage space.
In one embodiment, when the swap memory space is a memory space in a physical disk, a plurality of swap memory spaces are mounted on a swap memory device of the host;
and when the exchange storage space is a space in a virtual disk, a preset mapping relation is formed between a virtual storage address in the virtual disk and a physical address in exchange storage equipment of the host.
In a second aspect, the present invention provides a host memory management device, including:
a memory for storing a computer program;
and the processor is used for realizing the steps of the host memory management method when executing the computer program.
In a third aspect, the present invention provides a non-volatile storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the host memory management method described above.
In a fourth aspect, the present invention provides a server, including a disk, a host memory, and the host memory management device described above, where a processor in the host memory management device is connected to the disk and the host memory, respectively.
The invention provides a host memory management method, a device, a storage medium and a server, and relates to the field of data storage. And creating independent sub-memories and exchange storage spaces for each virtual machine process, creating independent virtual control groups for each virtual machine process, and triggering the virtual control groups to adjust resources according to the received allocation instructions and the current memory use conditions. In the invention, independent memory resources and virtual control groups are provided for each virtual machine process, so that the problem that a uniform resource management mode is adopted for all virtual machine processes is avoided, each virtual machine process can use the memory according to own requirements without being influenced by other virtual machine processes, the dynamic adjustment of the resources can avoid resource competition and waste, and the utilization rate of the whole memory resources is improved
Drawings
For a clearer description of embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a flowchart of a method for managing a host memory according to the present invention;
FIG. 2 is a schematic diagram of a host according to the present invention;
FIG. 3 is a schematic diagram of a pressure state of a host according to the present invention;
FIG. 4 is a schematic flow chart of determining a preset time period according to a pressure level according to the present invention;
Fig. 5 is a schematic diagram of a host memory management device according to the present invention.
Detailed Description
The core of the invention is to provide a host memory management method, a device, a storage medium and a server, which provide independent memory resources and virtual control groups for each virtual machine process, so that the problem that a uniform resource management mode is adopted for all virtual machine processes is avoided, each virtual machine process can use memory according to own requirements, the influence of other virtual machine processes is avoided, the resources can be dynamically adjusted, the resource competition and waste can be avoided, and the overall memory resource utilization rate is improved.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In a first aspect, as shown in fig. 1 and fig. 2, the present invention provides a host memory management method, applied to a processor in a server, where the server further includes a disk and a host memory, and the host memory management method includes:
S11: creating a plurality of virtual machine processes, and distributing at least one sub-memory and at least one exchange storage space for each virtual machine process, wherein the host memory comprises a plurality of sub-memories, and the disk comprises a plurality of exchange storage spaces;
In this step, a virtual machine process is created and allocated with sub-memory and swap memory. Wherein virtual machine processes are virtualized instances running on servers, each virtual machine process representing a separate operating system environment in which applications and services can be run. Creating multiple virtual machine processes allows the server to run multiple isolated operating system instances simultaneously, improving the utilization and flexibility of server resources. The sub-memory is a portion of memory space allocated from the host memory to each virtual machine process, and each virtual machine process is allocated at least one sub-memory for storing applications and data that it is running. The swap memory space is a virtual or physical memory located on the disk, and is used to store part of the data of the virtual machine processes when the memory is insufficient, and each virtual machine process allocates at least one swap memory space (such as swap-1 and swap-n in fig. 2) so as to perform page swap when needed, and release the memory for other processes to use.
Host memory is random access memory on a physical server for storing programs and data for running virtual machine processes. A disk is a device for persistent storage that includes swap memory space for storing data and page files for virtual machine processes.
Through the step, independent memory resources can be provided for each virtual machine process, so that the virtual machine processes can obtain enough memory space during operation, and memory exchange can be performed by utilizing the exchange memory space when needed, thereby ensuring the stability and performance of the whole system.
S12: creating a virtual control group corresponding to each virtual machine process, wherein the virtual control group is used for managing the resource use of the virtual machine process corresponding to each virtual machine process, and the resources of the virtual machine process at least comprise a sub-memory and an exchange storage space;
Further, to effectively manage the resource usage of each virtual machine process, a corresponding virtual control group needs to be created for each virtual machine process. A virtual control group is a resource management mechanism provided by the Linux kernel for grouping and managing the resource usage of a set of processes or threads. In this embodiment, the virtual control group is configured to manage resources of a virtual machine process corresponding to the virtual control group, including memory and disk resources.
In particular, the virtual control group allows an administrator to define and control resource usage limitations of the process group, including CPU (Central Processing Unit ), memory, disk I/O, and the like. For each virtual machine process, the virtual control group may set resource quota, priority, limits, and monitoring policies to ensure that it can use system resources in the intended manner. Through the virtual control group, an administrator can manage and adjust the resource usage of each virtual machine process in a fine granularity mode so as to meet the requirements of the overall performance and stability of the system.
The virtual control group not only manages the memory resources of the virtual machine process, but also covers the swap storage space. This means that the virtual control group can monitor and adjust the data exchange between the memory and disk of the virtual machine process to maximize the performance and resource utilization of the system.
In summary, the present step can effectively manage and optimize the overall resource usage by creating a corresponding virtual control group for each virtual machine process, thereby improving the performance, stability and manageability of the server.
S13: when the virtual machine process is operated, the virtual control group is triggered to adjust the resources of the virtual machine process according to the received allocation instruction, the size of the sub-memory, the size of the exchange storage space and the data stored in the sub-memory.
In this step, the virtual control group may receive an allocation instruction from a system administrator or an automation management system. These instructions may be dynamic, such as increasing or decreasing memory allocation for the virtual machine processes, or static, such as setting resource allocation for each virtual machine process at initial configuration. The virtual control group needs to know the size of the allocated sub-memory and swap memory space of each virtual machine process, which is the basis for adjusting the resource allocation, because the resource requirements of the virtual machine process are closely related to its allocated memory and disk space. The virtual control group also needs to know the data currently stored in the sub-memory. This may include applications, caches, temporary files, etc., and depending on the amount and nature of data stored, resources may be adjusted to meet the needs of the virtual machine process, e.g., freeing memory to make room, etc.
The virtual control group may take various measures to adjust the resource allocation of the virtual machine process based on the received allocation instructions, the size of the sub-memories, the size of the swap memory space, and the data stored in the sub-memories. In this way, the resources of the virtual machine processes can be dynamically adjusted according to real-time conditions to ensure that they are able to obtain sufficient resources and optimized as needed to improve overall performance and stability.
In one embodiment, adjusting the resources of the virtual machine process according to the received allocation instruction, the size of the sub-memory, the size of the swap memory space, and the data stored in the sub-memory includes:
determining whether preset data stored in the sub-memory is migrated to the exchange storage space according to the current actually used sub-memory of the virtual machine process corresponding to the allocation instruction, the pressure state of the host memory, the size of the sub-memory and the exchange storage space;
if yes, the preset data stored in the sub-memory corresponding to the virtual machine process is migrated to the exchange storage space.
In this embodiment, when an allocation instruction is received, it is determined which virtual machine process is to be resource-adjusted according to the instruction, and the current actually used sub-memory amount of the virtual machine process is checked, which may be implemented by monitoring the memory usage of the process, where this measure is used to determine whether the resource allocation needs to be adjusted. In addition, the pressure state of the host memory, i.e. the current use condition of the host memory, is also considered. If host memory is already in a high load state, some memory resources may need to be freed up to meet the needs of other processes. The size of the sub-memory allocated by the virtual machine process and the available space for swap memory may also be checked, which may help the system determine if there is sufficient space to migrate the data in the sub-memory to swap memory.
Based on the information, whether the preset data stored in the sub-memory needs to be migrated to the exchange storage space is determined. This determination may be based on a variety of factors, such as host memory pressure, actual memory usage of the virtual machine process, and the like. This is performed if it is considered that releasing the pre-set data in the sub-memory can effectively relieve the host memory pressure or meet the needs of other processes. If the system decides to migrate the preset data to the swap memory space, it will migrate the preset data stored in the sub-memory corresponding to the virtual machine process to the swap memory space. This frees up host memory and provides room for other processes. Meanwhile, by migrating data to the exchange storage space, the memory resources of the virtual machine process can be ensured to be effectively utilized, and resource waste is avoided.
In summary, the embodiment can dynamically adjust the resource allocation of the virtual machine process according to the real-time situation, ensure the stability and performance of the system, and maximally utilize the available resources.
In one embodiment, before determining whether to migrate the preset data stored in the sub-memory to the swap storage space according to the current actually used sub-memory of the virtual machine process corresponding to the allocation instruction, the pressure state of the host memory, the size of the sub-memory, and the swap storage space, the method further includes:
Acquiring a first current residual memory of a host and a first reserved memory of the host;
Calculating the ratio of the first current residual memory to the first reserved memory or the ratio of the first reserved memory to the first current residual memory;
and determining the pressure level of the host memory according to the ratio, wherein the pressure level is used for representing the pressure state of the host memory.
In this embodiment, before determining whether to migrate preset data in the sub-memory to the swap memory space, the current remaining memory and reserved memory of the host are obtained first. The current remaining memory represents the amount of unallocated memory available on the current host, while reserved memory is the portion of memory reserved by the system for other critical tasks or emergency situations. Then, the ratio of the current residual memory to the reserved memory, or the ratio of the reserved memory to the current residual memory, is calculated. This ratio may help evaluate the pressure condition of the host memory, i.e., whether the host memory has approached or exceeded the limit of reserved memory.
And determining the pressure level of the memory of the host according to the calculated ratio. Pressure levels are used to characterize the pressure state of the host memory and are typically divided into multiple levels, e.g., low, medium, high, etc. The lower the ratio of the current residual memory to the reserved memory is calculated, or the higher the ratio of the reserved memory to the current residual memory is, the higher the pressure of the host memory is, the lower the resource utilization rate is, and the stability and the performance of the system can be influenced.
As shown in fig. 3, the pressure level of the host memory is divided into 4 levels of lower, middle and higher, specifically, when the ratio of the first current remaining memory to the first reserved memory is greater than 2, the pressure level is determined to be lower; when the ratio of the first current residual memory to the first reserved memory is between 1 and 2, determining that the pressure level is low; when the ratio of the first current residual memory to the first reserved memory is 1.5-1, determining that the pressure level is medium; and when the ratio of the first current residual memory to the first reserved memory is smaller than 0.5, determining that the pressure level is high.
In summary, the present embodiment can more comprehensively evaluate the use condition and the pressure state of the host memory, so as to more accurately determine whether to migrate the preset data in the sub-memory to the swap memory space. This helps the system more intelligent and accurate when adjusting the resource allocation, improves the overall performance and the stability of system.
As shown in fig. 4, in one embodiment, after determining the pressure level of the host memory according to the ratio, the method further includes:
determining a corresponding preset time period according to the pressure grade;
Every preset time period, entering a step of determining whether preset data stored in the sub-memory is migrated to the exchange storage space according to the current actually used memory of the virtual machine process corresponding to the allocation instruction, the pressure state of the main memory, the size of the sub-memory and the exchange storage space.
In this embodiment, after determining the pressure level of the host memory, a corresponding preset time period is determined according to the pressure level calculated in the foregoing. Different pressure levels may require different adjustment frequencies, and thus a preset time period may be set for each pressure level. For example, a higher pressure host may require more frequent resource adjustments, while a lower pressure host may maintain the current configuration for a longer period of time. At the end of each preset time period, a step of resource adjustment is entered. At this time, the system will check the current actual used memory of the virtual machine process corresponding to the allocation instruction, the pressure state of the host memory, the size of the sub-memory, the swap storage space, etc. Based on the information, it is re-evaluated whether the preset data stored in the sub-memory needs to be migrated to the swap memory space. If it is deemed that the resource allocation needs to be adjusted at this time to cope with the current stress situation or to meet the demands of the virtual machine process, it will perform the corresponding operation.
In summary, the present embodiment can dynamically optimize resource allocation according to the pressure condition of the host memory and the requirements of the virtual machine process through regular resource inspection and adjustment, thereby improving the performance and stability of the system. This periodic adjustment also allows the system to accommodate changing workloads, better meeting the needs of the user.
In one embodiment, determining whether to migrate preset data stored in the sub-memory to the swap memory space according to a currently actually used memory of the virtual machine process corresponding to the allocation instruction, a pressure state of the host memory, a size of the sub-memory, and the swap memory space includes:
Acquiring a current actually used sub-memory of a virtual machine process and an actually allocated sub-memory allocated for the virtual machine process;
Acquiring a second reserved memory of the virtual machine process, wherein actually allocating the sub-memory comprises using the memory and the second reserved memory, and the current actually used sub-memory is a subset of the used memory;
acquiring average use memory of a virtual machine process in a preset test time period;
comparing the sizes of the current actually used sub-memories, the average used memories and the actually allocated sub-memories;
If the current actual used sub-memory is less than the average used memory and less than the actual allocated sub-memory, determining to transfer preset data stored in the sub-memory to the exchange memory space;
The preset data are data stored in the sub-memory for a time exceeding a preset retention time, the data volume of the preset data is determined to be a target data volume, and the target data volume is the minimum value of the first data volume and the second data volume;
The first data quantity is the difference between the actually allocated sub-memory and the average used memory, and the second data quantity is the difference between the actually allocated sub-memory and the second reserved memory.
The present embodiment describes the step of determining whether to migrate preset data stored in a sub-memory of a virtual machine process to a swap memory space according to a variety of factors. Specifically, the current actually used sub-memory of the virtual machine process and the actually allocated sub-memory allocated for the virtual machine process are obtained first, and these information can help the system to know the current memory usage situation of the virtual machine process and the limitation of the available memory. Further, a second reserved memory of the virtual machine process is also obtained, where the second reserved memory is a portion of memory reserved by the system for the virtual machine process, and is used for coping with sudden memory requirements. Furthermore, the average use memory of the virtual machine process in the preset test period is counted, which can help the system to know the typical memory use condition of the virtual machine process in a period of time. If the running time of the virtual machine process exceeds the preset test time period, selecting the memory use condition in the latest preset test time period to determine the average use memory; if the running time of the virtual machine process does not exceed the preset test time period, the memory use condition of all the running time of the virtual machine process is selected to determine the average used memory.
Specifically, the sizes of the current actually used sub-memory, the average used memory and the actually allocated sub-memory are compared to determine whether the current memory usage condition of the virtual machine process is within a preset range. If the current actually used sub-memory is smaller than the average used memory and the average used memory is smaller than the actually allocated sub-memory, the system determines to migrate the preset data stored in the sub-memory to the swap memory space. In this case, the amount of memory actually required by the current virtual machine process is considered to be insufficient to fully utilize the amount of actual sub-memory allocated by the current virtual machine process, and a part of memory resources can be released by migrating preset data stored in the sub-memory to the swap storage space. The purpose of this is to optimize the utilization of memory resources, ensure that the virtual machine process can obtain enough memory when needed, and avoid wasting the practically allocated memory resources. In other words, although the amount of memory currently used by the virtual machine process is low, due to the large amount of memory actually allocated, the system may optionally migrate data stored in a portion of the memory to the swap memory space, so as to use memory resources to meet the needs of other processes, or to prepare for the needs from time to time.
The preset data refers to data stored in the sub-memory for a time exceeding a preset retention time. The data volume is determined by taking smaller values from the difference value between the actual allocation sub-memory and the average use memory and the difference value between the actual allocation sub-memory and the second reserved memory.
According to the embodiment, whether resource allocation needs to be adjusted or not can be judged according to the memory use condition of the virtual machine process and the preset threshold value, so that the resource utilization efficiency and the performance of the system are improved.
In one embodiment, after migrating preset data stored in a sub-memory corresponding to a virtual machine process to the exchange storage space, the method further includes:
marking a sub-memory area for storing preset data as available;
Updating a preset mapping table, wherein the preset mapping table stores the corresponding relation between preset data in the sub-memory and the address of the exchange storage space.
In this embodiment, after the preset data is successfully migrated to the swap memory space, the sub-memory area originally used for storing the data becomes free and available. Thus, these sub-memory regions are marked as available so that subsequent memory allocations can utilize these free memory regions. The preset mapping table is also updated: the preset mapping table is a data structure for recording the corresponding relation between preset data in the sub-memory and the address of the exchange storage space. After the preset data are migrated to the exchange storage space, the preset mapping table needs to be updated to reflect the new storage locations of the data. Updating the preset mapping table can ensure that the system can accurately find preset data stored in the exchange storage space when needed and reload the preset data into the sub-memory so as to meet the requirements of virtual machine processes.
In summary, by performing these steps, the system ensures the integrity of migration and management of the preset data. The sub-memory area for storing the preset data is marked as an available state, so that the space can be effectively recovered and reused, and the utilization efficiency of memory resources is improved. Meanwhile, the accuracy and the reliability of the system in processing the memory requirement of the virtual machine process are ensured by updating the preset mapping table.
In one embodiment, updating the preset mapping table further comprises: and determining demand data loaded in advance from the exchange storage space according to task demands, positioning and reading the demand data in the exchange storage space according to the information stored in the preset mapping table, and loading the demand data read from the exchange storage space into the sub-memory corresponding to the virtual machine process in advance by preset time.
Specifically, the data to be loaded in advance is determined according to the task requirement, which may be based on the requirement prediction of the task to be executed on the memory, for example, an application program is about to be started, and related data needs to be preloaded into the memory, so as to speed up the starting and improve the responsiveness. Then, according to the information stored in the preset mapping table, the required data is positioned and read in the exchange storage space, and the information comprises the position, the size and the like of the data in the exchange storage space. Finally, the required data read from the exchange storage space is loaded into the sub-memory corresponding to the virtual machine process in advance, so that the required data is ensured to be available in the memory when the task is executed, and the delay of waiting for loading the data from the exchange storage space is avoided.
According to the embodiment, the data is loaded into the memory in advance, so that delay caused by loading the data from the exchange storage space when the task is started can be avoided, and the response speed of the task is improved. The pre-loading data can reduce frequent access to the exchange storage space, reduce the I/O load of the system and help to improve the overall performance of the system. The starting speed and the response speed of the task are increased, the user experience can be improved, the user can acquire the required service or information faster, and the user satisfaction is improved.
In one embodiment, when the exchange storage space is a storage space in a physical disk, a plurality of exchange storage spaces are mounted on an exchange storage device of the host;
when the exchange storage space is the space in the virtual disk, the virtual storage address in the virtual disk and the physical address in the exchange storage device of the host are in a preset mapping relation.
The embodiment describes the use mode of exchanging storage space and the preset mapping relation thereof under two conditions, which are based on a physical disk and a virtual disk respectively. When swap memory is based on physical disks, several swap memory is mounted on a host's swap memory device (e.g., the swap device in FIG. 2), which is typically a disk partition or disk volume on the host dedicated to swapping memory data. When data is to be migrated from memory to swap storage, the data is written to swap storage on these physical disks. When the swap memory space is based on virtual disks, the space in the virtual disks, which are typically provided by the physical disks or other storage devices of the host, is used as the swap memory space, which the system treats as a separate storage device. At this time, a preset mapping relationship between the virtual storage address in the virtual disk and the physical address in the exchange storage device of the host is established. Thus, when data needs to be read or written, the storage position of the data is determined according to the mapping relations.
The memory resources required by the memory exchange operation can be better isolated and managed by mounting the exchange memory space on the exchange memory equipment of the host, so that the situation of mixed storage of other data is avoided. The virtual disk-based exchange storage space has greater flexibility and expansibility, and the size and configuration of the storage space can be easily adjusted according to the needs without modifying or expanding physical hardware.
In summary, according to the embodiment, according to different storage requirements and environmental characteristics, a suitable exchange storage space type can be selected and a preset mapping relationship can be established, so that memory exchange operation can be more effectively managed, and the performance and reliability of the system can be improved.
In a second aspect, as shown in fig. 5, the present invention provides a host memory management device, including:
a memory 51 for storing a computer program;
the processor 52 is configured to implement the steps of the host memory management method described above when executing the computer program.
For the description of the host memory management device, please refer to the above embodiment, and the description of the present invention is omitted herein.
In a third aspect, the present invention provides a nonvolatile storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the host memory management method described above.
For the description of the nonvolatile storage medium, refer to the above embodiments, and the description of the present invention is omitted herein.
In a fourth aspect, the present invention provides a server, including a disk, a host memory, and the host memory management device described above, where a processor in the host memory management device is connected to the disk and the host memory, respectively.
For the description of the server, refer to the above embodiments, and the description of the present invention is omitted herein.
In a fifth aspect, the present invention provides a computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the host memory management method described above. For the description of the computer program/instructions, refer to the above embodiments, and the disclosure is not repeated here. It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A host memory management method, applied to a processor in a server, the server further comprising a disk and a host memory, the host memory management method comprising:
Creating a plurality of virtual machine processes, and distributing at least one sub-memory and at least one exchange storage space for each virtual machine process, wherein the host memory comprises a plurality of sub-memories, and the disk comprises a plurality of exchange storage spaces;
Creating a virtual control group corresponding to each virtual machine process, wherein the virtual control group is used for managing the resource use of the virtual machine process corresponding to each virtual machine process, and the resources of the virtual machine process at least comprise the sub-memory and the exchange storage space;
and triggering the virtual control group to adjust the resources of the virtual machine process according to the received allocation instruction, the size of the sub-memory, the size of the exchange storage space and the data stored in the sub-memory when the virtual machine process is operated.
2. The host memory management method of claim 1, wherein adjusting the resources of the virtual machine process according to the received allocation instruction, the size of the sub-memory, the size of the swap memory space, and the data stored in the sub-memory comprises:
Determining whether preset data stored in the sub-memory is migrated to the exchange storage space according to a current actually used sub-memory of the virtual machine process, a pressure state of the host memory, the size of the sub-memory and the exchange storage space corresponding to the allocation instruction;
If yes, the preset data stored in the sub-memory corresponding to the virtual machine process are migrated to the exchange memory space.
3. The method of claim 2, wherein determining whether to migrate the preset data stored in the sub-memory to the swap memory space according to a currently actually used sub-memory of the virtual machine process corresponding to the allocation instruction, a pressure state of the host memory, a size of the sub-memory, and the swap memory space further comprises:
acquiring a first current residual memory of the host and a first reserved memory of the host;
Calculating the ratio of the first current residual memory to the first reserved memory or the ratio of the first reserved memory to the first current residual memory;
and determining the pressure grade of the host memory according to the ratio, wherein the pressure grade is used for representing the pressure state of the host memory.
4. The method of host memory management according to claim 3, further comprising, after determining the pressure level of the host memory according to the ratio:
determining a corresponding preset time period according to the pressure level;
And every preset time period, entering a step of determining whether preset data stored in the sub-memory is migrated to the exchange memory space according to the current actually used memory of the virtual machine process corresponding to the allocation instruction, the pressure state of the host memory, the size of the sub-memory and the exchange memory space.
5. The method of claim 3, wherein determining whether to migrate the preset data stored in the sub-memory to the swap memory space according to the current actually used memory of the virtual machine process, the pressure state of the host memory, the size of the sub-memory, and the swap memory space corresponding to the allocation instruction comprises:
Acquiring a current actually used sub-memory of the virtual machine process and an actually allocated sub-memory allocated for the virtual machine process;
Acquiring a second reserved memory of the virtual machine process, wherein the actually allocated sub-memory comprises a used memory and the second reserved memory, and the current actually used sub-memory is a subset of the used memory;
Acquiring average use memory of the virtual machine process in a preset test time period;
Comparing the sizes of the current actually used sub-memory, the average used memory and the actually allocated sub-memory;
if the current actual used sub-memory is less than the average used memory and less than the actual allocated sub-memory, determining to migrate preset data stored in the sub-memory to the exchange storage space;
The preset data are data stored in the sub-memory for a time exceeding a preset residence time, the data size of the preset data is determined to be a target data size, and the target data size is the minimum value of the first data size and the second data size;
The first data amount is the difference between the actually allocated sub-memory and the average used memory, and the second data amount is the difference between the actually allocated sub-memory and the second reserved memory.
6. The host memory management method according to claim 2, wherein after migrating the preset data stored in the sub-memory corresponding to the virtual machine process to the swap memory space, further comprising:
marking a sub-memory area for storing the preset data as available;
and updating a preset mapping table, wherein the preset mapping table stores the corresponding relation between preset data in the sub-memory and the address of the exchange storage space.
7. The host memory management method according to any one of claims 1 to 6, wherein when the swap memory space is a memory space in a physical disk, a plurality of swap memory spaces are mounted on a swap memory device of the host;
and when the exchange storage space is a space in a virtual disk, a preset mapping relation is formed between a virtual storage address in the virtual disk and a physical address in exchange storage equipment of the host.
8. A host memory management device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the host memory management method of any one of claims 1-7 when executing said computer program.
9. A non-volatile storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the host memory management method of any of claims 1-7.
10. A server comprising a disk, a host memory, and the host memory management device of claim 8, wherein a processor in the host memory management device is coupled to the disk and the host memory, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410706077.4A CN118467172A (en) | 2024-05-31 | 2024-05-31 | Host memory management method, device, storage medium and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410706077.4A CN118467172A (en) | 2024-05-31 | 2024-05-31 | Host memory management method, device, storage medium and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118467172A true CN118467172A (en) | 2024-08-09 |
Family
ID=92151688
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410706077.4A Pending CN118467172A (en) | 2024-05-31 | 2024-05-31 | Host memory management method, device, storage medium and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118467172A (en) |
-
2024
- 2024-05-31 CN CN202410706077.4A patent/CN118467172A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11106579B2 (en) | System and method to manage and share managed runtime memory for java virtual machine | |
| EP2687991A2 (en) | Methods And Structure For Improved Flexibility In Shared Storage Caching By Multiple Systems Operating As Multiple Virtual Machines | |
| JP5063069B2 (en) | Memory allocation method, apparatus, and program for multi-node computer | |
| US20170017511A1 (en) | Method for memory management in virtual machines, and corresponding system and computer program product | |
| JP7125964B2 (en) | Computer system and management method | |
| US9367441B2 (en) | Method for managing physical memory of a data storage and data storage management system | |
| WO2008006674A1 (en) | Reserve pool management in virtualized storage systems | |
| US8332611B2 (en) | Method and system for managing memory | |
| CN112269656B (en) | Application configuration method, device and system based on multi-core processor | |
| CN118244995B (en) | Method for adjusting SSD load, storage system, storage access method and product | |
| CN115421924A (en) | Memory allocation method, device and equipment | |
| CN109960569B (en) | Virtualization processing method and device | |
| CN114442910B (en) | Method for managing storage system, electronic device and computer readable medium | |
| CN110543355A (en) | method for automatically balancing cloud platform resources | |
| KR20200117032A (en) | Hybrid memory system | |
| US10007434B1 (en) | Proactive release of high performance data storage resources when exceeding a service level objective | |
| US20190173770A1 (en) | Method and system for placement of virtual machines using a working set computation | |
| US20090187911A1 (en) | Computer device with reserved memory for priority applications | |
| CN118467172A (en) | Host memory management method, device, storage medium and server | |
| US10592297B2 (en) | Use minimal variance to distribute disk slices to avoid over-commitment | |
| US11914512B2 (en) | Writeback overhead reduction for workloads | |
| CN116185631A (en) | Load balancing method, device, equipment and medium for solid state disk with multi-core structure | |
| CN116204134B (en) | Cache management method, device, equipment, storage system and storage medium | |
| US12039203B1 (en) | Data storage system employing delayed responses to influence rate of bulk storage operations | |
| US20190339898A1 (en) | Method, system and computer program product for managing data storage in data storage systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |