JP6543219B2 - Virtual machine allocation apparatus and resource management method - Google Patents

Description

  The present invention relates to a virtual machine placement device for placing virtual machines on a plurality of physical servers, and a resource management method for managing resources of virtual machines.

In recent years, as typified by a virtual appliance of Network Functions Virtualization (NFV), an application that requires low processing delay (hereinafter simply referred to as an application) is executed using a virtual machine (VM). The technology to operate is attracting attention.
In such applications, in order to reduce the delay, the virtual CPUs of VMs and the cores of physical CPUs are often fixedly allocated. This is effective in reducing the delay since the VM is intended to occupy the CPU time resource of the core and is not dispatched to other VMs.

  Moreover, in such an application, there is also a case where the virtual CPU of VM and the core of physical CPU are fixedly assigned with the limitation of the core on the same physical CPU. This enables communication between processes and threads in VMs to be performed in the same physical CPU, so that the cache hit rate can be increased, which is extremely effective as a means for reducing delay.

Conventionally, such a method of fixedly allocating a virtual CPU of a VM to a physical CPU can be realized by a function (CPU pinning) provided by KVM (Kernel-based Virtual Machine: kernel virtualization base) or the like ( Non-Patent Document 1). In this method, a command is manually issued via a computer, and a physical server (server hardware) that has acquired the command assigns virtual CPUs and physical CPUs of VMs operating on the server.
On the other hand, resource management systems such as OpenStack automatically allocate resources to VMs (see Non-Patent Document 2). This resource management system allocates resources by arranging VMs on a physical server with low CPU load.

"Setting of KVM processor affinity", [online], [search on May 20, 2016], Internet <URL: https://docs.fedoraproject.org/ja-JP/Fedora/13/html/Virtualization_Guide/ch25s06 .html> “Virtualization Driver Guest CPU / Memory Placement”, [online], [search May 20, 2016], Internet <https://wiki.openstack.org/wiki/VirtDriverGuestCPUMemoryPlacement>

In the method of the conventional non-patent document 1, since resources such as a CPU are manually allocated, there is a problem that it takes time and trouble in operation.
In addition, in the method of the conventional non-patent document 2, although resources can be automatically allocated to VMs, it can not be applied to an application in which virtual CPUs of VMs and cores of physical CPUs are fixedly allocated. You can only deploy VMs on low-load physical servers.
As described above, in the conventional method, it is difficult to automatically allocate resources in a system on the premise that virtual CPUs of VMs and cores of physical CPUs are fixedly allocated.

  Also, in the conventional method, when resources of VM are secured and resources are recovered after VM stop, resources in use and small unused resources are mixed, and large unused resources can not be secured. So-called resource fragmentation occurs. Therefore, in the conventional method, even when the unused resources in the entire system are sufficient, there may be a problem that the unused resources required by the VM can not be secured on the same physical CPU.

  The present invention has been made in view of such problems, and provides a virtual machine (VM) with resources without human intervention and capable of preventing fragmentation of resources, and a virtual machine allocation apparatus The task is to provide a resource management method.

In order to solve the above-mentioned subject, in a computer system provided with a plurality of physical servers having a plurality of CPUs composed of a plurality of cores, in a virtual machine arranging apparatus according to claim 1, A virtual machine allocation apparatus allocated and arranged in the physical server, which associates the physical server with the CPU and the core, and stores resource management data indicating which virtual machine the core is being used for Storage means, free resource determination means for determining whether or not the required number of cores required by a new virtual machine can be secured in advance on the same CPU with reference to the resource management data, and the free resource determination means If the required number of cores can not be secured on the same CPU, a core to which a virtual machine in operation is assigned, Arrangement to arrange the new virtual machine in the core of the same CPU on the same CPU as the execution unit for reassignment to another CPU of the physical server on which the virtual machine is operating or to the CPU of the other physical server An execution unit, and the free resource determination unit, when the required number of cores can not be secured by the same CPU, the core assigned to the active virtual machine in each physical server, The physical server unit resource determining unit that determines whether or not the required number of cores can be secured on the same CPU by reassignment to the CPU, and all the number of required cores can not be secured in the physical server. By reassigning the core assigned to the active virtual machine to the CPU of another physical server among physical servers, the required core The and a full physical server resources determining means for determining whether or not can be secured on the same CPU, characterized in Rukoto.

  According to this configuration, the virtual machine placement device can use the free resource determination means to determine whether or not the number of cores required by the newly placed virtual machine can be secured on the same CPU. Then, the virtual machine allocation apparatus can change the CPU core allocation in the physical server and the CPU core allocation between physical servers by the compaction execution unit. By this, even when the resource (core) is divided, the virtual machine placement device can effectively utilize the resource by compaction.

Also , the virtual machine placement device determines whether or not the number of cores required by the virtual machine to be newly placed can be secured on the same CPU by means of physical server unit compaction by means of physical server unit resource determination means. be able to. In addition, the virtual machine placement device determines whether or not the number of cores required by the virtual machine to be newly placed can be secured on the same CPU by compaction among physical servers by the all physical server resource determination means. Can. By this, the virtual machine placement device can secure resources hierarchically between physical servers and between physical servers.

Further, in order to solve the above problem, according to a resource management method according to claim 2 , in a computer system provided with a plurality of physical servers having a plurality of CPUs constituted of a plurality of cores, the virtual machine placement device comprises A resource management method for securing resources to be allocated to the same CPU core in advance and arranged in the physical server, and the CPU unit resource determination means associates the physical server, the CPU, and the core, and The same step of determining whether or not the required number of cores required by a new virtual machine can be secured in advance within the same CPU with reference to resource management data indicating which virtual machine is used by which core. When the required number of cores can not be secured in the CPU, the physical server unit resource determination unit Whether or not the required number of cores can be secured on the same CPU by reassigning cores assigned to active virtual machines to other CPUs in individual physical servers with reference to source control data Determining the core number in the physical server by the physical server internal allocation changing unit when the required number of cores can be secured in the physical server; and changing the core allocation in the physical server; If the required number of cores can not be secured, all physical server resource determination means refers to the resource management data, and all physical servers are assigned cores to active virtual machines among other physical servers. Between the physical servers and determining whether or not the required number of cores can be secured on the same CPU by reassignment to the CPU of If you can secure a number of the required cores, the physical servers inter-allocation changing means, characterized in that it comprises the steps of: changing the allocation of the core between the physical servers.

  According to this procedure, the resource management method can determine whether or not the number of cores required by the virtual machine to be newly arranged can be secured on the same CPU by compaction in the physical server. Further, the resource management method can determine whether or not the number of cores required by a virtual machine to be newly disposed can be secured on the same CPU by compaction among physical servers. Thus, the resource management method can secure resources hierarchically between physical servers and between physical servers.

According to the present invention, it is possible to automatically allocate resources in a system on the premise of fixedly allocating the virtual CPU of the virtual machine and the core of the physical CPU, thereby reducing the time and effort of manual operation and making an operation error. It can be prevented.
Further, according to the present invention, compaction can prevent fragmentation of resources, and resources can be effectively used.

It is a system outline figure for explaining an outline of a computer system (data center) concerning an embodiment of the present invention. FIG. 6 is an explanatory diagram for describing resource allocation between the physical server of FIG. 1 and a virtual machine operating on the physical server. It is a block block diagram which shows the structure of the virtual label arrangement | positioning apparatus which concerns on embodiment of this invention. It is a tree structure figure which shows the structure of the resource managed by the virtual label arrangement | positioning apparatus which concerns on embodiment of this invention. It is an example of a data structure of the resource management data managed by the virtual label placement apparatus according to the embodiment of the present invention, wherein (a) is a physical server list in a data center, (b) is a CPU list in a physical server, (c) Is a data structure diagram of the in-CPU core list. It is a data structure figure of the operation state list which shows the operation state (active / standby) of the application which operates with the virtual machine managed by the virtual label arrangement device concerning an embodiment of the present invention. It is explanatory drawing for demonstrating the concept which arrange | positions a virtual machine newly to a physical server. FIG. 6 is an explanatory diagram for explaining the compaction processing in the physical server, where (a) shows the order of many CPU free resources, (b) shows resource search for virtual machines, (c) shows new It is a figure which shows the state which arrange | positioned the virtual machine of. FIG. 6 is an explanatory diagram for describing a compaction process in all physical servers, where (a) is a diagram showing the allocation state of virtual machines for each CPU of all physical servers, (b) is an order of many free CPU resources FIG. It is an explanatory view for explaining change of assignment of a core in virtual CPU of a virtual machine, and physical CPU of a physical server. It is an explanatory view for explaining movement of a virtual machine between physical servers. FIG. 16 is an explanatory diagram for explaining movement of a virtual machine between physical servers in consideration of the operation state (active / standby) of an application operating in the virtual machine. It is a flowchart (the 1) which shows operation | movement of the virtual label arrangement | positioning apparatus which concerns on embodiment of this invention. It is a flowchart (the 2) which shows operation | movement of the virtual label arrangement | positioning apparatus which concerns on embodiment of this invention.

Embodiments of the present invention will be described with reference to the drawings.
<< Overall Configuration of Data Center >>
First, with reference to FIG. 1 and FIG. 2, an overall configuration of a data center S, which is an example of a computer system that performs placement of virtual machines and resource management thereof by a virtual machine placement device according to an embodiment of the present invention will be described. .

The data center S is a computer system provided with a plurality of servers connectable from the Internet, a network operator network, etc. (not shown).
The data center S is composed of a plurality of physical servers 1, 1, ..., and network devices whose illustration is omitted. Also, in this case, the data center S includes the virtual machine allocation device 2. The virtual machine layout device 2 need not necessarily be provided in the data center S, and may be provided outside the data center S that can be connected to the physical servers 1, 1,.

The physical server 1 is hardware that operates a virtual machine (VM) 12. Here, it is assumed that the physical server 1 is configured by a plurality of CPUs (Central Processing Units) 10, and each CPU (physical CPU) 10 is configured by a plurality of cores 11. Note that in FIG. 1, illustration of a storage device, a communication control device, and the like generally provided as the physical server 1 is omitted. The physical server 1 provides the virtual machine 12 operating on the physical server 1 with a resource (CPU resource) for operating an application.
As shown in FIG. 2, in the virtual machine (hereinafter, VM) 12 disposed on the physical server 1, a plurality of processes and threads operate in the virtual CPU (vCPU) 13. Further, each virtual CPU 13 is fixedly assigned to an individual core 11 of the same CPU (physical CPU) 10 and operates. In the following, this state is referred to as that the VM 12 is assigned to the core 11 of the CPU 10.
By this, the VM 12 can occupy the CPU time of the core 11, can not dispatch resources to other VMs, and can reduce delay.

The virtual machine placement device 2 manages resources in the data center (computer system) S, assigns the VMs 12 to the cores 11 of the same CPU 10, and places them on the physical server 1.
The virtual machine placement device 2 places the VM 12 on the physical server 1 when the CPU 10 of the physical server 1 has an enough space for the core 11 to operate the VM 12. On the other hand, if there is not enough space for the core 11 to operate the VM 12 in the CPU 10 of the physical server 1, the virtual machine placement device 2 resolves fragmentation of the core 11 and makes space for the core 11 to create the VM 12. Deploy.
The configuration of the virtual machine placement device 2 will be described in detail later with reference to FIG.
Thus, the data center S can automatically arrange the VMs 12 on the plurality of physical servers 1 by the virtual machine arrangement device 2 and can manage resources.

<< Configuration of virtual machine placement device >>
Next, the configuration of the virtual machine placement device 2 will be described with reference to FIG.
As shown in FIG. 3, the virtual machine placement device 2 includes a communication unit 200, a storage unit 210, and a control unit 220.

  The communication unit (communication means) 200 transmits and receives communication data to and from the physical server 1 via a communication line. For example, the communication unit 200 is configured by communication control means such as a NIC (Network Interface Card). The communication unit 200 transmits signals (control information and data) input from the control unit 220 to the physical server 1 via the communication line, and outputs the signal received from the physical server 1 to the control unit 220.

  The storage unit (storage unit) 210 stores data (resource management data) for managing resources in the data center S. Here, the application operating as the VM 12 is assumed to operate in the active / standby (ACT [operating system] / SBY [standby system] configuration, and the storage unit 210 is a list showing the operating state of the VM 12 (operating state) (List) is also stored. The storage unit 210 is configured of a general storage device such as a hard disk.

  Here, an example of resource management data stored in the storage unit 210 will be described with reference to FIGS. 4 and 5. As shown in FIG. 4, the resource management data is structured by treeing resources (physical server 1, CPU 10, core 11) in the order of physical server list in data center 211, CPU list in physical server 212, and core list in CPU 213. Data.

The data center physical server list 211 is a list of physical servers 1 constituting the data center S. For example, as shown in FIG. 5A, the physical server list in the data center 211 is data in which identification information SV ₁ , SV ₂ , ... (for example, IP address etc.) for identifying each physical server 1 is arranged. is there.
The physical server list in the data center 211 manages how many physical servers 1 the data center S is configured.

The physical server CPU list 212 is a list of CPUs 10 that constitute the physical server 1. For example, as shown in FIG. 5B, the physical server CPU list 212 arranges identification information CPU ₁ , CPU ₂ ,... (For example, consecutive numbers etc.) for identifying the CPUs 10 constituting each physical server 1. Data.
The physical server CPU list 212 manages how many CPUs 10 each physical server 1 is configured.

The in-CPU core list 213 is a list of the cores 11 that constitute the CPU 10. Further, the in-CPU core list 213 holds which VM 12 each core 11 is used and which virtual CPU 13 is associated with.
For example, as shown in FIG. 5C, the in-CPU core list 213 is data in which identification information CORE ₁ , CORE ₂ ,... It is. Furthermore, as shown in FIG. 5C, the in-CPU core list 213 holds the use state (used / not used) for each of the cores 11 and, if it is used, which virtual machine (VM) And 12) which virtual CPU 13 is associated with. Note that, in FIG. 5C, VM ₁ , VM ₂ ,... Are identification information (for example, consecutive numbers etc.) for identifying the VMs 12 individually. Further, vCPU ₁ , vCPU ₂ ,... Are identification information (for example, serial numbers etc.) for identifying the virtual CPU 13 individually.
The in-CPU core list 213 manages how many cores 11 each CPU 10 is configured of, and also manages the usage status of the cores 11.

Next, with reference to FIG. 6, an example of the operation state list stored in the storage unit 210 will be described. As shown in FIG. 6, the operation state list 214 indicates, for each of the VMs 12 (VM ₁ , VM ₂ ,...), The application APL being operated and the application APL in an active (ACT) state or standby (SBY) It is a list indicating whether it is a state. For example, FIG. 6 shows that the application APL ₁ operates in the active (ACT) state on VM ₁ and operates in the standby (SBY) state on VM ₂ .
Returning to FIG. 3, the description of the configuration of the virtual machine placement device 2 will be continued.

  The control unit (control means) 220 controls the operation of the virtual machine placement device 2. Here, the control unit 220 includes the placement position control unit 21, the free resource determination unit 22, the compaction execution unit 23, and the placement execution unit 24.

The placement position control unit 21 determines the placement position of the VM 12 in accordance with the use state of the resource in the data center S.
The placement position control unit 21 determines the placement position of the VM 12 by externally inputting the program of the VM 12 (including the application of the active system or the standby system) and the required number of cores (the required number of cores) from the outside. Do. The program of the VM 12 and the number of used cores are associated in advance and stored in the storage unit 210, and the arrangement position control unit 21 determines the arrangement position of the VM 12 by specifying the VM 12 to be arranged. It is also good.

Here, the placement position control means 21 includes identification information (APL) of an application included in the VM 12, identification information (ACT / SBY) indicating whether the application is an active system or a standby system, and a required core. The number and the number are notified to the free resource determination means 22. Then, the placement position control means 21 acquires, as a placement position, identification information of the CPU 10 on the physical server 1 with an available space where resources can be secured, and outputs it to the placement execution means 24 together with the program of the VM 12.
In addition, the arrangement position control means 21 has a procedure (compaction) indicating that it is necessary to eliminate fragmentation of the core 11 and reorganize (compaction) the core 11 in order to arrange the VM 12 from the free resource determination means 22. When notified as procedure), the compaction procedure is output to the compaction execution means 23. After the execution of the compaction, the placement position control means 21 outputs the program of the VM 12 and the placement position thereof to the placement execution means 24.

  The placement position control unit 21 displays a warning on a display device (not shown) when it is notified by the free resource determination unit 22 that the resource for placing the VM 12 can not be secured even if the compaction is performed. . In this case, the administrator may increase resources by adding the physical server 1.

The free resource determination unit 22 determines whether or not the necessary number of cores of the VMs 12 to be newly arranged can be secured by the same CPU 10 in all the physical servers 1.
Here, the available resource determination unit 22 includes a CPU-based resource determination unit 22 a, a physical server-based resource determination unit 22 b, and an all-physical server resource determination unit 22 c.

The CPU-based resource determination unit 22 a determines, for each of the CPUs 10 of the physical server 1, whether or not the required number of cores of the VM 12 to be newly arranged is available.
If the CPU (resource required number) can be secured by only one CPU 10, the CPU unit resource determination unit 22a uses the CPU 10 as an arrangement position of the VM 12 and identifies the physical server 1 and the identification information of the CPU 10 to the arrangement position control unit 21. Output In this case, the CPU unit resource determination means 22a does not output the compaction procedure to the placement position control means 21.
On the other hand, when the resource can not be secured by only one CPU 10, the CPU unit resource determination unit 22a outputs the required number of cores to the physical server unit resource determination unit 22b.

The CPU unit resource determination unit 22a refers to the resource management data (see FIG. 4) stored in the storage unit 210, and “unused” in the same CPU 10 in the CPU core list (see FIG. 5 (c)). When the number of cores (the number of vacant spaces) is equal to or more than the number of necessary cores, it is determined that resources can be secured in the CPU 10 thereof.
Note that the CPU unit resource determination means 22a is required core for the CPU 10 of the physical server 1 in which an operating system different from the operating system (operating system or standby system) of the application APL included in the newly allocated VM 12 is already arranged. Do not judge the number of vacancies.
As a result, the VM including the operation application and the VM including the standby application can be arranged on different physical servers 1.

Here, with reference to FIG. 7, a specific example of the process of the CPU-based resource determination unit 22a will be described. As shown in FIG. 7, for example, one physical server 1 (SV ₁ ) is configured by _four 18-core CPUs (CPU _{1 to} CPU ₄ ), and each CPU is used by a plurality of VMs 12, and Assume that the number of vacant units is seven, five, seven, and six.
In this state, when four are notified as the required number of cores of the VM 12 to be newly disposed, the CPU unit resource determination unit 22a is able to secure the resource first in the search order of the physical server 1 (SV ₁ ) Let CPU _{1 be} the placement position of VM 12.

On the other hand, when eight are notified as the required number of cores of the VM 12 to be newly arranged, the CPU unit resource determination unit 22a determines that the physical server 1 (SV ₁ ) can not secure the resources.
Returning to FIG. 3, the description of the configuration of the virtual machine placement device 2 will be continued.

  The physical server unit resource determination unit 22b, when the CPU unit resource determination unit 22a can not ensure the required number of cores by the same CPU, generates the other cores 11 allocated to the running VMs 12 in each physical server. By reassigning to the CPU 10, it is determined whether the required number of cores can be secured on the same CPU.

If the resource (number of required cores) can be secured only by the compaction of one physical server 1, the physical server unit resource determination means 22b arranges the identification information of the physical server 1 having the resource and the CPU 10 as the arrangement position. Output to position control means 21. Further, the physical server unit resource determination unit 22 b outputs the compaction procedure to the placement position control unit 21.
The physical server unit resource determination unit 22b is configured to acquire the required number of cores for the physical server 1 in which an operating system different from the operating system (operating system or standby system) of the application APL included in the newly arranged VM 12 is already arranged. It shall not be judged whether or not the vacancy can be secured.

Here, with reference to FIGS. 7 and 8, a specific example of the process of the physical server unit resource determination unit 22b will be described. FIG. 7 is a diagram described in the process of the CPU unit resource determination unit 22a, and thus the description thereof is omitted here.
In the state shown in FIG. 7, when the number (in this case, eight) greater than the number of vacant cores of each CPU 10 is notified as the required number of cores of VM 12 to be newly disposed, physical server unit resource judging means 22b , Rearranges the CPUs 10 of the physical server 1 in descending order of the number of free cores. Here, rearranging the CPUs 10 means that the physical server unit resource determination unit 22 b changes the order of the CPUs to be processed.
For example, as illustrated in FIG. 8A, the physical server unit resource determination unit 22b rearranges the arrangement of the CPUs 10 in FIG. 7 in the order of the CPUs ₁ , ₃ , ₃ , ₄ and ₂ .

Then, the physical server unit resource determination unit 22b determines whether the allocation of the cores 11 being used can be changed in order from the CPU 10 with the largest number of vacant cores to the CPU 10 with the smallest number of vacant cores. This is because if the assignment is changed from the CPU 10 with many vacant cores to the CPU 10 with few vacant cores, it is possible to secure more vacant cores. The assignment change of the core 11 is performed in units of VMs 12.
For example, in the example of FIG. 8A, three VMs 12 of vm1, vm2, and vm3 as change targets exist in the CPU 10 (CPU ₁ ) having the largest number of empty cores. Then, the physical server unit resource determination unit 22b calculates the number of cores lacking (the number of insufficient cores) to the required number of cores of the VM 12 to be newly disposed, and the combination of the existing VMs 12 has a combination larger than the number of insufficient cores. Generate a list. In the example of FIG. 8A, since the CPU ₁ has eight cores required and the number of insufficient cores is one, the corresponding combinations are only vm1, only vm2, only vm3, only vm1, vm2, and vm1. And vm3, vm2 and vm3, vm1 and vm2 and vm3, and seven ways.

Here, the physical server unit resource determination unit 22b searches for a combination that can be assigned to the CPU 10 with a smaller number of vacant cores in the descending order of the number of cores in each combination.
In the example of FIG. 8A, vm1, vm2 and vm3 (11 cores), vm1 and vm3 (9 cores), vm1 and vm2 (8 cores), vm1 (cores) in descending order of the number of cores The number is six, vm2 and vm3 (five cores), vm3 (three cores), and vm2 (two cores) in this order.

Then, the physical server unit resource determination unit 22b searches for a combination that can be allocated and changed to the CPU 10 with the smaller number of vacant cores in order from the one with the largest number of cores. In this case, each combination of vm1 and vm2 and vm3 (11 number of cores), vm1 and vm3 (9 number of cores), and vm1 and vm2 (8 numbers of cores) should be allocated with no space to any other CPU I can not Then, vm1 (the number of cores: 6) having the next largest number of cores can not be allocated to the CPU ₂ having the smallest number of empty cores, but as shown in FIG. It can be allocated to a small number of CPUs ₄ . That is, the physical server unit resource determination unit 22b secures the resource for the new VM 12 if the resource reorganization (compaction) is performed so that the VM (vm1) allocated to the CPU ₁ is allocated to the CPU _4. It is determined that

When the total number of cores is the same and the number of VMs 12 is different among the combinations which can be changed, it is desirable to prioritize the change with the smaller number of VMs 12 being selected. This is because as the number of VMs 12 to be changed increases, the time required for the change, the increase in the processing procedure, and the like affect the entire data center S as a whole.
As a result, as shown in FIG. 8C, the physical server unit resource determination unit 22b may arrange the new VM 12 (here, the number of cores 8) in the CPU ₁ by the compaction in the physical server 1. Determine that it is possible.

Here, the physical server unit resource determination unit 22b determines whether resources can be secured or not on the basis of the CPU 10 (CPU _{1 in} this case) with the largest number of vacant cores, but if resources can not be secured, the resources are sequentially vacant. The same processing may be performed in order from the CPU 10 having the large number of cores (in the example of FIG. 8, CPU ₁ → CPU ₃ → CPU ₄ → CPU ₂ ).

When the physical server unit resource determination unit 22b determines that the resource (the required number of cores) necessary for the new VM can be secured, identification information for identifying the physical server 1 and the CPU 10 that can be allocated (in the example of FIG. 8) , SV ₁ , CPU ₁ ) to the placement position control means 21. Further, in order to secure resources, a compaction procedure (in the example of FIG. 8, the assignment change of vm1 from the CPU ₁ to the CPU ₄ of SV ₁ ) is outputted to the placement position control means 21.
If the physical server unit resource determination unit 22b can not secure the required number of cores by compaction, the physical server 1 (the physical server following the physical server list in the data center 211 (see FIG. 5A)) The CPU unit resource determination unit 22a is instructed to search for a resource in Then, in the case where the required number of cores can not be secured due to compaction in all the physical servers 1, the physical server unit resource determination means 22b outputs the required number of cores to the all physical server resource determination means 22c.
Returning to FIG. 3, the description of the configuration of the virtual machine placement device 2 will be continued.

The all-physical server resource determination unit 22c, when the required number of cores can not be secured in the physical server 1, allows the cores allocated to the running VMs 12 among all the physical servers 1 to be the CPUs 10 of the other physical servers 1. By reassigning to, it is determined whether the required number of cores can be secured on the same CPU.
If all resources (number of required cores) can be secured by compaction, the all physical server resource determination unit 22c outputs the identification information of the physical server 1 having the resource and the CPU 10 to the placement position control unit 21. The all physical server resource determination unit 22 c outputs the compaction procedure to the placement position control unit 21.

For example, as illustrated in FIG. 9A, all physical server resource determination unit 22c has four CPUs (CPU _{1 to} CPU ₄ ) each having 18 cores of physical servers 1 (SV ₁ , SV ₂ , SV ₃ ), It is assumed that two CPUs (CPU ₅ and CPU ₆ ) and two CPUs (CPU ₇ and CPU ₈ ) are used, and the cores 11 are used by a plurality of VMs 12 in each CPU.
In this case, the all-physical server resource determination unit 22c rearranges the CPUs 10 in descending order of the number of free cores, as shown in FIG. 9 (b). That is, while the physical server unit resource determination means 22b performs rearrangement within one physical server 1 (see FIG. 8), the all physical server resource determination means 22c targets all physical servers 1 as follows. Rearrange the CPU 10.
The subsequent processing is the same as the processing (see FIG. 8) of the physical server unit resource determination unit 22b, and thus the description thereof is omitted.

However, as shown in FIG. 8B, when searching for a resource whose core allocation is to be changed, the all physical server resource determination unit 22c operates and standbys the same application in the physical server 1 after the change. However, the resource search is performed so as not to operate on the CPU 10 of the same physical server 1.
In addition, all physical server resource determination means 22c is required for the physical server 1 in which an operating system different from the operating system (operating system or standby system) of the application APL included in the VM 12 to be newly arranged is already arranged. It shall not be judged whether or not the vacancy can be secured.

If all physical server resource determination means 22c determines that the resources required for the new VM (number of required cores) can be secured, the identification information for specifying the physical server 1 and the CPU 10 that can be arranged is the placement position control means Output to 21 Also, a compaction procedure is output to the placement position control means 21 in order to secure resources.
Then, when all the physical servers 1 can not secure the required number of cores by compaction, the all physical server resource determination unit 22c outputs to the placement position control unit 21 that the resources can not be secured.
As described above, the all physical server resource determination unit 22c determines whether or not the necessary number of cores of the VM 12 to be newly arranged is available for all the physical servers 1 (that is, the data centers S).
Returning to FIG. 3, the description of the configuration of the virtual machine placement device 2 will be continued.

The compaction execution unit 23 selects the core 11 to which the VM 12 in operation is allocated based on the compaction procedure output from the arrangement position control unit 21, the other CPU 10 of the same physical server 1 in which the VM 12 is operating, Alternatively, they are reassigned (compaction) to the CPUs 10 of other physical servers 1. The compaction execution unit 23 operates when it is determined by the free resource determination unit 22 that the required number of cores can not be secured on the same CPU without performing compaction.
Here, the compaction execution unit 23 includes an intra-physical server allocation change unit 23 a and an inter-physical server allocation change unit 23 b.

The physical server internal allocation changing means 23 a changes the allocation of the VM 12 to the core 11 in the physical server 1. The intra-physical-server allocation changing means 23a operates when a procedure for changing the allocation of the cores 11 is designated in the same physical server 1 as the compaction procedure. That is, the in-physical server allocation change unit 23a executes the compaction procedure determined by the physical server unit resource determination unit 22b to be able to secure the resource.
For example, physical servers within allocation change means 23a, as shown in FIG. 10, the virtual CPU of VM12B assigned to the core of the CPU 10B ₁ (vCPUs), assigned to the other CPU 10B ₂ of the core of the same physical server 1B. In addition, since the method of this allocation is a conventional general method (nonpatent literature 1), detailed description is abbreviate | omitted here.

The intra-physical-server allocation changing unit 23a updates the resource management data stored in the storage unit 210 after the allocation change. That is, the physical server allocation changing means 23a deletes the VM 12 whose allocation has been changed from the "used VM" in the in-CPU core list 213 (see FIG. 5C) of the CPU which released the core, and the VM 12 uses it. Change the core's "use" to "unused". Further, the physical server allocation changing means 23a "unused" the cores 11 allocated to the VM 12 whose allocation has been changed in the in-CPU core list 213 (see FIG. 5C) of the CPU 10 newly allocated cores. While changing to "use", the identification information of the VM 12 whose allocation is changed to "use VM" and the identification information of the virtual CPU 13 allocated to the core 11 are set.
The physical server internal allocation changing means 23a executes the compaction and updates the resource management data, and then notifies the placement position control means 21 that the compaction is completed.

The inter-physical-server allocation changing unit 23 b changes the allocation of the VM 12 to the core 11 among the different physical servers 1. The inter-physical-server allocation changing means 23b operates when a procedure for changing the allocation of the cores 11 is specified between different physical servers 1 as a compaction procedure. That is, the inter-physical-server allocation changing unit 23b executes the compaction procedure determined by the all-physical server resource determining unit 22c that the resources can be secured.
For example, as illustrated in FIG. 11, the inter-physical-server allocation changing unit 23b allocates the VM 12B allocated to the core of the CPU 10B of the physical server 1B to the core of the CPU 10C of the physical server 1C. That is, the inter-physical server allocation changing unit 23b moves the VM 12B from the physical server 1B to the physical server 1C, and changes the core allocation.

The inter-physical-server allocation changing unit 23b updates the resource management data stored in the storage unit 210 after the allocation change. The update of the resource management data of the inter-physical-server allocation changing means 23b is the same as that of the in-physical-server allocation changing means 23a, except that the physical server 1 is straddled.
In order to move the VM 12 between different physical servers 1, the inter-physical-server allocation change unit 23 b needs to control the activation state of the VM 12 and perform the movement.

Here, with reference to FIGS. 11 and 12, the change of the core allocation of the VM 12 in which the application is operating in the active / standby configuration will be described.
In the “0. initial state”, in the physical server 1A, it is assumed that the application is running in the standby (SBY) state in the VM 12A and the application is running in the active (ACT) state in the VM 12B in the physical server 1B. . That is, in the physical servers 1A and 1B, it is assumed that applications are duplicated and operating.
Here, it is assumed that the VM 12B is moved from the physical server 1B to the physical server 1C.

  As “1. System switching”, the inter-server allocation changing means 23b changes the application being activated as the SBY state in the VM 12A of the physical server 1A to the ACT state, and is activating as the ACT state in the VM 12B of the physical server 1B Change the application to SBY state. This changes the operating system of the application.

  As “2. SBY stop”, the inter-physical server allocation change unit 23 b stops the application of the VM 12 B of the physical server 1 B. Even in this case, one system (operating system) of the application is operating on the physical server 1A.

  As “3. VM migration”, the inter-physical-server allocation changing unit 23b migrates the VM 12 (including the application) from the physical server 1B to the physical server 1C. That is, the inter-physical-server allocation changing unit 23b copies the VM 12 (including the application) to the physical server 1C and deletes the VM 12 from the physical server 1B. At this time, the inter-physical-server allocation changing unit 23b changes the core allocation as described above.

As “4. SBY startup”, the inter-physical-server allocation changing unit 23b starts the application of the VM 12B of the physical server 1C in the SBY state.
As “5. System switching”, the inter-server allocation changing means 23b changes the application being activated as the ACT state in the VM 12A of the physical server 1A to the SBY state, and in the SBY state as the SBY state in the VM 12B of the physical server 1C. Change the application to ACT state.

Here, the case where the application of the migration target VM 12B is in the ACT state in the initial state has been described.
However, when the application of the migration target VM 12B is in the SBY state in the initial state, that is, when the application of the VM 12A in which the application is duplicated is in the ACT state in the initial state, the inter-physical server allocation change unit 23b is shown in FIG. In the above, the processing from “1. system switching” to “4. SBY startup” may be performed.
In this way, the physical server inter-server allocation changing means 23b can change the core allocation without interrupting the service of the application.
Returning to FIG. 3, the description of the configuration of the virtual machine placement device 2 will be continued.

The placement execution means 24 places the new VM 12 at the placement position determined by the placement position control means 21.
The placement execution unit 24 copies the program of the new VM 12 to the physical server 1 designated as the placement position via the communication unit 200, and puts the virtual CPU 13 of the VM 12 in the empty core of the CPU 10 designated similarly as the placement position. assign.

The placement execution unit 24 refers to the resource management data (see FIG. 5) stored in the storage unit 210, and the cores of the CPU 10 of the designated physical server 1 are not used by the number of cores required. Allocate virtual CPU of VM12. Then, the placement execution unit 24 sets “used / unused” of the allocated core to “used” in the in-CPU core list 213 (refer to FIG. 5C) of the resource management data, and “used VM”. Then, the identification information of the newly arranged VM 12 and the identification information of the virtual CPU 13 are set.
In addition, the arrangement execution means 24 sets the identification information of the VM 12 for identifying the program of the new VM 12 in the operation state list 214 (see FIG. 6) stored in the storage unit 210, and the identification information and operation of the application. The identification information (ACT / SBY) indicating whether it is a system or a standby system is set.

  Note that, when instructed to delete the VM 12 from the outside, the placement execution unit 24 instructs the corresponding physical server 1 to delete the VM 12 via the communication unit 200, and the resource management data of the storage unit 210 is The CPU core list (see FIG. 5C) and the operation state list (see FIG. 6) are updated.

As described above, by configuring the virtual machine placement device 2, the virtual machine placement device 2 assigns the virtual CPU (vCPU) 13 of the virtual machine (VM) 12 to the core 11 of the CPU 10 and operates it. Allocation processing can be performed automatically.
In addition, even when resource fragmentation occurs by repeating acquisition of resources by allocation of VM 12 and recovery of resources by suspension, the virtual machine allocation device 2 secures resources for the VM 12 to be newly allocated by compaction. be able to.
Note that the virtual machine placement device 2 can operate a computer (a virtual machine placement program) that causes the computer to function as the above-described means.

<< Operation of virtual machine placement device >>
Next, the operation of the virtual machine placement device 2 will be described with reference to FIGS. 13 and 14 (for the configuration, refer to FIGS. 1 to 3 as needed).
First, when the virtual machine placement device 2 is instructed by the placement location control means 21 to place a new VM 12, the CPU unit resource determination means 22a of the free resource determination means 22 tries to reserve resources in CPU units.
That is, the free resource determination unit 22 selects the first physical server from the data center physical server list 211 (see FIG. 5A) stored in the storage unit 210 (step S1). The free resource determination unit 22 refers to the operation state list 214 (see FIG. 6) and operates the same application as another operation system (operating system or standby system) different from the application of the VM 12 to be newly added. If it is, the physical server that is operating is excluded from selection.

  Then, the CPU unit resource determination unit 22a selects the first CPU from the physical server CPU list 212 (see FIG. 5B) corresponding to the selected physical server (step S2).

  Then, with reference to the in-CPU core list 213 (see FIG. 5C) corresponding to the selected CPU 10, the CPU unit resource determination unit 22a determines whether or not the required number of cores of the VM 12 to be newly arranged is available. Is determined (step S3). That is, the CPU unit resource determination unit 22 a determines whether the number of “unused” of the cores 11 in the in-CPU core list 213 is equal to or more than the required number of cores.

Here, when there is a space required for the required number of cores (Yes in step S3), the virtual machine arrangement device 2 arranges the VM 12 with the selected CPU as the arrangement position by the arrangement execution unit 24 (step S4). Here, the CPU unit resource determination means 22a notifies the arrangement position control means 21 of the identification information of the selected physical server 1 and CPU 10 as the arrangement position, and the arrangement position control means 21 arranges the VM 12 at the arrangement position Is instructed to the placement execution means 24 to place the VM 12.
Then, the placement execution unit 24 updates the in-CPU core list (see FIG. 5C) of the resource management data of the storage unit 210 and the operation state list (see FIG. 6) (not shown).

On the other hand, when the selected CPU does not have the necessary number of cores available (No in step S3), the CPU unit resource determination unit 22a has the next CPU 10 in the physical server CPU list 212 (see FIG. 5B). It is determined whether or not (step S5).
Here, when the next CPU 10 exists (Yes in step S5), the CPU-based resource determination unit 22a selects the next CPU 10 (step S6), and returns the operation to step S3.
When the next CPU 10 does not exist (No in step S5), the CPU unit resource determination unit 22a shifts the operation to step S7 (FIG. 14) (A).

  In this case, the resource for the new VM 12 can not be secured in the CPU unit in the selected physical server 1. Therefore, the virtual machine allocation device 2 uses the physical server unit resource determination unit 22b to select resources for the entire selected physical server. Try to secure.

  That is, the physical server unit resource determination unit 22 b determines the internal CPUs corresponding to all the CPUs 10 included in the physical server 1 according to the physical server CPU list 212 (see FIG. 5B) corresponding to the selected physical server 1. It is determined with reference to the core list 213 (see FIG. 5C) whether or not the required number of cores of the VMs 12 to be newly disposed in the physical server 1 can be secured (step S7). At this time, the physical server unit resource determination unit 22b determines whether or not the resources (the required number of cores) can be secured by compaction in the physical server 1 as described with reference to FIGS. 7 and 8.

  Here, when the required number of cores can be secured (Yes in step S7), the virtual machine allocation device 2 executes the compaction in the physical server 1 by the in-physical server allocation changing means 23a of the compaction execution means 23 (step S8). ). Here, the physical server unit resource determination means 22b notifies the placement position control means 21 of the compaction procedure in which the resource is secured by the compaction, and the physical server internal allocation change means 23a executes the compaction procedure.

Then, the virtual machine placement device 2 places the VM 12 at the placement position where the resource is secured by the placement execution unit 24 (step S9). Here, the physical server unit resource determination unit 22b notifies the arrangement position control unit 21 of the identification information of the physical server 1 and the CPU 10 capable of securing resources by compaction as the arrangement position, and the arrangement position control unit 21 arranges the arrangement position The placement of the VM 12 is executed by instructing the placement execution unit 24 to place the VM 12 on the
Then, the placement execution unit 24 updates the in-CPU core list (see FIG. 5C) of the resource management data of the storage unit 210 and the operation state list (see FIG. 6) (not shown).

On the other hand, when the required number of cores can not be secured in the selected physical server 1 (No in step S7), the physical server unit resource determination unit 22b continues to the physical server list in data center 211 (see FIG. 5A). It is determined whether there is any physical server 1 (step S10).
Here, when the next physical server exists (Yes in step S10), the physical server unit resource determination unit 22b selects the next physical server 1 (step S11), and returns the operation to step S2 (FIG. 13). (B).

  The physical server unit resource determination unit 22b refers to the operation state list 214 (see FIG. 6) and the same application as another operation system (operating system or standby system) different from the application of the VM 12 to be newly added. If it is running, the running physical server is excluded from selection.

Then, when the next physical server 1 does not exist (No in step S10), the physical server unit resource determination unit 22b shifts the operation to step S12.
In this case, since resources for a new VM 12 can not be secured on a physical server basis, the virtual machine allocation device 2 attempts to secure resources on all physical servers 1 by the all-physical server resource determination unit 22c.

  That is, the all-physical server resource determination unit 22c stores the in-physical server CPU list 212 (FIG. 5 (b)) corresponding to all the physical servers 1 specified in the in-data center physical server list 211 (see FIG. 5 (a)). With reference to the in-CPU core list 213 (see FIG. 5C) corresponding to all the CPUs 10 included in all the physical servers 1, the required number of cores of the VM 12 to be newly disposed in the physical server 1). It is determined whether it is possible to secure (step S12). At this time, the all physical server resource determination unit 22c determines whether or not the resources (the number of necessary cores) can be secured in the data center S by compaction.

  Here, when the required number of cores can be secured (Yes in step S12), the virtual machine allocation device 2 executes compaction among the physical servers 1 by the inter-physical server allocation changing means 23b of the compaction execution means 23 (step S13). ). Here, the all physical server resource determination unit 22c notifies the placement position control unit 21 of the compaction procedure in which the resource is secured by the compaction, and the inter-physical server allocation change unit 23b executes the compaction procedure.

  Then, the virtual machine placement device 2 places the VM 12 at the placement position where the resource is secured by the placement execution unit 24 (step S14). Here, the all physical server resource determination means 22c notifies the arrangement position control means 21 of the identification information of the physical server 1 and CPU 10 capable of securing resources by compaction as the arrangement position, and the arrangement position control means 21 arranges the arrangement position The placement of the VM 12 is executed by instructing the placement execution unit 24 to place the VM 12 on the

Then, the placement execution unit 24 updates the in-CPU core list (see FIG. 5C) of the resource management data of the storage unit 210 and the operation state list (see FIG. 6) (not shown).
On the other hand, when the required number of cores can not be secured by all physical servers (No in step S12), the virtual machine placement device 2 causes the placement position control means 21 to display a warning on the display device (not shown) (step S15).
In this case, after a physical server is newly added in the data center S (step S16), a new VM is placed (step S17).

  By the above operation, the virtual machine arrangement device 2 can perform resource management in the data center S, and can secure resources in the VM 12 to be newly arranged by compaction even when resource fragmentation occurs.

As mentioned above, although embodiment of this invention was described, this invention is not limited to this embodiment.
For example, here, the virtual machine placement device 2 updates the resource management data each time the placement position control means 21 is instructed from outside to place or delete the VM 12.
However, the virtual machine placement device 2 updates (releases) the resource by receiving a VM termination notice from the physical server 1 in the data center S via the communication unit 200. It may be Alternatively, the placement execution unit 24 may periodically check the operation of the VM 12 and update (release) the resources of the VM 12 for which the operation check can not be made.
Thus, even when the physical server 1 voluntarily deletes the VM 12, the virtual machine allocation device 2 can manage the resources.

Furthermore, here, the free resource determination unit 22 searches whether or not a resource can be secured for each CPU 10 in a certain physical server 1, and if the resource can not be secured, whether or not the resource can be secured by performing compaction in the physical server 1 Was judged. That is, the free resource determination unit 22 determines whether the resources can be secured by sequentially repeating the search in the CPU unit and the search in the physical server unit for each physical server 1.
However, the free resource determination unit 22 searches whether all the CPUs 10 in all the physical servers 1 can secure resources, and when it is not possible, determines whether resources can be secured by performing compaction in units of physical servers. You may do it. That is, the free resource determination unit 22 may search for all physical servers 1 in units of physical servers after searching for CPU units in all physical servers 1.

S Data Center (Computer System)
1 Physical server (server hardware)
10 CPU
11 cores 12 virtual machines (VMs)
13 Virtual CPU (vCPU)
2 Virtual Machine Placement Device 21 Placement Location Control Means 22 Free Resource Determination Means 22a CPU Unit Resource Determination Means 22b Physical Server Unit Determination Means 22c All Physical Server Resource Determination Means 23 Compaction Execution Means 23a Allocation in Physical Server Allocation Means 23b Allocation Between Physical Servers Changing means 24 Arrangement executing means 200 Communication unit (communication means)
210 Storage unit (storage means)
220 Control unit (control means)

Claims (2)

A computer system comprising a plurality of physical servers having a plurality of CPUs configured with a plurality of cores, wherein virtual machines are allocated to cores of the same CPU in advance and arranged in the physical servers,
Storage means for correlating the physical server, the CPU, and the core, and storing resource management data indicating which virtual machine the core is being used for;
Free resource determination means for determining whether or not the required number of cores required by a new virtual machine can be secured on the same CPU with reference to the resource management data;
When the necessary resource count can not be secured on the same CPU by the free resource determination means, a core to which a virtual machine in operation is allocated is another CPU of a physical server on which the virtual machine is operating, or Compaction execution means for reassigning to CPUs of other physical servers;
Placement execution means for placing the new virtual machine in the secured core on the same CPU;
Equipped with
The free resource determination means
If the required number of cores can not be secured by the same CPU, the cores allocated to the active virtual machine are reassigned to another CPU in each physical server, so that the required number of cores is the same. A physical server unit resource determination unit that determines whether or not it can be secured on the CPU;
When the required number of cores can not be secured in the physical server, the core assigned to the active virtual machine is reassigned to the CPUs of other physical servers among all the physical servers. virtual machine placement apparatus according to claim Rukoto and a full physical server resources determining means for determining whether the number of cores can be secured on the same CPU. In a computer system provided with a plurality of physical servers having a plurality of CPUs composed of a plurality of cores, the virtual machine placement device reserves a resource for assigning virtual machines to the cores of the same CPU in advance and placing them on the physical servers Resource management method, and
The CPU-based resource determination unit associates the physical server, the CPU, and the core with each other, and refers to resource management data indicating which virtual machine the core is being used for, and creates a new virtual within the same CPU. Determining whether or not the necessary number of cores required by the machine can be secured in advance;
When the required number of cores can not be secured in the same CPU, the physical server unit resource determination unit refers to the resource management data, and the cores assigned to the operating virtual machines in each physical server are selected. Determining whether the required number of cores can be secured on the same CPU by reassigning to another CPU;
Changing the core allocation in the physical server by the physical server internal allocation changing unit when the required number of cores can be secured in the physical server;
When it is not possible to secure the required number of cores in the physical server, all physical server resource determination means refers to the resource management data, and a core allocated to a virtual machine in operation among all the physical servers Determining whether or not the required number of cores can be secured on the same CPU by reassigning to the CPUs of other physical servers.
Changing the core allocation among the physical servers by the physical server inter-allocation changing unit when the required number of cores can be secured between the physical servers;
A resource management method comprising:

Images