Background
Currently, the current state of the art commonly used in the industry is such that:
cloud brokers are a new trend in recent years to provide services to users using multi-cloud and hybrid-cloud, and are proposed as a basic cloud service model that enables cloud network selection by renting cloud service provider instances and multiplexing relatively small tenant demands to achieve cost minimization and profit maximization.
The multi-tenant multi-data center selects common scenes with 9 tenants and 10 data centers, and spans 4 continents. Different tenants select multiple data centers according to the availability and the preference of the data centers, and the prior art only uses a mechanism of one data center, cannot meet the requirement of comprehensive data analysis, and cannot ensure faster local data analysis.
In summary, the problems of the prior art are as follows:
(1) the prior art only uses a mechanism of one data center, cannot meet the requirement of data comprehensive analysis, cannot ensure faster local data analysis, and does not have lower cost.
(2) The prior art does not consider the price difference of reserved and real-time resources and cannot reflect the dynamic characteristics of tenant demands in time.
(3) The selection is limited in a plurality of cloud platforms owned by the same cloud service provider, and the selection is difficult to be expanded among the plurality of cloud service providers.
The difficulty and significance for solving the technical problems are as follows:
the scale is huge, the failure rate is high: at present, the number of interconnected servers in a public cloud data center exceeds 105Of the order of magnitude of (1), the number of switching nodes also reaching 104The data centers with increasingly large scale put new requirements on network architecture, transmission protocols and system management. Moreover, the network failure rate increases rapidly with increasing system size, with network configuration failures (38%) and failures of unknown origin (23% such as sudden switch stops forwarding traffic).
The flow is complicated, and the longitudinal extension cost is high: due to the Incast problem caused by a high-burst and high-dynamic many-to-one communication mode, the development of computation-intensive applications such as MapReduce and Hadoop and the wide use of virtualization technologies, not only is the complexity of network traffic behaviors caused, but also a severe transmission load is brought. Meanwhile, the data center has high occupied east-west flow rate and tree structure convergence rate, so that the longitudinal expansion cost of the data center is extremely high and cannot be continued.
Low resource utilization rate, various forms: the traditional data exchange (such as VLAN) and communication identification (such as IP) technology effectively avoids mutual interference when a plurality of applications in a data center are deployed at the same time, but also limits the flexibility of network resource multiplexing, so that the utilization rate of network resources is generally low. In addition, due to the traction of different performance requirements, the situation of coexistence of networks with different forms is formed, including an enhanced ethernet network, an InfiniBand high-speed interconnection storage network, a dedicated high-speed network and the like.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method and a system for self-adaptive scheduling of virtual network resources of a multi-tenant container cloud platform.
The invention is realized in such a way that a self-adaptive scheduling method for virtual network resources of a multi-tenant container cloud platform comprises the following steps:
based on a data center selection strategy of availability and user preference, describing communication paths among containers by using container pairs [ src docker, dstDocker ], assigning each tenant to the nearest data center from an alternative data center set available to a cloud service provider, and determining a data center subset with optimal multi-tenant data placement;
server selection policy based on application performance: the container tenant specifies the efficiency parameters of the application according to the application characteristics, and provides network services of different levels by setting efficiency coefficient combinations [ alpha, beta ];
the network resource self-adaptive scheduling mechanism under the multi-tenant container cloud data center comprises the following steps: on-line self-adaptive cloud network selection algorithm based on historical information, resource reservation charging period T is estimated and divided according to historical information of each tenant on resource usage of cloud service provideriAnd resource multiplexing charging time slot taui(ii) a Let niNumber of instances enabled for the ith multiplex charging slot, CiFor the total cost of the ith multiplexing charging time slot, the optimization target of the online self-adaptive cloud network selection algorithm based on the historical information is to determine the longest resource reservation charging period and the longest resource multiplexing charging time slot tau according to the historical information of the tenanti。
Further, a container pair [ srcDocker, dstDocker ] is used]Describing the communication path between containers, the number of paths for a container pair i is denoted p
iThe bandwidth allocated to the container pair is represented by a vector
Is represented by the formula, wherein x
ijRepresents the bandwidth allocated on path j for the ith container pair; current number ofThe global bandwidth allocation vector is represented as n according to the number of container pairs in the center
The routing matrix is represented as:
further, in the server selection policy based on application performance, the container tenant specifies the performance parameters of the application according to the application characteristics, and the format is as follows [ application id, srcDocker, dst docker, Bmin,α,β]In the formula BminExpressing the minimum bandwidth requirement of the application, alpha and beta respectively expressing the throughput and the delay sensitivity coefficient of the application, and combining alpha, beta by setting the efficiency coefficient]Providing different levels of network services, the performance function is:
where u represents the set of paths used by the container pair; v represents the set of links used by the path; x is the number ofkwRepresenting the bandwidth allocated to application k on link w, 1/ywRepresenting the expected value of the congestion delay on the link w; coefficient of performance αkAnd betakRespectively representing the throughput and delay sensitive characteristics of application k.
Further, a network resource adaptive scheduling mechanism under the multi-tenant container cloud data center further includes: modeling a system: the price difference between the resource reservation and the real-time lease is utilized to reduce the cost per se.
The invention also aims to provide a computer program for realizing the self-adaptive scheduling method of the virtual network resources of the multi-tenant container cloud platform.
The invention also aims to provide an information data processing terminal for realizing the self-adaptive scheduling method of the virtual network resources of the multi-tenant container cloud platform.
Another object of the present invention is to provide a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to execute the method for adaptive scheduling of virtual network resources of a multi-tenant container cloud platform.
Another objective of the present invention is to provide a system for adaptively scheduling virtual network resources of a multi-tenant container cloud platform, which implements the method for adaptively scheduling virtual network resources of the multi-tenant container cloud platform, and the system includes:
a data center selection strategy unit based on availability and user preference describes communication paths among containers by using container pairs [ src docker, dstDocker ], each tenant is assigned to the nearest data center from an alternative data center set available to a cloud service provider, and a data center subset with optimal multi-tenant data placement is determined;
the server based on the application efficiency selects a policy unit: the container tenant specifies the efficiency parameters of the application according to the application characteristics, and provides network services of different levels by setting efficiency coefficient combinations [ alpha, beta ];
a network resource adaptive scheduling mechanism unit under a multi-tenant container cloud data center, an online adaptive cloud network selection algorithm based on historical information, and a resource reservation charging period T estimated and divided according to the historical information of each tenant on the use of cloud service provider resourcesiAnd resource multiplexing charging time slot taui(ii) a Let niNumber of instances enabled for the ith multiplex charging slot, CiFor the total cost of the ith multiplexing charging time slot, the optimization target of the online self-adaptive cloud network selection algorithm based on the historical information is to determine the longest resource reservation charging period and the longest resource multiplexing charging time slot tau according to the historical information of the tenanti。
The invention also aims to provide a network charging platform carrying the self-adaptive scheduling system for the virtual network resources of the multi-tenant container cloud platform.
In summary, the advantages and positive effects of the invention are:
the invention provides a cloud service environment with multiple tenants and multiple data centers, which is characterized in that various network resources in a container cloud platform are adaptively scheduled in a cooperative mode, the problem of network resource management in the cloud computing environment is considered globally, and the benefit balance of both cloud service supply and demand parties is realized on the premise of ensuring a service level agreement.
The invention aims to estimate and divide the resource reservation charging period T according to the historical information of each tenant on the resource use of the cloud service provideriAnd resource multiplexing charging time slot taui. If n is setiNumber of instances enabled for the ith multiplex charging slot, CiFor the total charge of the ith multiplexing charging time slot, the optimization goal of the algorithm is to determine the longest resource reservation charging period and resource multiplexing charging time slot tau according to the tenant historical informationiThe benefit of the cloud broker is maximized.
Compared with a mechanism using only one data center, the mechanism using a plurality of data centers can not only meet the requirement of data comprehensive analysis, but also ensure faster local data analysis and have lower cost.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The prior art only uses a mechanism of one data center, cannot meet the requirement of data comprehensive analysis, cannot ensure faster local data analysis, and does not have lower cost.
The self-adaptive scheduling method for the virtual network resources of the multi-tenant container cloud platform provided by the embodiment of the invention comprises the following steps:
s101: based on the data center selection strategy of availability and user preference, using a container pair to describe a communication path between containers, starting from an alternative data center set available to a cloud service provider, assigning each tenant to the nearest data center, and determining a data center subset with optimal multi-tenant data placement;
s102: server selection policy based on application performance: the container tenant specifies the efficiency parameters of the application according to the application characteristics, and provides network services of different levels by setting efficiency coefficient combinations;
s103: the network resource self-adaptive scheduling mechanism under the multi-tenant container cloud data center comprises the following steps: an online self-adaptive cloud network selection algorithm based on historical information estimates and divides a resource reservation charging period and a resource multiplexing charging time slot according to the historical information of each tenant on the resource use of a cloud service provider; and setting the number of enabled instances of the ith multiplexing charging time slot as the total cost of the ith multiplexing charging time slot, and determining the longest resource reservation charging period and the longest resource multiplexing charging time slot according to the tenant historical information by using the optimization target of the online adaptive cloud network selection algorithm based on the historical information.
In step S101, a container pair [ src docker, dstDocker ] is used to describe a communication path between containers, each tenant is assigned to a nearest data center from an alternative data center set available to a cloud service provider, and a data center subset with optimal placement of multi-tenant data is determined;
in step S102, network services of different levels are provided by setting an efficiency coefficient combination [ alpha, beta ];
step S103: resource reservation charging period Ti and resource multiplexing charging time slot tau are estimated and divided according to historical information of cloud service provider resource usage of each tenanti(ii) a Setting ni as the number of enabled instances of the ith multiplexing charging time slot and Ci as the total cost of the ith multiplexing charging time slot, determining the longest resource reservation charging period and resource multiplexing charging time slot tau according to tenant historical information by the optimization objective of the online adaptive cloud network selection algorithm based on the historical informationi。
The invention is further described below with reference to specific assays.
One, data center selection policy based on availability and user preferences:
the multi-tenant multi-data center selects common scenes with 9 tenants and 10 data centers, and spans 4 continents. Different tenants select multiple data centers according to the availability and the preference of the data centers, and compared with a mechanism using only one data center, the mechanism using multiple data centers can meet the requirement of comprehensive data analysis, can ensure faster local data analysis and has lower cost.
A data center Tree topology represented by a Fat-Tree is shown in fig. 2 as an example, and is composed of an access layer, a convergence layer, and a core layer. Wherein the number of communication paths across the container cluster is determined by the number of core layer switches and the number of communication paths within the container cluster is determined by the number of switches of the aggregation layer within the cluster.
The topology may be described in a weighted undirected graph G ═ N, L, where N represents a set of switches; l ═ 1,2.., L } (L ≧ 2) represents a set of physical links; the bandwidth capacity and the residual capacity of the link are respectively used as vectors
And
and (4) showing.
The invention contemplates the use of a container pair srcDocker, dstDocker]Describing the communication paths between containers, if the number of paths available to a container for i is denoted as p
iThen the bandwidth allocated to the pair of containers may be represented by the vector
Is represented by the formula, wherein x
ijIndicating the bandwidth allocated on path j for the ith pair of containers. Assuming that the number of container pairs in the current data center is n, the global bandwidth allocation vector may be represented as
The routing matrix may be represented as:
the invention is to store the data of each tenant only on one data center, so that the data can meet the following targets:
preferred data placement: the maximum weighted distance between the tenant and the selected data center is minimized.
Transfer cost minimization data placement: the weighted distance sum between the tenant and the selected data center is minimized.
Fair data placement: the maximum distance between the tenant and the selected data center is minimized.
The total cost is minimized: the sum of the tenant costs is minimized.
Based on the above objectives, the present invention is intended to start with an alternative set of data centers available to the cloud service provider, attempting to assign each tenant to the closest (i.e., the lowest total cost between the user and the selected data center) data center to determine the best subset of data centers for multi-tenant data placement.
Secondly, selecting a strategy based on the server of the application efficiency:
in order to provide a bandwidth allocation service with a fine granularity distinction, the invention designs a broadband allocation mode based on application efficiency for a container tenant. The container tenant may specify the performance parameters of the application according to the application characteristics, and the format is as follows [ application ID, srcDocker, dstDocker, Bmin,α,β]In the formula BminExpressing the minimum bandwidth requirement of the application, alpha and beta respectively expressing the throughput and the delay sensitivity coefficient of the application, and combining alpha, beta by setting the efficiency coefficient]Providing different levels of network services, the invention is designedThe efficacy function is as follows:
where u represents the set of paths used by the container pair; v represents the set of links used by the path; x is the number ofkwRepresenting the bandwidth allocated to application k on link w, 1/ywRepresenting the expected value of the congestion delay on the link w; coefficient of performance αkAnd betakRespectively representing the throughput and delay sensitive characteristics of application k. The performance function designed by the present invention is determined by all the container sets, paths and links used.
Thirdly, a network resource self-adaptive scheduling mechanism under the multi-tenant container cloud data center:
due to the constraint of multiple factors such as a virtualized resource pricing mode, a lease period and data transmission across cloud service providers, the service provided by a cloud broker by using a single cloud service provider instance is a basic mode for cloud network selection at present and in a future period. The cloud broker multiplexes the multi-tenant network requirements through mechanisms such as resource reservation and dynamic adjustment, and maximizes the self income on the premise of ensuring the user service agreement.
The invention is based on the following facts:
tenant requirements: according to the historical information and the demand plan of each tenant, the cloud broker can estimate the total demand of the tenant in a longer period T, and the increase of the total demand curve along with the time T is monotonically increased.
Cloud service provider pricing: the resource reservation cost strictly monotonically increases with increasing time T, while the average cost strictly monotonically decreases with increasing time T.
Modeling a system:
due to the dynamic change of the tenant demand, the cloud broker reduces its own cost by using the price difference between the resource reservation and the real-time lease, and an example of the network resource multiplexing mode commonly adopted by the cloud broker is shown in fig. 3.
An online self-adaptive cloud network selection algorithm based on historical information comprises the following steps:
according to the network resource multiplexing mode shown in fig. 3, the invention proposes to estimate and divide the resource reservation charging period T according to the historical information of each tenant on the resource usage of the cloud service provideriAnd resource multiplexing charging time slot taui. If n is setiNumber of instances enabled for the ith multiplex charging slot, CiFor the total charge of the ith multiplexing charging time slot, the optimization goal of the algorithm is to determine the longest resource reservation charging period and resource multiplexing charging time slot tau according to the tenant historical informationiThe benefit of the cloud broker is maximized.
As shown in fig. 4, an embodiment of the present invention provides a system for adaptive scheduling of virtual network resources of a multi-tenant container cloud platform, including:
the data center selection strategy unit 1 based on availability and user preference describes communication paths among containers by using container pairs [ src docker, dstDocker ], assigns each tenant to the nearest data center from an alternative data center set available to a cloud service provider, and determines a data center subset with optimal multi-tenant data placement;
the server selection policy unit 2 based on the application efficiency: the container tenant specifies the efficiency parameters of the application according to the application characteristics, and provides network services of different levels by setting efficiency coefficient combinations [ alpha, beta ];
a network resource adaptive scheduling mechanism unit 3 under a multi-tenant container cloud data center, an online adaptive cloud network selection algorithm based on historical information, and resource reservation charging period T estimated and divided according to the historical information of each tenant on the use of cloud service provider resourcesiAnd resource multiplexing charging time slot taui(ii) a Let niNumber of instances enabled for the ith multiplex charging slot, CiFor the total cost of the ith multiplexing charging time slot, the optimization target of the online self-adaptive cloud network selection algorithm based on the historical information is to determine the longest resource reservation charging period and the longest resource multiplexing charging time slot tau according to the historical information of the tenanti。
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When used in whole or in part, can be implemented in a computer program product that includes one or more computer instructions. When loaded or executed on a computer, cause the flow or functions according to embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.)). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.