CN115665049A - Method for balancing and distributing weight for multiple live loads by cloud platform - Google Patents

Abstract

The invention provides a method for a cloud platform to distribute weight for balancing multiple live loads, which belongs to the field of cloud computing and comprises the following steps: 1) Grouping the load balancing clusters; grouping according to the position relation with a physical machine where a back-end server is located, and setting an affinity group and an anti-affinity group; 2) Configuring a weight of each packet; the affinity group setting weight is N (N > 1), the anti-affinity group weight is K (K > 1), and the connection number is distributed according to the proportion of N to K. When the load balancing node communicates with the back-end server, virtual machine communication before the host machine is crossed can be reduced, and the performance of qps inquired every second of the whole load balancing can be effectively improved.

Description

A method for cloud platform to distribute weights for multi-active load balancing

technical field

The invention relates to the field of cloud computing, in particular to a method for cloud platform to distribute weights for multi-active load balancing.

Background technique

Cloud computing is one of the hottest topics in the field of IT infrastructure in recent years. Its virtual abstraction of computing, network, storage and other resources provides users with extremely convenient resource usage and flexible resource expansion capabilities. At the same time, the rapid development of technologies such as computer communication networks has made network overloading and overloading commonplace. In order to solve the network load problem, it is necessary to use load balancing technology to provide more flexible, convenient, fast and efficient cloud computing service capabilities. Load balancing is based on the existing network structure, which provides a cheap, effective and transparent method to expand the bandwidth of network devices and servers, increase throughput, strengthen network data processing capabilities, and improve network flexibility and availability .

Currently, there are many different load balancing technologies to meet different application requirements, such as software/hardware load balancing, local/global load balancing, higher network layer load balancing, and link aggregation technologies.

The current multi-active load balancing relies on equal-cost routing (implemented by multi-path routing in linux), and the hash strategy is based on three layers (source IP hash) and four layers (based on source IP, destination IP, source port, destination port, transport protocol hash), each next hop of the equal-cost route is completely equal, and cannot be flexibly adjusted according to the specific conditions of each load balancing node.

Although the traditional multi-active load balancing implemented by equal-cost routing can be expanded horizontally by increasing the number of virtual machines, the resource usage will also increase exponentially, and the load balancing nodes and back-end servers are randomly distributed on physical machines. It will increase the network communication between virtual machines on many different physical machines, increasing the load and communication pressure of the physical network.

Contents of the invention

In order to solve the above technical problems, the present invention provides a method for cloud platform to distribute weights for multi-active load balancing. By flexibly configuring the weight of multi-path routing, the number of access traffic and connections can be distributed more reasonably, and the QPS performance of multi-active load balancing that relies on multi-path routing is enhanced.

Technical scheme of the present invention is:

A method for assigning weights in multi-active load balancing, comprising:

1) Group the load balancing clusters;

2) Flexible configuration of the weight of each group.

Grouping load balancing refers to grouping according to the location relationship with the physical machines where the backend servers are located, including affinity groups and anti-affinity groups.

The weight of each group can be flexibly configured. The weight of the affinity group is set to N (N>1), and the weight of the anti-affinity group is set to K (K>1). The number of connections is allocated according to the ratio of N to K; the weight can also be customized , but the affinity group weight is greater than the anti-affinity group weight.

further,

First, use the multi-path routing supported by the Linux system kernel to issue an equivalent route whose destination address is the load-balanced VIP on the virtual router vrouter, so that the same destination address can use multiple links at the same time in this network environment, and the next hop is The business network ip address of each load balancing node.

On this basis, when the virtual router vrouter sends the route whose destination address is the load balancing VIP, the weight is adjusted according to whether the load balancing node is compatible with the back-end server or not. If it is compatible, the weight should be set higher , on the contrary, the weight is set lower. In this way, when the load balancing node communicates with the back-end server, it can reduce the virtual machine communication before crossing the host machine, and can effectively improve the qps performance of the overall load balancing query per second.

An affinity group is a load-balancing cluster located on the same physical machine as the back-end server, which loads more than half of the network traffic and acts as a master server; an anti-affinity group is a load-balancing cluster located on a different physical machine from the back-end server , which is responsible for less than half of the network traffic and acts as a backup server.

Going one step further,

After the configuration of the load balancing node and the virtual router vrouter is completed, the configuration of accessing nat from the external network is performed.

First, you need to bind the vip to a floating ip, and then use the public network ip and the floating ip to configure nat on the egress firewall; the traffic accessing the public network ip first reaches the egress firewall, and a dnat is made on the egress firewall to convert the destination ip The fip bound to the load-balanced vip, and then through the snat ns of the network node, reaches the virtual router vrouter, and forwards the traffic to the load-balanced node according to the equivalent route whose destination ip is VIP, and the load-balanced node then uses nginx configuration to forward the traffic to the final application server server.

Going one step further,

Use the client of the public network to access the public network ip to verify the accessibility of the public network, use the client1 and client2 of the internal network to access the vip, and verify the accessibility of the internal network; and test the number of new connections to check the distribution of the number of connections Is it consistent with the set ratio.

The beneficial effect of the present invention is

1) Reduce the virtual machine communication before crossing the host machine, and reduce the pressure on the physical network;

2) The virtual router vrouter distributes more connections and network traffic to the load balancing nodes that are compatible with the backend server, which can improve the overall query per second qps performance of the load balancing.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are the Some embodiments of the invention are not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

The present invention provides a method for allocating weights for multi-active load balancing on a cloud platform. First, use the multi-path routing supported by the kernel of the Linux system to issue an equivalent route whose destination address is the load-balancing VIP on the virtual router vrouter. Realize that the same destination address uses multiple links at the same time in this network environment, and the next hop is the business network ip address of each load balancing node. On this basis, when the virtual router vrouter sends the route whose destination address is the load balancing VIP, the weight is adjusted according to whether the load balancing node is compatible with the back-end server or not. If it is compatible, the weight should be set higher , on the contrary, the weight is set lower. In this way, when the load balancing node communicates with the back-end server, it can reduce the virtual machine communication before crossing the host machine, and can effectively improve the qps performance of the overall load balancing query per second.

The load balancing cluster is divided into two groups, one is an affinity group and the other is an anti-affinity group. An affinity group is a load balancing cluster located on the same physical machine as the backend server, which loads most of the network traffic and acts as a master server. The anti-affinity group is a load balancing cluster located on a different physical machine from the backend server, which is responsible for a small part of network traffic and acts as a backup server.

The specific environment construction process is as follows:

1. Environment preparation requires two networks. One is the service management network. The console configures the load balancing nodes through this network, such as the configuration of the load balancer and the configuration of the virtual machine system of the load balancing node. It is mainly used for For management, business traffic will not pass through this network card; the second is the tenant business network, the private virtual network where the back-end server is located, used to provide VIP for external services, and communicate with the back-end server.

2. Two backend servers, server1 and server2, deploy a simple web service on the server side to test the forwarding effect of load balancing.

3. Apply for a VIP under the tenant business network, and create two sets of virtual machines on the load balancing node. One set of virtual machines is used as an affinity group, created on the same physical machine as the backend server, as the cluster carrying the main traffic, and the other set of virtual machines is used as an affinity group. As an anti-affinity group, the virtual machine is created on a physical machine different from the backend server, and serves as a backup server group to carry a small amount of traffic. Configure the vip and the load balancer on all load balancing nodes, and configure the vip on the lo network card by using ip addr add vip/32dev lo. In addition to the configuration of the load balancer, it is also necessary to disable the arp response for VIP on the load balancer node, using the following command:

echo 1>/proc/sys/net/ipv4/conf/eth1/arp_ignore

echo 1>/proc/sys/net/ipv4/conf/all/arp_ignore

echo 1>/proc/sys/net/ipv4/conf/lo/arp_ignore

echo 2>/proc/sys/net/ipv4/conf/eth1/arp_announce

echo 2>/proc/sys/net/ipv4/conf/all/arp_announce

echo 2>/proc/sys/net/ipv4/conf/lo/arp_announce

It is used to close the arp response on the network card, and enable the arp proxy on the virtual switch, so that no matter whether it is the traffic from the internal network or the external network to access the VIP, it will first pass through the virtual router vrouter, and the virtual router vrouter will distribute the traffic, avoiding two The load balancing node rushes to answer the arp response, resulting in a split-brain situation.

4. Issue equal-cost routes in the virtual router vrouter, set the weight of the affinity group to 3, set the weight of the anti-affinity group to 1, and configure the traffic distribution according to the three-to-one strategy. In this way, the access traffic reaches the virtual After the router vrouter, when establishing a tcp connection, it will follow a five-to-one ratio. For example, if there are 400 new connections, 300 will be allocated to the affinity group, and 100 will be allocated to the anti-affinity group. Use 172.16.122.1 as vip, 172.16.122.2 and 172.16.122.3 as affinity groups for tenant business network ip, 172.16.122.4 and 172.16.122.5 as tenant business network ip for anti-affinity groups, gateway network card name qr-2b749a55-20 For example, execute the following command in the virtual router vrouter:

ip route add 172.16.122.1 nexthop via 172.16.122.2 dev qr-2b749a55-20weight 1nexthop via 172.16.122.3dev qr-2b749a55-20 weight 1nexthop via172.16.122.4dev qr-2b749a55-20 weight 5nexthop via 172.16.122.5dev qr-2b749a55 -20 weight 5

And use the following command to enable arp proxy in the virtual router vrouter:

echo 1>/proc/sys/net/ipv4/conf/qr-2b749a55-20/proxy_arp

echo 1>/proc/sys/net/ipv4/conf/qr-2b749a55-20/proxy_arp_pvlan

In this way, no matter whether it is the traffic of accessing VIP from the internal network or the external network, it will first pass through the virtual router vrouter, and the virtual router vrouter will distribute the traffic, avoiding the situation where two load balancing nodes rush to answer the arp response and cause a split-brain situation.

5. After the configuration of the load balancing node and the virtual router vrouter is completed, configure the external network access nat. First, you need to bind vip to a floating ip, and then use the public network ip and the floating ip to configure nat on the egress firewall (the premise is that the floating network communicates with the egress firewall). The traffic accessing the public network ip first reaches the egress firewall, and a dnat is made on the egress firewall, the destination ip is converted into the fip bound to the load-balanced vip, and then reaches the virtual router vrouter through the snat ns of the network node, according to the destination ip This equal-cost route of vip forwards the traffic to the load balancing node, and the load balancing node forwards the traffic to the final application server server according to the configuration of nginx.

6. Use the client of the public network to access the public network ip to verify the accessibility of the public network, and use the client1 and client2 of the internal network to access the vip to verify the accessibility of the internal network. And do a test of the number of new connections to see if the distribution of the number of connections is consistent with the set five-to-one strategy.

Explanation of some terms in this article: vrouter, virtual machine router; load balancing vip, the virtual address of load balancing; qps, query per second, the number of queries per second, is the traffic processed by a specific query server within a specified time How much to measure.

The above descriptions are only preferred embodiments of the present invention, and are only used to illustrate the technical solution of the present invention, and are not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (8)

1. A cloud platform is aimed at the method for multi-active load balancing distribution weight, it is characterized in that, 1) Grouping the load balancing cluster; refers to grouping according to the location relationship with the physical machine where the backend server is located, and setting an affinity group and an anti-affinity group; 2) Configure the weight of each group; the weight of the affinity group is set to N (N>1), the weight of the anti-affinity group is set to K (K>1), and the number of connections is allocated according to the ratio of N to K. 2. The method of claim 1, wherein, The weight can be customized, and the weight of the affinity group is greater than that of the anti-affinity group. 3. The method of claim 2, wherein, Using the multi-path routing supported by the Linux system kernel, the virtual router vrouter sends the equivalent route with the destination address as the load-balanced VIP, so that the same destination address can use several links at the same time in this network environment, and the next hop is each The service network ip address of the load balancing node. 4. The method of claim 3, wherein, On this basis, when the virtual router vrouter sends the route whose destination address is the load-balancing VIP, the weight is adjusted according to whether the load-balancing node has an affinity relationship with the back-end server. The setting value is lowered. 5. The method of claim 1, wherein, The affinity group is a load balancing cluster located on the same physical machine as the backend server, which loads more than half of the network traffic and acts as a master server; The anti-affinity group is a load balancing cluster located on a different physical machine from the backend server, which is responsible for less than half of the network traffic and acts as a backup server. 6. The method of claim 4, wherein, After the configuration of the load balancing node and the virtual router vrouter is completed, the configuration of external network access nat is performed. 7. The method of claim 6, wherein, First, you need to bind vip to a floating ip, and then use the public network ip and the floating ip to configure nat on the egress firewall; The traffic accessing the public network ip first reaches the egress firewall, and a dnat is made on the egress firewall, the destination ip is converted into the fip bound to the load-balanced vip, and then reaches the virtual router vrouter through the snat ns of the network node, according to the destination ip This equal-cost route of vip forwards the traffic to the load balancing node, and the load balancing node forwards the traffic to the final application server server according to the configuration of nginx. 8. The method of claim 7, wherein, Use the public network client to access the public network ip to verify the accessibility of the public network, use the client1 and client2 of the internal network to access the vip, and verify the accessibility of the internal network; and test the number of new connections to check the distribution of the number of connections Is it consistent with the set ratio.