CN1878147A - Method for balancing load between multi network cards of network file system server - Google Patents

Abstract

The present invention provides a method for balancing the load among multiple network cards of a network file system server, comprising: the network file system server configures a network address for each network card; selects a main network card among multiple network cards, and creates a socket for each network card; Bind the network card with the socket structure; the network file system client obtains the server's Ethernet address; the client selects a network address in a round-robin manner from the obtained server network card addresses to send a file access request; the server After the end receives the request, it is processed by the network file system service process, and the processing result is sent to the client through the network card that accepts the request. The invention makes full use of the bandwidth resources of the server network and improves the file access throughput rate.

Description

Method for Balancing the Load among Multiple Network Cards of Network File System Server

technical field

The invention relates to a network file system server, in particular to a method for balancing the load among multiple network cards of the network file system server.

Background technique

The ever-increasing demand for sharing files between different hosts has promoted the development of distributed file systems, and NFS is the earliest distributed file system. With the increasing operation scale of scientific computing and other applications, the storage capacity of Network File System (NFS) servers is constantly expanding, so NFS servers are beginning to be equipped with 100M and 1000M Ethernet cards. With the user's requirements for network speed and bandwidth, most of the current NFS servers are equipped with multiple network cards to increase the data transmission speed.

However, in all current NFS implementations, the NFS client can only communicate with the NFS server through one network card. For a high-performance I/O server with multiple physical network cards, this implementation cannot fully utilize the server-side network load support capability. .

The current NFS server generally adopts the following methods for configuring multiple network cards:

(1) The multiple network cards of the server are configured on different network segments, and the clients are divided into several groups at the same time. The network cards of each group of clients are configured on the same network segment, and the clients of different groups are configured on different network segments. The client uses the NFS server network card address on the same network segment as itself to mount the network file system, and all subsequent file accesses access the server through the server network card.

The above-mentioned NFS multi-network card configuration method makes the utilization rate of the network card not high. When a group of clients accessing the server has a large load and causes network congestion, while another group of clients assigned another network card has a light load, it will cause The load of the two network cards is unbalanced, and the I/O bandwidth of the server cannot be fully utilized. In addition, once the network fails, the client can no longer access the NFS server because the backup network cannot be selected. If multiple network cards are configured on the same network segment, the server will always select the first network card to send data according to the routing rules, and it is difficult to use the transmission capabilities of multiple network cards at the same time.

(2) Using the network card binding technology, the main idea of the multi-network card binding technology is to bind multiple network cards of the server into a virtual network card. Multiple clients access the NFS server through a virtual network card. When the server network device sends data, multiple physical network cards are driven by the network device to send data packets in turn.

The above configuration method generally needs the support of the switch, because the switch also needs to send the data packets to multiple physical network cards of the server in turn. However, for switches that do not support round-robin sending, the load balancing effect of the binding mode is not ideal, and even errors occur.

Contents of the invention

The purpose of the present invention is to overcome the current NFS server configuration and use of multiple network cards can not well solve the problem of load balancing between multiple network cards, can not fully utilize the bandwidth provided by the network, the defect of low data transmission efficiency, thereby providing A method that can better balance the load among multiple network cards.

In order to achieve the above object, the present invention provides a method for balancing the load between multiple network cards of a network file system server, which is applied to a network file system composed of a network file system server, a switch, and a client. The method includes the following steps:

1), configure network addresses for all network cards on the network file system server;

2), obtain the physical address of all network cards on the network file system server, the network address and the title of physical equipment, and record in the network card information table of the network file system server;

3), select a network card from all network cards on the network file system server as the main network card, and the remaining network cards are slave network cards, and specify the port number for the main network card;

4), the network file system server according to the network card information table that step 2) obtains, creates corresponding socket structure for each network card, and designates a port number for each from network card, initializes described socket structure, will The network address obtained in port number and step 2) is filled in the described socket structure;

5), select the socket structure of the main network card as the main socket, and register the port number of the main network card in the port mapper of the server;

6), bind each network card with the socket structure specified by the network card;

7), the client sends to the server a request to obtain the server network card physical address information, network address information and port number information, and saves the result returned by the server;

8), according to the information of the network card obtained in step 7), and the automatic setting client arp configuration file that comes with the client, fill the network address and physical address of all network cards of the server into the file of the arp protocol to read the static analysis network address , and then run the arp configuration file;

9), the client uses the network address of the main network card of the server to install the network file system;

10), the client communicates with the server through the switch, the client intercepts the remote procedure call protocol packet, the address field of the packet includes the port number and the network address of the destination network card, and the client obtains the server network address from step 7) Select one of the port address pairs as the server network card target address, and modify the network address and port number of the destination network card in the address field of the data packet;

11), determine whether the selected server network card target address in step 10) is valid, if the network card is valid, then perform the next step, otherwise, perform step 13);

12), the client sends the remote procedure call protocol packet to the server, if sending fails, then the mark network card is invalid, and the time that the failure starts is set, then reselect the network card according to step 10), if sending successfully, then perform step 15);

13), the network card is in an invalid state, detect whether the invalid time interval threshold is reached from the failure time to the current time, if the invalid time interval threshold is reached, execute the next step, otherwise reselect the network card according to step 10);

14) Test whether the network is connected. If the network is connected, resend the data packet. After the transmission is successful, the effective bit of the network card is marked as valid. If the network is not connected, reset the fault time to the current time, and then follow step 10 ) to re-select the network card;

15), after the server receives the data packet from the network card, the socket structure created in step 4) passes the data packet to the network file system service thread for processing, and the server returns the processed result to the server through the network card that received the request. client.

In the above technical solution, in the step 1), the network addresses are configured in the same network segment.

In the above technical solution, in the step 2), ifconfig-a is used to obtain the network addresses, physical addresses and names of physical devices of all network cards on the network file system server.

In the above technical solution, in the step 3), the port number of the main network card is port 2049.

In the above technical solution, in the step 6), the "sk_bound_dev_if" field of all socket structures is set to "1", so that the network card is bound together with the specified socket structure of the network card .

In the above technical solution, in step 10), when the client selects one of the server network address and port address pairs obtained in step 7) as the target address of the server network card, a strategy of round robin selection is adopted.

In the above technical solution, the multi-network card refers to more than 2 network cards.

By adopting the method of the invention, each client can make full use of the network bandwidth and improve the network transmission efficiency.

Description of drawings

Fig. 1 is the composition figure of the network file system that the present invention adopts;

Fig. 2 is the operation of the method for balancing network file system server load among multiple network cards of the present invention at the client;

Fig. 3 is the operation on the server end of the method for balancing the load between multiple network cards of the network file system server of the present invention;

Fig. 4 is the flowchart of selecting target address to send data request in the method of the present invention;

FIG. 5 is a structural diagram of a network file system client and server.

Detailed ways

The method of the present invention will be specifically described below in conjunction with the accompanying drawings and specific embodiments.

As shown in Figure 1, it is a structural diagram of a common NFS system. The method of this embodiment utilizes the system, which includes a client, a switch and a server, wherein the switch does not need to support round-robin operation, and the server has multiple NIC.

Figure 5 is a further illustration of the client and server. The client includes: a system call layer, a virtual file system layer, a network file system client module, a request load distribution module, and a remote procedure call client module. The server side includes: a system call layer, a virtual file system layer, a network file system server-side module, a network channel concurrent module, and a remote procedure call server-side module.

When the method of the present invention is used to balance the load among multiple network cards of the network file system server, the server and the client need to be operated separately, and the operation process is described below respectively.

Operations on the server:

Step 1.1. Configure network addresses for all network cards on the NFS server. These addresses can be configured to belong to the same network segment or to belong to different network segments.

Step 1.2, use ifconfig-a to obtain the network addresses, physical addresses and names of physical devices of all network cards on the NFS server, and record the obtained information of each network card in a table. For example, the network address of the first network card on the NFS server is 159.211.10.101, its physical device name is eth0, and its physical address is 00:0C:76:9D:37:69. The above information is recorded in the network card information table middle.

Step 1.3. Select a master network card from the network cards of the NFS server, and other network cards in the server are slave network cards. When selecting the main network card, you can choose a network card as the main network card arbitrarily, and specify the port number of the main network card as port 2049.

Step 1.4, the server creates a corresponding socket structure for each network card according to the network card information table obtained in step 1.2, and initializes the socket structure, specifies a port number for the network card, and uses the network address and port obtained in step 1.2 number into the socket structure as described. Through the above operations, the number of data transmission channels between the network card and the NFS service thread is expanded, from the original sharing of one channel to the number of channels matching the number of network cards, which increases the concurrency of data transmission and improves data transmission performance.

Step 1.5, select the socket of the main network card as the main socket, and its service port number is 2049, and register the port number with the server port mapper, and the other sockets do not register the port number with the port mapper.

The function of the port mapper is that when the client does not know the service port number, it can query the server. According to the information registered in the port mapper, the server returns the port number of the service requested by the user.

In this step, multiple service ports (i.e. sockets) are specified for the NFS server, but only the public default port 2049 is registered with the portmapper. When installing (mount) the server-side NFS file system, in order to retain the original mount protocol, only the address of the server-side main network card is specified.

Step 1.6. At the same time, set the "sk_bound_dev_if" field of all sockets to "1", so that the socket structure is bound to the specified network card. Bind the socket with the specified network card, so that when the network device sends a data packet, it will not query the routing table to send data, avoiding the problem of always selecting the first network card to send for network cards in the same network segment.

After completing the operation on the server side, the server sends the network address of the main network card and the server port number to the client, and then completes the following operations in the client:

Step 2.1, the client sends to the server a request to obtain the server network card physical address information, network address information and port number information, and saves the result returned by the server in the table; the server network card described in this step includes the aforementioned main network card and from the network card.

Step 2.2, according to the obtained network card information and automatically set the client arp configuration file, fill the network addresses and physical addresses of multiple network cards of the server into the file of the arp protocol for reading statically resolved network addresses; then run arp to make the configuration take effect, In the following steps, the ethernet protocol can convert the target IP address into the physical address of the network card, and can transmit data on the media medium.

Step 2.3, the client uses the network address of the main network card of the server to install (mount) the NFS file system. After completing this step, the client can use the network file system.

Step 2.4, when the client communicates with the server through the switch, the client intercepts the remote procedure call protocol (RPC) packet, and the address field of the packet includes the destination network card port number and network address sent, by default, The destination network card and the network address are the addresses of the main network card in the server, and the port number of the network card is port number 2049. The client selects a target address from the server network address and port address pair obtained in step 2.1, and modifies the address and port number of the destination network card in the address field in the RPC data packet, so that the network layer can send the data packet to Specifies the server NIC. When selecting a target address from the server network address and port address pair, a round-robin selection strategy is adopted, and the dynamic selection of the network card can be realized according to the strategy.

Step 2.5, judging whether the target address of the server network card selected in the previous step is valid, if the network card flag is valid, then perform the next step, otherwise, perform step 2.7;

Step 2.6, the client sends a remote procedure call protocol (RPC) packet to the server. If the sending fails, the network card is marked as invalid, and the time when the fault starts is set, and then the network card is reselected according to step 2.4. If the sending is successful, then perform step 2.9

Step 2.7, the network card is in an invalid state, detect whether the invalid time interval threshold has been reached from the fault time to the current time, if the invalid time interval threshold is reached, perform the next step, otherwise, reselect the network card according to step 2.4;

Step 2.8. Test whether the network is connected. If the network is connected, resend the data packet. After the transmission is successful, the valid bit of the network card is marked as valid. If the network is not connected, reset the fault time to the current time, and then follow step 2.4 Reselect the network card;

Step 2.9: After the server receives the data packet from the network card, it passes the created socket to the network file system service thread (nfsd), and the server still sends the processed result to the client from the network card that received the request.

Fig. 4 is a flow example of a client initiating a server file access request. Taking the client write operation as an example, the client application calls the write operating system, and the virtual file system layer converts the operation into multiple network file system write operation requests, and the request load distribution module intercepts the request, selects the target address for it, and remote procedure call The client module sends the request to the specified server target address. The server sends the received request to the network file system server module for processing, writes the data back to the disk, and then sends the response information to the network channel concurrent module, which binds the network card device and the socket according to the initialization settings. The principle transmits the response packet to the remote procedure call server-side module, and sends it to the client from the network card that accepts the request packet corresponding to the response packet.

Claims (7)

1. A method for balancing the load between multiple network cards of a network file system server, applied to a network file system composed of a network file system server, a switch, and a client, wherein the method is operated on the server and the client respectively, comprising the following steps : 1), first configure network addresses for all network cards on the network file system server; 2), obtain the physical address of all network cards on the network file system server, the network address and the title of physical equipment, and record in the network card information table of the network file system server; 3), select a network card from all network cards on the network file system server as the main network card, and the remaining network cards are slave network cards, and specify the port number for the main network card; 4), the network file system server according to the network card information table that step 2) obtains, creates corresponding socket structure for each network card, and designates a port number for each from network card, initializes described socket structure, will The network address obtained in port number and step 2) is filled in the described socket structure; 5), select the socket structure of the main network card as the main socket, and register the port number of the main network card in the port mapper of the server; 6), bind each network card with the socket structure specified by the network card; 7), the client sends to the server a request to obtain the server network card physical address information, network address information and port number information, and saves the result returned by the server; 8), according to the information of the network card obtained in step 7), and the automatic setting client arp configuration file that comes with the client, fill the network address and physical address of all network cards of the server into the file of the arp protocol to read the static analysis network address , and then run the arp configuration file; 9), the client uses the network address of the main network card of the server to install the network file system; 10), the client communicates with the server through the switch, the client intercepts the remote procedure call protocol packet, the address field of the packet includes the port number and the network address of the destination network card, and the client obtains the server network address from step 7) Select one of the port address pairs as the server network card target address, and modify the network address and port number of the destination network card in the address field of the data packet; 11), determine whether the selected server network card target address in step 10) is valid, if the network card is valid, then perform the next step, otherwise, perform step 13); 12), the client sends the remote procedure call protocol packet to the server, if sending fails, then the mark network card is invalid, and the time that the failure starts is set, then reselect the network card according to step 10), if sending successfully, then perform step 15); 13), the network card is in an invalid state, detect whether the invalid time interval threshold is reached from the failure time to the current time, if the invalid time interval threshold is reached, execute the next step, otherwise reselect the network card according to step 10); 14) Test whether the network is connected. If the network is connected, resend the data packet. After the transmission is successful, the effective bit of the network card is marked as valid. If the network is not connected, reset the fault time to the current time, and then follow step 10 ) to re-select the network card; 15), after the server receives the data packet from the network card, the socket structure created in step 4) passes the data packet to the network file system service thread for processing, and the server returns the processed result to the server through the network card that received the request. client. 2. The method for balancing load among multiple network cards of a network file system server according to claim 1, characterized in that, in said step 1), said network addresses are configured in the same network segment. 3. The method for balancing the load between multiple network cards of a network file system server according to claim 1, characterized in that, in said step 2), use ifconfig-a to obtain the network addresses of all network cards on the network file system server, The physical address and the name of the physical device. 4. The method for balancing the load among multiple network cards of a network file system server according to claim 1, characterized in that, in the step 3), the port number of the main network card is port 2049. 5. The method for balancing the load among multiple network cards of a network file system server according to claim 1, characterized in that, in said step 6), the "sk_bound_dev_if" field of all socket structures is set to "1" , so that the network card is bound to the socket structure specified by the network card. 6. The method for balancing the load among multiple network cards of a network file system server according to claim 1, characterized in that, in said step 10), the server network address and port obtained by the client from step 7) When one of the address pairs is selected as the target address of the server network card, a round-robin selection strategy is adopted. 7. The method for balancing the load among multiple network cards of a network file system server according to claim 1, wherein said multiple network cards refer to more than two network cards.

Images