Cross-host communication method and system of Docker container
Technical Field
The present application relates to the field of communications technologies, and in particular, to a cross-host communication method and system for a Docker container.
Background
The Docker container is an open-source application container engine, so that developers can package their applications and dependence packages into a portable container, and then distribute the container to any popular Linux machine, and can also realize virtualization.
In the related art, the Docker container is originally provided with three network models, namely bridge, host and none. The bridge model and the none model cannot realize cross-host communication of the Docker containers, and the host model can realize cross-host communication but cannot simultaneously perform network isolation between the Docker containers on the same host.
Disclosure of Invention
In view of the above, the present application provides a cross-host communication method and system for a Docker container.
Specifically, the method is realized through the following technical scheme:
a method of cross-host communication for a Docker container, the method comprising:
controlling the Docker to create an OVS bridge after starting;
the control Docker acquires network configuration information of the application Docker after detecting that the application Docker on the host computer is started;
the control Docker sends the network configuration information to a distributed coordination service for storage, and the distributed coordination service is used for maintaining the network configuration information of the application Docker on each host computer in the networking;
when monitoring that the network configuration information of the application Docker in the distributed coordination service is updated, the control Docker generates a forwarding flow table according to the updated network configuration information;
the control Docker issues the forwarding flow table to the OVS network bridge;
and the OVS bridge guides the cross-host communication between application Dockers according to the forwarding flow table.
A cross-host communication system of Docker containers, the system comprising: control Docker, OVS bridge, and application Docker, wherein,
after the control Docker is started, the OVS bridge is established;
the control Docker acquires network configuration information of the application Docker after detecting that the application Docker on the host computer is started;
the control Docker sends the network configuration information to a distributed coordination service for storage, and the distributed coordination service is used for maintaining the network configuration information of the application Docker on each host computer in the networking;
when monitoring that the network configuration information of the application Docker in the distributed coordination service is updated, the control Docker generates a forwarding flow table according to the updated network configuration information;
the control Docker issues the forwarding flow table to the OVS network bridge;
and the OVS bridge guides the cross-host communication between application Dockers according to the forwarding flow table.
As can be seen from the above description, the network configuration information of the application Docker in the networking can be maintained through the distributed coordination service, and the control Docker can generate the forwarding flow table for the started application Docker in time through monitoring the distributed coordination service and issue the forwarding flow table to the OVS bridge to guide forwarding, so that cross-host communication between the application dockers is realized. Meanwhile, the network configuration information of the application Docker is maintained by adopting the distributed coordination service, and compared with the Gossip protocol, the method is simpler and more reliable to implement.
Drawings
Fig. 1 is a schematic diagram of a Docker Overlay network architecture in the related art.
Fig. 2 is a flowchart illustrating a cross-host communication method for a Docker container according to an exemplary embodiment of the present application.
FIG. 3 is a block diagram illustrating a networking architecture for cross-host communication for a Docker container, according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
In the related art, cross-host communication of Docker can be realized through a Docker overlay network. Referring to the networking architecture shown in fig. 1, when an Overlay network is created, a network namespace may be created on a host, then a br0 bridge may be created in the network namespace, a virtual network device of Vxlan (virtual extensible LAN) is created to connect to the br0 bridge, and a virtual network card eth0 inside the Docker container may also connect to the br0 bridge.
The host may manage the cluster relationship through Gossip protocol, when a host is started, other hosts may receive a new node join event, and through the relevant information of this event, may update the fdb (forwarding database) entry of the local br0 bridge. Referring to fig. 1 again, if the Docker container 1 on the host a accesses the Docker container 3 on the host B for the first time, because the Docker container 1 does not have the MAC address of the Docker container 3, the Linux kernel sends an L3miss, the kernel converts the event to an event notification to the user state, and then converts the event notification to an inquiry request, and broadcasts the inquiry request to other host hosts in the Overlay network. The Docker container 3 on the host B will acknowledge its MAC address. Based on this process, the Docker container 1 may acquire the MAC address of the Docker container 3. Next, in conjunction with the br0FDB entry, Docker container 1 and Docker container 3 may communicate via Vxlan 1.
However, the implementation of the Overlay network described above has the following problems:
1) the Overlay network of the Docker container requires that the Linux kernel version is at least above 3.8, but the current popular low-version (2.6.32) kernel in the market cannot support the Linux kernel version, so that the application range of the Overlay network is small.
2) Nodes of the Docker container Overlay network discover to pass the Gossip protocol. The Gossip protocol is a dynamic discovery protocol, when a new node is added, information synchronization with the existing node needs to be completed, and an FDB table entry of Vxlan equipment needs to be updated, so that the reliability is poor.
3) The Docker container Overlay network performs MAC Address addressing in a manner of broadcasting an ARP (Address Resolution Protocol) request message, where the ARP broadcast message is transmitted in a cluster and consumes a certain network bandwidth. In addition, the Docker container is required to have a unique MAC address within the network, increasing management complexity.
4) Two virtual network cards are arranged in a Docker container Overlay network, one virtual network card is used for cross-host communication among the Docker containers, and the other virtual network card is used for mutual access with an external network, so that the complexity of application processing is increased.
5) The Docker container Overlay network does not support the architecture of multiple subnets, cannot plan the subnets, and has poor flexibility.
6) The Docker container Overlay network adopts native Vxlan virtual equipment of Linux, and the encapsulation of Vxlan protocols requires the participation of a CPU of a host computer, so that the performance loss of the host computer is increased.
In order to solve the above problems, the present application provides a cross-host communication scheme for a Docker container.
Fig. 2 is a flowchart illustrating a cross-host communication method for a Docker container according to an exemplary embodiment of the present application.
Referring to fig. 2, the cross-host communication method of the Docker container may include the following steps:
step 201, after the Docker is started, the OVS bridge is created.
In this embodiment, a control Docker may be configured in the host, and is used for issuing a service Docker forwarding flow table, where the control Docker may be implemented by a CPU of the host. Specifically, the control Docker may create an ovs (open vswitch) bridge on the host after booting. In one example, a control Docker may create the OVS bridge based on a CPU. In another example, when the host is configured with an OVS-supporting smart network card, an OVS bridge may also be created in the smart network card to save the CPU overhead.
In this embodiment, the low-version Linux kernel may also support the OVS technology, and the problem of a small application range of the Overlay network may be effectively solved by using the OVS technology to implement virtual interaction.
Step 202, controlling a Docker to acquire network configuration information of the application Docker after detecting that the application Docker on the host is started.
In this embodiment, after the control Docker is started, the start of the application Docker on the host computer may be detected through a plug-in mechanism between dockers. After detecting that the application Docker is started, network configuration information of the application Docker may be obtained. Wherein the network configuration information may include: the ID, IP address, MAC address, OVS port, IP address of the host, etc. of the application Docker.
And 203, controlling the Docker to send the network configuration information to a distributed coordination service for storage, wherein the distributed coordination service is used for maintaining the network configuration information of the application Docker on each host in the networking.
In this embodiment, after acquiring the network configuration information of the application Docker, the control Docker may send the network configuration information to the distributed coordination service for storage. The distributed coordination service is usually an external service, and can be used for maintaining network configuration information of application dockers on host computers in networking. The distributed coordination service may be zookeeper or the like, which is not particularly limited in this application.
In this embodiment, if cross-host communication between dockers is implemented based on a Vxlan protocol, the network configuration information may further include a Vxlan ID to which the application Docker belongs. Correspondingly, the distributed coordination service may implement isolated storage of application Docker network configuration information based on Vxlan ID, such as: in the distributed coordination service, a storage directory of network configuration information can be established by taking Vxlan ID as a name, and the control Docker can store the network configuration information of the application Docker in the directory of the Vxlan ID so as to facilitate management of the network configuration information of the application Docker.
And 204, controlling the Docker to generate a forwarding flow table according to the updated network configuration information when monitoring that the network configuration information of the application Docker in the distributed coordination service is updated.
In this embodiment, the control Docker may start a monitoring thread to monitor the update of the application Docker network configuration information in the distributed coordination service, where the update includes addition, modification, and the like. Such as: if the network configuration information of the application Docker is newly added in the distributed coordination service is monitored, the control Docker can generate a forwarding flow table according to the newly added network configuration information. If it is monitored that the network configuration information of the application Docker stored in the distributed coordination service is modified, the control Docker may also generate a forwarding flow table according to the modified network configuration information.
In this embodiment, the forwarding flow table includes network configuration information corresponding to the application Docker, such as: IP address, MAC address, IP address of the host, etc.
And step 205, controlling the Docker to issue the forwarding flow table to the OVS bridge.
And step 206, the OVS bridge guides the cross-host communication between the application Dockers according to the forwarding flow table.
Based on the foregoing step 204, after the forwarding flow table is generated by the control Docker, the forwarding flow table may be issued to the OVS bridge, and a subsequent OVS bridge may direct the cross-host communication between application dockers according to the issued forwarding flow table, for example: and encapsulating and decapsulating the message and the like.
As can be seen from the above description, the network configuration information of the application Docker in the networking can be maintained through the distributed coordination service, and the control Docker can generate the forwarding flow table for the started application Docker in time through monitoring the distributed coordination service and issue the forwarding flow table to the OVS bridge to guide forwarding, so that cross-host communication between the application dockers is realized. Meanwhile, the network configuration information of the application Docker is maintained by adopting the distributed coordination service, and compared with the Gossip protocol, the method is simpler and more reliable to implement.
The technical scheme of the present application is described in detail below by three aspects of MAC address acquisition, service packet forwarding, and application of Docker external communication, respectively.
First, MAC address acquisition of application Docker
In this embodiment, before the application Docker communicates with another application Docker, the application Docker may obtain the MAC address of the other application Docker through the ARP request packet. After receiving the ARP request message, the OVS bridge may construct an ARP reply message based on a preset MAC address, and send the ARP reply message to an application Docker.
Referring to the networking architecture shown in fig. 3, assuming that an application Docker1 on a host a sends an ARP request message 1 to request an MAC address of an application Docker3 on a host B, after receiving the ARP request message 1, an OVS bridge 1 may determine whether the ARP request message 1 is a cross-network segment ARP request message or a same-network segment ARP request message according to a destination IP address (i.e., an IP address of application Docker 3) of the ARP request message 1. Specifically, if the destination IP address of the ARP request message 1 and the source IP address (i.e., the IP address of the application Docker 1) belong to the same network segment, it may be determined that the ARP request message 1 is the ARP request message of the same network segment, and the OVS bridge 1 may further construct an ARP reply message 1 according to the preset pseudo MAC address, and send the ARP reply message 1 to the Docker 1. The pseudo MAC address may be set by an administrator, which is not particularly limited in this application.
With reference to fig. 3, it is assumed that the application Docker1 on the host a sends an ARP request message 2 to request an MAC address of the application Docker4 on the host B, and after receiving the ARP request message 2, the OVS bridge 1 may determine whether the ARP request message 2 is a cross-network segment ARP request message or a same-network segment ARP request message according to a destination IP address (i.e., an IP address of the application Docker 4) of the ARP request message 2. Specifically, if the destination IP address and the source IP address of the ARP request packet 2 (i.e., the IP address of the application Docker 1) belong to different network segments, it may be determined that the ARP request packet 2 is a cross-segment ARP request packet, and the OVS bridge 1 may further construct an ARP reply packet 2 according to the MAC address of the container gateway, and send the ARP reply packet 2 to the Docker 1.
Based on the above description, in this example, for the application Docker, the MAC addresses of other application dockers in the same network segment are all fixed pseudo MAC addresses, and the MAC addresses of application dockers in different network segments are all MAC addresses of the container gateway, and the acquisition of the application Docker MAC addresses is realized through an ARP proxy mechanism, so that the ARP request packet can be prevented from being transmitted across hosts in a cluster, and a large amount of network bandwidth is saved.
Second, service message forwarding
In this embodiment, after receiving a service packet sent by an application Docker, the OVS bridge may query, in a forwarding flow table, host address information where a target application Docker is located according to a destination IP address of the service packet. Then, the OVS bridge may encapsulate the service packet according to the queried host address information, and then send the encapsulated service packet to the host where the target application Docker is located through the tunnel. And the target application Docker is an application Docker to which the service message is sent.
With continued reference to fig. 3, assuming that the OVS bridge 1 receives the service packet sent by the application Docker1, the OVS bridge 1 may determine the target application Docker according to the destination IP address of the service packet. Assuming that the destination IP address of the service packet is the IP address of the application Docker3, it may be determined that the target application Docker is Docker 3. OVS bridge 1 may query the forwarding flow table for network configuration information of application Docker3, such as: by applying the information of the IP address, Vxlan port, and the like of the host B where the Docker3 is located, based on the queried network configuration information of the host B, the OVS bridge 1 may perform Vxlan tunnel encapsulation on the service packet, and send the encapsulated service packet to the OVS bridge 2 of the host B through the Vxlan tunnel.
It should be noted that, the present application is described by taking the Vxlan protocol as an example, and in practical application, the present application is not limited to the Vxlan protocol.
In this embodiment, after receiving a tunnel encapsulation service packet from another host, the OVS bridge may decapsulate the tunnel encapsulation service packet to obtain a decapsulated service packet. Based on the destination IP address of the service packet, the OVS bridge may query the forwarding flow table for the real MAC address of the target application Docker. The OVS bridge can modify the destination MAC address of the service message into the real MAC address, modify the source MAC address of the service message into a pseudo MAC address when determining that the service message is a service message in the same network segment, and modify the source MAC address of the service message into the MAC address of the container gateway when determining that the service message is a cross-network segment service message. After the modification of the MAC address is completed, the OVS bridge may send the modified service packet to the target application Docker.
Continuing with fig. 3, and still taking the above example as an example, after receiving the encapsulated service packet through the Vxlan tunnel, OVS bridge 2 may perform decapsulation to restore the service packet. The OVS bridge 2 may query the forwarding flow table according to the destination IP address of the service packet (the IP address of the application Docker 3), so as to query the real MAC address of the application Docker3, and modify the destination MAC address of the service packet into the real MAC address of the application Docker 3. For the source MAC address of the service packet, before modification, the OVS bridge 2 needs to determine the subnet, and if the service packet is a service packet in the same network segment (the IP addresses of the application Docker1 and the application Docker3 belong to the same network segment), the source MAC address of the service packet may be modified to a pseudo MAC address. If the service packet is a cross-network segment service packet (the IP addresses of the application Docker1 and the application Docker3 belong to different network segments), the source MAC address of the service packet may be modified to the MAC address of the container gateway. The OVS bridge 2 may send the service packet with the modified MAC address to the application Docker3, so as to implement cross-host communication between the application Docker1 and the application Docker 3.
From the above description, it can be seen that the cross-host communication scheme of the Docker container provided by the application can support the planning of the sub-network, and improves the flexibility of the service.
It should be noted that the encapsulation and forwarding of the service packet may be implemented by a virtual device established on the OVS bridge. Specifically, after the OVS bridge is created by the control Docker, Vxlan equipment and Veth equipment may also be created on the OVS bridge, where the Vxlan equipment may be used for encapsulation and decapsulation of a service packet, and the Veth equipment may be used for implementing a container gateway function, and the like. The processing and implementation of this part can refer to the related art, and the detailed description of this application is omitted here.
Third, apply Docker external communication
In this embodiment, a forwarding flow entry with the lowest priority may be configured in advance, where the forwarding flow entry may be: the destination MAC address is the address of the container gateway MAC forwarded to the virtual host gateway. Specifically, after receiving a service packet sent by an application Docker, if it is determined that a destination MAC address of the service packet is an MAC address of a container gateway and matches the forwarding flow table entry with the lowest priority, the OVS bridge may determine that the service packet is a service packet that needs to be sent to an external network, and send the service packet to a virtual host gateway, where the virtual host gateway sends the service packet to an external network in combination with a snat or a dnat of an iptable.
Specifically, the control Docker may configure the address of the container gateway into the virtual host gateway after creating the OVS bridge. Subsequently, if the OVS bridge receives the service packet matching the forwarding flow entry with the lowest priority, it may be determined that the service packet is a service packet sent to an external network, and then the service packet may be sent to the virtual host gateway through the container gateway and sent to the external network by the virtual host gateway. On the contrary, if the OVS bridge receives the service packet from the virtual host gateway, the source MAC address of the service packet may be modified into the MAC address of the container gateway, and then the modified service packet is sent to the corresponding application Docker.
As can be seen from the above description, the application can simultaneously implement cross-host communication between application dockers and communication between the application dockers and an external network through the container gateway, without configuring two virtual network cards, and the processing is simpler for the application.
Corresponding to the foregoing embodiments of the cross-host communication method of the Docker container, the present application also provides embodiments of a cross-host communication system of the Docker container. The cross-host communication system of the Docker container may include: control Docker, OVS bridge, and application Docker.
Wherein the control Docker creates the OVS bridge after starting;
the control Docker acquires network configuration information of the application Docker after detecting that the application Docker on the host computer is started;
the control Docker sends the network configuration information to a distributed coordination service for storage, and the distributed coordination service is used for maintaining the network configuration information of the application Docker on each host computer in the networking;
when monitoring that the network configuration information of the application Docker in the distributed coordination service is updated, the control Docker generates a forwarding flow table according to the updated network configuration information;
the control Docker issues the forwarding flow table to the OVS network bridge;
and the OVS bridge guides the cross-host communication between application Dockers according to the forwarding flow table.
Optionally, the OVS bridge receives an ARP request message sent by an application Docker;
if the OVS network bridge determines that the ARP request message is a cross-network-segment ARP request message, constructing an ARP response message based on the MAC address of the container gateway, and returning the ARP response message to the application Docker;
and if the OVS network bridge determines that the ARP request message is the same-network-segment ARP request message, constructing an ARP response message based on a preset pseudo MAC address, and returning the ARP response message to the application Docker.
Optionally, after receiving a service packet sent by an application Docker on a host, the OVS bridge queries, in the forwarding flow table, address information of the host where a target application Docker is located according to a destination IP address of the service packet;
and the OVS network bridge encapsulates the service message according to the host address information and sends the encapsulated service message to the host where the target application Docker is located through a tunnel.
Optionally, after receiving the tunnel encapsulation service packet from another host, the OVS bridge decapsulates the tunnel encapsulation service packet;
the OVS network bridge inquires a real MAC address of the target application Docker in the forwarding flow table according to the destination IP address of the decapsulated service message;
the OVS bridge modifies a target MAC address of the decapsulated service message into the real MAC address, modifies a source MAC address of the service message into the pseudo MAC address when determining that the decapsulated service message is a same-network-segment service message, and modifies the source MAC address of the service message into an MAC address of the container gateway when determining that the decapsulated service message is a cross-network-segment service message;
and the OVS network bridge sends the modified service message to the target application Docker.
Optionally, after receiving a service packet sent by an application Docker on a host, the OVS bridge sends the service packet to a virtual host gateway if it is determined that the service packet matches a forwarding flow entry with the lowest priority, and the virtual host gateway sends the service packet to an external network;
and the forwarding flow table entry with the lowest priority sends the service message with the destination MAC address as the MAC address of the container gateway to the virtual host gateway.
Optionally, the control Docker creates an OVS bridge in the smart network card.
Optionally, Vxlan equipment and Veth equipment are also created on the OVS bridge;
the Vxlan equipment is used for encapsulating and de-encapsulating the service message, and the Veth equipment is used for realizing the function of the container gateway.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.