CN114968895A - Heterogeneous interconnection system and cluster - Google Patents

Abstract

The present application discloses a heterogeneous interconnection system and a cluster in the field of computer technology. In this application, each cabinet includes two switches and at least one server, each CPU in each server is connected to a CXL interface of at least one computing device through the CXL protocol; and each CPU in each server is interconnected through QPI/UPI. In a cabinet, one switch is connected to the network cards of each server in the current cabinet and connected to the public network, so that the current cabinet can be connected to the public network; the other switch is connected to the RDMA interface of each computing device in the current cabinet, so that each computing device can RDMA data is directly fetched. Each computing device connected to the same server is also connected to the data exchange device corresponding to the current server through its own cable interface, so that each computing device in a server can interact through the data exchange device connected to the cable interface, ensuring data transmission efficiency and improving system fault tolerance. A heterogeneous interconnection cluster provided by the present application also has the above technical effects.

Description

A heterogeneous interconnection system and cluster

technical field

The present application relates to the field of computer technology, and in particular, to a heterogeneous interconnection system and a cluster.

Background technique

At present, hardware devices are often used to improve the processing rate, such as: using GPU (Graphics Processing Unit, graphics processor), FPGA (Field-Programmable Gate Array, field programmable gate array) for machine learning and the like. There are various hardware devices currently available, and there is no unified connection standard. When multiple hardware devices are managed in a system, the connection between these hardware devices, any hardware device and the host will be more complicated. The connections are likely to affect the overall computational efficiency of the system.

Therefore, how to interconnect multiple hardware devices in a system is a problem to be solved by those skilled in the art.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present application is to provide a heterogeneous interconnection system and cluster to interconnect multiple hardware devices in one system. Its specific plan is as follows:

In a first aspect, the present application provides a heterogeneous interconnection system, including: at least one cabinet and a network access layer connected to a public network;

Wherein, each cabinet includes: a first switch, a second switch and at least one server; each server includes at least one CPU, each CPU is connected to a CXL interface of at least one computing device through the CXL protocol, and each computing device is also provided with RDMA Interface and Cable interface; different CPUs in the same server are interconnected through QPI/UPI;

the first switch is connected to the network access layer and the network card of each server in the current cabinet;

The second switch is connected to the RDMA interface of each computing device in the current cabinet;

Each computing device connected to the same server is connected to the data exchange device corresponding to the server through its own cable interface.

Optionally, each computing device connected to the same server is used to process different computing tasks.

Optionally, the computing device is a GPU, FPGA, DPU or AI chip.

Optionally, the CXL interface is implemented based on the PCIe standard.

Optionally, the network access layer includes at least one connection layer, each connection layer includes at least one connection device, and connection devices belonging to different connection layers are interconnected.

Optionally, the connection device is a switch or a router.

Optionally, the network access layer is a spine-leaf two-layer interconnection architecture.

Optionally, the network access layer of the spine-leaf two-layer interconnection architecture includes: a first connection layer connecting the public network, and a second connection layer connecting the first connection layer and the first switch in each cabinet;

Wherein, the connection devices in the first connection layer are less than the connection devices in the second connection layer, and the connection devices belonging to different connection layers are interconnected.

Optionally, each server has at least one network card.

In a second aspect, the present application provides a heterogeneous interconnection cluster, including: a plurality of heterogeneous interconnection systems described in any one of the above.

It can be seen from the above solutions that the present application provides a heterogeneous interconnection system, including: at least one cabinet and a network access layer connected to the public network; wherein each cabinet includes: a first switch, a second switch, and at least one server; Each server includes at least one CPU, and each CPU is connected to the CXL interface of at least one computing device through the CXL protocol, and each computing device is also provided with an RDMA interface and a Cable interface; different CPUs in the same server are interconnected through QPI/UPI; all The first switch is connected to the network access layer and the network card of each server in the current cabinet; the second switch is connected to the RDMA interface of each computing device in the current cabinet; each computing device connected to the same server is connected through its own Cable interface The data exchange device corresponding to the server.

It can be seen that in this application, each cabinet includes: a first switch, a second switch and at least one server, and each server includes at least one CPU, and each CPU is connected to the CXL interface of at least one computing device through the CXL protocol, and each The computing device is also provided with an RDMA interface and a Cable interface; different CPUs in the same server are interconnected through QPI/UPI; the first switch connects the network access layer and the network cards of each server in the current cabinet; the second switch connects each of the current cabinets. The RDMA interface of the computing device; each computing device connected to the same server is connected to the data exchange device corresponding to the server through its own Cable interface. It can be seen that different CPUs in each server are interconnected through QPI/UPI, which has high transmission efficiency; and each CPU is connected to the CXL interface of at least one computing device through the CXL protocol, and each computing device also has an RDMA interface and a Cable interface; the second switch is connected to the RDMA interface of each computing device in the current cabinet, so each computing device in a cabinet can directly fetch RDMA data, and the data transmission efficiency is higher; each computing device connected to the same server through its own Cable interface Connect the data exchange device corresponding to the server, so each computing device in a server can also interact through the data exchange device connected to the cable interface, which can ensure the data transmission efficiency and improve the system fault tolerance rate. In each cabinet, the first switch is connected to the network card of each server in the current cabinet, and at the same time, the first switch is also connected to the network access layer connected to the public network, so the first switch can enable each server and each computing device in the current cabinet Access the public network through the network access layer. According to the present application, more types of computing devices can be accommodated in one system, and the number of cabinets and servers can be easily expanded, and the solution has high scalability.

Correspondingly, a heterogeneous interconnection cluster provided by the present application also has the above technical effects.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without any creative effort.

1 is a schematic diagram of a heterogeneous interconnection system disclosed in the application;

2 is a schematic diagram of a topology structure in a server node disclosed in the present application;

3 is a schematic diagram of an interconnection topology inside a cabinet disclosed in the application;

FIG. 4 is a schematic diagram of an interconnection topology between cabinets disclosed in the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

At present, there are various hardware devices available, and there is no unified connection standard. When multiple hardware devices are managed in a system, the connection between these hardware devices, any hardware device and the host will be more complicated, and Complex connections are likely to affect the overall computational efficiency of the system. To this end, the present application provides a heterogeneous interconnection system and cluster, which can interconnect multiple hardware devices in one system, and easily expand the number of cabinets and servers.

Referring to FIG. 1 , an embodiment of the present application discloses a heterogeneous interconnection system, including: at least one cabinet and a network access layer connected to a public network; wherein each cabinet includes: a first switch, a second switch, and at least one A server; each server includes at least one CPU, each CPU is connected to the CXL interface of at least one computing device through the CXL protocol, and each computing device is also provided with an RDMA interface and a Cable interface; different CPUs in the same server communicate with each other through QPI/UPI. The first switch connects the network access layer and the network cards of each server in the current cabinet; the second switch connects the RDMA interface of each computing device in the current cabinet; the computing devices connected to the same server pass their own The cable interface is connected to the data exchange device corresponding to the server. Among them, each cabinet constitutes the data center shown in FIG. 1 .

It can be seen that each computing device in a cabinet can directly fetch RDMA data, and each computing device in a server can interact at high speed through the Cable interface. Each computing device connected to the same server is used to process different computing tasks. For example, some computing devices perform image recognition tasks, and some computing devices perform text classification tasks.

As shown in FIG. 1 , the data center includes N cabinets, and each cabinet includes N server nodes and two switches. Each server node includes: a server, N computing devices, and a data exchange device. N is a natural number.

In a specific implementation, different CPUs within the same server are interconnected via QPI/UPI. QPI (QuickPath Interconnect) is a fast path interconnection, which can realize direct interconnection between chips. UPI (Ultra PathInterconnect) is also a fast channel interconnection with higher communication rate and lower power consumption.

In a specific implementation manner, the computing device may be a GPU, an FPGA, a DPU (Data Processing Unit, processor distributed processing unit) or an AI (Artificial Intelligence, artificial intelligence) chip.

In a specific implementation manner, each CPU of each server is connected to a CXL interface of at least one computing device through a CXL (Compute Express Link) protocol, and each server has at least one network card. That is, each CPU of any server is connected with the CXL interface of any computing device through the CXL protocol. Since two ends connected using the CXL protocol can share IO, cache and memory, the CPU and computing devices connected using the CXL protocol can share IO, cache and memory. It can be seen that the computing device is provided with a CXL interface, and the CXL interface can be implemented based on the PCIe standard. From this, it can be determined that a computing device has a CXL interface, an RDMA interface and a Cable interface at the same time.

In a specific embodiment, the network access layer includes at least one connection layer, each connection layer includes at least one connection device, and the connection devices belonging to different connection layers are interconnected. The connecting device is a switch or a router.

In a specific implementation manner, the network access layer may be a spine-leaf two-layer interconnection architecture. The network access layer of the spine-leaf two-layer interconnection architecture includes: a first connection layer (ie, the spine layer in FIG. 4 ) connecting the public network, connecting the first connection layer and the first connection layer in each cabinet The second connection layer of the switch (ie, the Leaf layer in FIG. 4 ); wherein, the connection devices in the first connection layer are less than the connection devices in the second connection layer, and the connection devices belonging to different connection layers are interconnected.

In this embodiment, the data center includes at least one cabinet, and each cabinet includes: a first switch, a second switch, and at least one server, and different CPUs in each server are interconnected through QPI/UPI, which has a higher transmission rate. Efficiency; and each CPU is connected to the CXL interface of at least one computing device through the CXL protocol, and each computing device is also provided with an RDMA interface and a Cable interface; the second switch is connected to the RDMA interface of each computing device in the current cabinet, so in one cabinet Each computing device in the same server can directly fetch RDMA data, and the data transmission efficiency is higher; each computing device connected to the same server is connected to the data exchange device corresponding to the server through its own Cable interface, so each computing device in a server can also pass the Cable interface. The data exchange devices connected to the interface interact with each other, which can ensure the efficiency of data transmission and improve the fault tolerance rate of the system. In each cabinet, the first switch is connected to the network card of each server in the current cabinet, and at the same time, the first switch is also connected to the network access layer connected to the public network, so the first switch can enable each server and each computing device in the current cabinet Access the public network through the network access layer.

It can be seen that this embodiment can accommodate more types of computing devices in one system, and it is easier to expand the number of cabinets and servers, and the solution has high scalability.

Based on the core idea of the present application, the following embodiments provide an artificial intelligence computing-oriented interconnection system, which has a multi-level coupling hybrid topology structure, and is capable of interconnecting server CPUs, heterogeneous computing devices, and server nodes. , Inter-cabinet interconnection.

The topology within the server node is shown in Figure 2. In Figure 2, the various heterogeneous computing acceleration devices GPU, FPGA, DPU, AI chip, etc. are collectively referred to as XPU. As shown in Figure 2, XPU0~XPU7. An XPU contains three interfaces: CXL standard interface, high-bandwidth Cable interface, and custom RDMA interface. Among them, the XPU adopts PCIe (Peripheral Component Interconnect express, a high-speed serial computer expansion bus standard) that supports CXL to realize the CXL standard interface, and uses the CXL standard interface to interconnect with the server CPU, which can realize the connection between the XPU memory and the server host memory. Memory consistency. Generally, within a single server node, according to the definition of the CXL standard, each server CPU can mount 4 XPUs. In the server node, the QPI/UPI interface is used for high-speed interconnection between different CPUs in a server.

In Figure 2, CPU0 is connected to XPU0 to XPU3, and CPU1 is connected to XPU4 to XPU7. At the same time, XPU0 to XPU7 are uniformly connected to the XPU switching device. In addition, XPU0 to XPU7 are connected to the iRDMA Switch in a unified manner.

It should be noted that although CXL can provide standardized and efficient interconnection for XPU, the connection performance is limited by the PCIe physical bandwidth carrying the CXL standard, and cannot meet the high-bandwidth data transmission requirements between XPU devices in artificial intelligence computing scenarios. Therefore, within the server node, a cable interconnection interface standard for high-bandwidth transmission is also designed, and XPU switching equipment is used to enable different XPU devices in the server node to realize high-bandwidth data exchange.

At the same time, the iRDMA Switch (that is, the second switch) inside the cabinet summarizes the RDMA interfaces of each XPU device in the node to realize efficient data sharing of XPU devices between different nodes in the cabinet, and also enables each server in a cabinet to exchange RDMA data. . In order to improve the expansion efficiency of server nodes and other devices in the network, each server also has a general-purpose network card NIC device for data interaction between the server and other devices in the network. These NIC devices are connected to the Top of Rack Switch (ie first switch). Hereinafter, Top of Rack Switch will be referred to as TOR Switch for short.

Specifically, the interconnection topology inside the cabinet is shown in FIG. 3 . Two layers of switch devices are configured inside the cabinet: iRDMA Switch (ie, the second switch in FIG. 3 ) and TOR Switch (ie, the first switch in FIG. 3 ). iRDMASwitch is mainly responsible for RDMA data exchange between different XPU devices in the cabinet, and can provide low-latency, high-bandwidth data exchange for XPUs between nodes in the cabinet. TOR Switch is mainly responsible for data exchange between network card devices between different nodes in the cabinet, and at the same time, it is responsible for exchanging data inside the cabinet with the network.

Further, the interconnection topology between the cabinets is shown in FIG. 4 . Each cabinet is connected to the network access layer of the leaf-spine network Spine-Leaf two-layer interconnection architecture to ensure the scalability of the ultra-large-scale data center. Among them, the TORSwitch in each cabinet is interconnected with the Leaf switch in the spine-leaf architecture (ie, the leaf switch in Figure 4), so as to meet the large-scale scalability requirements between cabinets. Each Leaf switch is connected to each Spine switch (ie, the spine switch in Figure 4), forming a full-mesh topology. Specifically, the Leaf switch is an access switch, which forms a Leaf layer and is used to connect cabinets. The spine layer is the backbone of the network and is responsible for connecting all Leaf switches. Since every Leaf switch is connected to every Spine switch, if a Spine switch goes down, the throughput performance of the data center will only be slightly degraded. If a link is full, the expansion process is straightforward: adding a spine switch can expand the uplink of each leaf, increasing the bandwidth between the leaf and the spine. If the number of ports at the access layer becomes a bottleneck, simply add a new Leaf switch, connect it to each spine switch, and configure it accordingly. This easy-to-scale feature optimizes the network scaling process. When there are no bottlenecks in the access ports and uplinks of the Leaf layer, this architecture achieves non-blocking. In a Spine-Leaf architecture, any connection from one server to another goes through the same number of devices (unless the two servers are under the same Leaf switch), which ensures predictable latency because a packet only needs to be The destination can be reached through a spine switch and another leaf switch. The data exchange between the network of the data center and the core network is completed through Spine-Leaf.

It can be seen that this embodiment uses QPI/UPI interconnection between the CPU and the CPU, the CXL interconnection standard between the CPU and the XPU device, and the high-bandwidth Cable interconnection between the XPU and the XPU, and the XPU is used for switching. device for data interaction. In the rack rack, the customized RDMA protocol is used for interconnection and communication between XPUs, and the iRDMA Switch is used for data interaction. Further, the spine-leaf architecture is used to interconnect different cabinets. The TORSwitch in each cabinet is interconnected with all leaf switches to form a Full-Mesh interconnection network, so that the leaf switches are connected to the core network through the first layer of the spine switches. Interconnection to realize data interaction between different data centers.

Starting from the hybrid heterogeneous interconnection topology of the hyper-heterogeneous system, this embodiment proposes a multi-level hybrid heterogeneous interconnection topology, which can realize the interconnection between different heterogeneous acceleration devices XPUs within nodes, between nodes, and between cabinets. The efficient synergy between the devices improves the overall energy efficiency of the system, supports the connection of different heterogeneous computing acceleration devices, and meets the performance requirements for interconnection bandwidth and data exchange delay in different artificial intelligence computing scenarios.

The following describes a heterogeneous interconnection cluster provided by an embodiment of the present application. A heterogeneous interconnection cluster described below and a heterogeneous interconnection system described above may refer to each other.

The embodiment of the present application discloses a heterogeneous interconnection cluster, which includes: a plurality of heterogeneous interconnection systems described in any of the foregoing embodiments.

Specifically, a heterogeneous interconnection system includes: at least one cabinet and a network access layer connected to a public network; wherein each cabinet includes: a first switch, a second switch and at least one server; each server includes at least one CPU, each CPU is connected to the CXL interface of at least one computing device through the CXL protocol, and each computing device is also provided with an RDMA interface and a Cable interface; different CPUs in the same server are interconnected through QPI/UPI; the first switch connects all The network card of each server in the network access layer and the current cabinet; the second switch is connected to the RDMA interface of each computing device in the current cabinet; each computing device connected to the same server is connected to the data exchange corresponding to the server through its own Cable interface equipment. Among them, each cabinet constitutes the data center shown in FIG. 1 .

In a specific implementation, each computing device connected to the same server is used to process different computing tasks.

In a specific implementation, the computing device is a GPU, an FPGA, a DPU or an AI chip.

In a specific implementation, the CXL interface is implemented based on the PCIe standard.

In a specific embodiment, the network access layer includes at least one connection layer, each connection layer includes at least one connection device, and the connection devices belonging to different connection layers are interconnected.

In a specific implementation, the connecting device is a switch or a router.

In a specific implementation manner, the network access layer is a spine-leaf two-layer interconnection architecture.

In a specific embodiment, the network access layer of the spine-leaf two-layer interconnection architecture includes: a first connection layer connecting the public network, and a second connection layer connecting the first connection layer and the first switch in each cabinet; Wherein, the connection devices in the first connection layer are less than the connection devices in the second connection layer, and the connection devices belonging to different connection layers are interconnected.

It can be seen that, according to this embodiment, different data centers can form a heterogeneous interconnection cluster, and different data centers can perform data interaction.

References in this application to "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method or apparatus comprising a series of steps or elements is not necessarily limited to those steps or elements expressly listed , but may include other steps or elements not expressly listed or inherent to these processes, methods or apparatus.

It should be noted that the descriptions involving "first", "second", etc. in this application are only for the purpose of description, and should not be construed as indicating or implying their relative importance or implying the number of indicated technical features . Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In addition, the technical solutions between the various embodiments can be combined with each other, but must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of such technical solutions does not exist. , is not within the scope of protection claimed in this application.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments may be referred to each other.

The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other form of readable storage medium that is well known.

The principles and implementations of the present application are described herein by using specific examples. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application. There will be changes in the specific implementation and application scope. To sum up, the content of this specification should not be construed as a limitation on the application.

Claims (10)

1. A heterogeneous interconnection system, comprising: at least one cabinet and a network access layer connected to a public network; Wherein, each cabinet includes: a first switch, a second switch and at least one server; each server includes at least one CPU, each CPU is connected to a CXL interface of at least one computing device through the CXL protocol, and each computing device is also provided with RDMA Interface and Cable interface; different CPUs in the same server are interconnected through QPI/UPI; the first switch is connected to the network access layer and the network card of each server in the current cabinet; The second switch is connected to the RDMA interface of each computing device in the current cabinet; Each computing device connected to the same server is connected to the data exchange device corresponding to the server through its own cable interface. 2 . The heterogeneous interconnection system according to claim 1 , wherein each computing device connected to the same server is used to process different computing tasks. 3 . 3. The heterogeneous interconnection system according to claim 2, wherein the computing device is a GPU, an FPGA, a DPU or an AI chip. 4. The heterogeneous interconnection system according to claim 1, wherein the CXL interface is implemented based on the PCIe standard. 5. The heterogeneous interconnection system according to any one of claims 1 to 4, wherein the network access layer includes at least one connection layer, each connection layer includes at least one connection device, and the network access layer belongs to different connection layers. Connect the device interconnect. 6. The heterogeneous interconnection system according to claim 5, wherein the connection device is a switch or a router. 7. The heterogeneous interconnection system according to any one of claims 1 to 4, wherein the network access layer is a spine-leaf two-layer interconnection architecture. 8 . The heterogeneous interconnection system according to claim 7 , wherein the network access layer of the spine-leaf two-layer interconnection architecture comprises: a first connection layer connected to a public network, a first connection layer connected to the first connection layer and the second connection layer of the first switch in each cabinet; Wherein, the connection devices in the first connection layer are less than the connection devices in the second connection layer, and the connection devices belonging to different connection layers are interconnected. 9. The heterogeneous interconnection system according to claim 5, wherein each server has at least one network card. 10. A heterogeneous interconnection cluster, comprising: a plurality of heterogeneous interconnection systems according to any one of claims 1 to 9.

Images