CN114024910B - Extremely low-delay reliable communication system and method for financial transaction system - Google Patents
Extremely low-delay reliable communication system and method for financial transaction system Download PDFInfo
- Publication number
- CN114024910B CN114024910B CN202111276461.8A CN202111276461A CN114024910B CN 114024910 B CN114024910 B CN 114024910B CN 202111276461 A CN202111276461 A CN 202111276461A CN 114024910 B CN114024910 B CN 114024910B
- Authority
- CN
- China
- Prior art keywords
- delay
- reliable
- network card
- message
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2425—Traffic characterised by specific attributes, e.g. priority or QoS for supporting services specification, e.g. SLA
- H04L47/2433—Allocation of priorities to traffic types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2491—Mapping quality of service [QoS] requirements between different networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/901—Buffering arrangements using storage descriptor, e.g. read or write pointers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses an extremely low-delay reliable communication system and method for a financial transaction system. The low-delay switch is used for forwarding transaction data packets, and the delay is hundreds of ns; the low-delay network card is used for packaging and unpacking data, managing a link, and encoding and decoding; the FPGA is used for carrying out hardware acceleration processing on the reliable communication message protocol; the server runs a high-performance communication component for reliable communication protocol software processing, enabling one-to-many or many-to-many scene publish subscriptions. The invention relates to the technical field of financial transactions, which is applied to the issuing of quotations of a transaction center system, a quotation transaction gateway of the transaction center and a transaction counter. The technology and the method of the invention adopt a large-scale programmable FPGA, the software and the hardware are accelerated cooperatively, the flexible customization can be realized according to the service characteristics, and the capacity is large and the processing speed is high; the method can effectively cope with high concurrency scenes, and has short system response time.
Description
Technical Field
The invention relates to the technical field of financial transactions, in particular to an extremely low-delay reliable communication system and method for a financial transaction system.
Background
The transaction processing performance is an important index for reflecting the core competitiveness of the electronic transaction platform, the communication delay is one of the core factors for influencing the overall transaction performance, and the current global leading electronic transaction platform has reduced the communication delay to below 50 microseconds.
The conventional transaction platform is composed of a quotation system and a quotation gateway, wherein the communication component is used for the communication of the central information system and the data transmission component, as shown in fig. 1: after passing through the exchanger, the quotation data are received by the low-delay network card, are transmitted to the quotation gateway for performing FIX and SBE coding and decoding conversion after being processed by the communication component based on reliable transmission of connection, and are sent to a transaction user terminal through the communication component, one quotation system corresponds to a plurality of quotation gateways, and one quotation gateway can be connected to a plurality of transaction users, namely, 1-to-1 and 1-to-more scenes exist.
One existing communication method is a TCP mode, a client/server connection method is adopted, a plurality of TCPs are used for connecting a quotation system and a quotation gateway, and a plurality of TCPs are used for connecting a transaction user and the quotation gateway. The other communication mode is UDP mode, peer/peer connection mode, and multicast IP address is combined, and multicast address is adopted between the quotation system and quotation gateway and between the trading users.
Both of these two treatment modes have its disadvantages:
In scheme 1, the TCP connection state machine has higher performance requirements on multiple servers in queue processing, larger transmission delay and incapability of adapting to high throughput and high concurrency scenes.
In the scheme 2, since the UDP message is not connected and reliable, retransmission is not carried out after the UDP message is lost, all service subscriptions under the multicast address are consistent, and the subscriptions and consumption cannot be carried out according to the needs.
Disclosure of Invention
(One) solving the technical problems
Aiming at the defects of the prior art, the invention provides an extremely low-delay reliable communication system and method for a financial transaction system, which are used for carrying out hardware acceleration processing on transaction data interacted with a network in an FPGA (field programmable gate array) by adopting the FPGA and the low-delay network card so as to carry out reliable communication message identification, current limiting, stream classification and multi-queue distribution. The high-performance communication assembly performs reliable UDP transport layer protocol, pub/Sub communication mode processing, and realizes zero copy of the whole process in a shared memory mode. The software and hardware cooperation can fully use low delay to improve processing performance, and the problems of high delay, low performance and large jitter instability of the traditional transaction system are solved. The invention is used for extremely low-delay communication of a financial transaction system, realizes a high-performance and high-concurrency reliable communication mode, and reconstructs a data transmission flow. To support the need for extremely low latency in transaction processing in future high throughput, high concurrency scenarios.
(II) technical scheme
In order to achieve the above purpose, the invention is realized by the following technical scheme: an extremely low-delay reliable communication system for a financial transaction system comprises a low-delay switch, a low-delay network card, an FPGA and a server;
the low-delay switch is used for forwarding transaction data packets, and the delay is hundreds of ns;
the low-delay network card is used for data encapsulation and decapsulation, link management, encoding and decoding;
The FPGA is used for carrying out hardware acceleration processing on the reliable communication message protocol, distributing a plurality of queues and carrying out CPU hash calculation;
the server is used for providing reliable data transmission service through a high-performance transmission protocol stack and a communication mode, and is used for processing after application software transaction in a daemon and shared library cooperation mode.
Preferably, the low-latency network card is an intelligent network card, and the intelligent network card is used for flexibly unloading processing tasks unsuitable for a CPU, meeting the processing requirements of a data plane network and being compatible with the ecology of the existing network protocol.
Preferably, the existing network protocol is one of Ethernet, RDMA or InfiBand.
The invention also discloses a method for extremely low-delay reliable communication of the financial transaction system, which comprises the following steps:
S1, after receiving market data, the low-delay network card performs IP/UDP message flow limiting, classification and decoding through an FPGA, and then performs multi-queue hash distribution to a server;
s2, the data message is directly sent to a user state network protocol stack of the high-performance component for UDP processing through a network card driver in the high-performance component by receiving kernel bypass/RDMA;
S3, after the user state protocol stack is processed, a control message and a data message are separated through reliable transmission layer protocol processing, the control message enters session processing, and the data message enters queue cache;
S4, searching the cached message through a Pub/Sub publishing and subscribing mode to obtain all subscribers, and updating the write pointer of the shared buffer area of the subscribers to inform the subscribers that the message arrives;
S5, the application software compares the read-write pointers of the messages to obtain the messages in a mode of inquiring the shared buffer area, and the messages are free from copying in the whole process;
s6, the high-performance communication component provides a configurable parameter interface and performance and monitoring functions, and configures the FPGA to accelerate multi-queues and routing.
Preferably, the high-performance component in step S2 includes a network card driver, a user mode protocol stack, a reliable UDP communication and Pub/Sub publish-subscribe component, and the high-performance component operates in a Linux system and operates in a dynamic library so form, and together with upper software, the high-performance component operates independently in a user process.
Preferably, the high-performance component adopts a layered architecture, provides an API for upper-layer software to use, and simultaneously provides a configurable parameter interface and performance and monitoring functions in the component, and a system management process is used for configuring network card multi-queue receiving and routing.
Preferably, the idea of hierarchical design of the high-performance component is to treat the service by adopting the idea of componentization/service, abstract and layer each module of the whole system, and mutually decouple or loosely couple each layer.
Preferably, the reliable transport layer protocol in the step S3 is used for one-to-many or many-to-many scene publishing and subscribing, high-scalability multi-core expansion (gateway, single machine), flexible adaptation dpdk/rdma/bypass intelligent network card, capable of configuring queue parameters and dynamically loading flow control algorithm, and the reliable transport layer protocol is a self-defined reliable UDP transport protocol, which supports flow control algorithm and congestion control algorithm, and capable of self-defining modification.
(III) beneficial effects
The invention provides an extremely low-delay reliable communication system and method for a financial transaction system. Compared with the prior art, the method has the following beneficial effects:
(1) The extremely low-delay reliable communication system and the extremely low-delay reliable communication method for the financial transaction system can be flexibly customized according to service characteristics by adopting a large-scale programmable FPGA and the cooperative acceleration of software and hardware, and have large capacity, high processing speed and flexible protocol; the FPGA processing at the communication transmission end can effectively cope with high concurrency scenes; the system response time is short, the throughput is typically 40B/1000w, the delay is less than 2.4us, and the number of concurrent users is 10K/s, so that the system has high concurrency effect; the FPGA hardware protocol is accelerated, the flow is balanced, the protocol is processed, the priority is mapped, and the high-availability QOS scene is effectively treated.
(2) The very low-delay reliable communication system and the method for the financial transaction system fully utilize NUMA characteristics of a server CPU by adopting distributed multi-process deployment, and can be horizontally expanded according to the specification of a service; in addition, a json configuration file is adopted to modify NUMA binding cores to a system network card, a CPU and a memory, so that the system time delay is effectively reduced, and high expandability is realized; and (3) distributed software processing, namely multi-core CPU resources are fully utilized, and multi-thread lock-free processing is performed.
(3) The extremely low-delay reliable communication system and method for the financial transaction system adopt the technology of an intelligent network card in the message receiving and sending direction, for example, the RDMA network card can unload the whole transmission processing to network card hardware to release a CPU, network card receiving interrupt irqbalance, multiple queues and RSS hash. The technology of penetrating the Linux Kernel comprises DPDK and Kernel bypass, and the system can be flexibly adapted. To support intelligent network cards such as Mellanox, exanic, solarflare for various scenarios. And the intelligent network card RDMA, kernel bypass and DPDK user state straight-through technologies are flexibly adapted.
(4) The extremely low-delay reliable communication system and the extremely low-delay reliable communication method for the financial transaction system are characterized by unidirectional release aiming at a financial quotation system and multiple reception. The system supports the mode that the multicast IP address bears UDP messages, receives quotations at the equipment added into the multicast group, and does not need to copy respectively. Aiming at a unicast scene, the system realizes the many-to-many transmission through subscribing the publishing mode. The software does not copy the message in multiple ways, and the FPGA copies UDP messages Wen Duolu; the unicast multicast scene is a one-to-many, many-to-many scene realized based on the Pub/Sub communication model, and can be flexibly expanded.
(5) The system can be flexibly deployed on gateway equipment with high performance and high concurrency, can be deployed on a server counter with small specification, and can be deployed on a client of a single communication node. Modifying daemon kernel binding and memory specification through json file; the business process interacts with the daemon process through the communication library to support business applications with different specifications; by adopting the interaction mechanism of daemon process and user instance process, large-scale business application is supported, and flexible deployment is realized.
(5) The system and the method for extremely low-delay reliable communication of the financial transaction system adopt a copy-free idea, and a business process and a daemon share a transmitting and receiving buffer area of a large page memory. The sending direction also sends the service process to the FPGA to send DMA, zero copy is realized, and message copying and multipath distribution are completed by the FPGA; in the receiving direction, after receiving DMA (direct memory access) to a buffer area, each subscribed business process reads a block of buffer area, a plurality of subscribers do not need to copy, and the whole process is zero-copy; the processing time delay is effectively reduced by directly carrying out offload to FPGA processing; through the sending/receiving buffer area based on the large page shared memory, offload is directly used to hardware, and copying is avoided.
Drawings
FIG. 1 is a schematic diagram of a conventional market transaction data application scenario in an embodiment of the present invention;
FIG. 2 is a diagram of the interaction relationship between high performance communication components in an embodiment of the present invention;
FIG. 3 is a diagram of a layered architecture of a communication assembly in an embodiment of the present invention;
FIG. 4 is a flow chart of data processing in an embodiment of the invention;
FIG. 5 is a diagram of a data processing entity object lookup in an embodiment of the present invention;
FIG. 6 is a diagram of the acceleration processing of FPGA hardware in an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1-6, the embodiment of the present invention provides a technical solution: an extremely low-delay reliable communication system for a financial transaction system comprises a low-delay switch, a low-delay network card, an FPGA and a server;
the low-delay switch is used for forwarding transaction data packets, and the delay is hundreds of ns;
the low-delay network card is used for data encapsulation and decapsulation, link management, encoding and decoding;
the FPGA is used for carrying out hardware acceleration processing on a reliable communication message protocol, distributing a plurality of queues and carrying out hash calculation;
the server is used for providing reliable data transmission service through a high-performance transmission protocol stack and a communication mode, and is used for processing after application software transaction in a daemon and shared library cooperation mode.
In the embodiment of the invention, the low-delay network card is an intelligent network card, and the intelligent network card is used for flexibly unloading processing tasks unsuitable for a CPU, meeting the processing requirements of a data plane network and being compatible with the ecology of the existing network protocol, wherein the existing network protocol is one of Ethernet, RDMA or InfiBand.
The embodiment of the invention also provides a method for extremely low-delay reliable communication of the financial transaction system, which comprises the following steps:
S1, after receiving market data, the low-delay network card performs IP/UDP message flow limiting, classification and decoding through an FPGA, and then performs multi-queue hash distribution to a server;
s2, the data message is directly sent to a user state network protocol stack of the high-performance component for UDP processing through a network card driver in the high-performance component by receiving kernel bypass/RDMA;
S3, after user state protocol stack processing, separating out control messages and data messages, wherein the control messages enter session processing, the data messages enter queue cache, the reliable transport layer protocol is used for one-to-many or many-to-many scene publishing and subscribing, high-expansion multi-core expansion (gateway, single machine) is flexibly adapted to dpdk/rdma/bypass intelligent network card, queue parameters and dynamic loading flow control algorithm can be configured, and the reliable transport layer protocol is a self-defined reliable UDP transport protocol which supports flow control algorithm and congestion control algorithm and can be self-defined and modified;
S4, searching the cached message through a Pub/Sub publishing and subscribing mode to obtain all subscribers, and updating the write pointer of the shared buffer area of the subscribers to inform the subscribers that the message arrives;
S5, the application software compares the read-write pointers of the messages to obtain the messages in a mode of inquiring the shared buffer area, and the messages are free from copying in the whole process;
s6, the high-performance communication component provides a configurable parameter interface and performance and monitoring functions, and configures the FPGA to accelerate multi-queues and routing.
In step S2, the high-performance component includes a network card driver, a user mode protocol stack, a reliable UDP communication and Pub/Sub publishing and subscribing component, and the high-performance component operates under a Linux system to independently operate with an upper layer software in a user process, the high-performance component adopts a layered architecture, provides an API for the upper layer software to use, and simultaneously provides a configurable parameter interface and performance and monitoring functions in the component, the system management process is used for configuring network card multi-queue receiving and routing, the idea of the layered design is that the idea of componentization/service is adopted for managing services, abstract layering is carried out on each module of the whole system, and mutual decoupling or loose coupling is carried out between each layer.
As shown in fig. 3, in the embodiment of the present invention, the transmission medium: the protocol of service network communication is unified, multimedia driving is developed and supported in a socket language mode, and network acceleration in a Libc/Kernel bypass/Rdma/Dpdk mode is adapted.
Common components: common library codes and function calls which are frequently used in the development process of a large number of platforms are packaged, so that the development and the use are simpler and more convenient; such as threads, atoms, counters, queues, and time.
Transmission protocol: reliable communication schemes based on the UDP protocol include transport layer message format, connection session establishment, reception and transmission, retransmission mode, congestion control, and flow control.
Socket service: and packaging to realize a socket-style programming interface, a client/server processing mode and message caching. The service process is convenient to carry out non-modification transplanting, and the operation efficiency is improved.
A communication frame: based on the communication framework of the publisher, subscriber and message Que modes, one-to-one, one-to-many, many-to-one and many-to-many communication is realized. From the perspective of the frame, the high stability, the high availability and the high performance are achieved, so that business service operation can be more relieved.
Service interface layer: from the aspect of service requirements, a plurality of API interfaces are supported, including Socket style and C/C++/Java API, and a plurality of application and service modes are adapted, so that the platform is more powerful.
Application and operation: the operation and maintenance only need to pay attention to daily service deployment, configuration, monitoring and other operations. Providing a testing tool, a configuration tool, performance statistics, log tracking, message grabbing and other tools.
In the embodiment of the invention, as shown in fig. 4, the high-performance component comprises a management plane and a data plane, and the interaction of each component module is shown in the figure, and sequentially passes through the intelligent network card hardware, the Kernel bypass/Rdma/Dpdk protocol stack, the adaptation layer, the transmission layer, the protocol layer and the communication session layer from bottom to top, and then passes through the interface layer to be processed by the application process.
Management surface: see lines of small arrows in the figure, the application layer may operate through Tools or business processes using the Api interface, api will issue management commands to Config and Perf modules, and finally configuration and performance modules will read or modify the tables or data of the components, changing their behavior.
Data plane: see the large arrow line in the figure, split into two paths, receive and transmit.
And (3) transmitting: and the application process message producer writes the message into the shared memory queue by calling Api, the daemon Publisher reads the message from the shared memory, sends the message to the corresponding sending channel endpoint according to the subscription relation table, starts sending processing by sender, carries out Segment segmentation aggregation through the transmission layer, encapsulates the reliable UDP protocol DATA format, calls the socket sending function of the adaptation layer, and finally sends the message to the intelligent network card through the UDP protocol stack or bypass protocol stack.
And (3) receiving: after receiving the message, the intelligent network card polls in a poll or Epoll mode through a UDP protocol stack or bypass protocol stack, and after receiving the message by calling an adaptation layer socket receiving function by a receiving thread, the intelligent network card analyzes a reliable UDP protocol DATA format by a transmission layer, performs Segment segmentation splitting processing and then receives the message by a corresponding receiver. And the daemon subscnriber subscribers transmit messages from the corresponding receiving channels according to the subscription relation and write the messages into the corresponding Subscriber shared memory queues. The application process message consumer reads the message from the shared memory queue for further processing by invoking Api polling.
Algorithm surface: based on reliable UDP communication, a reliable mechanism for message transmission needs to be added above the transport layer, packet loss processing policy ARQ retransmission, flow control mechanism sliding window mechanism, and congestion mechanism including congestion control algorithm. The algorithm supports flexible iterative optimization and has a scalable algorithm framework.
Protocol layer: reliable UDP communication supports connection establishment, heartbeat mechanism, ACK/NAK status, RTT measurement, DATA message. And the packet loss rate, the effective bandwidth acquisition and the timing mechanism are supported.
Communication frame layer: one-to-one, one-to-many, many-to-many communication modes are supported as needed, and here, a publisher-subscriber mode is adopted.
The transmission direction is:
1. The sending thread scans and reads the shared memory queue and judges the packet;
2. The message producer maintains a publisher relationship graph Public Image;
3. According to the release relation diagram, a transmission reference counter is maintained and transmitted to a corresponding channel;
Reception direction:
1. Receiving a thread epoll polling message, and judging a received packet;
2. The message consumer maintains a subscriber relationship map Subscrible Image;
3. According to the subscription relation diagram, a receiving reference counter is maintained, and a corresponding channel is received;
4. and writing the received shared memory for the corresponding application process to poll.
Configuration direction:
1. The configuration thread polls the shared memory and judges that the configuration message arrives;
2. the configuration comprises adding and deleting publishers and subscriber relation graphs;
3. configuring a reading counter and queue information;
4. the transport layer configures the channels with respect to the protocol layer.
In the embodiment of the invention, the high-performance component operates in a mode of independent process, and the instance of each application makes a cross-way with the daemon through a message mechanism.
As shown in fig. 5, according to the system scheme and functionality, the transport flow includes stran send transport, socktran socket transport, rtran receive transport and ctran control transport four types:
The stran thread is responsible for sender processing: netpubs the array stores N netpub network published objects, each netpub corresponds to N sep transmitting endpoint objects, each sep can support N sdest transmitting target addresses, and the N sep can support 1-to-many and many-to-many scenes of the publication.
The socktran thread is responsible for the sock processing: the send_list stores an array of send sockets, there are N transock transmit sockets,
Each socket supports a client/server model, with an accept_list accept reply list and a list_list snoop list. Corresponding to 1 socket at the transmitting end. The socket only supports a1 to 1 scenario.
Rtran thread is responsible for the receiving end processing: the inst array stores N netsub network subscription objects, recv_list is a receiving socket list, each netsub corresponds to N pe publishing entity objects, each pe can support an array conns [ ] connected by N rep receiving endpoints, and a receiving node rep can receive N rdest receiving target addresses.
The ctran thread is responsible for configuration and command issuing: and distributing and storing a sep sending node, netpub network objects, inst receiving instances, a pe publishing unit, a rep receiving endpoint, netsub network subscription objects and a socket_list socket. Actions for manipulating these objects, managing the heartbeat of these objects, timeout, delete, etc.
In the embodiment of the invention, the hardware acceleration processing of the FPGA is as shown in fig. 6:
to support distributed, high performance, high scalability. The characteristics of the CPU of the multi-core server can be fully utilized according to N stran threads, N socktran threads and N rtran threads. Priority mapping is carried out on the distribution of the sending directions stran and socketran in the FPGA, protocol processing (1 is used for copying messages which are needed for a plurality of times) is carried out, and finally the messages are sent to a physical port after flow control; the receiver firstly carries out flow supervision (aiming at high burst scene flow balance) of the port, and carries out protocol processing, and finally, the receiver is hashed to the CPU by the RSS receiving thread and is received and processed by the corresponding rtran thread.
And all that is not described in detail in this specification is well known to those skilled in the art.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (5)
1. A method of implementing an extremely low latency reliable communications system for a financial transaction system, comprising: the system comprises a low-delay switch, a low-delay network card, an FPGA and a server;
the low-delay switch is used for forwarding transaction data packets, and the delay is hundreds of ns;
the low-delay network card is used for data encapsulation and decapsulation, link management, encoding and decoding;
the FPGA is used for carrying out hardware acceleration processing on the reliable communication message protocol, distributing a plurality of queues and carrying out CPU hash calculation;
The server is used for providing reliable data transmission service through a high-performance transmission protocol stack and a communication mode, and providing post-transaction processing for application software in a daemon and shared library cooperation mode;
The implementation method of the extremely low-delay reliable communication system for the financial transaction system specifically comprises the following steps:
s1, after receiving market data, the low-delay network card performs IP/UDP message flow limiting, classification and decoding through an FPGA, and then performs multi-queue hash distribution to a server CPU;
s2, the data message is directly sent to a user state network protocol stack of a high-performance component for UDP processing through a network card driver in the high-performance component, the high-performance component comprises the network card driver, the user state protocol stack, reliable UDP communication and Pub/Sub publishing and subscribing components, the high-performance component operates under a Linux system and is in a dynamic library so form and independently operates with upper software in a user process, the high-performance component adopts a layered architecture, an API is provided for the upper software to use, a configurable parameter interface and a performance and monitoring function are provided in the component, and a system management process is used for configuring FPGA multi-queue receiving and routing;
S3, after the user state protocol stack is processed, a control message and a data message are separated through reliable transmission layer protocol processing, the control message enters session processing, and the data message enters queue cache;
S4, searching the cached message through a Pub/Sub publishing and subscribing mode to obtain all subscribers, and updating the write pointer of the shared buffer area of the subscribers to inform the subscribers that the message arrives;
S5, the application software compares the read-write pointers of the messages to obtain the messages by inquiring the shared buffer area, and the messages are free from copying in the whole process.
2. A method of implementing an extremely low latency reliable communications system for a financial transaction system according to claim 1, wherein: the low-delay network card is an intelligent network card, and the intelligent network card is used for flexibly unloading processing tasks unsuitable for a CPU, meeting the processing requirements of a data plane network and being compatible with the existing network protocol.
3. A method of implementing an extremely low latency reliable communication system for use in a financial transaction system according to claim 2, wherein: the existing network protocol is one of ethernet or infiniband.
4. A method of implementing an extremely low latency reliable communications system for a financial transaction system according to claim 1, wherein: the idea of hierarchical design of the high-performance components is to treat services by adopting the idea of componentization/service, abstract and layer each module of the whole system, and mutually decouple or loosely couple each layer.
5. A method of implementing an extremely low latency reliable communications system for a financial transaction system according to claim 1, wherein: the reliable transport layer protocol in the step S3 is used for one-to-many or many-to-many scene publishing and subscribing, high-scalability multi-core expansion, flexible adaptation dpdk/RDMA/bypass intelligent network card, can configure queue parameters and dynamically load a flow control algorithm, is a self-defined reliable UDP transport protocol, supports the flow control algorithm and congestion control algorithm, and can be self-defined and modified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111276461.8A CN114024910B (en) | 2021-10-29 | 2021-10-29 | Extremely low-delay reliable communication system and method for financial transaction system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111276461.8A CN114024910B (en) | 2021-10-29 | 2021-10-29 | Extremely low-delay reliable communication system and method for financial transaction system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114024910A CN114024910A (en) | 2022-02-08 |
| CN114024910B true CN114024910B (en) | 2024-07-05 |
Family
ID=80059347
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111276461.8A Active CN114024910B (en) | 2021-10-29 | 2021-10-29 | Extremely low-delay reliable communication system and method for financial transaction system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114024910B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115002087B (en) * | 2022-05-31 | 2024-07-02 | 济南浪潮数据技术有限公司 | Audio and video data transmission method, system, device and storage medium |
| CN115344527B (en) * | 2022-10-18 | 2023-01-13 | 上海特高信息技术有限公司 | High-speed market data caching method based on FPGA |
| CN116016612A (en) * | 2022-10-25 | 2023-04-25 | 平安科技（深圳）有限公司 | Construction method, device, terminal and storage medium of multi-system link map |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110460540A (en) * | 2019-06-28 | 2019-11-15 | 杨涛 | A kind of low latency high frequency transaction system and method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108289063B (en) * | 2017-09-11 | 2021-07-27 | 上海金融期货信息技术有限公司 | Low-delay communication system |
| CN110611624A (en) * | 2018-06-15 | 2019-12-24 | 上海仪电（集团）有限公司中央研究院 | Massive market quotation data acceleration system and acceleration method based on FPGA |
| CN110517136A (en) * | 2019-07-30 | 2019-11-29 | 上海兮通信息科技有限公司 | A kind of quotation accelerated processing method and system |
| US11477123B2 (en) * | 2019-09-26 | 2022-10-18 | Apple Inc. | Methods and apparatus for low latency operation in user space networking |
| CN113485823A (en) * | 2020-11-23 | 2021-10-08 | 中兴通讯股份有限公司 | Data transmission method, device, network equipment and storage medium |
-
2021
- 2021-10-29 CN CN202111276461.8A patent/CN114024910B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110460540A (en) * | 2019-06-28 | 2019-11-15 | 杨涛 | A kind of low latency high frequency transaction system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114024910A (en) | 2022-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114024910B (en) | Extremely low-delay reliable communication system and method for financial transaction system | |
| US8458280B2 (en) | Apparatus and method for packet transmission over a high speed network supporting remote direct memory access operations | |
| EP3719657A1 (en) | Communication with accelerator via rdma-based network adapter | |
| US10521283B2 (en) | In-node aggregation and disaggregation of MPI alltoall and alltoallv collectives | |
| WO2018133454A1 (en) | Method for controlling remote service access path, and relevant apparatus | |
| US9294569B2 (en) | Cell fabric hardware acceleration | |
| CN113485823A (en) | Data transmission method, device, network equipment and storage medium | |
| US10114792B2 (en) | Low latency remote direct memory access for microservers | |
| US20070208820A1 (en) | Apparatus and method for out-of-order placement and in-order completion reporting of remote direct memory access operations | |
| CN108494817A (en) | Data transmission method, relevant apparatus and system | |
| WO2022148363A1 (en) | Data transmission method and data transmission server | |
| WO2024037296A1 (en) | Protocol family-based quic data transmission method and device | |
| CN110505244A (en) | Long-range tunnel access technique gateway and server | |
| CN118227343B (en) | A data processing method, system, device, equipment, medium and product | |
| CN116471242A (en) | RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method | |
| CN118200253A (en) | RDMA UD transmission-oriented reliable communication method, electronic equipment and readable medium | |
| WO2020187124A1 (en) | Data processing method and device | |
| CN117812027A (en) | RDMA (remote direct memory access) acceleration multicast method, device, equipment and storage medium | |
| WO2024221928A1 (en) | Packet transmission method and device | |
| CN107623645A (en) | A Real-time Data Exchange System of Power System Based on Data Flow Forwarding | |
| Baymani et al. | Exploring RapidIO technology within a DAQ system event building network | |
| CN113676544A (en) | Cloud storage network and method for realizing service isolation in entity server | |
| AU2014200243B2 (en) | System(s) and method(s) for multiple sender support in low latency fifo messaging using tcp/ip protocol | |
| WO2024140068A2 (en) | Message transmission method and system, and network apparatus | |
| CN118101605B (en) | Astronomical radio telescope terminal high-performance data exchange method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |