DE102021202661A1 - TDMA NETWORK WITH COMMODITY NIC / SWITCH - Google Patents

Abstract

A network element that includes one or more network ports, a network timing circuit, and a packet processing circuit. The network ports are configured to communicate with a communication network. The network time switch is configured to track a network time defined in the communication network. In some embodiments, the packet processing circuit is configured to receive a definition of one or more time slots that are synchronized with the network time and to send outgoing packets to the communication network depending on the time slots. In some embodiments, the packet processing circuit is configured to process incoming packets received from the communication network depending on the time slots.

Description

FIELD OF THE INVENTION

The present invention relates generally to computer networks and, more particularly, to time division multiplexing (TDM) and time division multiple access (TDMA) communications over networks.

BACKGROUND OF THE INVENTION

Various techniques are known in the art for enforcing the Time Division Multiplexing (TDM) discipline in networks such as Ethernet. For example, describes "Practical TDMA for Datacenter Ethernet" published by Department of Computer Science and Engineering, University of California, San Diego, April 2012, Vattikonda et al. , A design and implementation of a TDMA Medium Access Control (MAC) layer for off-the-shelf Ethernet hardware that allows end hosts to forego the reliability and congestion control of TCP.

In another example, US patent application publication describes 2019/0319730 Techniques for operating a time division multiplexed (TDM) MAC module, and includes examples of enabling the use of shared resources assigned to ports of a network interface based on a timeslot mechanism, the shared resources being assigned to packet data transmitted through the ports received or sent via the network interface.

SUMMARY OF THE INVENTION

The invention is defined by the claims. To illustrate the invention, aspects and embodiments are described herein which may or may not fall within the scope of the claims.

An embodiment of the present invention described herein provides a network element that includes one or more network ports, a network timing circuit, and a packet processing circuit. The network ports are configured to communicate with a communication network. The network time switch is configured to track a network time defined in the communication network. The packet processing circuit is configured to receive a definition of one or more time slots that are synchronized with the network time and to send outgoing packets to the communication network depending on the time slots.

In some embodiments, the one or more time slots include multiple time slots assigned to the network element in a periodic schedule that is synchronized with the network time. In one embodiment, the packet processing circuit is configured to send the outgoing packets only during the one or more time slots. In another embodiment, the packet processing circuit is configured to generate a series of dummy packets to record network times corresponding to the exit of the dummy packets and to send the outgoing packets to the communication network depending on one or more time slots and the recorded network times .

In some embodiments, the network ports are configured to send the outbound packets to a wireless network that uses time division multiple access (TDMA). In other embodiments, the network ports are configured to send the outgoing packets to an optical switching network that operates in TDMA. In some embodiments, in addition to sending the outgoing packets depending on the time slots, the packet processing circuit is configured to send additional packets to the communication network independently of the time slots. In some embodiments, the outbound packets include Ethernet packets or Infiniband packets.

According to one embodiment of the present invention, a network element is additionally provided which comprises one or more network ports, a network timing circuit and a packet processing circuit. The network ports are configured to communicate with a communication network. The network time switch is configured to track a network time defined in the communication network. The packet processing circuit is configured to receive a definition of one or more time slots that are synchronized with the network time and to process incoming packets received from the communication network depending on the time slots.

In an exemplary embodiment, the packet processing circuit is configured to process an incoming packet only when an arrival time of the incoming packet is during the one or more time slots.

According to an embodiment of the present invention, a network element is also provided which has one or more network ports, a network timer and a Includes packet processing circuitry. The network ports are configured to communicate with a communication network. The network time switch is configured to track a network time defined in the communication network. The packet processing circuit is configured to put outgoing packets in one or more queues in order to assign credits to the queues depending on the network time and to transmit the outgoing packets to the communication network according to the assigned credits.

According to an embodiment of the present invention, a network element is further provided which comprises one or more network ports, a network timing circuit and a packet processing circuit. The network ports are configured to communicate with a communication network. The network time switch is configured to track a network time defined in the communication network. The packet processing circuit is configured to queue outgoing packets pending transmission to the communication network and to transmit a pending outgoing packet to the communication network in synchronization with an event defined according to the network time.

According to one embodiment of the present invention, a network element is also provided which comprises one or more network ports, a network timing circuit and a packet processing circuit. The network ports are configured to communicate with a communication network. The network time switch is configured to track a network time defined in the communication network. The packet processing circuit is configured to transmit outbound packets to the communication network and stop the transmission of an outbound packet at a specified time according to the network time.

According to one embodiment of the present invention, a network element is additionally provided which comprises one or more network ports, a network timing circuit and a packet processing circuit. The network ports are configured to communicate with a communication network. The network time switch is configured to track a network time defined in the communication network. The packet processing circuit is configured to receive incoming packets from the communication network to determine arrival times of the incoming packets according to the network time and to distribute the incoming packets to a plurality of queues based on the arrival times.

According to an embodiment of the present invention, a method for communication is also provided which, in a network element that is connected to a communication network, comprises tracking a network time which is defined in the communication network. A definition of one or more time slots that are synchronized to the network time is received. Outgoing packets are sent from the network element to the communication network depending on the time slots.

According to one embodiment of the present invention, a method for communication is additionally provided which, in a network element which is connected to a communication network, comprises tracking a network time which is defined in the communication network. A definition of one or more time slots that are synchronized to the network time is received. Incoming packets received from the communication network are processed depending on the time slots.

According to one embodiment of the present invention, a method for communication is additionally provided which, in a network element which is connected to a communication network, comprises tracking a network time which is defined in the communication network. Outgoing packets are placed in one or more queues. Credits are assigned to the queues based on the network time. The outgoing packets are transmitted from the network element to the communication network according to the allocated credits.

According to one embodiment of the present invention, a method for communication is also provided which, in a network element that is connected to a communication network, comprises tracking a network time that is defined in the communication network. Outgoing packets that are waiting for transmission to the communication network are placed in the queue. A pending outgoing packet is transmitted from the network element to the communication network synchronously with an event which is defined according to the network time.

According to one embodiment of the present invention, a method for communication is additionally provided which, in a network element that is connected to a communication network, comprises tracking a network time that is defined in the communication network. Outgoing packets are transmitted from the network element to the communication network. The transmission of an outgoing packet becomes a stopped for a certain time according to the network time.

According to an embodiment of the present invention, a method for communication is also provided which, in a network element that is connected to a communication network, comprises tracking a network time which is defined in the communication network. Incoming packets are received in the network element from the communication network. Arrival times of the incoming packets are determined according to the network time. Incoming packets are distributed to multiple queues based on arrival times.

Any feature of an aspect or embodiment can be applied to other aspects or embodiments in any suitable combination. In particular, each feature of a method aspect or an embodiment can be applied to a device aspect or an embodiment and vice versa.

The present invention will be better understood from the following detailed description of its embodiments, together with the drawings, in which:

Figure list

• 1 Figure 3 is a block diagram schematically illustrating a time division multiple access (TDM) node in a radio access network according to an embodiment of the present invention;
• 2 Figure 3 is a block diagram schematically illustrating Precision Time Protocol (PTP) support circuitry 200 in accordance with an embodiment of the present invention;
• 3 Figure 13 is a diagram schematically illustrating values of PTP counters according to an embodiment of the present invention;
• 4th Figure 3 is a block diagram schematically illustrating time-based input packet processing in a network adapter in accordance with embodiments of the present invention;
• 5 Figure 3 is a block diagram schematically illustrating the timed transmission of packets from a network adapter in accordance with embodiments of the present invention; and
• 6th Figure 13 is a block diagram schematically illustrating an optical switching system according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

OVERVIEW

Communication networks such as the Enhanced Common Public Radio Interface (eCPRI), the Optical Data Center Network (ODCN), Video over IP (e.g. Society of Motion Picture and Television Engineers (SMPTE) 2110 ) and others use time division multiple access (TDM) or sometimes time division multiple access (TDMA) for communication between endpoints, where multiple data sources share the same physical medium during different time intervals called time slots.

eCPRI is described, for example, in the eCPRI Specification V2.0 (2019-05-10) by Ericsson AB, Huawei Technologies Co. Ltd, NEC Corporation and Nokia. Optical data center networks are used, for example, in "NEPHELE: an end-to-end scalable and dynamically reconfigurable optical architecture for application-aware SDN cloud datacenters", IEEE Communications Magazine (Volume: 56, Issue: 2, Feb. 2018. DOI: 10.1109 / MCOM.2018.1600804) , by Paraskevas Bakopoulos et al. described.

TDMA multiplexing in high-performance networks requires good synchronization between the endpoints, which is normally achieved using high-precision time bases. Special circuits as described in the Xilinx RoE Framer IP documentation (Xilinx PB056 (v2.1) October 30, 2019) can also be used to send and receive data in the TDM network; however, such specialized circuitry can be expensive and inflexible.

Embodiments of the present invention disclosed herein provide network time-dependent network communications using network elements that include inexpensive network adapters such as network interface controllers (NICs) in the context of Ethernet or host channel adapters (HCAs) in the context of InfiniBand. Although the description below relates primarily to network adapters, the disclosed techniques are not limited to network adapters and can be used with any suitable network elements including, for example, switches and routers.

In some embodiments described herein, a network element (e.g., network adapter, switch, router, or the like) includes one or more network ports for communicating with a communications network, network timing circuitry, and packet processing circuitry. The network time switch is configured to track a network time defined in the communication network.

In some embodiments, the packet processing circuit is configured to receive a definition of one or more time slots that are synchronized with the network time and to send outgoing packets to the communication network or to send incoming packets received from the communication network depending on the time slots to process.

In an exemplary embodiment, the packet processing circuit is configured to enqueue outgoing packets in one or more queues to assign credits to the queues based on the network time and to transmit the outgoing packets to the communication network according to the assigned credits.

In another exemplary embodiment, the packet processing circuit is configured to transmit a pending outbound packet to the communication network in synchronism with an event defined according to the network time.

In yet another embodiment, the packet processing circuit is configured to stop the transmission of an outgoing packet at a specified time according to the network time.

In some embodiments, the packet processing circuit is configured to receive incoming packets from the communication network to determine arrival times of the incoming packets according to the network time and to distribute the incoming packets to multiple queues based on the arrival times.

In another exemplary embodiment, the packet processing circuit is configured to receive one or more time slot assignments assigned to the network element in a TDM schedule that is synchronized with the network time. The packet processing circuit sends outgoing packets to the communication network and / or processes incoming packets received from the communication network only during the time slot assignments assigned to the network element.

In other words, in some embodiments, a particular TDM plan is defined in at least a portion of the communication network. As part of this TDM plan, specific time slots are assigned to a specific network element. In some embodiments, the network element is only allowed to send packets to the network (often to a pre-assigned destination) during the assigned time slots. Additionally or alternatively, in some embodiments the network element is allowed to process received packets only if the packet arrival times match the assigned time slots. This mechanism is useful, for example, for interacting with a wireless or optical switching network that operates in TDMA. Examples of such use cases are described below.

In some embodiments, the network element may send the packet to a specific software entity, such as a queue, in response to the time the network element receives the packet.

According to an exemplary embodiment, a network adapter includes one or more input ports that receive TDM transmissions over an Ethernet network; the network adapter generates time stamps corresponding to the arrival time of the incoming packets, classifies the packets according to the time stamps and other packet information (such as various headers) and, in response to the classification, forwards packets to a host processor connected to the network adapter (in the Hereinafter referred to as “the processor” or “the host”).

According to embodiments, the network adapter can further include one or more output ports; the network adapter receives packets from the processor and sends the packets to the output ports in predefined time slots.

In some embodiments, the receipt and / or transmission of TDM-based packets require a precise time base derived from a high-precision clock included in the network adapter. In one embodiment, the network adapter includes a clock that is frequency and phase locked to the network timeline, and the processor includes a master clock ("wall clock") configured to keep track of the network adapter's clock. This reduces the real-time load on the processor.

In one embodiment, the network adapter includes a precision time protocol, PTP, clock that synchronizes with the network timeline and commands such as "adjust time" and "time." set ”, as implemented in the open source PTP daemon of the Linux community - PTP41 and defined, for example, in the“ Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems ”(IEEE 1588-2008). In some embodiments, the network adapter further comprises a syntonization circuit that continuously adjusts the frequency source of the network adapter to the frequency source of the time base of the network and supports commands such as “adjust frequency” and “set frequency”. For example, certain aspects of network adapters with built-in PTP clocks are described in the U.S. Patent Application 16 / 779,611 , filed on February 2, 2020, and in the U.S. Patent Application 16 / 782,075 , filed February 5, 2020, both assigned to the assignee of the present patent application, the disclosures of which are incorporated herein by reference.

According to some embodiments, in order to send packets at precisely predefined time slots, the network adapter sends an evenly spaced infinite sequence of dummy packets (which typically do not leave the network adapter). Whenever a packet is sent, the network adapter sends a completion queue entry (CQE) consisting of a timestamp (which can be derived from the high-precision clock described above) and a producer index (PI) to the processor and to a Loop-back circuit used to synchronize outgoing packets. Each PI corresponds directly to a different time and can be used to give an accurate time. The processor sends packets to be transmitted to various queues within the network adapter (for example, each virtual machine in the processor sends packets to a different queue) and adds "waiting for PI" messages to the packets; thus the processor precisely controls the time slot in which the network adapter sends the packet from the output port. In some embodiments, the network element further comprises a time-based circuit that blocks any packet that the output circuit may attempt to send to the network outside of the assigned time slot (for example due to a software error or back pressure from the network or for other reasons).

According to one embodiment, the network adapter comprises a packet control circuit which is configured to control packets by adapting a parameter set to predefined values, the parameter set comprising a time of arrival parameter. In order to receive a packet in a predefined time slot, the network adapter stamps the incoming packets with a time stamp (for example using the high-precision clocks described above). The processor then controls the packet control circuitry to adjust a timing parameter corresponding to the assigned time slot of TDM packets. In some embodiments, the packet control circuit is configured to adjust the timeline value to a range of values.

It should be noted that any of the disclosed time-dependent techniques apply to all packets or to a selected subset of packets, e.g. B. packets that are assigned to one or more specified ports or data flows can be applied.

SYSTEM DESCRIPTION

1 Figure 3 is a block diagram schematically showing a time division multiplexed (TDM) node 100 illustrated in a radio access network according to an embodiment of the present invention. The knot 100 includes a network adapter that connects to a TDM network 104 , a processor 106 , a radio circuit 108 and a cellular antenna (or antenna array) 110. The radio circuit 108 communicates wirelessly via the antenna 110 with multiple cellular devices (not shown) and is connected to the processor 106 tied together. The processor 106 is configured to control the radio circuit and data between the antenna and, through the network adapter 102 to broadcast to the TDM network. The processor 106 is further configured to receive a time slot assignment for sending and receiving packets to or from the network. The time slot assignment can be received from a processor (not shown), via the network adapter, or in some other way; in some embodiments, the time slot allocation is done by the processor 106 which can send time slot assignments to other nodes of the TDM network; in other embodiments, each processor on the network generates the time slot assignment independently. The time slots typically include transmission time slots and reception time slots for sending and receiving packets, respectively.

The network adapter 102 includes an input port 112 configured to receive packets from the network; an output port 114 configured to send packets over the network; a time-based packet control unit 118 (also known as a "packet processing circuit"); and a queue 120 for time-controlled sending.

The input port 112 receives packets from the TDM network 104 and sends the packets to the time-based packet control 118 . the Time-based packet control controls the packets according to rules that include conditions that the packet processor should check and actions that the packet processor take when the conditions are met. The conditions may include a range of time and optionally some other conditions derived, for example, from the headers of the packet; for example, the action can be to send the packet to the processor 106 forward or discard the packet and send an appropriate notification to the processor.

In one embodiment, the time domain condition is set in response to the receive time slot assignment that the processor 106 sends to the network adapter. For example, if the receive time slot allocation indicates that packets are destined for the current TDM node 100 are determined to be received in the first 50 microseconds of every 1 millisecond cycle is calculated by the network adapter (or in some embodiments, the processor 106 ) in each cycle the start time and the end time during which the packet can be received. The time-based package control 118 can only forward packets to the processor that are received within the assigned time slots (since none for the processor 106 If certain packets should be received outside of the assigned incoming time slot, the packet control checks 118 in some embodiments, not whether the packet arrives at the assigned input timeslot). Aspects of time-conscious packet control are also cited in the above U.S. Patent Application 16 / 782,075 , filed on February 5, 2020.

The queue 120 Timed Sending is configured to send packets over the TDM network 104 through the output port 114 according to the transmission time slot allocation that the processor 106 receive, send. In one embodiment, the queue 120 include multiple queues for different packet flows for scheduled sending (for example, each queue can be a separate virtual machine within the processor 106 assigned) and each data flow can be assigned a sub-timeslot within the timeslot assigned to the TDM node 100 assigned. The network adapter translates the time slot assignment into a network time for the assigned time slot, and the Timed-Transmit-Queue sends a packet from one of the separate queues over the outgoing port 114 to the network.

The network adapter 102 furthermore comprises a time-based transmission blocking unit 122 configured to block the transmission of packets outside the assigned time slot. In some embodiments, the processor hangs 106 display a start time and an end time indicator to some or all of the outgoing packets; the time-based transmission blocking unit compares the start and end times with the current time, and blocks the transmission of packets before or after the assigned time slot.

In some embodiments, the network adapter comprises 102 also a network timing circuit (not shown), such as a Precision Time Clock (PTP), which is used to precisely time the TDM time slots. The network timing is described below.

Thus, a TDM network as specified for the front link by eCPRI can be implemented using a network adapter connected to a processor; Incoming packets received at predefined receive time slots are routed to the radio circuit, and data from the radio circuit are packetized and sent over the network in predefined transmit time slots. In embodiments according to the present invention, time slots are checked by hardware and therefore the processor does not need to check time slots by polling, which would consume considerable computing resources and would force an accurate synchronization of the processes.

Although the above description (and further descriptions herein below) relates to network adapters, other types of network elements may be used in alternative embodiments, such as switches, routers, and the like.

It goes without saying that the configuration of the TDM node 100 including the network adapter 102 are exemplary configurations presented for conceptual clarity only. Other suitable configurations can be used in alternative embodiments of the present invention. For example, more network ports connecting to the same TDM network, other TDM networks, and / or non-TDM networks can be used. In some embodiments, a single port is implemented. In other embodiments, bidirectional input / output ports are used.

In one embodiment, the network adapter calculates network timeline values that correspond to the start and stop times of time slots in a first TDM cycle, and then calculates timeline values that correspond to the start and stop times of further TDM cycles by adding the duration of a TDM cycle is repeatedly added to the start and stop time values (note that non-TDM queues can coexist with the TDM queues described above).

TIME SYNCHRONIZATION

In some embodiments according to the present invention, the network adapter comprises a precision time clock, PTP. PTP is defined in IEEE 1588-2008 "Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems" (hereinafter "1588") and comprises a hierarchical master-slave architecture for clock distribution. In this architecture, a timeline distribution system consists of one or more communication media (network segments) and one or more clocks that are constantly synchronized with one another. Thus, the PTP timeline value can be adapted to the network timeline. The PTP timeline value is usually an 80-bit integer consisting of a day field (days from January 1, 1970, midnight) and a nanosecond field.

In embodiments, a software PTP client, typically running on the processor, operates a "wall clock" by continuously reading the network timeline from the network adapter. The PTP client can send commands such as “adjust time”, “set time” and “read time” to the PTP circuit. The PTP circuit includes an adjustable oscillator, counter and synchronization circuitry to support such commands.

Fluctuations in the adjustable oscillator are measured, calculated and then attenuated by readjusting the oscillator frequency (e.g. by changing the n and m values of a PLL and / or by changing the voltage input to a voltage controlled oscillator (VCO)).

In some embodiments, the network adapter further comprises circuitry that generates time stamps at a time when an outgoing packet leaves the network adapter and at a time when an incoming packet enters the network adapter; the network adapter then sends the timestamps to the PTP client, which uses the timestamps to calculate the network propagation delay from the network adapter to a peer network element that includes the network timeline ("master timeline").

• ■ Austauschen von Paketen mit einem Netzwerkelement, das den Haupttaktgeber umfasst;
• ■ Extrahieren der Master-Zeitleiste aus den eingehenden Paketen;
• ■ Messen der Netzwerkverzögerungen unter Verwendung von Zeitstempeln, die den ausgetauschten Paketen zugeordnet sind;
• ■ Anpassen der Zeit gemäß der Master-Zeitleiste und den Verzögerungen, Berechnen der Abweichung der Oszillatorfrequenz (z. B. durch Teilen der Zeitabweichung durch die verstrichene Zeit seit der vorherigen Messung); und
• ■ Anpassen des Oszillators, um die gemessene Frequenzdabweichung abzuschwächen.

To discipline a clock (that is, get it to track the frequency and phase of a master clock), a PTP client does the following continuously:

• ■ exchanging packets with a network element comprising the master clock;
• ■ Extract the master timeline from the incoming packets;
• ■ measuring network delays using time stamps associated with the exchanged packets;
• ■ Adjusting the time according to the master timeline and delays, calculating the deviation of the oscillator frequency (e.g. by dividing the time deviation by the time elapsed since the previous measurement); and
• ■ Adjusting the oscillator in order to reduce the measured frequency deviation.

2 Figure 3 is a block diagram schematically showing PTP support circuitry 200 illustrated in accordance with an embodiment of the present invention. The support circuits are with an input port 112 and an output port 114 ( 1 ) that communicate with the network. PTP support circuits 200 include an input timestamp generator 202 configured to generate accurate timestamps when packets from the network enter the network adapter, an output timestamp generator 204 configured to generate accurate timestamps when packets are output from the network adapter to the network, a PTP client software 206 and a PTP circuit 208 . The PTP client software 206 typically runs on the processor 106 . The timestamp generators 202 and 204 and the PTP circuit 208 are typically part of the network adapter 102 in 1 .

The PTP circuit 208 includes a programmable oscillator 210 (e.g. a high-precision phase locked loop (PLL), a voltage controlled oscillator (VCO) or a combination of a PLL and a VCO) and a network time counter 212 that counts the oscillator cycles in nanoseconds and days. In one embodiment, the counter comprises 80 Bits. The PTP circuit forwards the counter value to the input time stamp generator 202 and the output timestamp generator 204 to synchronize the timestamps with the network timeline.

1. i. Kommunizieren von Paketen mit einem Peer-Netzwerkelement, das die Netzwerkzeitleiste umfasst;
2. ii. Empfangen von Zeitstempeln von den Zeitstempelgeneratoren 202 und 204, die sich auf die genaue Eingangs- und Ausgangszeit der Pakete beziehen;
3. iii. Berechnen der Oszillatorabweichung und entsprechendes Anpassen der programmierbaren Oszillatorfrequenz.
4. iv. Messen/Berechnen der Ausbreitungsverzögerung;
5. v. Berechnen der genauen Zeit und entsprechendes Einstellen des Zählers 212.

The PTP client software is configured to:

1. i. Communicating packets with a peer network element comprising the network timeline;
2. ii. Receiving timestamps from the timestamp generators 202 and 204 that focus on the Obtain precise arrival and departure times of parcels;
3. iii. Calculate the oscillator deviation and adjust the programmable oscillator frequency accordingly.
4. iv. Measuring / calculating the propagation delay;
5. v. Calculate the exact time and set the counter accordingly 212 .

To further increase PTP accuracy, in embodiments the network adapter compensates for internal pipeline delays when using the PTP timeline. This can be done by passing the PTP time through a pipeline with an appropriate delay or by adding the difference in the pipeline delay to the counter 212 is added / subtracted.

Thus, the network adapter 102 according to the in 2 The illustrated exemplary embodiment keeping a time counter synchronized with a remote network timeline.

The configuration of the PTP support circuits 200 , in the 2 is an exemplary configuration shown for conceptual clarity only. Any other suitable configurations may be used in alternative embodiments. The network time counter can, for example, count in other resolutions or be subdivided into other fields; all or part of the client software functions can be implemented in specialized circuitry (rather than by the processor 102 ), the timestamp that the input timestamp generator 202 generated, can be concatenated with the corresponding incoming packet (e.g. as headers) and the frequency deviation measurement and setting can be divided into a coarse part and a fine part.

3 is a diagram 300 , which schematically shows the values of the PTP counters according to an embodiment of the present invention. A vertical timeline 302 represents the time on the network adapter ("slave") and a horizontal timeline 304 represents the network timeline ("master"). A 45 degree diagram 306 shows the ideal case where the slave clock exactly follows the master clock. A curve 308 represents the master-slave relationship when there is no syntonization - the gap between the diagram 306 and the diagram 308 increases continuously. The diagram 310 represents the master-slave relationship with synchronization. As can be seen, the slope of the graph becomes 310 adjusted if a discrepancy is detected; if the corrected frequency deviates further, the slope is corrected again so that a good match is achieved. When the time deviation is measured, the PTP client does not set the PTP time to the network time that the PTP client calculates (by subtracting the propagation delay from the received network time) because such settings create a discontinuity in the diagram 310 can cause; instead it becomes the slope of the graph 310 modified so that the diagram 310 the diagram 306 crosses. In alternative embodiments, the PTP client may occasionally set the time.

4th Figure 13 is a block diagram schematically describing time-based input packet processing (also referred to as “packet processing circuitry”) in a network adapter in accordance with embodiments of the present invention. The time-based packet processing 118 (also in 1 shown) includes an input timestamp generator 202 ( 2 ), which is configured to generate accurate timestamps when packets are entered into the network adapter (corresponding to a PTP time entry generated by the network timer, not shown), a parser 404 configured to extract the headers of incoming packets, a rule search engine 406 ; a look-up table 408 and a control unit 416 .

The look-up table 408 includes a time range field 410 , a header field 412 and a field of action 414 . Each entry in the look-up table 408 includes a time range entry, a single header entry, and a single action entry. The header entry can contain 1, 0, and X (doesn't matter) bits. The time range field can include, for example, an upper limit and a lower limit; and the action field includes control information for the packet (and in some embodiments other actions, e.g., security related). In some embodiments, the table 408 is stored in a ternary content-addressable memory (TCAM); in other embodiments, the table is stored in random access memory (RAM) using a hash function; and in still other embodiments, table 408 is stored in a combination of RAM and TCAM.

The rule search engine 406 receives the packet header from the parser 404 and the input timestamp from the input timestamp generator. The rule search engine communicates with the rule table 408 and looks for a match with the timestamp and packet header in the table. The rule search engine 406 can be a finite state machine that runs in one or more clock cycles after a Seek rules matching, or include a single cycle match engine.

If the rule search engine finds a matching rule with the packet header or the time stamp, the rule search engine can send control information for the packet to the control unit 416 send; if no matching rule is found, the rule search engine can send a discard notice to the controller so that a packet that is not received in the assigned time slot is ignored. For example, if the time slot allocation defines that the network adapter receives packets during a predefined time slot, the rules table 408 an entry defining an allowable time range corresponding to the assigned time slot, and an action field instructing the control unit to forward the packet to a predefined virtual machine.

The control unit 416 receives the incoming packets from the ingress port and takes appropriate action from the rule search engine 406 and performs the action. For example, the control unit 416 route the packet to one of multiple output queues, with each output queue associated with a separate virtual machine. If the time indicated by the timestamp is not within the assigned time slot, the control unit typically receives a discard action for the rule search engine and ignores the packet; in some embodiments, the controller can send such packets to a standard queue configured to handle non-TDM traffic.

In some embodiments, the packet control operation may be sequential and consume multiple cycles during which different parts of the packet header are parsed and searched for different rules. This is the purpose of the rule search engine 406 also configured to use the parser 404 to control to parse different header fields in different cycles, and the control unit 406 is also configured to access the rules table 408 to control to send different rules to the rule search engine.

Thus, according to the in 4th illustrated exemplary embodiment of the network adapters 102 TDMA communication from the network; a rule search engine compares the timestamp generated when the network adapter receives the packet with a range corresponding to the assigned time slot and determines an appropriate packet action - e.g. B. Ignoring the packet if the timestamp is not within the assigned time slot, or directing the packet to a destination derived from fields in the packet headers.

The configuration of the time-based inbound packet processing 118 , in the 4th is an exemplary configuration shown for conceptual clarity only. Any other suitable configurations may be used in alternative embodiments. For example, in some embodiments, a relative time can be used to reduce interactions with the processor: i) the time domain field defines a relative time measured from the beginning of the TDMA cycle; ii) the network adapter comprises a TDMA start time register which stores the network time of the start of the current TDMA cycle and increases it in each TDMA cycle by the duration of the TDMA cycle; and, iii) the rule search engine subtracts the TDMA start time register value from the time stamp and searches the rule table with the difference.

In one embodiment, the network adapter comprises separate search circuits for the time stamp and for the control rules, which can work simultaneously or serially; in one embodiment, searches are routed over a line.

In some embodiments, the network adapter can additionally or alternatively modify packet contents based on the arrival time.

PACKAGE LEVEL CONTROL

Aspects of packet level control techniques are, for example, in US Pat U.S. Patent Application 16 / 430,457 , filed June 4, 2019, assigned to the assignee of the present patent application, the disclosure of which is incorporated herein by reference. In a typical packet level control mechanism, a credit allocation circuit sends credits with picosecond resolution to send bits of information to various queues. A queue only issues a pending packet if the number of credits the queue has is greater than the total number of bits in the pending packet. The credit allocation circuit sends credits to the queues according to the bandwidths allocated to the queues and according to the elapsed time. For example, if the credit assignment circuit is called every 10 microseconds and one of the queues is assigned a bandwidth of 200 Mbps, the credit assignment circuit will send 2000 credits to the queue every time the credit assignment circuit is called.

In one embodiment, the credit allocation circuit is coupled to the PTP clock of the network adapter and thus closely follows the network clock.

By sending evenly spaced and fixed duration dummy packets, the network adapter creates equally spaced completion queue entries (CQEs) that correspond to the dummy packets and that include unique producer indexes (PIs). For example, if the processor sends 64-byte dummy packets at a rate that is continuously adjusted to 128,000,000 bytes per second, a CQE is generated every 500 nanoseconds. Since each generated CQE has a corresponding PI, there is a one-to-one correspondence between the PI and the PTP time of the network adapter.

5 Figure 13 is a block diagram schematically illustrating the timed transmission of packets from a network adapter in accordance with embodiments of the present invention. The queue 120 for time-controlled sending ( 1 ) includes Write-Queue-Entry (WQE) queues 122A until 122D , a dummy WQE 122E and rate controllers 124A until 124E . The dummy WQE 122E includes dummy packets that do not necessarily originate from the network adapter, but rather create completion queue entries that are sent to a dummy CQE queue 126 which includes CQE entries, each CQE entry including the producer index (PI) values that the network adapter assigns to the dummy packets. As explained above, the length of the dummy entries and the controlled rate are the dummy WQE 122E such that dummy CQE entries are generated at equal time intervals; since each CQE includes a separate PI, there is a one-to-one correspondence between the PI and when the dummy packets would leave the network adapter (as described above, the dummy packets are not sent to the network).

In the in 5 illustrated exemplary embodiments includes each WQE queue 122A until 122D in addition to the queue entries, commands that instruct the queue (or, more precisely, an undisplayed selector that is configured to extract entries from the WQE queues and send the appropriate packets) to wait until the one from the dummy PI extracted from the CQE queue matches the time of the assigned time slot. The wait commands can include time parameters or a PI value.

The passage of time 128 illustrates the timed exit of packets from the network adapter. According to the in 5 The illustrated exemplary embodiment cover those of the network element 102 allocated time slots for most of the TDMA cycle (as will be apparent this will not be the case in typical embodiments). The network adapter sends packets from WQE1 and WQE2 in odd timeslots and packets from WQE3 and WQE4 in even timeslots. In some embodiments, additional switching guarantees the queue 120 for time-controlled transmission, furthermore, that no packets are sent outside the assigned transmission time slot.

In summary, the network adapter's scheduled send queue includes 102 WQE queues for sending packets over the network, where the queues include WQE entries and wait entries. An additional dummy packet WQE queue can send evenly spaced dummy packets and receive a corresponding equally spaced CQE queue comprising PIs. The WQE queues do not send the stored packets until the waiting commands are executed, that is, until a PI is received that corresponds to the time specified in the waiting command.

It goes without saying that the configuration of the queue 120 for time-controlled sending, which are in 5 is an exemplary configuration shown for conceptual clarity only. Any other suitable configurations may be used in alternative embodiments. For example, in some embodiments the wait commands are not embedded in WQE1 through WQE4; rather, a circuit in the queue for timed send receives queue parameters from the processor (one per queue or one for multiple queues) and deactivates the output of the corresponding queue until the queue parameter matches the PI issued by the dummy CQE. The waiting parameters can be specified in time units or in PI units; in some embodiments, incremental rather than absolute PI or time values may be used. The number of WQE queues can vary and some or all of the rate controllers can be aggregated.

6th Fig. 3 is a block diagram schematically showing an optical switching system 600 illustrated in accordance with an embodiment of the present invention. The optical switching system can be, for example, a high performance data center comprising multiple processors through a fast optical switching network 602 are connected to each other.

Typically, routing based on packet headers cannot be performed in optical switching networks because the packets are not analyzed in the optical medium (to avoid optical-to-electrical conversion and processing delays). Instead a TDMA protocol is used and in each TDMA time slot the optical switching network makes optical connections from a set of input ports to a set of output ports, e.g. B. it establishes and clears line connections over time according to a predefined or real-time calculated schedule. (This is different from non-optical data center networks that implement packet switching instead of line switching.)

According to the in 6th The illustrated exemplary embodiment includes the optical switching network 602 multiple optical switches 604 (three optical switches 604A , 504B and 604C are in the exemplary embodiment of FIG 6th shown; any other suitable number may be used in alternative embodiments). The optical switches are interconnected by optical links that carry light modulated by packet data. Some or all of the optical switches can also be coupled to processors or additional optical switching networks.

A control unit 610 is configured to assign time slots to the optical switches and the coupled processors according to a preset schedule or a real-time calculated schedule generated by a scheduler; in some embodiments, the scheduler can be embedded in the control unit; in other embodiments, the scheduler is external to the control unit, which may or may not be in the control unit.

In the in 6th The exemplary embodiment illustrated is the optical switch 604C via an optical-electrical interface 606 with the input port 112 of the network adapter 102 ( 1 ) and via an electrical-optical interface 608 with the output port 114 coupled. The processor 106 receives the assignment of input timeslots from the control unit 106 and provides the time-based packet control 118 to receive packets at the assigned time slots (as with reference to FIG 4th has been described). Similarly, the processor receives the output timeslot assignment from the controller and sets the time-based queue to send packets on the assigned timeslots (as with reference to FIG 5 described).

In the 6th Configuration of the optical switching system shown 600 is an exemplary configuration shown for conceptual clarity only. In alternative embodiments, any other suitable configurations may be used, including, for example, systems with multiple optical switching networks and hybrid systems with optical and electrical switching networks. The control unit 610 can be implemented as a software program running on a processor (which can be dedicated to controlling the optical network or can be shared with other functions).

In some embodiments, one-way communication is implemented, the optical switch 604C is divided into two unidirectional switches, with the electrical-optical interface 608 is coupled to an optical output switch and the optical-electrical interface 606 is coupled to an optical input switch.

In the above descriptions, techniques for implementing TDM and TDMA networks using network adapters (or, in general, network elements) have been disclosed; two sample applications were demonstrated - eCPRI and optical switching systems. It should be understood that the techniques disclosed are in no way limited to eCPRI and optical switching systems. In alternative embodiments, any other suitable application may be used, including, for example, video-over-IP.

The various components of the network elements described herein, e.g. B. the network adapter 102 ( 1 and 6th ), the PTP support circuits 200 ( 2 ) and / or the time-based packet control unit 118 ( 4th ) can be implemented with suitable hardware, such as in one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs), using software, using hardware or using a combination of hardware and software elements.

In some embodiments, the processor comprises 106 a general purpose processor programmed in software to perform the functions described herein. For example, the software can be downloaded to the processor in electronic form, over a network, or from a host, or can alternatively or additionally be provided and / or on non-volatile tangible material Media, such as magnetic, optical, or electronic storage.

Although the embodiments described herein primarily address network adapters, the methods and systems described herein can also be used in other applications, such as, for example, in network switches.

It is therefore to be understood that the embodiments described above are given by way of example and that the present invention is not limited to what has been particularly shown and described herein above. Rather, the scope of the present invention encompasses both combinations and sub-combinations of the various features described above, as well as variations and modifications thereof which would become apparent to those skilled in the art upon reading the above description and which are not disclosed in the prior art. Documents which are incorporated by reference into the present patent application are to be regarded as an integral part of the application, except that to the extent that terms in these incorporated documents are defined in a manner that is explicitly or implicitly made in the present description Definitions contradicts, only the definitions in the present specification are to be considered.

It goes without saying that aspects and embodiments have only been described above by way of example and that changes in details can be made within the scope of protection of the claims.

Each apparatus, method, and feature disclosed in the description and claims and drawings (if any) may be provided independently or in any suitable combination.

Reference signs appearing in the claims are for illustrative purposes only and are not intended to limit the scope of the claims.

1. 1. Netzwerkelement, das Folgendes umfasst:
   • einen oder mehrere Netzwerk-Ports, die dafür konfiguriert sind, mit einem Kommunikationsnetzwerk zu kommunizieren;
   • eine Netzwerkzeitschaltung, die dafür konfiguriert ist, eine in dem Kommunikationsnetzwerk definierte Netzwerkzeit zu verfolgen; und
   • eine Paketverarbeitungsschaltung, die für Folgendes konfiguriert ist:
     • Empfangen einer Definition von einem oder mehreren Zeitschlitzen, die mit der Netzwerkzeit synchronisiert sind; und
     • Senden ausgehender Pakete in Abhängigkeit von den Zeitschlitzen an das Kommunikationsnetzwerk.
2. 2. Netzwerkelement nach Klausel 1, wobei der eine oder die mehreren Zeitschlitze mehrere Zeitschlitze umfassen, die dem Netzwerkelement in einem periodischen Zeitplan zugewiesen sind, der mit der Netzwerkzeit synchronisiert ist.
3. 3. Netzwerkelement nach Klausel 1, wobei die Paketverarbeitungsschaltung dafür konfiguriert ist, die ausgehenden Pakete nur während des einen oder der mehreren Zeitschlitze zu senden.
4. 4. Netzwerkelement nach Klausel 1, wobei die Paketverarbeitungsschaltung dafür konfiguriert ist, eine Reihe von Dummy-Paketen zu erzeugen, um Netzwerkzeiten entsprechend dem Austritt der Dummy-Pakete aufzuzeichnen, und die ausgehenden Pakete in Abhängigkeit von einem oder mehreren Zeitschlitzen und den aufgezeichneten Netzwerkzeiten an das Kommunikationsnetz zu senden.
5. 5. Netzwerkelement nach Klausel 1, wobei die Netzwerk-Ports dafür konfiguriert sind, die ausgehenden Pakete an ein drahtloses Netzwerk zu senden, das im Zeitmultiplex-Mehrfachzugriff (TDMA) arbeitet.
6. 6. Netzwerkelement nach Klausel 1, wobei die Netzwerk-Ports dafür konfiguriert sind, die ausgehenden Pakete an ein optisches Switching-Netzwerk zu senden, das im Zeitmultiplex-Mehrfachzugriff (TDMA) arbeitet.
7. 7. Netzwerkelement nach Klausel 1, wobei die Paketverarbeitungsschaltung zusätzlich zum Senden der ausgehenden Pakete in Abhängigkeit von den Zeitschlitzen dafür konfiguriert ist, zusätzliche Pakete unabhängig von den Zeitschlitzen an das Kommunikationsnetz zu senden.
8. 8. Netzwerkelement nach Klausel 1, wobei die ausgehenden Pakete Ethernet-Pakete oder Infiniband-Pakete umfassen.
9. 9. Netzwerkelement, das Folgendes umfasst:
   • einen oder mehrere Netzwerk-Ports, die dafür konfiguriert sind, mit einem Kommunikationsnetzwerk zu kommunizieren;
   • eine Netzwerkzeitschaltung, die dafür konfiguriert ist, eine in dem Kommunikationsnetzwerk definierte Netzwerkzeit zu verfolgen; und
   • eine Paketverarbeitungsschaltung, die für Folgendes konfiguriert ist:
     • Empfangen einer Definition von einem oder mehreren Zeitschlitzen, die mit der Netzwerkzeit synchronisiert sind; und Verarbeiten eingehender Pakete, die von dem Kommunikationsnetz empfangen werden, abhängig von den Zeitschlitzen.
10. 10. Netzwerkelement nach Klausel 1, wobei die Paketverarbeitungsschaltung dafür konfiguriert ist, ein eingehendes Paket nur dann zu verarbeiten, wenn eine Ankunftszeit des eingehenden Pakets während des einen oder der mehreren Zeitschlitze liegt.
11. 11. Netzwerkelement, das Folgendes umfasst:
    • einen oder mehrere Netzwerk-Ports, die dafür konfiguriert sind, mit einem Kommunikationsnetzwerk zu kommunizieren;
    • eine Netzwerkzeitschaltung, die dafür konfiguriert ist, eine in dem Kommunikationsnetzwerk definierte Netzwerkzeit zu verfolgen; und
    • eine Paketverarbeitungsschaltung, die für Folgendes konfiguriert ist:
      • Einreihen ausgehender Pakete in eine oder mehrere Warteschlangen;
      • Zuweisen von Krediten zu den Warteschlangen abhängig von der Netzwerkzeit; und
      • Senden der ausgehenden Pakete gemäß den zugewiesenen Krediten an das Kommunikationsnetz.
12. 12. Netzwerkelement, das Folgendes umfasst:
    • einen oder mehrere Netzwerk-Ports, die dafür konfiguriert sind, mit einem Kommunikationsnetzwerk zu kommunizieren;
    • eine Netzwerkzeitschaltung, die dafür konfiguriert ist, eine in dem Kommunikationsnetzwerk definierte Netzwerkzeit zu verfolgen; und
    • eine Paketverarbeitungsschaltung, die für Folgendes konfiguriert ist:
      • Einreihen ausgehender Pakete, die zur Übertragung an das Kommunikationsnetz anstehen, in die Warteschlange; und
      • Senden eines anstehenden ausgehenden Pakets synchron mit einem Ereignis, das gemäß der Netzwerkzeit definiert ist, an das Kommunikationsnetz.
13. 13. Netzwerkelement, das Folgendes umfasst:
    • einen oder mehrere Netzwerk-Ports, die dafür konfiguriert sind, mit einem Kommunikationsnetzwerk zu kommunizieren;
    • eine Netzwerkzeitschaltung, die dafür konfiguriert ist, eine in dem Kommunikationsnetzwerk definierte Netzwerkzeit zu verfolgen; und
    • eine Paketverarbeitungsschaltung, die für Folgendes konfiguriert ist:
      • Senden ausgehender Pakete an das Kommunikationsnetz; und
      • Stoppen der Übertragung eines ausgehenden Pakets zu einer bestimmten Zeit in Übereinstimmung mit der Netzwerkzeit.
14. 14. Netzwerkelement, das Folgendes umfasst:
    • einen oder mehrere Netzwerk-Ports, die dafür konfiguriert sind, mit einem Kommunikationsnetzwerk zu kommunizieren;
    • eine Netzwerkzeitschaltung, die dafür konfiguriert ist, eine in dem Kommunikationsnetzwerk definierte Netzwerkzeit zu verfolgen; und
    • eine Paketverarbeitungsschaltung, die für Folgendes konfiguriert ist:
      • Empfangen von eingehenden Paketen von dem Kommunikationsnetz;
      • Bestimmen von Ankunftszeiten der eingehenden Pakete gemäß der Netzwerkzeit; und
      • Verteilen der eingehenden Pakete basierend auf den Ankunftszeiten auf mehrere Warteschlangen.

The disclosure of this application also includes the following numbered clauses:

1. 1. Network element comprising:
   • one or more network ports configured to communicate with a communication network;
   • a network timing circuit configured to keep track of a network time defined in the communication network; and
   • a packet processing circuit configured to:
     • Receiving a definition of one or more time slots that are synchronized with the network time; and
     • Sending outgoing packets to the communication network depending on the time slots.
2. 2. The network element of clause 1, wherein the one or more time slots comprise a plurality of time slots assigned to the network element in a periodic schedule that is synchronized with the network time.
3. 3. The network element of clause 1, wherein the packet processing circuit is configured to send the outgoing packets only during the one or more time slots.
4. 4. The network element of Clause 1, wherein the packet processing circuit is configured to generate a series of dummy packets to record network times according to the exit of the dummy packets and the outgoing packets depending on one or more time slots and the recorded network times to send the communication network.
5. 5. The network element of clause 1, wherein the network ports are configured to send the outgoing packets to a wireless network using time division multiple access (TDMA).
6. 6. The network element of clause 1, wherein the network ports are configured to send the outgoing packets to an optical switching network employing time division multiple access (TDMA).
7. 7. The network element of clause 1, wherein the packet processing circuit, in addition to sending the outgoing packets depending on the time slots, is configured to send additional packets to the communication network independently of the time slots.
8. 8. The network element of clause 1, wherein the outbound packets comprise Ethernet packets or Infiniband packets.
9. 9. Network element comprising:
   • one or more network ports configured to communicate with a communication network;
   • a network timing circuit configured to keep track of a network time defined in the communication network; and
   • a packet processing circuit configured to:
     • Receiving a definition of one or more time slots that are synchronized with the network time; and processing incoming packets received from the communication network depending on the time slots.
10. 10. The network element of clause 1, wherein the packet processing circuitry is configured to process an incoming packet only if an arrival time of the incoming packet is during the one or more time slots.
11. 11. Network element comprising:
    • one or more network ports configured to communicate with a communication network;
    • a network timing circuit configured to keep track of a network time defined in the communication network; and
    • a packet processing circuit configured to:
      • Putting outgoing packets on one or more queues;
      • Assigning credits to the queues based on the network time; and
      • Sending the outgoing packets to the communication network according to the allocated credits.
12. 12. Network element comprising:
    • one or more network ports configured to communicate with a communication network;
    • a network timing circuit configured to keep track of a network time defined in the communication network; and
    • a packet processing circuit configured to:
      • Queuing outgoing packets pending transmission to the communications network; and
      • Sending a pending outgoing packet to the communication network in synchronization with an event that is defined according to the network time.
13. 13. Network element comprising:
    • one or more network ports configured to communicate with a communication network;
    • a network timing circuit configured to keep track of a network time defined in the communication network; and
    • a packet processing circuit configured to:
      • Sending outgoing packets to the communication network; and
      • Stop the transmission of an outgoing packet at a specified time according to the network time.
14. 14. Network element comprising:
    • one or more network ports configured to communicate with a communication network;
    • a network timing circuit configured to keep track of a network time defined in the communication network; and
    • a packet processing circuit configured to:
      • Receiving incoming packets from the communication network;
      • Determining arrival times of the incoming packets according to the network time; and
      • Distribute incoming packets to multiple queues based on arrival times.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 2019/0319730 [0003]
• US 16779611 [0038]
• US 16782075 [0038, 0045]
• US 16430457 [0075]

Non-patent literature cited

• Department of Computer Science and Engineering, University of California, San Diego, April 2012, Vattikonda et al. [0002]
• Society of Motion Picture and Television Engineers (SMPTE) 2110 [0022]
• "NEPHELE: an end-to-end scalable and dynamically reconfigurable optical architecture for application-aware SDN cloud datacenters", IEEE Communications Magazine (Volume: 56, Issue: 2, Feb. 2018. DOI: 10.1109 / MCOM.2018.1600804) [0023 ]

Claims (15)

Network element comprising: one or more network ports configured to communicate with a communication network; a network timing circuit configured to keep track of a network time defined in the communication network; and a packet processing circuit configured to: Receiving a definition of one or more time slots that are synchronized with the network time; and Sending outgoing packets to the communication network depending on the time slots. Network element after Claim 1 wherein the one or more time slots comprise a plurality of time slots assigned to the network element in a periodic schedule that is synchronized with the network time. Network element after Claim 1 or 2 wherein the packet processing circuit is configured to send the outgoing packets only during the one or more time slots. A network element according to any preceding claim, wherein the packet processing circuit is configured to generate a series of dummy packets to record network times corresponding to the exit of the dummy packets, and the outgoing packets depending on one or more time slots and the recorded network times to send the communication network. A network element as claimed in any preceding claim, wherein the network ports are configured to send the outgoing packets to a wireless network using time division multiple access (TDMA). Network element according to one of the Claims 1 until 4th wherein the network ports are configured to send the outgoing packets to an optical switching network that uses time division multiple access (TDMA). Network element according to one of the preceding claims, wherein the packet processing circuit, in addition to sending the outgoing packets as a function of the time slots, is configured to send additional packets to the communication network independently of the time slots. Network element according to one of the preceding claims, wherein the outgoing packets comprise Ethernet packets or Infiniband packets. The network element of any preceding claim, wherein the processing circuitry is further configured to: Putting outgoing packets on one or more queues; Assigning credits to the queues based on the network time; and Sending the outgoing packets to the communication network according to the allocated credits. The network element of any preceding claim, wherein the processing circuitry is further configured to: Queuing outgoing packets pending transmission to the communications network; and Sending a pending outgoing packet to the communication network in synchronization with an event that is defined according to the network time. The network element of any preceding claim, wherein the processing circuitry is further configured to: Sending outgoing packets to the communication network; and Stop the transmission of an outgoing packet at a specified time according to the network time. Network element comprising: one or more network ports configured to communicate with a communication network; a network timing circuit configured to keep track of a network time defined in the communication network; and a packet processing circuit configured to: Receiving a definition of one or more time slots that are synchronized with the network time; and Processing incoming packets received from the communication network depending on the time slots. Network element after Claim 12 wherein the packet processing circuit is configured to process an incoming packet only when an arrival time of the incoming packet is during the one or more time slots. Network element after Claim 12 or 13th wherein the processing circuit is further configured to: determine arrival times of the incoming packets according to the network time; and distributing the incoming packets to multiple queues based on the arrival times. A method comprising: keeping a network time defined in a communication network in a network timing circuit; Receiving at a packet processing circuit a definition of one or more time slots synchronized with the network time of the communication network; and at least one of: the packet processing circuit sending outgoing packets to the communication network in dependence on the time slots; and processing incoming packets received from the communication network by the packet processing circuit depending on the time slots.

Images