[go: up one dir, main page]

Li, 2022 - Google Patents

Efficient Shuffle for Flash Burst Computing

Li, 2022

View PDF
Document ID
7775685208875887436
Author
Li Y
Publication year

External Links

Snippet

Shuffle is the operation of exchanging arbitrary data among a group of servers, and it is a fundamental communication primitive in distributed computing. In particular, shuffle has been shown to be a fundamental bottleneck to the scalability of flash bursts, a radically new …
Continue reading at web.stanford.edu (PDF) (other versions)

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/54—Store-and-forward switching systems
    • H04L12/56—Packet switching systems
    • H04L12/5693—Queue scheduling in packet switching networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/25—Routing or path finding through a switch fabric
    • H04L49/253—Connections establishment or release between ports
    • H04L49/254—Centralized controller, i.e. arbitration or scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Programme initiating; Programme switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic regulation in packet switching networks
    • H04L47/10—Flow control or congestion control
    • H04L47/24—Flow control or congestion control depending on the type of traffic, e.g. priority or quality of service [QoS]
    • H04L47/2441—Flow classification
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic regulation in packet switching networks
    • H04L47/50—Queue scheduling
    • H04L47/62—General aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogramme communication; Intertask communication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/90—Queuing arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/20—Support for services or operations
    • H04L49/201—Multicast or broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/76—Architectures of general purpose stored programme computers
    • G06F15/78—Architectures of general purpose stored programme computers comprising a single central processing unit

Similar Documents

Publication Publication Date Title
US9503394B2 (en) Clustered dispersion of resource use in shared computing environments
Susanto et al. Stream: Decentralized opportunistic inter-coflow scheduling for datacenter networks
US10148744B2 (en) Random next iteration for data update management
US10931602B1 (en) Egress-based compute architecture for network switches in distributed artificial intelligence and other applications
US10735337B1 (en) Processing packets in an electronic device
Ousterhout A linux kernel implementation of the homa transport protocol
CN112041826A (en) Fine-grained traffic shaping offload for network interface cards
Liu et al. Task-aware TCP in data center networks
Li et al. OPTAS: Decentralized flow monitoring and scheduling for tiny tasks
CN108199985A (en) NoC arbitration method based on global node information in GPGPU
Wu et al. Network congestion avoidance through packet-chaining reservation
Aljoby et al. On SDN-enabled online and dynamic bandwidth allocation for stream analytics
Zahavi et al. Distributed adaptive routing for big-data applications running on data center networks
Liu et al. Joint two-tier network function parallelization on multicore platform
Wilke et al. Opportunities and limitations of quality-of-service in message passing applications on adaptively routed dragonfly and fat tree networks
Hu et al. Highest rank first: A new class of single-iteration scheduling algorithms for input-queued switches
Li Efficient Shuffle for Flash Burst Computing
Susanto et al. Creek: Inter many-to-many coflows scheduling for datacenter networks
Eugster et al. Essential traffic parameters for shared memory switch performance
Gadre et al. Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets
Wu et al. Multicast-based replication for Hadoop HDFS
Eugster et al. Admission control in shared memory switches
Meyer et al. Low latency packet processing in software routers
Wu et al. MCCS: A Service-based Approach to Collective Communication for Multi-Tenant Cloud
Wilke et al. Opportunities and limitations of Quality-of-Service (QoS) in Message Passing (MPI) applications on adaptively routed Dragonfly and Fat Tree networks.