[go: up one dir, main page]

WO2010008984A3 - Method and system for using shared memory with optimized data flow to improve input/output throughput and latency - Google Patents

Method and system for using shared memory with optimized data flow to improve input/output throughput and latency Download PDF

Info

Publication number
WO2010008984A3
WO2010008984A3 PCT/US2009/049963 US2009049963W WO2010008984A3 WO 2010008984 A3 WO2010008984 A3 WO 2010008984A3 US 2009049963 W US2009049963 W US 2009049963W WO 2010008984 A3 WO2010008984 A3 WO 2010008984A3
Authority
WO
WIPO (PCT)
Prior art keywords
data
clients
module
storage system
shared memory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2009/049963
Other languages
French (fr)
Other versions
WO2010008984A2 (en
Inventor
Jeffrey S. Kimmel
Steve C. Miller
Ashish Prakash
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NetApp Inc
Original Assignee
NetApp Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NetApp Inc filed Critical NetApp Inc
Publication of WO2010008984A2 publication Critical patent/WO2010008984A2/en
Publication of WO2010008984A3 publication Critical patent/WO2010008984A3/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G06F12/0868—Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
    • G06F12/084—Multiuser, multiprocessor or multiprocessing cache systems with a shared cache
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/26—Using a specific storage system architecture
    • G06F2212/263—Network storage, e.g. SAN or NAS
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/28—Using a specific disk cache architecture
    • G06F2212/283—Plural cache memories
    • G06F2212/284—Plural cache memories being distributed
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Multi Processors (AREA)

Abstract

The data path in a network storage system is streamlined by sharing a memory among multiple functional modules (e.g., N-module and D-module) of a storage server that facilitates symmetric access to data from multiple clients. The shared memory stores data from clients or storage devices to facilitate communication of data between clients and storage devices and/or between functional modules, and reduces redundant copies necessary for data transport. It reduces latency and improves throughput efficiencies by minimizing data copies and using hardware assisted mechanisms such as DMA directly from host bus adapters over an interconnection, e.g. switched PCI-e "network". This scheme is well suited for a "SAN array" architecture, but also can be applied to NAS protocols or in a unified protocol-agnostic storage system. The storage system can provide a range of configurations ranging from dual module to many modules with redundant switched fabrics for I/O, CPU, memory, and disk connectivity.
PCT/US2009/049963 2008-07-17 2009-07-08 Method and system for using shared memory with optimized data flow to improve input/output throughput and latency Ceased WO2010008984A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12/175,426 2008-07-17
US12/175,426 US8478835B2 (en) 2008-07-17 2008-07-17 Method and system for using shared memory with optimized data flow to improve input/output throughout and latency

Publications (2)

Publication Number Publication Date
WO2010008984A2 WO2010008984A2 (en) 2010-01-21
WO2010008984A3 true WO2010008984A3 (en) 2010-05-06

Family

ID=41531246

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2009/049963 Ceased WO2010008984A2 (en) 2008-07-17 2009-07-08 Method and system for using shared memory with optimized data flow to improve input/output throughput and latency

Country Status (2)

Country Link
US (1) US8478835B2 (en)
WO (1) WO2010008984A2 (en)

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140222758A1 (en) * 2009-08-14 2014-08-07 Ic Manage, Inc. Coherent File State Maintained Among Confederated Repositories By Distributed Workspace Apparatuses Backed Up By a File State Ledgerdemain Store
US8874534B2 (en) * 2009-08-14 2014-10-28 Ic Manage, Inc. File state subset satellites to provide block-based version control
US8392369B2 (en) 2010-09-10 2013-03-05 Microsoft Corporation File-backed in-memory structured storage for service synchronization
US9114386B2 (en) 2010-10-27 2015-08-25 Shell Oil Company Self-activating hydroprocessing catalyst and process for treating heavy hydrocarbon feedstocks
US9513814B1 (en) * 2011-03-29 2016-12-06 EMC IP Holding Company LLC Balancing I/O load on data storage systems
CN102761579B (en) * 2011-04-29 2015-12-09 国际商业机器公司 Storage are network is utilized to transmit the method and system of data
US9063939B2 (en) * 2011-11-03 2015-06-23 Zettaset, Inc. Distributed storage medium management for heterogeneous storage media in high availability clusters
US8732381B2 (en) 2011-11-09 2014-05-20 Hewlett-Packard Development Company, L.P. SAS expander for communication between drivers
US8819353B2 (en) * 2012-06-15 2014-08-26 Lsi Corporation Host bus adapters with shared memory and battery backup
WO2014098839A1 (en) * 2012-12-19 2014-06-26 Hewlett-Packard Development Company Nvram path selection
US10031820B2 (en) * 2013-01-17 2018-07-24 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Mirroring high performance and high availablity applications across server computers
US9825913B2 (en) 2014-06-04 2017-11-21 Nicira, Inc. Use of stateless marking to speed up stateful firewall rule processing
US9692698B2 (en) 2014-06-30 2017-06-27 Nicira, Inc. Methods and systems to offload overlay network packet encapsulation to hardware
FR3023620B1 (en) * 2014-07-09 2016-07-29 Stmicroelectronics (Grenoble 2) Sas METHOD FOR MANAGING THE OPERATION OF A TEST MODE OF A LOGIC COMPONENT WITH RESTORATION OF THE STATE PRECEDING THE TEST
US9838477B2 (en) * 2014-10-30 2017-12-05 Netapp, Inc. Techniques for storing and distributing metadata among nodes in a storage cluster system
US11068420B2 (en) * 2015-05-12 2021-07-20 Hewlett Packard Enterprise Development Lp Scalable software stack
US11038845B2 (en) 2016-02-23 2021-06-15 Nicira, Inc. Firewall in a virtualized computing environment using physical network interface controller (PNIC) level firewall rules
CN107015767B (en) * 2017-04-06 2020-06-23 南京三宝弘正视觉科技有限公司 NAS device, distributed processing system and method
KR102818037B1 (en) * 2020-01-20 2025-06-10 에스케이하이닉스 주식회사 System including a storage device for providing data to an application processor
US12068979B2 (en) * 2020-05-01 2024-08-20 EMC IP Holding Company, LLC System and method for dividing a physical ethernet port
US11962518B2 (en) 2020-06-02 2024-04-16 VMware LLC Hardware acceleration techniques using flow selection
TW202216293A (en) 2020-09-01 2022-05-01 荷蘭商蜆殼國際研究公司 A heavy hydrocarbon hydroprocessing catalyst and methods of making and using thereof
US11636053B2 (en) 2020-09-28 2023-04-25 Vmware, Inc. Emulating a local storage by accessing an external storage through a shared port of a NIC
US11736566B2 (en) 2020-09-28 2023-08-22 Vmware, Inc. Using a NIC as a network accelerator to allow VM access to an external storage via a PF module, bus, and VF module
US11593278B2 (en) 2020-09-28 2023-02-28 Vmware, Inc. Using machine executing on a NIC to access a third party storage not supported by a NIC or host
US11792134B2 (en) 2020-09-28 2023-10-17 Vmware, Inc. Configuring PNIC to perform flow processing offload using virtual port identifiers
US12021759B2 (en) 2020-09-28 2024-06-25 VMware LLC Packet processing with hardware offload units
US11829793B2 (en) 2020-09-28 2023-11-28 Vmware, Inc. Unified management of virtual machines and bare metal computers
US11995024B2 (en) 2021-12-22 2024-05-28 VMware LLC State sharing between smart NICs
US12229578B2 (en) 2021-12-22 2025-02-18 VMware LLC Teaming of smart NICs
US11863376B2 (en) 2021-12-22 2024-01-02 Vmware, Inc. Smart NIC leader election
US20230214265A1 (en) * 2022-01-05 2023-07-06 International Business Machines Corporation High availability scheduler event tracking
US12373237B2 (en) 2022-05-27 2025-07-29 VMware LLC Logical memory addressing by smart NIC across multiple devices
US11928062B2 (en) 2022-06-21 2024-03-12 VMware LLC Accelerating data message classification with smart NICs
US11899594B2 (en) 2022-06-21 2024-02-13 VMware LLC Maintenance of data message classification cache on smart NIC
US11928367B2 (en) 2022-06-21 2024-03-12 VMware LLC Logical memory addressing for network devices
US12481444B2 (en) 2022-06-21 2025-11-25 VMware LLC Smart NIC responding to requests from client device
CN120631662B (en) * 2025-08-08 2025-11-14 浪潮电子信息产业股份有限公司 Data transmission method, device, storage medium and computer program product

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6421711B1 (en) * 1998-06-29 2002-07-16 Emc Corporation Virtual ports for data transferring of a data storage system
US6574667B1 (en) * 1998-06-24 2003-06-03 Emc Corporation Dynamic routing for performance partitioning in a data processing network
US20050097273A1 (en) * 2003-10-29 2005-05-05 Hiroki Kanai Storage device controlling device and control method for storage device controlling device
EP1580656A1 (en) * 2004-03-17 2005-09-28 Hitachi Ltd. Storage control system and control method therefor
US7149922B2 (en) * 2003-01-07 2006-12-12 Hitachi, Ltd. Storage system

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7020754B1 (en) * 2001-12-13 2006-03-28 Emc Corporation Method and system for maintaining data integrity using dual write operations
US6795850B2 (en) * 2002-12-13 2004-09-21 Sun Microsystems, Inc. System and method for sharing memory among multiple storage device controllers
JP4477906B2 (en) * 2004-03-12 2010-06-09 株式会社日立製作所 Storage system
JP2005309739A (en) * 2004-04-21 2005-11-04 Hitachi Ltd Disk array device and cache control method for disk array device
JP4448005B2 (en) * 2004-10-22 2010-04-07 株式会社日立製作所 Storage system
JP4728060B2 (en) * 2005-07-21 2011-07-20 株式会社日立製作所 Storage device
US7500071B2 (en) * 2005-08-31 2009-03-03 International Business Machines Corporation Method for out of user space I/O with server authentication
JP4394670B2 (en) * 2006-09-28 2010-01-06 株式会社日立製作所 Disk control device and storage system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6574667B1 (en) * 1998-06-24 2003-06-03 Emc Corporation Dynamic routing for performance partitioning in a data processing network
US6421711B1 (en) * 1998-06-29 2002-07-16 Emc Corporation Virtual ports for data transferring of a data storage system
US7149922B2 (en) * 2003-01-07 2006-12-12 Hitachi, Ltd. Storage system
US20050097273A1 (en) * 2003-10-29 2005-05-05 Hiroki Kanai Storage device controlling device and control method for storage device controlling device
EP1580656A1 (en) * 2004-03-17 2005-09-28 Hitachi Ltd. Storage control system and control method therefor

Also Published As

Publication number Publication date
US8478835B2 (en) 2013-07-02
US20100017496A1 (en) 2010-01-21
WO2010008984A2 (en) 2010-01-21

Similar Documents

Publication Publication Date Title
WO2010008984A3 (en) Method and system for using shared memory with optimized data flow to improve input/output throughput and latency
JP6974270B2 (en) Intelligent high bandwidth memory system and logic die for it
US11238203B2 (en) Systems and methods for accessing storage-as-memory
US11907139B2 (en) Memory system design using buffer(s) on a mother board
EP3063640B1 (en) A method, apparatus and system for measuring latency in a physical unit of a circuit
WO2023129772A1 (en) Die-to-die interconnect
CN105740195B (en) Method and apparatus for enhanced data bus inversion encoding of OR chained buses
US9946664B2 (en) Socket interposer having a multi-modal I/O interface
EP3035198A1 (en) Low power entry in a shared memory link
CN105183683B (en) A kind of more fpga chip accelerator cards
WO2011068614A3 (en) Controller device coprocessor architecture
WO2023146671A1 (en) Retimers to extend a die-to-die interconnect
US20070098001A1 (en) PCI express to PCI express based low latency interconnect scheme for clustering systems
US11671522B2 (en) System and method for memory access in server communications
US20160116959A1 (en) Device power management state transition latency advertisement for faster boot time
WO2007106830A1 (en) Input/output agent having multiple secondary ports
US20200234396A1 (en) Heterogeneous computation and hierarchical memory image sensing pipeline
CN107430574A (en) Method and apparatus for the optimization of the IO of analysis system, processing and bandwidth of memory
US10318473B2 (en) Inter-device data-transport via memory channels
CN109845113B (en) Multi-channel cache memory and system memory device
US11544009B2 (en) Heterogeneous computation and hierarchical memory image sensing pipeline
EP2801032B1 (en) Bimodal functionality between coherent link and memory expansion
US9317089B2 (en) Mesh performance improvement using dual voltage data transfer
US20160110136A1 (en) ISOLATED SHARED MEMORY ARCHITECTURE (iSMA)
US20090177832A1 (en) Parallel computer system and method for parallel processing of data

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09798580

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09798580

Country of ref document: EP

Kind code of ref document: A2