[go: up one dir, main page]

SG11201707304RA - Data check method and storage system - Google Patents

Data check method and storage system

Info

Publication number
SG11201707304RA
SG11201707304RA SG11201707304RA SG11201707304RA SG11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA
Authority
SG
Singapore
Prior art keywords
data
node
data write
check
storage system
Prior art date
Application number
SG11201707304RA
Inventor
Mingchang Wei
Original Assignee
Huawei Tech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Tech Co Ltd filed Critical Huawei Tech Co Ltd
Publication of SG11201707304RA publication Critical patent/SG11201707304RA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0611—Improving I/O performance in relation to response time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0688—Non-volatile semiconductor memory arrays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1016—Performance improvement
    • G06F2212/1024—Latency reduction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1032—Reliability improvement, data loss prevention, degraded operation etc
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/15—Use in a specific computing environment
    • G06F2212/154—Networked environment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/25—Using a specific main memory architecture
    • G06F2212/254—Distributed memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/38—Universal adapter
    • G06F2213/3854—Control is performed at the peripheral side

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Memory System (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)
  • Detection And Correction Of Errors (AREA)

Abstract

DATA CHECK METHOD AND STORAGE SYSTEM A storage system includes a host, a check node, and multiple data nodes. The host is configured to: divide multiple data write requests into multiple data write request sets according to data node identifiers included in the data write requests, where each data write 5 request set includes multiple data write requests having a same data node identifier; and when sizes of all data write requests in each of a specified quantity of data write request sets reach a preset data amount, calculate check data of the specified quantity of data write request sets. The host is further configured to: send each data write request set to a data node indicated by an identifier, and send the check data to the check node. Therefore, on a basis of ensuring 10 data reliability, data of a same type can be stored in a same storage node and the data can be read from only one storage node, thereby improving data reading efficiency. 15 Figure 3
SG11201707304RA 2016-11-25 2016-11-25 Data check method and storage system SG11201707304RA (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/107355 WO2018094704A1 (en) 2016-11-25 2016-11-25 Data check method and storage system

Publications (1)

Publication Number Publication Date
SG11201707304RA true SG11201707304RA (en) 2018-06-28

Family

ID=62188856

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201707304RA SG11201707304RA (en) 2016-11-25 2016-11-25 Data check method and storage system

Country Status (9)

Country Link
US (1) US10303374B2 (en)
EP (1) EP3352071B1 (en)
JP (1) JP6526235B2 (en)
CN (1) CN109074227B (en)
AU (1) AU2016397189B2 (en)
BR (1) BR112017020736B8 (en)
CA (1) CA2978927C (en)
SG (1) SG11201707304RA (en)
WO (1) WO2018094704A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113302597A (en) * 2019-04-23 2021-08-24 华为技术有限公司 Distributed storage system and garbage recycling method in distributed storage system
CN111949434B (en) * 2019-05-17 2022-06-14 华为技术有限公司 RAID management method, RAID controller and system
JP7490394B2 (en) 2020-03-06 2024-05-27 株式会社日立製作所 Information sharing support method and information sharing support system
US12066980B2 (en) * 2020-05-12 2024-08-20 Hewlett Packard Enterprise Development Lp File system metadata
CN112365124A (en) * 2020-10-16 2021-02-12 重庆恢恢信息技术有限公司 Intelligent construction site construction material rapid distribution working method
CN114697349A (en) * 2020-12-28 2022-07-01 华为技术有限公司 Method for processing data by using intermediate device, computer system and intermediate device
CN113419684B (en) * 2021-07-09 2023-02-24 深圳大普微电子科技有限公司 Data processing method, device and equipment and readable storage medium
CN116166179A (en) * 2021-11-25 2023-05-26 华为技术有限公司 Data storage system, smart network card and computing node
CN114625318A (en) * 2022-02-28 2022-06-14 锐掣(杭州)科技有限公司 Data writing method, device and equipment applied to solid state disk
CN115933995B (en) * 2023-01-09 2023-05-09 苏州浪潮智能科技有限公司 Data writing method, device, electronic equipment and readable medium in solid-state hard disk

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3713788B2 (en) 1996-02-28 2005-11-09 株式会社日立製作所 Storage device and storage device system
US6742081B2 (en) * 2001-04-30 2004-05-25 Sun Microsystems, Inc. Data storage array employing block checksums and dynamic striping
JP4383321B2 (en) * 2004-11-09 2009-12-16 富士通株式会社 Storage control device and external storage device
US8332608B2 (en) * 2008-09-19 2012-12-11 Mediatek Inc. Method of enhancing command executing performance of disc drive
JP5317827B2 (en) * 2009-05-19 2013-10-16 日本電信電話株式会社 Distributed data management apparatus, method and program
CN102546755A (en) * 2011-12-12 2012-07-04 华中科技大学 Data storage method of cloud storage system
CN102968498B (en) * 2012-12-05 2016-08-10 华为技术有限公司 Data processing method and device
JP6005533B2 (en) * 2013-01-17 2016-10-12 株式会社東芝 Storage device and storage method
JP2014203233A (en) * 2013-04-04 2014-10-27 株式会社日立製作所 Storage system and method for updating data in storage system
CN104216664B (en) * 2013-06-26 2017-06-09 华为技术有限公司 Network volume creation method, date storage method, storage device and storage system
CN103647797A (en) * 2013-11-15 2014-03-19 北京邮电大学 Distributed file system and data access method thereof
CN103699494B (en) 2013-12-06 2017-03-15 北京奇虎科技有限公司 A kind of date storage method, data storage device and distributed memory system
WO2015100627A1 (en) * 2013-12-31 2015-07-09 华为技术有限公司 Data processing method and device in distributed file storage system
CN103761058B (en) * 2014-01-23 2016-08-17 天津中科蓝鲸信息技术有限公司 RAID1 and RAID4 mixed structure network store system and method
JP2016184372A (en) * 2015-03-27 2016-10-20 富士通株式会社 Storage system, information processing device, parity generation program, and parity generation method
CN107844268B (en) * 2015-06-04 2021-09-14 华为技术有限公司 Data distribution method, data storage method, related device and system
CN105404469B (en) * 2015-10-22 2018-11-13 浙江宇视科技有限公司 A kind of storage method and system of video data
CN105930103B (en) * 2016-05-10 2019-04-16 南京大学 A kind of correcting and eleting codes covering write method of distributed storage CEPH

Also Published As

Publication number Publication date
BR112017020736B1 (en) 2021-01-19
BR112017020736A2 (en) 2018-07-17
WO2018094704A1 (en) 2018-05-31
EP3352071B1 (en) 2019-08-28
BR112017020736B8 (en) 2021-08-17
US10303374B2 (en) 2019-05-28
AU2016397189B2 (en) 2019-07-25
CN109074227A (en) 2018-12-21
CA2978927A1 (en) 2018-05-25
CA2978927C (en) 2019-09-17
EP3352071A1 (en) 2018-07-25
AU2016397189A1 (en) 2018-06-14
CN109074227B (en) 2020-06-16
JP2019504369A (en) 2019-02-14
JP6526235B2 (en) 2019-06-05
EP3352071A4 (en) 2018-07-25
US20180364920A1 (en) 2018-12-20

Similar Documents

Publication Publication Date Title
SG11201707304RA (en) Data check method and storage system
MX2023000186A (en) SYSTEMS AND METHODS TO CREATE A UNIVERSAL REGISTRY.
PH12017500241A1 (en) File access method, device and storage system
EP4242821A3 (en) Distributed transactions with token-associated execution
RU2015104116A (en) METHOD AND STORAGE DEVICE FOR DISTRIBUTED FILE SYSTEM
WO2015066719A3 (en) Use of solid state storage devices and the like in data deduplication
SG11201907619VA (en) Blockchain data storage system and method
SG11201907942QA (en) Blockchain cluster processing system and method, computer device and storage medium
PH12019501501B1 (en) Method and apparatus for verifying block data in a blockchain
CN106687911A8 (en) The online data movement of data integrity is not damaged
SA516370693B1 (en) Ascertaining command completion in flash memories
GB2550800A (en) Managing data in storage according to a log structure
GB2492870A (en) Optimizing a file system for different types of applications in a compute cluster using dynamic block size granularity
CN104199784B (en) A kind of data migration method and device based on classification storage
WO2015020900A3 (en) Method and device for error correcting code (ecc) error handling
EP3822798A3 (en) Memory access technology and computer system
CN107454161A (en) A data backup method and device
BR112015018922A8 (en) device, method and one or more computer readable non-transient storage media for routine estimation
WO2014114817A3 (en) System and method for replicating data
CN110199270A (en) The management method and device of equipment are stored in storage system
CA2902868C (en) Managing operations on stored data units
CN103473266A (en) Solid state disk and method for deleting repeating data thereof
GB2532667A (en) Memory management
CN104217011A (en) Method and device for inquiring HBase secondary index table
CN101763433A (en) Data storage system and method