[go: up one dir, main page]

CN104536902A - Performance optimization method for IO (Input/Output) subsystem by testing server - Google Patents

Performance optimization method for IO (Input/Output) subsystem by testing server Download PDF

Info

Publication number
CN104536902A
CN104536902A CN201510043037.7A CN201510043037A CN104536902A CN 104536902 A CN104536902 A CN 104536902A CN 201510043037 A CN201510043037 A CN 201510043037A CN 104536902 A CN104536902 A CN 104536902A
Authority
CN
China
Prior art keywords
testing server
server
subsystem
performance
write
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201510043037.7A
Other languages
Chinese (zh)
Inventor
黄福超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
IEIT Systems Co Ltd
Original Assignee
Inspur Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Electronic Information Industry Co Ltd filed Critical Inspur Electronic Information Industry Co Ltd
Priority to CN201510043037.7A priority Critical patent/CN104536902A/en
Publication of CN104536902A publication Critical patent/CN104536902A/en
Pending legal-status Critical Current

Links

Landscapes

  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a performance optimization method for an IO (Input/Output) subsystem by a testing server, relating to the server IO subsystem. The method comprises the steps of arranging an SAS (Serial Attached SCSI) disc and a data disc on the testing server, and customizing read-write scheduling mode using a script according to the application diversity; through arranging an operation system on the testing server, after the system is normally logged, writing the optimization strategy and storing the optimization strategy; and finally, running the optimization strategy before testing. The method can effectively improve the performance of the continuous read-write mode of the magnetic disk and can improve the utilization rate of the hardware.

Description

一种测试服务器IO子系统的性能调优方法A Performance Tuning Method for Testing Server IO Subsystem

技术领域 technical field

本发明涉及服务器IO子系统,具体的说就是一种测试服务器IO子系统的性能调优方法。 The invention relates to a server IO subsystem, in particular to a performance tuning method for testing the server IO subsystem.

背景技术 Background technique

随着互联网发展越来越迅速,流媒体大量应用,有代表性的优酷、土豆等为网民提供视频服务。与此对应要求服务器具有极高的读写吞吐性能。客户在选用设备时会进行性能测试比较,性能高的产品占据优势。 With the rapid development of the Internet, a large number of streaming media applications, representative Youku, Tudou, etc. provide video services for Internet users. Correspondingly, the server is required to have extremely high read and write throughput performance. Customers will conduct performance tests and comparisons when selecting equipment, and products with high performance will take advantage.

通常Linux操作系统默认对用户的IO操作不做判断,按照同一种方式进行安排,当服务器压力逐渐增大时,服务器IO性能方面的表现逐渐下降,突出表现为IO带宽增长放缓出现瓶颈,系统的IO延时增长,拖慢整体应用程序的进度。顺序读写时,IO操作具有一定的预知性,当某一读写操作执行完成时,系统进行预读磁盘连续位置的数据,命中的情况下会提升读写效率。 Usually, the Linux operating system does not judge the user's IO operation by default, and arranges it in the same way. When the pressure on the server gradually increases, the performance of the server's IO performance gradually declines. The IO latency increases, slowing down the progress of the overall application. When reading and writing sequentially, the IO operation has a certain degree of predictability. When a certain read and write operation is completed, the system will pre-read the data in the continuous position of the disk. If there is a hit, the read and write efficiency will be improved.

操作系统Linux下的IO操作有默认的调度算法,不同应用程序产生不同类型的IO操作。以读写测试工具Iozone为代表的IO测试工具,在测试时依据测试进程产生IO操作,通过相同进程可能产生具有相同的异步读写方式,此时具有测试读写操作的IO调度测试数值会有提升。 The IO operation under the operating system Linux has a default scheduling algorithm, and different applications generate different types of IO operations. The IO test tool represented by the read-write test tool Iozone generates IO operations according to the test process during the test. The same process may generate the same asynchronous read-write mode. At this time, the IO scheduling test value with test read-write operations will have promote.

发明内容 Contents of the invention

本发明的目的是提供一种针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,通过该方法指导厂商、用户对现有设备进行调优,实现设备IO最大化。 The purpose of the present invention is to provide a performance tuning method for the IO subsystem of the read-write test tool Iozone test server, through which the manufacturer and the user are guided to tune the existing equipment to maximize the IO of the equipment.

本发明所述一种测试服务器IO子系统的性能调优方法,解决上述技术问题采用的技术方案如下:该方法是针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,需要一台服务器作为测试服务器,并在测试服务器上搭配SAS盘和数据盘,根据自身应用多样性使用脚本来自定义读写调度方式;通过在测试服务器上安装操作系统,正常登录系统后,编写调优策略,并保存所述调优策略;最后在测试之前,运行所述调优策略。 A kind of performance tuning method of test server IO subsystem described in the present invention, the technical scheme that solves the above-mentioned technical problem adopts is as follows: this method is the performance tuning method for Iozone test server IO subsystem of reading and writing test tool, needs a The server is used as a test server, and the test server is equipped with SAS disks and data disks, and scripts are used to customize the read and write scheduling methods according to the diversity of its own applications; by installing the operating system on the test server and logging in to the system normally, write the tuning strategy, And save the tuning strategy; finally, run the tuning strategy before testing.

优选的,所述测试服务器采用机架式服务器,并搭配2块SAS盘,且做raid1装操作系统,同时搭配12块数据盘,均做raid0装操作系统。 Preferably, the test server adopts a rack server, and is equipped with 2 SAS disks, and is configured as raid1 to install the operating system, and is configured with 12 data disks, and is configured as raid0 to install the operating system.

优选的,在测试服务器上安装操作系统CentOS6.4,并默认正常安装。 Preferably, the operating system CentOS6.4 is installed on the test server, and it is installed normally by default.

优选的,所述测试服务器采用NF5270M3,搭配的SAS盘为2块300G 2.5寸SAS盘,搭配的数据盘为12块2T 3.5寸盘。 Preferably, the test server adopts NF5270M3, and the matching SAS disks are two 300G 2.5-inch SAS disks, and the matching data disks are twelve 2T 3.5-inch disks.

本发明的一种测试服务器IO子系统的性能调优方法与现有技术相比具有的有益效果是:该方法采用测试设备,重新编写调优策略,通过脚本实现对算法、参数的选择设定,能够有效提升磁盘连续读写等方式的性能,提高硬件的利用率;通过该方法,可以对现有服务器的磁盘性能在Iozone测试下进行一定量提升,对硬件厂商、最终用户都有指导意义。 Compared with the prior art, the performance tuning method of a test server IO subsystem of the present invention has the following beneficial effects: the method uses test equipment, rewrites the tuning strategy, and realizes the selection and setting of algorithms and parameters through scripts , which can effectively improve the performance of disk continuous reading and writing, and improve the utilization rate of hardware; through this method, the disk performance of existing servers can be improved to a certain extent under the Iozone test, which has guiding significance for hardware manufacturers and end users .

具体实施方式 Detailed ways

为使本发明的目的、技术方案和优点更加清楚明白,以下结合具体实施例,对本发明所述一种测试服务器IO子系统的性能调优方法进一步详细说明。 In order to make the purpose, technical solution and advantages of the present invention clearer, the performance tuning method of the IO subsystem of the test server described in the present invention will be further described in detail below in conjunction with specific embodiments.

本发明所述一种测试服务器IO子系统的性能调优方法,是针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,根据自身应用多样性使用脚本来自定义读写调度方式,具备灵活性、实用性及可操作。 The performance tuning method of a test server IO subsystem described in the present invention is aimed at the performance tuning method of the read-write test tool Iozone test server IO subsystem, and uses scripts to customize the read-write scheduling method according to its own application diversity, and has Flexibility, practicality and operability.

本发明所述性能调优方法需要一台服务器作为测试服务器,并在测试服务器上搭配SAS盘和数据盘;然后在测试服务器上安装操作系统,正常登录系统后,编写调优策略,并保存所述调优策略;最后在测试之前,运行所述调优策略。 The performance tuning method described in the present invention needs a server as a test server, and a SAS disk and a data disk are matched on the test server; then an operating system is installed on the test server, and after a normal login to the system, an optimization strategy is written and saved. The tuning strategy described above; finally, before testing, the tuning strategy is run.

实施例: Example:

本实施例所述一种测试服务器IO子系统的性能调优方法,采用机架式服务器作为测试服务器,并搭配2块SAS盘,且做raid1装操作系统,同时搭配12块数据盘,均做raid0装操作系统;本实施例所述性能调优方法在测试服务器上安装操作系统CentOS6.4,并默认正常安装。 A performance tuning method for the test server IO subsystem described in this embodiment uses a rack-mounted server as the test server, and is equipped with 2 SAS disks, and is configured as raid1 to install the operating system, and is equipped with 12 data disks at the same time. The raid0 installs the operating system; the performance tuning method described in this embodiment installs the operating system CentOS6.4 on the test server, and it is installed normally by default.

本实施例所述测试服务器IO子系统的性能调优方法,其具体实施过程为:在测试服务器正常登录系统后,编写调优策略: The performance tuning method of the test server IO subsystem described in this embodiment, its specific implementation process is: after the test server logs in to the system normally, write the tuning strategy:

for i in {b..m}for i in {b..m}

dodo

     echo anticipatory >/sys/block/sd$i/queue/schedulerecho anticipatory >/sys/block/sd$i/queue/scheduler

  echo 15000 >/sys/block/sd$i/queue/iosched/read_expireecho 15000 >/sys/block/sd$i/queue/iosched/read_expire

  echo 30000 >/sys/block/sd$i/queue/iosched/write_expireecho 30000 >/sys/block/sd$i/queue/iosched/write_expire

  echo 4096 >/sys/block/sd$i/queue/nr_requestsecho 4096 >/sys/block/sd$i/queue/nr_requests

  echo 5 >/sys/block/sd$i/device/queue_depthecho 5 >/sys/block/sd$i/device/queue_depth

  blockdev --setra 512 /dev/sd$iblockdev --setra 512 /dev/sd$i

donedone

并保存上述调优策略。 And save the above tuning strategy.

下面对本实施例所述方法的上述调优策略进行详细解释: The above-mentioned tuning strategy of the method described in this embodiment is explained in detail below:

1)  第1-2行:在sdb-sdm12块盘上(默认测试环境12块数据盘)作如下操作; 1) Lines 1-2: Do the following operations on sdb-sdm12 disks (the default test environment is 12 data disks);

2)  第3行:默认系统IO策略是cfq(完全公平队列),而针对流媒体应用类的读写,连续读写概率大,改为anticipatory(预测性策略)将会预读连续扇区,增大读写效率; 2) Line 3: The default system IO policy is cfq (completely fair queue), and for streaming media applications, the probability of continuous reading and writing is high, changing to anticipatory (predictive strategy) will pre-read continuous sectors, Increase reading and writing efficiency;

3)  第4-5行:读写时间片超时设定,一般 read/write=1/2搭配出现,设置值大可以有效预防读写超时; 3) Lines 4-5: read and write time slice timeout setting, generally read/write=1/2 appear together, a large setting value can effectively prevent read and write timeout;

4)  第6-8行:将请求合并值、队列深度值按照对应值修改; 4) Lines 6-8: Modify the request merge value and queue depth value according to the corresponding value;

5)  第9行:结束。 5) Line 9: end.

通过本实施例所述测试服务器IO子系统的性能调优方法,在测试之前,运行上述调优策略。以下是该方法的效果验证,通过更改策略前后三次Iozone测试值比较(单位KB/s): Through the performance tuning method of the test server IO subsystem described in this embodiment, the above tuning strategy is run before the test. The following is the effect verification of this method, through the comparison of three Iozone test values before and after changing the strategy (in KB/s):

更改前: Before change:

 Initial write " 1812105.56 " Initial write " 1812105.56 "

Read " 1554452.91 " Read " 1554452.91 "

Initial write " 1816866.94 " Initial write " 1816866.94 "

         Read " 1544720.16 " Read " 1544720.16 "

 Initial write " 1813688.34 " Initial write " 1813688.34 "

Read " 1537983.47 Read " 1537983.47

   更改后: After the change:

 Initial write " 1887822.60 " Initial write " 1887822.60 "

 Read " 1595224.81 " Read " 1595224.81 "

Initial write " 1893080.30“ Initial write " 1893080.30"

Read " 1594529.09 " Read " 1594529.09 "

Initial write " 1882738.75 " Initial write " 1882738.75 "

Read " 1591268.60“ Read "1591268.60"

对比以上测试结果,调整后策略写最高提升:4.47%;读最高提升:3.72%,且数值较稳定。以上调优值均是针对流媒体测试,经过多次修改反复试验所得,若更改测试模型,可做相应修改,亦可增加读写数值。 Comparing the above test results, after the adjustment, the highest increase in writing strategy: 4.47%; the highest increase in reading: 3.72%, and the value is relatively stable. The above tuning values are all for streaming media testing. After many modifications and repeated tests, if you change the test model, you can make corresponding modifications, and you can also increase the read and write values.

除本实施例所述测试服务器IO子系统的性能调优方法,所采用的测试设备外,本发明所述方法还可以采用测试设备:测试服务器为NF5270M3(浪潮2路机架式服务器),搭配的SAS盘为2块300G 2.5寸SAS盘,搭配的数据盘为12块2T 3.5寸大盘。 In addition to the performance tuning method of the test server IO subsystem described in this embodiment and the test equipment used, the method of the present invention can also use test equipment: the test server is NF5270M3 (Inspur 2-way rack server), with The SAS disks are two 300G 2.5-inch SAS disks, and the matching data disks are twelve 2T 3.5-inch large disks.

综上所述,本发明提供了一种针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,通过脚本实现对算法、参数的选择设定,可以有效提升磁盘连续读写等方式的性能,提高硬件的利用率。 In summary, the present invention provides a performance tuning method for the IO subsystem of the read-write test tool Iozone test server. The selection and setting of algorithms and parameters can be realized through scripts, which can effectively improve the performance of continuous disk read-write and other methods. performance and improve hardware utilization.

上述具体实施方式仅是本发明的具体个案,本发明的专利保护范围包括但不限于上述具体实施方式,任何符合本发明的权利要求书的且任何所属技术领域的普通技术人员对其所做的适当变化或替换,皆应落入本发明的专利保护范围。 The above-mentioned specific embodiments are only specific cases of the present invention, and the scope of patent protection of the present invention includes but is not limited to the above-mentioned specific embodiments, any claims that meet the claims of the present invention and any ordinary skilled person in the technical field. Appropriate changes or substitutions should fall within the scope of patent protection of the present invention.

Claims (4)

1. the Performance tuning method of a testing server IO subsystem, it is characterized in that, for the Performance tuning method of readwrite tests instrument Iozone testing server IO subsystem, need a station server as testing server, and arrange in pairs or groups on testing server SAS dish and data disks, use the self-defined read-write scheduling mode of script according to self application diversity; By installing operating system on testing server, after normal login system, write tuning strategy, and preserve described tuning strategy; Before test, run described tuning strategy finally.
2. the Performance tuning method of a kind of testing server IO subsystem according to claim 1, is characterized in that, described testing server adopts rack-mount server, and 2 pieces of SAS dishes of arranging in pairs or groups, and do raid1 dress operating system, 12 blocks of data dishes of simultaneously arranging in pairs or groups, be all raid0 and fill operating system.
3. the Performance tuning method of a kind of testing server IO subsystem according to claim 2, is characterized in that, installing operating system CentOS6.4 on testing server, and gives tacit consent to normal mounting.
4. the Performance tuning method of a kind of testing server IO subsystem according to claim 3, is characterized in that, described testing server adopts NF5270M3, and the SAS dish of collocation is 2 pieces of 300G, 2.5 cun of SAS dishes, and the data disks of collocation is 12 pieces of 2T 3.5 cun dishes.
CN201510043037.7A 2015-01-28 2015-01-28 Performance optimization method for IO (Input/Output) subsystem by testing server Pending CN104536902A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510043037.7A CN104536902A (en) 2015-01-28 2015-01-28 Performance optimization method for IO (Input/Output) subsystem by testing server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510043037.7A CN104536902A (en) 2015-01-28 2015-01-28 Performance optimization method for IO (Input/Output) subsystem by testing server

Publications (1)

Publication Number Publication Date
CN104536902A true CN104536902A (en) 2015-04-22

Family

ID=52852432

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510043037.7A Pending CN104536902A (en) 2015-01-28 2015-01-28 Performance optimization method for IO (Input/Output) subsystem by testing server

Country Status (1)

Country Link
CN (1) CN104536902A (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104978250A (en) * 2015-06-23 2015-10-14 浪潮电子信息产业股份有限公司 Iozone-based HBA card performance test method
CN105138331A (en) * 2015-08-26 2015-12-09 浪潮电子信息产业股份有限公司 A method of distinguishing local disk and USB mobile disk based on LINUX system
US9886670B2 (en) 2014-06-30 2018-02-06 Amazon Technologies, Inc. Feature processing recipes for machine learning
US10102480B2 (en) 2014-06-30 2018-10-16 Amazon Technologies, Inc. Machine learning service
US10257275B1 (en) 2015-10-26 2019-04-09 Amazon Technologies, Inc. Tuning software execution environments using Bayesian models
US10318882B2 (en) 2014-09-11 2019-06-11 Amazon Technologies, Inc. Optimized training of linear machine learning models
US10452992B2 (en) 2014-06-30 2019-10-22 Amazon Technologies, Inc. Interactive interfaces for machine learning model evaluations
US10540606B2 (en) 2014-06-30 2020-01-21 Amazon Technologies, Inc. Consistent filtering of machine learning data
US10963810B2 (en) 2014-06-30 2021-03-30 Amazon Technologies, Inc. Efficient duplicate detection for machine learning data sets
US11100420B2 (en) 2014-06-30 2021-08-24 Amazon Technologies, Inc. Input processing for machine learning
US11182691B1 (en) 2014-08-14 2021-11-23 Amazon Technologies, Inc. Category-based sampling of machine learning data

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6434642B1 (en) * 1999-10-07 2002-08-13 Xilinx, Inc. FIFO memory system and method with improved determination of full and empty conditions and amount of data stored
CN101819547A (en) * 2010-03-25 2010-09-01 浪潮电子信息产业股份有限公司 Method for testing stability and reliability of storage subsystem
CN201698255U (en) * 2010-05-14 2011-01-05 上海浙大网新易得科技发展有限公司 Server capable of accessing disc at high speed
CN103268204A (en) * 2013-06-08 2013-08-28 北京百度网讯科技有限公司 Method and device for tuning solid state disk

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6434642B1 (en) * 1999-10-07 2002-08-13 Xilinx, Inc. FIFO memory system and method with improved determination of full and empty conditions and amount of data stored
CN101819547A (en) * 2010-03-25 2010-09-01 浪潮电子信息产业股份有限公司 Method for testing stability and reliability of storage subsystem
CN201698255U (en) * 2010-05-14 2011-01-05 上海浙大网新易得科技发展有限公司 Server capable of accessing disc at high speed
CN103268204A (en) * 2013-06-08 2013-08-28 北京百度网讯科技有限公司 Method and device for tuning solid state disk

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
何勇,陈晓峰: "《Greenplum企业应用实战》", 31 October 2014 *
杨鹏: "《Linux 服务器架设》", 31 December 2008 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10452992B2 (en) 2014-06-30 2019-10-22 Amazon Technologies, Inc. Interactive interfaces for machine learning model evaluations
US11100420B2 (en) 2014-06-30 2021-08-24 Amazon Technologies, Inc. Input processing for machine learning
US9886670B2 (en) 2014-06-30 2018-02-06 Amazon Technologies, Inc. Feature processing recipes for machine learning
US10102480B2 (en) 2014-06-30 2018-10-16 Amazon Technologies, Inc. Machine learning service
US12229642B2 (en) 2014-06-30 2025-02-18 Amazon Technologies, Inc. Efficient duplicate detection for machine learning data sets
US10540606B2 (en) 2014-06-30 2020-01-21 Amazon Technologies, Inc. Consistent filtering of machine learning data
US10963810B2 (en) 2014-06-30 2021-03-30 Amazon Technologies, Inc. Efficient duplicate detection for machine learning data sets
US12073298B2 (en) 2014-06-30 2024-08-27 Amazon Technologies, Inc. Machine learning service
US11544623B2 (en) 2014-06-30 2023-01-03 Amazon Technologies, Inc. Consistent filtering of machine learning data
US11182691B1 (en) 2014-08-14 2021-11-23 Amazon Technologies, Inc. Category-based sampling of machine learning data
US10318882B2 (en) 2014-09-11 2019-06-11 Amazon Technologies, Inc. Optimized training of linear machine learning models
CN104978250A (en) * 2015-06-23 2015-10-14 浪潮电子信息产业股份有限公司 Iozone-based HBA card performance test method
CN105138331A (en) * 2015-08-26 2015-12-09 浪潮电子信息产业股份有限公司 A method of distinguishing local disk and USB mobile disk based on LINUX system
US10257275B1 (en) 2015-10-26 2019-04-09 Amazon Technologies, Inc. Tuning software execution environments using Bayesian models

Similar Documents

Publication Publication Date Title
CN104536902A (en) Performance optimization method for IO (Input/Output) subsystem by testing server
US10642524B1 (en) Upgrading a write buffer in a storage system that includes a plurality of storage devices and a plurality of write buffer devices
US9128855B1 (en) Flash cache partitioning
US9569118B2 (en) Promoting consistent response times in a data storage system having multiple data retrieval mechanisms
US12405741B2 (en) Dynamic data relocation using cloud based ranks
US9405625B2 (en) Optimizing and enhancing performance for parity based storage
US11086535B2 (en) Thin provisioning using cloud based ranks
US8578092B2 (en) Method and apparatus to support determining storage area unit size
US20160034394A1 (en) Methods and systems for using predictive cache statistics in a storage system
US20090217067A1 (en) Systems and Methods for Reducing Power Consumption in a Redundant Storage Array
CN108292196A (en) Write data into the storage system of the storage device of storage device and Second Type including the first kind
KR102478392B1 (en) System and method for identifying ssds with lowest tail latencies
US20160188212A1 (en) Data storage system selectively employing multiple data compression techniques
CN110389710A (en) The method and apparatus for distributing storage resource
US20240134696A1 (en) Offloading Data Storage Device Processing Tasks to a Graphics Processing Unit
US8949528B2 (en) Writing of data of a first block size in a raid array that stores and mirrors data in a second block size
WO2016190893A1 (en) Storage management
US11461250B2 (en) Tuning data storage equipment based on comparing observed I/O statistics with expected I/O statistics which are defined by operating settings that control operation
JP2008065433A (en) Computer system and performance tuning method
CN104484135A (en) Method and device for quickly reading data
TW201445429A (en) Disk array system and data processing method
CN115729470A (en) Data writing method and related equipment
US20210240630A1 (en) Adapting Cache Processing Using Phase Libraries and Real Time Simulators
US7725654B2 (en) Affecting a caching algorithm used by a cache of storage system
Hollowell et al. The effect of flashcache and bcache on I/O performance

Legal Events

Date Code Title Description
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20150422

WD01 Invention patent application deemed withdrawn after publication