CN104536902A - Performance optimization method for IO (Input/Output) subsystem by testing server - Google Patents
Performance optimization method for IO (Input/Output) subsystem by testing server Download PDFInfo
- Publication number
- CN104536902A CN104536902A CN201510043037.7A CN201510043037A CN104536902A CN 104536902 A CN104536902 A CN 104536902A CN 201510043037 A CN201510043037 A CN 201510043037A CN 104536902 A CN104536902 A CN 104536902A
- Authority
- CN
- China
- Prior art keywords
- testing server
- server
- subsystem
- performance
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
Description
技术领域 technical field
本发明涉及服务器IO子系统,具体的说就是一种测试服务器IO子系统的性能调优方法。 The invention relates to a server IO subsystem, in particular to a performance tuning method for testing the server IO subsystem.
背景技术 Background technique
随着互联网发展越来越迅速,流媒体大量应用,有代表性的优酷、土豆等为网民提供视频服务。与此对应要求服务器具有极高的读写吞吐性能。客户在选用设备时会进行性能测试比较,性能高的产品占据优势。 With the rapid development of the Internet, a large number of streaming media applications, representative Youku, Tudou, etc. provide video services for Internet users. Correspondingly, the server is required to have extremely high read and write throughput performance. Customers will conduct performance tests and comparisons when selecting equipment, and products with high performance will take advantage.
通常Linux操作系统默认对用户的IO操作不做判断,按照同一种方式进行安排,当服务器压力逐渐增大时,服务器IO性能方面的表现逐渐下降,突出表现为IO带宽增长放缓出现瓶颈,系统的IO延时增长,拖慢整体应用程序的进度。顺序读写时,IO操作具有一定的预知性,当某一读写操作执行完成时,系统进行预读磁盘连续位置的数据,命中的情况下会提升读写效率。 Usually, the Linux operating system does not judge the user's IO operation by default, and arranges it in the same way. When the pressure on the server gradually increases, the performance of the server's IO performance gradually declines. The IO latency increases, slowing down the progress of the overall application. When reading and writing sequentially, the IO operation has a certain degree of predictability. When a certain read and write operation is completed, the system will pre-read the data in the continuous position of the disk. If there is a hit, the read and write efficiency will be improved.
操作系统Linux下的IO操作有默认的调度算法,不同应用程序产生不同类型的IO操作。以读写测试工具Iozone为代表的IO测试工具,在测试时依据测试进程产生IO操作,通过相同进程可能产生具有相同的异步读写方式,此时具有测试读写操作的IO调度测试数值会有提升。 The IO operation under the operating system Linux has a default scheduling algorithm, and different applications generate different types of IO operations. The IO test tool represented by the read-write test tool Iozone generates IO operations according to the test process during the test. The same process may generate the same asynchronous read-write mode. At this time, the IO scheduling test value with test read-write operations will have promote.
发明内容 Contents of the invention
本发明的目的是提供一种针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,通过该方法指导厂商、用户对现有设备进行调优,实现设备IO最大化。 The purpose of the present invention is to provide a performance tuning method for the IO subsystem of the read-write test tool Iozone test server, through which the manufacturer and the user are guided to tune the existing equipment to maximize the IO of the equipment.
本发明所述一种测试服务器IO子系统的性能调优方法,解决上述技术问题采用的技术方案如下:该方法是针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,需要一台服务器作为测试服务器,并在测试服务器上搭配SAS盘和数据盘,根据自身应用多样性使用脚本来自定义读写调度方式;通过在测试服务器上安装操作系统,正常登录系统后,编写调优策略,并保存所述调优策略;最后在测试之前,运行所述调优策略。 A kind of performance tuning method of test server IO subsystem described in the present invention, the technical scheme that solves the above-mentioned technical problem adopts is as follows: this method is the performance tuning method for Iozone test server IO subsystem of reading and writing test tool, needs a The server is used as a test server, and the test server is equipped with SAS disks and data disks, and scripts are used to customize the read and write scheduling methods according to the diversity of its own applications; by installing the operating system on the test server and logging in to the system normally, write the tuning strategy, And save the tuning strategy; finally, run the tuning strategy before testing.
优选的,所述测试服务器采用机架式服务器,并搭配2块SAS盘,且做raid1装操作系统,同时搭配12块数据盘,均做raid0装操作系统。 Preferably, the test server adopts a rack server, and is equipped with 2 SAS disks, and is configured as raid1 to install the operating system, and is configured with 12 data disks, and is configured as raid0 to install the operating system.
优选的,在测试服务器上安装操作系统CentOS6.4,并默认正常安装。 Preferably, the operating system CentOS6.4 is installed on the test server, and it is installed normally by default.
优选的,所述测试服务器采用NF5270M3,搭配的SAS盘为2块300G 2.5寸SAS盘,搭配的数据盘为12块2T 3.5寸盘。 Preferably, the test server adopts NF5270M3, and the matching SAS disks are two 300G 2.5-inch SAS disks, and the matching data disks are twelve 2T 3.5-inch disks.
本发明的一种测试服务器IO子系统的性能调优方法与现有技术相比具有的有益效果是:该方法采用测试设备,重新编写调优策略,通过脚本实现对算法、参数的选择设定,能够有效提升磁盘连续读写等方式的性能,提高硬件的利用率;通过该方法,可以对现有服务器的磁盘性能在Iozone测试下进行一定量提升,对硬件厂商、最终用户都有指导意义。 Compared with the prior art, the performance tuning method of a test server IO subsystem of the present invention has the following beneficial effects: the method uses test equipment, rewrites the tuning strategy, and realizes the selection and setting of algorithms and parameters through scripts , which can effectively improve the performance of disk continuous reading and writing, and improve the utilization rate of hardware; through this method, the disk performance of existing servers can be improved to a certain extent under the Iozone test, which has guiding significance for hardware manufacturers and end users .
具体实施方式 Detailed ways
为使本发明的目的、技术方案和优点更加清楚明白,以下结合具体实施例,对本发明所述一种测试服务器IO子系统的性能调优方法进一步详细说明。 In order to make the purpose, technical solution and advantages of the present invention clearer, the performance tuning method of the IO subsystem of the test server described in the present invention will be further described in detail below in conjunction with specific embodiments.
本发明所述一种测试服务器IO子系统的性能调优方法,是针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,根据自身应用多样性使用脚本来自定义读写调度方式,具备灵活性、实用性及可操作。 The performance tuning method of a test server IO subsystem described in the present invention is aimed at the performance tuning method of the read-write test tool Iozone test server IO subsystem, and uses scripts to customize the read-write scheduling method according to its own application diversity, and has Flexibility, practicality and operability.
本发明所述性能调优方法需要一台服务器作为测试服务器,并在测试服务器上搭配SAS盘和数据盘;然后在测试服务器上安装操作系统,正常登录系统后,编写调优策略,并保存所述调优策略;最后在测试之前,运行所述调优策略。 The performance tuning method described in the present invention needs a server as a test server, and a SAS disk and a data disk are matched on the test server; then an operating system is installed on the test server, and after a normal login to the system, an optimization strategy is written and saved. The tuning strategy described above; finally, before testing, the tuning strategy is run.
实施例: Example:
本实施例所述一种测试服务器IO子系统的性能调优方法,采用机架式服务器作为测试服务器,并搭配2块SAS盘,且做raid1装操作系统,同时搭配12块数据盘,均做raid0装操作系统;本实施例所述性能调优方法在测试服务器上安装操作系统CentOS6.4,并默认正常安装。 A performance tuning method for the test server IO subsystem described in this embodiment uses a rack-mounted server as the test server, and is equipped with 2 SAS disks, and is configured as raid1 to install the operating system, and is equipped with 12 data disks at the same time. The raid0 installs the operating system; the performance tuning method described in this embodiment installs the operating system CentOS6.4 on the test server, and it is installed normally by default.
本实施例所述测试服务器IO子系统的性能调优方法,其具体实施过程为:在测试服务器正常登录系统后,编写调优策略: The performance tuning method of the test server IO subsystem described in this embodiment, its specific implementation process is: after the test server logs in to the system normally, write the tuning strategy:
for i in {b..m}for i in {b..m}
dodo
echo anticipatory >/sys/block/sd$i/queue/schedulerecho anticipatory >/sys/block/sd$i/queue/scheduler
echo 15000 >/sys/block/sd$i/queue/iosched/read_expireecho 15000 >/sys/block/sd$i/queue/iosched/read_expire
echo 30000 >/sys/block/sd$i/queue/iosched/write_expireecho 30000 >/sys/block/sd$i/queue/iosched/write_expire
echo 4096 >/sys/block/sd$i/queue/nr_requestsecho 4096 >/sys/block/sd$i/queue/nr_requests
echo 5 >/sys/block/sd$i/device/queue_depthecho 5 >/sys/block/sd$i/device/queue_depth
blockdev --setra 512 /dev/sd$iblockdev --setra 512 /dev/sd$i
donedone
并保存上述调优策略。 And save the above tuning strategy.
下面对本实施例所述方法的上述调优策略进行详细解释: The above-mentioned tuning strategy of the method described in this embodiment is explained in detail below:
1) 第1-2行:在sdb-sdm12块盘上(默认测试环境12块数据盘)作如下操作; 1) Lines 1-2: Do the following operations on sdb-sdm12 disks (the default test environment is 12 data disks);
2) 第3行:默认系统IO策略是cfq(完全公平队列),而针对流媒体应用类的读写,连续读写概率大,改为anticipatory(预测性策略)将会预读连续扇区,增大读写效率; 2) Line 3: The default system IO policy is cfq (completely fair queue), and for streaming media applications, the probability of continuous reading and writing is high, changing to anticipatory (predictive strategy) will pre-read continuous sectors, Increase reading and writing efficiency;
3) 第4-5行:读写时间片超时设定,一般 read/write=1/2搭配出现,设置值大可以有效预防读写超时; 3) Lines 4-5: read and write time slice timeout setting, generally read/write=1/2 appear together, a large setting value can effectively prevent read and write timeout;
4) 第6-8行:将请求合并值、队列深度值按照对应值修改; 4) Lines 6-8: Modify the request merge value and queue depth value according to the corresponding value;
5) 第9行:结束。 5) Line 9: end.
通过本实施例所述测试服务器IO子系统的性能调优方法,在测试之前,运行上述调优策略。以下是该方法的效果验证,通过更改策略前后三次Iozone测试值比较(单位KB/s): Through the performance tuning method of the test server IO subsystem described in this embodiment, the above tuning strategy is run before the test. The following is the effect verification of this method, through the comparison of three Iozone test values before and after changing the strategy (in KB/s):
更改前: Before change:
Initial write " 1812105.56 " Initial write " 1812105.56 "
Read " 1554452.91 " Read " 1554452.91 "
Initial write " 1816866.94 " Initial write " 1816866.94 "
Read " 1544720.16 " Read " 1544720.16 "
Initial write " 1813688.34 " Initial write " 1813688.34 "
Read " 1537983.47 Read " 1537983.47
更改后: After the change:
Initial write " 1887822.60 " Initial write " 1887822.60 "
Read " 1595224.81 " Read " 1595224.81 "
Initial write " 1893080.30“ Initial write " 1893080.30"
Read " 1594529.09 " Read " 1594529.09 "
Initial write " 1882738.75 " Initial write " 1882738.75 "
Read " 1591268.60“ Read "1591268.60"
对比以上测试结果,调整后策略写最高提升:4.47%;读最高提升:3.72%,且数值较稳定。以上调优值均是针对流媒体测试,经过多次修改反复试验所得,若更改测试模型,可做相应修改,亦可增加读写数值。 Comparing the above test results, after the adjustment, the highest increase in writing strategy: 4.47%; the highest increase in reading: 3.72%, and the value is relatively stable. The above tuning values are all for streaming media testing. After many modifications and repeated tests, if you change the test model, you can make corresponding modifications, and you can also increase the read and write values.
除本实施例所述测试服务器IO子系统的性能调优方法,所采用的测试设备外,本发明所述方法还可以采用测试设备:测试服务器为NF5270M3(浪潮2路机架式服务器),搭配的SAS盘为2块300G 2.5寸SAS盘,搭配的数据盘为12块2T 3.5寸大盘。 In addition to the performance tuning method of the test server IO subsystem described in this embodiment and the test equipment used, the method of the present invention can also use test equipment: the test server is NF5270M3 (Inspur 2-way rack server), with The SAS disks are two 300G 2.5-inch SAS disks, and the matching data disks are twelve 2T 3.5-inch large disks.
综上所述,本发明提供了一种针对读写测试工具Iozone测试服务器IO子系统的性能调优方法,通过脚本实现对算法、参数的选择设定,可以有效提升磁盘连续读写等方式的性能,提高硬件的利用率。 In summary, the present invention provides a performance tuning method for the IO subsystem of the read-write test tool Iozone test server. The selection and setting of algorithms and parameters can be realized through scripts, which can effectively improve the performance of continuous disk read-write and other methods. performance and improve hardware utilization.
上述具体实施方式仅是本发明的具体个案,本发明的专利保护范围包括但不限于上述具体实施方式,任何符合本发明的权利要求书的且任何所属技术领域的普通技术人员对其所做的适当变化或替换,皆应落入本发明的专利保护范围。 The above-mentioned specific embodiments are only specific cases of the present invention, and the scope of patent protection of the present invention includes but is not limited to the above-mentioned specific embodiments, any claims that meet the claims of the present invention and any ordinary skilled person in the technical field. Appropriate changes or substitutions should fall within the scope of patent protection of the present invention.
Claims (4)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510043037.7A CN104536902A (en) | 2015-01-28 | 2015-01-28 | Performance optimization method for IO (Input/Output) subsystem by testing server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510043037.7A CN104536902A (en) | 2015-01-28 | 2015-01-28 | Performance optimization method for IO (Input/Output) subsystem by testing server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104536902A true CN104536902A (en) | 2015-04-22 |
Family
ID=52852432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510043037.7A Pending CN104536902A (en) | 2015-01-28 | 2015-01-28 | Performance optimization method for IO (Input/Output) subsystem by testing server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104536902A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104978250A (en) * | 2015-06-23 | 2015-10-14 | 浪潮电子信息产业股份有限公司 | Iozone-based HBA card performance test method |
| CN105138331A (en) * | 2015-08-26 | 2015-12-09 | 浪潮电子信息产业股份有限公司 | A method of distinguishing local disk and USB mobile disk based on LINUX system |
| US9886670B2 (en) | 2014-06-30 | 2018-02-06 | Amazon Technologies, Inc. | Feature processing recipes for machine learning |
| US10102480B2 (en) | 2014-06-30 | 2018-10-16 | Amazon Technologies, Inc. | Machine learning service |
| US10257275B1 (en) | 2015-10-26 | 2019-04-09 | Amazon Technologies, Inc. | Tuning software execution environments using Bayesian models |
| US10318882B2 (en) | 2014-09-11 | 2019-06-11 | Amazon Technologies, Inc. | Optimized training of linear machine learning models |
| US10452992B2 (en) | 2014-06-30 | 2019-10-22 | Amazon Technologies, Inc. | Interactive interfaces for machine learning model evaluations |
| US10540606B2 (en) | 2014-06-30 | 2020-01-21 | Amazon Technologies, Inc. | Consistent filtering of machine learning data |
| US10963810B2 (en) | 2014-06-30 | 2021-03-30 | Amazon Technologies, Inc. | Efficient duplicate detection for machine learning data sets |
| US11100420B2 (en) | 2014-06-30 | 2021-08-24 | Amazon Technologies, Inc. | Input processing for machine learning |
| US11182691B1 (en) | 2014-08-14 | 2021-11-23 | Amazon Technologies, Inc. | Category-based sampling of machine learning data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6434642B1 (en) * | 1999-10-07 | 2002-08-13 | Xilinx, Inc. | FIFO memory system and method with improved determination of full and empty conditions and amount of data stored |
| CN101819547A (en) * | 2010-03-25 | 2010-09-01 | 浪潮电子信息产业股份有限公司 | Method for testing stability and reliability of storage subsystem |
| CN201698255U (en) * | 2010-05-14 | 2011-01-05 | 上海浙大网新易得科技发展有限公司 | Server capable of accessing disc at high speed |
| CN103268204A (en) * | 2013-06-08 | 2013-08-28 | 北京百度网讯科技有限公司 | Method and device for tuning solid state disk |
-
2015
- 2015-01-28 CN CN201510043037.7A patent/CN104536902A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6434642B1 (en) * | 1999-10-07 | 2002-08-13 | Xilinx, Inc. | FIFO memory system and method with improved determination of full and empty conditions and amount of data stored |
| CN101819547A (en) * | 2010-03-25 | 2010-09-01 | 浪潮电子信息产业股份有限公司 | Method for testing stability and reliability of storage subsystem |
| CN201698255U (en) * | 2010-05-14 | 2011-01-05 | 上海浙大网新易得科技发展有限公司 | Server capable of accessing disc at high speed |
| CN103268204A (en) * | 2013-06-08 | 2013-08-28 | 北京百度网讯科技有限公司 | Method and device for tuning solid state disk |
Non-Patent Citations (2)
| Title |
|---|
| 何勇,陈晓峰: "《Greenplum企业应用实战》", 31 October 2014 * |
| 杨鹏: "《Linux 服务器架设》", 31 December 2008 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10452992B2 (en) | 2014-06-30 | 2019-10-22 | Amazon Technologies, Inc. | Interactive interfaces for machine learning model evaluations |
| US11100420B2 (en) | 2014-06-30 | 2021-08-24 | Amazon Technologies, Inc. | Input processing for machine learning |
| US9886670B2 (en) | 2014-06-30 | 2018-02-06 | Amazon Technologies, Inc. | Feature processing recipes for machine learning |
| US10102480B2 (en) | 2014-06-30 | 2018-10-16 | Amazon Technologies, Inc. | Machine learning service |
| US12229642B2 (en) | 2014-06-30 | 2025-02-18 | Amazon Technologies, Inc. | Efficient duplicate detection for machine learning data sets |
| US10540606B2 (en) | 2014-06-30 | 2020-01-21 | Amazon Technologies, Inc. | Consistent filtering of machine learning data |
| US10963810B2 (en) | 2014-06-30 | 2021-03-30 | Amazon Technologies, Inc. | Efficient duplicate detection for machine learning data sets |
| US12073298B2 (en) | 2014-06-30 | 2024-08-27 | Amazon Technologies, Inc. | Machine learning service |
| US11544623B2 (en) | 2014-06-30 | 2023-01-03 | Amazon Technologies, Inc. | Consistent filtering of machine learning data |
| US11182691B1 (en) | 2014-08-14 | 2021-11-23 | Amazon Technologies, Inc. | Category-based sampling of machine learning data |
| US10318882B2 (en) | 2014-09-11 | 2019-06-11 | Amazon Technologies, Inc. | Optimized training of linear machine learning models |
| CN104978250A (en) * | 2015-06-23 | 2015-10-14 | 浪潮电子信息产业股份有限公司 | Iozone-based HBA card performance test method |
| CN105138331A (en) * | 2015-08-26 | 2015-12-09 | 浪潮电子信息产业股份有限公司 | A method of distinguishing local disk and USB mobile disk based on LINUX system |
| US10257275B1 (en) | 2015-10-26 | 2019-04-09 | Amazon Technologies, Inc. | Tuning software execution environments using Bayesian models |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104536902A (en) | Performance optimization method for IO (Input/Output) subsystem by testing server | |
| US10642524B1 (en) | Upgrading a write buffer in a storage system that includes a plurality of storage devices and a plurality of write buffer devices | |
| US9128855B1 (en) | Flash cache partitioning | |
| US9569118B2 (en) | Promoting consistent response times in a data storage system having multiple data retrieval mechanisms | |
| US12405741B2 (en) | Dynamic data relocation using cloud based ranks | |
| US9405625B2 (en) | Optimizing and enhancing performance for parity based storage | |
| US11086535B2 (en) | Thin provisioning using cloud based ranks | |
| US8578092B2 (en) | Method and apparatus to support determining storage area unit size | |
| US20160034394A1 (en) | Methods and systems for using predictive cache statistics in a storage system | |
| US20090217067A1 (en) | Systems and Methods for Reducing Power Consumption in a Redundant Storage Array | |
| CN108292196A (en) | Write data into the storage system of the storage device of storage device and Second Type including the first kind | |
| KR102478392B1 (en) | System and method for identifying ssds with lowest tail latencies | |
| US20160188212A1 (en) | Data storage system selectively employing multiple data compression techniques | |
| CN110389710A (en) | The method and apparatus for distributing storage resource | |
| US20240134696A1 (en) | Offloading Data Storage Device Processing Tasks to a Graphics Processing Unit | |
| US8949528B2 (en) | Writing of data of a first block size in a raid array that stores and mirrors data in a second block size | |
| WO2016190893A1 (en) | Storage management | |
| US11461250B2 (en) | Tuning data storage equipment based on comparing observed I/O statistics with expected I/O statistics which are defined by operating settings that control operation | |
| JP2008065433A (en) | Computer system and performance tuning method | |
| CN104484135A (en) | Method and device for quickly reading data | |
| TW201445429A (en) | Disk array system and data processing method | |
| CN115729470A (en) | Data writing method and related equipment | |
| US20210240630A1 (en) | Adapting Cache Processing Using Phase Libraries and Real Time Simulators | |
| US7725654B2 (en) | Affecting a caching algorithm used by a cache of storage system | |
| Hollowell et al. | The effect of flashcache and bcache on I/O performance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150422 |
|
| WD01 | Invention patent application deemed withdrawn after publication |