TWI587152B - Method for extending life expectancy of disks in cloud-based service system and system using the same - Google Patents

Description

The party used to extend the disk life expectancy value in the cloud service system Method and system using the same

The present invention relates to a method for extending a disk life expectancy value and a system using the same, and more particularly to a method for extending a disk life expectancy value in a cloud service system and a system using the same.

The disk in the computer is the main device used to store data and supply the application. Regardless of the type, such as a hard drive, a solid state drive, or even a tape, after a long period of use, the disk will eventually fail and will not work. If the data backup or archive is not properly performed before the failure, the data on the disk will be lost because it may contain important data, such as the operating system and computer system configuration data on the disk, which will cause disaster. Often, the disk will show some signs before the failure. For example, the stored data disappears or the program runs frequently. Users can easily detect these signs and take action to replace the disk and store the data in it. Because the computer may have only a few disks, and the user can continuously observe the disk every day through the performance of the computer, this is feasible.

For the architecture that operates the cloud service system, it will encounter the same problem as the aforementioned disk. However, the more complicated situation is that the architecture usually contains a large number of disks for data access. Because of the nature and content of the stored data, a disk may be accessed more often than other disks. Frequent disk access is an important factor in shortening disk life values. However, it is very difficult to observe the physical properties of each disk frequently and continuously. For cloud service system administrators, it is not a cost-effective way to perform data backups and replace failed disks. Therefore, some techniques for monitoring cluster disks and predicting disk life values have been disclosed to provide a solution. For example, US Patent Application No. US2016232450 proposes a storage device life monitoring system and a storage device life monitoring method thereof. The method comprises the steps of: collecting operational behavior information corresponding to the storage devices; storing a plurality of training materials having operational behavior information and corresponding operational life values; and constructing storage device life predictions based on the operational behavior information and corresponding operational lifetime values a model; inputting operational behavior information of the storage devices to a storage device life prediction model to generate an estimated life value corresponding to each storage device; and re-architecting the storage according to the operational behavior information and the estimated life value of each storage device Device life prediction model. Thereby, the method can accurately predict the life of the storage device.

The aforementioned patent application uses data from a log, such as a system log, an application log, or a database log (operational behavior information) for training and predicting life. Although the information in the log may not inform the actual situation of the disk, it may be suggested by the record that some disk health status is obtained. It is because the abnormal value in the record is related to the real life value of the corresponding disk, and the historical data can be effectively used for prediction. If the method can borrow the contents of the log to accurately find the lifetime value for all the disks, for a specific model of the disk, based on the same manufacturing process and quality requirements, the real life value should be within a certain range, for example , use 4,000 to 5,000 hours. However, in fact, some disks of the same model can only work for a short period of time, some work for a long time, and the lifetime value of most disks falls within the predicted range. Even though two disks have similar operational behavior information, they may not have the same lifetime value. This means that some key factors are missing from the analysis.

For two disks with similar logs but different lifetime values, if you view certain performance data, such as IOPS (Input/Output Per Second), delay time, and throughput ( Throughput, or related information, such as Central Processing Unit (CPU) load or host memory usage, can be found that the two disks operate differently, and this difference may be the cause of different lifetime values. For example, two disks have similar access and failure records within one year, one being intensively accessed within three months and the other being accessed equally within a year. Therefore, in order to provide a prediction method for providing a more accurate life of the disk in the cloud service system, the input/output mode can be further analyzed to extend the life expectancy value of the disk.

This paragraph of text extracts and compiles certain features of the present invention. Other features will be revealed in subsequent paragraphs. The intention is to cover various modifications and similar arrangements in the spirit and scope of the appended claims.

The present invention proposes a method for extending a disk life expectancy value in a cloud service system and a system using the same. According to an aspect of the invention, the method comprises the steps of: A. collecting performance data from each disk in a cloud service system from historical data; B. filtering performance data, the performance data is from a faulty magnetic field. The disk or a disk has a lifetime value shorter than a preset value; C. group the disks according to the performance value level according to the life value level; D. normalize the performance data and the lifetime value to a unitless performance value And a unitless life value; E. Perform an LSTM (Long Short Term Memory) modeling algorithm for the unitless performance value of each disk in each group to obtain a unitless unit for each disk in a future period of time. The predicted trend of the performance value; F. based on the predicted trend of the unitless performance values in the group, respectively assign a specific unitless performance value to all the disks in each group; G. Perform k-average cluster (k- Means clustering), the input set is used to obtain an output set, wherein each input set represents a corresponding disk and includes a specific unitless performance value and a unitless life value; H. denormalize each output set to Get a performance limit and a goal respectively Command value; I. These and forming a disk storage device that is provided with a performance limit, so that each disk has in the next period of time not shorter lifetime expected life value to the target value. The method may further comprise a step J after step I: J. configuring one for each storage device A workload that has a performance requirement that matches or falls below that performance limit.

According to the present invention, the performance data may be delay time, throughput, central processing unit (CPU) load, memory usage, or IOPS (Input/Output Per Second) input/output operations per second. frequency). The unitless performance value may be calculated by dividing a performance data value from a first difference value among the smallest of all performance data values by a second difference value between the largest and smallest of all performance data values. The unitless life value may be calculated by dividing a third difference value between a lifetime value and the largest of all life values by a fourth difference value between the largest and the smallest of all life values. The disk can be a Hard Disk Drive (HDD) or a Solid State Disk (SSD). These life value levels can be evenly distributed over the range between the largest and the smallest of all life values. The particular unitless performance value may be derived by averaging the predicted unitless performance values of the group for the period of time in the future. This historical data can come from system logs, application logs, database logs, or S.M.A.R.T. (Self-Monitoring Analysis and Reporting Technology) logs. The central set of each cluster in step G can be selected as the output set.

Another aspect of the present invention is a cloud service system. The system includes: a host for operating a workload and performing data access; a plurality of disks connected to the host for storing data for workload access; and an expected life extension module, configured or installed For the host, used for historical data Collecting performance data for each disk; filtering performance data from a faulty disk or a disk having a lifetime value shorter than a preset value; grouping the corresponding performance data according to the life value level The disks; normalize the performance data and lifetime values to a unitless performance value and a unitless lifetime value; perform an LSTM modeling algorithm on the unitless performance value of each disk in each group to obtain The predicted trend of the unitless performance value of each disk in a future period of time; based on the predicted trend of the unitless performance values in the group, each of the disks in each group is assigned a specific unitless performance value; Performing a k-average clustering algorithm to obtain an output set with an input set, wherein each input set represents a corresponding disk and includes a specific unitless performance value and a unitless lifetime value; denormalizing each output set by Obtaining a performance limit and a target lifetime value respectively; and setting a performance limit for the disks forming the storage device, so that each disk has a expected life value not shorter than the target in the future time period Life values. The life extension extension module can be further configured to configure a workload for each storage device, the workload having a performance requirement that matches or falls below the performance limit.

In accordance with the present invention, the performance data can be latency, throughput, CPU load, memory usage, or IOPS. The unitless performance value may be calculated by dividing a performance data value from a first difference value among the smallest of all performance data values by a second difference value between the largest and smallest of all performance data values. The unitless life value may be calculated by dividing a third difference value between a lifetime value and the largest of all life values by a fourth difference value between the largest and the smallest of all life values. Where the disk can be a hard disk or a solid Hard drive. These life value levels can be evenly distributed over the range between the largest and the smallest of all life values. The particular unitless performance value may be derived by averaging the predicted unitless performance values of the group for the period of time in the future. This historical data can come from system logs, application logs, database logs, or S.M.A.R.T. logs. The central set of each cluster from the k-average clustering algorithm can be selected as the output set.

The present invention uses LSTM modeling and k-average clustering algorithms to find performance limits and target lifetime values so that a cluster of disks can be designated for a particular workload running on the cloud service system. The disk can be predicted for its minimum lifetime and can meet the workload's needs. In addition, the minimum life value is the longest life that the disk can continue to operate.

10‧‧‧Cloud Service System

100‧‧‧Host

101‧‧‧Central Processing Unit

102‧‧‧ memory

103‧‧‧Life expectancy extension module

201~230‧‧‧Disk

1 is a schematic diagram of a cloud storage device system in accordance with the present invention.

2 is a flow chart of a method for extending a disk life expectancy value in a cloud service system in accordance with the present invention.

Figure 3 shows a form showing the status of the disk.

Figure 4 is a form showing the results of the grouping.

Figure 5 shows a graph of the unitless performance values collected in the past half in the upper half and a graph showing the predicted trend of the unitless performance values for the next 24 hours in the lower half.

Figure 6 shows the IOPS prediction trend for the three disks in a high group.

Figure 7 is a chart showing the distribution of input sets.

The invention will be more specifically described by reference to the following embodiments.

An ideal architecture for implementing the present invention is shown in Figure 1. A cloud service system 10 includes a host 100 and 30 disks (the disks are numbered sequentially from 201 to 230 for the following comprehensive description). The host 100 can be a server that is used to operate the workload and perform data access for the workload. The host 100 receives from a client device such as a personal computer, a tablet computer, a smart phone, and the like through an internet, a local area network (LAN), or a wide area network (WAN). The hardware of the requirements of the remote device. Disks 201 through 230 are connected to the host 100 for storing data for workload access. Although the number of disks in this embodiment is 30, this is not meant to limit the application of the present invention. In fact, the cloud services system 10 can have any number of disks as long as the workload is met. The disk can be a Hard Disk Drive (HDD) or a Solid State Disk (SSD). In the present embodiment, the disks 201 to 230 are all solid state hard disks. For the application of the invention, the type of the disk should be consistent. Preferably, the models of the disks are the same (from the same manufacturer and have a consistent architecture). Thereby, a unified comparison based on the same situation can be performed. The results from this offering method can be more precise.

The host 100 has a number of key components: a central processing unit (CPU) 101, a memory 102, and a life expectancy extension module 103. The central processing unit 101 is responsible for the operation of the host 100. The memory 102 can be a static random access memory (SRAM) or a dynamic random access memory (DRAM) for temporarily storing data or programs to run the cloud service system 10. The life expectancy extension module 103 is a device that implements the method provided by the present invention and is configured into the host 100. The main function of the life expectancy extension module 103 is to collect performance data for each disk from the historical data; filter performance data from a faulty disk or a disk having a lifetime value shorter than a preset Values; according to the life value level, group the disks with corresponding performance data; normalize the performance data and lifetime values to a unitless performance value and a unitless lifetime value; for each disk in each group Performing an LSTM (Long Short Term Memory) modeling algorithm without unit performance value to obtain a predicted trend of the unitless performance value of each disk in a future period of time; based on the predicted trend of the unitless performance values in the group Specifying a specific unitless performance value for each disk in each group; performing a k-average cluster algorithm to obtain an output set from the input set, wherein each input set represents a corresponding disk and contains a specific No unit performance value and a unitless life value; denormalize each output set to obtain a performance limit and a target lifetime value respectively; and set a performance limit for the disks forming the storage device So that each disk has in the next period of time shorter than the expected life value target lifetime value. In addition, expected The life extension module 103 can further configure a workload for each storage device that has a performance requirement that matches or falls below the performance limit. These functions will be described in detail later in conjunction with the method of the present invention. It should be noted that in other embodiments, the life expectancy extension module may be installed in the host 100 (stored in the memory 102 and operated by the central processing unit 101) in a software type. In still other embodiments, the life expectancy extension module can be a standalone device and configured into the host 100 in parallel.

Please refer to FIG. 2, which is a flow chart of a method for extending the disk life expectancy value in the cloud service system 10 in accordance with the present invention. The first step of the method is to collect performance data (S01) from each disk in the cloud service system 10 from the historical data. Here, the performance data can be gathered from any component of the cloud services system 10, which may not be associated with the physical properties of the disk, but is associated with a portion of the cloud services system 10. For example, performance data can be latency, throughput, CPU load, memory usage, or IOPS. Historical data is continuously collected in the past, including performance data, metadata, or other required information, which may come from system logs, application logs, database logs, or SMART (Self-Monitoring Analysis and Reporting Technology, self-monitoring analysis and Reporting technology) logs. This means obtaining performance data from the sources mentioned above. In the present embodiment, it is explained by IOPS. In addition, performance data for any disk is uninterruptible, such as continuous and periodic recordings over the past 6 months, but missing performance data for a week in the third month, which is not acceptable. Performance data for interrupted records This is meaningless because the input/output mode of this performance cannot be found.

Then, the second step is to filter out performance data from a faulty disk or a disk having a lifetime value shorter than a preset value (S02). For a better understanding, please refer to FIG. 3, which is a form showing the status of the disks 201 to 230. Obviously, it can be seen from Fig. 3 that the state of some disks is "faulty" at the moment of analysis. A failed disk may be completely failed for replacement, and may also refer to a disk having poor performance, such as a disk having more than a certain degree of dead blocks or being easily overheated. The disks 221 to 225 are judged to be "faulty" and cannot be applied by the present invention. The remaining disks are all good. However, the lifetime values recorded for all disks are not the same. The lifetime value refers to the normal operating time (days) of the disk, which can continue to work for a long time or may fail quickly. The lifespan is different from the life, which is defined as the time at which a disk can work before it fails. In other words, the value of life is a certain value and the range of life values will change at any time. Except for performance data from "faulty" disks that are not available, performance data from good disks but collection times are too short (lifetime values) are not representative. The extent to which the lifetime value is so short that the data of the disk is not representative is not limited by the present invention. In the present embodiment, a disk having a lifetime value shorter than 50 days is removed. Therefore, the use of the disks 226 to 230 is abandoned.

The next step is to group the disks (good and have a lifetime value longer than 50 days) with the corresponding performance data according to the life value level (S03). See section Figure 4, which is a form showing the results of grouping. Grouping or binning is a data pre-processing technique used to reduce the effects of secondary observation errors. The number of groups is not limited. In the present embodiment, the number of groups is 5 groups, which are divided into higher, higher, medium, lower, and lower. In other embodiments, the number of groups can be 3 groups and divided into high, medium, and low. The life value level is the limit value of two adjacent groups. Preferably, the life value levels are evenly distributed over the range between the largest and the smallest of all life values. Here, the life value levels are set to 122, 187, 252, and 317 (days). Thus, as can be seen from FIG. 4, the disk 214 is in the upper group, the disks 213, 215 and 208 are in the high group, the disks 218, 203, 219, and 204 are in the middle group, the disk 216, 201, 220, 212, 211, 217, and 202 are in the lower group, disks 205, 210, 209, 206, and 207 are in the lower group. The performance data and corresponding disks are grouped into a group of the aforementioned groups.

The fourth step of the method is to normalize the performance data and lifetime values to a unitless performance value and a unitless life value (S04). According to the present invention, the unitless performance value is calculated by dividing a first difference value between a performance data value and a minimum of all performance data values by a second difference value between the largest and the smallest of all performance data values. Out. The unitless life value is calculated by dividing a third difference value between the largest of all life values and all of the life values by a fourth difference value between the largest and the smallest of all life values. The calculations for both are similar, but the application goals are different. Please see figure 4. Take the regularization of the life value as an example. In Figure 4, the smallest of all life values is 56, the largest being 382. The fourth difference value is 326 (obtained by 382 minus 56). For disk 216, its lifetime value is 154, the third difference value is 98 (obtained by 154 minus 56) and the unit life value is 0.301 (obtained by dividing 98 by 326).

Next, an LSTM (Long Short Term Memory) modeling algorithm is performed on the unitless performance value of each disk in each group to obtain a predicted trend of the unitless performance value of each disk in a future period of time (S05) ). The LSTM modeling algorithm is a type of Artificial Neural Network (ANN), each of which has a special design that is suitable for predicting trends in long-term data. The detailed design of the LSTM modeling algorithm is not the focus of the present invention, and any LSTM modeling algorithm can be applied, although the results of the derivation are somewhat different. See Figure 5, which shows a graph of unitless performance values collected over the past half in the upper half and a chart showing the predicted trend of unitless performance values for the next 24 hours in the lower half. The second chart is for the same disk, such as disk 208 in the high group. Each point on the solid line in the upper half of the graph is a record of the IOPS of the disk 208. The interval for collecting data may be 5 minutes, 10 minutes, 30 minutes, or one hour, and the present invention is not limited thereto. The dotted line in the lower half shows the predicted trend of IOPS over the next 24 hours. There are 20 disks for analysis, and there are 20 charts with no unit performance value prediction trend in the future.

Then, based on the predicted trend of the unitless performance values in the group, a specific unitless performance value is assigned to each of the disks in each group (S06). It should be noted that the purpose of the present invention is to predict the time of the disk in the future. The expected life value, and the configuration of the optimal disk combination to different workloads has extended the lifetime value of all disks, and the result should be feasible for the disk in the "future time". For example, the future time may refer to the next hour, the next 6 hours, the next hour after the first 6 hours, and the like. Therefore, the specified specific unitless performance value can be changed with different definitions of "this period of time in the future". For a better understanding of step S06, please refer to Figure 6, which shows the predicted IOPS trend of the three disks in the high group. In order to find the results by this method for the next 6 hours, three predicted values (black dots) were sampled from the graphs. The specific unitless performance value is obtained from the predicted unitless performance values at the 6th hour of the average high group (which can be used at any one of the next 6 hours). As shown in Figure 4, it is 0.43. For other groups, specific unitless performance values are also given in Figure 4. Of course, other methods, such as a weighted average or a geometric mean, can also be used to find the specific unitless performance value, which is not limited by the present invention.

The seventh step of the method is to perform a k-means clustering algorithm to obtain an output set from the input set (S07). Here, each input set represents a corresponding disk and contains a specific unitless performance value and a unitless life value. Figure 7 is a chart showing the distribution of input sets. The horizontal axis value is the specific unitless performance value (based on IOPS), and the vertical axis value is the unitless life value. The input set is indicated by a solid diamond. After the operation, the k-average cluster algorithm can divide the input set into three or more clusters (in this embodiment, three clusters are used) regardless of which group each input set may belong to. In each cluster in step S07 The central set is selected as the output set. As shown in Figure 7, the output set for each cluster is indicated by a hollow circle.

The next step is to denormalize each output set to obtain a performance limit and a target lifetime value, respectively (S08). In step S08, the denormalization means multiplying the values in the output set by the corresponding second difference value and the fourth difference value, and adding respective minimum values to obtain an IOPS value and a lifetime value. For high IOPS and high lifetime value clusters, the performance limit is 1542 IOPS and 284 days lifetime value. For medium IOPS and shorter lifetime value clusters, the performance limit is 1150 IOPS and 85 days lifetime value. For low IOPS and medium life value clusters, the performance limit is 544 IOPS and 147 days lifetime value. Next, a performance limit is set for the disks forming the storage device so that each disk has a desired life value not shorter than the target lifetime value for the period of time in the future (S09). The "storage device" used here is the same as the "cluster", that is, several disks are connected for a specific workload, and should not be confused with the "cluster" in the k-average cluster algorithm. If the disk setting in a storage device has an IOPS performance limit of 1542, this means that each disk in the storage device has a desired lifetime of 284 days when it has a workload application with a minimum IOPS lower than 1542.

When all the disks have performance limits set, the final step of the method is to configure a workload for each storage device (several disks) that has a performance requirement that matches or falls below the performance limit (S10). ). In such cases, the disks can be predicted to have a minimum lifetime and the workload (IOPS) requirements can be met. In addition, the minimum lifetime value is the disk energy dimension. The longest life of operation. However, it is emphasized that the result of step S09 is only established within the "future time". When the definition of "this period of time in the future" changes, for example, from the next hour to 6 hours, the result of step S09 will also change. The disk configuration in the storage device is dynamic.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and those skilled in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

Claims (20)

A method for extending a disk life expectancy value in a cloud service system, comprising the steps of: A. collecting performance data from each disk in a cloud service system from historical data to generate a plurality of performance data; The performance data from a faulty disk or a disk having a lifetime value shorter than a preset value is filtered out from the plurality of performance data collected in step A; C. according to the lifetime value level, corresponding performance Data grouping the disks; D. normalizing the performance data and lifetime values to a unitless performance value and a unitless life value; E. performing LSTM (Long) on the unitless performance value of each disk in each group Short Term Memory) modeling algorithm to obtain the predicted trend of the unitless performance value of each disk in a future period of time; F. based on the predicted trend of the unitless performance values in the group, respectively for each group All of the disks specify a specific unitless performance value; G. perform a k-means clustering algorithm to obtain an output set from the input set, where each input set represents a corresponding disk and contains a specific unitless performance And a lifetime value without unit;. H unconventional set of outputs each respectively to a performance limit value and a target life; and I. to form a disk storage device of the plurality of setting a performance limit, so that each The expected life value of a disk for a period of time in the future is not shorter than the target lifetime value. The method of claim 1, further comprising a step J after step I: J. configuring a workload for each storage device, the workload having a performance requirement that matches or falls below the performance limit. The method of claim 1, wherein the performance data is a delay time, a throughput, a central processing unit (CPU) load, a memory usage, or an IOPS (Input/Output Per). Second, number of input/output operations per second). The method of claim 1, wherein the unitless performance value is obtained by dividing a first difference between a performance data value and a minimum of all performance data values by a maximum and a minimum of all performance data values. Calculated by a second difference value between the two. The method of claim 1, wherein the unitless life value is obtained by dividing a third difference between a lifetime value and a maximum of all life values by a maximum and a minimum of all life values. Calculated by a fourth difference value between. The method of claim 1, wherein the disk is a Hard Disk Drive (HDD) or a Solid State Disk (SSD). The method of claim 1, wherein the life value levels are evenly distributed among the largest and smallest of all life values. Inside. The method of claim 1, wherein the specific unitless performance value is obtained by averaging the predicted unitless performance values of the group in the future period of time. The method of claim 1, wherein the historical data is from a system log, an application log, a database log, or a S.M.A.R.T. (Self-Monitoring Analysis and Reporting Technology) log. The method of claim 1, wherein the central set of each cluster in step G is the output set. A cloud service system includes: a host for operating a workload and performing data access; a plurality of disks connected to the host for storing data for workload access; and an expected life extension module, configured Or installed on the host to collect performance data from each disk in the historical data to generate a plurality of performance data; a disk from a fault or a disk having a lifetime value shorter than a preset value The performance data is filtered out from the collected plurality of performance data; according to the life value level, the disks are grouped according to the corresponding performance data; the performance data and the lifetime value are conventionalized into a unitless performance value and none Unit life value; perform an LSTM modeling algorithm on the unitless performance value of each disk in each group to get each disk in The predicted trend of the unitless performance value for a period of time; based on the predicted trend of the unitless performance values in the group, each of the disks in each group is assigned a specific unitless performance value; performing k-average clustering Algorithm, the input set is used to obtain an output set, wherein each input set represents a corresponding disk and includes a specific unitless performance value and a unitless life value; each of the output sets is denormalized to obtain a performance limit respectively And a target lifetime value; and setting a performance limit for the disks forming a storage device such that each disk has a desired lifetime value that is not shorter than the target lifetime value for a period of time in the future. The cloud service system of claim 11, wherein the life extension extension module is further configured to configure a workload for each storage device, the workload having a performance requirement that matches or falls below the performance limit. The cloud service system of claim 11, wherein the performance data is delay time, throughput, CPU load, memory usage, or IOPS. The cloud service system of claim 11, wherein the unitless performance value is obtained by dividing a first difference between a performance data value and a minimum of all performance data values by a maximum of all performance data values. Calculated with a second difference value from the smallest. The cloud service system of claim 11, wherein the unitless life value is one of a maximum value of a lifetime value and all life values. The third difference value is calculated by dividing by a fourth difference value between the largest and the smallest of all life values. The cloud service system of claim 11, wherein the disk is a hard disk or a solid state disk. The cloud service system of claim 11, wherein the life value levels are evenly distributed within a range between a maximum and a minimum of all life values. The cloud service system of claim 11, wherein the specific unitless performance value is obtained by averaging the predicted unitless performance values of the group in the future period of time. The cloud service system of claim 11, wherein the historical data is from a system log, an application log, a database log, or an S.M.A.R.T. log. The cloud service system of claim 11, wherein a central set of each cluster from the k-average cluster algorithm is selected as the output set.