JP7553136B1 - STORAGE SYSTEM, STORAGE, AND METHOD AND PROGRAM FOR CONTROLLING STORAGE SYSTEM - Google Patents

Abstract

When a data copy operation with guaranteed order from a primary volume to a secondary volume is stopped and then restarted, data in the secondary volume is appropriately evacuated.
[Solution] When a copy operation that guarantees order from the primary volume to the secondary volume is stopped, the storage system copies each piece of data that is determined to have a difference between the secondary volume and the save volume during the copy operation that guarantees order from the primary volume to the secondary volume from the secondary volume to the save volume before copying each piece of data that is determined to have a difference between the primary volume and the secondary volume during the stopped period from the primary volume to the secondary volume, and then copies each piece of data that is determined to have a difference based on first difference information stored in a second storage means from the primary volume to the secondary volume, and then resumes the copy operation.
[Selection] Figure 26

Description

The present invention relates to a storage system, storage, a control method for a storage system, and a program.

One method for backing up data from a main site storage to a remote site storage is to perform data updates at the main site and data copies at the remote site asynchronously while maintaining the order of data (see, for example, Patent Document 1). In this method, for example, if a buffer that temporarily holds the data to be copied overflows, the copy operation is stopped, differential copying is performed until consistency is achieved, and then the copy operation is resumed. This differential copying is performed without considering the order of data, so if a failure occurs at the main site while the differential copying is being performed, the data at the remote site becomes meaningless (see paragraphs 0007 to 0009 of Patent Document 1). To address this problem, a backup area that holds the state before the differential copying is performed at the remote site can be provided to restore at least the data that is consistent before the differential copying (see paragraphs 0010 to 0011 of Patent Document 1). When providing a backup area, in order to backup the data to be overwritten before overwriting the differential data, it is necessary to search for an empty area, backup the data, and create and store information indicating the location where the data was stored and the backup destination, etc. (see the patent document prior to Patent Document 1 (Patent Publication No. 11-15604, paragraph 0052)). In addition, if a backup area is provided, it will be necessary to prepare a volume capacity that is twice the current capacity (paragraph 0014 of Patent Document 1).

The above-mentioned problems with differential copying between the main site and the remote site can be solved, for example, by providing a replica of the secondary volume of the copy destination at the remote site and backing up all data of the secondary volume to the replica (Patent Document 2). In this case, data recovery using the save volume when a failure occurs is performed based on the data at the time when the data of the secondary volume was backed up to the replica. In the remote copy system described in Patent Document 2, data is backed up from the secondary volume to the replica at a preset scheduled time.

JP 2015-52844 A JP 2008-181288 A

As described in Patent Document 1, when data to be overwritten is evacuated to a backup area during a differential copy from a main site to a remote site, the backup area must be secured and information about the evacuated data must be created and stored according to the contents of the differential copy. For this reason, the processing and storage capacity required for the backup may become an issue in some cases.

In addition, when creating a replica at a preset scheduled time as described in Patent Document 2, the amount of data that can be recovered can be an issue depending on the interval between the time the failure occurred and the scheduled time.

The present invention aims to provide a storage system, storage, a control method for a storage system, and a program that solve the above-mentioned problems.

One aspect of the present invention comprises a first storage having a primary volume, and a second storage connected to the first storage via a communication line and having a secondary volume set as a copy pair with the primary volume, the first storage further comprising a first storage means for temporarily holding data copied from the primary volume to the secondary volume with guaranteed order, and a second storage means for storing first difference information indicating whether there is a difference in data for each of a predetermined first storage area for the primary volume and the secondary volume, the second storage further comprising a save volume for copying and storing all data of the secondary volume, and a previous copy of the data for the secondary volume and the save volume via the first storage means for each of a predetermined second storage area for the secondary volume and the save volume. and a third storage means for storing second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume. When the copy operation is stopped, before copying each piece of data that is determined to have a difference based on the second difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, the storage system copies each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume, and then copies each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, and then resumes the copy operation.

One aspect of the present invention is a first storage device comprising a primary volume and connected via a communication line to a second storage device comprising a secondary volume set as a copy pair with the primary volume, the first storage device further comprising a first storage means for temporarily holding data copied from the primary volume to the secondary volume with guaranteed order, and a second storage means for storing first difference information indicating whether or not there is a difference in data for each predetermined first storage area for the primary volume and the secondary volume, the second storage device further comprising a secondary volume, a save volume for copying and storing all data of the secondary volume, and a second storage means for storing first difference information indicating whether or not there is a difference in data for each predetermined second storage area for the secondary volume and the save volume, the second storage device further comprising a first storage means for temporarily holding data copied from the primary volume to the secondary volume with guaranteed order, and a second storage means for storing first difference information indicating whether or not there is a difference in data for each predetermined first storage area for the secondary volume and the save volume, the second storage device further comprising a second storage means for temporarily ... and a third storage means for storing second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume, and when the copy operation is stopped, each piece of data that is determined to have a difference based on the second difference information is copied from the secondary volume to the save volume before each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period is copied from the primary volume to the secondary volume, and then each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period is copied from the primary volume to the secondary volume, and then the copy operation is resumed.

One aspect of the present invention is a second storage connected via a communication line to a first storage having a primary volume and having a secondary volume set as a copy pair with the primary volume, the first storage comprising: the primary volume; a first storage means for temporarily holding data copied from the primary volume to the secondary volume with guaranteed order; and a second storage means for storing first difference information indicating whether there is a difference in data for each predetermined first storage area for the primary volume and the secondary volume; the second storage further comprises a save volume for copying and storing all data of the secondary volume; and a second storage means for storing first difference information indicating whether there is a difference in data for each predetermined second storage area for the secondary volume and the save volume, the second storage further comprising: a save volume for copying and storing all data of the secondary volume; and a third storage means for storing second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume, and when the copy operation is stopped, each piece of data that is determined to have a difference based on the second difference information is copied from the secondary volume to the save volume before each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period is copied from the primary volume to the secondary volume, and then each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period is copied from the primary volume to the secondary volume, and then the copy operation is resumed.

One aspect of the present invention is a control method for controlling a first storage having a primary volume and a second storage connected to the first storage via a communication line and having a secondary volume set as a copy pair with the primary volume, the first storage further comprising a first storage means for temporarily holding data copied from the primary volume to the secondary volume with guaranteed order, and a second storage means for storing first difference information indicating whether or not there is a difference in data for each of a predetermined first storage area for the primary volume and the secondary volume, the second storage further comprising a save volume for copying and storing all data of the secondary volume, and a control method for controlling the primary volume via the first storage means for each of a predetermined second storage area for the secondary volume and the save volume. and a third storage means for storing second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume, and the method includes, when the copy operation is stopped, a step of copying each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume before copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, a step of subsequently copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, and a step of subsequently resuming the copy operation.

In one aspect of the present invention, when controlling a first storage having a primary volume and a second storage connected to the first storage via a communication line and having a secondary volume set as a copy pair with the primary volume, the first storage further comprises a first storage means for temporarily holding data copied from the primary volume to the secondary volume with guaranteed order, and a second storage means for storing first difference information indicating whether or not there is a difference in data for each predetermined first storage area for the primary volume and the secondary volume, and the second storage further comprises a save volume for copying and storing all data of the secondary volume, and a second storage means for storing first difference information indicating whether or not there is a difference in data for each predetermined second storage area for the secondary volume and the save volume from the primary volume via the first storage means. and a third storage means for storing second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data to the secondary volume. When the copy operation is stopped, the program causes a computer to execute the steps of copying each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume before copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, then copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, and then resuming the copy operation.

The storage system, storage, storage system control method, and program of the present invention make it possible to appropriately evacuate data from the secondary volume when the operation of copying data from the primary volume to the secondary volume with guaranteed order is stopped and then resumed.

1 is a block diagram illustrating an example of the configuration of a storage system according to an embodiment of the present invention. 11 is a flowchart illustrating an example of processing when a group for order-guaranteed copying is set in the embodiment of the present invention. 11 is a schematic diagram showing an example of group settings for order-guaranteed copying according to an embodiment of the present invention; FIG. 11 is a flowchart illustrating an example of an operation when a volume for order-guaranteed copying is constructed in the embodiment of the present invention. 1 is a schematic diagram illustrating an example of a data flow when a volume for order-guaranteed copying is constructed in an embodiment of the present invention; 11 is a schematic diagram illustrating an example of the contents of a difference information buffer when a volume for order-guaranteed copying is constructed in an embodiment of the present invention. FIG. 11 is a flowchart illustrating an example of an operation when constructing a save volume for order guarantee copying in an embodiment of the present invention. 1 is a schematic diagram illustrating an example of a data flow when a save volume for order guarantee copying is constructed in an embodiment of the present invention; 11 is a schematic diagram illustrating an example of the contents of a difference information buffer when a save volume for order guarantee copying is constructed in an embodiment of the present invention. FIG. 11 is a flowchart illustrating an example of an operation of an order guarantee copy in an embodiment of the present invention. FIG. 11 is a schematic diagram for explaining an example of data flow in an example of the operation of order guarantee copying according to an embodiment of the present invention; 11 is a flowchart illustrating an example of an operation performed when stopping order guarantee copying according to an embodiment of the present invention. 11 is a flowchart showing an example of an operation performed when writing from a host in a state in which order guarantee copying is stopped in an embodiment of the present invention. 10 is a schematic diagram illustrating an example of data flow in an example of an operation performed when writing from a host in a state in which order guarantee copying is stopped in an embodiment of the present invention. FIG. 13 is a schematic diagram illustrating an example of the contents of a difference information buffer in an example of an operation performed when writing from a host in a state in which order assurance copying is stopped in an embodiment of the present invention. FIG. 10 is a flowchart illustrating an example of an operation when resuming order guarantee copying in the first embodiment of the present invention. 10 is a flowchart illustrating an example of a copy operation to a save volume when resuming order guarantee copying in the first embodiment of the present invention. 10 is a schematic diagram illustrating an example of data flow in an operation example when resuming order assurance copying in the first embodiment of the present invention; FIG. 11 is a schematic diagram illustrating an example of the contents of a difference information buffer when resuming order assurance copying in the first embodiment of the present invention; FIG. 10 is a flowchart showing an example of an operation of differential copying from a primary volume to a secondary volume when order guarantee copying is resumed in the first embodiment of the present invention. 10 is a schematic diagram illustrating an example of data flow in an operation example when resuming order assurance copying in the first embodiment of the present invention; FIG. 11 is a schematic diagram illustrating an example of the contents of a difference information buffer when resuming order assurance copying in the first embodiment of the present invention; FIG. 13 is a flowchart illustrating an example of an operation when resuming order guarantee copying in the second embodiment of the present invention. 13 is a flowchart showing an example of an operation of differential copying from a primary volume to a secondary volume when order guarantee copying is resumed in the second embodiment of the present invention. 13 is a flowchart illustrating an example of an operation performed when stopping an order guarantee copy in the third embodiment of the present invention. FIG. 2 is a block diagram showing an example of a minimum configuration according to an embodiment of the present invention. 10 is a flowchart illustrating an example of a minimum configuration of an operation according to an embodiment of the present invention. FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the same or corresponding components in each drawing will be designated by the same reference numerals and their explanations will be omitted as appropriate.

[First embodiment]
(Storage System Configuration)
FIG. 1 is a block diagram for explaining an example of the configuration of a storage system according to the first, second and third embodiments of the present invention. The storage system 3 shown in FIG. 1 includes a storage 10 installed at a main site 1 and a storage 20 installed at a remote site 2. The remote site 2 is located in an area away from the main site 1 for the purpose of disaster recovery, for example. In the following, the "storage 10 installed at the main site 1" will also be simply referred to as the "main site 1". Furthermore, the "storage 20 installed at the remote site 2" will also be simply referred to as the "remote site 2". The main site 1 and the remote site 2 are connected by a remote line (communication line) 50 such as a LAN (Local Area Network) or a WAN (Wide Area Network).

The storage 10 is connected to a host computer (hereinafter simply referred to as "host") 100 via a communication line 51 such as a SAN (Storage Area Network). The host 100 writes data to the storage 10 and reads data, and also sets the operating conditions of the storage system 3. The storage 20 is connected to the host 200 via a communication line 52. The host 200 writes data to the storage 20 and reads data, and also sets the operating conditions of the storage system 3. The host 100 and the host 200 may or may not be included in the storage system 3. The host 200 may be omitted. There may be multiple hosts 100 and 200. In the following, the operation of writing data from the host 100 to the storage 10 is referred to as "I/O" (input or output) or "host I/O". The written data is also referred to as "I/O data". The I/O data is managed, for example, by an operating system (OS) running on the host 100. The I/O data is issued, for example, in the form of a request to write to the storage 10 in a specific block unit by specifying the destination volume and the storage area within the volume.

Storage 10 comprises a storage controller (also called a storage processor, array controller, etc.) SC1 and a storage array 30. Storage 20 comprises a storage controller SC2 and a storage array 40. Storage arrays 30 and 40 each comprise, for example, a plurality of disk drives and a controller. A plurality of volumes are set in storage arrays 30 and 40. A volume is a unit of data management in storage (external storage device) and is also called a logical volume. The multiple volumes can be divided into groups. Each group (also called a volume group) can contain multiple volumes. Volumes belonging to the same group maintain data consistency at the same time.

In the storage system 3 of this embodiment, data of the primary volume updated by the host 100 is copied and stored in a buffer (storage area such as memory or disk) in the main site 1 with order guaranteed. The data stored in the buffer in the main site 1 is transferred to the storage 20 in the remote site 2 separately asynchronously with the host I/O. A secondary volume with the update order guaranteed is then created on the storage 20 in the remote site 2. Hereinafter, the series of operations in which a copy of data updated in the primary volume is written to the secondary volume with order guaranteed is referred to as "order guaranteed copy" or "order guaranteed copy operation". Data copied by order guaranteed copy is also referred to as order guaranteed data.

In this embodiment, as an example, a primary volume 18 (hereinafter also simply referred to as "volume 18") is set in the storage array 30, and a secondary volume 28 (hereinafter also simply referred to as "volume 28") is set in the storage array 40 as a copy pair of the primary volume 18. Also, a save volume 29 that copies and stores all data of the secondary volume 28 is set in the storage array 40. Also, "volume 18" and "volume 28" are set to belong to "group 3". Below, the configuration and operation will be explained using examples of the primary volume 18, secondary volume 28, and save volume 29 as necessary.

Volumes 18, 28 and evacuation volume 29 each include a corresponding area A (RA), area B (RB), area C (RC), area D (RD), and area E (RE). For example, when performing an order-guaranteed copy, the same data is stored in area D of volume 18 and area D of volume 28, and the same data is stored in area D of volume 28 and area D of evacuation volume 29.

(Storage Controller Configuration)
The storage controller SC1 has the following functional components, which are configured by a combination of a computer, hardware such as peripheral circuits and devices of the computer, and software such as programs executed by the computer: The storage controller SC1 has an I/O processing unit 11, a cache memory 12, an order assurance processing unit 13, an order assurance buffer 14, a remote transmission processing unit 15, a difference management processing unit 16, and a difference information buffer 17.

First, the order assurance buffer 14 and the difference information buffer 17 will be described. The order assurance buffer 14 is a storage area such as a memory or a disk that stores data for order assurance copying. In the order assurance buffer 14, I/O data issued from the host 100 to the primary volume 18 when performing order assurance copying is stored together with relational data by the order assurance processing unit 13. Here, relational data includes data representing, for example, the position, timestamp, sequence number, etc. of the updated primary volume 18. Note that, hereinafter, the data stored in the order assurance buffer 14 is also referred to as order assurance buffer data.

The difference information buffer 17 is a storage area such as a memory or a disk that stores difference information. The difference information buffer 17 is used at the main site 1 for managing the difference between the volume 18 and the volume 28 at the remote site 2. As shown in FIG. 6, for each area A (RA), B (RB), C (RC), D (RD), E (RE), ... of the volume 18, the difference information buffer 17 holds data "no difference" when the data matches with each area A (RA), B (RB), C (RC), D (RD), E (RE), ... of the volume 28, and holds data "difference" when they do not match. Note that FIG. 6 is a schematic diagram for explaining an example of the contents of the difference information buffer (difference information buffers 17 and 27) when constructing a volume for order-guaranteed copying in an embodiment of the present invention. The contents of the difference information buffer 17 are updated by the difference management processing unit 16. The difference information buffer 17 has a setting of valid or invalid, which is set by the difference management processing unit 16, and if invalid, the contents are not updated.

Next, the I/O processing unit 11 receives and processes I/O data from the host 100. For example, the I/O processing unit 11 receives I/O data written from the host 100 in the cache memory 12 and calls the order assurance processing unit 13. In response to this, the order assurance processing unit 13 copies the data to the order assurance buffer 14 as described below, and reports the completion to the I/O processing unit 11. Furthermore, when the I/O processing unit 11 receives a report from the order assurance processing unit 13 that the data has been copied to the order assurance buffer 14, it calls the difference management processing unit 16, and the difference management processing unit 16 sets the difference information buffer 17 to indicate that there is a difference. Furthermore, when the I/O processing unit 11 receives a report from the order assurance processing unit 13 that the I/O data has been copied to the order assurance buffer 14, it reports the completion to the host 100. Furthermore, the I/O processing unit 11 writes the I/O data written to the cache memory 12 to the volume 18 asynchronously with the host I/O.

Cache memory 12 is a storage area such as a memory or disk.

The order assurance processing unit 13 processes the order of I/O. The order assurance processing unit 13 constructs a group of order assurance copies based on, for example, instructions from the host 100. The order assurance processing unit 13 performs, for example, setting of volumes 18, 28 and evacuation volume 29, setting of order assurance buffers 14 and 24, setting of difference information buffers 17 and 27, etc. When the order assurance processing unit 13 resumes the order assurance copy, it instructs the difference management processing unit 26 of the remote site 2 to copy the data of volume 28 to the evacuation volume 29. In addition, in response to a call from the I/O processing unit 11, the order assurance processing unit 13 copies the I/O data stored in the cache memory 12 to the order assurance buffer 14 and reports the completion to the I/O processing unit 11. In addition, when the order assurance copy is executed, the order assurance processing unit 13 instructs the remote transmission processing unit 15 to transfer the I/O data in the order assurance buffer 14 to the remote site 2. In addition, when the order assurance copy is stopped, the order assurance processing unit 13 stops copying I/O data to the order assurance buffer 14.

The remote transmission processing unit 15 performs processing to transmit a copy of the order assurance buffer 14 and a differential copy to the remote site 2 in response to instructions from the order assurance processing unit 13 or the differential management processing unit 16. Here, the copy of the order assurance buffer 14 is data obtained by duplicating the order assurance buffer data stored in the order assurance buffer 14. In addition, the differential copy is data obtained by duplicating the data of the primary volume 18 based on the contents of the differential information buffer 17 (if "differences exist") when the order assurance copy is resumed.

The differential management processing unit 16 performs differential copying. The differential management processing unit 16 manages the setting of the differential information buffer 17 as enabled or disabled, and the setting of the contents of the differential information buffer 17 ("with difference" or "without difference"). When constructing the secondary volume 28, which is a copy pair of the primary volume 18, the differential management processing unit 16 transfers all data of volume 18 from the remote transmission processing unit 15 to the remote site 2 in order to copy all data of volume 18 to volume 28. In addition, when resuming the order-guaranteed copy, the differential management processing unit 16 transfers a differential copy of the data of volume 18 from the remote transmission processing unit 15 to the remote site 2 based on the contents of the differential information buffer 17.

The storage controller SC2 has the following functional components, which are made up of a combination of hardware such as a computer, peripheral circuits and devices of the computer, and software such as programs executed by the computer. That is, the storage controller SC2 has an I/O processing unit 21, a cache memory 22, an order assurance processing unit 23, an order assurance buffer 24, a remote reception processing unit 25, a difference management processing unit 26, and a difference information buffer 27.

First, the order assurance buffer 24 and the difference information buffer 27 will be described. The order assurance buffer 24 is a storage area such as a memory or a disk that stores order assurance data. In the order assurance buffer 24, the order assurance processing unit 23 stores the order assurance buffer data received from the main site 1 when performing order assurance copying, together with related data.

The differential information buffer 27 is a storage area such as a memory or a disk that stores differential information. The differential information buffer 27 is used at the remote site 2 for managing the differences between the volume 28 and the backup volume 29. As shown in FIG. 6, for each area A (RA), B (RB), C (RC), D (RD), E (RE), ... of the volume 28, the differential information buffer 27 holds data of "no difference" when the data matches with each area A (RA), B (RB), C (RC), D (RD), E (RE), ... of the volume 29, and holds data of "difference" when the data does not match. The contents of the differential information buffer 27 are updated by the differential management processing unit 26. In this example, the size of the area for managing the presence or absence of differences in the differential information buffer 27 and the differential information buffer 17 is the same, but the size of the area may be different. For example, the difference information buffer 27 may combine the two areas A (RA) and B (RB) managed by the difference information buffer 17 into a single area to manage whether or not there is a difference.

The I/O processing unit 21 processes the received I/O. When the remote reception processing unit 25 receives data, the I/O processing unit 21 is called by the remote reception processing unit 25, and calls the order assurance processing unit 23 to have the order assurance buffer 24 receive the data. Also, when the remote reception processing unit 25 receives data, the I/O processing unit 21 is called by the remote reception processing unit 25, and calls the difference management processing unit 26 to instruct it to set the difference information buffer 27 to indicate that there is a difference. Also, when the I/O processing unit 21 receives a report from the order assurance processing unit 23 that the reception of data to the order assurance buffer 24 has been completed, it reports the completion to the remote reception processing unit 25.

The cache memory 22 is a storage area such as a memory or disk that stores data. The cache memory 22 temporarily holds I/O data sent by the host 200, for example.

The order assurance processing unit 23 processes the order assurance buffer data received from the main site 1. When the remote reception processing unit 25 receives data, the order assurance processing unit 23 is called by the I/O processing unit 21, receives the data into the order assurance buffer 24, and reports completion to the I/O processing unit 21. Also, when the remote reception processing unit 25 receives data, the order assurance processing unit 23 is called by the I/O processing unit 21, calls the difference management processing unit 26, and sets the difference information buffer 27 to indicate that there is a difference. Also, the order assurance processing unit 23 copies the data in the order assurance buffer 24 to the volume 28.

The order assurance buffer 24 is a storage area such as a memory or disk that stores the received order assurance data.

The remote reception processing unit 25 receives a copy of the in-order guarantee buffer 14 from the main site 1 and a differential copy from the volume 18. When the remote reception processing unit 25 receives the data, it calls the I/O processing unit 21. When the I/O processing unit 21 reports completion to the remote reception processing unit 25, the remote reception processing unit 25 reports completion to the remote transmission processing unit 15 at the main site 1.

The difference management processing unit 26 performs differential copying. The difference management processing unit 26 also updates the contents of the difference information buffer 27. The difference management processing unit 26 also copies data from the volume 28 to the evacuation volume 29 in response to instructions from the order assurance processing unit 13 when constructing a volume. The difference management processing unit 26 also copies data from the volume 28 to the evacuation volume 29 for the area with differences in the difference information buffer 27 that was set during the order assurance copying operation.

(Example of storage system operation)
Next, an example of the operation of the storage system 3 shown in FIG. 1 will be described. In the description of this example of the operation, the following example is used as the flow of operations and data. That is, the initial state is a state in which data D0 is stored in area D (RD) of volume 18. Next, a copy pair for order-guaranteed copying is constructed. Next, data D1 is written from the host 100 to area D (RD) of volume 18 while order-guaranteed copying is being performed. Next, the operation of order-guaranteed copying is stopped. Next, data D2 is written from the host 100 to area D (RD) of volume 18 while order-guaranteed copying is stopped. After that, a process for resuming order-guaranteed copying is performed.

(Building a group of ordered copies)
First, the procedure by which the order assurance processing unit 13 constructs a group for order assurance copying will be described with reference to Fig. 2. Note that in this embodiment, one pair is set in one group, but two or more pairs may be set. Fig. 2 is a flowchart showing an example of processing when a group for order assurance copying is set in an embodiment of the present invention. The processing shown in Fig. 2 is executed, for example, in response to an exchange between the host 100 and the main site 1 using a specified user interface.

In the process shown in FIG. 2, first, the order assurance buffer 14 is set to "group 3" at the main site 1 (step S101). Next, the order assurance buffer 24 is set to "group 3" at the remote site 2 (step S102). Next, the volume 18 at the main site 1 and the volume 28 at the remote site 2 are set as a pair (copy pair) (step S103). Next, the difference information buffer 17 is set at the main site 1 for difference management between the volume 18 and the volume 28 at the remote site 2 (step S104). Next, the volumes 18 and 28 are set as "group 3" (step S105). Next, the save volume 29 is set in the volume 28 (step S106). Next, the difference information buffer 27 is set at the remote site 2 for difference management between the volume 28 and the save volume 29 (step S107). Next, the difference management processing unit 16 at the main site 1 is instructed to copy data of the volume 18 at the time of construction (step S108). Next, the differential management processing unit 26 at the remote site 2 is instructed to copy the data of the volume 28 during construction (step S109).

When the processes from step S101 to step S109 have been carried out, the setting status of "group 3" will be as shown in FIG. 3. FIG. 3 is a schematic diagram showing an example of group settings for order-guaranteed copying in an embodiment of the present invention. At the main site 1, volume 18, order-guaranteed buffer 14, and difference information buffer 17 are set to "group 3". At the remote site 2, volume 28, evacuation volume 29, order-guaranteed buffer 24, and difference information buffer 27 are set to "group 3".

(Copying volume data when constructing a group for order-guaranteed copying)
Next, the operation of the difference management processing unit 16 that receives a volume data copy instruction in step S108 of Fig. 2 when a group is constructed at the main site 1, and the receiving operation at the remote site 2 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the operation when constructing a volume for order-guaranteed copying in an embodiment of the present invention. The volume data copy instruction when constructing a group is issued based on, for example, the operation of the host 100. Note that before data is copied, the contents of each area of the difference information buffer 17 are set to have differences, and the contents of each area of the difference information buffer 27 are set to have no differences.

In the process shown in FIG. 4, first, the differential management processing unit 16 of the main site 1 transfers data D0 in area D of volume 18 from the remote transmission processing unit 15 to the remote site 2 (step S201). The remote reception processing unit 25 of the remote site 2 receives the data D0 and calls the I/O processing unit 21 (step S202). Next, the I/O processing unit 21 of the remote site 2 instructs the differential management processing unit 26, which sets area D of volume 28 in the differential information buffer 27 to indicate that there is a difference (step S203). Next, the I/O processing unit 21 of the remote site 2 copies data D0 received from the main site 1 to volume 28 and reports completion to the main site 1 (step S204). The differential management processing unit 16 of the main site 1 sets area D of volume 18 in the differential information buffer 17 to indicate that there is no difference (step S205).

The processes from step S201 to step S205 are repeated until the entire copy of volume 18 at main site 1 is completed (repeated steps S201 to S205 and step S206: No). When the entire copy is completed, the differential management processing unit 16 reports the completion to the calling order assurance processing unit 13.

If two or more pairs are set in "group 3", steps S201 to S206 are repeated to perform copying for all pairs in "group 3". When the copying during construction is completed, the data in volumes 18 and 28 will match. When viewed from area D (RD), volumes 18 and 28 will be data D0 (Figure 5). Furthermore, difference information buffer 17 will show no differences, and difference information buffer 27 will show differences (Figure 6). Figure 5 is a schematic diagram for explaining an example of the data flow when constructing a volume for order-guaranteed copying in an embodiment of the present invention.

(Operation of the difference management processing unit 26 that receives an instruction to copy data of the volume 28 when constructing a group)
7, the operation of the differential management processing unit 26 that receives an instruction to copy data of the volume 28 in step S109 of Fig. 2 when constructing a group at the remote site 2 will be described. Fig. 7 is a flowchart showing an example of the operation when constructing a save volume for order guarantee copying in an embodiment of the present invention.

In the process shown in FIG. 7, first, the difference management processing unit 26 copies data D0 in area D (RD) of volume 28 to area D (RD) of the save volume 29 (step S301). Next, the difference management processing unit 26 sets area D (RD) of the difference buffer 27 to no difference (step S302). The difference management processing unit 26 repeats steps S301 to S303 until copying of the entire volume is complete (repeating steps S301 to S302 and No in step S303). When copying of the entire volume is complete, the difference management processing unit 26 reports completion to the calling order assurance processing unit 13.

If two or more pairs are set in "Group 3," steps S301 to S303 are repeated to perform copying for all pairs in "Group 3."

When copying from volume 28 at remote site 2 to save volume 29 is complete, the data in volume 28 and the save volume will be consistent. When viewed from area D (RD), volume 28 and save volume 29 will be data D0 (Figure 8). Furthermore, the difference information buffer 28 will have no differences (Figure 9). Note that Figure 8 is a schematic diagram for explaining an example of the data flow when constructing a save volume 29 for order-guaranteed copying in an embodiment of the present invention. Figure 9 is a schematic diagram for explaining an example of the contents of the difference information buffer 27 when constructing a save volume 29 for order-guaranteed copying in an embodiment of the present invention.

(Order-guaranteed copy operation)
Next, an order guarantee copy operation when a write is performed from the host 100 to the volume 18 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the order guarantee copy operation in the embodiment of the present invention.

In the process shown in FIG. 10, the I/O processing unit 11 receives data D1 written from the host 100 into the cache memory 12 and calls the order assurance processing unit 13 (step S401). Next, the order assurance processing unit 13 copies data D1 to the order assurance buffer 14 and reports completion to the I/O processing unit 11 (step S402). Next, the I/O processing unit 11 reports completion to the host 100 (step S403). Note that the data D1 written to the cache memory 12 is written asynchronously to the volume 18.

Next, the order assurance processing unit 13 instructs the remote transmission processing unit 15 to transfer data D1 in the order assurance buffer 14 to the remote site 2. The remote transmission processing unit 15 also transfers the data to the remote site 2 (step S404).

Meanwhile, the remote receiving processor 25 at the remote site 2 calls the I/O processor 21, which then calls the order assurance processor 23 (step S405). The order assurance processor 23 then receives the data D1 into the order assurance buffer 24 and reports completion to the I/O processor 21 (step S406). The I/O processor 21 then reports completion to the remote receiving processor 25, which then reports completion to the remote sending processor 15 at the main site 1 (step S407).

Next, the order assurance processing unit 23 calls the difference management processing unit 26, which sets the presence of a difference in the difference information buffer 27 (in this example, the storage area of the difference information buffer 27 corresponding to area D (RD) of volume 28) (step S408).

Then, the order assurance processing unit 23 copies the data D1 to area D (RD) of the volume 28 (step S409).

When the order-guaranteed copy is performed, the area D (RD) of volume 18 and volume 28 will be in the same state as data D1, and data will remain as D0 because it has not been copied to the area D (RD) of the evacuation volume 29 (Figure 11). Figure 11 is a schematic diagram for explaining an example of data flow in an example of the operation of order-guaranteed copy according to an embodiment of the present invention.

(Stopping sequence-guaranteed copying)
Next, a case where order guarantee copying is stopped will be described with reference to Fig. 12. The order guarantee copying operation is stopped, for example, when the order guarantee buffer 14 is about to overflow. Fig. 12 is a flowchart showing an example of the operation when order guarantee copying is stopped in an embodiment of the present invention. In the process shown in Fig. 12, first, the order guarantee processing unit 13 of the main site 1 stops copying to the order guarantee buffer 14 (step S501). Next, the difference management processing unit 16 enables the difference information buffer 17 (step S502).

(Writing from the host 100 to the volume 18 while the order-guaranteed copy is stopped)
Next, a case where writing is performed from the host 100 to the volume 18 while order guarantee copying is stopped will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the operation when writing from the host 100 while order guarantee copying is stopped in an embodiment of the present invention.

In the process shown in FIG. 13, first, the I/O processing unit 11 receives data D2 written from the host 100 into the cache memory 12 (step S601). Next, the I/O processing unit 11 calls the difference management processing unit 16 (step S602), and the difference management processing unit 16 sets the area D (RD) of the volume 18 in the difference information buffer 17 to indicate that there is a difference, and reports completion to the I/O processing unit 11 (step S603).

The I/O processing unit 11 then reports completion to the host 100 (step S604). The data D2 written to the cache memory 12 is written to the volume 18 asynchronously. In this case, the data D2 is written to the volume 18, but is not stored in the order assurance buffer 14, and a difference is recorded in the storage area of the difference information buffer 17 corresponding to area D of the volume 18.

In volume 18, area D (RD) where writing has been performed from host 100 to volume 18 becomes data D2 (Figure 14). Also, in the difference buffer 17, area D where writing has been performed from host 100 to volume 18 becomes a differential (Figure 15). Figure 14 is a schematic diagram for explaining an example of data flow in an operation example when writing from host 100 with order assurance copying stopped in an embodiment of the present invention. Figure 15 is a schematic diagram for explaining an example of the contents of difference information buffer 17 in an operation example when writing from host 100 with order assurance copying stopped in an embodiment of the present invention.

Figure 14 shows a state in which the host 100 writes data D1 to region D of volume 18 when performing order-guaranteed copying, and then writes data D2 to region D of volume 18 when order-guaranteed copying is stopped. In this case, at the main site 1, data D2 is stored in the cache memory 12, and data D2 is stored in region D (RD) of volume 18. Data D1 is also stored in the order-guaranteed buffer 14. Meanwhile, at the remote site 2, data D1 is stored in the order-guaranteed buffer 24, and data D1 is stored in region D (RD) of volume 28. Data D0 is also stored in region D (RD) of the evacuation volume 29.

(Resuming sequence-guaranteed copying)
Next, the case where order guarantee copying is resumed will be described with reference to Fig. 16. Fig. 16 is a flowchart showing an example of operation when order guarantee copying is resumed in the first embodiment of the present invention. Order guarantee copying is resumed, for example, in response to an instruction from the host 100, or when the cause of the suspension has been eliminated.

In the process shown in FIG. 16, first, the order assurance processing unit 13 instructs the differential management processing unit 26 at the remote site 2 to copy the data in the volume 28 to the backup volume 29 (step S701).

Next, the order assurance processing unit 13 instructs the difference management processing unit 16 at the main site 1 to copy from volume 18 to volume 28 (step S702).

Next, the differential management processing unit 13 enables the order guarantee buffer 14 (step S703). After this, writing from the host 100 becomes an order guarantee copy using the order guarantee buffer 14.

(Copying data from volume 28 to save volume 29 when resuming order-guaranteed copying)
Next, copying of data from the volume 28 to the save volume 29 in step S701 in Fig. 16 will be described with reference to Fig. 17. Fig. 17 is a flowchart showing an example of a copying operation to the save volume 29 when resuming order guarantee copying in the first embodiment of the present invention.

In the process shown in FIG. 17, the difference management processing unit 26 first copies data D1 from the volume 28 to the save volume 29 for the area D with differences in the difference information buffer 27 that was set in step S408 of FIG. 10 during the order-guaranteed copy operation (step S801 -> step S802: Yes -> step S803). Note that in step S801, the difference management processing unit 26 calculates the area with differences from the difference information buffer 27. Also, in step S802, the difference management processing unit 26 determines whether or not there is an area with differences. Also, in step S802, the difference management processing unit 26 reads data of the area with differences from the volume 28 and copies it to the save volume 29.

Next, the difference management processing unit 26 sets the copied area D of the difference information buffer 27 to no difference (step S804).

The difference management processing unit 26 repeats steps S801 to S804 until all areas with differences are gone.

If there are two or more pairs in the group, steps S801 to S804 are repeated.

When the copying is completed, area D of the save volume 29 becomes data D1 (Figure 18). Also, area D of volume 28 in the difference buffer 27 becomes in a state of no difference (Figure 19). Figure 18 is a schematic diagram for explaining an example of the data flow in an operation example when resuming order assurance copying in the first embodiment of the present invention. Figure 19 is a schematic diagram for explaining an example of the contents of the difference information buffer 27 when resuming order assurance copying in the first embodiment of the present invention.

Figure 18 shows the state at the stage where data in volume 28 has been copied (evacuated) to evacuation volume 29 before resuming order assurance copying from a stopped state. In this case, at the main site 1, data D2 is stored in the cache memory 12, and data D2 is stored in area D (RD) of volume 18. Data D1 is also stored in the order assurance buffer 14. Meanwhile, at the remote site 2, data D1 is stored in the order assurance buffer 24, and data D1 is stored in area D (RD) of volume 28. Data D1 is also stored in area D (RD) of evacuation volume 29.

(Differential copying from the volume 18 of the main site 1 to the volume 28 of the remote site 2 when resuming the order-guaranteed copying).
Next, the differential copying from the volume 18 at the main site 1 to the volume 28 at the remote site 2 in step S702 in Fig. 16 will be described with reference to Fig. 20. Fig. 20 is a flowchart showing an example of the operation of differential copying from the primary volume 18 to the secondary volume 28 when resuming order guarantee copying in the first embodiment of the present invention.

In the process shown in FIG. 20, first, the differential management processing unit 16 copies data D2 from volume 18 to volume 28 for the area with a difference (area D (RD) in this example) set in the differential information buffer 17 when the order-guaranteed copy was stopped (step S901 → step S902: Yes → step S903). Note that in step S901, the differential management processing unit 16 calculates the area with a difference in the differential information buffer 17. In step S902, the differential management processing unit 16 determines whether or not there is an area with a difference. In step S903, the data with a difference in volume 18 is transferred from the remote transmission processing unit 15 to the remote site 2.

The differential management processing unit 26 at the remote site 2 sets the differential information buffer 27 to indicate that there is a difference, in the same way as when copying during construction, copies the data to the volume 28, and reports completion (steps S904 to S906). In step S904, the remote receiving processing unit 25 receives the data and calls the I/O processing unit 21. In step S905, the I/O processing unit 21 instructs the differential management processing unit 26 to set the differential information buffer 27 to indicate that there is a difference. In step S906, the I/O processing unit 21 copies the data D2 received from the main site 1 to the volume 28, and reports completion to the main site 1.

After receiving the completion report at the main site 1, the difference management processing unit 16 sets the area copied to the difference information buffer 17 to no differences (step S907). Steps S901 to S907 are repeated until all areas with differences in the difference information buffer 17 have been copied (step S901 → step S902: yes → repeat steps S903 to S907).

If there are two or more pairs in "Group 3," steps S901 to S907 are repeated for each pair.

When copying is complete, area D of volume 28 becomes data D2 (Figure 21). The difference buffer 17 has no difference, and the copied area of the difference buffer 28 has a difference (Figure 22). Figure 21 is a schematic diagram for explaining an example of data flow in an operation example when resuming order assurance copying in the first embodiment of the present invention. Figure 22 is a schematic diagram for explaining an example of the contents of the difference information buffer when resuming order assurance copying in the first embodiment of the present invention.

Figure 21 shows the state at the stage where data with differences has been copied from volume 18 to volume 28 after the data of volume 28 has been copied to evacuation volume 29 and before resuming order assurance copying from the stopped state. In this case, at main site 1, data D2 is stored in cache memory 12, and data D2 is stored in area D (RD) of volume 18. Data D1 is also stored in order assurance buffer 14. Meanwhile, at remote site 2, data D1 is stored in order assurance buffer 24, and data D2 is stored in area D (RD) of volume 28. Data D1 transferred from main site 1 to remote site 2 in the order assurance copying operation is also stored in area D (RD) of evacuation volume 29.

(Action and Effects)
According to this embodiment, when order assurance copying is resumed, before differential copying is performed using the difference information buffer 17 at the main site 1, data D1 in volume 28 at the time when order assurance copying was stopped is automatically copied to the evacuation volume 29, making it possible to secure the data D1 before the order assurance copying was stopped while maintaining its order. That is, according to the first embodiment, when the operation of copying data with order assurance from the primary volume to the secondary volume is resumed after being stopped, the data in the secondary volume can be appropriately evacuated.

[Second embodiment]
A second embodiment of a storage system according to the present invention will be described with reference to Figs. 23 and 24. Fig. 23 is a flowchart showing an example of operation when order guarantee copying is resumed in the second embodiment of the present invention. The process shown in Fig. 23 corresponds to the process shown in Fig. 16 in the first embodiment. Fig. 24 is a flowchart showing an example of operation of differential copying from a primary volume to a secondary volume when order guarantee copying is resumed in the second embodiment of the present invention. The process shown in Fig. 24 corresponds to a combination of the process shown in Fig. 17 and the process shown in Fig. 20 in the first embodiment.

In the first embodiment, as described with reference to Figures 16, 17 and 20, first, when resuming order-guaranteed copying, before the resumption, data with differences in volume 28 is copied to evacuation volume 29 (step S701 in Figure 16, Figure 17), and after completion, data with differences in volume 18 is differentially copied to volume 28 (step S702 in Figure 16, Figure 20). On the other hand, in the second embodiment, when resuming order-guaranteed copying, before the resumption, copying from volume 28 to evacuation volume 29 and differential copying of data with differences in volume 18 to volume 28 are performed in parallel for each area. Other configurations and operations are basically the same in the first and second embodiments. Below, the parts that differ from the first embodiment are described.

In the second embodiment, in the restart process shown in FIG. 23, first, the order assurance processing unit 13 instructs the difference management processing unit 26 at the remote site 2 to copy data from the volume 28 to the evacuation volume 29 (step S1001). Next, the difference management processing unit 26 at the remote site 2 reports completion to the order assurance processing unit 13 at the main site 1 without waiting for the copy to be completed (step S1002). Next, the order assurance processing unit 13 instructs the difference management processing unit 16 to copy data that was written to the volume 18 while the order assurance copy was stopped (step S1003). Next, the difference management processing unit 16 enables the order assurance buffer 14 (step S1004). Thereafter, writing from the host 100 becomes an order assurance copy using the order assurance buffer 14.

Next, referring to FIG. 24, the copy operation from volume 18 to volume 28 by the differential management processing unit 16 in step S1003 in FIG. 23 will be described. In the process shown in FIG. 24, first, the differential management processing unit 16 instructs the remote transmission processing unit 15 to remotely transfer data D2 in area D (RD) of volume 18 to the area with a difference set in the differential information buffer 17 when the order guarantee copy was stopped, and transmits the data to the remote site 2 (step S1101 → step S1102: Yes → step S1103). In step S1101, the differential management processing unit 16 calculates the area with a difference in the differential information buffer 17. In step S1102, the differential management processing unit 16 determines whether or not there is an area with a difference. In step S1103, the data with a difference in volume 18 is transferred from the remote transmission processing unit 15 to the remote site 2.

Next, the remote reception processing unit 25 of the remote site 2 receives the data (step S1104). Next, if the area in the differential information buffer 27 set during the sequence guarantee copy operation has a difference (step S1105 → step S1106: Yes), the differential management processing unit 26 of the remote site 2 copies data D1 of area D (RD) from volume 28 to the evacuation volume 29 (steps S1107 to S1108), and then copies data D2 of area D (RD) of volume 18 to volume 28 (steps S1109 to S1110). On the other hand, if the area does not have a difference (step S1105 → step S1106: No), the data D2 of area D (RD) of volume 18 is copied directly to volume 28 (steps S1109 to S1110).

In step S1104, the remote reception processing unit 25 receives the data. In step S1105, the difference management processing unit 26 checks the difference information buffer 27. In step S1106, the difference management processing unit 26 determines whether or not the area has a difference. In step S1107, the difference management processing unit 26 copies data from the volume 28 to the save volume 29. In step S1108, the difference management processing unit 26 sets the difference information buffer 27 to no difference. In step S1109, the difference management processing unit 26 sets the difference information buffer 27 to have a difference. In step S1110, the difference management processing unit 26 copies data D2 in area D (RD) of volume 18 to volume 28, and the remote site 2 reports completion to the main site 1.

At the main site 1, the difference management processing unit 16 sets the area copied to the difference information buffer 17 to no differences (step S1111).

The processing of steps S1101 to S1111 is repeated until all areas with differences in the difference information buffer 17 have been copied (step S1111 → step S1102: Yes → repeat S1103 to S1111).

If there are two or more pairs in the group, steps S1101 to S1111 are repeated.

As described above, in the second embodiment, by adding a pre-copy process for areas with differences (step S1106: Yes → steps S1107 and S1108), the order assurance data D1 before the order assurance copying was stopped can be secured in the evacuation volume 29. Also, by performing differential copying from volume 18 to volume 28 in parallel for each area without waiting for the copy from volume 28 to the evacuation volume 29 to be completed, the time until the order assurance copying is resumed can be shortened. Note that, as described above, if the size of each area of the difference information buffer 17 and the size of each area of the difference information buffer 27 are made different, the determination process shown in FIG. 25 is made to take the difference in size into consideration.

[Third embodiment]
A third embodiment of a storage system according to the present invention will be described with reference to Fig. 25. Fig. 25 is a flowchart showing an example of an operation when stopping order guarantee copying in the third embodiment of the present invention. The process shown in Fig. 25 corresponds to a combination of the process shown in Fig. 12 and the process shown in Fig. 17 in the first embodiment.

In the first embodiment, when resuming order-guaranteed copying, data with differences in volume 28 is copied to evacuation volume 29 before the resumption (step S701 in FIG. 16, FIG. 17). On the other hand, in the third embodiment, when order-guaranteed copying is stopped, copying of volume 28 to evacuation volume 29 is executed after the stop, for example before the conditions for restart are met. Other configurations and operations are basically the same in the first and third embodiments. Below, the parts that differ from the first embodiment are explained.

In the process shown in FIG. 25, after the order assurance processing unit 13 stops the order assurance copy, it instructs the difference management processing unit 26 at the remote site 2 to copy data from the volume 28 to the evacuation volume 29 (steps S1201 to S1203). Note that in step S1201, the order assurance processing unit 13 stops copying to the order assurance buffer 14. In step S1202, the order assurance processing unit 13 enables the difference information buffer 27 at the remote site 2. In step S1203, the order assurance processing unit 13 instructs the start of copying from the volume 28 at the remote site 2 to the evacuation volume 29.

The difference management processing unit 26 copies data D1 from the volume 28 to the save volume 29 for the area with difference in the difference information buffer 27 (steps S1204 to S1207). At this time, the difference information buffer 27 is set to "no difference" for the area where copying has been completed (step S1207). The processing of steps S1204 to S1207 is repeated until copying of all areas with difference in the difference information buffer 27 is completed (until step S1205: No). In step S1204, the difference management processing unit 26 calculates the area with difference from the difference information buffer 27. In step S1205, the difference management processing unit 26 determines whether or not there is an area with difference in the difference information buffer 27. In step S1206, the difference management processing unit 26 reads data of the area with difference from the volume 28 and copies it to the save volume 29. In step S1207, the difference management processing unit 26 sets "no difference" in the difference information buffer 27.

If there are two or more pairs in "Group 3," steps S1204 to S1207 are repeated for all pairs.

In the third embodiment, the above process is performed from the time when the order-guaranteed copy is stopped, thereby shortening the time it takes to copy data to the evacuation volume 29 when the order-guaranteed copy is resumed.

[Actions and Effects of Each Embodiment]
In the in-order copy operation, when the stopped operation is resumed, the secondary volume at the remote site may temporarily be in an in-order-guaranteed state. If, for example, the main site becomes unavailable during this period when the in-order is not guaranteed, it may be difficult to resume operations using the volume of the storage at the remote site. For this reason, maintaining in-order guaranteed data and reducing the time required to maintain the data may be an issue. In response to this, in each embodiment of the present invention, before copying the data of the primary volume 18 to the secondary volume 28 at the remote site 2, the data of the volume 28 is copied and evacuated to the evacuation volume 29 using the difference information buffer 27 at the remote site 2. As a result, the data of the volume 28 in an in-order guaranteed state is maintained in the evacuation volume 29. That is, in the storage system of this embodiment, when the copy operation that guarantees the order from the primary volume to the secondary volume is stopped, before copying each data that is determined to have a difference between the primary volume and the secondary volume during the stop period from the primary volume to the secondary volume, each data that is determined to have a difference between the secondary volume and the save volume during the copy operation that guarantees the order from the primary volume to the secondary volume before the stop is copied from the secondary volume to the save volume, and then each data that is determined to have a difference is copied from the primary volume to the secondary volume, and then the copy operation is resumed. Therefore, according to this embodiment, when the copy operation of data that guarantees the order from the primary volume to the secondary volume is stopped and then resumed, the data of the secondary volume can be appropriately saved. Also, in the second embodiment, the time until the order guarantee is resumed can be shortened by simultaneously copying from the volume 28 to the save volume 29 and copying from the volume 18 to the volume 28.

[Computer Configuration]
FIG. 28 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 700 includes a CPU (Central Processing Unit) 710, a main storage device 720, an auxiliary storage device 730, and an interface 740. For example, a non-volatile storage medium 750 is connected to the interface 740. The above-mentioned host computer 100, host computer 200, storage 10, storage 20, storage controller SC1, storage controller SC2, storage array 30, and storage array 40 may have the same configuration as the computer 700. The operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730 and deploys it in the main storage device 720, and executes the above-mentioned processing according to the program. The CPU 710 also secures a storage area corresponding to each of the above-mentioned storage units in the main storage device 720 according to the program.

The program may be for implementing some of the functions to be performed by the computer 700. For example, the program may be implemented by combining it with other programs already stored in the auxiliary storage device 730, or by combining it with other programs implemented in other devices. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions implemented by the CPU 710 may be implemented by the integrated circuit.

Examples of the auxiliary storage device 730 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), and a semiconductor memory. The auxiliary storage device 730 may be an internal medium directly connected to the bus of the computer 700, or an external medium connected to the computer 700 via the interface 740 or a communication line. In addition, when the program is distributed to the computer 700 via a communication line, the computer 700 that receives the program may expand the program in the main storage device 720 and execute the above-mentioned processing. In at least one embodiment, the auxiliary storage device 730 is a non-transitory tangible storage medium.

[Modifications, etc.]
Although the embodiment of the present invention has been described above with reference to the drawings, the specific configuration is not limited to the above embodiment, and includes design changes and the like within the scope of the gist of the present invention. In addition, a part or all of the program executed by the computer in the above embodiment can be distributed via a computer-readable recording medium or a communication line.

[Minimum configuration of embodiment]
Fig. 26 is a block diagram showing an example of a minimum configuration of a configuration according to an embodiment of the present invention. Fig. 27 is a flowchart showing an example of a minimum configuration of an operation according to an embodiment of the present invention.

As shown in FIG. 26, the storage system 60 according to the embodiment of the present invention may include at least a first storage 61 and a second storage 62. The first storage 61 includes a primary volume 611. The second storage 62 includes a secondary volume 621 that is connected to the first storage 61 via a communication line 71 and that is set as a copy pair with the primary volume 611. The first storage 61 further includes a first storage unit 612 and a second storage unit 613. The first storage unit 612 temporarily holds data that is copied from the primary volume 611 to the secondary volume 621 with the order guaranteed. The second storage unit 613 stores first difference information that indicates whether there is a difference in data for each of the first storage areas A, B, C, ... for the primary volume 611 and the secondary volume 621. The second storage 62 further includes a save volume 622 and a third storage unit 623. The save volume 622 copies and stores all data of the volume 621. The third storage means 623 stores second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data from the primary volume 611 to the secondary volume 621 via the first storage means 612 for each of the specified second storage areas a, b, c, ... for the secondary volume 621 and the save volume 622.

When the copy operation that ensures the order of data from the primary volume 611 to the secondary volume 621 via the first storage means 612 is stopped, the storage system 60 copies each piece of data that is determined to have a difference based on the second difference information from the secondary volume 621 to the evacuation volume 622 before copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means 613 during the stopped period from the primary volume 611 to the secondary volume 621 (step S1).

The storage system 60 then copies each piece of data that is determined to have a difference based on the first difference information stored in the second storage means 613 during the suspension period from the primary volume 611 to the secondary volume 621 (step S2).

The storage system 60 then resumes the copy operation that ensures data order from the primary volume 611 to the secondary volume 621 via the first storage means 612 (step S3).

As shown in FIG. 27, the control method for the storage system 60 according to an embodiment of the present invention may include at least steps S1, S2, and S3.

The correspondence between the components of the present invention and the components of the embodiment is as follows: "Storage system" corresponds to storage system 3. "First storage" corresponds to storage 10. "Second storage" corresponds to storage 20. "Primary volume" corresponds to primary volume 18. "Communication line" corresponds to remote line 50. "Secondary volume" corresponds to secondary volume 28. "Evacuation volume" corresponds to evacuation volume 29. "First storage means" corresponds to order guarantee buffer 14. "First storage area" corresponds to areas A (RA), B (RB), C (RC), and D (RD). "Second storage means" corresponds to difference information buffer 17. "First difference information" corresponds to information indicating whether there is a difference or not in areas A (RA), B (RB), C (RC), and D (RD). "Second storage area" corresponds to areas A (RA), B (RB), C (RC), and D (RD). The "second difference information" corresponds to information indicating whether there is a difference or not for areas A (RA), B (RB), C (RC), and D (RD). The "third storage means" corresponds to the difference information buffer 27. Note that the above correspondence is a correspondence with one configuration example of the components of the present invention, and does not limit the components of the present invention.

[Additional Notes]
A part or all of the above-described embodiments can be described as, but are not limited to, the following notes.

(Appendix 1)
a first storage including a primary volume;
a second storage connected to the first storage via a communication line and including a secondary volume set as a copy pair with the primary volume;
the first storage further comprises a first storage means for temporarily storing data copied from the primary volume to the secondary volume with order guaranteed, and a second storage means for storing first difference information indicating whether or not there is a difference in data between the primary volume and the secondary volume for each predetermined first storage area;
the second storage further comprises a save volume for copying and storing all data of the secondary volume, and a third storage means for storing, for each predetermined second storage area, second difference information for the secondary volume and the save volume, which indicates whether or not there is a difference that has occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume via the first storage means;
When the copy operation is stopped, before copying each piece of data that is determined to have a difference based on the second difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, each piece of data that is determined to have a difference based on the second difference information is copied from the secondary volume to the save volume;
Then, each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the suspension period is copied from the primary volume to the secondary volume;
The storage system then resumes the copy operation.

(Appendix 2)
copying each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume;
copying each piece of data that is determined to have a difference based on the first difference information from the primary volume to the secondary volume;
2. The storage system according to claim 1, wherein the first storage area and the second storage area are processed in parallel.

(Appendix 3)
The storage system according to claim 1 or 2, wherein when the copy operation is stopped, copying of each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume is performed after the copy operation is stopped.

3, 60...storage system, 10, 20...storage, 100, 200...host computer, SC1, SC2...storage controller, 30, 40...storage array, 11, 21...I/O processor, 12, 22...cache memory, 13, 23...order guarantee processor, 14, 24...order guarantee buffer, 15...remote transmission processor, 16, 26...difference management processor, 17, 27...difference information buffer, 18, 611...primary volume, 25...remote reception processor, 28, 621...secondary volume, 29, 622...evacuation volume, 50...remote line, 61...first storage, 62...second storage, 612...first storage means, 613...second storage means, 623...third storage means, 71...communication line

Claims (8)

a first storage including a primary volume;
a second storage connected to the first storage via a communication line and including a secondary volume set as a copy pair with the primary volume ;
Equipped with
the first storage further comprises a first storage means for temporarily storing data copied from the primary volume to the secondary volume with order guaranteed, and a second storage means for storing first difference information indicating whether or not there is a difference in data between the primary volume and the secondary volume for each predetermined first storage area;
the second storage further comprises a save volume for copying and storing all data of the secondary volume, and a third storage means for storing, for each predetermined second storage area, second difference information for the secondary volume and the save volume, which indicates whether or not there is a difference that has occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume via the first storage means;
When the copy operation is stopped, before copying each piece of data that is determined to have a difference based on the second difference information stored in the second storage means during the stop period from the primary volume to the secondary volume, each piece of data that is determined to have a difference based on the second difference information is copied from the secondary volume to the save volume;
Then, each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the suspension period is copied from the primary volume to the secondary volume;
The storage system then resumes the copy operation. copying each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume;
copying each piece of data that is determined to have a difference based on the first difference information from the primary volume to the secondary volume;
The storage system according to claim 1 , wherein the first storage area and the second storage area are processed in parallel. 2. The storage system according to claim 1, wherein when the copy operation is stopped, copying of each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume is performed after the copy operation is stopped. A first storage device includes a primary volume and is connected via a communication line to a second storage device includes a secondary volume that is set as a copy pair with the primary volume,
the first storage further comprises a first storage means for temporarily storing data copied from the primary volume to the secondary volume with order guaranteed, and a second storage means for storing first difference information indicating whether or not there is a difference in data between the primary volume and the secondary volume for each predetermined first storage area;
the second storage comprises: the secondary volume; a save volume for copying and storing all data of the secondary volume; and a third storage means for storing, for each predetermined second storage area, second difference information for the secondary volume and the save volume, which indicates whether or not there is a difference that has occurred during a copy operation that ensures data order from the primary volume to the secondary volume via the first storage means;
When the copy operation is stopped, each piece of data that is determined to have a difference based on the second difference information in the second storage is copied from the secondary volume to the save volume, and then each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stoppage period is copied from the primary volume to the secondary volume;
Then, each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the suspension period is copied from the primary volume to the secondary volume;
Then, the copy operation is resumed.
Storage. A second storage device is connected to a first storage device having a primary volume via a communication line, and has a secondary volume set as a copy pair with the primary volume,
the first storage comprises: the primary volume; a first storage means for temporarily holding data copied from the primary volume to the secondary volume with order guaranteed; and a second storage means for storing first difference information indicating whether or not there is a difference in data between the primary volume and the secondary volume for each predetermined first storage area;
the second storage further comprises a save volume for copying and storing all data of the secondary volume, and a third storage means for storing, for each predetermined second storage area, second difference information for the secondary volume and the save volume, which indicates whether or not there is a difference that has occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume via the first storage means;
When the copy operation is stopped, before each piece of data that is determined to have a difference based on the second difference information stored in the second storage means during the stop period is copied from the primary volume to the secondary volume, each piece of data that is determined to have a difference based on the second difference information is copied from the secondary volume to the save volume;
Then, each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the suspension period is copied from the primary volume to the secondary volume;
Then, the copy operation is resumed.
Storage. a first storage including a primary volume;
a second storage connected to the first storage via a communication line and including a secondary volume set as a copy pair with the primary volume ;
A control method for controlling
the first storage further comprises a first storage means for temporarily storing data copied from the primary volume to the secondary volume with order guaranteed, and a second storage means for storing first difference information indicating whether or not there is a difference in data between the primary volume and the secondary volume for each predetermined first storage area;
the second storage further comprises a save volume for copying and storing all data of the secondary volume, and a third storage means for storing, for each predetermined second storage area, second difference information for the secondary volume and the save volume, which indicates whether or not there is a difference that has occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume via the first storage means;
when the copy operation is stopped, copying each piece of data that is determined to have a difference based on the second difference information from the secondary volume to the save volume before copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stop period from the primary volume to the secondary volume;
thereafter, copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the suspension period from the primary volume to the secondary volume;
and thereafter, resuming the copy operation. a first storage including a primary volume;
a second storage connected to the first storage via a communication line and including a secondary volume set as a copy pair with the primary volume ;
Equipped with
the first storage further comprises a first storage means for temporarily storing data copied from the primary volume to the secondary volume with order guaranteed, and a second storage means for storing first difference information indicating whether or not there is a difference in data between the primary volume and the secondary volume for each predetermined first storage area;
a computer of the first storage in a storage system comprising: the second storage, the secondary volume; a save volume for copying and storing all data of the secondary volume; and a third storage means for storing second difference information for each predetermined second storage area for the secondary volume and the save volume, the second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume via the first storage means,
a means for copying, when the copy operation is stopped, each piece of data that is determined to have a difference based on the second difference information in the second storage from the secondary volume to the save volume, after the data has been copied from the secondary volume to the save volume, each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the stoppage of the copy operation, from the primary volume to the secondary volume;
thereafter, a means for copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the suspension period from the primary volume to the secondary volume;
Thereafter, means for resuming said copying operation;
A program that functions as a a first storage including a primary volume; and a second storage connected to the first storage via a communication line and including a secondary volume that is set as a copy pair with the primary volume;
the first storage further comprises a first storage means for temporarily storing data copied from the primary volume to the secondary volume with order guaranteed, and a second storage means for storing first difference information indicating whether or not there is a difference in data between the primary volume and the secondary volume for each predetermined first storage area;
a computer of the second storage in a storage system comprising: the secondary volume; a save volume for copying and storing all data of the secondary volume; and a third storage means for storing second difference information for each predetermined second storage area for the secondary volume and the save volume, the second difference information indicating whether or not there is a difference that occurred during a copy operation that ensures the order of data from the primary volume to the secondary volume via the first storage means,
a means for copying, when the copy operation is stopped, each piece of data determined to have a difference based on the second difference information from the secondary volume to the save volume before each piece of data determined to have a difference based on the first difference information stored in the second storage means during the stoppage period is copied from the primary volume to the secondary volume;
thereafter, a means for copying each piece of data that is determined to have a difference based on the first difference information stored in the second storage means during the suspension period from the primary volume to the secondary volume;
Thereafter, the means for resuming the copying operation
A program that functions as a

Images