JP2994070B2 - Method and apparatus for determining the state of a pair of mirrored data storage units in a data processing system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

FIELD OF THE INVENTION The present invention relates to maintaining a mirror copy of computer data in a redundant data storage unit, and more particularly, to determine which unit contains the current data when the system is reinitialized. About deciding.

[0002]

2. Description of the Related Art The demand for increasing the data storage capacity of modern computer systems requires large-capacity data storage devices. Such devices are typically magnetic disk drives, which are mechanically complex with many components that are prone to failure. A typical computer system includes several such units. As the user's data storage requirements increase, the number of such storage units in the system increases. A single storage unit failure can be very fatal to the system. Many systems cannot operate until the failed unit has been repaired or replaced and the lost data has been recovered. Increasing the number of storage units increases the probability of a single unit failing, causing a system failure. At the same time, computer users will be more dependent on the consistent availability of their systems. Therefore, an improved method of maintaining system operation in the event of a single storage unit failure and returning the system to a normal operating mode when the failure condition is corrected is important.

One method for solving this problem is the "mirror" method. This method maintains a duplicate set of storage devices containing the same data as the original. This duplicate set can be used to perform the task of providing data to the system when one unit in the original set fails. One system can have a duplicate set of all stored data (fully mirrored) or a duplicate set of some subset of that data (partially mirrored). This mirror is becoming important when computer users need improved system reliability and availability.

[0004] Users of systems that include mirror storage expect the highest reliability from the storage. Since the nature of the mirror is that if one storage unit fails, the other unit can be used instead, the system can necessarily operate such that only one of the pair of mirror units functions. Must be something. When both units in the mirror pair are functioning and contain current data, they are said to be synchronized. If one of the storage units in the mirror pair fails and the other continues to operate, the data in the failed unit immediately becomes outdated. A "failure" of a unit means that data cannot be read from or written to the unit. This may be interpreted to mean that the storage unit itself does not work or that other elements of the system, such as the I / O processor, fail. Recovering the operation of the failed storage unit leaves the data on the storage medium intact, such as when a circuit card containing control logic is replaced.

Since the system can operate even when the disk units forming the mirror pair are out of synchronization, it is necessary to know the state of the mirror pair, that is, which of them currently contains data. If the system fails for any reason, the state of its storage unit must be able to be reconfigured when the power is restored and the system itself is reinitialized. If the system is shut down and the failed storage unit is repaired or replaced, reinitialization causes the operating system to verify that the data contained in the repaired or replaced unit is unreliable and to replace those units. The process of resynchronization must be initiated so that the repaired or replaced unit can be matched with that of the unit that did not fail.

[0006] One way to know the status of the storage units forming a mirror pair is to store status information in both units. The system reads this state information upon re-initialization. If both units are working and the stored state information of both units is synchronized with each other, the system determines the state. If one storage unit fails while all other devices of the operating system are operating properly and the system itself is fully operational, this operating system must ensure that only the non-failed unit has current data. And record the new status information in the fault-free unit. When reinitialized after repair, the status information of the unit that has not failed contains only the current data, and the status record of the unit that failed at that time indicates that both units have synchronized or some unknown condition. Is shown. The operating system may then determine that only non-faulty units contain the current data.

However, for one of the mirrored units during system reinitialization, there is generally no report that both units are synchronized while the other is not responding. In this case, the system cannot reliably determine the state of the mirror pair. It may be recognized that both units are synchronized by the corresponding unit when the system power is reduced. This state is, for example, A
This can happen if the unit fails, is repaired without losing its stale data, the system is reinitialized, and the B unit does not respond. Failure to respond during reinitialization does not necessarily mean that the storage unit has been destroyed. It is possible that the power switch will be turned off or any number of other situations will interfere with the response of the unit, especially if repairs are performed during system power down.

[0008] In the above situation, the operating system may not be able to make a state decision, or may make an incorrect state decision. If the operating system cannot make a status decision, it will generally ask the user for the correct status. Interrogating the user is a very unreliable method for status determination because of the large number of storage units and because the association of a logical address to a physical location need not always be apparent. Predicting the state or not knowing the state is clearly undesirable for a mirrored or fault tolerant computer system because the user does not have the reliability and availability that one would expect.

It is therefore an object of the present invention to provide an extended method and apparatus for determining the status of a mirrored data storage unit.

It is another object of the present invention to provide an extended method and apparatus for determining the state of a mirrored data storage unit when multiple devices fail.

It is another object of the present invention to provide greater redundancy and reliability to information tracking the state of a mirrored storage unit in a data processing system.

Another object of the present invention is to reduce human error,
It is to provide a method and apparatus for determining a state of a mirror pair data storage unit.

[0013]

SUMMARY OF THE INVENTION A data processing system having a pair of mirrored storage units maintains a record of the state of this unit pair ("state record") in system memory. This state information is also stored in separate state records, one for each unit, so that the state when the system is reinitialized can be determined. The state of the mirror pair is also stored in another state record in another location. In the preferred embodiment, this additional status record is stored in non-volatile RAM. When the state changes, the operating system updates the new state,
Write to the storage unit that is still functioning and another status record as described above . Only limited state changes are allowed to prevent ambiguous situations.

When the system is reinitialized, the system reads the status information stored in the storage unit. If no unit can be read, the system retrieves the status from another record and replaces it as read from the non-responsive unit. A pair of states read from the two units (or replaced by another state record) index a unique entry in the state fetch table. The input contained in the location indexed in these two fetched states is the correct current state of these mirrored storage units, which is loaded into memory for system operation.

In a preferred embodiment, the system includes a number of mirrored storage units. The pair is a "load source" and contains the data needed to re-initialize the system. Each unit of the load source pair includes a status record called an "I / O device configuration table". The input / output device configuration table of the load source unit includes the status of all storage units of the system. The other status record includes the status of the load source pair, but does not include the status of other storage units. When the system is reinitialized, the system determines the state of the load source pair as described above. Once the status of the load source pair is known, the status of the remaining devices can be obtained directly from the input / output device configuration table of any current load source unit.

[0016]

FIG. 1 is a block diagram of the main components of a computer system 100 according to a preferred embodiment of the present invention. FIG.
A suitably programmed system processor 101 communicates with one or more I / O processors 120 and 130 via a system bus 102 as shown at 10 through 10. The main memory 103 is connected to the system bus 102 and directly to the system processor 101. Operating system 1 including input / output device configuration table 106 and state derivation table 116 in normal operation
05 is in the main memory 103. A non-volatile RAM 104 containing system critical data is connected to the system processor 101. The non-volatile RAM 104 contains another status record 115 of the mirror-to-unit status of the load source unit. Another status record 115 is the system serial number 107, load source unit 10
9 mirror unit status, load source unit A 11
0 I / O address and load source unit B1
11 I / O addresses. The non-volatile RAM 104 may further include a serial number 108 of the non-volatile RAM circuit card itself. The I / O processors 120 and 130 include one or more storage units 121, 122, 131 and 13
2 given. In this embodiment, the storage unit 121,
Although 122, 131 and 132 are magnetic disk drive units, other storage units such as optical disks and tapes may be used. The storage units A121 and B131 form one mirror pair and store the same data. In this embodiment, the units A121 and B131 are provided to the I / O processors 120 and 130, respectively, to increase the redundancy of the system. However, these units can also be provided to the same processor. Although two I / O processors 120 and 130 are shown, this system
Including more than one I / O system may also involve one I / O system.
An O processor may be included. Also, one I / O
The number of storage units provided to the processor is variable. In this embodiment, the computer system 100 is an IB
Although it is a MAS / 400 computer system, other systems may be used.

The respective storage units 121, 122, 13
Reference numerals 1 and 132 include a logical identifier 140, a system serial number 141, and a status record 143 which is an input / output device configuration table. Identifier 140 identifies the disk unit to the system. The system serial number 141 is the serial number of the system to which the unit is given. In another embodiment, the disk units further include the serial number 142 of the non-volatile RAM circuit card 104.

FIG. 2 shows details of the input / output device configuration table. This includes information identifying the configuration and status of the storage unit. Each storage unit contains a complete I / O device configuration table with entries for all storage units provided to the system. Each record 20 in this table
1, 202 corresponds to a pair of mirrored disk units or one unmirrored unit. This table is indexed by the logical unit number 210 of those pairs or non-mirrored units. In this embodiment, logical unit number 1 is stored for a load source pair or unit. Other logical unit numbers are arbitrary. When this logical unit number is linked with the destination A or B, the logical identifier 14 of one disk unit is used.
0. In addition to the logical unit number 210, as shown in FIG.
11. The pair is divided into three sub-records: a sub-record 212 for data unique to the unit A and a sub-record 213 for data unique to the unit B.
The common data sub-record 211 has a mirroring flag 215
And the mirroring unit status 214. The mirror flag 215 is a one-bit field and is used to indicate whether the record is for a pair of mirrored disk units or one non-mirrored unit. “1” indicates that the record is a pair of mirror units. Mirrored unit status 214 is 1
Byte field indicating the current state of the mirror pair.
If the mirroring flag is set to "0" (not mirrored), the mirror unit status is ignored. In the example of FIG. 2, the record 201 for the logical unit number 1 is a disk unit that is a mirror pair, and the current state indicates that both units are operating normally, and the logical unit number The third record 202 is for one unmirrored unit. Mirror all or some of the disk units in a particular system because each unit number has its own entry in the I / O device configuration table to indicate whether it is a mirror pair. Can be done or not mirrored at all. Units A and B
Sub-records 212 and 213 for each of
It contains address fields 216, 218 and unit serial numbers 217, 219 of the respective disk units. If the record is for one non-mirrored disk unit as shown for record 202, the information for that unit is stored in subrecord 212 of unit A 'and subrecord 213 of unit B is used. I can't. Sub record 21
1, 212, 213 can include other fields used for other purposes.

Five states can be stored in the mirrored unit status field 214 of the input / output device configuration table. The states and their associated meanings are as follows. The A and B units in the ABOK mirror pair synchronize with the current data. AOK Only the A unit contains the current data, the B unit is unreliable. Only the BOK B unit contains the current data, the A unit is unreliable. AOKRB A unit currently contains data and B unit is in the process of being resynchronized with A unit. BOKRA B unit currently contains data and A unit is in the process of being resynchronized with B unit. This system does not store the "unknown" status in the mirror unit status field.

During normal system operation, the operating system maintains a copy of the I / O device configuration table in the main memory 103 (FIG. 1), against which the system checks when status information about the storage unit is needed. . When there is a change in the state of the storage unit, the new state is written to the I / O device configuration table in the memory and the I / O device configuration table of each operation storage unit. If this state change involves a load source disk unit, the new state is
It is also written in M104. The following rules must be consulted during a state change to ensure that the system does not go into an indeterminate state. 1. The system must not back level both mirrored units in a pair. If both units fail, the system itself is stopped. In this case, the last failed unit contains the current data. 2. The system does not write a new mirror state to the failed disk unit. 3. Only the following state changes are allowed: ABOK to AOK (B unit failure) ABOK to BOK (A unit failure) AOKRB to AOK (B unit cannot be resynchronized) BOKRA to BOK (A unit resynchronization) AOK to AOKRB (B unit resynchronization start) BOK to BOKRA (A unit resynchronization start) AOKRB to ABOK (B unit resynchronization success) BOKRA to ABOK (A unit) When computer system 100 is reinitialized, the system first retrieves the mirrored unit state of the load source pair from the mirrored state data of the two load source units and the non-volatile RAM. One state is associated with each of these load source units. The state "associated with" one load source is the state stored in the unit if it can be read, and the nonvolatile RAM if the state stored in the storage unit cannot be read.
This is the state stored in another state record 115 in 104. There is a unique resulting state of the mirrored load source pair corresponding to the pair of states associated with the A unit and the B unit. This result state is the true state of the load source pair, and is limited by the input / output device configuration table 301 in FIG. The input / output device configuration table 301 stored in 116 of the operating system 105 in the main memory 103 includes an input 302 associated with the load source unit A and an input 3 associated with the load source unit B.
03 is required. The system retrieves the state of the mirror load source pair by accessing the entries of the column corresponding to the state associated with unit A and the row corresponding to the state associated with unit B of this table. “Unknown” input states 304 and 305 indicate the disk unit or the non-volatile RA
Not read from M, disk unit or non-volatile R
Used only when none of the AMs can be read.
The entry of this table marked with “*” is not possible.
Input of " ^* AOK" and " ^* BOK" is not possible when using the state read from a disk unit, but non-volatile RA
This is possible when using the state read from M. " ^**
"AOK" and " ^** BOK" are input when B is faulty.
It occurs only for a state change from RA to ABOK, or for a state change from AOKRB to ABOK when A is faulty. Computer system 100 recognizes and processes the two-step change.

As an example of the use of I / O device configuration table 301 when unit A contains state ABOK, unit B does not respond, and non-volatile RAM contains state BOK, the system was read from non-volatile RAM. The status BOK is replaced with the status read from the B unit. Once these values have been entered into this table, the resulting state is contained in column ABOK and row BOK. The result state is BOK. This is the situation that occurs when the A unit has failed and its data has become stale, the system power has dropped, the A unit has resumed operation, and the B unit has not responded during system reinitialization.

FIGS. 4-10 show details of the steps of the reinitialization procedure (IPL) according to the present invention. FIG. 4 shows the input / output device configuration task. The process begins by reading the storage unit's logical identifier from one of the load source units 402. This load source unit accessed at this initial stage is referred to herein as a primary load source unit. In this embodiment, this primary load source unit is the first unit to respond to a poll, and is a logical A or B unit. The process then loads the input / output device configuration table (DCT) from main load source unit 403 into main memory. Next, at 404, a loop is entered to determine the status of all units. The storage unit reports in a random pattern as it enters the line. The process waits for each unit to report at 408, obtains the logical identifier of the reporting unit at 409, and calls the search mirroring unit status routine at 410 when this unit is one of the mirror pairs 405.

This search mirroring unit status routine 4
Numeral 10 returns the status of the reporting unit based on information available about the system at that time. On return from routine 410, if both load source units are reporting at 411, and also if the called subroutine returns the state for the load source pair and at 411 the primary load source does not contain the current data, At 413, the system restarts its reinitialization procedure with another load source unit. If not, the process returns to the loop 404.

The details of the routine 410 are shown in FIG. In this routine 421, if this unit is a load source unit, the load source state is acquired (Get Load Source St
ate) Branch to the routine 422. Otherwise, the I / O device configuration check table routine 423 is called. If this routine 423 is DCT valid at 424, routine 410 restores the state value of the mirrored pair stored in the DCT at 425, otherwise returns to an unknown state at 426.

FIG. 6 shows the load source state acquisition routine called at 422. This is followed by NVRA at 431
The M check routine, the storage state acquisition routine for disk unit A at 432, and the storage state acquisition routine for disk B at 433, to retrieve the load source state from the state returned from two calls to the storage state acquisition routine. The state derivation table 301 is accessed. These returned status values are the statuses associated with load source units A and B.

FIG. 7 shows the NVRAM inspection routine. This routine performs a simple validity check and sets a flag indicating whether NVRAM is available. One reason for performing this check is that the non-volatile RAM unit (in this case, the circuit card) has been replaced and may therefore be insufficient to respond for NVRAM to determine the state of the load source unit. It is. In this routine, the data in NVRAM and 1
Compare with the data read from the next load source disk unit. If the NVRAM and the primary load source unit do not contain the same system serial number at 442 or do not contain the same I / O address as the primary load source unit at 443,
VRAM is not enabled. NVRAM is 4
If it contains an invalid value for the state of the load source unit at 44, it cannot be used.

FIG. 8 shows the storage state acquisition routines 432 and 433. If a particular disk unit has reported, this routine branches at 451 to return to the state read from that disk unit if it reported at 452. If this unit did not report, check if NVRAM is available.
Check in 3. 45 if NVRAM is available
At 5 the state stored in NVRAM is restored, otherwise at 454 the "unknown state" is restored.

The DCT checking routine called at 423 is shown in FIG. This routine sets a flag indicating whether DCT is enabled. If the load source is not a mirror pair, the routine branches at 461 to immediately set DCT enabled. Otherwise 462
Calls the load source status acquisition routine to obtain the status of the primary load source device. This is the device that first loaded the DCT table into memory. When the load source state acquisition routine returns to a state indicating that the primary load source device has current data, the process branches at 463 to set DCT enabled. Otherwise, set DCT to disabled. Then return to the calling routine.

The input / output device configuration task shown in FIG. 4 continues indefinitely. At the same time, the main IPL task shown in FIG. 10 starts. After the first part of the IPL task has been completed at 471, the task returns the report to all requested storage units, ie, at least one storage unit containing the current data for all of the stored data in the system. It is checked at 472 whether or not it has been performed. If so, the IPL is allowed to continue at 473; otherwise, the error handling routine is called.

For example, if the system is functioning in all respects, and both load source units now contain data but the B unit is turned off, then the system does the following: That is, an input / output device configuration task is entered, and an identifier, a system serial number, and a DCT are read from the unit A at 402 and 403. Since unit A is a mirror at 405, a search mirroring unit status routine is called at 410. It calls the load source state acquisition routine 422 at 421. The get load source state routine calls the NVRAM check routine at 431, which returns NVRAM to valid. Next, at 432, the storage state acquisition routine for the unit A is called, and this returns to the ABOK state. Next, the storage state acquisition routine of the unit B is called at 433. Since unit B did not report at 451, and NVRAM was enabled at 453, AB
Return the status to NVRAM that is OK. The Load Source State Acquisition Routine checks 434 the resulting state of the state fetch table that is ABOK. Next, the process returns to the mirroring unit status search routine (FIG. 5), which returns to the input / output device configuration task (FIG. 4). Since both load source units have not reported at 411, the task returns to the loop at 404 and waits for a report of the next unit at 408. After a sufficient amount of time has elapsed, the main IPL task doing other work at 471 checks at 472 if there is at least one current unit for all data in the system. The A unit allows the IPL to continue at 473 since it was determined to be current.

In this embodiment, the system is designed so that any number of storage units can be mirrored. For example, it is possible to mirror some of these units and not mirror the load source unit. However, this means that upon re-initialization, the only source of mirrored state information will be one load source unit, losing much of the redundancy provided by the mirrored storage means.

One input / output device configuration table is stored in every disk unit. A copy of this table in a unit other than the load source unit is not used in the reinitialization procedure described above, and exists only for storage purposes. In other embodiments, these copies of the DCT can be used instead of the load source state stored in non-volatile RAM. In yet another embodiment, rather than using DCT for the status information of all other units, the status of each mirrored disk unit pair is determined using the same procedure used to determine the status of the load source unit. Can be determined individually. Since these units are brought online and randomly report, in yet another embodiment the state determination procedure is first applied to the mirrored pair reporting and then the state of the remaining mirror pairs is determined. The status information from the input / output device configuration table of one of the current units of the mirror pair performing the first report is used.

In the preferred embodiment of the present invention, another status record is stored in non-volatile RAM. This record indicates that if the data persists when the system is powered down,
It may be stored at any location other than the two load source units. For example, it may be stored on a third disk storage unit of the same type as the load source unit, tape, or other computer system remote or directly connected to the computer system 100.

In yet another embodiment, the serial number of a non-volatile RAM may be stored on the RAM itself and on a disk unit. These serial numbers are compared in the NVRAM test routine for other levels of redundancy in the test of the non-volatile RAM.

[Brief description of the drawings]

FIG. 1 is a block diagram of a system that uses a mirrored storage element of the present invention.

FIG. 2 is a diagram showing an input / output device configuration table included in one of the storage units or a main memory.

FIG. 3 is a diagram showing a status retrieval table used to determine the status of a pair of mirrored storage units according to the present invention.

FIG. 4 is a flowchart showing a routine of an input / output device configuration task.

FIG. 5 is a flowchart showing a mirroring unit state search routine in FIG. 4;

FIG. 6 is a flowchart showing a routine for calling a load source state in FIG. 5;

FIG. 7 is a flowchart showing an NVRAM inspection routine in FIG. 6;

FIG. 8 is a flowchart showing a routine for acquiring a storage state of units A and B in FIG. 6;

FIG. 9 is a flowchart showing a DCT inspection routine in FIG. 5;

FIG. 10 is a flowchart showing a routine of a main IPL task.

[Explanation of symbols]

 Reference Signs List 100 computer system 101 system processor 102 system bus 103 main memory 104 nonvolatile RAM 106 input / output device configuration table 107 system serial number 108 nonvolatile RAM circuit card 109, 110, 111 load source unit 116 status extraction table 120, 130 I / O Processor

 ────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventors Michael, James, McDermott Minnesota, U.S.A., Oronoko, One Hundred, And, Fifthines, Street, N, W JP, A) JP-A-62-139172 (JP, A)

Claims (10)

(57) [Claims] 1. A method for determining a state of a pair of mirrored data storage devices in a data processing system, the method comprising: determining a state of a mirrored pair of data storage devices from a first storage device of the pair of mirrored data storage devices; Retrieving a first status record indicating a status; retrieving a second status record indicating a status associated therewith from a second storage device of the pair of mirrored data storage devices; Retrieving another status record from another device from the device; and if the status record cannot be retrieved from the data storage device, the status associated with one of the data storage devices being included in the another status record. Replacing with the information, the pair of mirrored data associated with the first and second data storage devices. It said method comprising the steps of determining the state of the data storage device. 2. The method of claim 1, wherein the step of determining a state of the pair of mirrored data storage devices comprises the step of determining the state of the mirrored pair from a state retrieval table corresponding to a state associated with the first and second storage devices. The method of claim 1 including retrieving the state of the data storage device. 3. The method according to claim 1, wherein said another device is a nonvolatile random access memory. 4. A method for determining the status of a pair of mirrored data storage devices in a data processing system having at least two pairs of mirrored data storage devices, the method comprising: From storage,
Retrieving a first status record to obtain a status associated therewith; retrieving a second status record from a second storage device of the first pair of storage devices to obtain a status associated therewith; Retrieving another status record from a different device than the storage device pair; and when the status record cannot be retrieved from the storage device, the status related to one of the storage devices of the first storage device pair is changed to the another status. Replacing the first storage device with status information included in a record; and determining a status corresponding to a status pair associated with the first and second storage devices in the first storage device pair.
Determining a state of the pair of storage devices; and extracting a state of each mirrored state device other than the first pair of storage devices from the first pair of storage devices determined to obtain the current data by the determining step. The method comprising: 5. The method of claim 1, wherein said determining step includes retrieving a state of said first mirror pair from an entry in a state retrieval table corresponding to states associated with the first and second storage devices of said first storage device pair. The method of claim 4. 6. The method according to claim 4, wherein said another device is a nonvolatile random access memory. 7. A data processing system having a pair of mirrored data storage devices, the first of said pair of data storage devices having a status record for storing a status of said pair of data storage devices. A storage device, a second storage device of the pair of data storage devices having a status record for storing a status of the pair of data storage devices, and a storage device connected to the first and second storage devices and storing them. State retrieval means for retrieving a status record to be retrieved; means for retrieving another status record from a device different from the pair of data storage devices; and when the status record cannot be retrieved from the data storage device, Means for replacing a state associated with one of the states with state information contained in the another state record; It said data processing system comprising a means, for determining a state of said mirrored pair of data storage device associated with the device. 8. The status determination means further includes a status retrieval table having a unique status entry corresponding to each of a pair of a status associated with the first storage device and a status associated with the second storage device, The data processing system of claim 7, wherein the state of the data storage device is determined by retrieving entries corresponding to both states from the table. 9. A storage device further comprising: a third storage device for storing data; and a fourth storage device for storing a copy of the data stored in the third storage device. Using the information contained in the status record stored in the second storage device, the third
9. The data processing system according to claim 7, wherein the state of the storage device is automatically determined. 10. The data processing system according to claim 7, wherein said another device is a nonvolatile random access memory.