CN100447731C - Redundant Storage Virtualization Computer System - Google Patents

Abstract

The invention provides a redundant storage virtualization computer system, which comprises: a host for sending out input/output request; a set of redundant storage virtualization controllers coupled to the host for performing IO operations in response to IO requests issued by the host; and a plurality of physical storage devices for providing storage space to the computer system. Each physical storage device is coupled to the set of redundant storage virtualization controllers, the set of redundant storage virtualization controllers comprises a first storage virtualization controller and a second storage virtualization controller, the first storage virtualization controller and the second storage virtualization controller are coupled to the host, the first storage virtualization controller and the second storage virtualization controller are communicated through a PCI-Express link, and in the set of redundant storage virtualization controllers, when one storage virtualization controller is offline, the other storage virtualization controller replaces functions originally executed by the offline storage virtualization controller.

Description

Redundant Storage Virtualization Computer System

technical field

The present invention relates to a redundant storage virtualization computer system (redundant storage virtualization computer system), in particular to a redundant storage virtualization computer system using a local bus as a connection between two storage virtualization controllers.

Background technique

The so-called storage virtualization (storage virtualization) is a technology that virtualizes physical storage space, which combines different sections of physical storage devices (PSD, physical storage devices) into logical storage that can be accessed by a host system. Body (logical storage entity)-herein referred to as "logical media unit" (LMU, logical media unit). This technology is mainly used for disk array (RAID) storage virtualization. Through this disk array technology, smaller physical storage devices can be combined into larger capacity, fault-tolerant, and high-performance logical media units.

The main purpose of the storage virtualization controller (SVC, storage virtualization controller) is to form a combined image (map) of each section of the physical storage medium into a logical media unit visible to the host system. The input/output (IO) request sent by the host system will be analyzed and interpreted after being received, and related operations and data will be compiled into an I/O request of the physical storage device. This process can be indirect, such as using cache, delay (such as: write-back (write-back)), anticipate (such as: read-ahead (read-ahead)), cluster (group) and other operations to enhance performance And other operating characteristics, so a host I/O request does not necessarily directly correspond to a physical storage device I/O request in a one-to-one manner.

An external (or stand-alone) storage virtualization controller is a storage virtualization controller connected to the host system via an I/O interface, and it can be connected to a device located outside the host system, generally In general, external storage virtualization controllers usually operate independently of the host.

The main motivation for configuring a pair of storage virtualization controllers as a redundant pair is to allow the host to continue to perform data access tasks without interruption even if a single storage virtualization controller fails or fails. This can be achieved by adding a function to the storage virtualization controllers so that in the event that one controller malfunctions or fails completely, the other controller can take over its work.

The configuration of redundant storage virtualization controller pairs is divided into two categories. The first type is active-standby mode (active-standby). In this mode, one of the storage virtualization One storage virtualization controller) presents, manages, and processes all I/O requests for all logical media units in the storage virtualization subsystem, while another storage virtualization controller (commonly referred to as a secondary storage virtualization controller) will only It is in a standby state (standby), and when the main storage virtualization controller fails or fails, it can take over the main storage virtualization controller at any time. The second is the active-active mode (active-active), in this mode, the two storage virtualization controllers simultaneously present and manage the input and output requests of various logical media units in the storage virtualization subsystem and processing. In the active-active mode, the two storage virtualization controllers are always ready to take over the other if the other storage virtualization controller malfunctions or becomes disabled due to a malfunction. Active-active mode, usually provides better performance, because the resources of its two storage virtualization controllers (such as: central processing unit (CPU, central processing unit) time, internal bus bandwidth, etc.) The virtualization controller can load more I/O request services than it does.

Regardless of whether it is an active-passive mode or an active-active mode, a basic function of a redundant storage virtualization computer system is that when a storage virtualization controller fails, another storage virtualization controller can take over (Take Over) The work of the storage virtualization controller in which the situation occurs, for example: continue to complete the access to the data in the direct access storage device. Therefore, in order to be able to construct a redundant storage virtualization computer system, an inter-controller communications channel (ICC, inter-controller communications channel) must first be established between its storage virtualization controllers, and information can be transmitted through this communication channel; Or, each storage virtualization controller must also be able to know the current working information of its companion's storage virtualization controller at any time, that is to say, the data between the two controllers must be almost synchronized and consistent, so that the situation with its companion can occur Time can take over its work to achieve the effect of redundancy.

For the part of establishing the communication channel between controllers, Fiber Channel Arbitrated Loop (FC-AL) or Small Computer System Interface (SCSI, parallel small computer system interface) or serial advanced technology access interface (SATA, serial advanced technology attachment) are generally used at present. ) and other communication links, the main factor for using these links is that these links have the characteristics of supporting long distances and being able to connect external cables to facilitate the connection between two independent devices.

Please refer to FIG. 1 , which mainly shows a block diagram of a storage virtualization controller of an existing redundant storage virtualization system. Wherein, the first storage virtualization controller 100 has a redundant controller communication (RCC, redundant controller communication) connection controller 136 for establishing a communication channel ICC with the second storage virtualization controller 100'.

Since the communication channel ICC between the two controllers 100, 100' adopts communication links such as Fiber Channel Arbitrated Loop (FC-AL) or SCSI or SATA, it is different from the local bus (local bus) adopted by the circuit in the controller, therefore, In these controllers at both ends of the inter-controller communication channel (ICC), an RCC connection controller 136 as a conversion internal bus and the external connection interface or buffer of the ICC must be arranged, so that the overall circuit is complicated and the cost is also high. .

Furthermore, to solve the problem that each storage virtualization controller must know the current working information of its companion (another storage virtualization controller) at any time and maintain data synchronization and consistency between the two external storage virtualization controllers, in principle, when the storage When the data of the virtualization controller changes, this change must be made known to its peers, so that the two storage virtualization controllers can almost maintain synchronization, and then when a situation occurs in one storage virtualization controller, the other storage virtualization controller A complete successful takeover was possible with almost no data time lag. Therefore, in some cases, the data transmission system on the communication channel ICC is quite frequent, which undoubtedly may increase the workload of the central processing unit in the storage virtualization controller, so that its performance is greatly affected. Therefore, how to deal with communication The data transmission on the channel ICC has also become a very important subject.

Contents of the invention

The main purpose of the present invention is to provide a redundant storage virtualization computer system with simplified circuits and reduced costs.

The present invention discloses a computer system, which includes: a host, used to issue I/O requests; a set of redundant storage virtualization controllers, used to perform I/O operations in response to the I/O requests sent by the host, including There are a first and a second storage virtualization controller coupled to the host, and a local bus (local bus) is used for communication between the first and second storage virtualization controllers; and a set of physical storage devices, Coupled to these storage virtualization controllers, it is used to provide the storage space of the computer system; wherein, when the first storage virtualization controller has a situation, the second storage virtualization controller will automatically take over from the first storage virtualization controller where the situation occurs Functions originally performed by storage virtualization controllers.

The present invention also discloses a storage virtualization subsystem, which includes: a set of redundant storage virtualization controllers, which are used to perform I/O operations in response to an I/O request from a host, and include a set of redundant storage virtualization controllers for coupling to the host a first and a second storage virtualization controller, and a local bus (local bus) is used for communication between the first and second storage virtualization controllers; and a set of physical storage devices coupled to these storage virtualization controllers The virtualization controller is used to provide computer system storage space; wherein, when a situation occurs in the first storage virtualization controller, the second storage virtualization controller will automatically take over from the first storage virtualization controller in which the situation occurs function that was previously performed.

According to a feature of the embodiment, the first and second storage virtualization controllers in the aforementioned computer system or storage virtualization subsystem of the present invention are located on the same circuit board. According to a feature of the embodiment, the local bus in the computer system or the storage virtualization subsystem of the present invention is connected to the two storage virtualization controllers by means of an external cable or a backplane.

According to a feature of the embodiment, the local bus in the aforementioned computer system or storage virtualization subsystem of the present invention is a Peripheral Component Link (PCI) bus, a Peripheral Component Link Expansion (PCI-X) bus or a Peripheral Component Link Express (PCI-Express) bus. )bus. According to a feature of the embodiment, the first and second storage virtualization controllers in the aforementioned computer system or storage virtualization subsystem of the present invention each include a local bus interface for establishing the first and second storage virtualization controllers local bus communication channel between them.

According to a feature of the embodiment, the aforementioned local bus interfaces in the aforementioned computer system or storage virtualization subsystem of the present invention are each located in a central processing unit chipset, and are set up by pins of at least one of the two central processing unit chipsets. It is determined that one of the local bus interfaces changes its operation mode, so that a connection can be established between the local bus interfaces of the two storage virtualization controllers.

According to a feature of the embodiment, in the aforementioned computer system or storage virtualization subsystem of the present invention, software is used to fill in the register of one of the aforementioned local bus interfaces so that the local bus interface can change the operation mode, so that these storage virtualization A connection is established between the local bus interfaces of the controllers.

According to a feature of the embodiment, the aforementioned local bus interface in the aforementioned computer system or storage virtualization subsystem of the present invention further has the feature of cross-connection.

According to a feature of the embodiment, in the computer system or the storage virtualization subsystem of the present invention, at least one of the local bus interfaces implements an automatic switching mechanism, which uses the feature of cross-connection to switch the operation mode of the interface, so that the first A connection can be established with the second storage virtualization controller.

In addition, the present invention discloses a storage virtualization controller, which includes: a central processing circuit for performing I/O operations in response to an I/O request from a host, and can be coupled to another storage virtualization device using a local bus Controller; at least one I/O device connection controller, coupled to the central processing circuit; at least one host-side I/O device port, set in one of the aforementioned at least one I/O device connection controller, for coupling to the host ; at least one device-side input-output device port is provided in one of the aforementioned at least one input-output device connection controller for coupling to at least one physical storage device; and a memory is connected to the central processing circuit and is used for To buffer the data transmitted between the host and the physical storage device through the central processing circuit.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, the central processing circuit includes a central processing unit and a central processing unit chipset. The CPU chipset is used as an interface between the CPU and other electronic components, including: a first local bus interface coupled to the other storage virtualization controller via the local bus; An internal main bus, used as a communication link between the main electronic components in the CPU chipset, for communicating data signals and control signals therebetween; a CPU interface, used to couple to the CPU and the The internal main bus is used as the communication interface between the central processing unit and other electronic components; a memory controller is used to couple to the memory and the internal main bus. The data will be stored in the internal memory, and the data in the internal memory will also be transmitted to the internal main bus through the memory controller; and at least one second local bus interface for coupling to the I/O device connection The controller and the internal main bus serve as a communication interface between the two.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, the aforementioned central processing unit chipset further includes a register, which belongs to the first local bus interface, and a storage space in the register is planned and defined as The CPU writes and transmits data to another storage virtualization controller for related information.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, the aforementioned register is located in the first local bus interface or a register array located in the CPU chipset.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, the aforementioned first local bus interface is a peripheral component connection express bus interface, a peripheral component connection expansion bus interface or a peripheral component connection bus interface.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, the pin setting of the aforementioned central processing unit chipset is used to make the aforementioned first local bus interface change its operation mode, so as to communicate with another Correspondingly, the storage virtualization controllers can establish connections with each other.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, software is used to fill in a register of the aforementioned first local bus interface so that the first local bus interface changes the operation mode so as to communicate with another storage The virtualization controllers are corresponding to establish connections with each other.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, the aforementioned first local bus interface further has the feature of a cross connection.

According to a feature of the embodiment, in the aforementioned storage virtualization controller of the present invention, the aforementioned first local bus interface implements an automatic switching mechanism, which uses the feature of cross-connection to switch the operation mode of the interface, so that it can communicate with another storage The virtualization controllers are corresponding to establish connections with each other.

The present invention further discloses a method for establishing a communication channel between storage virtualization controllers, comprising the following steps: the storage virtualization controller sends a message containing information about its operation mode through a local bus interface serving as an end of the inter-controller communication channel ; When at least one of the storage virtualization controllers receives the operation mode information of another storage virtualization controller through the local bus interface, it compares the received operation mode information with its own operation mode to determine Whether the connection can be established; if it is determined that the connection can be established, then establish a communication channel between the two storage virtualization controllers, and if it is determined that the connection cannot be established, then at least one of the storage virtualization controllers will switch the operation of the local bus interface The mode is corresponding to the operation mode of the local bus interface of another storage virtualization controller, so as to establish mutual connection.

According to a feature of the embodiment, the local bus interface in the aforementioned method of the present invention is a PCI-Express bus interface.

According to a feature of the embodiment, in the method of the present invention, the local bus has the feature of cross-connection, and when the comparison judges that the connection cannot be established, the storage virtualization controller uses the feature of the cross-connection to switch the communication interface mode of operation.

According to a feature of the embodiment, in the above-mentioned method of the present invention, the step executed after the result of comparison and judgment is that the connection cannot be established includes a matching mechanism, so that the operation of the communication interfaces at both ends of the communication channel between these storage virtualization controllers The modes correspond to each other, so that connections can be established.

According to a feature of the embodiment, the aforementioned matching mechanism in the aforementioned method of the present invention includes the following steps: randomly select a timing critical parameter value, and then start timing; If the information shows that the operation mode has been changed, the matching is completed and the connection is established; if the time reaches the critical value and no information has been received that the other party’s operation mode has changed, the step of switching the operation mode will be executed, and the message will be sent after the conversion is completed. One contains new operating mode status information to the other party; and re-judging whether the operating mode types between the two are different. If not, the matching is completed and the connection is established. If they are the same, the matching mechanism is re-executed until the matching Finish.

According to a feature of the embodiment, the step of switching the operation mode performed by the matching mechanism in the method of the present invention further includes the following steps: if the operation mode information sent by the other party is received when switching the operation mode, the switching is suspended and maintained The previously set operating mode.

According to a feature of the embodiment, in the aforementioned method of the present invention, before the aforesaid abort conversion, it is first compared whether the operation mode of the other party is different from its own operation mode before the conversion, and if so, steps such as the abort conversion are performed; .

The present invention further discloses a method for data transmission between storage virtualization controllers in a computer system, comprising the following steps: A. A central processing unit of a storage virtualization controller according to a pre-defined data transmission protocol format (data-transfer-protocol format) establishes at least one corresponding discrete aggregation table (SG-list) for the discrete data (scattered data) to be transmitted to another storage virtualization controller; B. The central processing unit will store the discrete aggregation The address of the table is written into a register; and C. a local bus interface reads the scatter-gather table in a memory according to the address written in the register, and according to the address indicated by a scatter-gather data (scatter-gather data) in the table Read the discrete data into the memory, and transmit the discrete data to another storage virtualization controller through a local bus.

According to a feature of the embodiment, the local bus in the aforementioned method of the present invention is a peripheral device connection bus, a peripheral device connection expansion bus or a peripheral device connection express bus.

According to a feature of the embodiment, in the aforementioned method of the present invention, after the central processing unit writes the address storing the discrete aggregation table into the register, the register transmits the address information to the local bus interface, and triggers the local bus interface to execute step C. .

According to a feature of the embodiment, in the aforementioned method of the present invention, the central processing unit writes the address of the discrete aggregation table into the register and is defined as a storage space for writing a specific address for storing the address of the discrete aggregation table.

According to a characteristic of the embodiment, the data transmission protocol format of the discrete aggregation table in the aforementioned method of the present invention includes the following fields: the data quantity field in the table, which is used to indicate the number of discrete aggregation data contained in the table; the source starting address field, used to indicate the memory start address where each piece of discrete data to be transmitted is stored; the data length field is used to indicate the length of each piece of discrete data to be transmitted; and the target start address field is used to indicate the transmission of discrete data The target address to be stored.

According to a feature of the embodiment, the execution of the local bus interface in step C. of the method of the present invention includes the following steps: read out the control information in the discrete aggregation table header in the memory according to the information sent by the central processing unit, according to The quantity of the discrete aggregated data reads down the control information of the discrete aggregated data in sequence; and according to the indication of the source start address field and the discrete data length field content in each piece of discrete data read , read the pieces of discrete data one by one, and transmit each piece of discrete data together with the target start address of the piece of discrete data to another storage virtualization controller through the local bus.

According to a feature of the embodiment, the discrete data read by the local bus interface in the method of the present invention is first temporarily stored in a buffer, and then transmitted together with the target start address.

According to a feature of the embodiment, when the local bus interface reads and transmits the discrete data in the method of the present invention, it can read and transmit the discrete data in batches according to the actual capacity of the buffer.

According to a feature of the embodiment, the aforementioned method of the present invention further includes a step of receiving the discrete data and the target start address by the local bus interface of another storage virtualization controller, and storing the discrete data in the memory of the target address.

According to a feature of the embodiment, the data transmission protocol format of the discrete aggregation table in the aforementioned method of the present invention further includes a data direction field, which is used to indicate the execution of a write or read operation, so that the local bus interface performs corresponding operations according to the content of this field. access operations.

According to a characteristic of the embodiment, in the aforementioned method of the present invention, after the step of reading the contents of the discrete aggregation table in the local bus interface in step C., further includes: performing corresponding writing according to the content of the data direction field input or read operation; if the content indicates that the write operation is performed, then perform the subsequent steps of reading the discrete data and transmitting the discrete data to the other storage virtualization controller according to each field of the discrete aggregation table; if the content indicates When performing a read operation, at least part of the field content in the discrete aggregation table is transmitted to another storage virtualization controller through the local bus; and after receiving the discrete data transmitted by another storage virtualization controller, the discrete The target starting address corresponding to the data is stored in the memory.

According to a feature of the embodiment, the CPU chipset in the method of the present invention transmits the entire discrete aggregation table content to another storage virtualization controller.

According to a feature of the embodiment, the above-mentioned method of the present invention further includes another CPU chipset of the storage virtualization controller receiving the content of the discrete aggregation table, according to the indication of the source start address field and the content of the data length field, according to Steps to sequentially read each piece of discrete data and return the discrete data.

According to a feature of the embodiment, the data transmission protocol format of the discrete aggregation table in the aforementioned method of the present invention further includes the following fields: a first interrupt field, which is used to set whether after completing the data transmission listed in this table, the central processing unit The chipset needs to generate an interrupt signal to notify the CPU; and the second interrupt field is used to set another storage virtual controller to generate an interrupt signal after receiving the discrete data and completing the corresponding operation.

According to a feature of the embodiment, step C. in the aforementioned method of the present invention further includes: the CPU chipset transmits the content of the aforementioned second interrupt field to another storage virtualization controller; After the data transmission indicated in the discrete gathering table is completed, it is determined whether to generate an interrupt signal to the central processing unit according to the indication of the first interrupt field.

According to a characteristic of the embodiment, the data transmission protocol format of the discrete aggregation table in the aforementioned method of the present invention further includes a field of the address of the next discrete aggregation table, which is used to store the memory address stored in the next discrete aggregation table, so as to indicate the central processing After completing the data transmission indicated by the contents of a discrete gathering table, the processor chipset can read the next discrete gathering table according to the content of the field of the address of the next discrete gathering table, and then continue processing in conjunction.

According to a feature of the embodiment, the aforementioned method of the present invention includes a system planning setting that indicates that there is no value for the next discrete aggregation table, and when the content of the address field of the next discrete aggregation table is the value, it will display no There are other discrete aggregation tables that need linkage processing.

According to a feature of the embodiment, in the aforementioned method of the present invention, the aforementioned value indicating that there is no next discrete aggregation table is 0.

According to a characteristic of the embodiment, in the above-mentioned method of the present invention, the following steps are further included between steps A and B: check whether there is an unprocessed discrete aggregation table; The newly created discrete aggregate table is linked with the previously unprocessed discrete aggregate table; and otherwise, step B is performed.

According to a characteristic of the embodiment, in the above-mentioned method of the present invention, the following steps are further included before performing the linkage program: the central processing unit sends a suspend request to the local bus interface to request suspend transmission of data to another storage virtual machine The related actions of the controller; and when the local bus interface receives the suspend request, it executes a suspend mechanism, and after completing the suspend mechanism, replies a suspend approval to the central processing unit; and further includes the following steps after performing the linkage procedure : The central processing unit notifies the local bus interface to release the suspension state; and when the local bus interface receives the cancellation notification, it resumes the suspended action and continues to process and transmit data.

According to a feature of the embodiment, the above-mentioned pause mechanism in the method of the present invention is to suspend the reading and transmission of each piece of discrete data after the discrete data being processed is completed, and record the pause point to facilitate the cancellation of the pause. Continuation processing can be resumed.

According to a feature of the embodiment, the aforementioned suspending mechanism in the aforementioned method of the present invention is an action of suspending entry into the next discrete aggregation table after all the discrete data in the discrete data table being processed have been transmitted.

According to a feature of the embodiment, the aforementioned linkage program in the aforementioned method of the present invention comprises: the central processing unit reads the address data in the next discrete gathering table address field in the discrete gathering table stored in the register; According to the address in the read register, it is judged whether the local bus interface has a discrete aggregation table that needs continuous processing thereafter; The content of the field is the starting address for storing the newly created first discrete aggregation table; Gathering table address field content, change it to store the start address of the newly created first discrete gathering table, and set the content of the next discrete gathering table address field of the last one in the newly created discrete gathering table to It must be consistent with the content of the address field of the next discrete aggregation table stored in one of the previously unprocessed discrete aggregation tables before being changed. According to a feature of the embodiment, one of the unprocessed discrete gather tables in the aforementioned method of the present invention refers to a discrete gather table stored in a register when the local bus interface is performing a suspend mechanism.

According to a feature of the embodiment, one of the aforementioned unprocessed discrete aggregate tables in the aforementioned method of the present invention refers to the last discrete aggregate table among the unprocessed linked discrete aggregate tables.

According to a feature of the embodiment, the aforementioned method of the present invention includes setting up a table that records the memory addresses stored in all discrete gathering tables, and is used for the central processing unit to inquire about not yet to change the content of the field that stores the address of the next discrete gathering table. The memory address where the processed discrete aggregation table is stored. According to a feature of the embodiment, in the aforementioned method of the present invention, the central processor reads the next discrete aggregation table address field of the discrete aggregation table stored in the address according to the address information of the next discrete aggregation table in the read register The content is repeated in sequence until the unprocessed discrete gathering table that the central processor intends to change the field content storing the address of the next discrete gathering table is read.

According to a characteristic of the embodiment, when the central processing unit needs to transfer the established discrete aggregation table in the aforementioned method of the present invention, the execution includes the following steps: checking the discrete aggregation table that has not been processed at present; and judging that it intends to transfer Whether the discrete aggregation table belongs to the discrete aggregation table that has not been processed, if so, it will be transferred, otherwise, it will not be transferred.

According to a characteristic of the embodiment, in the aforementioned method of the present invention, the central processing unit is used to read the address data stored in the address field of the next discrete aggregation table in the discrete aggregation table in the register, so as to know the current progress of data transmission.

According to a feature of the embodiment, the above-mentioned method of the present invention further includes the following steps before finally mobilizing: the central processing unit sends a suspend request to the local bus interface for requesting suspend transmission of data to another storage virtualization controller action; and when the local bus interface receives the suspend request, it executes a suspend mechanism, and after completing the suspend mechanism, replies a suspend approval to the central processing unit; The bus interface releases the suspended state; and when the local bus interface receives the release notification, it resumes the suspended action and continues to execute processing and transmitting data.

The present invention further discloses another method for data transmission between storage virtualization controllers in a computer system, including the following steps: a central processing unit of a storage virtualization controller sends a data transmission request to a central processing unit chipset; a first local bus interface in the CPU chipset forwards requests to another storage virtualization controller; and a second local bus interface in the CPU chipset of another storage virtualization controller The bus interface performs corresponding processing after receiving the request.

According to a characteristic of the embodiment, the local bus in the aforementioned method of the present invention is a peripheral component connection bus interface, a peripheral component connection expansion bus interface or a peripheral component connection express bus interface.

According to a feature of the embodiment, the step of the central processor sending the data transmission request in the method of the present invention includes: the central processing unit sends the data transmission request to a central processing unit interface in the central processing unit chipset; the central processing unit interface placing the data transfer request on an internal main bus in the CPU chipset; and reading the data transfer request by the first local bus interface.

According to a feature of the embodiment, the data transmission request in the aforementioned method of the present invention includes a write data and a target start address, the target start address is used to indicate the memory address for writing the write data, And the corresponding processing performed by the other storage virtualization controller includes: writing the write data into the memory according to the target start address.

According to a feature of the embodiment, the content of the data transmission request in the method of the present invention includes identification information, so that the first local bus interface can identify and read the data transmission request.

According to a feature of the embodiment, the aforementioned method of the present invention includes: the storage virtualization controller defines the physical memory address of another storage virtualization controller as a virtual memory address, so that it is different from its own physical memory address It will not be repeated, and the identification information is a memory address, and the memory address is expressed in the form of the virtualized memory address.

According to a characteristic of the embodiment, in the aforementioned method of the present invention, the address of the virtual memory is defined by directly following the address of the physical memory itself.

According to a feature of the embodiment, the data transmission request in the method of the present invention includes a memory address, and the memory address is represented by a virtualized memory address as the identification information.

According to a feature of the embodiment, the aforementioned method of the present invention includes the step of converting the aforementioned memory address into a corresponding physical memory address by the first or second local bus interface.

According to a feature of the embodiment, the data transmission request in the aforementioned method of the present invention includes an access command for instructing to perform a write or read operation.

According to a feature of the embodiment, the corresponding processing performed by another storage virtualization controller in the method of the present invention includes the second establishment of a local bus interface to interpret the access instruction in the request to perform the indicated corresponding operation .

According to a feature of the embodiment, the data transmission request in the aforementioned method of the present invention further includes a data length and a data source start address, and the corresponding processing performed by another storage virtualization controller refers to The source start address and data length are read into the memory to read the data and send the data back to the first local bus interface.

According to a feature of the embodiment, the aforementioned method of the present invention further includes: when the first local bus interface receives the returned data from another storage virtualization controller, it transmits it to the central processing unit.

Description of drawings

FIG. 1 is a block diagram of a conventional redundant external storage virtualization controller.

FIG. 2 is a block diagram of a storage virtualization computer system according to the present invention.

FIG. 3 is a block diagram of a storage virtualization controller according to the present invention.

FIG. 4 is a block diagram of an embodiment of the central processing circuit shown in FIG. 3 .

FIG. 5 is a block diagram of an embodiment of the CPU chipset/parity engine shown in FIG. 4 .

FIG. 6 is a flowchart of an automatic conversion mechanism of a storage virtualization subsystem.

Fig. 7 is a flowchart of a unilateral automatic switching mechanism.

FIG. 8 is a flow chart of the storage virtualization controller in another automatic conversion mechanism.

FIG. 9 is a flowchart of a data transmission method according to the present invention.

Figure 10 is a scatter-gather table format.

FIG. 11 is a hypothetical example of data transmission.

FIG. 12 is another hypothetical example of data transmission.

FIG. 13 is yet another hypothetical example of data transmission.

FIG. 14 is yet another hypothetical example of data transmission.

FIG. 15 is another flow chart of data transmission performed by the central processing unit.

FIG. 16 is a flow chart of an embodiment of the insertion or continuation program in FIG. 15 .

FIG. 17 is a flow chart of another embodiment of the insertion or continuation program in FIG. 15 .

FIG. 18 is a flow chart of transferring a small amount of data between storage virtualization controllers according to the present invention.

Detailed ways

With the development of science and technology, the local bus (local bus) has developed from the peripheral component interconnect (PCI, peripheral component interconnect) bus to the peripheral component interconnect extended (PCI-X, peripheral component interconnect extended) bus, and the peripheral component interconnect express (PCI-Express) bus .

The PCI-Express is different from other local buses in that it breaks through the limitations of the previous local bus that cannot or is limited on the backplane (backplane), and the limited transmission distance of its electrical characteristics. PCI-Express The Express interface can not only connect external cables, but also communicate online through cables, and its electrical characteristics can also make the transmission distance up to 7 meters.

Please refer to FIG. 2. FIG. 2 is a schematic block diagram of an embodiment of the present invention. The system includes a host 10 and a storage virtualization subsystem 20 (SVS, redundant storagevirtualization subsystem). The storage virtualization subsystem 20 includes a set of storage virtualization controllers (including first and second storage virtualization controllers (SVC1, SVC2) 200, 200', and a plurality of physical storage devices 420. The storage virtualization controller 200, 200' can be a disk array controller or a JBOD emulator.

Although only one host 10 is connected to a storage virtualization subsystem 20 as shown in FIG. The system 20 , or multiple hosts 10 are connected to multiple storage virtualization subsystems 20 . The host 10 can be a host computer, such as a server system, workstation, personal computer system or other related computers, and the host 10 can also be another storage virtualization controller.

The storage virtualization subsystem has a structure, and an inter-controller communication channel ICC is provided between the two storage virtualization controllers 200, 200' for exchanging information with each other. The communication channel ICC between the controllers is PCI-Express. The reason why PCI-Express is adopted in this implementation is that PCI-Express can not only walk on the backplane, but also can be connected to an external cable for transmission and supports longer transmission distances. Therefore, it is more suitable for the design of external storage virtualization controller, so that when a storage virtualization controller has a problem (such as: failure or failure), storage virtualization can be carried out in another normal operation Repair or replace a faulty storage virtualization controller while the controller is still in service. However, the present invention should not be limited thereto. According to the design of the controller, any local bus used by the circuit in the controller can be applied. For example, for the design that the two controllers are located on the same circuit board, PCI or PCI-X, etc.

In one embodiment, all the physical storage devices 420 in the storage virtualization subsystem 20 can be combined to form a physical storage device array 400 .

Fig. 3 is a block diagram of an embodiment of the storage virtualization controller 200, 200' connected to the host 10 and the physical storage device array 400 in the present invention. Here, the first storage virtualization controller (SVC1) 200 is taken as an example for illustration, but in fact, the second storage virtualization controller 200' is also the same. In this embodiment, the first storage virtualization controller (SVC1) 200 includes a host-side I/O device connection controller 220, a central processing circuit (CPC, central processing circuit) 240, a memory 280, and a device-side I/O The device is connected to the controller 300 . Although described here as separate functional blocks, in practical applications, some or even all of the functional blocks can be integrated on a single chip.

The host-side I/O device connection controller 220 is connected to the host 10 and the central processing circuit 240, and is used as an interface and buffer between the first storage virtualization controller (SVC1) 200 and the host 10, and can receive data transmitted from the host 10. Incoming I/O requests and related data are converted and/or mapped to the central processing circuit 240 . The host-side I/O device connection controller 220 may include one or more host-side ports for coupling to the host 10 . The types of ports referred to herein may be: fiber channel supporting fabric, point-to-point, public loop and/or dedicated loop in target mode, parallel small computer system interface operating in target mode (parallel SCSI, parallel small computer system interface), Ethernet network that supports Internet SCSI (iSCSI, internet SCSI) protocol and operates in target mode, serial attached SCSI (SAS, serial-attached SCSI) operating in target mode, and operates in Serial advanced technology access interface (SATA, serial advanced technology attachment) in target mode.

The device-side I/O device connection controller 300 is interposed between the central processing circuit 240 and the physical storage device array 400 and used as an interface and buffer between the storage virtualization controller 200 and the physical storage device array 400 . The device-side I/O device connection controller 300 receives I/O requests and related data from the central processing circuit 240 , and maps and/or transmits them to the physical storage device array 400 . The device-side I/O device connection controller 300 may include one or more device-side ports for coupling to the physical storage device array 400 . The type of port mentioned here is to cooperate with the physical storage device used by the system, which can be: FC-AL, SCSI, serial-attached SCSI (SAS, serial-attached SCSI) and serial advanced technology access interface (SATA, serial advanced technology attachment).

Moreover, although here is an example where the corresponding I/O device connection controllers 220, 300 are respectively provided at the host end and the device end, in another embodiment of the present invention, there may be only one I/O device connection controller device, so that the host-side port coupled to the host 10 and the device-side port coupled to the physical storage device array 400 are both set in the I/O device connection controller. When the central processing circuit 240 receives a host I/O request from the host-side I/O device connection controller 220, the central processing circuit 240 will analyze the I/O request, and perform some operations in response to the I/O request, and send all The requested data and/or reports and/or information are transmitted from the first storage virtualization controller 200 to the host 10 via the host-side I/O device connection controller 220 . After analyzing the incoming I/O request of the host computer 10, if the received request is a read request and one or more operations are executed as a response, the central processing circuit 240 will internally or from the memory 280 or through this There are two ways to obtain the requested data and send the data to the host 10 . If the requested data cannot be obtained internally or does not exist in the internal memory 280, the read request will be sent to the physical storage device array 400 via the device-side I/O device connection controller 300, and then the requested data will be stored by the physical The device array 400 is transmitted to the memory 280 , and then transmitted to the host 10 via the host-side I/O device connection controller 220 . When the write request (write request) imported by the host 10 is transmitted to the central processing circuit 240, after the write request is analyzed and one or more operations are performed, the central processing circuit 240 is connected to the controller through the host-side I/O device 220 receives incoming data from host 10 and stores it in memory 280 . For both synchronous or asynchronous device operation, data is transferred via central processing circuit 240 to physical storage device array 400 . When the write request is a writeback request, the IO complete report will be sent to the host 10 first, and then the central processing circuit 240 will perform the actual write operation; If the write request is a write through request, the I/O completion report will be sent to the host 10 after the data has actually been written into the physical storage device array 400 . The memory 280 is connected to the central processing circuit 240 and serves as a buffer for buffering data transmitted between the host 10 and the physical storage device array 400 through the central processing circuit 240 . In actual application, the memory 280 may be dynamic random access memory (DRAM, dynamic random access memory), or more specifically, the DRAM may also be synchronous dynamic random access memory (SDRAM, synchronous dynamic random access memory).

In this embodiment, the central processing circuit 240 of the first storage virtualization controller (SVC1) 200 is directly connected to the central processing circuit (not shown in the figure) of the second storage virtualization controller (SVC2) 200' , that is to construct a communication channel (ICC) between the first storage virtualization controller (SVC1) 200 and the second storage virtualization controller (SVC2) 200'.

In addition, the device-side I/O device connection controller 300 is coupled to the physical storage device array 400, and the physical storage device array 400 is also coupled to the second storage virtualization controller (SVC2) 200'.

In this configuration, the second storage virtualization controller (SVC2) 200' can be attached to the first storage virtualization controller (SVC1) 200, and the physical storage device array 400 can be controlled by the two storage virtualization controllers. 200 accesses. What's more, the control/data signal sent by the host 10 can be transmitted from the central processing circuit 240 to the second storage virtualization controller (SVC2) 200 or further transmitted to a second physical storage device array (not shown in the figure ).

Please continue to refer to FIG. 3 , in this embodiment, an enclosure management service circuit 360 (EMS circuit, enclosure management service circuit) can be attached to the central processing circuit 240 as a management system for accommodating the entity storage device array 400 enclosures. circuit, the cabinet management service circuit 360 is used to control the power supply of the physical storage device array and perform other management, and a liquid crystal display module 350 (liquid crystal display module, LCD module) is used to display the operating status of the subsystem. However, the storage virtualization subsystem 20 also has other configurations. For example, depending on the functional design of various products, the cabinet management service circuit 360 or the LCD module 350 can be omitted, or the cabinet management service circuit 360 or the LCD module 350 can be omitted. Body Management Service The box management service circuit 360 is integrated in the central processing circuit 240 .

See also Fig. 4, be an embodiment of central processing circuit 240, wherein include CPU chipset/parity engine 244 (CPU chipset/parity engine), a central processing unit 242 (CPU), read-only memory 246 (ROM, read only memory) and a non-volatile random access memory 248 (NVRAM, non-volatile random access memory). Wherein the CPU 242 may be, for example, a Power PC CPU, and the ROM 246 may be a flash memory for storing basic input/output system (BIOS) and/or other system programs to be executed to control subsystems when booting operation. The NVRAM 248 is used to store relevant information about the execution state of the I/O operation of the physical storage device array, and is used for inspection when an abnormal power supply shutdown occurs before the I/O operation is completed. ROM 246, NVRAM 248, LCD module 350 and cabinet management service circuit 360 are all connected to CPU chipset/parity engine 244 via an X-bus (X-bus). Again, the NVRAM 248 is an optional item, which can be omitted in another configuration of the present invention. Moreover, although the CPU chipset/parity engine 244 is described here as an integrated functional block, in actual application, the CPU chipset and the parity engine can be separately arranged on different chips.

Please continue to refer to FIG. 4 , the CPU 242 must be coupled with other electronic components (such as memory 280 , etc.) through the CPU chipset/parity engine 244 .

FIG. 5 shows an embodiment of the CPU chipset/parity engine 244 in the present invention. Wherein CPU chipset/parity engine 244 includes parity engine 260, CPU interface 910, memory controller 920, peripheral component connection fast (PCI Express) interface 930,932,934, X-Bus interface 940, transmission control protocol and Internet protocol (TCP/IP, transmission control protocol/internetprotocol) direct memory access (DMA, direct memory access) 980, arbitrator (Arbiter) 982, internal local bus (IL, internal local) 990 and internal main (IM, internal main) The bus 950, wherein the IM bus 950 is connected to the parity engine 260, the CPU interface 910, the memory controller 920, and the PCI-E interfaces 930, 932, 934 for communicating data signals and control signals therebetween.

The data and control signals sent by the host-side I/O device connection controller 220 enter the CPU chipset/parity engine 244 through the PCI-E interface 930 . The transmission rate of the PCI-E interface 930 connected to the host-side I/O device connection controller 220 may be, for example, 1.5 Gbit/sec. When the PCI-E interface 930 has an IM bus 950 (IM Bus), the data and control signals will be sent to the memory controller 920 or the CPU interface 910.

When the CPU interface 910 receives data and control signals from the IM bus 950, it will be sent to the CPU 242 for further processing, and the communication pipeline between the CPU interface 910 and the CPU 242 can be, for example, 64- bit data transmission lines and 32-bit address lines.

In the memory controller 920, there is an error correction code circuit (ECC circuit, error correction code circuit) (not shown in the figure), which is used to generate an ECC code, and the way it is generated can be, for example, 8- After the bit data is operated with exclusive OR (XOR), a single-bit ECC code is generated. Next, the memory controller 920 stores the data and the ECC code in the memory 280 . The memory 280 can be, for example, SDRAM. And the data in the memory 280 can also be sent to the IM bus 950 . And the memory controller 920 can be designed such that when data is transmitted from the memory 280 to the IM bus 950, the memory controller 920 will perform single-bit auto-correction (1-bit auto-correction) and multi-bit error detection (multi-bit error detection) on the data. error detecting) and other functions.

The parity engine 260 will respond to the instructions of the CPU 242 to perform a parity function of a specific disk array type. Of course, under some specific conditions, such as RAID0, the parity engine 260 can be turned off without performing the parity function.

The IL bus 990 (IL Bus) is connected to the CPU interface 910 and other low-speed device interfaces.

The register array 984 (Reg.array) is used to temporarily store the state of the CPU chip/parity engine 244, and control the data flow in the IM Bus 950. In addition, a pair of Universal Asynchronous Receiver and Transmitter (UART, universal asynchronous receiver and transmitter,) functional block 986 is used as an external interface of the CPU chip/parity engine 244, and the interface specification is RS232.

The CPU chipset/parity engine 244 is connected to the device-side I/O device connection controller 300 through the PCI-E interface 932 .

The TCP/IPDMA980 is used to perform the functions of checksum calculation and DMA operation. The arbitrator 982 is used to arbitrate the right to use the IM bus 950 .

In this embodiment, if data is to be transferred from the memory 280 to the second storage virtualization controller 200′, after entering the PCI-E interface 934 through the IM bus 950, a PCI-E communication channel is used, such as: An external cable (cable) or a backplane (backplane) can be directly coupled to the PCI-E interface 934' in the second storage virtualization controller 200', without any intermediate conversion as in the prior art Buffer interface (eg: redundant communication link controller).

In actual application, the PCI-E interfaces 930, 932 can be replaced by peripheral component interconnect extended (PCI-X, peripheral component interconnect extended) interfaces, or replaced by peripheral component interconnect (PCI) interfaces.

When the subsystem 20 is powered on, the CPU chip set of each storage virtualization controller will know why and whether the connection is established correctly for the device externally connected to the controller through each PCI-E interface. And if the PCI-E interface is to be used to establish a PCI-E transmission channel between the two devices, the physical layer mode (PHY-mode) of the PCI-E interface connected to the two devices must be operated in the downstream (down stream) mode and the upstream ( up stream) mode, otherwise it will not be possible to establish a connection between the two for data transmission.

Generally speaking, the storage virtualization controller is an active component, so the PHY-mode of its PCI-E interface is set to down stream.

However, as mentioned above, the settings of the two storage virtualization controllers are consistent, that is to say, for the two storage virtualization controllers, it establishes the PCI-E interface of the inter-controller communication channel ICC between the two controllers Both PHY-modes are set to down stream, so the two controllers cannot successfully establish a connection at all. In order to solve this problem, in this embodiment, a conversion mechanism is proposed so that one of the controllers can be converted to upstream, so as to successfully establish an online data transmission.

Please refer to FIG. 6 , which is a flowchart of an automatic conversion mechanism for a storage virtualization subsystem. The automatic conversion mechanism is executed by the PCI-E interface of the CPU chipset in the storage virtualization controller.

First, the two storage virtualization controllers 200, 200' send information including their PHY-mode information through the PCI-E interfaces 934 at both ends of the inter-controller communication channel ICC, so the two storage virtualization controllers 200, 200 The PCI-E interface 934 of ' can receive the PHY-mode information of the other side (step S810);

Compare whether the PHY-mode of the two is different and can establish connection (step S820);

If the result of the comparison is that the information of another storage virtualization controller shows that its PHY-mode is the same as its own PHY-mode, then the PHY-mode will be changed using the characteristics of the cross link of PCI-Express, and the first and the second will be matched. The PHY-modes of the PCI-E interfaces at both ends of the communication channel ICC between two storage virtualization controllers operate in down stream and up stream respectively to establish a connection (step S830 and step S840).

The aforementioned matching method can be achieved by only allowing the PCI-E interface at one end to change the PHY-mode, that is, when the information received by the PCI-E interface shows that the PHY-mode at the other end is the same as itself, only The PCI-E at one end will change its PHY-mode.

It is assumed that only one-sided conversion is allowed to be performed by the second storage virtualization controller. When the subsystem is powered on, the two storage virtualization controllers respectively send a message including their PHY-mode information through the PCI-E interfaces at both ends of the inter-controller communication channel ICC. Therefore, the inter-controller communication channel ICC is at the second The PCI-E interface of the storage virtualization controller will receive the PHY-mode information of the PCI-E interface in the first storage virtualization controller. Please refer to FIG. 7 , which is a flow chart of the second storage virtualization controller.

After the PCI-E interface of the second storage virtualization controller receives the PHY-mode information of the first storage virtualization controller PCI-E, it is judged whether the connection can be established by the received PHY-mode information (step S410, S420);

If the PCI-E interface of the second storage virtualization controller detects that the information of another storage virtualization controller shows that its PHY-mode is the same as its own PHY-mode in the aforementioned determination procedure (step S420), then the -The feature of the cross link (cross link) of Express changes the PHY-mode of its storage virtualization controller, and sends new PHY-mode information to the first storage virtualization controller (step S430) after conversion, and then enters the establishment Online program (step S440).

The PCI-E interface at the ICC side of the inter-controller communication channel of the first storage virtualization controller waits until it receives the second storage virtualization controller's message after transmitting the setting information including its PHY-mode as down stream. The PCI-E interface transmits the information showing that its PHY-mode is up stream through ICC, and then establishes the connection between the two controllers according to the content of the information.

To sum up, in this implementation solution, when the PHY-modes at both ends of the communication channel are the same among the storage virtualization controllers, only one storage virtualization controller will use the cross-link to convert its PHY-mode, and the other The storage virtualization controller does not perform any actions. That is, the designs of the two storage virtualization controllers in this embodiment are different, so the ICs of the CPU chipsets used by the two will be different.

That is to say, in the aforementioned method, the controllers must be used in pairs so that the communication channel ICC therebetween can be connected normally.

However, in some environments or situations, it is expected or required that the software and hardware of the ICC communication interface adopted by the two storage virtualization controllers forming the storage virtualization controller pair should be the same. Here, another automatic conversion mechanism is proposed, wherein the procedures performed by the two storage virtualization controllers to establish the PCI-E interface of the ICC are the same, enabling the aforementioned situation. Figure 8 shows the storage virtualization controller side PCI A flowchart of an implementation scheme implemented by the -E interface.

First, when the PCI-E interface of the storage virtualization controller receives the information of another connection end, it will analyze to understand the state of the connection end (step S510),

Then, compare whether the PHY-mode of the connection end is the same as itself (step S520), if not, directly proceed to establish a connection procedure (step S530);

If the PHY-modes of the two are the same, enter a matching procedure S540;

First, a parameter T value (timing critical value) will be selected at random (random), and then start timing to know whether the T value is reached (step S542);

Thereafter, if before the time reaches the T value, receiving the new PHY-mode information of the other party shows that it has changed the PHY-mode (step S544), then end the matching procedure and enter the online procedure of setting up (step S530);

And when the information that the other party's PHY-mode has changed has not been received for a long time, and the time reaches the T value, as shown in the figure, step S546 judges whether the time t has reached the T value, if t<T, then go back to step S544, Otherwise, carry out a conversion PHY-mode procedure (step S548), is to utilize cross-link to convert PHY-mode from the type that is set originally to another type, as: originally be down stream then change to upstream or up The stream is converted to a down stream, and after the conversion is completed, a new PHY-mode status message is sent to the other party;

Afterwards, return to step S520, determine whether the PHY-mode types between the two are different and can establish a connection, if they are the same, then enter the matchmaking procedure S540 again, and then randomly select the setting parameters again at this time T.

Among them, the reason why it is necessary to go back to step S520 after the execution of step S548 is that although it has been determined that the PHY-modes of the two are the same before the conversion, the two storage virtualization controllers may execute cross-mode at the same time. When the link changes the PHY-mode, in this case, the connection between the two still cannot be established. Therefore, after the conversion is completed, it is still necessary to judge whether the PHY-mode between the two is different, and confirm that the matching is completed to ensure that Establish a connection. And if both change the PHY-mode status at the same time, since the two parties will send and display new PHY-mode status information to the other party after the change, it can still be learned that the matchmaking is not successful through comparison in step S520, so it can be obtained Match again.

In addition, it can be designed that during the conversion process in step S548, if the PHY-mode information sent by the other party is received, the conversion is stopped to maintain the originally set PHY-mode and jump to the step of establishing a connection procedure (S530).

Or, in step S548, during the conversion process, if the PHY-mode information transmitted by the other party is received, the conversion is suspended, and whether the PHY-mode state of the other party is different from the PHY-mode before the conversion step S548 is performed by itself, if so, Then stop the conversion process to maintain the settings before the conversion, and directly jump to step S530 to establish a connection process, and if it is found that the PHY-mode of the other party is still the same as before the conversion, continue to complete the conversion. In order to avoid the situation that after the other party completes the conversion and is no longer the same as yourself, you are converted to the same party as the other party and still cannot establish a connection.

Here is an example for illustration. First, when the subsystem is powered on, the PCI-E interfaces of the two storage virtualization controllers will both send out a message showing the operating status of the PHY-mode as down stream, etc., which of course includes The PCI-E interface used to establish the communication channel ICC between controllers.

When the PCI-E interface of the second storage virtualization controller receives the information sent by the other party, it will analyze it to understand its PHY-mode, and when it finds that its PHY-mode is also down stream, both will enter the matching process, and each Randomly select a T value (assuming that the first storage virtualization controller is selected as T1, and the second storage virtualization controller is selected as T2), and start timing. When the T value is reached, the cross-link is performed, because The T value is randomly selected, so there is a high probability that the two controllers will select different T values, that is, T1≠T2, and the side that selects the smaller T value will enter the conversion program first, and the PHY-mode Convert to up stream, and send information containing the new PHY-mode to the other party after conversion, and because the T value selected by the other party is relatively large, when receiving the new PHY-mode sent by the other party information, it may not have reached the selected T value, or it has just reached the T value and is executing the conversion PHY-mode procedure. If it is the former, after receiving the information that the other party has converted to the up stream, it will directly enter the connection procedure of establishing the communication channel ICC between the two controllers. If it is the latter, it will stop the conversion and maintain the PHY of the original down stream -mode, and then successfully establish the inter-controller communication channel ICC between the two. For example: assuming T1<T2, the first storage virtualization controller will first enter the conversion program, and convert the PHY-mode to up stream, and send information including the new PHY-mode (up stream) to The second storage virtualization controller, and because the T2 value selected by the second storage virtualization controller is larger, after the first storage virtualization controller completes the conversion process, the second storage virtualization controller has not yet started The conversion is halfway through the conversion, so the second storage virtualization controller known by the first storage virtualization controller is still in the down-stram mode, so it will consider that the matching is completed and enter the online program, and the second storage virtualization controller That is, if the selected T2 value has not been reached when receiving the new PHY-mode information transmitted by the first storage virtualization controller, since the second storage virtualization controller is in the down stream, it will also Enter the online program, therefore, the inter-controller communication channel ICC can be correctly established between the two, or the second storage virtualization controller receives the new PHY-mode information of the first storage virtualization controller and knows that it is changed to upstream When the value of T2 is just reached and the PHY-mode conversion program is being executed, the second storage virtualization controller will stop the conversion and maintain the original PHY-mode of the down stream, and then successfully establish the inter-controller communication channel ICC between the two .

And if unfortunately, the T value selected by the two is the same, it may happen that the two complete the conversion process and transmit the new PHY-mode information at the same time, that is, after the conversion is completed, both parties will receive that the other party’s PHY-mode is up stream Therefore, after comparison, it will be known that the PHY-modes of the two parties are the same, so the two will detect that the matching is not successful, and will re-enter the matching procedure, and both parties will re-select the T value randomly, as long as the selected T value If it is different, the connection can be established successfully according to the above. And because the time of each coordination and matching is extremely short, and because the T value is selected by a random number, the probability that the two controllers continuously select the same T value is almost zero. Within a certain period of time, one must be coordinated as a down stream and the other as an up stream, and the inter-controller communication channel ICC is successfully established.

Please note that the operation processes adopted by the first and second storage virtualization controllers in this embodiment are the same (as shown in FIG. 8 ), that is to say, the design adopted by the first storage virtualization controller is the same as that of the second storage virtualization controller The two storage virtualization controllers are the same, so in this embodiment, the inter-controller communication channel ICC can be successfully established between the two storage virtualization controllers whose designs are completely consistent.

Of course, the two storage virtualization controllers need to use the PCI-Express connection to establish the inter-controller communication channel ICC. In addition to the aforementioned mechanism of automatically switching and establishing the connection through the PCI-E interface, other mechanisms that are not automatically established by the PCI-E interface can also be used. finished mechanism. For example: use the pin (pin) setting of the CPU chipset/colocation engine 244 to make (select or force) the PCI-E interface 934 to become upstream; or, utilize software to fill in the CPU chipset/colocation engine 244 The register of the PCI-E interface 934 makes (selects or forces) the PHY-mode of the PCI-E interface 934 to become upstream. However, in these two methods, the former needs to be manually set on site, and the latter needs to be set by software detection, which may cause a waste of time, so it is not ideal. However, it is more suitable for other embodiments of the present invention that adopt local bus interfaces without cross link features (such as PCI interfaces).

Because, when the power is turned on, the storage virtualization controller will use the external connection interfaces to send information to each device connected to the controller, and wait for the response of each device to know whether the connection has been successfully established and the information on each connection. What is the connected device. In principle, in the design of the entire system, except for the communication channel between redundant storage virtualization controller pairs, the external communication channels of other controllers are connected to passive devices, so their PHY-mode is up stream, so storage The virtualization controller can also use it to detect the connection status of each connection interface when starting up, and judge that the connection channel is an inter-controller communication channel ICC when it knows that the connected device is also operating in the down stream.

In order to improve the performance of the central processing unit of the storage virtualization controller, please continue to refer to FIG. 9, the present invention proposes a method for data transmission, including:

The central processing unit establishes a corresponding discrete aggregation table (Scatter-Gather (SG)) for data to be transmitted to another storage virtualization controller according to a pre-defined data-transfer-protocol format -list) (step S910);

The central processing unit sends a message containing the address of the SG-List to write to the register (register) belonging to the PCI-E interface that establishes the communication channel between the controllers (step S920); and

The PCI-E interface reads the SG-List according to the address information of the register, and reads the scattered data (scattered data) to be transmitted in the memory according to the content of the SG-List, and passes the scattered data through the controller The inter-communication channel ICC is transmitted to another storage virtualization controller (step S930).

The above-mentioned registers belonging to the interface for establishing the controller communication channel can be designed to be located in the PCI-E interface, or can be designed in a register functional block (such as a register array).

When the central processing unit writes to the register, since this register is designed to belong to the PCI-E interface of the communication channel between the establishment controllers, the register will send the address information to the establishment of communication between the controllers The PCI-E interface of the channel, and trigger (trigger) the interface.

In one embodiment, some specific address spaces in the register are planned and defined for writing the memory address storing the SG-List. Such a design will make when the central processing unit writes the memory address storing the SG-List to these specific register addresses, this writing action is equivalent to a trigger signal, so that the register can trigger the engine in the PCI-E interface To execute data transfer related procedures. In this way, the central processing unit only needs to write the memory address storing the SG-List into these specific register addresses.

Please refer to FIG. 10 , which is an embodiment of the SG-List format in the present invention. According to this embodiment, the SG-List content includes: the data quantity field (list-entry-count) in the table, in order to indicate the number of discrete aggregated data (SG data) contained in the table; the source starting address field (Source -base-Addr), indicating the memory start address where each piece of discrete data to be transmitted is stored; the data length field (Data-Length), indicating the length of the piece of discrete data to be transmitted stored in the aforementioned memory start address; and Destination-Base-Addr field (Destination-Base-Addr), indicating the destination address of the transmission discrete data to be stored; and so on.

When the central processing unit writes the memory address storing the SG-List into the register, the PCI-E interface reads out the discrete aggregation data in the SG-List according to the memory address indicated by the register, and The discrete aggregated data in each field defined above is operated, for example: after reading the control information in the header according to the address information of the register written by the central processing unit to the memory address, according to the table in the header The quantity indicated in the quantity field reads down the discrete aggregation data of that quantity sequentially, and the information of the entire SG-List will be stored in the register, and then establishes the PCI-E interface of the inter-controller communication channel according to each discrete The source start address field of the data and the indication of the content of the discrete data length field are sequentially transmitted to another storage virtualization controller through the ICC together with the target start address of the discrete data after reading the discrete data The PCI-E interface that establishes the communication channel between the controllers, when the PCI-E interface of another storage virtualization controller that establishes the communication channel between the controllers receives the information, it will store the discrete data in the target address according to the content .

And the first interrupt (INT) field can also be included in the SG-List, which is used to set whether after the data transmission listed in this table is completed, the interface establishing the communication channel between the controllers needs to generate an interrupt (interrupt) signal; And the second interrupt (Ints) field is used to set whether the destination end (another storage virtualization controller) completes writing these discrete data to the target address corresponding to each discrete data, and generates an interrupt (interrupt) signal notification CPU of itself (another storage virtualization controller).

If the SG-List defined in advance by the system has these fields, the interface for establishing the inter-controller communication channel will transmit the content of the second interrupt field, so that the interface for establishing the inter-controller communication channel of another storage virtualization controller can be obtained. Act according to the setting of this field, for example: 1 is to generate an interrupt signal to its own CPU, 0 is not to generate it. And after the interface for establishing the communication channel between controllers completes the data transmission indicated in the SG-List, it will determine whether to generate an interrupt signal to the central processing unit according to the indication of the first interrupt field of the SG-List , such as: 1 means generate, 0 means not generate.

In addition, due to factors such as memory space configuration, it may happen that a single SG-List cannot be used to list all the discrete aggregation data. In this case, several SG-Lists can be used to list and solve the problem. Please refer to FIG. 10 again. In this preferred embodiment, the format of the aforementioned SG-List still has a field (Next-SG-List Addr.) of the next SG-List address to indicate the next SG - The memory address stored in the List, so when the interface that establishes the communication channel between controllers reads the content of a certain SG-List, it can know where the next SG-List to be processed is stored according to the content of this field, and then Read the next SG-List according to the address, so that these SG-Lists can automatically generate a joint relationship, and when there is no next SG-List, this field will be set to 0, so that the SG-List can be known List is the last list. Therefore, the central processing unit does not need to write the address stored in the SG-List into the register for each SG-List, as long as the address of the first SG-List is written into the register, the communication between the controllers should be established. The interface of the channel will automatically complete all related SG-Lists.

Please refer to FIG. 11 , which shows a hypothetical example. Assuming that the central processing unit accepts a request sent by a host, the request-related information and data are scattered and stored in various parts of the memory through the CPU chipset, and the central processing unit is based on the memory address and length of these information and data storage. Four SG-Lists are generated.

As shown in the figure, suppose the address stored in the first SG-List is 0000_0100. After these SG-Lists are established, the CPU only needs to send the information containing the address 0000_0100 to write to the register in the CPU chipset that is set to belong to the interface for establishing the communication channel between controllers. Read the information in the SG-List from the PCI-E interface to the memory address 0000-0100, and obtain the control information (including the first interrupt (INT), the next SG-List address (Next-SG-List-Addr.), table The number of data in the table (list-entry-count) and other fields), and then read the information stored in the address 0000_0110 and 0000_0120 according to the number of data "2" displayed in the data number field in the table (including Source start address (Source-base-Addr), data length (Data-Length), target start address (Destination-Base-Addr) and other fields), and after reading all the contents of the first SG-List, Then read the discrete data into the buffer according to the initial address (1000_0000) and length (0000_0010) of the first discrete data stored in the memory, together with the target initial address ( A100_0000) are transmitted to another storage virtualization controller. When reading and transmitting the discrete data, the discrete data can be read and transmitted in batches according to the actual capacity of the buffer. Then complete the reading and transmission of each following discrete data in sequence according to the aforementioned method (such as: read the second discrete data in the memory according to the starting address (1100_0000) and data length (0000_0020) for storing the second discrete data) The first piece of discrete data is sent to the buffer (buffer), and the target starting address (A200_0000) to be stored for the second piece of discrete data is also sent to another storage virtualization controller). After completing the reading and transmission of all the discrete data listed in the first SG-List, and since the first interrupt (INT) field of the first SG-List is 0, the inter-controller communication is established The PCI-E interface of the channel will not send an interrupt signal to notify the central processing unit, that is, to read the second SG according to the information (0000_020) in the field of the next SG-List address (Next-SG-List Addr) to the memory address 0000_0200 -List, also obtain the content in the second SG-List according to the above method, and also complete the transmission of each discrete data listed in the list according to the above method, and then also according to Next-SG-List Addr. The internal information reads the third SG-List, and repeats each SG-List one by one until the Next-SG-List Addr address in the SG-List is set to 0000_000 (the fourth SG-List). In this example, since only the first interrupt (INT) field content of the fourth SG-List is 1, after completing the transmission action of the fourth SG-List, the PCI- The E interface will generate an interrupt signal to the CPU, so the CPU can know that the data of the four SG-Lists has been transmitted.

Using the aforementioned method, when performing redundant data transmission actions, the central processing unit only needs to maintain the SG-List and transmit and store the first SG-List Addr to the CPU chipset, and then all read discrete data and transmit discrete data Actions will be performed by the CPU chipset, requiring little or no CPU work.

Furthermore, when the inter-controller communication channel interface is established, after completing each transmission data operation, it reports to the central processor according to the INT field setting in the SG-list, so that the central processor can know that these discrete data have been successfully transmitted to Another storage virtualization controller can release the memory space for storing the transferred SG-List. At this time, the method of replying to the central processing unit every time an SG-List is completed can be adopted, that is, the INT field of each SG-List is all set to 1; The address of the next SG-List Addr is 0000_0) before replying to the CPU. As in the previous example, only the INT field of the last SG-List is set to 1, and the INT fields of other SG-Lists are all set to 0. The advantage of the former is that the memory space can be vacated in real time because the SG-List is restored every time a single SG-List is completed, which provides better flexibility and performance in the configuration and use of the memory space. As for which return method to adopt, it can be determined by the central processing unit according to the actual situation of the system.

In addition, generally speaking, in a redundant storage virtualization computer system, because the two storage virtualization controllers must maintain almost synchronization, when a storage virtualization controller has a change, it must notify the other storage virtualization controller , and usually the IO operation is very complicated when the system is running, so there may be a situation that new data or information needs to be transmitted to another storage virtualization controller continuously.

A better implementation method is proposed below, which is to add the following steps to the foregoing:

When the PCI-E interface that establishes the communication channel between controllers is processing a certain SG-List, the CPU chipset receives new information or data, and then the central processing unit establishes a corresponding new SG-List based on these information or data. (s), then the central processing unit can insert or connect the new SG-List(s) to the old unfinished SG-List(s), for example: by changing the ongoing connection of the PCI-E interface The data in the Next-SG-list Addr field of a SG-list that has not been processed in the memory of SG-lists or the data in the Next-SG-List Addr. field to be performed in the register, change it to The initial address of the newly established SG-List(s) is stored, so that there is also a linkage relationship between the SG-List(s) before and after, so that the PCI-E interface is automatically linked.

Further description is as follows, please refer to Figure 15, the CPU chipset notifies the central processing unit after receiving new data and storing the new data in the memory, and the central processing unit will establish several associated New SG-Lists (step S602).

Then the central processing unit judges whether there is an old SG-List(s) that has not been processed (step S604), and this step can be obtained by setting up the PCI-E interface of the inter-controller communication channel to send an Interrupt signal to the central processing unit It is known that if the central processing unit does not receive the Interrupt signal, the central processing unit knows that it has triggered the discrete data to be transmitted before completing the transmission. Assuming that if there is no old SG-List that has not been processed yet, write the start address of the newly established SG-List(s) into the register belonging to the interface for establishing the communication channel between the controllers to start the establishment of the controller The PCI-E interface of inter-communication channel carries out the transmission process of new data (step S606);

And if there are still existing SG-List(s) that have not been processed, the central processing unit sends a pause request (Pause Req.) to the PCI-E interface that establishes the communication channel between the controllers, requesting the interface to pause the action first (step S608). When the interface receives the request, it will execute a pause mechanism at an appropriate time, suspend the action at hand, and reply a pause approval (Pause Gnt) to the central processing unit after the pause. The suspension mechanism can be, for example: suspend the reading and transmission of subsequent discrete data after the discrete data being processed is completed, and record the suspension point so that the continuous processing can be resumed when the suspension is cancelled, or the SG- After all the discrete data in the List have been transmitted, the action to enter the next SG-List is paused.

After receiving the suspension approval reply, the central processing unit starts to insert or connect the continuous program (step S610), mainly to select a register or a certain unprocessed SG-List in the previous group of SG-Lists, and newly generate The Next-SG-List Addr. setting of the last SG-List in SG-Lists is consistent with the original setting of Next-SG-List Addr. , and change the Next-SG-List Addr. in the register or the unprocessed SG-List of the selected previous group in the memory to the starting address stored in the newly generated SG-Lists (store the first SG-List List address), that is, to complete the action of inserting the newly generated SG-Lists into the previous group of SG-Lists, so that the two groups of SG-Lists are linked.

After completing the joint relationship between the newly generated SG-List(s) and the existing SG-List to be processed, then notify the PCI-E interface that establishes the communication channel between the controllers to release the suspension state, and then establish the communication channel between the controllers The interface will resume the paused action, and continue to automatically link and process according to the new joint relationship. (step S620)

The reason why the aforementioned pause request needs to be sent before the central processor performs two sets of SG-Lists linkage is to prevent the central processor from changing the information in a certain SG-List in the memory or in the Next-SG-ListAddr. field in the register. At the same time, the PCI-E interface that establishes the communication channel between the controllers also reads the content of the SG-List, and a conflict occurs and an error occurs.

The insertion or connection sequence (S610) will be described below. Please refer to FIG. 16 , which is an embodiment of an insertion or connection program. In this embodiment, the newly generated SG-List(s) is inserted directly after the SG-List being processed by the register.

Since the interface for establishing a communication channel between controllers will first obtain the content of the SG-List when a certain SG-List is to be executed, so when the aforementioned situation occurs, the central processing unit can read the relevant Next-List stored in the register. The information in the SG-List Addr. field can be used to know whether the PCI-E interface has the next SG-List to be processed or where the next SG-List to be processed is stored. Therefore, as shown in Figure 16, the first step in the program of inserting or connecting the continuous operation at the central processing unit is that the central processing unit first reads the Next-SG-List Addr field in the SG-List being processed in the register Address data in (step S612), and after obtaining the Next-SG-List Addr. information that this PCI-E interface would have carried out originally, it directly adds the Next-SG-List of the last SG-List in the newly generated SG-Lists. The setting of SG-List Addr. is consistent with the original setting of Next-SG-List Addr. in the previously read register, and the Next-SG-List Addr. in this register is changed to the newly generated SG-List Addr. The starting address (the address of storing the first SG-List) stored in the Lists is to complete the action of inserting the newly generated SG-Lists into the previous group of SG-Lists, so that the two groups of SG-Lists are linked (step S614 ). That is to say, insert the newly generated SG-Lists directly before the SG-List to be processed after the SG-List being processed by the interface, and after completing the data transmission corresponding to the newly generated SG-Lists , it will jump back to continue the unfinished SG-List(s) of the previous group that would have been continued at the insertion point. Of course, if the insertion point is just after the last SG-List of the previous group of SG-List(s) (the Next-SG-List Addr. in the read register is 0000_000), then it will not jump back to continue The unfinished SG-List(s) of the previous group, because there is no unfinished SG-List(s) of the previous group.

Take the aforementioned FIG. 11 as an example, and refer to FIG. 12 again. First, as shown in Figure 11, the central processing unit establishes four interlinked SG-Lists according to the storage address of the discrete data that the central processing unit wants to transmit to another storage virtualization controller (companion), and will contain the first The address information of each SG-List is written into the register of the PCI-E interface as the communication channel between the controllers, and the interface reads the content of the SG-List according to the address information of the register, and performs operations such as reading and transmitting data.

In addition, the CPU chipset receives new data and stores the new data into the memory to notify the central processing unit, and the central processing unit also establishes several new SG-Lists linked to each other for these new data (please Refer to Fig. 12), so that this data can be transmitted to its companions for storage, please note that, as in Fig. 11, although the example of generating many SG-Lists and multiple discrete data is used here, the same actual reason Factors such as data volume and memory space configuration may only generate a new SG-List or a single discrete data content.

If when the central processing unit sets up the SG-Lists corresponding to the new data (as shown in Figure 12), the PCI-E interface of the communication channel between the controllers has not yet processed the four SG-Lists of Figure 11, at this time , the central processing unit sends a pause request (Pause Req.) to this interface, requesting to pause and enter the action of processing the next SG-List first, when the interface receives the request, it will execute a pause mechanism to suspend the SG-List in hand All actions, and reply a pause approval (Pause Gnt) to the central processing unit. After receiving the suspension approval reply, the central processing unit starts to carry out the linkage program, and the central processing unit reads the information in the relevant Next-SG-List Addr. field in the register.

Assuming that the interface is processing the second SG-List in FIG. 11 at this time, the address information read by the CPU will be 0000_040. Then the central processing unit sets the Next-SG-List Addr. field of the last SG-List in Fig. 12 to 0000_040, and changes the Next-SG-List Addr. in this register to 0000_050 (the first one in Fig. 12 address stored in the SG-List), then complete the action of inserting the newly generated SG-List (as shown in Figure 12) between the second and third SG-List shown in Figure 11.

And if the aforementioned interface that establishes a communication channel between controllers just in time handles the fourth (last) SG-List in Fig. 11 when receiving the suspension request sent by the central processing unit, the register read by the central processing unit The Next-SG-List Addr. in the SG-List will be 0000_000, then the Next-SG-List Addr. in the last SG-List shown in Fig. The Next-SG-List Addr. in the read register is consistent), and the Next-SG-List Addr. in the register is changed to 0000_050 (the address stored in the first SG-List in Figure 12) That's it.

After completing the interlocking procedure, the central processing unit then sends a notification of releasing the suspension state to the interface that establishes the communication channel between the controllers, so that the interface cancels the suspension mechanism, continues the action before the suspension mechanism, and handles the SG at the time of the suspension -List, it will then read the newly generated first SG-List according to the Next-SG-List Addr written by the CPU during the pause period, and perform all pending Data transfer corresponding to the processed SG-List(s).

Please refer to FIG. 17 again, which shows another embodiment of the interlocking program of insertion and connection. Different from FIG. 16, the linkage insertion point selected in this embodiment is between or after the SG-List(s) that have not yet been processed to establish the interface of the inter-controller communication channel.

As in Fig. 16, the first step of inserting the linked program at the central processing unit is still that the central processing unit reads the address data in the Next-SG-List Addr field in the SG-List being processed in the register (step S612), then the central processing unit can judge whether the address in the register is 0000_000, to know whether there is an SG-List(s) (step S616) that needs to be processed subsequently; if the address is set to 0000_000, it means that There is no next SG-List to be processed next, so the central processing unit only needs to change the Next-SG-List Addr. in the register to the first SG-List of the newly generated SG-List(s). address (step S617), the linkage between two sets of SG-Lists can be completed.

If the Next-SG-List Addr. of this register is not 0000_000, it means that there are other SG-List(s) that are originally set to be processed but have not been processed. At this time, the central processing unit can select other SG-List(s) The next unprocessed SG-List is inserted. Read the Next-SG-List Addr. in the selected SG-List in the memory, set the Next-SG-ListAddr. Next-SG-List Addr. in the selected SG-List is changed to the address stored in the first SG-List that is newly generated, which completes the linkage program (step S618).

For example, the central processing unit selects the last SG-List in the SG-Lists of the previous group for linkage, because the Next-SG-List Addr. -The Next-SG-List Addr. of the List is originally the same (0000_000), so you only need to change the Next-SG-List Addr. of the last SG-List in the previous group of SG-Lists to the newly generated SG- The initial memory address stored in List(s) is to complete the linkage of these two sets of SG-Lists. In this case, after the suspension state is released, the PCI-E interface that establishes the communication channel between controllers will automatically continue to process the newly generated SG-List(s) after processing and transmitting the previous group of SG-List(s) . That is to say, the newly generated SG-List(s) is inserted and linked after the previous SG-List(s). And to find the last SG-List in the previous group of SG-Lists, you can use to set up a table that stores the memory addresses of all SG-Lists, and use this table to query and obtain, or according to the address in the read register Next-SG-List Addr, the address information to the address to read the Next-SG-List Addr. of the SG-List stored in the address, if the Next-SG-List Addr. is not yet 0000_000, then follow Sequentially read the Next-SG-List Addr of the next SG-List until the Next-SG-ListAddr is 0000_000.

Also take Figure 11 and Figure 12 as an example, also assume that when the interface for establishing the communication channel between the controllers proceeds to a certain SG-List in Figure 11 and receives a pause request sent by the central processor, the register will execute the pause mechanism, and Send a pause acknowledgment to the CPU upon completion. Then the central processing unit reads the Next-SG-List Addr. in the register, if the Next-SG-List Addr. in the register is 0000_000, as: the interface of establishing the communication channel between the controllers is processing the fourth in Fig. 11 SG-List, then at this time, change the Next-SG-List Addr. in the register from 0000_000 to the address (0000_050) stored in the first SG-List among the SG-Lists to be processed as shown in Figure 12 ), the two sets of SG-Lists can be linked together, and then the PCI-E interface that establishes the communication channel between the controllers is triggered to release the suspension state, then the interface will continue to perform the action before the suspension mechanism, and will process the transmission After completing the three discrete data in the fourth SG-List of the previous group, according to the address (0000_050) in the Next-SG-List Addr. field in the register, then process the newly generated SG-Lists in Figure 12.

And if the Next-SG-List Addr. in the read register is not 0000_000, for example: if the inter-controller communication channel interface is being established and is being processed and suspended in the second SG-List shown in Figure 11, in the register The information in the Next-SG-List Addr. field is 0000_040, so the CPU reads the field information and knows that the address of the SG-List that has not been processed by the PCI-E interface but will be processed next is 0000_040. Then the central processing unit can select the desired insertion point according to the situation or system setting. For example, the selection is inserted after the last SG-List. Then you can use the central processing unit to obtain the Next-SG-List Addr of the third SG-List in the memory address of 0000_0400. SG-List Addr., because it is 0000_000, it means that there is no linked SG-List next. After finding the last SG-List, change its Next-SG-List Addr. from 0000_000 to that shown in Figure 12 The address 0000_050 stored in the first SG-List shown makes the two groups of SG-Lists generate a linkage relationship (as shown in Figure 13), and then notifies the establishment of the communication channel interface between the controllers to release the suspension state, so that the establishment of the control The inter-device communication channel interface continues to complete the data transmission actions corresponding to each SG-List in sequence. That is to say, the actions before the inter-controller communication channel interface continuation pause mechanism continue to operate, and after the second SG-List in the previous group of SG-Lists is processed, the third SG-List in the previous group of SG-Lists is still processed sequentially. The first SG-List and the fourth SG-List, and after processing the last SG-List of the previous group of SG-Lists (the fourth SG-List whose storage address is 0000_0300), it will automatically continue to process the second group of SG-Lists Lists without sending any information from the CPU.

Or, it is inserted after the next SG-List to be processed is selected to establish the communication channel interface between controllers. Then the CPU also utilizes the Next-SG-List Addr. (0000_040) in the read register to obtain the Next-SG-List Addr. in the memory address 0000_0400 as 0000_030, and change the content of this field in the memory as shown in Figure 12 The address 0000_050 stored in the first SG-List of the newly generated SG-Lists shown in Figure 12, and fill in the Next-SG-List Addr. field of the last SG-List of the newly generated SG-Lists shown in Figure 12 The aforementioned reading of the Next-SG-ListAddr.(0000_030) at the insertion place makes the two sets of SG-Lists generate a linkage relationship (as shown in FIG. 14 ). Then the same central processing unit notifies the establishment of the inter-controller communication channel interface to release the suspended state, so that the establishment of the inter-controller communication channel interface continues to complete the data transmission action corresponding to each SG-List in sequence. That is to say, the action before establishing the inter-controller communication channel interface continues the pause mechanism, and after processing the original establishment of the inter-controller communication channel interface, the processing SG-List will be continued (the third SG in Figure 11 -List), it will automatically jump to process the inserted SG-Lists first, and then jump back to process the last SG-List of the previous group (the fourth SG-List whose storage address is 0000_0300), without central processing sender any information.

Of course, in addition to the last or the last SG-List can be selected as the insertion point, the selected insertion point can also be the second or third SG-List to be processed, as long as it does not exceed the last one. Can. As in the previous embodiments, when new data to be transmitted is generated, as long as the communication channel interface between the controllers is established and the data of a certain group is being processed and transmitted, a linkage relationship between the old and new data will be generated by using the aforementioned mechanism, so that the central The processor does not need to wait until the interface of the inter-controller communication channel is established to report that the old data has been transmitted, and then writes the start address of the SG-List(s) of the new data to the register and then triggers the establishment of the inter-controller communication channel interface. The communication channel interface will automatically process and transmit this new data. And because in theory, in the operation of the subsystem, the storage virtualization controller will constantly have data to be sent to its companion (another storage virtualization controller) for storage, that is, the previous or even multiple data has not been completely New data will be generated when it is sent to peers. Therefore, as far as the aforementioned mechanism is concerned, the central processing unit only needs to establish the SG-List(s) corresponding to the maintenance data and the first SG-List(s) generated after the system starts. The storage address of the List is written into the register of the establishment of the communication channel interface between the controllers of the ICC, and then the interface of the communication channel between the establishment of the controllers is continuously reading and transmitting data, which will effectively and greatly share the work of the central processing unit, and then Improve the working performance of the central processing unit.

Furthermore, when the central processing unit needs to integrate or modify or delete a certain SG-List(s), as the aforementioned insertion or continuous continuous program, the central processing unit can read the relevant Next-SG-List stored in the register The information in the Addr. field tells whether the PCI-E interface has the next SG-List to be processed or where the next SG-List to be processed is stored, that is, the central processing unit can use this to know which SG-List is an interface for establishing an inter-controller communication channel that has not yet been processed. Therefore, the central processing unit can determine whether a certain SG-List(s) that it intends to integrate or modify or delete is an SG-List(s) that has not yet been processed by establishing an inter-controller communication channel interface, and if so, the central processing unit can The SG-List is integrated or modified or deleted.

Of course, when the central processing unit performs the aforementioned modification or deletion program, before reading the information in the register, it is also possible to send a pause request to request the establishment of an inter-controller communication channel interface to first suspend the actions related to the transmission of redundant data. The reading action is performed after receiving the suspension approval, so as to avoid conflicts with the subsequent modification or deletion actions of the central processing unit due to the establishment of the inter-controller communication channel interface for continuous transmission work. And similarly, after the central processing unit completes the entire modification or deletion program, it will notify the establishment of the inter-controller communication channel interface to release the suspended state, so that it can continue to operate.

In addition, the central processing unit can combine the above-mentioned insertion or continuation of the continuous program and the modification or deletion program according to the actual situation, for example: in a hypothetical example (Fig. To change the Next-SG-List Addr. field data of the fourth SG-List in the memory to the start address of the newly generated SG-List(s) to be processed continuously, you can use it according to actual needs (for example: CPU) If you want to set the communication channel interface between the controllers to generate an interrupt signal to notify the CPU when all the SG-Lists of the new joint relationship are completed), change the setting of the Int field of the SG-List, and change the data in this field to If it is 0, the inter-controller communication channel interface will not send an Interrupt signal to the central processing unit after completing the fourth SG-List. Or such as: when modifying or deleting the program, according to the current situation or needs, you can use the Next-SG-List Addr. of the SG-Lists that have not been established for the inter-controller communication channel interface to read and process to reset. Determine the joint relationship between these unprocessed SG-Lists.

In addition to the aforementioned method of transferring data to another storage virtualization controller and then storing it in its memory, if the system is designed, one storage virtualization controller can store data in the memory of another storage virtualization controller in a redundant configuration. When fetching (write/read) operations, there is a data direction (Dir) field in the SG-List to indicate the execution of write or read operations, for example: 1 for write (Data Out), 0 for Read (Data In) (see Figure 10).

Therefore, the central processing unit writes the memory address storing the SG-List into the register, and the interface to establish the communication channel between the controllers reads out the data in the SG-List according to the address information transmitted by the register, and according to the aforementioned The instruction information in each defined field operates.

If the Dir field is set to indicate the write operation, then the interface can follow the indications of the source start address field of each piece of discrete data and the content of the discrete data length field as described in the foregoing examples, sequentially following the read operation. Fetch the piece of discrete data, and then transmit the piece of discrete data, the target start address of the piece of discrete data, and the instruction to perform the write operation to the establishment control of another storage virtualization controller through the communication channel between the controllers Inter-device communication channel interface.

And if it is a read operation, in one embodiment, the interface for establishing an inter-controller communication channel can transmit the content of the SG-List to the interface for establishing an inter-controller communication channel of another storage virtualization controller, and the other end interface After receiving and storing in the register, read each piece of discrete data sequentially according to the source start address field of each piece of discrete data and the content of the length field of the discrete data, and then store the information related to the discrete data and the discrete data (such as : Target start address, data length) is sent back to the interface for establishing a communication channel between controllers, and the interface stores the data in the memory according to the target start address. In another embodiment, the whole SG-List content is sent to the other end, but only the field data related to each discrete data in the SG-List is sent, and even the fields related to each discrete data are sent in batches Data (for example: source start address and data length) can be transmitted with an instruction indicating the read operation, and after receiving the discrete data obtained from the other end according to the source start address and data length, then The start address of the target to store the discrete data. Finally, regarding the transmission of a small amount of data between storage virtualization controllers, the present invention proposes another method for data transmission, which can further improve the overall performance of storage virtualization controllers.

Please refer to FIG. 18 , which shows another method for transferring data between two storage virtualization controllers proposed by the present invention. After the two storage virtualization controllers coordinate to establish a communication channel (step S702), if the central processing unit of one of the storage virtualization controllers has information that needs to be transmitted to the other, the central processing unit will directly send the data transmission The request information is sent to the CPU chipset (step S704), and the CPU chipset will respond to the information and transmit the information to be delivered to another storage virtualization controller (step S706), and the storage virtualization controller in another storage virtualization controller The CPU chipset receives and processes directly.

Please cooperate with FIG. 5, according to the method proposed by the present invention, when the central processing unit 242 needs to transmit information to the second storage virtualization controller, one of the implementation methods for transmitting the data transmission request information is for the central processing unit 242 sends the information to be sent to the other end to the IM BUS 950 through the CPU interface 910, and is read by the PCI-E interface 934 establishing the inter-controller communication channel ICC and sent to the second storage virtualization controller 200'. In addition, when the PCI-E interface 934 of a storage virtualization controller (for example: the first storage virtualization controller) receives information transmitted by another storage virtualization controller (for example, the second storage virtualization controller), Then it is sent to the memory controller 920 through the IM BUS950, and then stored in the memory 280.

An embodiment is proposed below for further description. First, it is set that for each storage virtualization controller, the memory of another storage virtualization controller in redundant configuration is regarded as an extension of its own memory. Assuming that the memory spaces of the first storage virtualization controller and the second storage virtualization controller are both 2G, the physical memory address of each storage virtualization controller is set to 0000_0000-7FFF_FFFF, and another storage The memory address of the virtualization controller is considered as 8000_0000-FFFF_FFFF. In this example, the memory address of another storage virtualization controller is directly connected to its own memory address. It is virtual memory address 2G+P. However, indirect connection can also be used in practice. For example, the physical memory address P of another storage virtualization controller is regarded as the virtual memory address 3G+P, as long as it does not overlap with its own physical memory address in principle. Furthermore, although the two storage virtualization controllers have the same storage space as an example here, in practice, the memory space of the two controllers may be different. For example, the first storage virtualization controller is 2G but the second storage virtualization controller is 2G. The virtualization controller is 1G, and the physical or virtual memory address also depends on the memory space. And according to the present invention, it is assumed that for each storage virtualization controller, the other side of the redundant configuration has a part or all of the memory available for it to access.

Assuming that when a storage virtualization controller intends to write certain data into the memory address 2100_0000 of another storage virtualization controller, the CPU 242 directly transmits the target start address (A100_0000) of the write object and the write data to the CPU interface 910, and then the interface 910 transmits the address (A100_0000) and write data to the IM BUS950, and is read by the PCI-E interface 934 coupled to another storage virtualization controller. Because the target starting address in the information sent by the central processing unit 910 is to point to the virtual memory address of another storage virtualization controller, and among the external interfaces in the CPU chipset 244, only the PCI-E interface 934 is used. Designed as an interface to communicate with another storage virtual machine controller, so that each component can be identified and only the interface that establishes the communication channel between the controllers will read and perform data transfer.

In one embodiment, after the PCI-E interface 934 establishing the inter-controller communication channel reads the target start address A100_0000 and writes data, it first converts the target start address A100_0000 into the corresponding physical memory address 2100_0000, It is sent to another storage virtualization controller together with the written data. When another storage virtualization controller receives information such as the physical memory address 2100_0000 and write data, it writes the data into the physical address 2100_0000.

In another embodiment, the step of converting the virtual memory address into the corresponding physical address is performed by the interface of the receiving end. That is, the PCI-E interface 934 establishing the inter-controller communication channel transmits the target start address A100_0000 and the write data to another storage virtualization controller. And when the PCI-E interface of another storage virtualization controller reads information such as the target start address A100_0000 and write data transmitted by the other end, it converts the target start address A100_0000 (virtual memory address) into the corresponding The corresponding physical address is 2100_0000, and the data is written into the physical address 2100_0000.

According to this embodiment, when the communication channel ICC is established between two storage virtualization controllers, as far as each storage virtualization controller is concerned, another storage virtualization controller with redundant configuration is regarded as a terminal device, so It can access the memory it has. Therefore, in addition to the aforementioned data transmission and writing to another storage virtualization controller, data in the memory of another storage virtualization controller may also be read.

If a storage virtualization controller is designed to directly write and read the memory of the other side of the redundant configuration, the data transmission request issued by the central processing unit of the storage virtualization controller must include access command to indicate a write or read operation.

If it is to perform a write operation, it is as described above. And if it is a read operation, the data transmission request still includes the starting address of the data source and the data length. Likewise, since the source start address is a virtual memory address pointing to another storage virtualization controller, the interface that establishes the inter-controller communication channel will read the request and forward it to another storage virtualization controller in the redundant configuration after processing. The storage virtualization controller, and transfers to the central processing unit after receiving the report information from another storage virtualization controller.

When the inter-controller communication channel interface established in the CPU chipset receives a request sent by another controller, it interprets the storage/fetch instructions in the request and performs corresponding write or read data operations. If it is a write operation, it is as described above. If it is a read operation, the data will be read according to the source start address and data length and reported to the requester. The entire storage/retrieval process is executed by the CPU chipset itself. And like the aforementioned write operation, the step of converting the virtual memory address in the data transmission request into the corresponding physical memory address can be performed by the interface establishing the communication channel between the controllers at the sending end or the receiving end. That is to say, this method mainly uses the CPU to directly issue transmission requests to the CPU chipset, and the interface that establishes the inter-controller communication channel reads it and transmits it to the storage virtualization controller at the other end, so the CPU does not need to perform these requests. The data establishes the corresponding SG-List.

And if an interface establishing a communication channel between controllers receives data transmitted by another storage virtualization controller, the entire data storage/retrieval process is executed by the CPU chipset without using its central processing unit at all. .

Therefore, according to the structure of this embodiment, one storage virtualization controller can directly perform access operations to another storage virtualization controller in redundant configuration, so that when it is damaged and the other side takes over its work, it can recover In this case, it is possible to directly read the contents of the other party's memory, or the other party actively transmits relevant information, so that information such as the working status and progress after taking over can be obtained, and the subsequent processing can be resumed. It can also treat the memory of another storage virtualization controller as an extension of its own memory.

And because of the aforementioned method, the central processing unit does not establish a corresponding SG-List for the data, and it can show obvious effects when transmitting a small amount of data, because for a small amount of data, for example: a data of only 1 byte, if you need According to the data transmission protocol format to establish the corresponding SG-List and other actions, the SG-List created by the CPU in the memory may be as high as 16bytes, and the CPU chipset still needs to read the SG-List and process it according to its content, so It appears to be very uneconomical. In a preferred embodiment, the aforementioned data transmission method with SG-List characteristics can be used. The storage virtualization controller can decide which method to use to transmit data to another storage virtualization controller according to the amount of data to be transmitted, so that the system performance can be optimized.

In summary, in the redundant storage virtualization computer system proposed by the present invention, the communication channel between the two storage virtualization controllers is directly achieved by using a local bus (such as PCI-Express in the embodiment), which is different from the conventionally used External connections such as FC-AL, SATA or SCSI are completely different, and compared, the present invention does not need to go through an intermediate referral circuit, therefore, its circuit design is not only comparatively simplified, and cost is also lower.

Moreover, in order to solve the problem that the two storage virtualization controllers are active devices and cannot really connect the communication channel between the two, in the embodiment of the present invention, the local bus interface is used to automatically send the The function of including its own interface setting status and other information to the other connection end, whereby each storage virtualization controller can understand the mode of its other connection end to establish a communication channel interface, so it can detect whether the two are in a reliable state In the state of establishing the connection, if it is in the state of being unable to establish the connection, then enter the switching program to switch the operation mode of the local bus interface of at least one communication channel end so that the connection can be successfully established.

For the conversion program, in addition to proposing the implementation of unilateral conversion, an automatic matching embodiment is also proposed, which uses random random number selection to determine a waiting time value, and switches the interface operation mode after the time is reached. It will happen that the selected waiting time values are different, so one of them must arrive first and be converted first, so the operation modes of the interfaces at the two ends of the communication channel between the two storage virtualization controllers can be successfully matched.

In addition, in the part of how to use the communication channel formed by the local bus to transmit data, an implementation method is proposed, so that the central processing unit only needs to establish and maintain the SG corresponding to the data to be transmitted according to a pre-defined data transmission protocol format. List, and write the storage address of the first SG-List into the registers in the CPU chipset. Other operations to read data and transfer these data will be completely determined by the controller who establishes the communication channel between the controllers. The interface is executed, so it can effectively reduce the problem of affecting the performance of the central processing unit caused by the requirements of the redundant storage virtualization controllers to maintain mutual data synchronization and consistency. And in a further embodiment, it is possible to create a linkage relationship between the new data generated before the transmission of the previous data is completed, so that the interface establishing the communication channel between the controllers can be automatically continued for processing.

Finally, in order to optimize the performance of storage virtualization controllers, a method for transmitting information between redundant storage virtualization controllers is also proposed for the small amount of data transmission between redundant storage virtualization controllers , which is mainly used after the communication channel is established between the two storage virtualization controllers, the central processing unit in the controller directly transmits the request information to the other end through the ICC, without the need to construct the SG-List, therefore, for a small amount of data transmission In addition, the system processing performance can be increased.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the patent of the present invention.

Claims (61)

1. computer system includes:

One main frame is used for sending output and goes into request;

One group of redundant storage virtualization controller, being used to carry out output goes into operation and goes into request with the output that responds this main frame and send, it includes one first and one second storage virtualization controller that is coupled to this main frame, this first and this second storage virtualization controller between be to utilize a local bus to communicate; And

One group object memory storage is coupled to these storage virtualization controllers, is used to provide this computer system stores space;

Wherein, when this first storage virtualization controller situation occurred, this second storage virtualization controller will automatically be taken over the original function of carrying out of first storage virtualization controller of this situation occurred;

Wherein this local bus is to adopt the mode of cable or backboard to be connected in this two to deposit virtualization controller, this first with this second storage virtualization controller respectively include a local bus interface, in order to set up the local bus communication channel between this first and second storage virtualization controller, and one by this two local bus interface is gone to change its operator scheme, so that set up online between the local interface of these storage virtualization controllers.

2. computer system as claimed in claim 1, wherein, this local bus is one of following:

Perimeter component links bus, perimeter component links expansion bus or perimeter component links quick bus.

3. computer system as claimed in claim 1, wherein, this two local bus interface is each positioned at a central processing unit chipset, and at least one the pin that utilizes this two central processing units chipset is set and is made this described one local bus interface of this two central processing units chipset go to change its operator scheme, so that set up online between the local bus interface of these storage virtualization controllers.

4. computer system as claimed in claim 1, wherein, be to utilize software to fill in one register of this two local bus interface and make described one of this two local bus interface to go to change operator scheme, so that the local bus interface of these storage virtualization controllers is set up is online.

5. computer system as claimed in claim 1, wherein, these local bus interface also have the feature that intersection links.

6. computer system as claimed in claim 5, wherein, at least one of these local bus interface carried out an automatic transfer machine system, the feature of utilizing this intersection to link is changed described one operator scheme of these local bus interface, so that can set up online between this first and second storage virtualization controller.

7. Storage Virtualization subsystem includes:

One group of redundant storage virtualization controller, being used to carry out output goes into operation and goes into request with the output that responds a main frame and send, it includes one first and one second storage virtualization controller that is used for being coupled to this main frame, this first and this second storage virtualization controller between utilize a local bus to communicate; And

One group object memory storage is coupled to these storage virtualization controllers, is used to provide this Storage Virtualization subsystem stores space;

Wherein, when this first storage virtualization controller situation occurred, then this second storage virtualization controller will automatically be taken over the original function of carrying out of first storage virtualization controller of this situation occurred;

Wherein this local bus is to adopt the mode of cable or backboard to be connected in this two to deposit virtualization controller, this first with this second storage virtualization controller respectively include a local bus interface, in order to set up the local bus communication channel between this first and second storage virtualization controller, and one by this two local bus interface is gone to change its operator scheme, so that set up online between the local interface of these storage virtualization controllers.

8. subsystem as claimed in claim 7, wherein, this local bus is one of following: perimeter component links bus, perimeter component links expansion bus or perimeter component links quick bus.

9. subsystem as claimed in claim 7, wherein, this two local bus interface is each positioned at a central processing unit chipset, and at least one the pin that utilizes this two central processing units chipset is set and is made this described one local bus interface of this two central processing units chipset go to change its operator scheme, so that set up online between the local bus interface of these storage virtualization controllers.

10. subsystem as claimed in claim 7, wherein utilize software to fill in one register of this two local bus interface and make described one of this two local bus interface to go to change operator scheme, so that set up online between the local bus interface of these storage virtualization controllers.

11. subsystem as claimed in claim 7, wherein, these local bus interface also have the feature that intersection links.

12. subsystem as claimed in claim 11, wherein, at least one of these local bus interface is to carry out an automatic transfer machine system, the feature of utilizing this intersection to link is changed described one operator scheme of these local bus interface, so that can set up online between this first and second storage virtualization controller.

13. a method of setting up the communication channel between the storage virtualization controller may further comprise the steps:

These storage virtualization controllers are by comprising the information of its operator scheme as the local bus interface transmission one of communication channel end between controller;

In these storage virtualization controllers at least one compares to judge whether to set up the operation mode information of this another storage virtualization controller of being received and the operator scheme of self online when receiving the operation mode information of another this storage virtualization controller by this local bus interface;

If it is online that judgement can be set up, then directly set the communication channel of setting up between this two storage virtualization controller according to this local bus interface; And

If be judged as can't set up online, then at least one in these storage virtualization controllers will be changed the operator scheme of this local bus interface, so that corresponding, and then set up each other online with the operator scheme of this local bus interface of another this storage virtualization controller;

Be to utilize a local bus to communicate between one first and one second storage virtualization controller in these storage virtualization controllers wherein, this local bus is to adopt the mode of cable or backboard to be connected in this two to deposit virtualization controller, this first with this second storage virtualization controller respectively include this local bus interface, in order to set up the local bus communication channel between this first and second storage virtualization controller, and one by this two local bus interface is gone to change its operator scheme, so that set up online between the local interface of this storage virtualization controller.

14. method as claimed in claim 13, wherein, these local bus interface are that perimeter component links quick bus interface.

15. method as claimed in claim 13, wherein, this local bus is to have the feature that intersect to link, and online for setting up when this comparison judged result, and then this storage virtualization controller is an operator scheme of utilizing the feature of this intersection binding to come translation interface.

16. method as claimed in claim 13, wherein, when judged result relatively for setting up in the step performed after online, include one and bring mechanism together so that the operator scheme of these local bus interface corresponds to each other, and set up online.

17. method as claimed in claim 16, wherein, this match mechanism system may further comprise the steps:

A selected at random timing critical parameters value then starts timing;

If before the time reaches this critical value, receive the new operation mode information of the other side and show that it has changed operator scheme, then finish bring together and set up online;

If when the time reaches this critical value, and do not receive the information that the other side's operator scheme has changed, then carry out this conversion operations mode step, and contain new operational mode status information to the other side in converting back transmission one; And

Whether the operator scheme kenel that rejudges between the two is inequality, if inequality then brining together finished and set up online, if identical, then re-executes this match mechanism, brings together up to this and finishes.

18. method as claimed in claim 17, wherein, in this match mechanism, more may further comprise the steps in the step of this execution conversion operations pattern: if when carrying out the conversion operations pattern, receive the operation mode information that the other side transmits, then end conversion and keep the operator scheme that originally set.

19. method as claimed in claim 18 wherein, before this ends conversion, is whether the operator scheme of carrying out the other side relatively earlier is different with itself operator scheme before the conversion, if just carry out this terminations switch process, otherwise continue execution conversion.

20. a method of carrying out data transmission in a computer system between storage virtualization controller comprises following steps:

A. the central processing unit of this storage virtualization controller one discrete data of another storage virtualization controller is defeated by in tendency to develop according to the Data Transport Protocol form of a predefined is set up at least one corresponding discrete gathering table;

B. this central processing unit address that will deposit this discrete gathering table writes a register; And

C. a local bus interface writes in address to an internal memory in the register according to this and reads this discrete gathering table, this discrete data is read in a address according to indicated this discrete data of storage of discrete aggregate data in this table to this internal memory, and transmits this discrete data by a local bus and give another storage virtualization controller;

Be to utilize a local bus to communicate between one first and one second storage virtualization controller in these storage virtualization controllers wherein, this local bus is to adopt the mode of cable or backboard to be connected in this two to deposit virtualization controller, this first with this second storage virtualization controller respectively include this local bus interface, in order to set up the local bus communication channel between this first and second storage virtualization controller, and one by this two local bus interface is gone to change its operator scheme, so that set up online between the local interface of this storage virtualization controller.

21. method as claimed in claim 20, wherein, this local bus is for one of following: perimeter component links bus, perimeter component links expansion bus and perimeter component links quick bus.

22. method as claimed in claim 20, wherein, after the address that this discrete gathering table deposited this by this central processing unit write this register, this register was that this address information is sent to this local bus interface, and triggered this local bus interface and carry out this step C.

23. method as claimed in claim 20, wherein, this central processing unit is that this address of depositing this discrete gathering table is write the storage space that is defined in this register as the particular address that writes this usefulness of depositing discrete gathering table address.

24. method as claimed in claim 20, wherein, the Data Transport Protocol form of this discrete gathering table is to comprise following field:

Data bulk field in the table is in order to the number of the discrete aggregate data that includes in the indicating gauge;

The source origing address field (OAF) is failed its stored internal memory start address of discrete data in order to indicate every tendency to develop;

Data length field is in order to indicate the length of the defeated discrete data of this every tendency to develop; And

The target origing address field (OAF) is in order to the destination address of indicating the transmitting discrete data to desire to deposit.

25. method as claimed in claim 24, wherein, among the step C, this local bus interface is carried out and be may further comprise the steps:

Read according to the control information that information that this central processing unit sent will disperse to this internal memory in the gathering table header, according to the quantity of this discrete aggregate data down read in regular turn this quantity should discrete aggregate data control information; And

According to this source origing address field (OAF) in this each discrete data that is read and the indication of this data length field content, read this discrete data one by one, and will this every discrete data be transferred to this another storage virtualization controller together with this target start address of this discrete data by this local bus.

26. method as claimed in claim 25, wherein, this discrete data that this local bus interface read is to be temporary in an impact damper earlier, just transfers out together with this target start address thereafter.

27. method as claimed in claim 26 wherein, when this local bus interface reads and transmit this discrete data at this, reads this discrete data and transmits according to the actual capacity of this impact damper in batches.

28. method as claimed in claim 25 wherein, comprises that more the local bus interface of this another storage virtualization controller receives this discrete data and this target start address, this discrete data is stored in the step in the internal memory of this destination address.

29. method as claimed in claim 25, wherein, the Data Transport Protocol form that is somebody's turn to do discrete gathering table more comprises a data direction field, carries out writing or read operation in order to indication, makes this local bus interface carry out corresponding accessing operation according to this field contents.

30. method as claimed in claim 29, wherein, at step C, execute the step that reads discrete gathering table content in this local bus interface after, more include:

Content according to this data direction field is carried out corresponding writing or read operation;

If this content is designated as the execution write operation, then carries out the discrete gathering of these follow-up foundations and show each field and read this discrete data and transmitting discrete data step to this another storage virtualization controller;

If read operation is carried out in the indication of this content, in the gathering table that then should disperse at least the part field contents be transferred to this another storage virtualization controller by this local bus; And

After receiving this discrete data that this another storage virtualization controller passes, deposit in this internal memory according to corresponding this target start address of this discrete data again.

31. method as claimed in claim 30, wherein, the chipset of this central processing unit is to give this another storage virtualization controller with this whole discrete gathering table content delivery.

32. method as claimed in claim 30, wherein, after the central processing unit chipset that more comprises this another storage virtualization controller receives these discrete gathering table contents, indication according to this source origing address field (OAF) and this data length field content, read every discrete data in regular turn, and the step that returns these discrete datas.

33. method as claimed in claim 24, wherein, the Data Transport Protocol form of this discrete gathering table more comprises following field:

First break field, whether in order to set in finishing this table after the listed data transmission, the chipset of this central processing unit need produce a look-at-me and notify this central processing unit; And

Second break field in order to set this another storage virtualization controller after receiving these discrete datas and finishing corresponding operation, produces a look-at-me.

34. method as claimed in claim 33, wherein, step C more includes:

This central processing unit chipset sends this second break field content to this another storage virtualization controller; And

The chipset of this central processing unit determines whether producing the action of a look-at-me to central processing unit according to the indication of this first break field in finishing this discrete gathering table after the indicated data transmission.

35. method as claimed in claim 24, wherein, the Data Transport Protocol form that is somebody's turn to do discrete gathering table more comprises next discrete field of assembling table address, show stored memory address in order to deposit next discrete gathering, with the chipset of indicating this central processing unit after finishing the indicated data transmission of a discrete gathering table content, according to the discrete content of assembling the field of table address of this next one, read next discrete gathering table, and then interlock continues to handle.

36. method as claimed in claim 35, wherein, more include systems organization and be set with the numerical value that an expression does not have next discrete gathering table existence, when the discrete content of assembling the table address field of this next one is this numerical value, show that promptly not having other discrete gathering table of existence needs interlock to handle.

37. method as claimed in claim 36, wherein, this numerical value is to be 0.

38. method as claimed in claim 35 more may further comprise the steps between steps A and B:

Check and whether have the discrete gathering table that still has been untreated;

If have, then carry out an interlock program, use so that newly-established discrete gathering table and the aforementioned discrete gathering table that still has been untreated produce interlock; And

Otherwise, execution in step B.

39. method as claimed in claim 38, wherein:

Carry out more may further comprise the steps before the interlock program at this:

This central processing unit sends out one and suspends request to this local bus interface, suspends the relevant action that transfers data to this another storage virtualization controller in order to request; And

Receiving this time-out request when this local bus interface, is to carry out one to suspend mechanism, and after finishing this time-out mechanism, answer one suspends approval and gives this central processing unit; And

After carrying out the interlock program, this more may further comprise the steps:

This central processing unit notifies this local bus interface to remove halted state; And

Receiving this releasing notice when this local bus interface, is the action that recovery is suspended, and continues to carry out to handle and transmits data.

40. method as claimed in claim 39, wherein, this time-out mechanism is the discrete data of handling to be finished the reading of suspending thereafter the transmission back of each discrete data transmit action, and notes down and recover to continue processing when the breakpoint is suspended in order to cancellation.

41. method as claimed in claim 39, wherein, this time-out mechanism is that all discrete datas in the discrete data table that will handle are all finished the action that transmission back time-out enters next discrete gathering table.

42. method as claimed in claim 38, wherein, this interlock program comprises:

This central processing unit reads the next discrete address date of assembling in the table address field in the discrete gathering table that is stored in this register;

This central processing unit judges according to the address in this register that is read whether this local bus interface has the discrete gathering table that need continue and handle thereafter;

If judged result is the discrete gathering table that need not to continue and handle, then changing the field contents of depositing next discrete gathering table address in this register is the start address of depositing this newly-established first discrete gathering table; And

If judged result is to have the discrete gathering table that must continue and handle, then change one the field contents of depositing next discrete gathering table address of these discrete gathering tables that still have been untreated, change it into deposit this newly-established first discrete gathering table start address, and the next discrete table address field contents of assembling of depositing of last sets that to deposit the next gathering table address field contents that disperses consistent with one of the aforementioned discrete gathering table that still has been untreated before change not this in will this newly-established discrete gathering table.

43. method as claimed in claim 42, wherein, one of the discrete gathering table that this still has been untreated is meant when this local bus interface is suspended mechanism, is stored in the discrete gathering table in the register.

44. method as claimed in claim 42, wherein, last discrete gathering table that is meant in the discrete gathering table of these interlocks that still have been untreated of the discrete gathering table that this still has been untreated.

45. method as claimed in claim 42, wherein, comprise that more setting up a record has the table of all discrete gathering memory addresss that table is deposited, use and inquire about the discrete gathering of still being untreated that this desire change deposits the field contents of next discrete gathering table address for this central processing unit and show stored memory address.

46. method as claimed in claim 42, wherein, this central processing unit system reads the discrete table address field contents of assembling of the next one that is stored in the discrete gathering table in this address according to the discrete table address information of assembling of the next one in this register that is read to this address, repeat it in regular turn, up to reading till desire change of this central processing unit institute deposits the discrete gathering table that still has been untreated of the next discrete field contents of assembling table address.

47. method as claimed in claim 20 wherein, when this central processing unit need be transferred these discrete gathering tables of having set up, is to carry out may further comprise the steps:

Check the discrete gathering table that has been untreated at present; And

Judge whether its discrete gathering table that is intended to transfer belongs among these discrete gathering tables that still have been untreated, if then transfer, otherwise, do not transfer.

48. method as claimed in claim 47 wherein, is to utilize this central processing unit to read to be stored in the address date in the next discrete gathering table address field in the interior gathering table that should disperse of this register, to know the present progress of data transmission.

49. method as claimed in claim 47, wherein

Before transferring, more may further comprise the steps:

This central processing unit sends out one and suspends request to this local bus interface, suspends the relevant action that transfers data to this another storage virtualization controller in order to request; And

Receiving this time-out request when this local bus interface, is to carry out one to suspend mechanism, and after finishing this time-out mechanism, answer one suspends approval and gives this central processing unit; And

After transferring, more may further comprise the steps:

This central processing unit notifies this local bus interface to remove halted state; And

Receiving this releasing notice when this local bus interface, is the action that recovery is suspended, and continues to carry out to handle and transmits data.

50. transmit the method for data in the computer system between storage virtualization controller, include following steps:

The central processing unit of one storage virtualization controller is to send a data transfer request to a central processing unit chipset;

One first local bus interface in this central processing unit chipset is this request to be changeed pass to another storage virtualization controller; And

One second local bus interface of this another storage virtualization controller will be carried out corresponding processing after receiving this request;

Be to utilize a local bus to communicate between one first and one second storage virtualization controller in these storage virtualization controllers wherein, this local bus is to adopt the mode of cable or backboard to be connected in this two to deposit virtualization controller, this first with this second storage virtualization controller respectively include this local bus interface, in order to set up the local bus communication channel between this first and second storage virtualization controller, and one by this two local bus interface is gone to change its operator scheme, so that set up online between the local interface of this storage virtualization controller.

51. method as claimed in claim 50, wherein, these local bus interface are for one of following: perimeter component links bus interface, perimeter component links expansion bus interface and perimeter component links quick bus interface.

52. method as claimed in claim 50, wherein, the step that this central processing unit sends data transfer request comprises:

This central processing unit transmits this data transfer request to the central processing unit interface in this central processing unit chipset;

This central processing unit interface is placed on the main bus in inside in this central processing unit chipset with this data transfer request; And

Read this data transfer request by this first local bus interface.

53. method as claimed in claim 50, wherein, this data transfer request content is to include a discriminant information, this first local bus interface is differentiated and is read this data transfer request.

54. method as claimed in claim 53, wherein, more include: this storage virtualization controller is that the physical memory address of this another storage virtualization controller of definition is represented with a virtual memory address form, make with its own physical memory address own and can not repeat, and this discriminant information is to be a memory address, and this memory address is also represented with this virtual memory address form.

55. method as claimed in claim 54 wherein, is to adopt the mode of the physical memory address of directly continuing own to go to define this virtual memory address.

56. method as claimed in claim 54 comprises that more in this first and second local bus interface is to carry out the step that this memory address is converted to corresponding physical memory address.

57. method as claimed in claim 50, wherein, this data transfer request is to comprise access instruction, carries out one of following operation in order to indication: write and read.

58. method as claimed in claim 57, wherein, the corresponding processing that this another storage virtualization controller carried out is to comprise that this access instruction that this second local bus interface is separated in the read request indicates corresponding operation to carry out this.

59. as claim 50 or 57 described methods, wherein, this data transfer request is to comprise that one writes data, an and target start address, this target start address is to write the memory address that this writes data in order to indication, and this another storage virtualization controller to carry out this corresponding processing be to comprise: according to this target start address this is write in the data write memory.

60. method as claimed in claim 59, wherein, this data transfer request more comprises a data length, and a Data Source start address, and corresponding processing that this another storage virtualization controller carries out is meant according to this Data Source start address and this data length reading of data and these data are returned to this first local bus interface to this internal memory.

61. method as claimed in claim 60 more includes: receiving after the passback data of this another storage virtualization controller it is to send central processing unit to when this first local bus interface.

Images