CN117251115A - Channel management method, system, equipment and medium of disk array - Google Patents
Channel management method, system, equipment and medium of disk array Download PDFInfo
- Publication number
- CN117251115A CN117251115A CN202311511539.9A CN202311511539A CN117251115A CN 117251115 A CN117251115 A CN 117251115A CN 202311511539 A CN202311511539 A CN 202311511539A CN 117251115 A CN117251115 A CN 117251115A
- Authority
- CN
- China
- Prior art keywords
- disk
- packet
- engine
- channel
- disk array
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007726 management method Methods 0.000 title claims abstract description 19
- 238000012795 verification Methods 0.000 claims description 61
- 238000000034 method Methods 0.000 claims description 48
- 238000012545 processing Methods 0.000 claims description 12
- 238000003491 array Methods 0.000 claims description 9
- 238000013468 resource allocation Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 abstract description 4
- 238000011161 development Methods 0.000 abstract description 4
- 238000012827 research and development Methods 0.000 abstract description 3
- 238000004904 shortening Methods 0.000 abstract description 2
- 238000012384 transportation and delivery Methods 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000013500 data storage Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a channel management method, a system, equipment and a medium of a disk array, comprising the following steps: receiving a demand configuration relation of a disk issued by a host; extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured; determining an optimal packet of a plurality of channel packets within the engine to be configured; and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk. The flexible selection of the IO engines of the disk configuration is realized, channel groups can be dynamically managed, the optimal groups in all the IO engines are searched, the disks are bound with the allocable channels in the optimal groups, the channel load balancing is realized, and the requirements of the express delivery under different scenes can be supported. The user does not need to design a brand new ASIC chip for each new application, thereby reducing the research and development cost, shortening the development time and accelerating the product marketing speed.
Description
Technical Field
The present invention relates to the field of data storage, and in particular, to a method, system, device, and medium for channel management of a disk array.
Background
With the rapid increase of the traffic of modern data centers, more and more data is run and stored on a large server; at the same time, the requirements on the data storage speed and the security are also increasing. In this context, RAID techniques have evolved. With the increasingly strict requirements of performance, security and the like, under the high-performance and high-reliability scenarios, the software RAID card (Redundant Array of Independent Disks ) is limited by the CPU and memory resources of the operating system, and the performance requirements cannot be met gradually, so that the system is suitable for general users and small enterprises. In practical application scenarios, the hardware RAID card can alleviate some adverse factors caused by the software RAID card, however, when the end user selects the hardware RAID scheme, the cost is actually too high, and the configuration is complicated to be the biggest concern.
Under the prior art, many hardware resources of the hardware RAID card are actually deployed in place in advance at the beginning of hardware design, and can not be simply and quickly reconfigured according to the actual demands of users in the later stage. Therefore, the customized hardware RAID card has special advantages in a specific scene, but has poor configuration flexibility and highly binds the current requirements of users, so that the whole life cycle cost is very high and is not the optimal strategy. In addition, the customization strategies are not capable of allocating the hardware resources after hardware shaping, and cannot improve the utilization rate of the expensive hardware engine resources.
Therefore, a channel management method for a disk array is needed to solve the above-mentioned problems.
Disclosure of Invention
Accordingly, it is necessary to provide a channel management method for a disk array to solve the above-mentioned problems.
In a first aspect, the present application provides a method for channel management of a disk array, where the method includes:
receiving a demand configuration relation of a disk issued by a host;
extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured;
determining an optimal packet of a plurality of channel packets within the engine to be configured;
and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
In some embodiments, a method for grouping a plurality of channel groups in the engine to be configured includes:
extracting the size of a grouping layer selector in the requirement configuration relation;
and determining the number of channel groups and dividing corresponding IO channels for each channel group according to the size of the group layer selector and the total number of IO channels in the engine to be configured.
In some embodiments, the determining the number of channel packets and each channel packet dividing the corresponding IO channel according to the size of the packet layer selector and the total number of IO channels in the engine to be configured includes:
Dividing the total number of IO channels in the engine to be configured by the size of the grouping layer selector, and calculating to obtain the number of channel groups;
the number of IO lanes matching the size of the packet layer selector is divided for each lane packet.
In some embodiments, before extracting the engine configuration relationship in the requirement configuration relationship and determining one or more engines to be configured, the method further includes:
carrying out validity check on the issued demand configuration relation, and judging whether the demand configuration relation passes the validity check or not;
if the required configuration relation does not pass the validity check, feeding back configuration error information to the host for reconfiguration;
and if the demand configuration relation passes the validity check, extracting an engine configuration relation in the demand configuration relation, and determining one or more engines to be configured.
In some embodiments, the verifying validity of the issued requirement configuration relationship includes:
executing first verification to verify whether the disk type is a supportable type;
executing a second verification to verify whether the current channel resources are allocated enough;
and executing third verification to verify the validity of the engine to be configured.
In some embodiments, the determining whether the requirement configuration relationship passes a validity check includes:
if the first verification, the second verification and the third verification are all passed, judging that the required configuration relationship passes the validity verification;
and if any one of the first verification, the second verification and the third verification fails, judging that the required configuration relationship fails the validity verification.
In some embodiments, the determining an optimal packet of a plurality of channel packets within the engine to be configured comprises:
selecting any one of the plurality of channel packets as an optimal packet, and defining the remaining plurality of channel packets as pending packets;
selecting one undetermined packet, and comparing the optimal packet with the number of allocable channels in IO channels in the undetermined packet;
if the number of the allocable channels of the optimal packet is smaller than that of the undetermined packet, updating the undetermined packet into the optimal packet, otherwise, defining the undetermined packet as a disadvantaged packet until the comparison of the undetermined packet and the optimal packet is completed.
In some embodiments, the method further comprises:
And updating the value obtained by subtracting one from the number of the allocable channels in the optimal packet to the latest number of the allocable channels in the optimal packet so as to search the corresponding optimal packet when the disk is inserted next time.
In some embodiments, the method further comprises:
when a hardware disk connected with the disk array is pulled out, unbinding an IO channel bound with the hardware disk;
updating the unbinding IO channel into an allocable channel.
In some embodiments, before the receiving the demand configuration relationship of the disk issued by the host, the method further includes:
generating disk array resource information according to the purposes, the priorities and the load conditions of the disk arrays;
and uploading the disk array resource information to a host so that a user configures the number of IO engines for the disk array.
In some embodiments, the method includes building a storage environment:
the mainboard sends the required configuration information to a disk array card through an NVMe protocol, and one or more disk arrays are arranged on the disk array card;
the disk array card uploads the disk array resource information to a host through a PCIe protocol;
the disk array card transmits data to a hardware disk through an NVMe protocol;
And uploading data to the disk array card by the hardware disk through a PCIe protocol.
In some embodiments, after implementing the connection of the disk array to all of the hardware disks, the method further comprises:
receiving an IO task issued by a host, wherein the IO task comprises an issued target disk;
and simultaneously issuing the IO task to a corresponding hardware disk through one or more IO channels bound with the target disk according to the configuration relation between the target disk and the IO channels.
In a second aspect, the present application provides a channel management system for a disk array, the system comprising:
the data processing module is used for receiving the demand configuration relation of the magnetic disk issued by the host;
the engine configuration module is used for extracting an engine configuration relation in the requirement configuration relation and determining one or more engines to be configured;
a grouping dividing module, configured to determine an optimal grouping among a plurality of channel groupings in the engine to be configured;
and the resource allocation module is used for binding the disk with any allocable channel in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
In a third aspect, the present application provides an electronic device, including:
One or more processors;
and a memory associated with the one or more processors, the memory for storing program instructions that, when read for execution by the one or more processors, perform the following:
receiving a demand configuration relation of a disk issued by a host;
extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured;
determining an optimal packet of a plurality of channel packets within the engine to be configured;
and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
In a fourth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program that causes a computer to perform the operations of:
receiving a demand configuration relation of a disk issued by a host;
extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured;
determining an optimal packet of a plurality of channel packets within the engine to be configured;
and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
The beneficial effect that this application realized is:
the application provides a channel management method of a disk array, which comprises the following steps: receiving a demand configuration relation of a disk issued by a host; extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured; determining an optimal packet of a plurality of channel packets within the engine to be configured; and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk. The flexible selection of the IO engines of the disk configuration is realized, channel groups can be dynamically managed, the optimal groups in all the IO engines are searched, the disks are bound with the allocable channels in the optimal groups, the channel load balancing is realized, and the requirements of the express delivery under different scenes can be supported. The user does not need to design a brand new ASIC chip for each new application, thereby reducing the research and development cost, shortening the development time and accelerating the product marketing speed.
Furthermore, the application provides an interface to the host (board card configuration service) upwards, so that a host user can send a configuration relation according to the condition of hardware resources, and the flexibility of channel resource management is improved.
Furthermore, the method and the device realize quick searching of the optimal grouping and bind the newly added disk with the optimal grouping, thereby ensuring the maximum parallelism of hardware and maximally improving the hardware speed and the use efficiency.
Drawings
For a clearer description of the technical solutions in the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art, wherein:
FIG. 1 is a flowchart of a method for managing channels of a disk array according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a storage environment according to an embodiment of the present application;
FIG. 3 is a schematic diagram of packet partitioning according to an embodiment of the present application;
FIG. 4 is a flowchart of an optimal packet determination provided by an embodiment of the present application;
FIG. 5 is a schematic diagram of a channel management method of a disk array according to an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of a channel management system of a disk array according to an embodiment of the present disclosure;
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be understood that throughout the description of this application, unless the context clearly requires otherwise, the words "comprise," "comprising," and the like in the description and the claims are to be construed in an inclusive sense rather than an exclusive or exhaustive sense; that is, it is the meaning of "including but not limited to".
It should also be appreciated that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Furthermore, in the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.
It should be noted that the terms "S1", "S2", and the like are used for the purpose of describing steps only, and are not intended to be limited to the order or sequence of steps or to limit the present application, but are merely used for convenience in describing the method of the present application and are not to be construed as indicating the sequence of steps. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.
As described in the background art, the existing hardware RAID card has the problems of high cost and complex configuration; and because of being a specialized RAID card, therefore more suitable for the users that highly match with this RAID card; further, as long as the custom user needs have been slightly altered, the original hardware RAID card will perform poorly when in use. In addition, many hardware resources of the hardware RAID card are deployed in place in advance at the beginning of hardware design, and cannot be simply and quickly reconfigured according to the actual demands of users in the later period. In summary, the existing hardware RAID card has poor flexibility when configuring hardware resources, and highly binds the current demands of users, resulting in higher development cost. Therefore, the channel management method for the disk array dynamically realizes the redistribution of hardware resources, reduces the research and development cost, shortens the development time and accelerates the product marketing speed.
Example 1
The embodiment of the application provides a channel management method of a disk array, which is applied to a hardware disk array card, and the channel resource is allocated to a disk newly inserted into the disk array by using the method disclosed by the embodiment of the application, as shown in a flow chart shown in fig. 1, and specifically comprises the following contents:
s1, receiving a demand configuration relation of a disk issued by a host.
Specifically, the above-mentioned demand configuration relationship is a demand relationship of a disk needing to perform channel resource configuration on each CPU, including a disk array type, the number of disks in the disk array, the number of packet layer selectors, the IO engines configured for the disk array, and an engine configuration relationship, where the engine configuration relationship is a configuration relationship between the disk and the engine, and may be set by a user on a host, that is, the user may flexibly configure the IO engines corresponding to the newly inserted disk, and it may be understood that one disk may be configured with one IO engine, or may be configured with multiple engines, and at most, the number of IO engines configured by the disk array corresponding to the disk may not exceed the number of IO engines configured by the disk array.
Before performing step S1, the present application also needs to perform the following two pre-steps:
step a1, the embodiment of the present application needs to build a storage environment of an SDH (Software Defined Hardware ) architecture first. Specifically, as shown in fig. 2, the storage environment includes a host, a hardware RAID card (hereinafter simply referred to as a disk array card) and a hardware physical disk (i.e. a hardware disk), where the host issues data or instructions to the disk array card through an NVMe protocol, for example, when the disk array card configures the required configuration information, and one or more disk arrays are installed on the disk array card; the disk array card uploads data to a host, such as disk array resource information, via PCIe (non-volatile memory interface specification) protocol; the disk array card transmits data to the hardware disk through NVMe (Non Volatile Memory Express, nonvolatile memory interface Specification) protocol; and uploading data to the disk array card by the hardware disk through the PCIE protocol.
In step a2, the embodiment of the present application further needs to implement an engine configuration for a disk array, where the disk array is an array set of a plurality of disks. Specifically, based on the storage environment already built in the foregoing, the disk array card may upload the disk array resource information such as the usage, priority, and current load condition of one or more disk arrays installed on the disk array card to the host, so as to display the disk array resource information to the user, and further, the user may actively match a suitable IO engine for the disk array according to the uploaded disk array resource information and the actual requirement for the IO task, so as to obtain higher hardware computing power and IO parallel capability. The hardware in the traditional sense can not show the resources of the bottom hardware to the user, but the hardware driver completes the static contract configuration of the hardware interface, and shields the perception of the host user on the bottom hardware, so that the host user loses the ability of deeply understanding and flexibly configuring the hardware. Similarly, the hardware configuration of the traditional disk array card is also the same, hardware details such as a CPU layout mode, the number of IO engines, channel resource division and the like are not revealed to a user, and the disk array resource information and the like are upwards provided with an interface to a host (board card configuration service) through the mode, so that a host user can issue a configuration relation according to the condition of hardware resources, and flexible and dynamic allocation of the hardware resources is realized.
S2, extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured.
In some implementation scenarios, in order to ensure the success of configuring channel resources for a disk, the embodiment of the application also proposes that after the disk array card receives a demand configuration relationship issued by a host, validity check is further required to be performed on the issued demand configuration relationship, and whether the demand configuration relationship passes the validity check is judged; if the required configuration relation does not pass the validity check, feeding back configuration error information to the host for reconfiguration; and if the demand configuration relation passes the validity check, extracting an engine configuration relation in the demand configuration relation, and determining one or more engines to be configured corresponding to the disk.
Specifically, the disk array type of the disk array, the number of disks in the disk array and the IO engine specification of the disk array configuration in the requirement configuration relation are extracted first. Executing first verification to verify whether the disk type is the disk type supported by the hardware disk array card, wherein it can be understood that the disk type is consistent with the disk array type; when the disk type is the disk type supported by the disk array card, the first verification is passed; when the disk type is a disk type which is not supported by the disk array card, the first verification is failed. Executing second verification, namely verifying whether the current residual channel resources are enough to support allocation to the disk, and specifically estimating whether the residual resources can meet the allocation according to the number of the disks in the current disk array and the number of IO channels in an IO engine configured by the disk array; if the number of the disks is greater than or equal to the number of IO channels in the IO engines configured by the disk array, namely all the IO channels are occupied, the IO engines cannot be allocated to the disks at the moment, the configuration of the new disks cannot be met by the residual resources, and the second verification fails; otherwise, if the number of the disks is smaller than the number of the IO channels in the IO engine configured by the disk array, the remaining resources can be considered to meet the configuration of the new disk, and the second verification is passed. Executing third verification, and verifying the validity of the IO engine configured by the disk array, namely judging whether the specification of the IO engine is legal or not, and if the specification of the IO engine is legal, passing the third verification; when the specification of the IO engine is illegal, the third verification is failed; such as configuring the illegal engine, the third verification is not passed. If the first verification, the second verification and the third verification are all passed, judging that the requirement configuration relationship passes the validity verification; if any one of the first verification, the second verification and the third verification fails, judging that the demand configuration relationship fails the validity verification. When the required configuration relation does not pass the validity check, the configuration error information is fed back to the host so that the user can reconfigure the configuration, and the configuration error information can be uploaded to the host through a PCIe protocol; only after the demand configuration relationship passes the validity check, determining one or more engines to be configured, which are configured to the disk by the user, according to the engine configuration relationship in the demand configuration relationship.
After the legitimacy check of the issued demand configuration relation is passed, extracting an engine configuration relation in the demand configuration relation, and determining one or more engines to be configured corresponding to the disk of the IO channel to be configured, wherein the engine configuration relation is a relation between the engine to be configured and the disk selected by a user from the IO engines.
S3, IO channel resource allocation of the disk on each engine to be allocated is executed.
Specifically, the executing the IO channel resource configuration on the disk includes the following steps:
b1, planning and grouping the IO channels on each engine to be configured.
And extracting the size of the grouping layer selector in the demand configuration relation issued by the host. And determining the number of channel groups and dividing corresponding IO channels for each channel group according to the number of the group layer selectors and the total number of IO channels in the engine to be configured. Dividing the total number of IO channels in the engine to be configured by the size of the grouping layer selector, and calculating to obtain the number of channel groups; each channel group is divided into a number of IO channels matching the size of the hierarchical group selector. For example, there are X total IO channels in the engine to be configured, where the size of the packet layer selector is N, the number of channel packets divided at this time is X/N, and the number of IO channels divided in each packet is N.
Referring to the grouping schematic shown in fig. 3, an illustration is given: assume that Y engines are to be configured, and for ease of understanding, one of the engines 0 is taken as an example to describe. If n×4 IO channels exist in the engine 0, the size of the packet layer selector is 4, there are n channel packets in total, and the number of IO channels in each group is 4. IO lanes (channels) within each Group (Group) may be grouped as shown, with channels 0 through 3 belonging to Group 0, channels 4 through 7 belonging to Group 1, and so on. It will be appreciated that fig. 3 is only an embodiment of packet division, and is merely for illustration and not for limitation, and that, in addition to the manner of channel packet division in fig. 3, the number of IO channels matching the size of the packet layer selector may be randomly selected as a channel packet.
b2, searching the optimal grouping in the engine to be configured.
Referring to fig. 4, the determining step of the optimal packet includes determining the optimal packet according to a greedy algorithm:
step 1, selecting any one channel packet G0 in a plurality of channel packets as an optimal packet LOS, representing that the channel resources in the packet remain the most, and defining the remaining channel packets except G0 in all the channel packets as pending packets;
Step 2, selecting a group G1 to be determined, and comparing the optimal group LOS with the number of allocable channels in IO channels in the group G1 to be determined, wherein the number of allocable channels is not occupied IO channels;
step 3, if the number of the allocable channels of the optimal packet LOS is smaller than the number of the undetermined packets G1, updating the undetermined packets G1 into the optimal packet LOS, namely taking the G1 as a new LOS; conversely, the pending packet G1 is defined as a bad packet;
repeating the step 2 and the step 3 until the comparison of all the undetermined packets and the optimal packets is completed, namely, the comparison is completed until the last GX in the N undetermined packets is completed, and finally, the output optimal packets are the optimal channel packets with the most residual resources in each channel packet.
It can be understood that after the above steps are completed, the embodiment of the present application further proposes to update the value obtained by subtracting one from the remaining allocable channel number of the optimal packet at this time to the latest allocable channel number in the optimal packet, so as to find the corresponding optimal packet when the disk is inserted next time. The requirement of ensuring the load balance of the hardware processing capacity from the channel group is met, and the final optimal group is the channel group which is currently optimally inserted into the newly added disk binding, so that the maximum parallelism of the hardware is ensured, and the hardware rate and the use efficiency are improved to the greatest extent.
b3, binding the disk and the optimal packet.
Specifically, the disk can bind any one of the allocable channels in the optimal packet, and the connection between the disk array card and the hardware disk is further realized by binding IO channels with the disk.
When the disk array is newly inserted into the disk, the channel grouping with the most channel resource residues is found on each engine to be configured for the disk according to the engine configuration relation, and the hardware binding relation of the disk in the current engine is completed. In addition, when a hardware disk connected with the disk array is pulled out, the IO channel bound with the hardware disk is unbinding; updating the unbinding IO channel into an allocable channel.
It can be understood that after the disk array receives the IO task issued by the host after the reasonable configuration of the channel resources of the disk is completed, according to the target disk included in the IO task and according to the relationship between the disk and the bound IO channel determined in the foregoing, the IO task is issued to the corresponding hardware disk through one or more IO channels bound with the target disk.
In addition, the embodiment of the application also provides a method for reconfiguring resources on the disk inserted into the disk array, which specifically includes uploading the disk array resource information to the host for showing to the user, and uploading the state information of the IO channels bound by the disk in the disk array to the host, so that the user can determine which disks are already installed in the current disk array, and which hardware resources are bound by the disks. According to the state information of the IO channels bound by the disk in the disk array, the user can unbind one or more of the plurality of IO channels bound by the disk when the current resource is tense, and the unbinding is realized by reconfiguring the engine configuration relation of the IO engine corresponding to the disk. The method for allocating the IO channels to the disk according to the configuration relation of the engine is consistent with the above, and the description is omitted again.
The method converts the function of resource management provided in RAID card hardware equipment into a virtual instance which can be configured and managed on a software level; the flexible channel resource allocation is realized, the channel resources of the determined engine to be configured are dynamically grouped, the hardware channel resource load balance after grouping is ensured, and finally, the flexible and configurable hardware engine and the channel resource dynamic load balance are realized.
Example two
Corresponding to the first embodiment, the present application further provides a method for managing channels of a disk array, and the flowchart shown in fig. 5 specifically includes:
5100. receiving a demand configuration relation of a disk issued by a host;
preferably, before extracting the engine configuration relation in the requirement configuration relation and determining one or more engines to be configured, the method further includes:
5120. carrying out validity check on the issued demand configuration relation, and judging whether the demand configuration relation passes the validity check or not;
preferably, the verifying validity of the issued requirement configuration relationship includes:
5121. executing first verification to verify whether the disk type is a supportable type;
5122. Executing a second verification to verify whether the current channel resources are allocated enough;
5123. and executing third verification to verify the validity of the engine to be configured.
Preferably, the determining whether the requirement configuration relationship passes the validity check includes:
5124. if the first verification, the second verification and the third verification are all passed, judging that the required configuration relationship passes the validity verification;
5125. and if any one of the first verification, the second verification and the third verification fails, judging that the required configuration relationship fails the validity verification.
5130. If the required configuration relation does not pass the validity check, feeding back configuration error information to the host for reconfiguration;
5140. and if the demand configuration relation passes the validity check, extracting an engine configuration relation in the demand configuration relation, and determining one or more engines to be configured.
Preferably, before the receiving the demand configuration relation of the disk issued by the host, the method further includes:
5150. generating disk array resource information according to the purposes, the priorities and the load conditions of the disk arrays;
5160. and uploading the disk array resource information to a host so that a user configures the number of IO engines for the disk array.
Preferably, the method comprises:
5170. building a storage environment, comprising:
5171. the mainboard sends the required configuration information to a disk array card through an NVMe protocol, and one or more disk arrays are arranged on the disk array card;
5172. the disk array card uploads the disk array resource information to a host through a PCIe protocol;
5173. the disk array card transmits data to a hardware disk through an NVMe protocol;
5174. and uploading data to the disk array card by the hardware disk through a PCIe protocol.
5200. Extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured;
5300. determining an optimal packet of a plurality of channel packets within the engine to be configured;
preferably, the grouping method of the plurality of channel groups in the engine to be configured includes:
5310. extracting the size of a grouping layer selector in the requirement configuration relation;
5320. and determining the number of channel groups and dividing corresponding IO channels for each channel group according to the size of the group layer selector and the total number of IO channels in the engine to be configured.
Preferably, the determining the number of channel packets and dividing each channel packet into corresponding IO channels according to the number of the packet layer selectors and the total number of IO channels in the engine to be configured includes:
5321. Dividing the total number of IO channels in the engine to be configured by the size of the grouping layer selector, and calculating to obtain the number of channel groups;
5322. the number of IO lanes matching the size of the packet layer selector is divided for each lane packet.
Preferably, determining an optimal packet of a plurality of channel packets within the engine to be configured includes:
5330. selecting any one of the plurality of channel packets as an optimal packet, and defining the remaining plurality of channel packets as pending packets;
5340. selecting one undetermined packet, and comparing the optimal packet with the number of allocable channels in IO channels in the undetermined packet;
5350. if the number of the allocable channels of the optimal packet is smaller than that of the undetermined packet, updating the undetermined packet into the optimal packet, otherwise, defining the undetermined packet as a disadvantaged packet until the comparison of the undetermined packet and the optimal packet is completed.
Preferably, the method further comprises:
5360. and updating the value obtained by subtracting one from the number of the allocable channels in the optimal packet to the latest number of the allocable channels in the optimal packet so as to search the corresponding optimal packet when the disk is inserted next time.
5400. And binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
Preferably, the method further comprises:
5500. when a hardware disk connected with the disk array is pulled out, unbinding an IO channel bound with the hardware disk;
5600. updating the unbinding IO channel into an allocable channel.
Preferably, after the connection between the disk array and all the hardware disks is implemented, the method further includes:
5700. receiving an IO task issued by a host, wherein the IO task comprises an issued target disk;
5800. and simultaneously issuing the IO task to a corresponding hardware disk through one or more IO channels bound with the target disk according to the configuration relation between the target disk and the IO channels.
Example III
Corresponding to all the embodiments described above, as shown in fig. 6, an embodiment of the present application further provides a channel management system of a disk array, including:
the data processing module 610 is configured to receive a demand configuration relationship of a disk issued by a host;
an engine configuration module 620, configured to extract an engine configuration relationship in the requirement configuration relationship, and determine one or more engines to be configured;
A packet dividing module 630, configured to determine an optimal packet among a plurality of channel packets in the engine to be configured;
and the resource allocation module 640 is configured to bind the disk with any one of the allocable channels in the optimal packet in the engine to be configured so as to realize connection between the disk array card and the disk.
In some embodiments, the data processing module 610 is further configured to perform validity check on the issued requirement configuration relationship, and determine whether the requirement configuration relationship passes the validity check; if the required configuration relation does not pass the validity check, feeding back configuration error information to the host for reconfiguration; and if the demand configuration relation passes the validity check, extracting an engine configuration relation in the demand configuration relation, and determining one or more engines to be configured.
In some embodiments, the grouping module 630 is further configured to extract a size of a grouping layer selector in the requirement configuration relationship; and determining the number of channel groups and dividing corresponding IO channels for each channel group according to the size of the group layer selector and the total number of IO channels in the engine to be configured.
In some embodiments, the packet dividing module 630 is further configured to calculate, according to the total number of IO channels in the engine to be configured divided by the size of the packet layer selector, the number of channel packets; the number of IO lanes matching the size of the packet layer selector is divided for each lane packet.
In some embodiments, the data processing module 610 is further configured to perform a first verification that the disk type is a supportable type; executing a second verification to verify whether the current channel resources are allocated enough; and executing third verification to verify the validity of the engine to be configured.
In some embodiments, the data processing module 610 is further configured to determine that the requirement configuration relationship passes a validity check when the first verification, the second verification, and the third verification are all passed; and when any one of the first verification, the second verification and the third verification fails, judging that the required configuration relationship fails the validity verification.
In some embodiments, the packet dividing module 630 further selects any one of the plurality of channel packets as an optimal packet and defines the remaining plurality of channel packets as pending packets; selecting one undetermined packet, and comparing the optimal packet with the number of allocable channels in IO channels in the undetermined packet; if the number of the allocable channels of the optimal packet is smaller than that of the undetermined packet, updating the undetermined packet into the optimal packet, otherwise, defining the undetermined packet as a disadvantaged packet until the comparison of the undetermined packet and the optimal packet is completed.
In some embodiments, the packet dividing module 630 is further configured to update the value obtained by subtracting one from the number of allocable channels in the optimal packet to the latest number of allocable channels in the optimal packet, so as to find the corresponding optimal packet when the disk is inserted next time.
In some embodiments, the resource allocation module 640 is further configured to unbind the IO channels bound to the hardware disk when the hardware disk connected to the disk array is unplugged; updating the unbinding IO channel into an allocable channel.
In some embodiments, the data processing module 610 is further configured to generate disk array resource information according to usage, priority, and loading conditions of the disk array; and uploading the disk array resource information to a host so that a user configures the number of IO engines for the disk array.
In some embodiments, the system further includes an environment preparation module 650 (not illustrated) for building a storage environment: the mainboard sends the required configuration information to a disk array card through an NVMe protocol, and one or more disk arrays are arranged on the disk array card; the disk array card uploads the disk array resource information to a host through a PCIe protocol; the disk array card transmits data to a hardware disk through an NVMe protocol; and uploading data to the disk array card by the hardware disk through a PCIe protocol.
In some embodiments, the data processing module 610 is further configured to receive an IO task issued by a host, where the IO task includes an issued destination disk; and simultaneously issuing the IO task to a corresponding hardware disk through one or more IO channels bound with the target disk according to the configuration relation between the target disk and the IO channels.
Example IV
Corresponding to all the embodiments described above, an embodiment of the present application provides an electronic device, including: one or more processors; and a memory associated with the one or more processors, the memory for storing program instructions that, when read for execution by the one or more processors, perform the following:
receiving a demand configuration relation of a disk issued by a host;
extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured;
determining an optimal packet of a plurality of channel packets within the engine to be configured;
and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
Fig. 7 illustrates an architecture of an electronic device, which may include a processor 710, a video display adapter 711, a disk drive 712, an input/output interface 713, a network interface 714, and a memory 720, among others. The processor 710, the video display adapter 711, the disk drive 712, the input/output interface 713, the network interface 714, and the memory 720 may be communicatively connected via a bus 730.
The processor 710 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided in the present application.
The Memory 720 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. The memory 720 may store an operating system 721 for controlling the execution of the electronic device 700, and a Basic Input Output System (BIOS) 722 for controlling the low-level operation of the electronic device 700. In addition, a web browser 723, a data storage management system 724, an icon font processing system 725, and the like may also be stored. The icon font processing system 725 may be an application program that specifically implements the operations of the foregoing steps in the embodiments of the present application. In general, when implemented in software or firmware, the relevant program code is stored in memory 720 and executed by processor 710.
The input/output interface 713 is used to connect with an input/output module to enable information input and output. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
The network interface 714 is used to connect communication modules (not shown) to enable communication interactions of the device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
Bus 730 includes a path to transfer information between various components of the device (e.g., processor 710, video display adapter 711, disk drive 712, input/output interface 713, network interface 714, and memory 720).
In addition, the electronic device 700 may also obtain information of specific acquisition conditions from the virtual resource object acquisition condition information database, for performing condition judgment, and so on.
It should be noted that although the above devices illustrate only the processor 710, the video display adapter 711, the disk drive 712, the input/output interface 713, the network interface 714, the memory 720, the bus 730, etc., the device may include other components necessary to achieve normal execution in an implementation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the present application, and not all the components shown in the drawings.
From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a cloud server, or a network device, etc.) to perform the method described in the embodiments or some parts of the embodiments of the present application.
Example five
Corresponding to all the above embodiments, the present embodiments also provide a computer-readable storage medium, characterized in that it stores a computer program, which causes a computer to perform the following operations:
receiving a demand configuration relation of a disk issued by a host;
extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured;
determining an optimal packet of a plurality of channel packets within the engine to be configured;
and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a system or system embodiment, since it is substantially similar to a method embodiment, the description is relatively simple, with reference to the description of the method embodiment being made in part. The systems and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing description of the preferred embodiments of the present application is not intended to limit the invention to the particular embodiments of the present application, but to limit the scope of the invention to the particular embodiments of the present application.
Claims (15)
1. A channel management method for a disk array, applied to a disk array card, the method comprising:
receiving a demand configuration relation of a disk issued by a host;
extracting an engine configuration relation in the requirement configuration relation, and determining one or more engines to be configured;
determining an optimal packet of a plurality of channel packets within the engine to be configured;
and binding the disk with any one of the allocable channels in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
2. The method of claim 1, wherein the grouping method of the plurality of channel groupings within the engine to be configured comprises:
extracting the size of a grouping layer selector in the requirement configuration relation;
and determining the number of channel groups and dividing corresponding IO channels for each channel group according to the size of the group layer selector and the total number of IO channels in the engine to be configured.
3. The method of claim 2, wherein determining the number of lane packets and each lane packet dividing a corresponding IO lane according to the size of the packet layer selector and the total number of IO lanes within the engine to be configured comprises:
dividing the total number of IO channels in the engine to be configured by the size of the grouping layer selector, and calculating to obtain the number of channel groups;
the number of IO lanes matching the size of the packet layer selector are divided for each lane packet.
4. The method of claim 1, wherein the extracting engine configuration relationships in the demand configuration relationships, before determining one or more engines to be configured, further comprises:
carrying out validity check on the issued demand configuration relation, and judging whether the demand configuration relation passes the validity check or not;
if the required configuration relation does not pass the validity check, feeding back configuration error information to the host for reconfiguration;
and if the demand configuration relation passes the validity check, extracting an engine configuration relation in the demand configuration relation, and determining one or more engines to be configured.
5. The method of claim 4, wherein the verifying the validity of the issued demand configuration relationship comprises:
executing first verification to verify whether the disk type is a supportable type;
executing a second verification to verify whether the current channel resources are allocated enough;
and executing third verification to verify the validity of the engine to be configured.
6. The method of claim 5, wherein the determining whether the demand configuration relationship passes a validity check comprises:
if the first verification, the second verification and the third verification are all passed, judging that the required configuration relationship passes the validity verification;
and if any one of the first verification, the second verification and the third verification fails, judging that the required configuration relationship fails the validity verification.
7. The method of claim 1, wherein the determining an optimal packet of a plurality of channel packets within the engine to be configured comprises:
selecting any one of the plurality of channel packets as an optimal packet, and defining the remaining plurality of channel packets as pending packets;
selecting one undetermined packet, and comparing the optimal packet with the number of allocable channels in IO channels in the undetermined packet;
If the number of the allocable channels of the optimal packet is smaller than that of the undetermined packet, updating the undetermined packet into the optimal packet, otherwise, defining the undetermined packet as a disadvantaged packet until the comparison of the undetermined packet and the optimal packet is completed.
8. The method of claim 7, wherein the method further comprises:
and updating the value obtained by subtracting one from the number of the allocable channels in the optimal packet to the latest number of the allocable channels in the optimal packet so as to search the corresponding optimal packet when the disk is inserted next time.
9. The method of claim 7, wherein the method further comprises:
when a hardware disk connected with the disk array is pulled out, unbinding an IO channel bound with the hardware disk;
updating the unbinding IO channel into an allocable channel.
10. The method of claim 1, wherein prior to receiving the disk demand configuration relationship issued by the host, the method further comprises:
generating disk array resource information according to the purposes, the priorities and the load conditions of the disk arrays;
and uploading the disk array resource information to a host so that a user configures the number of IO engines for the disk array.
11. The method of claim 10, comprising building a storage environment:
the mainboard sends the required configuration information to a disk array card through an NVMe protocol, and one or more disk arrays are arranged on the disk array card;
the disk array card uploads the disk array resource information to a host through a PCIe protocol;
the disk array card transmits data to a hardware disk through an NVMe protocol;
and uploading data to the disk array card by the hardware disk through a PCIe protocol.
12. The method of claim 1, wherein after the connection of the disk array to all of the hardware disks is made, the method further comprises:
receiving an IO task issued by a host, wherein the IO task comprises an issued target disk;
and simultaneously issuing the IO task to a corresponding hardware disk through one or more IO channels bound with the target disk according to the configuration relation between the target disk and the IO channels.
13. A channel management system for a disk array, the system comprising:
the data processing module is used for receiving the demand configuration relation of the magnetic disk issued by the host;
the engine configuration module is used for extracting an engine configuration relation in the requirement configuration relation and determining one or more engines to be configured;
A grouping dividing module, configured to determine an optimal grouping among a plurality of channel groupings in the engine to be configured;
and the resource allocation module is used for binding the disk with any allocable channel in the optimal group in the engine to be configured so as to realize the connection between the disk array card and the disk.
14. An electronic device, the electronic device comprising:
one or more processors;
and a memory associated with the one or more processors, the memory for storing program instructions that, when read for execution by the one or more processors, perform the method of any of claims 1-12.
15. A computer readable storage medium, characterized in that it stores a computer program, which causes a computer to perform the method of any one of claims 1-12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311511539.9A CN117251115B (en) | 2023-11-14 | 2023-11-14 | Channel management method, system, equipment and medium of disk array |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311511539.9A CN117251115B (en) | 2023-11-14 | 2023-11-14 | Channel management method, system, equipment and medium of disk array |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117251115A true CN117251115A (en) | 2023-12-19 |
| CN117251115B CN117251115B (en) | 2024-02-09 |
Family
ID=89131617
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311511539.9A Active CN117251115B (en) | 2023-11-14 | 2023-11-14 | Channel management method, system, equipment and medium of disk array |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117251115B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6904497B1 (en) * | 2001-09-21 | 2005-06-07 | Adaptec, Inc. | Method and apparatus for extending storage functionality at the bios level |
| TW200535609A (en) * | 2004-03-12 | 2005-11-01 | Netcell Corp | Disk controller methods and apparatus with improved striping, redundancy operations and interfaces |
| US20190004705A1 (en) * | 2017-06-30 | 2019-01-03 | Lenovo (Beijing) Co., Ltd. | Method for configuring disk array of electronic device and related electronic device |
| CN113407180A (en) * | 2021-05-28 | 2021-09-17 | 济南浪潮数据技术有限公司 | Configuration page generation method, system, equipment and medium |
| CN115543223A (en) * | 2022-11-30 | 2022-12-30 | 苏州浪潮智能科技有限公司 | Method, medium, equipment and disk array card for flexibly forming disk array card |
-
2023
- 2023-11-14 CN CN202311511539.9A patent/CN117251115B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6904497B1 (en) * | 2001-09-21 | 2005-06-07 | Adaptec, Inc. | Method and apparatus for extending storage functionality at the bios level |
| TW200535609A (en) * | 2004-03-12 | 2005-11-01 | Netcell Corp | Disk controller methods and apparatus with improved striping, redundancy operations and interfaces |
| US20190004705A1 (en) * | 2017-06-30 | 2019-01-03 | Lenovo (Beijing) Co., Ltd. | Method for configuring disk array of electronic device and related electronic device |
| CN113407180A (en) * | 2021-05-28 | 2021-09-17 | 济南浪潮数据技术有限公司 | Configuration page generation method, system, equipment and medium |
| CN115543223A (en) * | 2022-11-30 | 2022-12-30 | 苏州浪潮智能科技有限公司 | Method, medium, equipment and disk array card for flexibly forming disk array card |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117251115B (en) | 2024-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8954487B2 (en) | Management server and method for providing cloud computing service | |
| CN104468803B (en) | A kind of virtual data center method for mapping resource and equipment | |
| CN104123186B (en) | Method for distributing business and device | |
| CN107040582A (en) | A kind of data processing method and device | |
| CN117480494A (en) | Coordinated container scheduling for improved resource allocation in virtual computing environments | |
| CN103164253A (en) | Virtual machine deployment system and virtual machine deployment method | |
| CN108874502B (en) | Resource management method, device and equipment of cloud computing cluster | |
| CN109032751B (en) | Virtual machine deployment method and OMM virtual machine | |
| CN112862098A (en) | Method and system for processing cluster training task | |
| CN114244717A (en) | Configuration method and device of virtual network card resources, computer equipment and medium | |
| CN109873714B (en) | Cloud computing node configuration updating method and terminal equipment | |
| CN113051250A (en) | Database cluster capacity expansion method and device, electronic equipment and storage medium | |
| CN104702691A (en) | Distributed load balancing method and device | |
| CN106790403B (en) | Method for realizing mobile cloud computing intermediate platform and method for realizing distribution | |
| JP5515889B2 (en) | Virtual machine system, automatic migration method and automatic migration program | |
| CN118051166A (en) | Storage processing method and related device | |
| CN117251115B (en) | Channel management method, system, equipment and medium of disk array | |
| CN112261125B (en) | Centralized unit cloud deployment method, device and system | |
| CN110780992B (en) | A cloud computing platform optimization deployment method, system, terminal and storage medium | |
| CN118034588A (en) | Data migration method, device, computer equipment and readable storage medium | |
| CN112925640B (en) | Cluster training node distribution method and electronic equipment | |
| CN114173396A (en) | Method and device for determining terminal networking time, electronic equipment and storage medium | |
| CN114253663A (en) | Virtual machine resource scheduling method and device | |
| CN117149099B (en) | Calculation and storage split server system and control method | |
| US20240160465A1 (en) | Orchestration model integration method in heterogeneous cluster environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |