CN107395415A - A kind of data processing method and system - Google Patents

Abstract

The present application provides a data processing method and system. The data processing system includes a cluster composed of multiple servers, a switch and a storage system, wherein the cluster adopts Microsoft cluster service technology, and the storage system is used as a local disk of each server in the cluster. The method includes: a first server sends a data processing request to the switch, and forwards it to the storage system via the switch; the first server is any server in the cluster; the storage system responds to the data processing the request, sending the response result to the switch, and forwarding to the first server via the switch. This application proposes a technical solution combining the storage system and MSCS technology. For the entire cluster system, it can not only achieve high data availability, but also reduce the investment and maintenance costs of enterprises.

Description

A data processing method and system

technical field

The present application relates to the field of computer technology, in particular to a data processing method and system.

Background technique

In the process of reading or writing data, if the server fails or pauses, the data will not be read or stored normally. For this reason, the industry has launched cluster software, which can avoid single node failure or downtime through such software, and guarantee Data can still be accessed or stored.

However, the deployment and maintenance of this cluster environment, for some small and medium-sized enterprises, not only increases the investment cost, but also increases the later maintenance cost.

Contents of the invention

In view of this, the present application provides a data processing method and system to reduce the cost of deploying and maintaining a cluster environment. The technical scheme is as follows:

Based on one aspect of the present application, the present application provides a data processing method, which is applied to a data processing system, and the data processing system includes a cluster composed of multiple servers, a switch and a storage system, wherein the cluster adopts Microsoft cluster service technology , the storage system is used as a local disk of each server in the cluster; the method includes:

The first server sends a data processing request to the switch, and forwards it to the storage system via the switch; the first server is any server in the cluster;

The storage system responds to the data processing request, sends a response result to the switch, and forwards the response result to the first server via the switch.

Preferably, when the first server fails or goes down, the method further includes:

Start a switching mechanism based on the Microsoft cluster service technology, switch all resources on the first server to a second server, the second server is different from the first server in the cluster, and is the same as the first server Any server for server redundancy;

The second server sends the data processing request to the switch, and forwards it to the storage system via the switch;

The storage system responds to the data processing request, sends a response result to the switch, and forwards the response result to the second server via the switch.

Preferably, the data processing request includes a data read request; the storage system responds to the data processing request, and sending a response result to the switch includes:

The storage system searches for target data according to the data read request, and sends the target data to the switch.

Preferably, the data processing request includes a data write request; the storage system responds to the data processing request, and sending a response result to the switch includes:

The storage system writes the data to be written into a designated storage area according to the data write request, and sends a writing completion result to the switch.

Based on another aspect of the present application, the present application also provides a data processing system, the data processing system includes a cluster composed of a plurality of servers, a switch and a storage system, the cluster adopts Microsoft cluster service technology, and the storage system as a local disk for each server in the cluster;

The first server in the cluster is used to send a data processing request to the switch; the first server is any server in the cluster;

The switch is configured to forward the data processing request to the storage system;

The storage system is configured to respond to the data processing request, and send a response result to the switch;

The switch is further configured to forward the response result to the first server.

Preferably, when the first server fails or goes down, the data processing system is configured to start a switching mechanism based on the Microsoft cluster service technology to switch all resources on the first server to the second server , the second server is any server in the cluster that is different from the first server and redundant with the first server;

The second server is configured to send the data processing request to the switch;

The switch is configured to forward the data processing request to the storage system;

The storage system is configured to respond to the data processing request and send a response result to the switch;

The switch is also used to forward the response result to the second server.

Preferably, the data processing request includes a data read request; the storage system is configured to respond to the data processing request, and sending the response result to the switch includes:

The storage system searches for target data according to the data read request, and sends the target data to the switch.

Preferably, the data processing request includes a data write request; the storage system is configured to respond to the data processing request, and sending the response result to the switch includes:

The storage system writes the data to be written into a designated storage area according to the data write request, and sends a writing completion result to the switch.

In the data processing method and system provided by the present application, the data processing system includes a cluster composed of multiple servers, a switch, and a storage system, wherein the cluster adopts MSCS (Microsoft Cluster Service, Microsoft Cluster Service) technology, and the storage system is used as each cluster in the cluster. The server's local disk. This application proposes a technical solution combining the storage system and MSCS technology. For the entire cluster system, it can not only achieve high data availability, but also reduce the investment and maintenance costs of enterprises.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present application, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.

Fig. 1 is a schematic structural diagram of a data processing system provided by the present application;

Fig. 2 is a flow chart of a data processing method provided by the present application;

Fig. 3 is a sequence diagram of a data reading method provided by the present application;

FIG. 4 is a timing diagram of a data writing method provided by the present application;

FIG. 5 is a flow chart of another data processing method provided by the present application.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

As shown in FIG. 1 , it shows a schematic structural diagram of the data processing system provided by the present application. The data processing system provided in the present application may also be called a cluster system or a cluster environment, and specifically includes: a cluster 100 composed of multiple servers, a switch 200 and a storage system 300 . The cluster 100 adopts Microsoft Cluster Service (MSCS, Microsoft Cluster Service) technology, and the storage system 300 respectively establishes a mapping relationship with each server in the cluster 100 as a local disk shared by each server in the cluster 100 .

This application innovatively proposes to combine the MSCS technology with the storage system 300 so that all servers in the cluster 100 share the storage system 300 . When this application is actually applied, there are application programs installed on multiple servers in MSCS, but at any moment, the application program is only online on one node. When an application fails or the server is shut down, the application is transferred to another available node and restarted (this process is called "failover"), so as to ensure the normal reading and writing of data to the greatest extent, and improve data quality. high availability. And for the entire cluster system, there is no need to consider and design the storage capacity of the local disks of each server in the cluster, which reduces the difficulty for developers to write and identify applications in the cluster system, thereby reducing the design cost of the cluster system, that is, Reduce the input and maintenance costs of the enterprise.

Based on the data processing system provided by this application, this application also provides a data processing method, as shown in Figure 2, the method includes:

Step 201, the first server sends a data processing request to a switch, and forwards it to a storage system via the switch.

Wherein, the first server is any server in the cluster 100 .

Step 202, the storage system responds to the data processing request, sends the response result to the switch, and forwards the response result to the first server via the switch.

In the following, the applicant will describe the data processing requests as data reading requests and data writing requests respectively.

As shown in Figure 3, it shows a timing diagram of a data reading method provided by the present application, including:

Step 301, the first server sends a data read request to the switch.

The data read request includes information such as an address and an identifier of the data to be read.

Step 302, the switch forwards the data read request to the storage system.

Step 303, after receiving the data read request, the storage system searches for target data according to the data read request.

Step 304, the storage system sends the found target data to the switch.

After receiving the data read request, the storage system searches for target data according to information such as the address and identifier of the data to be read included in the data read request, and then sends the found target data to the switch.

Step 305, the switch forwards the target data to the first server.

As shown in Figure 4, it shows a timing diagram of a data writing method provided by the present application, including:

Step 401, the first server sends a data writing request to the switch.

The data writing request includes the data to be written, and may also include information such as a storage address of the data to be written.

Step 402, the switch forwards the data write request to the storage system.

Step 403, after receiving the data writing request, the storage system writes the data to be written into the designated storage area.

After the storage system receives the data write request, if the data write request includes the storage address of the data to be written, it writes the data to be written into the storage area corresponding to the storage address; if the data write request does not include the storage address of the data to be written If the storage address of the input data is specified, the data to be written can be written into the default storage area in the storage system.

Step 404, the storage system sends the writing completion result to the switch.

After the storage system finishes storing the data to be written, it sends the writing completion result to the switch.

Step 405, the switch forwards the writing completion result to the first server.

Step 406, the first server receives the write completion result, and determines that the data to be written has been successfully written.

The above embodiments are based on the fact that each server in the cluster 100 is in a normal working state. When a server is down, the data processing method provided by the present application further includes, as shown in Figure 5:

Step 501: Start a switching mechanism based on MSCS technology, and switch all resources on the first server to a second server. The second server is any server in the cluster that is different from the first server and redundant with the first server.

After starting the switching mechanism based on the MSCS technology, the cluster system starts a failover process to switch all resources on the first server, including application programs, to the second server.

It should be noted that the existing technology of the failover method based on MSCS technology is very mature, for details, please refer to http://baike.baidu.com/link? url=5egqoiGa-S1sdnW22WR9uK5p6Wq9VKjSxUFQWstxQB-hWUnFKjaQEzozazLyg5Xldj8Wyn_2LiuKUAxYA8ELB_, the applicant will not repeat it here.

Step 502, the second server sends the data processing request to the switch, and forwards the data processing request to the storage system via the switch.

Step 503, the storage system responds to the data processing request, sends the response result to the switch, and forwards the response result to the second server via the switch.

Different from the previous embodiments, in this embodiment, when the first server fails or goes down and the first server cannot communicate, the data processing system will automatically trigger the switching mechanism to switch all resources on the first server to The second server, the second server replaces the first server to complete the related processing of data. Wherein, the subsequent implementation process of the second server sending the data read request or data write request to the switch is similar to the subsequent implementation process of the first server sending the data read request or data write request to the switch. Taking the above-mentioned Fig. 3 and Fig. 4 as an example, this embodiment only needs to replace the first server in Fig. 3 and Fig. 4 with the second server.

Applying the data processing method and system provided by the present application, the data processing system includes a cluster composed of multiple servers, a switch and a storage system, wherein the cluster adopts MSCS technology, and the storage system serves as a local disk of each server in the cluster. This application proposes a technical solution combining the storage system and MSCS technology. For the entire cluster system, it can not only achieve high data availability, but also reduce the investment and maintenance costs of enterprises.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

A data processing method and system provided by this application has been introduced in detail above. In this paper, specific examples are used to illustrate the principle and implementation of this application. The description of the above embodiment is only used to help understand the method of this application. and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation and application scope. limits.

Claims (8)

1. A data processing method, applied to a data processing system, characterized in that, the data processing system includes a cluster composed of a plurality of servers, a switch and a storage system, wherein the cluster adopts Microsoft cluster service technology, and the storage The system serves as a local disk of each server in the cluster; the method includes: The first server sends a data processing request to the switch, and forwards it to the storage system via the switch; the first server is any server in the cluster; The storage system responds to the data processing request, sends a response result to the switch, and forwards the response result to the first server via the switch. 2. The method according to claim 1, wherein when the first server fails or goes down, the method further comprises: Start a switching mechanism based on the Microsoft cluster service technology, switch all resources on the first server to a second server, the second server is different from the first server in the cluster, and is the same as the first server Any server for server redundancy; The second server sends the data processing request to the switch, and forwards it to the storage system via the switch; The storage system responds to the data processing request, sends a response result to the switch, and forwards the response result to the second server via the switch. 3. The method according to claim 1 or 2, wherein the data processing request includes a data read request; and the storage system responds to the data processing request, and sending a response result to the switch includes: The storage system searches for target data according to the data read request, and sends the target data to the switch. 4. The method according to claim 1 or 2, wherein the data processing request includes a data write request; the storage system responds to the data processing request, and sending a response result to the switch includes: The storage system writes the data to be written into a designated storage area according to the data write request, and sends a writing completion result to the switch. 5. A data processing system, characterized in that, the data processing system includes a cluster composed of a plurality of servers, a switch and a storage system, the cluster adopts Microsoft cluster service technology, and the storage system is used as each server in the cluster the server's local disk; The first server in the cluster is used to send a data processing request to the switch; the first server is any server in the cluster; The switch is configured to forward the data processing request to the storage system; The storage system is configured to respond to the data processing request, and send a response result to the switch; The switch is further configured to forward the response result to the first server. 6. The system according to claim 5, wherein when the first server breaks down or goes down, the data processing system is configured to start a switching mechanism based on the Microsoft cluster service technology, and the All resources on the first server are switched to a second server, and the second server is any server in the cluster that is different from the first server and redundant with the first server; The second server is configured to send the data processing request to the switch; The switch is configured to forward the data processing request to the storage system; The storage system is configured to respond to the data processing request and send a response result to the switch; The switch is also used to forward the response result to the second server. 7. The system according to claim 5 or 6, wherein the data processing request includes a data read request; the storage system is configured to respond to the data processing request, and send a response result to the switch including : The storage system searches for target data according to the data read request, and sends the target data to the switch. 8. The system according to claim 5 or 6, wherein the data processing request includes a data write request; the storage system is configured to respond to the data processing request, and send a response result to the switch including : The storage system writes the data to be written into a designated storage area according to the data write request, and sends a writing completion result to the switch.