GB2335516A - Failure recovery in distributed transaction avoids heuristic damage - Google Patents

Abstract

Distributed transactions are processed in a client/server computer system using the two phase commit protocol, coordinated by server process 22. Upon completion of the first phase, the resource objects 231, 241, 251 etc. involved in the transaction are placed in a prepared state; server process 23 performs its own two-phase commit protocol with respect to resource objects 261-263 of server process 26. Should a failure occur during this prepared state, such that server process 22 has "gone down", some of the resource objects, e.g. 261-263 may make an heuristic decision to commit or rollback to avoid prolonged locking of the resources. An heuristic direction is agreed in advance amongst the transactionally involved servers having an ability to take heuristic decisions, thus ensuring that all the resource objects follow the same direction; the recovered server process 22 also follows this direction when issuing the phase two commands of the two phase commit protocol (fig. 4). Thus heuristic damage - which would occur if some resources commit while others rollback - is avoided.

Description

UK9-98-022 1 2335516 AN APPARATUS, METHOD AND COMPUTER PROGRAM PRODUCT FOR

CLIEW/SERVER COMPUTING WITH TRANSACTION RECOVERY IN PREPARED STATE AVOIDING HEURISTIC DAMAGE

Field of the Invention

The invention relates to the field of client/server (also known as "distributed") computing, where one computing device P,the client,') requests another computing device (,,the server") to perform part of the client's work. The client and server can also be both located on the same physical computing device.

Background of the Invention

Client/server computing has become more and more important over the past few years in the information technology world. This type of distributed computing allows one machine to delegate some of its work to another machine that might be, for example, better suited to perform that work. For example, the server could be a high-powered computer running a database program managing the storage of a vast amount of data, while the client is simply a desktop personal computer (PC) which requests information from the database to use in one of its local programs.

The benefits of client/server computing have been even further enhanced by the use of a well-known computer programming technology called objectoriented programming (OOP), which allows the client and server to be located on different (heterogeneous) "platforms". A platform is a combination of the specific hardware/software/operating system/communication protocol which a machine uses to do its work. COP allows the client application program and server application program to operate on their own platforms without worrying how the client application's work requests will be communicated and accepted by the server application. Likewise, the server application does not have to worry about how the OOP system will receive, translate and send the server application's processing results back to the requesting client application.

Details of how OOP techniques have been integrated with heterogeneous client/server systems are explained in US Patent No. 5,440,744 and European Patent Published Application No. EP 0 677,943 A2. These latter two publications are hereby incorporated by reference.

UK9-98-022 2 However, an example of the basic architecture will be given below for contextual understanding of the invention's environment.

As shown in Fig. 1, the client computer 10 (which could, for example, be a personal computer having the IBM OS/2 operating system installed thereon) has an application program 40 running on its operating system (11IBM11 and "OS/211 are trademarks of the International Business machines corporation). The application program 40 will periodically require work to be performed on the server computer 20 and/or data to be returned from the server 20 for subsequent use by the application program 40. The server computer 20 can be, for example, a high-powered mainframe computer running on IBM's MVS operating system (11MVS11 is also a trademark of the IBM corp.). For the purposes of the present invention it is irrelevant whether the requests for communications services to be carried out by the server are instigated by user interaction with the first application program 40, or whether the application program 40 operates independently of user interaction and makes the requests automatically during the running of the program.

when the client computer 10 wishes to make a request for the server computer 20's services, the first application program 40 informs the first logic means 50 of the service required. It may for example do this by sending the first logic means the name of a remote procedure along with a list of input and output parameters. The first logic means 50 then handles the task of establishing the necessary communications with the second computer 20 with reference to definitions of the available communications services stored in the storage device 60. All the possible services are defined as a cohesive framework of object classes 70, these classes being derived from a single object class. Defining the services in this way gives rise to a great number of advantages in terms of performance and reusability.

To establish the necessary communication with the server 20, the first logic means 50 determines which object class in the framework needs to be used, and then creates an instance of that object at the server, a message being sent to that object so as to cause that object to invoke one of its methods. This gives rise to the establishment of the connection with the server computer 20 via the connection means 80, and the subsequent sending of a request to the second logic means 90.

UK9-98-022 3 The second logic means 90 then passes the request on to the second application program 100 (hereafter called the service application) running on the server computer 20 so that the service application 100 can perform the specific task required by that request, such as running a data retrieval procedure. once this task has been completed the service application may need to send results back to the first computer 10. The server application 100 interacts with the second logic means 90 during the performance of the requested tasks and when results are to be sent back to the first computer 10. The second logic means 90 establishes instances of objects, and invokes appropriate methods of those objects, as and when required by the server application 100, the object instances being created from the cohesive framework of object classes stored in the storage device 110.

Using the above technique, the client application program 40 is not exposed to the communications architecture. Further the service application 100 is invoked through the standard mechanism for its environment; it does not know that it is being invoked remotely.

The object management Group (OMG) is an international consortium of organizations involved in various aspects of client/server computing on heterogeneous platforms with distributed objects as is shownn in Fig. 1. The OMG has set forth published standards by which client computers (e.g. 10) communicate (in OOP form) with server machines (e.g. 20). As part of these standards, an Object Request Broker (called CORBA-the Common object Request Broker Architecture) has been defined, which provides the objectoriented bridge between the client and the server machines. The ORB decouples the client and server applications from the object oriented implementation details, performing at least part of the work of the first and second logic means 50 and 90 as well as the connection means 80.

As part of the CORBA software structure, the OMG has set forth standards related to "transactions,, and these standards are known as the OTS or Object Transaction Service. See, e.g., CORBA Object Transaction Service Specification 1.0, OMG Document 94.8.4. Computer implemented transaction processing systems are used for critical business tasks in a number of industries. A transaction defines a single unit of work that must either be fully completed or fully purged without action. For example, in the case of a bank automated teller machine from which a customer seeks to withdraw money, the actions of issuing the money, reducing the balance of money on hand in the machine and reducing the

UK9-98-022 4 customer's bank balance must all occur or none of them must occur. Failure of one of the subordinate actions would lead to inconsistency between the records and the actual occurrences.

Distributed transaction processing involves a transaction that affects resources at more than one physical or logical location. In the above example, a transaction affects resources managed at the local automated teller device as well as bank balances managed by a bank's main computer. Such transactions involve one particular client computer (e.g, 10) communicating with one particular server computer (e.g., 20) over a series of client requests which are processed by the server. The OMWs OTS is responsible for coordinating these distributed transactions.

usually, an application running on a client process begins a transaction which may involve calling a plurality of different servers, each of which will initiate a server process to make changes to its local database according to the instructions contained in the transaction. The transaction finishes by either committing the transaction (and thus all servers finalize the changes to their local databases) or aborting the transaction (and thus all servers llrollbackll or ignore the changes to their local databases). To communicate with the servers during the transaction (e.g., instructing them to either commit or abort their part in the transaction) one of the processes involved must maintain state data for the transaction. This usually involves the process to set up a series of transaction state objects, one of which is a Coordinator object which coordinates the transaction with respect to the various server processes.

The basic software architecture involved in providing an implementation of the OTS is shown in Fig. 2. A client process 21 which wants to begin a transaction (e.g., to withdraw money from a bank account) locates a process which is capable of creating and holding the transaction objects that will maintain the state of the transaction. As the modern tendency is to create clients that are ',thin,, (and thus have only the minimum functionality), the client process 21 will usually not be able to maintain the transaction objects locally and must look for a server process for this purpose.

The OTS (or another service, such as the CORBA Lifecycle service) selects server A process 22 on which to create the transaction state objects 221 (which include the Coordinator object, Control object and UK9-98-022 Terminator object). Upon locating the server A process 22, client process 21 sends (arrow with encircled number 1) a message to server A process 22 to instruct server A process 22 to create the transaction state objects 221. The Control object (known in the OTS as CosTransactions::Control) provides access to the other two transaction state objects. The Terminator object (known in the OTS as CosTransact ions::Terminator) is used to end the transaction. The Coordinator object (known in the OTS as CosTrans actions::Coordinator) maintains a list, in local storage 222, of resource objects (known in the OTS as CosTransactions::Resource) that have made updates to their respective data during the transaction. This list is required so that the Coordinator object can consistently call the resource objects at the end of the transaction to command them to commit their transactional changes (make their local data changes final) or to rollback such changes (bring the local data back to the state it was in before the transaction started). A rollback would be necessary, for example, where the transaction could not finish because one of the resources was not working properly.

Server A process 22 then creates the transaction state objects 221 and sends a reply (arrow with encircled number 2) containing the transaction context to client 21. Client 21 then sends, for example, a debit bank account command (arrow with encircled number 3) to server B process 23 (the process containing the resource, for example, bank account, object 231 which the client process 21 wishes to withdraw money from). This latter command carries with it the transaction context supplied to the client 21 by the server A process 22. In this way, the resource object 231 in process 23 can register itself (arrow with encircled number 4) with the transaction objects 221 in process 22 so that the resource object 231 can be commanded (arrow with encircled number 5) to commit or rollback by the transaction state objects 221 at the end of the transaction.

in the above operation, when the transaction state objects 221 are created, they must log information about themselves and the transaction they represent in local storage 222, so that the transaction will be recoverable in case of a server failure which temporarily prevents the server A process 22 from continuing with the transaction.

As part of the transaction, the client process 21 then makes similar calls to server C process 24 (to access the resource object 241) UK9-98-022 6 and server D process 24 (to access the resource object 251). Server B process 23, in carrying out its part of the transaction, may need to call another server process, such as server E process 26, to access the resource objects 261, 262 and 263 located in process 26.

Since the number of server processes and resources involved in Fig. 2 is becoming large, the need for careful synchronization of all of the database changes involved becomes readily apparent. The usual way to go about achieving this synchronization is to carry out a two-phase commit process. The transaction objects 221 first command (phase 1) each of their directly registered resources (231, 241 and 251 in the Fig. 2 example) to prepare to commit their database changes. Each of these resources then responds to the transaction objects 221 to indicate that it has prepared to commit its changes, and the resources will not allow any more changes to be made to the databases. The resources are then said to be sitting "in doubt" (also known as in a "prepared" state) waiting for the transaction objects 221 to give a synchronized final command (phase 2) to commit all database changes made during the transaction. Server B process 23, which has called another server process 26, would carry out its own two-phase commit protocol with respect to the resource objects 261, 262, and 263, as part of its participation in the main two-phase commit protocol discussed above.

Since resource objects that are "in doubt" are holding locks to their respective database resources, no other transaction can access such resources while the two-phase commit protocol is still pending in the prepared state. Should the server A process 22 become unavailable at this stage, for example, if the server running this process becomes faulty or totally unusable (e.g., a thunderstorm has caused a power failure in the area where the server is located), the resources will be made to wait "in doubt" until the server A process 22 has again been made available. As remaining "in doubt" for an extended period of time can often work a severe hardship in many servers (due to the fact that no other transaction can have access to the server resources), a systems administrator of a server holding resource objects is often given the authority to make a guess (called a heuristic decision) as to how the transaction will turn out, should the server be left "in doubt" for a long period of time. This way, the resource can release its locks before the second phase of the two-phase commit has completed so that another transaction can have access to the resource's associated data. If the guess turns out to be right, then everything is fine, however, if UK9-98-022 7 the guess turns out to be wrong, then a condition known as heuristic damage has occurred.

For example, assume that resource objects 261, 262 and 263 have taken heuristic decisions to rollback a transaction while the transaction is "in doubt" because the server A process 22 has "gone downn (meaning that it has become unavailable, due to, for example, a local power loss due to a thunderstorm). When the server A process 22 becomes available again and is restarted, it will check its local datastore 222 and determine that the three server processes 23, 24 and 25 are directly registered in the transaction that was pending before the server A process went down. The server A process 22 also determines from its local store that, just before the power loss, the transaction was in doubt (in the prepared state) as all directly registered resource objects 231, 241 and 251 had indicated previously that they are prepared to commit their changes. AS the server A process 22 does not yet know that the resources 261, 262 and 263 of server E process 26 have taken heuristic decisions to rollback, the normal course of action would then be for server A process 22 to move on to the second phase and command its directly registered resources to commit their resource changes. Server C process 24 and server D process 25 respond by committing their changes, releasing their locks and informing server A process 22 that they have committed their changes as commanded, thus completing their part of the second phase. However, server B process 23 responds that heuristic rollback has occurred.

In the above example, at the end of the second phase, resource objects 241 and 251 have committed their changes but resource objects 261, 262 and 263 have rolled back their changes. Thus, heuristic damage has occurred. when heuristic damage has occurred, a transaction has completed improperly and this could have disastrous results, such as wrongly debiting or crediting a customer's bank account balance.

Heretofore, the present state of the art in this technical area of distributed processing of transactions in heterogeneous computing environments has not provided a way to prevent such heuristic damage from occurring when the server containing the transaction objects recovers from a failure which occurred during a time when a transaction was in the prepared state. Thus, the present state of the art is left with an environment in which heuristic damage can easily occur, resulting in disastrous results.

UK9-98-022 Summary of the Invention

8 According to a first aspect, the present invention provides a server for use in a heterogeneous client/server computing system which coordinates the processing of distributed transactions in the client/server computing system, which has a means for coordinating a first phase of a two phase commit protocol used in the processing of a transaction amongst a plurality of transactionally involved resources in order to place the plurality of resources in a prepared state at the completion of the first phase; a means for recovering from a failure which has occurred subsequent to the end of the first phase of the two phase commit protocol; and a means for, upon recovering from the failure, coordinating a second phase of the two phase commit protocol by following an heuristic direction, the heuristic direction having been agreed upon in advance by each transactionally involved server that has an ability to take an heuristic decision. This prevents heuristic damage from occurring.

Preferably, the following of the heuristic direction involves sending a commit command to each transactionally-involved resource if the heuristic direction is commit and sending a rollback command to each transactional ly- involved resource if the heuristic direction is rollback.

According to a second aspect, the invention provides a method of carrying out the functionality of the server described above in the first aspect.

According to a third aspect, the invention provides a computer program product stored on a computer readable storage medium for, when run on a computer, carrying out the functionality of the first aspect.

According to a fourth aspect, the invention provides a server for use in a heterogeneous client/server computing system which carries out transactions, the server having: a resource; a means for updating values associated with the resource; a means for taking an heuristic decision with respect to the resource when the resource has been placed in a prepared state during the processing of a distributed transaction; characterized in that the heuristic decision is always taken in a direction which has been agreed amongst other servers in the heterogeneous client/server computing system.

UK9-98-022 9 According to a fifth aspect, the invention provides a method of operating a server for use in a heterogeneous client/server computing system which carries out transactions, the method having steps of: registering a local resource with a transaction; updating values associated with the resource as part of the transaction; receiving a command to enter into a prepared state as part of a two phase commit protocol of the transaction; taking an heuristic decision with respect to the resource when the resource has been placed in a prepared state during the processing of a distributed transaction; characterized in that the heuristic decision is always taken in a direction which has been agreed amongst other servers in the heterogeneous client/server computing system.

Thus, with the present invention, heuristic damage is prevented in a consistent and predictable manner, during situations when a server containing the transaction objects recovers from a failure which occurred during a time when a transaction was in the prepared state. In these conditions, it is now guaranteed that all resource objects will follow the same heuristic direction and the recovered server process will also follow this same heuristic direction when issuing the phase two commands of the two phase protocol.

Brief Description of the Drawings

The invention will be better understood by the below description of preferred embodiments thereof to be read while referring to the following figures.

Figure 1 is a block diagram of a well-known heterogeneous client/server architecture using object technology, in the context of which preferred embodiments of the present invention can be applied; Figure 2 is a block diagram showing an implementation of an objectbased transaction service implementation used in explaining both the prior art and the preferred embodiment of the present invention;

Figure 3 is a block diagram showing an implementation according to a preferred embodiment of the present invention; and Figure 4 is a flowchart showing the steps which take place within a server process according to the preferred embodiment of the present UK9-98-022 invention, when the server process recovers after having been unavailable.

Detailed Description of the Preferred Embodiments

The Common Object Request Broker (CORBA) object Transaction Service (OTS) supplies an interface object known as "Current" which has a "begin,, method used by client application programs (source code) running in a client process 21 (see Fig. 2) to signal a beginning of a transaction. when the client application is being built or executed on a particular client architecture and contains the "begin" method, the underlying OTS software will proceed (as described above with respect to the prior art) to create the transaction by setting up the transaction objects 221 in a server process 22.

The preferred embodiment of the present invention will now be described, using the example architecture of Fig. 2 as a model. AS was described above, the resources 231, 241 and 251 of server B process 23, server C process 24 and server D process 25, respectively, are directly registered with the transaction objects 221. Server B process 23 has called resource objects 261, 262 and 263, in server E process 26, so resource objects 261, 262 and 263 are registered directly with server B process 23.

AS was also explained above, the transaction is now in doubt, since phase one is completed but phase two has not yet started. At this point, server A process 22 goes down. The operator of server E process 26 commands its resource objects 261, 262 and 263 to take an heuristic decision, since the transaction has been in doubt for a period of time longer than a preset threshold. Typically, a systems administrator would program a configurable variable to assume a particular value Plcommitn or Ilrollbackn) so that this is the direction the heuristic decision will take every time the transaction is in doubt for a period longer than the preset threshold.

when the server A process 22 returns to an operable state after having been down, the server A process checks its local data store 222 to determine the state the transaction was in before the server 22 went down, so that the server 22 can begin to resume the processing of the transaction (see step 41 of Fig. 4).

UK9-98-022 11 According to the preferred embodiment of the present invention, a configurable variable 32 (see Fig. 3) called heuristic-direction is assigned to each server process that is involved in the transaction. The configurable variable 36 is the one that was just described above and is assigned to the server E process 26. As described above, this variable has been programmed by the systems administrator of server E process 26 to assume the value nrollbackn so that whenever a heuristic decision is to be taken, the direction of the decision will be a rollback (rather than a commit). Server A process 22 also has a configurable variable 32 called heuristic-direction which also has been programmed by the systems administrator of server A process 22 to assume the value llrollbackll.

The systems administrators of the various servers (if they are not one and the same entity) need to communicate with each other so that each can program the heuristic_direction variable of its respective server to the same value ("commit" or %rollback n).

Thus, once the server A process 22 has recovered from a failure and checked its local store 222 to learn the state of the transaction before the failure, the server A process 22 determines from this stored data whether the transaction was in the prepared state (step 42) (also known as the state where the directly registered resource objects are in doubt) If the stored state data of the transaction indicates that the transaction was in the prepared state when the server 22 went down, the server 22 determines the value of server 22's heuristic_direction variable (step 43). if the value is commit, server 22 sends a commit command (step 44) to directly registered resources 231, 241 and 251 during phase two of the two phase commit protocol. If the value is rollback, server 22 sends a rollback command (step 45) to directly registered resources 231, 241 and 251 during phase two of the two phase commit protocol.

In the example that has been used above, the server 22 goes down while the resource objects 231, 241, 251, 261, 262 and 263 are in the prepared state (at the end of phase one of the two phase commit protocol). During the time when the server 22 is down, the operator of server 26 decides that its resources 261, 262 and 263 have been tied up by this transaction for an unacceptably long period of time (or, this decision is taken automatically by measuring the length of time during which the resource objects have been placed in the prepared state and comparing the measured time to a preset threshold), so a heuristic UK9-98-022 12 decision is taken with respect to resources 261, 262 and 263. Since the value of heuristic_direction variable 36 is Ilrollbackll, the heuristic decision that server 26 will take is in the rollback direction (rather than in the commit direction). This frees the locks held on resources 261, 262 and 263 and cancels any data changes to these resources which occurred during the transaction. Then, when the server 22 recovers from the failure, server 22 checks (step 41) its local datastore 222, determines that the transaction was in the prepared state when the server 22 failed ("went down,') (step 42) and then checks the value of the heuristic-direction variable 32 (step 43). The value of variable 32 of server 22 is Ilrollbackll, the same value as the variable 36 of server 26 (this is because there was a previous agreement between servers to set the value of this variable to be the same, systemwide). Thus, the server 22 begins phase two of the two phase commit protocol by sending (step 45) a rollback command to server 22's directly registered resource objects (231, 241 and 251) Thus, the heuristic decision taken by server 26 has not caused heuristic damage to the system.

Basically, any server (such as server 26) that has the ability to take an heuristic decision is provided with a configurable heuristic_direction variable and the value of all of these variables are set to be the same. Then, further, the server which contains the transaction objects 221 is also provided with a configurable heuristic-direction variable and the value of this variable is set to be the same as the value of theheuristic-direction variables of the other servers. This way, the servers which have the ability to take a heuristic decision (e.g., server 26) can be freely allowed to take such a decision to free up its resources from an oppressively long-running transaction (caused by, e.g., a server going down), without the fear that this decision will cause heuristic damage. All such servers will take an heuristic decision in the same direction, and the server containing the transaction objects will follow this same direction when issuing commands to resources during phase two of the two phase commit protocol.

Before the present invention, the server 22 had the freedom to take a decision in either direction (commit or rollback). The preferred embodiment of the present invention removes this freedom by constraining such servers to now always follow an agreed direction, such a value being agreed on in advance with respect to all servers that can take an UK9-98-022 13 heuristic decision and with respect to the server containing the transaction objects.

UK9-98-022 14

Claims (8)

1. A server for use in a heterogeneous client/server computing system which coordinates the processing of distributed transactions in the client/server computing system, the server comprising:

means for coordinating a first phase of a two phase commit protocol used in the processing of a transaction amongst a plurality of transactionally involved resources in order to place the plurality of resources in a prepared state at the completion of the first phase; means for recovering from a failure which has occurred subsequent to the end of the first phase of the two phase commit protocol; and means for, upon recovering from the failure, coordinating a second phase of the two Phase commit protocol by following an heuristic direction, the heuristic direction having been agreed upon in advance by each transactionally involved server that has an ability to take an heuristic decision.

2. The server of claim 1 wherein the following of the heuristic direction involves sending a commit command to each transactionallyinvolved resource if the heuristic direction is commit and sending a rollback command to each transactionally-involved resource if the heuristic direction is rollback.

3. A method of operating a server for use in a heterogeneous client/server computing system which coordinates the processing of distributed transactions in the client/server computing system, the method comprising steps of:

coordinating a first phase of a two phase commit protocol used in the processing of a transaction amongst a plurality of transactionally involved resources in order to place the plurality of resources in a prepared state at the completion of the first phase; recovering from a failure which has occurred subsequent to the end of the first phase of the two phase commit protocol; and coordinating a second phase of the two phase commit protocol by following an heuristic direction, the heuristic direction having been UK9-98-022 agreed upon in advance by each transactionally involved server that has an ability to take an heuristic decision.

is

4. The method of claim 3 wherein the following of the heuristic direction involves sending a commit command to each transactionallyinvolved resource if the heuristic direction is commit and sending a rollback command to each transactionally-involved resource if the heuristic direction is rollback.

5. A computer program product stored on a computer readable storage medium for, when run on a computer, carrying out a server processing method for use in a heterogeneous client/server computing system which carries out transactions, said method comprising steps of:

coordinating a first phase of a two phase commit protocol used in the processing of a transaction amongst a plurality of transactionally involved resources in order to place the plurality of resources in a prepared state at the completion of the first phase; recovering from a failure which has occurred subsequent to the end of the first phase of the two phase commit protocol; and coordinating a second phase of the two phase commit protocol by following an heuristic direction, the heuristic direction having been agreed upon in advance by each transactionally involved server that has an ability to take an heuristic decision.

6. The product of claim 5 wherein the following of the heuristic direction involves sending a commit command to each transactionallyinvolved resource if the heuristic direction is commit and sending a rollback command to each transactionally-involved resource if the heuristic direction is rollback.

7. A server for use in a heterogeneous client/server computing system which carries out transactions, said server comprising:

a resource; means for updating values associated with the resource; UK9-98-022 16 means for taking an heuristic decision with respect to the resource when the resource has been placed in a prepared state during the processing of a distributed transaction; characterized in that the heuristic decision is always taken in a direction which has been agreed amongst other servers in said heterogeneous client/server computing system.

8. A method of operating a server for use in a heterogeneous client/server computing system which carries out transactions, said method comprising steps of:

registering a local resource with a transaction; updating values associated with the resource as part of the transaction; receiving a command to enter into a prepared state as part of a two phase commit protocol of the transaction; taking an heuristic decision with respect to the resource when the resource has been placed in a prepared state during the processing of a distributed transaction; characterized in that the heuristic decision is always taken in a direction which has been agreed amongst other servers in said heterogeneous client/server computing system.