CN113986941A - Transaction batch processing method and device - Google Patents
Transaction batch processing method and device Download PDFInfo
- Publication number
- CN113986941A CN113986941A CN202111424141.2A CN202111424141A CN113986941A CN 113986941 A CN113986941 A CN 113986941A CN 202111424141 A CN202111424141 A CN 202111424141A CN 113986941 A CN113986941 A CN 113986941A
- Authority
- CN
- China
- Prior art keywords
- processed
- processing
- transaction
- batch
- records
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 85
- 238000005096 rolling process Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010923 batch production Methods 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
- G06F16/2386—Bulk updating operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/235—Update request formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a transaction batch processing method and device, which can be applied to the financial field or other fields. By acquiring batch data; dividing the batch data to obtain a plurality of transactions, wherein each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed; executing the affair, and performing related business logic processing on a batch of data by using an SQL statement to obtain a processing result; designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result. In the scheme, batch data is divided into a plurality of transactions, a batch of data processed by each transaction is processed by using SQL statements, and corresponding transaction control operation is performed according to an obtained processing result, so that the performance and the data consistency are ensured.
Description
Technical Field
The invention relates to the technical field of database transaction processing, in particular to a transaction batch processing method and device.
Background
Batch processing is a common transaction processing mode for databases, and in the process of batch processing, when batch processing is abnormal due to individual record format errors or logic problems, transactions are usually rolled back first and then are processed item by item, the record causing the errors is skipped, and the skipped record is recorded in an abnormal log, at the moment, only the transaction caused by the skipped record is rolled back, and the success of other normally recorded transactions is not influenced.
Therefore, the existing batch processing method does not introduce a mode of eliminating errors and recursively calling, cannot guarantee the performance to the maximum under the condition of individual record exception, and does not consider savepoint in some conditions so as to guarantee the transaction consistency of a single record.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for processing transactions in batch to achieve the purpose of ensuring performance and transaction consistency.
In order to achieve the above purpose, the embodiments of the present invention provide the following technical solutions:
the first aspect of the embodiment of the invention discloses a transaction batch processing method, which comprises the following steps:
acquiring batch data to be processed;
dividing the batch data to obtain a plurality of transactions, wherein each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed;
executing the transaction, and performing related business logic processing on the batch of data by using an SQL statement to obtain a processing result;
designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result.
Optionally, the designing a generic mechanism, and applying the generic mechanism to process the transaction according to the processing result includes:
and designing a general mechanism, and if the processing result is a successful processing result, submitting the transaction by applying the general mechanism.
Optionally, the designing a generic mechanism, and applying the generic mechanism to process the transaction according to the processing result includes:
designing a general mechanism, and if the processing result is a processing failure result, rolling back the transaction by applying the general mechanism;
and processing the records to be processed one by one until all the records to be processed are processed.
Optionally, before the processing the records to be processed one by one, the method further includes:
a rollback point is set.
Optionally, the processing the records to be processed one by one until all the records to be processed are processed includes:
processing the records to be processed one by one again;
and when any record to be processed has an error, skipping the record to be processed, and rolling back the transaction generated by the record to be processed to the roll-back point.
Optionally, the method further includes:
acquiring the remaining records to be processed;
and performing recursive call processing on the remaining records to be processed by using the SQL statement again until all the remaining records to be processed are processed.
The second aspect of the embodiments of the present invention discloses a transaction batch processing apparatus, which includes:
the acquisition module is used for acquiring batch data to be processed;
the dividing module is used for dividing the batch data to obtain a plurality of transactions, each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed;
the processing module is used for executing the transaction and performing related business logic processing on the batch of data by using an SQL statement to obtain a processing result; designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result.
Optionally, the processing module, configured to design a general mechanism and apply the general mechanism to process the transaction according to the processing result, is specifically configured to:
and designing a general mechanism, and if the processing result is a successful processing result, submitting the transaction by applying the general mechanism.
Optionally, the processing module, configured to design a general mechanism and apply the general mechanism to process the transaction according to the processing result, is further specifically configured to:
designing a general mechanism, and if the processing result is a processing failure result, rolling back the transaction by applying the general mechanism; and processing the records to be processed one by one until all the records to be processed are processed.
Optionally, before the processing module processes the records to be processed one by one, the processing module is further configured to:
a rollback point is set.
Based on the above method and apparatus for processing transactions in batch provided by the embodiments of the present invention, the method includes: acquiring batch data; dividing the batch data to obtain a plurality of transactions, wherein each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed; executing the transaction, and performing related business logic processing on the batch of data by using an SQL statement to obtain a processing result; designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result. In the scheme, batch data is divided into a plurality of transactions, a batch of data processed by each transaction is processed by using SQL statements, and corresponding transaction control operation is performed according to an obtained processing result, so that the performance and the data consistency are ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart illustrating a method for processing transaction batches according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating another transaction batch processing method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of processing records to be processed one by one according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a transaction batch processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein.
The transaction batch processing method and device provided by the invention can be used in the financial field or other fields, for example, can be used in a multi-employee pay application scenario in the financial field. Other fields are any fields other than the financial field, for example, the field of database transactions. The above is merely an example, and does not limit the application field of the transaction batch processing method and apparatus provided by the present invention.
In order to facilitate understanding of the technical solution of the present invention, technical terms appearing in the present invention are explained:
batch treatment: refers to a process that processes multiple instructions or data collectively in a batch. For example, a text file is processed in batch; banks batch-process the brokerage payroll business of multiple employees. Compared with single-pen online transaction processing, batch processing improves performance by processing data in a centralized manner, and transaction consistency is also guaranteed.
Transaction: is computer terminology, commonly used in databases, refers to a complete unit of data modification. The transaction should have 4 attributes: atomicity, consistency, isolation, durability. Operations in a transaction are either all done or none.
A transaction save point: the database provides Savepoint which divides a large transaction into smaller parts, transaction points can be marked at any position in a long transaction by using Savepoint, and subsequently, the transaction points can be rolled back to the designated transaction point mark by a command.
It can be known from the background art that the existing batch processing method does not introduce a mode of removing errors and recursively calling, cannot maximally guarantee performance under the condition of individual record exception, and does not consider savepoint in some cases to guarantee transaction consistency of a single record.
In the scheme, batch data is divided into a plurality of transactions, a batch of data processed by each transaction is processed by using an SQL statement, and corresponding transaction control operation is performed according to an obtained processing result, so that the performance and the data consistency are ensured.
As shown in fig. 1, a schematic flow chart of a transaction batch processing method according to an embodiment of the present invention is provided, where the method mainly includes the following steps:
step S101: and acquiring batch data to be processed.
In the process of implementing step S101 specifically, when performing database operation, first, data corresponding to a database operation object is acquired, that is, batch data to be processed is acquired.
Step S102: and dividing the batch data to obtain a plurality of transactions.
In step S102, a batch of data is processed per transaction.
A batch of data contains a plurality of records to be processed.
In the process of implementing step S102, the acquired batch data is divided into a plurality of transactions, and a batch of data is processed in each transaction, that is, a batch of records to be processed is processed in each transaction.
Step S103: and executing the transaction, and performing related business logic processing on a batch of data by using the SQL statement to obtain a processing result.
In step S103, the processing result includes a processing success result and a processing failure result.
Business logic is logic that processes a batch of data or a piece of data.
In the process of implementing step S103 specifically, any transaction is executed, and a batch of data processed by the transaction is processed by using an SQL statement, so as to obtain a processing success result or a processing failure result.
Step S104: designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result.
In the embodiment of the invention, the general mechanism is to design a recursive calling mode through pseudo codes.
In a particular implementation, transactions are processed through pseudo code recursive calls.
In the process of specifically implementing step S104, a general mechanism is designed, and if the obtained processing result is a processing success result, the general mechanism is applied to submit the executed transaction, and if the obtained processing result is a processing failure result, the general mechanism is applied to rollback the executed transaction.
Based on the transaction batch processing method provided by the embodiment of the invention, batch data is obtained; dividing the batch data to obtain a plurality of transactions, wherein each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed; executing the affair, and performing related business logic processing on a batch of data by using an SQL statement to obtain a processing result; designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result. In the scheme, batch data is divided into a plurality of transactions, a batch of data processed by each transaction is processed by using SQL statements, and corresponding transaction control operation is performed according to an obtained processing result, so that the performance and the data consistency are ensured.
Based on the above-mentioned transaction batch processing method shown in fig. 1 in the embodiment of the present invention, as shown in fig. 2, a flow diagram of another transaction batch processing method provided in the embodiment of the present invention is shown, where the transaction batch processing method mainly includes the following steps:
step S201: and acquiring batch data to be processed.
Step S202: and dividing the batch data to obtain a plurality of transactions.
Step S203: and executing the transaction, and performing related business logic processing on a batch of data by using the SQL statement to obtain a processing result.
It should be noted that the execution principle and procedure of the above steps S201 to S203 are the same as the execution principle and procedure of the steps S101 to S103 disclosed in fig. 1, and it can be referred to that, which is not described herein again.
Step S204: designing a general mechanism, and determining whether the processing result is a successful processing result, if so, executing step S205, and if not, executing step S206.
In the process of implementing step S204 specifically, it is determined whether the processing result is a processing success result, if so, step S205 is executed, and if not, step S206 is executed to indicate that the processing result is a processing failure result.
Step S205: a generic mechanism is applied to commit the transaction.
In the process of specifically implementing step S205, if it is determined that the obtained processing result is a successful processing result, the currently executed transaction is submitted by applying a general mechanism.
Step S206: a generic mechanism is applied to roll back transactions.
In the process of implementing step S206 specifically, if it is determined that the obtained processing result is a processing failure result, the currently executed transaction is rolled back by applying a general mechanism.
Alternatively, the currently executing transaction may be rolled back in a recursive manner.
Step S207: and processing the records to be processed one by one until all the records to be processed are processed.
In the process of implementing step S207, all the records to be processed are processed one by one, starting again until the processing of all the records to be processed is completed.
Optionally, before performing step S207 to process the records to be processed one by one, the method further includes:
a rollback point is set.
In an embodiment of the invention, the rollback point is a transaction savepoint.
It should be noted that the rollback point is used to roll back the transaction to the specified rollback point when any pending record has a transmission error, that is, to roll back the transaction to the specified transaction holding point when any pending record has a transmission error.
Preferably, in a specific embodiment, the rollback point may be denoted savepoint.
Optionally, in step S207, the records to be processed are processed one by one until all the records to be processed are processed, as shown in fig. 3, which is a schematic flow diagram for processing records to be processed one by one according to an embodiment of the present invention, and the method mainly includes the following steps:
step S301: and processing the records to be processed one by one again.
In the process of implementing step S301, all the records to be processed are processed one by one.
Step S302: and judging whether any record to be processed has an error, if so, executing the step S303, otherwise, returning to the step S301.
Step S303: and skipping the pending records and rolling back the transaction generated by the pending records to a roll-back point.
In the process of implementing step S303 specifically, if it is determined that an error occurs in a certain record to be processed, the record to be processed is skipped, the transaction generated by the record to be processed is rolled back to the roll-back point, and roll-back processing is performed, so that it is ensured that no transaction actually occurs in the skipped error record.
Optionally, after rolling back the transaction generated by the record to be processed to the roll-back point in step S303, the method further includes:
step S304: and acquiring the remaining records to be processed.
In the process of specifically implementing step S304, after the records to be processed with errors are removed, the remaining records to be processed have a low error probability, so that the remaining records to be processed need to be processed, and at this time, the remaining records to be processed need to be acquired first.
Step S305: and performing recursive call processing on the remaining records to be processed by using the SQL statement again until all the remaining records to be processed are processed.
In the process of implementing step S305 specifically, the obtained remaining records to be processed are used as a batch of data again, and the SQL statement is used to perform recursive call processing on the batch of data, so as to ensure performance, and if any remaining record to be processed is incorrect, recursive loop operation is repeatedly performed until processing of all remaining records to be processed is completed, and at this time, processing of all records to be processed is completed.
Based on the transaction batch processing method provided by the embodiment of the invention, when the processing result is a processing failure result, a general mechanism is applied to rollback the currently executed transaction and process the records to be processed one by one, when an error occurs in one record to be processed, the record to be processed is skipped, and the transaction generated by the record to be processed is rolled back to a rollback point, so that the skipped error record is ensured not to actually generate the transaction, and SQL statements are used again to carry out recursive call processing on the rest records to be processed until all the rest records to be processed are processed, so that the performance and the data consistency are ensured.
In order to better understand the transaction batch processing method provided by the above embodiments of the present invention, the following description will be made in detail by taking an example.
For example, a recursive call is made using pseudo-code, and the associated pseudo-code is as follows:
the key code is explained below.
Firstly, a new transaction is started to be executed by utilizing a begin transaction statement, and a batch of data in the divided transactions is used as input to be subjected to subsequent processing, wherein the batch of data comprises a plurality of records to be processed.
Then, a batch of data of the new transaction is processed once by using a result (process _ all (transactions) statement and an SQL statement to obtain a processing result, and the submission or rollback of the whole transaction is judged according to the processing result.
If the whole new transaction is currently rolled back, all the records to be processed are reprocessed one by using the for (rec entries) statement, and a roll-back point is set by using the set savepointbf before one is processed by using the process _ one statement.
Next, a pending record is processed using the process _ one statement.
And when a certain record to be processed has an error, rolling back the transaction generated by the record to be processed to a roll-back point, and recording the log. The piece of pending recording is skipped.
Then, the pending record with the successful previous processing is submitted.
And then, obtaining the rest records to be processed by using a get _ left statement, recursively calling the rest records to be processed by using a do process (records _ left) statement, and interrupting the recursive calling and submitting the whole transaction after the rest records to be processed are processed.
If the current records are 0, and all the records to be processed are processed, the recursive call is exited.
Based on the transaction batch processing method provided by the embodiment of the invention, when the processing result is a processing failure result, a general mechanism is applied to rollback the currently executed transaction and process the records to be processed one by one, when an error occurs in one record to be processed, the record to be processed is skipped, and the transaction generated by the record to be processed is rolled back to a rollback point, so that the skipped error record is ensured not to actually generate the transaction, and SQL statements are used again to carry out recursive call processing on the rest records to be processed until all the rest records to be processed are processed, so that the performance and the data consistency are ensured.
Corresponding to the above transaction batch processing method shown in fig. 1 in the embodiment of the present invention, an embodiment of the present invention further provides a transaction batch processing apparatus. As shown in fig. 4, the transaction batch processing apparatus includes: an acquisition module 41, a dividing module 42 and a processing module 43.
The obtaining module 41 is configured to obtain batch data to be processed.
And the dividing module 42 is configured to divide the batch data to obtain a plurality of transactions.
Each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed.
The processing module 43 is configured to execute a transaction, and perform related business logic processing on a batch of data by using an SQL statement to obtain a processing result; designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result.
It should be noted that, the specific principle and the execution process of each module or each unit in the transaction batch processing apparatus disclosed in the above embodiment of the present invention are the same as those of the method for implementing transaction batch processing in the above embodiment of the present invention, and reference may be made to corresponding parts in the method for implementing transaction batch processing in the above embodiment of the present invention, and details are not described here again.
Based on the transaction batch processing device provided by the embodiment of the invention, batch data is obtained; dividing the batch data to obtain a plurality of transactions, wherein each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed; executing the affair, and performing related business logic processing on a batch of data by using an SQL statement to obtain a processing result; designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result. In the scheme, batch data is divided into a plurality of transactions, a batch of data processed by each transaction is processed by using SQL statements, and corresponding transaction control operation is performed according to an obtained processing result, so that the performance and the data consistency are ensured.
Optionally, based on the processing module 43 shown in fig. 4, the processing module 43 for designing a general mechanism and applying the general mechanism to process a transaction according to a processing result is specifically configured to:
and designing a general mechanism, and if the processing result is a successful processing result, submitting the transaction by using the general mechanism.
Based on the transaction batch processing device provided by the embodiment of the invention, when the processing result is the successful processing result, the currently executed transaction is submitted, so that the performance and the transaction consistency are ensured.
Optionally, based on the processing module 43 shown in fig. 4, the processing module 43 for designing a general mechanism and applying the general mechanism to process a transaction according to a processing result is further specifically configured to:
designing a general mechanism, and if the processing result is a processing failure result, rolling back the transaction by applying the general mechanism; and processing the records to be processed one by one until all the records to be processed are processed.
Based on the transaction batch processing device provided by the embodiment of the invention, when the processing result is the processing failure result, the current executed transaction is rolled back by applying a general mechanism, and the records to be processed are processed one by one again, so that the performance and the data consistency are ensured.
Optionally, based on the processing module 43 shown in fig. 4, before the processing module 43 processes the records to be processed one by one, the processing module is further configured to:
a rollback point is set.
Based on the transaction batch processing device provided by the embodiment of the invention, the rollback point is set, so that when an error occurs in a certain subsequent record to be processed, the transaction generated by the record to be processed is rolled back to the rollback point, and the performance and the data consistency are ensured.
Optionally, based on the processing module 43 shown in fig. 4, the processing module 43 for processing the records to be processed one by one until all the records to be processed are processed is specifically configured to:
processing the records to be processed one by one again; and when any record to be processed has an error, skipping the record to be processed, and rolling back the transaction generated by the record to be processed to a roll-back point.
Based on the transaction batch processing device provided by the embodiment of the invention, when a certain record to be processed has an error, the transaction generated by the record to be processed is rolled back to the roll-back point, so that the performance and the data consistency are ensured.
Optionally, based on the processing module 43 shown in fig. 4, the processing module 43 for processing the records to be processed one by one until all the records to be processed are processed is further specifically configured to:
acquiring the remaining records to be processed; and performing recursive call processing on the remaining records to be processed by using the SQL statement again until all the remaining records to be processed are processed.
Based on the transaction batch processing device provided by the embodiment of the invention, the residual records to be processed are recursively called by reusing the SQL statements, so that the performance and the data consistency are ensured.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for batch processing of transactions, the method comprising:
acquiring batch data to be processed;
dividing the batch data to obtain a plurality of transactions, wherein each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed;
executing the transaction, and performing related business logic processing on the batch of data by using an SQL statement to obtain a processing result;
designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result.
2. The method of claim 1, wherein designing the generic mechanism and applying the generic mechanism to process the transaction according to the processing result comprises:
and designing a general mechanism, and if the processing result is a successful processing result, submitting the transaction by applying the general mechanism.
3. The method of claim 1, wherein designing the generic mechanism and applying the generic mechanism to process the transaction according to the processing result comprises:
designing a general mechanism, and if the processing result is a processing failure result, rolling back the transaction by applying the general mechanism;
and processing the records to be processed one by one until all the records to be processed are processed.
4. The method of claim 3, further comprising, prior to said processing said pending records on a per-record basis:
a rollback point is set.
5. The method according to claim 3 or 4, wherein the processing the records to be processed one by one until all records to be processed are processed comprises:
processing the records to be processed one by one again;
and when any record to be processed has an error, skipping the record to be processed, and rolling back the transaction generated by the record to be processed to the roll-back point.
6. The method of claim 5, further comprising:
acquiring the remaining records to be processed;
and performing recursive call processing on the remaining records to be processed by using the SQL statement again until all the remaining records to be processed are processed.
7. An apparatus for batch processing of transactions, the apparatus comprising:
the acquisition module is used for acquiring batch data to be processed;
the dividing module is used for dividing the batch data to obtain a plurality of transactions, each transaction processes a batch of data, and the batch of data comprises a plurality of records to be processed;
the processing module is used for executing the transaction and performing related business logic processing on the batch of data by using an SQL statement to obtain a processing result; designing a general mechanism, and applying the general mechanism to process the transaction according to the processing result.
8. The apparatus according to claim 7, wherein the processing module configured to design a generic mechanism and apply the generic mechanism to process the transaction according to the processing result is specifically configured to:
and designing a general mechanism, and if the processing result is a successful processing result, submitting the transaction by applying the general mechanism.
9. The apparatus according to claim 7, wherein the processing module for designing a generic mechanism and applying the generic mechanism to process the transaction according to the processing result is further specifically configured to:
designing a general mechanism, and if the processing result is a processing failure result, rolling back the transaction by applying the general mechanism; and processing the records to be processed one by one until all the records to be processed are processed.
10. The apparatus of claim 9, further configured to, before the processing module processes the to-be-processed records item by item:
a rollback point is set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111424141.2A CN113986941A (en) | 2021-11-26 | 2021-11-26 | Transaction batch processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111424141.2A CN113986941A (en) | 2021-11-26 | 2021-11-26 | Transaction batch processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113986941A true CN113986941A (en) | 2022-01-28 |
Family
ID=79732236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111424141.2A Pending CN113986941A (en) | 2021-11-26 | 2021-11-26 | Transaction batch processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113986941A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117632140A (en) * | 2023-12-07 | 2024-03-01 | 易方达基金管理有限公司 | Business process processing method, device and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106033439A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method and system for processing distributed transaction |
| CN110990182A (en) * | 2019-12-03 | 2020-04-10 | 腾讯科技（深圳）有限公司 | Transaction processing method, device, equipment and storage medium |
-
2021
- 2021-11-26 CN CN202111424141.2A patent/CN113986941A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106033439A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method and system for processing distributed transaction |
| CN110990182A (en) * | 2019-12-03 | 2020-04-10 | 腾讯科技（深圳）有限公司 | Transaction processing method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| DATANGXIAJUN: ""java批处理异常继续执行", Retrieved from the Internet <URL:https://blog.csdn.net/datangxiajun/article/details/52150093> * |
| 蓝蓝223: "Java 数据库编程6---事务处理", Retrieved from the Internet <URL:https://blog.csdn.net/qq_21808961/article/details/80984122/> * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117632140A (en) * | 2023-12-07 | 2024-03-01 | 易方达基金管理有限公司 | Business process processing method, device and storage medium |
| CN117632140B (en) * | 2023-12-07 | 2024-07-05 | 易方达基金管理有限公司 | Business process processing method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110032428B (en) | Method and device for executing distributed transaction | |
| CN106445644B (en) | Treating method and apparatus based on the distributed transaction that an improved stage submits | |
| US20140358855A1 (en) | Replicating data to a database | |
| CN107437222B (en) | Processing method and system of online business data based on front end of bank counter | |
| CN111444199B (en) | Data processing method and device, storage medium and processor | |
| CN102203779B (en) | Method for updating data and control apparatus thereof | |
| CN114253673A (en) | Transaction processing method and transaction processing device of distributed system | |
| US20210286614A1 (en) | Causality determination of upgrade regressions via comparisons of telemetry data | |
| US10769307B2 (en) | Processing system using natural language processing for performing dataset filtering and sanitization | |
| CN107169767B (en) | Transaction processing method and system | |
| CN111752740A (en) | Microservice anomaly compensation method, apparatus, device, and computer-readable storage medium | |
| CN112765126A (en) | Database transaction management method and device, computer equipment and storage medium | |
| CN113986941A (en) | Transaction batch processing method and device | |
| CN116225630A (en) | Transaction processing method, apparatus, electronic device, storage medium, and program product | |
| CN113704277B (en) | A database-based breakpoint resuming method and related devices | |
| CN110232069B (en) | Customer information intelligent input method and device based on big data | |
| CN105574020B (en) | Database operation method and device | |
| CN106776052B (en) | Shared resource access method and device | |
| CN115934272A (en) | Online batch task processing method and device | |
| CN115729946A (en) | Method, device, equipment and storage medium for processing distributed transaction | |
| CN111143277B (en) | Data processing method and device and electronic equipment | |
| CN114625743A (en) | Data updating method and device for personnel master data and electronic equipment | |
| CN118733202B (en) | Method and related device for optimizing the processing flow of database sub-transactions | |
| US11831490B1 (en) | Systems, methods, and media for performing information technology service management correlation for infrastructure environment functions | |
| US20250190436A1 (en) | Data query method and apparatus, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |