CN104298570A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN104298570A CN104298570A CN201410648492.5A CN201410648492A CN104298570A CN 104298570 A CN104298570 A CN 104298570A CN 201410648492 A CN201410648492 A CN 201410648492A CN 104298570 A CN104298570 A CN 104298570A
- Authority
- CN
- China
- Prior art keywords
- phase process
- state
- failed
- multiple phase
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 238000000034 method Methods 0.000 claims description 563
- 230000008569 process Effects 0.000 claims description 558
- 230000000694 effects Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000006116 polymerization reaction Methods 0.000 description 2
- 230000018199 S phase Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Landscapes
- Retry When Errors Occur (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a data processing method and a data processing device. The data processing method comprises the following steps: acquiring data to be processed; sequentially executing multiple stages of processing on the data to be processed, wherein the multiple stages of processing refer to processing stages sequentially and continuously executed on the data to be processed; recording the processing states of the multiple stages of processing, wherein the processing states comprise processing success and processing failure; judging whether the state of processing failure exists in the processing states or not; if yes, executing the stage of processing corresponding to the state of processing failure again. Through the data processing method and the data processing device, the problem in the prior art that the data processing efficiency is low is solved, and the effect of improving the data processing efficiency is realized.
Description
Technical field
The present invention relates to data processing field, in particular to a kind of data processing method and device.
Background technology
Flow chart of data processing of the prior art often only has the input of raw data, and intermediate treatment process and net result export.More complicated for data, the flow chart of data processing that the processing time is longer, occur mistake in intermediate treatment process or extremely often cause the failure of whole treatment scheme, this just needs again to process total data, and the efficiency of data processing is low.
For the inefficient problem of data processing in prior art, at present effective solution is not yet proposed.
Summary of the invention
Fundamental purpose of the present invention is to provide a kind of data processing method and device, to solve the inefficient problem of data processing in prior art.
To achieve these goals, according to an aspect of the embodiment of the present invention, a kind of data processing method is provided.Data processing method according to the present invention comprises: obtain pending data; Multiple phase process is performed successively to described pending data, described multiple phase process be to described pending data perform continuously successively processing stage; Record the treatment state of described multiple phase process, described treatment state comprises and processes successfully and process unsuccessfully; Judge in described treatment state, whether to there is the failed state of described process; And if judge that described treatment state exists the failed state of described process, then re-execute and describedly process phase process corresponding to failed state.
Further, after judging that described treatment state exists the failed state of described process, and before re-executing phase process corresponding to the failed state of described process, described data processing method also comprises: determine from described treatment state and perform the failed state of described process that first time in described multiple level processes occurs, the phase process re-executing the failed state of described process corresponding comprises: that from described multiple phase process, determines that described first time occurs describedly processes phase process corresponding to failed state; Re-execute the phase process determined and the later phase process of the described phase process determined.
Further, after multiple phase process is performed successively to described pending data, described data processing method also comprises: the intermediate processing results of multiple phase process described in buffer memory, wherein, the phase process re-executing the failed state of described process corresponding comprises: the described result processing a upper phase process of phase process corresponding to failed state obtaining buffer memory; Phase process corresponding to the failed state of described process is performed to the result obtained.
Further, the treatment state recording described multiple phase process comprises: the timestamp recording described multiple phase process, wherein, executing in described multiple phase process, to upgrade timestamp corresponding to the phase process that executes after each phase process be Preset Time stamp, judges that whether there is the failed state of described process in described treatment state comprises: whether the timestamp judging described multiple phase process is successively that described Preset Time stabs; If judge that the timestamp of described multiple phase process is not described Preset Time stamp, then determine that timestamp is not that the state of the phase process of described Preset Time stamp is for process unsuccessfully.
Further, the treatment state recording described multiple phase process comprises: according to the implementation of described multiple phase process in number record table successively recording processing state be the numbering of the phase process run succeeded, wherein, described multiple phase process is previously provided with numbers one to one with described multiple phase process, judges that whether there is the failed state of described process in described treatment state comprises: the phase process judging whether to exist non-record number from described number record table; If judge the phase process that there is non-record number, then determine that the state of the phase process of non-record number is for process unsuccessfully.
To achieve these goals, according to the another aspect of the embodiment of the present invention, provide a kind of data processing equipment.Data processing equipment according to the present invention comprises: acquiring unit, for obtaining pending data; First performance element, for performing multiple phase process successively to described pending data, described multiple phase process be to described pending data perform continuously successively processing stage; Record cell, for recording the treatment state of described multiple phase process, described treatment state comprises and processes successfully and process unsuccessfully; Whether judging unit, exist the failed state of described process for judging in described treatment state; And second performance element, if for judging that described treatment state exists the failed state of described process, then re-execute and describedly process phase process corresponding to failed state.
Further, described data processing equipment also comprises: determining unit, for after judging that described treatment state exists the failed state of described process, and before re-executing phase process corresponding to the failed state of described process, determine from described treatment state and perform the state that in described multiple level processes, the described process of first time appearance is failed, described second performance element comprises: the first determination module, for determining the phase process that state that described process that described first time occurs is failed is corresponding from described multiple phase process; First execution module, for re-executing the phase process determined and the later phase process of the described phase process determined.
Further, described data processing equipment also comprises: buffer unit, for after performing multiple phase process successively to described pending data, the intermediate processing results of multiple phase process described in buffer memory, wherein, described second performance element comprises: acquisition module, the result of a upper phase process of the phase process that the state that the described process for obtaining buffer memory is failed is corresponding; Second execution module, for performing phase process corresponding to the failed state of described process to the result obtained.
Further, described record cell comprises: the first logging modle, for recording the timestamp of described multiple phase process, wherein, executing in described multiple phase process, to upgrade timestamp corresponding to the phase process that executes after each phase process be Preset Time stamp, described judging unit comprises: the first judge module, for judging whether the timestamp of described multiple phase process is described Preset Time stamp successively; Second determination module, if for judging that the timestamp of described multiple phase process is not described Preset Time stamp, then determines that timestamp is not that the state of the phase process of described Preset Time stamp is for process unsuccessfully.
Further, described record cell comprises: the second logging modle, for the implementation according to described multiple phase process in number record table successively recording processing state be the numbering of the phase process run succeeded, wherein, described multiple phase process is previously provided with numbers one to one with described multiple phase process, described judging unit comprises: the second judge module, for judging whether the phase process that there is non-record number from described number record table; 3rd determination module, if for judging the phase process that there is non-record number, then determines that the state of the phase process of non-record number is for process unsuccessfully.
According to the embodiment of the present invention, by recording the treatment state of multiple phase process, judge in treatment state, whether to there is the failed state of process, if judge that treatment state exists the failed state of process, then re-execute the phase process that the failed state of process is corresponding, for the successful phase process of process, then without the need to again processing, like this, again above-mentioned multiple phase process is performed to pending data when avoiding abnormal in data processing, solve the inefficient problem of data processing in prior art, reach the effect of the efficiency improving data processing.
Accompanying drawing explanation
The accompanying drawing forming a application's part is used to provide a further understanding of the present invention, and schematic description and description of the present invention, for explaining the present invention, does not form inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the process flow diagram of the data processing method according to the embodiment of the present invention; And
Fig. 2 is the schematic diagram of the data processing equipment according to the embodiment of the present invention.
Embodiment
It should be noted that, when not conflicting, the embodiment in the application and the feature in embodiment can combine mutually.Below with reference to the accompanying drawings and describe the present invention in detail in conjunction with the embodiments.
The present invention program is understood better in order to make those skilled in the art person, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the embodiment of a part of the present invention, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, should belong to the scope of protection of the invention.
It should be noted that, term " first ", " second " etc. in instructions of the present invention and claims and above-mentioned accompanying drawing are for distinguishing similar object, and need not be used for describing specific order or precedence.Should be appreciated that the data used like this can be exchanged, in the appropriate case so that embodiments of the invention described herein.In addition, term " comprises " and " having " and their any distortion, intention is to cover not exclusive comprising, such as, contain those steps or unit that the process of series of steps or unit, method, system, product or equipment is not necessarily limited to clearly list, but can comprise clearly do not list or for intrinsic other step of these processes, method, product or equipment or unit.
Embodiments provide a kind of data processing method.The method can be realized by computer equipment.
Fig. 1 is the process flow diagram of the data processing method according to the embodiment of the present invention.As shown in Figure 1, to comprise step as follows for this data processing method:
Step S102, obtains pending data.
Pending data can be need the data through multiple phase process, such as daily record data, in storing daily record data, first can carry out classification and the polymerization of daily record data, then carry out multiple stage such as storing.
Step S104, performs multiple phase process successively to pending data, multiple phase process be to pending data perform continuously successively processing stage.
Above-mentioned multiple phase process is all for the treatment of pending data, wherein, be associated successively between multiple phase process, namely the input data of each phase process are the result of a upper phase process, that is to say that a rear phase process needs to use the result of a phase process.
Step S106, records the treatment state of multiple phase process, and treatment state comprises and processes successfully and process unsuccessfully.
Performing in the process of multiple phase process successively to pending data, record the treatment state of each phase process successively.Particularly, the timestamp of each phase process can be recorded, be determined the treatment state of each phase process by timestamp.Also can be utilize signature identification to record the treatment state of multiple phase process, after each phase process completes, if process successfully, signature identification becomes the first mark, otherwise, then become the second mark, like this, just can by identifying that each signature identification determines whether the phase process of its correspondence processes successfully or failure.
Step S108, judges whether there is the failed state of process in treatment state.
Step S110, if judge that treatment state exists the failed state of process, then re-executes the phase process that the failed state of process is corresponding.
From the treatment state of record, judge whether to process failed state, namely judge in multiple phase process, whether to there is the process failed stage, if had, then re-execute the phase process that process is failed.Due to multiple phase process be perform successively processing stage, so, when one of them phase process failure, its follow-up phase process all processes failure usually, therefore, this phase process and later phase process thereof all re-execute, and the phase process before this phase process is without the need to re-executing, thus improve the efficiency of data processing.
According to the embodiment of the present invention, by recording the treatment state of multiple phase process, judge in treatment state, whether to there is the failed state of process, if judge that treatment state exists the failed state of process, then re-execute the phase process that the failed state of process is corresponding, for the successful phase process of process, then without the need to again processing, like this, again above-mentioned multiple phase process is performed to pending data when avoiding abnormal in data processing, solve the inefficient problem of data processing in prior art, reach the effect of the efficiency improving data processing.
Preferably, after judging that treatment state exists the failed state of process, and before re-executing phase process corresponding to the failed state of process, data processing method also comprises: from treatment state, determine the state performing the process failure that first time in multiple level processes occurs, the phase process re-executing the failed state of process corresponding comprises: from multiple phase process, determine the phase process that the state of the process failure that first time occurs is corresponding; Re-execute the phase process determined and the later phase process of the phase process determined.
Because multiple phase process successively order performs, wherein, phase process needs the result using last phase process next time.When judging that the existence of multiple phase process processes failed state, from treatment state, then determine that the state that process is failed appears in first time, particularly, the state of each phase process can be judged from front to back successively according to the order of multiple phase process, when first time is judged to process failed state, then be there is the state that process is failed in this state first time in treatment state; Also can judge the state of each phase process according to the order of multiple phase process from back to front successively, when first time is judged to process successful state, then the state that process is failed be appearred in the front result once judged first time in treatment state.
After determining the state that first time appearance process is failed, determine the phase process that this state is corresponding to re-execute data processing from this phase process determined.
Consist of example with multiple phase process by two phase process to be described, wherein, two phase process comprise first stage process and subordinate phase process, first stage process and subordinate phase be treated to pending data perform continuously successively processing stage, wherein, perform multiple phase process successively to pending data to comprise: perform first stage process successively to pending data, obtain the first result; Subordinate phase process is performed to the first result, obtains the second result.In the process of execute phase process, record the first state and the second state, the first state is the treatment state of first stage process, and the second state is the treatment state of subordinate phase process.
If the first state and the second state are the successful state of process, then the second result is then final result.If the first state is the state that process is failed, then re-execute first stage process.Certainly, because first stage process is before subordinate phase process, when the first stage processes unsuccessfully, subordinate phase process also processes failure usually, therefore, when the first state is when processing failed state, then need to re-execute first stage process and subordinate phase process.Be when processing failed state, then only need re-execute subordinate phase process when only there being the second state.
In the embodiment of the present invention, after can judging the treatment state of phase process from front to back successively, then determining the phase process that re-executes, also can be the treatment state from judging forward each phase process processing stage of last successively.
The embodiment of the present invention preferably judges the treatment state of each phase process from back to front successively.Particularly, when multiple phase process comprises first stage process and subordinate phase process, first judge whether the second state is the state that process is failed, if so, then continue to judge whether the first state is the state that process is failed; Otherwise, then think that first stage process and subordinate phase process all process successfully.
According to the embodiment of the present invention, by judging the treatment state of each phase process successively from back to front, when judging that treatment state is the successful state of process, then without the need to judging again, and determine the phase process failure after the success of pending data processing or this process success status.
It should be noted that, in the embodiment of the present invention, multiple phase process can also comprise phase III process, fourth stage process etc., and its principle, with first stage process and subordinate phase process, does not repeat here.
Preferably, after multiple phase process is performed successively to pending data, data processing method also comprises: the intermediate processing results of the multiple phase process of buffer memory, wherein, the phase process re-executing the failed state of process corresponding comprises: the result of a upper phase process of the phase process that the state of the process failure of acquisition buffer memory is corresponding; Phase process corresponding to the failed state of process is performed to the result obtained.
After each phase process completes, by the intermediate result of each for buffer memory phase process, like this, when determining to there is the process failed stage in multiple phase process, and when re-executing the phase process of this process failure, last result can be obtained from the intermediate result of buffer memory, carry out subsequent treatment, ensure that the correctness of the phase process re-executed.
Preferably, the treatment state recording multiple phase process comprises: the timestamp recording multiple phase process, wherein, executing in multiple phase process, to upgrade timestamp corresponding to the phase process that executes after each phase process be Preset Time stamp, judges that whether there is the failed state of process in treatment state comprises: whether the timestamp judging multiple phase process is successively that Preset Time stabs; If judge that the timestamp of multiple phase process is not Preset Time stamp, then determine that timestamp is not that the state of the phase process of Preset Time stamp is for process unsuccessfully.
The treatment state of each phase process is recorded by logging timestamp.After each phase process is complete, if process successfully, then the timestamp in this stage is changed to Preset Time stamp; If process unsuccessfully, then timestamp does not change, like this, judge each phase process whether success or failed time, by identifying that timestamp corresponding to each stage judges, the accuracy for the treatment of state identification can be improve.
An application scenarios of the embodiment of the present invention is described below in detail.
Be treated to example with daily record data, treatment scheme can be split as multiple stage by function, such as log read, Data classification, data aggregate three phases.
Create two cache table (LogData table and ClassifyData table), for the intermediate result that log cache reads and Data classification obtains.Be buffered in after log read in LogData table, be buffered in after Data classification in ClassifyData table.What obtain due to data aggregate is net result data, does not therefore need buffer memory.
Create a time sheet TimeRecord, for recording the timestamp of each execute phase.Four row are needed in this example in TimeRecord table.The log read time: ReadTime; Data sorting time: ClassifyTime; The data aggregate time: AggTime; The timestamp of whole flow process: EndTime.
When process starts, first judge whether the timestamp AggTime of the last stage data aggregate in TimeRecord equals EndTime, if equal with EndTime, treatment scheme before explanation is success all, this needs to start once new process, namely empties whole cache table and the process that starts anew.
If AggTime is not equal to EndTime, then judge that whether other times stamp is equal with EndTime one by one.If ReadTime equals EndTime, log read process success in last process is described, Data classification and stage afterwards do not process or process failure.Then process from Data classification, the data during the log read data LogData that the former data used are buffer memory shows.If there is multiple stage in treatment scheme, if then judge EndTime and stage S-phase etc., then perform from the S+1 stage, the data buffer storage of operational phase S is as input.
After each stage completes by data buffer storage to corresponding tables of data, and upgrade the timestamp in corresponding stage in TimeRecord table and EndTime timestamp is current time, to ensure that the timestamp in the stage be finished is equal with EndTime timestamp simultaneously.Until whole treatment scheme terminates.
In another in optional embodiment, the treatment state recording multiple phase process comprises: according to the implementation of multiple phase process in number record table successively recording processing state be the numbering of the phase process run succeeded, wherein, multiple phase process is previously provided with and numbers one to one with multiple phase process.Judge that whether there is the failed state of process in treatment state comprises: the phase process judging whether to exist non-record number from number record table; If judge the phase process that there is non-record number, then determine that the state of the phase process of non-record number is for process unsuccessfully.
Be provided with the numbering corresponding with it to each phase process, in the process performing multiple phase process, each phase process is complete, if this phase process process success, then in number record table, records the numbering of this phase process; Otherwise, if process unsuccessfully, then the number of not being on the permanent staff record sheet records its numbering.Like this, when judging whether to exist in treatment state the failed state of process, then whether can record numbering to judge according at number record table, if do not have record, then corresponding phase process process unsuccessfully.
Further, if there is the failed phase process of process, then from number record table, determine the phase process of first non-record number, from this phase process, re-execute data processing.
The embodiment of the present invention additionally provides a kind of data processing equipment.This device can realize its function by computer equipment.It should be noted that, the data processing equipment of the embodiment of the present invention may be used for performing the data processing method that the embodiment of the present invention provides, and the data processing equipment that the data processing method of the embodiment of the present invention also can be provided by the embodiment of the present invention performs.
Fig. 2 is the schematic diagram of the data processing equipment according to the embodiment of the present invention.As shown in the figure, this data processing equipment comprises: acquiring unit 10, first performance element 20, record cell 30, judging unit 40 and the second performance element 50.
Acquiring unit 10 is for obtaining pending data.
Pending data can be need the data through multiple phase process, such as daily record data, in storing daily record data, first can carry out classification and the polymerization of daily record data, then carry out multiple stage such as storing.
First performance element 20 for performing multiple phase process successively to described pending data, described multiple phase process be to described pending data perform continuously successively processing stage.
Above-mentioned multiple phase process is all for the treatment of pending data, wherein, be associated successively between multiple phase process, namely the input data of each phase process are the result of a upper phase process, that is to say that a rear phase process needs to use the result of a phase process.
Record cell 30 is for recording the treatment state of described multiple phase process, and described treatment state comprises and processes successfully and process unsuccessfully.
Performing in the process of multiple phase process successively to pending data, record the treatment state of each phase process successively.Particularly, the timestamp of each phase process can be recorded, be determined the treatment state of each phase process by timestamp.Also can be utilize signature identification to record the treatment state of multiple phase process, after each phase process completes, if process successfully, signature identification becomes the first mark, otherwise, then become the second mark, like this, just can by identifying that each signature identification determines whether the phase process of its correspondence processes successfully or failure.
Whether judging unit 40 exists the failed state of described process for judging in described treatment state.
If the second performance element 50 for judging that described treatment state exists the failed state of described process, then re-executes the phase process that the failed state of described process is corresponding.
From the treatment state of record, judge whether to process failed state, namely judge in multiple phase process, whether to there is the process failed stage, if had, then re-execute the phase process that process is failed.Due to multiple phase process be perform successively processing stage, so, when one of them phase process failure, its follow-up phase process all processes failure usually, therefore, this phase process and later phase process thereof all re-execute, and the phase process before this phase process is without the need to re-executing, thus improve the efficiency of data processing.
According to the embodiment of the present invention, by recording the treatment state of multiple phase process, judge in treatment state, whether to there is the failed state of process, if judge that treatment state exists the failed state of process, then re-execute the phase process that the failed state of process is corresponding, for the successful phase process of process, then without the need to again processing, like this, again above-mentioned multiple phase process is performed to pending data when avoiding abnormal in data processing, solve the inefficient problem of data processing in prior art, reach the effect of the efficiency improving data processing.
Preferably, data processing equipment also comprises: determining unit, for after judging that treatment state exists the failed state of process, and before re-executing phase process corresponding to the failed state of process, the state performing the process failure that first time in multiple level processes occurs is determined from treatment state, second performance element comprises: the first determination module, for determining the phase process that the state of the process failure that first time occurs is corresponding from multiple phase process; First execution module, for re-executing the phase process determined and the later phase process of the phase process determined.
Because multiple phase process successively order performs, wherein, phase process needs the result using last phase process next time.When judging that the existence of multiple phase process processes failed state, from treatment state, then determine that the state that process is failed appears in first time, particularly, the state of each phase process can be judged from front to back successively according to the order of multiple phase process, when first time is judged to process failed state, then be there is the state that process is failed in this state first time in treatment state; Also can judge the state of each phase process according to the order of multiple phase process from back to front successively, when first time is judged to process successful state, then the state that process is failed be appearred in the front result once judged first time in treatment state.
After determining the state that first time appearance process is failed, determine the phase process that this state is corresponding to re-execute data processing from this phase process determined.
Consist of example with multiple phase process by two phase process to be described, wherein, two phase process comprise first stage process and subordinate phase process, first stage process and subordinate phase be treated to pending data perform continuously successively processing stage, wherein, perform multiple phase process successively to pending data to comprise: perform first stage process successively to pending data, obtain the first result; Subordinate phase process is performed to the first result, obtains the second result.In the process of execute phase process, record the first state and the second state, the first state is the treatment state of first stage process, and the second state is the treatment state of subordinate phase process.
If the first state and the second state are the successful state of process, then the second result is then final result.If the first state is the state that process is failed, then re-execute first stage process.Certainly, because first stage process is before subordinate phase process, when the first stage processes unsuccessfully, subordinate phase process also processes failure usually, therefore, when the first state is when processing failed state, then need to re-execute first stage process and subordinate phase process.Be when processing failed state, then only need re-execute subordinate phase process when only there being the second state.
In the embodiment of the present invention, after can judging the treatment state of phase process from front to back successively, then determining the phase process that re-executes, also can be the treatment state from judging forward each phase process processing stage of last successively.
The embodiment of the present invention preferably judges the treatment state of each phase process from back to front successively.Particularly, when multiple phase process comprises first stage process and subordinate phase process, first judge whether the second state is the state that process is failed, if so, then continue to judge whether the first state is the state that process is failed; Otherwise, then think that first stage process and subordinate phase process all process successfully.
According to the embodiment of the present invention, by judging the treatment state of each phase process successively from back to front, when judging that treatment state is the successful state of process, then without the need to judging again, and determine the phase process failure after the success of pending data processing or this process success status.
It should be noted that, in the embodiment of the present invention, multiple phase process can also comprise phase III process, fourth stage process etc., and its principle, with first stage process and subordinate phase process, does not repeat here.
Preferably, data processing equipment also comprises: buffer unit, for after performing multiple phase process successively to described pending data, the intermediate processing results of multiple phase process described in buffer memory, wherein, described second performance element 50 comprises: acquisition module, the result of a upper phase process of the phase process that the state that the described process for obtaining buffer memory is failed is corresponding; Second execution module, for performing phase process corresponding to the failed state of described process to the result obtained.
After each phase process completes, by the intermediate result of each for buffer memory phase process, like this, when determining to there is the process failed stage in multiple phase process, and when re-executing the phase process of this process failure, last result can be obtained from the intermediate result of buffer memory, carry out subsequent treatment, ensure that the correctness of the phase process re-executed.
Preferably, described record cell comprises: the first logging modle, for recording the timestamp of described multiple phase process, wherein, executing in described multiple phase process, to upgrade timestamp corresponding to the phase process that executes after each phase process be Preset Time stamp, described judging unit comprises: the first judge module, for judging whether the timestamp of described multiple phase process is described Preset Time stamp successively; Second determination module, if for judging that the timestamp of described multiple phase process is not described Preset Time stamp, then determines that timestamp is not that the state of the phase process of described Preset Time stamp is for process unsuccessfully.
The treatment state of each phase process is recorded by logging timestamp.After each phase process is complete, if process successfully, then the timestamp in this stage is changed to Preset Time stamp; If process unsuccessfully, then timestamp does not change, like this, judge each phase process whether success or failed time, by identifying that timestamp corresponding to each stage judges, the accuracy for the treatment of state identification can be improve.
In another in optional embodiment, record cell comprises: the second logging modle, for the implementation according to multiple phase process in number record table successively recording processing state be the numbering of the phase process run succeeded, wherein, multiple phase process is previously provided with numbers one to one with multiple phase process, judging unit comprises: the second judge module, for judging whether the phase process that there is non-record number from number record table; 3rd determination module, if for judging the phase process that there is non-record number, then determines that the state of the phase process of non-record number is for process unsuccessfully.
Be provided with the numbering corresponding with it to each phase process, in the process performing multiple phase process, each phase process is complete, if this phase process process success, then in number record table, records the numbering of this phase process; Otherwise, if process unsuccessfully, then the number of not being on the permanent staff record sheet records its numbering.Like this, when judging whether to exist in treatment state the failed state of process, then whether can record numbering to judge according at number record table, if do not have record, then corresponding phase process process unsuccessfully.
Further, if there is the failed phase process of process, then from number record table, determine the phase process of first non-record number, from this phase process, re-execute data processing.
It should be noted that, for aforesaid each embodiment of the method, in order to simple description, therefore it is all expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not by the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in instructions all belongs to preferred embodiment, and involved action and module might not be that the present invention is necessary.
In the above-described embodiments, the description of each embodiment is all emphasized particularly on different fields, in certain embodiment, there is no the part described in detail, can see the associated description of other embodiments.
In several embodiments that the application provides, should be understood that, disclosed device, the mode by other realizes.Such as, device embodiment described above is only schematic, the such as division of described unit, be only a kind of logic function to divide, actual can have other dividing mode when realizing, such as multiple unit or assembly can in conjunction with or another system can be integrated into, or some features can be ignored, or do not perform.Another point, shown or discussed coupling each other or direct-coupling or communication connection can be by some interfaces, and the indirect coupling of device or unit or communication connection can be electrical or other form.
The described unit illustrated as separating component or can may not be and physically separates, and the parts as unit display can be or may not be physical location, namely can be positioned at a place, or also can be distributed in multiple network element.Some or all of unit wherein can be selected according to the actual needs to realize the object of the present embodiment scheme.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, also can be that the independent physics of unit exists, also can two or more unit in a unit integrated.Above-mentioned integrated unit both can adopt the form of hardware to realize, and the form of SFU software functional unit also can be adopted to realize.
If described integrated unit using the form of SFU software functional unit realize and as independently production marketing or use time, can be stored in a computer read/write memory medium.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words or all or part of of this technical scheme can embody with the form of software product, this computer software product is stored in a storage medium, comprises all or part of step of some instructions in order to make a computer equipment (can be personal computer, mobile terminal, server or the network equipment etc.) perform method described in each embodiment of the present invention.And aforesaid storage medium comprises: USB flash disk, ROM (read-only memory) (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), portable hard drive, magnetic disc or CD etc. various can be program code stored medium.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1. a data processing method, is characterized in that, comprising:
Obtain pending data;
Multiple phase process is performed successively to described pending data, described multiple phase process be to described pending data perform continuously successively processing stage;
Record the treatment state of described multiple phase process, described treatment state comprises and processes successfully and process unsuccessfully;
Judge in described treatment state, whether to there is the failed state of described process; And
If judge that described treatment state exists the failed state of described process, then re-execute the phase process that the failed state of described process is corresponding.
2. data processing method according to claim 1, is characterized in that,
After judging that described treatment state exists the failed state of described process, and before re-executing phase process corresponding to the failed state of described process, described data processing method also comprises: determine from described treatment state and perform the state that in described multiple level processes, the described process of first time appearance is failed
The phase process re-executing the failed state of described process corresponding comprises: that from described multiple phase process, determines that described first time occurs describedly processes phase process corresponding to failed state; Re-execute the phase process determined and the later phase process of the described phase process determined.
3. data processing method according to claim 1 and 2, is characterized in that, after performing multiple phase process successively to described pending data, described data processing method also comprises:
The intermediate processing results of multiple phase process described in buffer memory,
Wherein, the phase process re-executing the failed state of described process corresponding comprises: the described result processing a upper phase process of phase process corresponding to failed state obtaining buffer memory; Phase process corresponding to the failed state of described process is performed to the result obtained.
4. data processing method according to claim 1 and 2, is characterized in that,
The treatment state recording described multiple phase process comprises: the timestamp recording described multiple phase process, and wherein, to upgrade timestamp corresponding to the phase process that executes after each phase process be Preset Time stamp executing in described multiple phase process,
Judge whether to exist in described treatment state the failed state of described process to comprise: whether the timestamp judging described multiple phase process is successively that described Preset Time stabs; If judge that the timestamp of described multiple phase process is not described Preset Time stamp, then determine that timestamp is not that the state of the phase process of described Preset Time stamp is for process unsuccessfully.
5. data processing method according to claim 1 and 2, is characterized in that,
The treatment state recording described multiple phase process comprises: according to the implementation of described multiple phase process in number record table successively recording processing state be the numbering of the phase process run succeeded, wherein, described multiple phase process is previously provided with numbers one to one with described multiple phase process
Judge that whether there is the failed state of described process in described treatment state comprises: the phase process judging whether to exist non-record number from described number record table; If judge the phase process that there is non-record number, then determine that the state of the phase process of non-record number is for process unsuccessfully.
6. a data processing equipment, is characterized in that, comprising:
Acquiring unit, for obtaining pending data;
First performance element, for performing multiple phase process successively to described pending data, described multiple phase process be to described pending data perform continuously successively processing stage;
Record cell, for recording the treatment state of described multiple phase process, described treatment state comprises and processes successfully and process unsuccessfully;
Whether judging unit, exist the failed state of described process for judging in described treatment state; And
Second performance element, if for judging that described treatment state exists the failed state of described process, then re-execute the phase process that the failed state of described process is corresponding.
7. data processing equipment according to claim 6, is characterized in that,
Described data processing equipment also comprises: determining unit, for after judging that described treatment state exists the failed state of described process, and before re-executing phase process corresponding to the failed state of described process, determine from described treatment state and perform the state that in described multiple level processes, the described process of first time appearance is failed
Described second performance element comprises: the first determination module, for determining the phase process that state that described process that described first time occurs is failed is corresponding from described multiple phase process; First execution module, for re-executing the phase process determined and the later phase process of the described phase process determined.
8. the data processing equipment according to claim 6 or 7, is characterized in that, described data processing equipment also comprises:
Buffer unit, for after performing multiple phase process successively to described pending data, the intermediate processing results of multiple phase process described in buffer memory,
Wherein, described second performance element comprises: acquisition module, the result of a upper phase process of the phase process that the state that the described process for obtaining buffer memory is failed is corresponding; Second execution module, for performing phase process corresponding to the failed state of described process to the result obtained.
9. the data processing equipment according to claim 6 or 7, is characterized in that,
Described record cell comprises: the first logging modle, and for recording the timestamp of described multiple phase process, wherein, to upgrade timestamp corresponding to the phase process that executes after each phase process be Preset Time stamp executing in described multiple phase process,
Described judging unit comprises: the first judge module, for judging whether the timestamp of described multiple phase process is described Preset Time stamp successively; Second determination module, if for judging that the timestamp of described multiple phase process is not described Preset Time stamp, then determines that timestamp is not that the state of the phase process of described Preset Time stamp is for process unsuccessfully.
10. the data processing equipment according to claim 6 or 7, is characterized in that,
Described record cell comprises: the second logging modle, for the implementation according to described multiple phase process in number record table successively recording processing state be the numbering of the phase process run succeeded, wherein, described multiple phase process is previously provided with numbers one to one with described multiple phase process
Described judging unit comprises: the second judge module, for judging whether the phase process that there is non-record number from described number record table; 3rd determination module, if for judging the phase process that there is non-record number, then determines that the state of the phase process of non-record number is for process unsuccessfully.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410648492.5A CN104298570B (en) | 2014-11-14 | 2014-11-14 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410648492.5A CN104298570B (en) | 2014-11-14 | 2014-11-14 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104298570A true CN104298570A (en) | 2015-01-21 |
| CN104298570B CN104298570B (en) | 2018-04-06 |
Family
ID=52318306
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410648492.5A Active CN104298570B (en) | 2014-11-14 | 2014-11-14 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104298570B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105511975A (en) * | 2015-11-27 | 2016-04-20 | 北京奇虎科技有限公司 | Exception handling method and device |
| CN107729207A (en) * | 2017-09-15 | 2018-02-23 | 青岛海信电器股份有限公司 | A kind of monitoring method and device of application program main thread |
| CN108021486A (en) * | 2017-11-21 | 2018-05-11 | 平安科技（深圳）有限公司 | Electronic device, the method and storage medium of collage-credit data processing |
| CN109408286A (en) * | 2018-09-17 | 2019-03-01 | 北京京东金融科技控股有限公司 | Data processing method, device, system, computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1740983A (en) * | 2004-08-25 | 2006-03-01 | 埃沃列姆公司 | Process management system |
| CN1977509A (en) * | 2004-06-29 | 2007-06-06 | 西门子公司 | Method of selecting one server out of a server set |
| CN101620549A (en) * | 2009-08-13 | 2010-01-06 | 恒生电子股份有限公司 | Performance analysis method and device |
| CN102750192A (en) * | 2012-06-06 | 2012-10-24 | 杭州华三通信技术有限公司 | Method and device for data plane anomaly detection |
| CN103678051A (en) * | 2013-11-18 | 2014-03-26 | 航天恒星科技有限公司 | On-line fault tolerance method in cluster data processing system |
| US20140195856A1 (en) * | 2013-01-07 | 2014-07-10 | Fujitsu Limited | Analysis method, analysis apparatus, and computer-readable recording medium storing analysis program |
-
2014
- 2014-11-14 CN CN201410648492.5A patent/CN104298570B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1977509A (en) * | 2004-06-29 | 2007-06-06 | 西门子公司 | Method of selecting one server out of a server set |
| CN1740983A (en) * | 2004-08-25 | 2006-03-01 | 埃沃列姆公司 | Process management system |
| CN101620549A (en) * | 2009-08-13 | 2010-01-06 | 恒生电子股份有限公司 | Performance analysis method and device |
| CN102750192A (en) * | 2012-06-06 | 2012-10-24 | 杭州华三通信技术有限公司 | Method and device for data plane anomaly detection |
| US20140195856A1 (en) * | 2013-01-07 | 2014-07-10 | Fujitsu Limited | Analysis method, analysis apparatus, and computer-readable recording medium storing analysis program |
| CN103678051A (en) * | 2013-11-18 | 2014-03-26 | 航天恒星科技有限公司 | On-line fault tolerance method in cluster data processing system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105511975A (en) * | 2015-11-27 | 2016-04-20 | 北京奇虎科技有限公司 | Exception handling method and device |
| CN105511975B (en) * | 2015-11-27 | 2019-06-04 | 北京奇虎科技有限公司 | An exception handling method and device |
| CN107729207A (en) * | 2017-09-15 | 2018-02-23 | 青岛海信电器股份有限公司 | A kind of monitoring method and device of application program main thread |
| CN107729207B (en) * | 2017-09-15 | 2021-01-01 | 海信视像科技股份有限公司 | Application program main thread monitoring method and device |
| CN108021486A (en) * | 2017-11-21 | 2018-05-11 | 平安科技（深圳）有限公司 | Electronic device, the method and storage medium of collage-credit data processing |
| CN109408286A (en) * | 2018-09-17 | 2019-03-01 | 北京京东金融科技控股有限公司 | Data processing method, device, system, computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104298570B (en) | 2018-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3678346A1 (en) | Blockchain smart contract verification method and apparatus, and storage medium | |
| CN109885614B (en) | Data synchronization method and device | |
| US10997733B2 (en) | Rigid-body configuration method, apparatus, terminal device, and computer readable storage medium | |
| CN104298570A (en) | Data processing method and device | |
| CN103714086A (en) | Method and device used for generating non-relational data base module | |
| CN104463141A (en) | Fingerprint template input method and device based on intelligent card | |
| CN104317850A (en) | Data processing method and device | |
| CN104298736A (en) | Method and device for aggregating and connecting data as well as database system | |
| CN112199935A (en) | Data comparison method and device, electronic equipment and computer readable storage medium | |
| EP3108400B1 (en) | Virus signature matching method and apparatus | |
| JP2019508774A (en) | Database system optimization method, system, electronic device and storage medium | |
| CN102486748B (en) | Method and device for performance test | |
| CN106874322A (en) | A kind of data table correlation method and device | |
| CN106933919B (en) | Data table connection method and device | |
| CN105243042A (en) | Data burning method and data burning device | |
| CN105446982A (en) | Data storage system management method and device | |
| CN104391927A (en) | Method for realizing dimensionality reconstruction of multidimensional data model | |
| CN103970792A (en) | Index-based file comparison method and device | |
| CN110347683B (en) | Data table merging processing method and device | |
| CN106686025A (en) | Data collection method and apparatus thereof | |
| CN106203753A (en) | The circulation method and system of problems of operation in software operation | |
| CN104268277A (en) | Data reading method and device for database | |
| CN106803202B (en) | Method and device for extracting transaction records to be tested | |
| CN113077344B (en) | Block chain-based transaction method, device, electronic equipment and storage medium | |
| CN104991963B (en) | Document handling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Parallel processed data prcessing method and apparatus thereof Effective date of registration: 20190531 Granted publication date: 20180406 Pledgee: Shenzhen Black Horse World Investment Consulting Co.,Ltd. Pledgor: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. Registration number: 2019990000503 |
|
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Patentee after: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Patentee before: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. |
|
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20240604 Granted publication date: 20180406 |