CN109299096A

CN109299096A - A kind of processing method of pipelined data, device and equipment

Info

Publication number: CN109299096A
Application number: CN201811116228.1A
Authority: CN
Inventors: 赵阳阳
Original assignee: Alibaba Group Holding Ltd
Current assignee: Advanced New Technologies Co Ltd; Advantageous New Technologies Co Ltd
Priority date: 2018-09-25
Filing date: 2018-09-25
Publication date: 2019-02-01

Abstract

This specification embodiment discloses processing method, device and the equipment of a kind of pipelined data, which comprises when detecting that target pipelined data generates, obtains the business dimension attribute of the corresponding target service of the target pipelined data；Based on the business dimension attribute, by target pipelined data storage into the target data grouping at least one corresponding data grouping of the business dimension attribute.

Description

A kind of processing method of pipelined data, device and equipment

Technical field

This specification is related to field of computer technology more particularly to a kind of processing method of pipelined data, device and equipment.

Background technique

With the continuous development of network finance and network payment, network finance and the corresponding portfolio of network payment and day are all Increase, in this way will generate a large amount of pipelined data.And how a large amount of pipelined datas of generation store becoming and need to solve Major issue certainly.

In general, the related system of the chains such as network finance and network payment road often uses a large amount of pipelined datas of generation Point library divides the mode of table to land storage, and is related in the relevant operation of pipelined data, and storage can be landed by the way of dividing table. But when the portfolio of certain business is larger, using table mode is divided, the pipelined data of the business of certain specified dimension Data volume it is still huge, in this way, often being led when needing to carry out the operation such as relevant business diagnosis processing to pipelined data It causes big affairs problem to occur, and then easily causes the shake of database, and business scenario higher for timeliness can not expire Sufficient actual demand, for this reason, it may be necessary to which providing the flowing water grouping scheme that one kind can be real-time, adaptive is grouped place to pipelined data Reason.

Summary of the invention

The purpose of this specification embodiment is to provide processing method, device and the equipment of a kind of pipelined data, to provide one Flowing water grouping scheme that kind can be real-time, adaptive is grouped processing to pipelined data.

To realize that above-mentioned technical proposal, this specification embodiment are achieved in that

A kind of processing method for pipelined data that this specification embodiment provides, which comprises

When detecting that target pipelined data generates, the business dimension of the corresponding target service of the target pipelined data is obtained Spend attribute；

Based on the business dimension attribute, the target pipelined data is stored corresponding extremely to the business dimension attribute In target data grouping in a few data grouping.

Optionally, described to be based on the business dimension attribute, by target pipelined data storage to the business dimension In target data grouping at least one corresponding data grouping of attribute, comprising:

If including the business dimension attribute in database, current time and the target pipelined data are obtained Grouping information；

According to the grouping information of the target pipelined data, the time and the corresponding storage of the target pipelined data are arrived In the target data grouping.

If in database not including the business dimension attribute, the corresponding data point of the business dimension attribute are created Group includes that target data is grouped in the data grouping of creation；

The current time is obtained, and the time and the corresponding storage of the target pipelined data are arrived to the institute of the database It states in target data grouping.

Optionally, the method also includes:

Receive the pipelined data processing request of the target service；

According to the business dimension attribute of the target service, obtain and the business dimension attribute phase from the database At least one corresponding data grouping；

According to the initial time of the pipelined data stored at least one described data grouping and deadline, respectively to institute The pipelined data stored at least one data grouping is stated to be handled.

Optionally, the flowing water number of predetermined quantity can be stored in the corresponding each data grouping of the business dimension attribute According to.

Optionally, the grouping information is that the flowing water of the target pipelined data identifies, and the flowing water mark includes flowing water Number, transaction one or more of odd numbers, alternatively, the grouping information is to be based on business dimension with the target pipelined data Spend the relevant information of storage order under attribute.

A kind of processing unit for pipelined data that this specification embodiment provides, described device include:

Dimensional attribute obtains module, for obtaining the target pipelined data when detecting that target pipelined data generates The business dimension attribute of corresponding target service；

Flowing water memory module, for being based on the business dimension attribute, by target pipelined data storage to the industry It is engaged in the target data grouping at least one corresponding data grouping of dimensional attribute.

Optionally, the flowing water memory module, comprising:

Information acquisition unit, if for including the business dimension attribute in database, obtain current time and The grouping information of the target pipelined data；

Flowing water storing unit, for the grouping information according to the target pipelined data, by the time and the target The corresponding storage of pipelined data is into target data grouping.

Optionally, the flowing water memory module, comprising:

It is grouped creating unit, if creating the business dimension for not including the business dimension attribute in database The corresponding data grouping of attribute is spent, includes that target data is grouped in the data grouping of creation；

Flowing water storing unit is deposited for obtaining the current time, and by the time and target pipelined data correspondence It stores up in the target data grouping of the database.

Optionally, described device further include:

Request receiving module, the pipelined data for receiving the target service handle request；

Be grouped determining module, for the business dimension attribute according to the target service, obtain from the database with At least one corresponding data grouping of the business dimension attribute；

Processing module, for the initial time and cut-off according to the pipelined data stored at least one described data grouping Time is respectively handled the pipelined data stored at least one described data grouping.

A kind of processing equipment for pipelined data that this specification embodiment provides, the processing equipment packet of the pipelined data It includes:

Processor；And

It is arranged to the memory of storage computer executable instructions, the executable instruction makes the place when executed Manage device:

By the technical solution of above this specification embodiment offer as it can be seen that this specification embodiment is by detecting target When pipelined data generates, the business dimension attribute of the corresponding target service of target pipelined data is obtained, then, is based on target service Business dimension attribute, by business dimension attribute corresponding at least one data point of target pipelined data storage to target service In target data grouping in group, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates With according to the business dimension attribute of the corresponding target service of target pipelined data, by the storage of target pipelined data to accordingly at least In a data grouping in one data grouping, thereby may be ensured that pipelined data processing real-time, meet timeliness compared with The demand of high business scenario.It, therefore, can be with moreover, because pipelined data is stored in a manner of an at least data grouping To be subsequent in the pipelined data of processing target business, pipelined data therein can be fished for as unit of data grouping, and by The limited amount of pipelined data in each data grouping, and quantity will not be excessive, this way it is possible to avoid in issued transaction mistake Occurs big affairs problem in journey.

Detailed description of the invention

In order to illustrate more clearly of this specification embodiment or technical solution in the prior art, below will to embodiment or Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only The some embodiments recorded in this specification, for those of ordinary skill in the art, in not making the creative labor property Under the premise of, it is also possible to obtain other drawings based on these drawings.

Fig. 1 is a kind of processing method embodiment of pipelined data of this specification；

Fig. 2 is a kind of structural schematic diagram of the processing system of pipelined data of this specification；

Fig. 3 is the processing method embodiment of this specification another kind pipelined data；

Fig. 4 is a kind of processing device embodiment of pipelined data of this specification；

Fig. 5 is a kind of processing equipment embodiment of pipelined data of this specification.

Specific embodiment

This specification embodiment provides processing method, device and the equipment of a kind of pipelined data.

In order to make those skilled in the art more fully understand the technical solution in this specification, below in conjunction with this explanation Attached drawing in book embodiment is clearly and completely described the technical solution in this specification embodiment, it is clear that described Embodiment be only this specification a part of the embodiment, instead of all the embodiments.The embodiment of base in this manual, Every other embodiment obtained by those of ordinary skill in the art without making creative efforts, all should belong to The range of this specification protection.

Embodiment one

As shown in Figure 1, this specification embodiment provides a kind of processing method of pipelined data, the executing subject of this method can Think terminal device or server, wherein the terminal device can such as personal computer equipment, can also be such as mobile phone, plate The mobile terminal devices such as computer, the terminal device can be the terminal device that user uses.The server can be independent clothes Business device, is also possible to the server cluster being made of multiple servers, moreover, the server can be the backstage clothes of a certain business Business device is also possible to the background server etc. of certain website (such as websites or payment application).This method can be used for In real time, adaptively processing etc. is grouped to pipelined data, in order to improve the treatment effeciency of pipelined data, in the present embodiment with Executing subject is to be illustrated for server, the case where for terminal device, can be handled according to following related contents, herein It repeats no more.This method can specifically include following steps:

In step s 102, when detecting that target pipelined data generates, the corresponding target industry of target pipelined data is obtained The business dimension attribute of business.

Wherein, target pipelined data can be the pipelined data of any business, such as the transaction flow water number in payment transaction According in, financial business for pipelined data of reconciliation etc..Different target services can be corresponding with different pipelined datas.Business Dimensional attribute can be the attribute that a dimensional information for being included based on target service is constituted, for example, target service includes machine One dimensional information of structure time, then its business dimension attribute can be authority time, alternatively, being also possible to based on target service institute The attribute that the multiple dimensional informations for including are constituted, for example, target service includes two dimensions such as authority time and accounting authority identification Information is spent, then its business dimension attribute can be authority time and accounting authority identification, in practical applications, the business dimension category Property can be and be arranged by way of such as above-mentioned text information, further, it is also possible to corresponding code is arranged in each dimensional information, The dimensional information can also be arranged by way of above-mentioned code, and specifically such as, authority time 20180917 calculates authority identification It is 3009, then the business dimension attribute can be 20180917/3009 etc..

In an implementation, with the continuous development of network finance and network payment, network finance and the corresponding industry of network payment Business amount is growing day by day, in this way will generate a large amount of pipelined data.And how a large amount of pipelined datas of generation to be stored The major issue solved as needs.

In general, the related system of the chains such as network finance and network payment road often uses a large amount of pipelined datas of generation Point library divides the mode of table to land storage, and is related in the relevant operation of pipelined data, and storage can be landed by the way of dividing table. But when the portfolio of certain business is larger, using table mode is divided, the pipelined data of the business of certain specified dimension Data volume it is still huge, in this way, often being led when needing to carry out the operation such as relevant business diagnosis processing to pipelined data It causes big affairs problem to occur, and then easily causes the shake of database.Such as reconciliation centring system to corresponding pipelined data into When row collation process, the single settlement day of specified services, there are a large amount of pipelined datas, after carrying out inner part list processing, based on single The operations such as the pipelined data verification of tables of data, filing, editing and processing will appear the big affairs problem of database, specifically such as, specify industry It is engaged under single settlement day, includes 200,000,000 pipelined datas in transaction details, carry out a point list processing, be stored to 100 tables of data In after, the quantity of the pipelined data for including under individual data table is 2,000,000, in the batch pipelined data core for carrying out individual data table Pair process in, single affairs submission amount that network finance and network payment side, mechanism side handle pipelined data 4,000,000 with On, when carrying out batch filing to the pipelined data in individual data table, the affairs submission amount being related to is 8,000,000 or more, due to thing Submission amount of being engaged in is larger, causes corresponding affairs larger, database can be caused to shake when submitting affairs, influence regular traffic on line.

It to solve the above-mentioned problems, can be by using timed task to the pipelined data generated at " particular point in time " Mark grouping is carried out, for example, can combine according to the dimension of corresponding service, meets what the dimension combined under each tables of data Pipelined data is grouped mark processing.When needing using pipelined data, data dimension and grouping convection current water number can be passed through According to being handled, in this manner it is ensured that the quantity that affairs are submitted when service operation for single time is relatively low, so as to solve major issue Business problem, still, aforesaid way business scenario higher for timeliness are unable to satisfy actual demand, for this purpose, it is also required to provide A kind of real-time, adaptive flowing water group technology is grouped processing to pipelined data, can specifically include the following contents:

As shown in Fig. 2, server can be based at scheduled business processing mechanism whenever executing a certain business Reason, can produce pipelined data (i.e. target pipelined data) during business processing, whenever the new target stream water number of generation According to when, can send a notification message to the server or request message, to notify the server to there is new pipelined data to generate, this When, the available target pipelined data of the server.For example, when user is transferred accounts using payment application to certain account, user The application program that can star payment application finds key of transferring accounts in the page that payment application provides, and user can be with The key of transferring accounts is clicked, payment application can obtain corresponding transfer page, Yong Huke from the background server of payment application To input the information of transfer amounts and the account in transfer page, payment application generates corresponding trading card, and after being sent to Platform server, background server can carry out processing of transferring accounts based on the trading card, in above-mentioned treatment process, whenever payment is applied When interacting with background server, corresponding pipelined data can produce, such as payment application obtains phase from background server When the transfer page answered, corresponding pipelined data can produce, trading card is sent to background server, Yi Jihou by payment application Platform server is based on the trading card and carries out processing of transferring accounts, and can produce corresponding pipelined data.Whenever the new flowing water number of generation According to when, can send a notification message to the server or request message, at this point, the available target pipelined data of the server. In this way, server can be the pipelined data for getting generation in real time.

It should be noted that server can be made to perceive above by sending a notification message or the mode of request message Or detect that new pipelined data generates, in practical applications, however it is not limited to detect target pipelined data through the above way Generation, can also realize, specifically may be set according to actual conditions otherwise, this specification embodiment does not do this It limits.

It may include the service identification of the affiliated business of target pipelined data in target pipelined data, server gets target After pipelined data, service identification therein can be extracted from target pipelined data, target can be determined by the service identification Target service belonging to pipelined data.It may include one or more dimensional informations and each dimensional information in each business Corresponding data, the dimensional information for including in the available target service of server, can extract from the dimensional information of acquisition One or more dimensional informations can construct the business dimension attribute of target service based on the dimensional information of extraction, wherein for The dimensional information of acquisition includes multiple situation, and multiple dimensional informations that server can will acquire are combined, and constitutes target Business dimension attribute of business etc..

In step S104, it is based on above-mentioned business dimension attribute, the business dimension attribute is arrived into the storage of target pipelined data In target data grouping at least one corresponding data grouping.

In an implementation, one or more data groupings can be set for each business dimension attribute, and each data grouping can To accommodate the pipelined data of certain amount or certain data volume, such as each data grouping can accommodate 1000 pipelined datas, Alternatively, each data grouping can accommodate the pipelined data etc. of 10MB.Server gets the business dimension attribute of target service Afterwards, it can be searched whether in the database of storage pipelined data comprising the business dimension attribute, if comprising being somebody's turn to do in database Business dimension attribute then shows that the pipelined data of target service has been stored before current time, at this point it is possible to by target Into database in one data grouping of the business dimension attribute, specific storage mode can be with a variety of more for pipelined data storage Sample, for example, may include the information such as serial number in each pipelined data, different pipelined datas, serial number can be different, can To determine the data grouping where target pipelined data according to the serial number of target pipelined data, specifically such as, if each data Grouping can accommodate 1000 pipelined datas, and if the serial number of target pipelined data is 2089, then target pipelined data should the 3rd In a data grouping, i.e., for accommodate serial number be 2001~3000 data grouping (wherein, the 1st data grouping by Serial number be 1~1000 pipelined data occupied by, the 2nd data grouping by serial number be 1001~2000 flowing water number According to occupied).Other than it can realize through the above way, multiple data groupings can also be set simultaneously, then, respectively successively Store pipelined data etc. into each data grouping, specifically may be set according to actual conditions, this specification embodiment to this not It limits.

This specification embodiment provides a kind of processing method of pipelined data, by detecting the generation of target pipelined data When, obtain the business dimension attribute of the corresponding target service of target pipelined data, then, the business dimension category based on target service Property, by target pipelined data storage to the number of targets at least one corresponding data grouping of business dimension attribute of target service According in grouping, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates, it can according to target stream Water number stores target pipelined data at least one corresponding data grouping according to the business dimension attribute of corresponding target service In a data grouping in, thereby may be ensured that pipelined data processing real-time, meet the higher business scenario of timeliness Demand.Moreover, because pipelined data is stored in a manner of an at least data grouping, therefore, can handled to be subsequent When the pipelined data of target service, pipelined data therein can be fished for as unit of data grouping, and due to each data point The limited amount of pipelined data in group, and quantity will not be excessive, this way it is possible to avoid occurring major issue in transaction process Business problem.

Embodiment two

As shown in figure 3, this specification embodiment provides a kind of processing method of pipelined data, the executing subject of this method can Think terminal device or server, wherein the terminal device can such as personal computer equipment, can also be such as mobile phone, plate The mobile terminal devices such as computer, the terminal device can be the terminal device that user uses.The server can be independent clothes Business device, is also possible to the server cluster being made of multiple servers, moreover, the server can be the backstage clothes of a certain business Business device is also possible to the background server etc. of certain website (such as websites or payment application).This method can be used for In real time, adaptively processing etc. is grouped to pipelined data, in order to improve the treatment effeciency of pipelined data, in the present embodiment with Executing subject is to be illustrated for server, the case where for terminal device, can be handled according to following related contents, herein It repeats no more.This method can specifically include following steps:

In step s 302, when detecting that target pipelined data generates, the corresponding target industry of target pipelined data is obtained The business dimension attribute of business.

In an implementation, it when whether detection pipelined data generates, can be accomplished in several ways, such as when new flowing water number When according to generating, it can be notified by way of notification message, alternatively, can be led to by way of external request message Know, specifically may be set according to actual conditions, this specification embodiment does not limit this.Whenever detecting new flowing water When data (i.e. target pipelined data) generate, available target pipelined data, in order to which the target pipelined data is stored, And searched when facilitating subsequent use, storage can be grouped to pipelined data.In view of different business, the business Business dimension attribute is often different, therefore, can be using the business dimension attribute of business as the pipelined data of storage different business Foundation, for this purpose, target service belonging to target pipelined data can be determined, and the one or more of available target service Attribute dimensions.

It should be noted that the business dimension attribute of target service may include one, also may include it is multiple, if mesh The business dimension attribute of mark business include it is multiple, then multiple business dimension attributes can be combined, obtain dimension combination, will After multiple business dimension attributes are combined, it can be indicated by way of the text after multiple business dimension combinations of attributes, example It then can be time/operator etc. after multiple business dimension combinations of attributes if business dimension attribute includes time and operator, or Person can also be indicated by the corresponding numerical value of different business dimensional attribute (being properly termed as business dimension attribute value), such as the time It is 20180918, operator A11 can then pass through the group of each business dimension attribute value after multiple business dimension combinations of attributes Conjunction value indicates, specific such as 20180918/A11.

In step s 304, if including above-mentioned business dimension attribute in database, current time and target are obtained The grouping information of pipelined data.

Wherein, grouping information, which can be, is used to indicate the relevant information which data grouping target pipelined data belongs to, Grouping information may include a variety of, such as grouping information can identify for the flowing water of target stream water number evidence, and flowing water mark can be with Including one or more of serial number, transaction odd numbers.Trading odd numbers specifically can such as order number, Invoice Number.Data Library can be the database for storing pipelined data, and the pipelined data stored in database can be carried out with business dimension attribute It distinguishes.

In an implementation, due to different business, corresponding business dimension attribute is often different, therefore, can pass through industry Business dimensional attribute determines the pipelined data that certain business dimension attribute whether has been stored in data, is based on this, determines in server After the business dimension attribute of target service, it can be searched in the database by the business dimension attribute, if in the database Including the business dimension attribute, then show before current time, the pipelined data of target service stored in the database, And the corresponding relationship of the existing service attribute and pipelined data in database, at this point, server does not need to re-create this The corresponding relationship and corresponding data grouping of service attribute and pipelined data, it is only necessary to which target pipelined data is stored in number According in library.In order to accurately store target pipelined data, can be obtained according to the storage rule of preset pipelined data The relevant information of target pipelined data, the corresponding storage of the relevant information that can be will acquire, the service attribute and target pipelined data In the database.A kind of storage mode of pipelined data is provided in the present embodiment, can specifically include the following contents:

In view of pipelined data drop into the time in database for some application scenarios, for user or technical staff Be it is important, therefore, the available current time, current time can be converted to timestamp, and can be by this when Between timestamp field value of stamp when falling into database as target pipelined data, such as current time is on September 18th, 2018, then Timestamp is converted to, can be 20180918, correspondingly, timestamp field value can be 20180918.

It should be noted that current time can also be the time only comprising clock, such as above-mentioned only comprising the date 10:21:38 etc. such as on September 18,10:21:38 etc. 2018, specifically can be with alternatively, may include the time of date and clock It is set according to actual conditions, this specification embodiment does not limit this.

In addition, each data grouping can store since the corresponding data grouping of business dimension attribute may include multiple The limited amount of pipelined data therefore can be in order to which target pipelined data to be accurately grouped, in target pipelined data Including grouping information, for this purpose, including target pipelined data can also be obtained after above-mentioned business dimension attribute in determining database Grouping information.

It is according to the grouping information of target pipelined data, above-mentioned time and target pipelined data is corresponding in step S306 It stores in target data grouping.

Wherein, target data grouping can be at least one corresponding data grouping of business dimension attribute of target service A data grouping.The pipelined data of predetermined quantity can be stored in the corresponding each data grouping of business dimension attribute.In advance Fixed number amount may be set according to actual conditions, such as 1000 or 5000 etc..

In an implementation, the corresponding data grouping of business dimension attribute of business may include multiple, and multiple data groupings In each data grouping may include one or more pipelined datas, the pipelined data for including in multiple data groupings is fallen into The timestamp of database may be difference, therefore, fall into database to record the pipelined data of a certain business dimension attribute, can To use database to be handled from ID mechanism is increased, specifically, in database may include business dimension attribute-timestamp from Increase ID table, it can be as shown in table 1.

Table 1

ID (from increasing, initial value 0)	Timestamp
		0	20180917
1	20180918

Wherein, the ID in table 1 can be continuously increased with falling into for pipelined data, the numerical value of ID can since 0, according to Secondary increase, in the case, when ID is 0, then it represents that the pipelined data of record is the initial value of first data grouping, works as ID Numerical value when being the integer P times of above-mentioned predetermined quantity M, then the pipelined data of current record is located in the P data grouping.In reality In the application of border, the initial value of ID is not limited to since 0, can be since preset any number, for example, can be from 1000 start to be sequentially increased or reduce.Business dimension attribute-timestamp can be specified services dimensional attribute from ID table is increased Under timestamp from increasing ID record sheet, i.e., each specified services dimensional attribute have one such as above-mentioned business dimension attribute-when Between stab from increase ID table.Based on above content, above-mentioned grouping information can be to be based on above-mentioned business dimension with target pipelined data The relevant information of storage order under attribute, it can for the numerical value of above-mentioned ID or based on ID numerical value determined etc..

In addition, can recorde the business dimension of different business for the ease of checking or the business dimension attribute of inquiry business Attribute stores the flowing water number into data grouping since any business dimension attribute may include one or more data groupings According to time can be different, in this way, in different data grouping when can store the initial time and cut-off of pipelined data Between, it is based on this, can also include business business dimension attribute-grouped record table in database, it can be as shown in table 2.

Table 2

Wherein, the major key in table 2 may be set according to actual conditions, such as the above-mentioned serial number being arranged successively, alternatively, can be with Pass through the character representation etc. of other forms.

In addition, what is mainly stored in database is pipelined data, it therefore, in database can also include that pipelined data is practical The detail storage table (being referred to as flowing water business record table) for falling into database, can be as shown in table 3.

Table 3

Wherein, the major key in table 3 may be set according to actual conditions, such as the above-mentioned serial number being arranged successively, alternatively, can be with Pass through the character representation etc. of other forms.N can be the positive integer more than or equal to 1.

The timestamp that server can determine current time is stored into above-mentioned table 1, when due to increasing one in table 1 Between stab, then the numerical value of corresponding ID can increase automatically, to obtain the corresponding ID numerical value of target pipelined data, can based on To the numerical value of ID determine data grouping locating for target pipelined data (i.e. target data grouping) and the current institute of target service The quantity for the data grouping for including (concrete processing procedure may refer to above-mentioned related content, and details are not described herein).Then, may be used Target pipelined data to be stored in target data grouping, and the number can be determined based on the quantity of obtained data grouping According to the initial time stamp for the grouping for including in grouping and the deadline stamp of grouping, and obtained above-mentioned timestamp is recorded It states in table 2.Furthermore it is also possible to which the relevant information of target pipelined data is recorded in table 3.

By taking the business scenario of the detail reconciliation at the reconciliation center in financial business, data filing, flowing water editor carry out as an example, The business business dimension attribute of the business includes: the mechanism date, operator, currency type, calculates mechanism ID, then the date is for mechanism For 20180303, if including 20180303/sysOperator/156/3009 business business dimension attribute with include The business business dimension attribute of 20180303/sysOperator/840/3009, respectively there are two business business dimension attribute-when Between stab from increasing ID table (i.e. table 1), then the business business dimension category in business business dimension attribute-grouped record table (i.e. table 2) Property be 20180303/sysOperator/156/3009 and 20180303/sysOperator/840/3009 combination " two Class " traffic packets record and an actual services record sheet.

It should be noted that after the corresponding timestamp of current time is set in the database, if the ID in above-mentioned table 1 When numerical value is the integral multiple of predetermined quantity, the business dimension attribute is recorded in business dimension attribute-grouped record table (i.e. table 2) Deadline stamp of (or business dimension combinations of attributes), the initial time stamp of current group and current group etc..Complete target stream After the grouping of water data storage procedure, actual data grouping can pass through the grouped record table under the business dimension attribute of business Initial time (or initial time stamp) and deadline (or deadline stamp), corresponding data can be got and be grouped the stream for including The set of water number evidence.

In step S308, if in database not including above-mentioned business dimension attribute, the business dimension attribute is created Corresponding data grouping includes that target data is grouped in the data grouping of creation.

In an implementation, if by searching for determining the business dimension attribute for not including target service in database, then showing There are no the pipelined datas for storing target service in database, at this point it is possible to create the business dimension attribute in the database Corresponding data grouping.While server creates above-mentioned data grouping, it can also establish as shown in above-mentioned table 1, table 2 and table 3 Message store table, to record the pipelined data storage of target service to the relevant information during database.

In step s310, the current time is obtained, and the time and the corresponding storage of target pipelined data are arrived into database Target data grouping in.

The concrete processing procedure of above-mentioned steps S310 may refer to the related content of above-mentioned steps S306, no longer superfluous herein It states.

By above-mentioned processing can by the pipelined data generated in real time store into corresponding data grouping, when need using It, can be by following step S312~step S316 treatment process, from data grouping or when handling the pipelined data of certain business Middle taking-up pipelined data is handled, and specifically may refer to following related contents.

In step S312, the pipelined data processing request of target service is received.

Wherein, the service identification of target service can be carried in pipelined data processing request, which can be The title of target service or coding etc..

In step S314, according to the business dimension attribute of target service, obtained and the business dimension category from database At least one corresponding data grouping of property.

In an implementation, it after server gets pipelined data processing request, can be mentioned from pipelined data processing request Service identification is taken, corresponding target service can be determined by the service identification of extraction, it is then possible to obtain the one of target service A or multiple business dimension attributes, wherein if the business dimension attribute of target service include it is multiple, target industry can be assembled It is engaged in corresponding business business dimension attribute, so that multiple business business dimension attributes are combined (alternatively, can also be by multiple industry Business business dimension attribute value is combined).It can the fishing from above-mentioned business business dimension attribute-grouped record table (i.e. table 2) Take the corresponding grouped record set of business business dimension attribute of target service.

In step S316, according to the initial time and cut-off of the pipelined data stored at least one above-mentioned data grouping Time is respectively handled the pipelined data stored at least one data grouping.

In an implementation, server fishes for target service from above-mentioned business business dimension attribute-grouped record table (i.e. table 2) The corresponding grouped record set of business business dimension attribute after, each grouped record can be successively traversed, from above-mentioned such as table 3 According to initial time stamp (i.e. initial time) and deadline stamp (i.e. deadline) in flowing water business record table, fish for current The corresponding pipelined data of data grouping set.At this point, server can carry out the pipelined data under current data grouping Business procession.

It should be noted that be pipelined data in data grouping due to fishing for, and the flowing water in a data grouping The limited amount of data, and quantity will not be excessive, in this way, transaction process can avoid the occurrence of big affairs problem.

Based on above content, this specification embodiment combines database from ID technology is increased, under specified services operation " the business business dimension attribute of business " has corresponding ID to record respectively, and the time one when in conjunction with pipelined data storage is set to increasing It is long, that is, the timestamp generated the feature increasing when each pipelined data stores, when the timestamp of specified pipelined data is inserted What is generated after entering increases ID certainly, when ID value is the multiple of predetermined quantity, then it is assumed that and current pipelined data is the boundary of data grouping, when Front border is the record of current data grouping, the time in true flowing water business record table to the timestamp in previous boundary Stamp can corresponding with data grouping timestamp associate, be grouped in real time in pipelined data storing process to reach Purpose.In the treatment process of pipelined data storage, if finding not setting up currently " the business business dimension attribute of business " corresponding Timestamp record (i.e. from increasing ID) table, then can automatically create a data grouping, and flow current pipelined data as first Water number is according to storing into the data grouping, to achieve the purpose that adaptive grouping.

On the basis of this specification embodiment, it can also extend for different business business dimension combinations of attributes, no Same business business dimension attribute value combination etc., and the threshold value (i.e. predetermined quantity) of different data groupings is configured, to reach The purpose of more flexible adaptive grouping.

Flowing water under the present invention lands process, the grouping for completing flowing water in real time, adaptive.

Embodiment three

The above are the processing methods for the pipelined data that this specification embodiment provides, and are based on same thinking, this specification Embodiment also provides a kind of processing unit of pipelined data, as shown in Figure 4.

The processing unit of the pipelined data includes: that business dimension attribute obtains module 401 and flowing water memory module 402, In:

Business dimension attribute obtains module 401, for obtaining the target stream when detecting that target pipelined data generates Water number according to corresponding target service business dimension attribute；

Flowing water memory module 402 stores the target pipelined data to described for being based on the business dimension attribute In target data grouping at least one the corresponding data grouping of business dimension attribute.

In this specification embodiment, the flowing water memory module 402, comprising:

In this specification embodiment, described device further include:

In this specification embodiment, predetermined quantity can be stored in the corresponding each data grouping of the business dimension attribute Pipelined data.

In this specification embodiment, the grouping information is that the flowing water of the target pipelined data identifies, the flowing water mark Knowing includes one or more of serial number, transaction odd numbers, alternatively, the grouping information is to be based on the target pipelined data The relevant information of storage order under the business dimension attribute.

This specification embodiment provides a kind of processing unit of pipelined data, by detecting the generation of target pipelined data When, obtain the business dimension attribute of the corresponding target service of target pipelined data, then, the business dimension category based on target service Property, by target pipelined data storage to the number of targets at least one corresponding data grouping of business dimension attribute of target service According in grouping, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates, it can according to target stream Water number stores target pipelined data at least one corresponding data grouping according to the business dimension attribute of corresponding target service In a data grouping in, thereby may be ensured that pipelined data processing real-time, meet the higher business scenario of timeliness Demand.Moreover, because pipelined data is stored in a manner of an at least data grouping, therefore, can handled to be subsequent When the pipelined data of target service, pipelined data therein can be fished for as unit of data grouping, and due to each data point The limited amount of pipelined data in group, and quantity will not be excessive, this way it is possible to avoid occurring major issue in transaction process Business problem.

Example IV

The above are the processing units for the pipelined data that this specification embodiment provides, and are based on same thinking, this specification Embodiment also provides a kind of processing equipment of pipelined data, as shown in Figure 5.

The processing equipment of the pipelined data can be server provided by the above embodiment or terminal device.

The processing equipment of pipelined data can generate bigger difference because configuration or performance are different, may include one or More than one processor 501 and memory 502 can store one or more storages in memory 502 using journey Sequence or data.Wherein, memory 502 can be of short duration storage or persistent storage.The application program for being stored in memory 502 can be with Including one or more modules (diagram is not shown), each module may include one in the processing equipment to pipelined data Family computer executable instruction.Further, processor 501 can be set to communicate with memory 502, in pipelined data Processing equipment on execute memory 502 in series of computation machine executable instruction.The processing equipment of pipelined data can be with Including one or more power supplys 503, one or more wired or wireless network interfaces 504, one or more Input/output interface 505, one or more keyboards 506.

Specifically in the present embodiment, the processing equipment of pipelined data includes memory and one or more Program, perhaps more than one program is stored in memory and one or more than one program may include one for one of them A or more than one module, and each module may include that series of computation machine in processing equipment to pipelined data is executable Instruction, and be configured to execute this or more than one program by one or more than one processor to include for carrying out Following computer executable instructions:

It is described to be based on the business dimension attribute in this specification embodiment, by target pipelined data storage to institute It states in the target data grouping at least one the corresponding data grouping of business dimension attribute, comprising:

In this specification embodiment, further includes:

Receive the pipelined data processing request of the target service；

This specification embodiment provides a kind of processing equipment of pipelined data, by detecting the generation of target pipelined data When, obtain the business dimension attribute of the corresponding target service of target pipelined data, then, the business dimension category based on target service Property, by target pipelined data storage to the number of targets at least one corresponding data grouping of business dimension attribute of target service According in grouping, in this way, whenever detecting that new pipelined data (i.e. target pipelined data) generates, it can according to target stream Water number stores target pipelined data at least one corresponding data grouping according to the business dimension attribute of corresponding target service In a data grouping in, thereby may be ensured that pipelined data processing real-time, meet the higher business scenario of timeliness Demand.Moreover, because pipelined data is stored in a manner of an at least data grouping, therefore, can handled to be subsequent When the pipelined data of target service, pipelined data therein can be fished for as unit of data grouping, and due to each data point The limited amount of pipelined data in group, and quantity will not be excessive, this way it is possible to avoid occurring major issue in transaction process Business problem.

It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims It is interior.In some cases, the movement recorded in detail in the claims or step can be come according to the sequence being different from embodiment It executes and desired result still may be implemented.In addition, process depicted in the drawing not necessarily require show it is specific suitable Sequence or consecutive order are just able to achieve desired result.In some embodiments, multitasking and parallel processing be also can With or may be advantageous.

In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example, Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit. Designer nearly all obtains corresponding hardware circuit by the way that improved method flow to be programmed into hardware circuit.Cause This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device (Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, designs and makes without asking chip maker Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " is patrolled Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development, And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language (Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL (Advanced Boolean Expression Language)、AHDL(Altera Hardware Description Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL (Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby Hardware Description Language) etc., VHDL (Very-High-Speed is most generally used at present Integrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also answer This understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages, The hardware circuit for realizing the logical method process can be readily available.

Controller can be implemented in any suitable manner, for example, controller can take such as microprocessor or processing The computer for the computer readable program code (such as software or firmware) that device and storage can be executed by (micro-) processor can Read medium, logic gate, switch, specific integrated circuit (Application Specific Integrated Circuit, ASIC), the form of programmable logic controller (PLC) and insertion microcontroller, the example of controller includes but is not limited to following microcontroller Device: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320 are deposited Memory controller is also implemented as a part of the control logic of memory.It is also known in the art that in addition to Pure computer readable program code mode is realized other than controller, can be made completely by the way that method and step is carried out programming in logic Controller is obtained to come in fact in the form of logic gate, switch, specific integrated circuit, programmable logic controller (PLC) and insertion microcontroller etc. Existing identical function.Therefore this controller is considered a kind of hardware component, and to including for realizing various in it The device of function can also be considered as the structure in hardware component.Or even, it can will be regarded for realizing the device of various functions For either the software module of implementation method can be the structure in hardware component again.

System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity, Or it is realized by the product with certain function.It is a kind of typically to realize that equipment is computer.Specifically, computer for example may be used Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment The combination of equipment.

For convenience of description, it is divided into various units when description apparatus above with function to describe respectively.Certainly, implementing this The function of each unit can be realized in the same or multiple software and or hardware when specification one or more embodiment.

It should be understood by those skilled in the art that, the embodiment of this specification can provide as method, system or computer journey Sequence product.Therefore, complete hardware embodiment, complete software embodiment or knot can be used in this specification one or more embodiment The form of embodiment in terms of conjunction software and hardware.Moreover, this specification one or more embodiment can be used at one or more A wherein includes computer-usable storage medium (the including but not limited to magnetic disk storage, CD- of computer usable program code ROM, optical memory etc.) on the form of computer program product implemented.

The embodiment of this specification is referring to the method, equipment (system) and computer journey according to this specification embodiment The flowchart and/or the block diagram of sequence product describes.It should be understood that flow chart and/or box can be realized by computer program instructions The combination of the process and/or box in each flow and/or block and flowchart and/or the block diagram in figure.It can provide this A little computer program instructions are to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices Processor to generate a machine so that the finger executed by the processor of computer or other programmable data processing devices It enables and generates to specify in one or more flows of the flowchart and/or one or more blocks of the block diagram The device of function.

These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.

These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one The step of function of being specified in a box or multiple boxes.

In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net Network interface and memory.

Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium Example.

Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data. The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM), Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.

It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including described want There is also other identical elements in the process, method of element, commodity or equipment.

It will be understood by those skilled in the art that the embodiment of this specification can provide as the production of method, system or computer program Product.Therefore, this specification one or more embodiment can be used complete hardware embodiment, complete software embodiment or combine software With the form of the embodiment of hardware aspect.Moreover, this specification one or more embodiment can be used it is one or more wherein It include computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, the light of computer usable program code Learn memory etc.) on the form of computer program product implemented.

This specification one or more embodiment can computer executable instructions it is general on It hereinafter describes, such as program module.Generally, program module includes executing particular task or realization particular abstract data type Routine, programs, objects, component, data structure etc..Can also practice in a distributed computing environment this specification one or Multiple embodiments, in these distributed computing environments, by being executed by the connected remote processing devices of communication network Task.In a distributed computing environment, the local and remote computer that program module can be located at including storage equipment is deposited In storage media.

All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method Part explanation.

The foregoing is merely the embodiments of this specification, are not limited to this specification.For art technology For personnel, this specification can have various modifications and variations.It is all made any within the spirit and principle of this specification Modification, equivalent replacement, improvement etc., should be included within the scope of the claims of this specification.

Claims

1. a kind of processing method of pipelined data, which comprises

When detecting that target pipelined data generates, the business dimension category of the corresponding target service of the target pipelined data is obtained Property；

Based on the business dimension attribute, by target pipelined data storage to the business dimension attribute corresponding at least one In target data grouping in a data grouping.

2. the target pipelined data is stored according to the method described in claim 1, described be based on the business dimension attribute Into the target data grouping at least one corresponding data grouping of the business dimension attribute, comprising:

If including the business dimension attribute in database, the grouping of current time and the target pipelined data are obtained Information；

According to the grouping information of the target pipelined data, by the time and the corresponding storage of the target pipelined data described in In target data grouping.

3. the target pipelined data is stored according to the method described in claim 2, described be based on the business dimension attribute Into the target data grouping at least one corresponding data grouping of the business dimension attribute, comprising:

If in database not including the business dimension attribute, the corresponding data grouping of the business dimension attribute is created, It include that target data is grouped in the data grouping of creation；

The current time is obtained, and the time and the corresponding storage of the target pipelined data are arrived to the mesh of the database It marks in data grouping.

4. according to the method described in claim 1, the method also includes:

Receive the pipelined data processing request of the target service；

According to the business dimension attribute of the target service, obtain from the database corresponding with the business dimension attribute At least one data grouping；

According to the initial time of the pipelined data stored at least one described data grouping and deadline, respectively to it is described extremely The pipelined data stored in a few data grouping is handled.

5. method according to any of claims 1-4, in the corresponding each data grouping of the business dimension attribute The pipelined data of predetermined quantity can be stored.

6. according to the method described in claim 5, the grouping information is the flowing water mark of the target pipelined data, the stream It includes one or more of serial number, transaction odd numbers that water gauge, which is known, alternatively, the grouping information is and the target pipelined data Based on the relevant information of storage order under the business dimension attribute.

7. a kind of processing unit of pipelined data, described device include:

Dimensional attribute obtains module, for it is corresponding to obtain the target pipelined data when detecting that target pipelined data generates Target service business dimension attribute；

Flowing water memory module ties up target pipelined data storage to the business for being based on the business dimension attribute It spends in the target data grouping at least one corresponding data grouping of attribute.

8. device according to claim 7, the flowing water memory module, comprising:

Information acquisition unit, if obtaining current time and described for including the business dimension attribute in database The grouping information of target pipelined data；

Flowing water storing unit, for the grouping information according to the target pipelined data, by the time and the target flowing water The corresponding storage of data is into target data grouping.

9. device according to claim 8, the flowing water memory module, comprising:

It is grouped creating unit, if creating the business dimension category for not including the business dimension attribute in database Property corresponding data grouping, include that target data is grouped in the data grouping of creation；

Flowing water storing unit is arrived for obtaining the current time, and by the time and the corresponding storage of the target pipelined data In the target data grouping of the database.

10. device according to claim 7, described device further include:

Be grouped determining module, for the business dimension attribute according to the target service, obtain from the database with it is described At least one the corresponding data grouping of business dimension attribute；

Processing module, when for according to the initial time and cut-off of the pipelined data stored at least one described data grouping Between, the pipelined data stored at least one described data grouping is handled respectively.

11. the device according to any one of claim 7-10, the corresponding each data grouping of the business dimension attribute In can store the pipelined data of predetermined quantity.

12. device according to claim 11, the grouping information is that the flowing water of the target pipelined data identifies, described Flowing water mark includes one or more of serial number, transaction odd numbers, alternatively, the grouping information is and the target stream water number According to based on the relevant information of storage order under the business dimension attribute.

13. a kind of processing equipment of pipelined data, the processing equipment of the pipelined data include:

Processor；And

It is arranged to the memory of storage computer executable instructions, the executable instruction makes the processing when executed Device: