CN107085605A - Method, device and electronic equipment for data packaging - Google Patents

Abstract

The disclosure provides a method and a device for data packaging and an electronic device. The method comprises the following steps: triggering a corresponding timing task according to the last triggering time and frequency; extracting a data set to be packaged corresponding to the timing task from a database; acquiring a corresponding packaging strategy according to the strategy parameters of the timing task; and executing a packing scheduling operation on the data set to be packed according to the packing strategy to complete the packing of the data set to be packed. According to the data packaging method and device, the timing task is triggered through the last time and the last time, the data set to be packaged of the timing task is obtained, and the data set to be packaged is packaged according to the preset packaging strategy, so that configurable packaging of data can be achieved.

Description

Method, device and electronic equipment for data packaging

technical field

The present disclosure relates to the field of computer technology, in particular to a method, device and electronic equipment for data packaging.

Background technique

The existing capital platform (for example, the company's financial transaction processing and financial verification platform makes the transaction transparent and automated, which effectively shortens the company's capital transaction processing time and cost, and at the same time strengthens capital supervision through this system) and When banks and third-party payments conduct transaction transmission (including real-time, batch-processing receipt and payment transactions, asynchronous transaction result queries, etc.), there are at least the following problems:

1. For the task object created by TaskTimer, since the frequency and trigger time of the task are fixed and configured in the XML file, only simple periodic tasks can be configured. When a new task is created during project implementation, it is necessary to change the task cycle, change the XML configuration, and reload the service. Such work is very cumbersome and is not conducive to long-term operation and maintenance of the project and implementation support.

2. The policy group for data classification is shared by all data. You cannot configure a separate policy group for a specified data collection. If you need to use a different policy group for a data collection, you can only add special rules in the code.

3. After the data is subcontracted for the first time, the subsequent work is done in the process processing class named by KEY. Each process class is configured with the custom parameters required to process the package. This process-oriented writing method, such as adding a new channel, is to copy the original process class, alter it, and accumulate a lot of redundancy. The code is very inconvenient to maintain.

Therefore, there is a need for an improved method, apparatus and electronic device for data packaging.

Contents of the invention

In order to overcome the problems existing in related technologies, the present disclosure provides a method, device and electronic equipment for data packaging, so as to realize the packaging of data sets within the specified range of timing tasks.

According to a first aspect of an embodiment of the present disclosure, a method for data packaging is provided, including:

Trigger the corresponding scheduled task according to the last trigger time and frequency;

Extracting the data set to be packaged corresponding to the timing task from the database;

Obtaining a corresponding packaging policy according to the policy parameters of the timing task;

Executing a packaging scheduling job on the data set to be packaged according to the packaging policy to complete the packaging of the data set to be packaged.

In an exemplary embodiment of the present disclosure, triggering a corresponding timed task according to the last trigger time and frequency includes:

storing the last trigger time and frequency of the timing task in the database;

Obtaining the next trigger time of the scheduled task according to the last trigger time and the frequency;

Time monitoring is performed, and when the current time is equal to the next trigger time, the timing task is triggered.

In an exemplary embodiment of the present disclosure, the method further includes:

According to business requirements, create a packaging scheduler for the data set to be packaged;

The timed task and its packaging and grouping policy parameters, initial trigger time and frequency are created through the page of the packaging scheduler and stored in the database.

In an exemplary embodiment of the present disclosure, the package scheduler is a QuartzJob package scheduler.

In an exemplary embodiment of the present disclosure, the packaging policy includes a subpackaging policy, an unpacking policy and a package naming policy.

In an exemplary embodiment of the present disclosure, the package naming policy includes a sequence reset policy.

In an exemplary embodiment of the present disclosure, performing a packaging scheduling job on the data set to be packaged according to the packaging strategy, and completing the packaging of the data set to be packaged includes:

According to the packing and grouping policy parameters, allocating the data in the data set to be packed with the same packing policy into the same data package;

Traverse the data set to be packaged to complete the subpackage of the data set to be packaged.

In an exemplary embodiment of the present disclosure, according to the packing and grouping policy parameters, allocating the data in the data set to be packaged with the same packing policy into the same data package includes:

Encapsulating the packaging and grouping strategy parameters in a HashMap, and delivering to the timing task;

Composing the packetization strategy parameters into a packetization strategy key;

Put the data in the data set to be packaged with the same subpackage policy key into the same data package.

In an exemplary embodiment of the present disclosure, performing a packaging scheduling job on the data set to be packaged according to the packaging strategy, and completing the packaging of the data set to be packaged further includes:

The packetized data packets are unpacked according to the unpacking strategy.

In an exemplary embodiment of the present disclosure, unpacking the packetized data packets according to the unpacking strategy includes:

Configure the maximum number of data packets in the database according to the requirements of the channel to be sent;

Loading the maximum number of data packets configured in the database into the cache during initialization;

When unpacking, according to the packing and grouping policy parameters, the maximum number of corresponding data packets is taken out from the cache, and the data packets are unpacked.

In an exemplary embodiment of the present disclosure, performing a packaging scheduling job on the data set to be packaged according to the packaging strategy, and completing the packaging of the data set to be packaged further includes:

Create a package number for the unpacked data package according to the package naming strategy.

In an exemplary embodiment of the present disclosure, creating a package number for the unpacked data package according to the package naming strategy includes:

According to the packaging and grouping strategy parameters, configure naming strategy parameters, output rules, splicing order and sequence reset strategy in the database;

Read the sequence reset policy to determine whether the custom sequence needs to be reset;

According to the splicing sequence, the naming rules are read sequentially, and the output strings of each naming rule are spliced into a complete package number.

In an exemplary embodiment of the present disclosure, the naming rule of the packet number is formulated according to the requirements of the channel to be sent, and includes the file name and/or the serial number of the message.

In an exemplary embodiment of the present disclosure, the packaged data package is returned to the database for storage.

According to a second aspect of an embodiment of the present disclosure, a device for packaging data is provided, including:

The task trigger module is used to trigger the corresponding timing task according to the last trigger time and frequency;

A data extraction module, configured to extract the data set to be packaged corresponding to the timing task from the database;

A packaging policy acquisition module, configured to acquire a corresponding packaging policy according to the policy parameters of the timing task;

A packaging scheduling module, configured to execute a packaging scheduling job on the data set to be packaged according to the packaging strategy, and complete the packaging of the data set to be packaged.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

processor; and

a memory for storing executable instructions of the processor;

Wherein, the processor is configured to execute any one of the methods for data packaging described above.

The method, device, and electronic device for data packaging in the example implementations of the present disclosure trigger the timing task through the last trigger time and frequency, obtain the data set to be packaged for the timing task, and the data set to be packaged according to the preset packaging strategy Data sets are packaged, so that configurable packaging of data can be achieved.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention. Apparently, the drawings in the following description are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to these drawings without creative efforts.

Fig. 1 is a schematic flowchart of a method for data packaging according to an exemplary embodiment.

Fig. 2 is a schematic diagram of a system architecture showing a method for data packaging according to an exemplary embodiment.

Fig. 3 is a schematic diagram of a page of a packaging scheduler according to an exemplary embodiment.

Fig. 4 is a schematic diagram of a page for inputting an execution cycle expression according to the packaging scheduler shown in Fig. 3 .

Fig. 5 is a schematic diagram of a job list page according to an exemplary embodiment.

Fig. 6 is a schematic block diagram of a device for packing data according to an exemplary embodiment.

detailed description

Exemplary embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments. communicated to those skilled in the art. In the drawings, the same reference numerals denote the same or similar structures, and thus their detailed descriptions will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other modules, steps, etc. may be employed. In other instances, well-known structures are not shown or described in detail to avoid obscuring aspects of the disclosure.

Fig. 1 is a schematic flowchart of a method for data packaging according to an exemplary embodiment.

Referring to FIG. 1 , this example embodiment firstly provides a method for data packaging, which may include the following steps.

In step S100, a corresponding timed task is triggered according to the last trigger time and frequency.

In an exemplary embodiment, the method may further include: according to business requirements, creating a packaging scheduler for the data set to be packaged; creating the timed task and its packaging and grouping policy parameters through the page of the packaging scheduler, The initial trigger time and frequency are also stored in the database.

In an exemplary embodiment, the packing scheduler may be a QuartzJob packing scheduler. However, the present disclosure is not limited thereto, and this is only an example for illustration.

QuartzJob is a job scheduling tool implemented in pure Java, which is equivalent to Job in the database, scheduled tasks in Windows, and Cron under Unix/Linux, but Quartz can control the scheduling more finely, which is more flexible and convenient than TaskTimer before the project revision a lot.

Fig. 3 is a schematic diagram of a page of a packaging scheduler according to an exemplary embodiment. Fig. 4 is a schematic diagram of a page for inputting an execution cycle expression according to the packaging scheduler shown in Fig. 3 . In Figure 3, the QuartzJob packaging scheduler created by Bank of Communications is used as an example to illustrate. On the page of the packaging scheduler, you can enter the packaging and grouping strategy parameters and the execution cycle expression (initial trigger time and frequency) as shown in Figure 4. Spend). As shown in Figure 4 above, a permanent traffic collection package scheduling is created, a subpackage strategy group is assigned, and it is executed every 5 minutes.

In the embodiment of the present invention, according to the requirements proposed by the business, a QuartzJob packaging scheduler is created for a set of data to be packaged within a specified range (for example, it can be created through pagination, similar to filling in a form), and the packaging and grouping strategy parameters are input (the existing ones are generally Require bank or third-party payment institution code, receipt/payment direction, timeliness, quantity per package, frequency, etc., of course, this disclosure is not limited to this), customize the execution time (trigger time) and period of the scheduler according to the QuartzTrigger expression ( Frequency). After the package scheduler is created, unless there is a later business change, it will be executed periodically without manual intervention.

In an exemplary embodiment, triggering the corresponding timed task according to the last trigger time and frequency includes: storing the last trigger time and frequency of the timed task in the database; The frequency obtains the next trigger time of the scheduled task; time monitoring is performed, and when the current time is equal to the next trigger time, the scheduled task is triggered. That is, in the embodiment of the present invention, the configuration of the trigger time and frequency of each task is not fixed in the XML file, but the last trigger time of each task and the frequency of configuration are stored in the database. , read the last trigger time and configured frequency of the task from the database, so as to calculate the next trigger time, and judge whether the task is triggered at the current time according to the next trigger time. And the new task of QuartzJob can be completed through the page, similar to filling in a common form, without writing code in the background. In this way, it is possible to overcome the tedious work of changing the task cycle, changing the XML configuration, and reloading the service when creating a new task in the prior art, and overcome the technical problems in the prior art that are not conducive to long-term operation and maintenance of the project and implementation support.

Fig. 5 is a schematic diagram of a job list page according to an exemplary embodiment. As shown in FIG. 5 , the difference between the last trigger time and the next trigger time indicates the frequency of automatic triggering of the scheduled task (scheduled task). In other embodiments, one of the scheduled tasks can also be selected, and the task can be manually triggered by clicking Execute. Therefore, the embodiment of the present invention uses the QuartzJob scheduler to be more flexible than the TaskTimer, and can freely choose manual execution or automatic execution.

In the embodiment of the present invention, configurable adjustment of frequency, quantity and task flow of timing tasks can be realized.

In step S110, the data set to be packaged corresponding to the timed task is extracted from the database.

In step S120, a corresponding packaging policy is obtained according to the policy parameters of the timed task.

The packaging strategy is an overall abstract process for the packaging of the fund platform, and its function is to improve code reuse and reduce redundancy and coupling. Each scheduling job is a concrete implementation of a packaging strategy.

In an exemplary embodiment, the packaging policy includes a subpackaging policy, an unpacking policy, and a package naming policy.

In an exemplary embodiment, the package naming policy includes a sequence reset policy.

The design pattern of the embodiment of the present invention is as follows: the strategy pattern, the factory pattern and the singleton pattern are referenced on the code level, all the redundant business codes are removed, and only the subpackage strategy, unpacking strategy, package naming strategy and sequence repetition are retained. configuration strategy. Each business processing class uses the form of an interface, configures the class name in the database, and uses the factory + lazy loading mode to obtain the class object when the project is first referenced. After the first loading, use the singleton mode + spring to ensure the packaging work atomicity. That is to configure the business process in the database, and abstract similar processing processes into abstract processing processes. When the source data calls the policy group, the system will load and instantiate the policy group corresponding to the source data, and then process the source data. When the source data of this group is processed by the instanced policy group, it will not be called for processing by other policy groups, and each processing group is relatively independent.

Among them, the above-mentioned strategy mode refers to defining a series of algorithms, encapsulating each algorithm, and making them interchangeable. This mode enables the algorithm to vary independently of the clients using it. Abstract multiple different data operations into multiple processes.

The aforementioned factory pattern refers to the creation of subsequent processes based on source data.

The above-mentioned singleton mode means that only one process can be created and used at the same time.

The above-mentioned subcontracting strategy is to extract the data allowed by the business in a batch according to the specified strategy. The subcontracting strategy refers to the first step of screening the source data according to the requirements of the business system.

The above-mentioned unpacking strategy means that the data extracted after the subpackaging strategy is split into a data format that meets the requirements of each request of the peer end according to the requirements (and strategies) of the bank or third-party payment institution.

The above-mentioned package naming strategy is based on the agreement between the fund platform and the bank and/or third-party payment institution. Every data sent to the bank or third-party payment institution contains a package name. These package names are used to distinguish and indicate that each transaction on the fund platform unique serial number.

The above-mentioned sequence reset strategy refers to the package name named according to the package naming strategy, and its custom sequence part may be repeated. Therefore, it needs to be checked, and if it is repeated, it will be reset.

In step S130, a packaging scheduling job is executed on the data set to be packaged according to the packaging policy, and the packaging of the data set to be packaged is completed.

The entire packaging and scheduling job can be divided into three steps: subpackaging, unpacking, and package naming. Each of them will be described below.

In an exemplary embodiment, according to the packaging policy, the package scheduling job is executed on the data set to be packaged, and completing the packaging of the data set to be packaged includes: according to the packaging and grouping policy parameters, grouping the data sets with the same package policy The data in the data set to be packaged is allocated to the same data package; the data set to be packaged is traversed to complete the subpackaging of the data set to be packaged.

When data enters the fund platform from other channels (such as various business systems), the first landing (stored in the database) will match the strategy group according to the source, amount, receipt/payment transaction and other attributes of the data. The policy group is similar to the workflow process, which is based on the rules extracted from known business scenarios. When new data matches the policy group, the flow of this data has been determined. The object-oriented writing method can be adopted in the embodiment of the present invention. The new channel writes the required data configuration in the database, and uses the adapter mode in the code to realize it.

In an exemplary embodiment, according to the packaging and grouping strategy parameters, allocating the data in the data set to be packaged with the same subpackaging strategy into the same data package includes: encapsulating the packaging and grouping strategy parameters in a HashMap, and passed to the timing task; the packaging and grouping strategy parameters are combined into a packetization strategy key; and the data in the data set to be packaged with the same packetization strategy key are placed in the same data packet.

In an exemplary embodiment, performing a packing scheduling job on the data set to be packed according to the packing strategy, and completing the packing of the data set to be packed further includes: performing the packetizing on the data packets after packing according to the unpacking policy unpacking.

In an exemplary embodiment, unpacking the subpackaged data packets according to the unpacking strategy includes: configuring the maximum number of data packets in the database according to the requirements of the channel to be sent; The maximum number of data packets in the database is limited and loaded in the cache; according to the packaging and grouping strategy parameters when unpacking, the maximum number of corresponding data packets is taken out from the cache, and the data packets are processed Unpacking process.

Perform secondary unpacking for each package after subpackaging. Each data packet has a maximum number limit, which varies according to the channel requirements to be sent. The maximum number limit is configured in the database, loaded into the cache when the project is initialized, and will be unpacked according to the subpackage policy key. The maximum number of transactions is taken out of the cache, and the second unpacking process is performed. For example, the maximum quantity per package of Bohai Bank is 2,000, while the maximum quantity per package of Baorong is 10,000. Both Bohai Bank and Baorong are third-party payment institutions. In other embodiments, the maximum number of transactions per data packet may be limited according to the third-party message and disk file, so that secondary unpacking is performed according to the maximum number of transactions in each data packet.

In an exemplary embodiment, executing a package scheduling job on the data set to be packaged according to the package policy, and completing the package of the data set to be packaged further includes: creating an unpacked data package according to the package naming policy package number.

In an exemplary embodiment, creating a packet number for the unpacked data packet according to the packet naming strategy includes: configuring naming strategy parameters, output rules, splicing order and sequence in the database according to the packaging and grouping strategy parameters Reset strategy; read the sequence reset strategy to determine whether the custom sequence needs to be reset; read the naming rules sequentially according to the splicing sequence, and splice the output strings of each naming rule into a complete package number.

In an exemplary embodiment, the naming rule of the packet number is formulated according to the requirements of the channel to be sent, which includes the file name and/or the serial number of the message.

Each data packet (the data packet after the second unpacking) needs to create a packet number, and the naming rules of the packet number are mainly based on the requirements of the channel to be sent. It can be the file name, or the serial number of the message (for example, the package name of the communication channel is the same as the FTP file name, and the package name of a single real-time transaction of Industrial and Commercial Bank of China is the serial number of the message). According to each subpackage policy key, complete naming strategy parameters (package naming methods corresponding to different naming strategies, naming strategy is a set of policies requiring package naming rules for each channel), output rules, splicing order and sequence reset are configured in the database strategy etc. First read the sequence reset policy to determine whether the custom sequence is reset. Then read the command rules sequentially according to the splicing sequence, and splice the output strings of each rule into a complete package name. Among them, this package name is a package name naming standard after discussion between the funding platform and the bank or third-party payment institution.

Still referring to Figure 3, in the example development of China Merchants Bank’s Bank-Insurance-Pass channel, the service provider stipulates the name of the offer file: type identification_organization code (4 digits)_date (8 digits)_custom serial number (5 digits) , according to the business structure of the fund platform, it is necessary to name the data collection package (that is, the data package, the business data collection to be sent) as the file name.

Wherein, the above-mentioned type identification can have 4 business types, for example, 101-public payment, 102-private payment, 103-public collection, 104-private collection. However, the present disclosure is not limited thereto.

Analyze the packaging naming strategy that needs to be invoked. The naming strategy can be configured in the BE_SEQRULE table (packaging strategy table, stored in the database), as shown in Table 1 below.

Table I

Among them, PART_NUM in Table 1 indicates the splicing sequence; PART_CLASS indicates the serial number of the naming strategy that needs to be called; PART_VALUE indicates the calling parameter of the naming strategy method;

PART_LENGTH indicates the length of the returned string of the named parameter, if the string is not enough, add _, and if the number is not enough, add 0.

Let's take a sample program as an example to illustrate:

For example, PART_CLASS=2, call Case 2: obtain the current eight-digit date partValue=ToolUtil.getNowDate(); PART_CLASS=5 read the sequence table and obtain the current sequence. The final output package number: 101_110Z_20161014_000001 (example).

The existing package naming strategy is already perfect, meeting the packaging needs of most channels. If special needs are encountered, the code level only needs to add a naming strategy and a sequence reset strategy, which is to increase the configuration of an abstract strategy. The configuration information of business parameters (generally including bank or third-party payment institution code, payment direction, timeliness, quantity per package, frequency, etc., but not limited to these, and specific situations need to be analyzed in detail) can be completed by the operation and maintenance team .

In an exemplary embodiment, the packaged data package is returned to the database for storage.

In the method for data packaging provided by the embodiment of the present invention, the code focuses on writing various strategies (subpackage strategy, unpacking strategy, package naming strategy and sequence reset strategy), business parameters (source system, receipt/payment direction, destination) Banks or third-party payment institutions, etc.) are configured in the database, which realizes the low coupling between functional modules and services, and can expand more services. After the strategy is perfected in the later period, the new business only needs to be handed over to the operation and maintenance team for configuration in the database (the database can be manually intervened, or a new strategy group can be added in a page). On the one hand, it solves the difference in transaction packaging between the fund platform and banks and third-party payment companies (including package naming rules, quantity per package, payment direction, etc. Package name naming standards, etc.), code redundancy, scalability and compatibility problems in the development process are relatively weak; on the other hand, it improves development efficiency, code repetition rate and post-project operation and maintenance costs and reduces development risks .

Fig. 2 is a schematic diagram of a system architecture showing a method for data packaging according to an exemplary embodiment.

The package scheduler monitors the time and finds the trigger that triggers the task at the current time point.

Among them, the scheduler Scheduler will open a thread Thread when it is started, and the Thread will continuously loop to the database according to the current time to query the trigger that will trigger the activity of the job at that time point. Task scheduling refers to setting a fixed time point, time interval or number of executions to automatically execute tasks. A scheduled task is a task that is automatically called up at a specified period and frequency at a specified time point or within a date range. Its scheduling execution does not require manual intervention. It is mainly divided into two categories according to the source. One is based on online transactions. The task reserved in the trading environment, one type is the task that is directly embedded in the database when the project goes online.

Responding to the trigger instruction, the scheduled task data to be instantiated in the database is instantiated as a scheduled task instance, and stored in the database.

Specifically, the trigger condition here may be: according to the preset scan cycle, scan whether there is scheduled task data in the database, and if it is found that the scheduled task data exists, then determine that the trigger instruction is satisfied.

Timed task data can only be processed later if it is instantiated into a specific timed task instance branch. During the instantiation of a scheduled task, it is necessary to refer to the task cycle data and the current system time to generate a scheduled task instance and store the generated scheduled task instance in the database. For example, a periodic task is batch payment at 17:00 every day, and the current time is 8:00 on November 1st, then a scheduled task instance with execution time at 17:00 on November 1st will be generated.

Then, invoke the scheduled task instance to be executed in the database.

Specifically, similar to the trigger instruction above, scan the database for a scheduled task instance according to the preset scan cycle, and if it finds that there is a scheduled task instance, call the JOB of the Spring-batch framework to execute the scheduled task instance to be executed .

After the scheduled task instance starts to execute, the capital platform obtains the data set to be packaged from the database (for example, db2).

Next, obtain the corresponding packaging strategy from the subpackage scheduling job cache, and execute the subpackage scheduling job on the data set to be packaged.

The packaging and grouping policy parameters created by the QuartzJob scheduler can be encapsulated in a HashMap (but the present disclosure is not limited thereto) and passed to the JOB (the task behind the instance, that is, the task to be executed). The data set to be packaged is obtained through SpringJdbc (which can be understood as database interaction, and the data set to be packaged is obtained from the database). The business layer forms the policy parameters into a key (for example, a subpackage strategy with a number, which is convenient for the system to quickly obtain), with the same The data collection of the key is placed in a list (a collection with serial numbers). Traverse all business data (all business data is equal to all data to be packaged) to complete the first subpackage (that is, grouping in Figure 2, grouping data using the same subpackaging strategy into the same group).

Then, according to the restriction of the third party (bank or third-party payment institution), the subpackaged data package is unpacked for the second time. Here, the number of packages after the second unpacking will not be less than 1. One instruction in each package is the transaction details to be transferred.

Second, perform data landing on the unpacked data packets. Among them, the data landing is the instruction table for data storage to the db2 database.

In the embodiment shown in FIG. 2 , it may further include: landing the instruction (for example, a service instruction issued by the service system) and the package information, and making the package in a state to be sent.

The various strategies that have been developed so far are perfect and sufficient to meet most of the channel needs. If there are special needs, the development team only needs to add corresponding strategies. The whole module becomes very refreshing, and the scalability is very strong. For hands-on colleagues who understand various strategies without touching the code, they can independently complete the packaging and scheduling operations of the new channel or modify the original packaging and scheduling operations through page-based operations, which greatly shortens the later development and maintenance. Work.

Fig. 6 is a schematic block diagram of a device for packing data according to an exemplary embodiment.

This example implementation also provides an apparatus 10 for data packaging, which may include: a task trigger module 100 , a data extraction module 110 , a packaging policy acquisition module 120 and a packaging scheduling module 130 .

The task triggering module 100 can be used to trigger corresponding timed tasks according to the last trigger time and frequency.

The data extraction module 110 may be configured to extract the data set to be packaged corresponding to the scheduled task from the database.

The packaging policy acquisition module 120 may be configured to acquire a corresponding packaging policy according to the policy parameters of the timed task.

The packaging scheduling module 130 may be configured to execute a packaging scheduling job on the data set to be packaged according to the packaging strategy, and complete the packaging of the data set to be packaged.

Regarding the above-mentioned apparatus for data packaging, the specific manner in which each module performs operations has been described in detail in the exemplary embodiment of the method for data packaging, and will not be described in detail here.

This exemplary embodiment also provides an electronic device, including: a processor; and a memory, configured to store executable instructions of the processor. Wherein, the processor may be configured to execute the method for packing data described in any of the above embodiments.

The above descriptions are only some exemplary embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure. within the scope of protection.

Claims (16)

1. a kind of method packed for data, it is characterised in that including：

Corresponding timed task is triggered according to last triggered time and frequency；

Extract that the timed task is corresponding to treat packing data collection from database；

Corresponding packing strategy is obtained according to the policing parameter of the timed task；

According to it is described packing strategy to it is described treat packing data collection perform packing schedule job, complete described in treat packing data collection Packing.

2. the method according to claim 1 packed for data, it is characterised in that according to last triggered time and frequency The corresponding timed task of degree triggering includes：

The last triggered time of the timed task and frequency are stored in the database；

The triggered time next time of the timed task is obtained according to the last triggered time and the frequency；

Time monitoring is carried out, when current time is equal to the triggered time next time, the timed task is triggered.

3. the method according to claim 1 packed for data, it is characterised in that methods described also includes：

It is described to treat that packing data collection creates packing scheduler according to business demand；

Pass through timed task and its packing grouping strategy parameter, initial trigger time described in the page creation of the packing scheduler With frequency and be stored in the database.

4. the method according to claim 3 packed for data, it is characterised in that the packing scheduler is QuartzJob packing schedulers.

5. the method according to claim 3 packed for data, it is characterised in that the packing strategy includes subpackage plan Summary, unpack strategy and bag naming policy.

6. the method according to claim 5 packed for data, it is characterised in that the bag naming policy includes sequence Reset strategy.

7. the method according to claim 5 packed for data, it is characterised in that according to the packing strategy to described Treat packing data collection perform packing schedule job, complete described in treat that the packing of packing data collection includes：

According to the packing grouping strategy parameter, by with the data distribution that packing data concentration is treated described in identical subpackage strategy Into same packet；

The traversal subpackage for treating to treat packing data collection described in the completion of packing data collection.

8. the method according to claim 7 packed for data, it is characterised in that according to the packing grouping strategy ginseng Number, will include with data distribution to the same packet that packing data concentration is treated described in identical subpackage strategy：

The packing grouping strategy parameter is encapsulated in HashMap, and is transferred to the timed task；

By the packing grouping strategy parameter group into subpackage strategy key；

It will be placed on the data that packing data concentration is treated described in identical subpackage strategy key in same packet.

9. the method according to claim 7 packed for data, it is characterised in that according to the packing strategy to described Treat packing data collection perform packing schedule job, complete described in treat that the packing of packing data collection also includes：

The packet after subpackage is unpacked according to the strategy of unpacking.

10. the method according to claim 9 packed for data, it is characterised in that according to the strategy of unpacking to dividing Packet after bag unpacked including：

Required according to channel to be sent, by the maximum stroke count limitation arrangement of packet in the database；

The maximum stroke count limitation loading of packet during initialization by configuration in the database is in the buffer；

According to the packing grouping strategy parameter when unpacking, the maximum stroke count limitation of corresponding data bag is taken out in the caching, Processing of unpacking is carried out to the packet.

11. the method according to claim 9 packed for data, it is characterised in that according to the packing strategy to institute State and treat the execution packing schedule job of packing data collection, treat that the packing of packing data collection also includes described in completion：

According to the bag naming policy to unpacking after packet create Bale No..

12. the method according to claim 11 packed for data, it is characterised in that according to the bag naming policy pair Packet after unpacking, which creates Bale No., to be included：

According to the packing grouping strategy parameter, naming policy parameter, output rule, splicing order are configured in the database And sequence resets strategy；

Read the sequence and reset strategy, judge whether custom list needs replacement；

According to splicing order, naming rule is successively read, each naming rule output string is spliced into complete Bale No..

13. the method according to claim 12 packed for data, it is characterised in that the naming rule root of the Bale No. Require to formulate according to channel to be sent, it includes the serial number of filename and/or message.

14. the method according to claim 11 packed for data, it is characterised in that return the packet for completion of packing It is back in the database and is stored.

15. a kind of device packed for data, it is characterised in that including：

Task trigger module, for triggering corresponding timed task according to last triggered time and frequency；

Data extraction module, for being extracted from database, the timed task is corresponding to treat packing data collection；

The tactful acquisition module of packing, for obtaining corresponding packing strategy according to the policing parameter of the timed task；

Packing scheduler module, for treating that packing data collection performs packing schedule job, completion to described according to the packing strategy The packing for treating packing data collection.

16. a kind of electronic equipment, it is characterised in that including：

Processor；And

Memory, the executable instruction for storing the processor；

Wherein, the processor, which is configured as performing in the claims 1-14 described in any one, is used for what data were packed Method.