CN113434307A - Task sending processing method, task processing method, device, system and equipment - Google Patents

Abstract

The application provides a task sending processing method, a task sending device, a task processing device, a task sending system and a task processing system. The method comprises the following steps: acquiring a task to be processed, wherein the task is an order task of order information to be supplemented or a task of article information to be added, and the task comprises a task type; determining a task queue corresponding to the task according to the task type of the task and the corresponding relation between the preset task type and the task queue; and sending the task to a task queue corresponding to the task. The method avoids the problem of high database pressure caused by frequent refreshing of the task table, and improves the task processing efficiency.

Description

Task sending processing method, task processing method, device, system and equipment

Technical Field

The present application relates to computer technologies, and in particular, to a method, an apparatus, a system, and a device for processing task transmission and task processing.

Background

With the development of computer technology, task processing by computers is a trend.

Currently, task processing relies primarily on databases. Namely, a task table is arranged in the database and used for storing the tasks uploaded by the user. And the task processing end pulls single or batch tasks from the task table at regular time for processing, and sends the processing result of the tasks to a downstream system of the service.

However, the method of regularly pulling tasks from the task table has the problem of frequent query of the task table, which causes excessive pressure on the database.

Disclosure of Invention

The application provides a task sending processing method, a task sending processing device, a task processing system and a task processing device, which are used for solving the problem that the database pressure is too high due to frequent query of a task table in a mode of regularly pulling tasks from the task table.

In a first aspect, the present application provides a method for processing task sending, which is applied to a task submitting end, and the method includes: acquiring a task to be processed, wherein the task is an order task to be supplemented with order information or a task to be added with article information, and the task comprises a task type; determining a task queue corresponding to the task according to the task type of the task and the corresponding relation between the preset task type and the task queue; and sending the task to a task queue corresponding to the task.

In a second aspect, the present application provides a task processing method, applied to a task processing end, where the task processing end corresponds to a task queue, and the method includes: acquiring a task to be processed in the task queue, wherein the task is an order task to be supplemented with order information or a task to be added with article information; calling a first task processing thread established in advance to process the task, and determining whether the task is successfully processed; and if the task is successfully processed, sending the processing result of the task to a downstream system of the service corresponding to the task.

In a third aspect, the present application provides a processing apparatus for task sending, including:

the acquisition module is used for acquiring a task to be processed, wherein the task is an order task to be supplemented with order information or a task to be added with article information, and the task comprises a task type;

the determining module is used for determining a task queue corresponding to the task according to the task type of the task and the corresponding relation between the preset task type and the task queue;

and the sending module is used for sending the task to a task queue corresponding to the task.

In a fourth aspect, the present application provides a task processing device, where the task processing device corresponds to a task queue, and the task processing device includes:

the acquisition module is used for acquiring a task to be processed in the task queue, wherein the task is an order task to be supplemented with order information or a task to be added with article information;

the processing module is used for calling a first task processing thread established in advance to process the task and determining whether the task is successfully processed;

and the sending module is used for sending the processing result of the task to a downstream system of the service corresponding to the task if the task is successfully processed.

In a fifth aspect, the present application provides a task processing system, comprising:

a processing device for task transmission according to the third aspect;

the task processing device of the fourth aspect;

the task queue database is used for storing a plurality of task queues;

the configuration center database is used for storing the processing device sent by the task and task configuration information of the task processing device, wherein the task configuration information of the processing device sent by the task comprises a corresponding relation between a task type and a task queue, and the task configuration information of the task processing device comprises a corresponding relation between an identifier of the task processing device and the task queue;

and the task event information database is used for storing task snapshot information of the task.

In a sixth aspect, the present application provides a computer device comprising: a memory, a processor;

a memory; a memory for storing the processor-executable instructions;

wherein the processor is configured to perform the method of the first or second aspect.

In a seventh aspect, the present application provides a computer-readable storage medium having stored thereon computer-executable instructions for implementing the method according to the first or second aspect when executed by a processor.

In an eighth aspect, the present application provides a computer program product comprising a computer program that, when executed by a processor, implements the method of the first or second aspect.

According to the task sending processing and task processing method, device, system and equipment, the task to be processed is obtained, wherein the task is an order task to be supplemented with order information or a task to be added with article information, and the task comprises a task type; determining a task queue corresponding to the task according to the task type of the task and the corresponding relation between the preset task type and the task queue; and sending the task to a task queue corresponding to the task. In the embodiment, the task is sent to the task queue, and the task processing end obtains the task from the task queue and processes the task, so that the problem of high database pressure caused by frequent refreshing of the task table due to the traditional dependence on the database task table can be solved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

FIG. 1 is an architecture diagram of a prior art task processing system;

FIG. 2 is an architecture diagram of a task processing system according to an embodiment of the present application;

fig. 3 is a flowchart of a task sending processing method according to an embodiment of the present application;

FIG. 4 is a logical block diagram of a task processing method implemented based on the system architecture of FIG. 2;

fig. 5 is a first flowchart of a task processing method provided in an embodiment of the present application;

fig. 6 is a second flowchart of a task processing method according to an embodiment of the present application;

fig. 7 is a flowchart three of a task processing method provided in the embodiment of the present application;

fig. 8 is a fourth flowchart of a task processing method provided in an embodiment of the present application;

fig. 9 is a schematic structural diagram of a processing device for task sending according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a task processing device according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a computer device according to an embodiment of the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

Fig. 1 is an architecture diagram of a prior art task processing system. As shown in fig. 1, the system includes: the system comprises a plurality of task submitting terminals 11, a database server 12 and a plurality of task processing terminals 13, wherein each task submitting terminal 11 in the plurality of task submitting terminals 11 is in communication connection with the database server 12, and each task processing terminal 13 in the plurality of task processing terminals 13 is in communication connection with the database server 12.

Each task submitting end 11 may be a mobile phone, a computer, an Ipad, a server, or other devices; each task processing terminal 13 may also be a mobile phone, a computer, an Ipad, a server, or the like.

The database server 12 may be one server or a server cluster including a plurality of servers, and the database server 12 stores a task table.

The task processing system can be applied to processing tasks generated in the business processing process. Specifically, the user may create a task through the task submitting terminal 11 and upload the task to the task table. And the task processing end pulls single or batch tasks from the task table at regular time for processing, and stores the processing result of the tasks in a database or directly sends the processing result to a downstream system of the service. The following describes the task processing procedure with a specific application scenario in conjunction with fig. 1:

in the E-market scene, order data are quickly put into a database (hereinafter referred to as the database) for guaranteeing that the order is quickly put into the database, and order data are quickly displayed for a user. The current processing mode is that an order system puts basic information of an order into a warehouse and submits a task of complete information of the order to be supplemented to a task list. And the task processing end starts a timing task, and the timing task is used for indicating the task processing end to pull a task from the task table at intervals, completely supplementing order information and sending the order information to a downstream system, such as an inventory system and a charging system.

For the above scenario, the task processing end needs to query the task table frequently, which results in a large database pressure.

In addition, as the number of tasks in the task table increases, the disk space of the database gradually decreases, and in order to solve the problem that the disk space decreases, the historical tasks need to be deleted at regular time, and deletion of the historical tasks also generates a large amount of database disk fragments and occupies the storage space of the database.

In addition, in order to guarantee the processing efficiency of the tasks, the time interval of the pull task may be set to be short, but the database pressure is too high. Therefore, the time interval of the task is not too short at present, and the task processing end pulls the task from the task table by means of the timed task, so that the task processing delay occurs, and the task processing speed cannot be guaranteed.

And once a database where the task table is located fails, the task cannot be created, abnormal upward throwing occurs, normal execution of non-task logic is affected, and task data cannot be pulled, so that normal processing of the task is affected.

In view of the above technical problems, the inventors of the present application propose the following technical idea: the task queue is used for replacing a database task table, the tasks created by the user are pushed to the task queue, the task submitting end is set to monitor the tasks in the task queue, and the tasks are obtained from the task queue under the condition that the tasks are monitored, so that the problem that the database pressure is too high due to frequent query of the database task table is solved, the database task table is not relied on, and the database pressure is reduced.

The task processing method provided by the present application is described in detail in specific embodiments with reference to the accompanying drawings. The following specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 2 is an architecture diagram of a task processing system according to an embodiment of the present application. As shown in fig. 2, the system architecture of the present embodiment includes: a task submitting terminal 21, a database server 22, a task processing terminal 23, a configuration center 24 and a task event library 25. Each task submitting terminal 21 of the plurality of task submitting terminals 21 is in communication connection with the database server 22, the configuration center 24 and the task event repository 25, respectively. Each task processing terminal 23 of the plurality of task processing terminals 23 is communicatively connected to the database server 22, the configuration center 24, and the task event repository 25, respectively.

Each task submitting terminal 21 may be a device such as a mobile phone, a computer, an Ipad, a server, and the like.

The database server 22 may be one server or a server cluster including a plurality of servers, and the database server 22 is provided with a task queue and a failure queue. The task Queue and the failure Queue in the database server can be realized based on a List structure in Redis, can also be realized based on a Queue of Zookeeper application, and can also be realized based on a dispatcher (high-performance Queue).

Each task processing terminal 23 may also be a mobile phone, a computer, an Ipad, a server, or the like.

The configuration center 24 may be a ZooKeeper based distributed configuration center.

The task event repository 25 may be an ElasticSearch database (ES database for short).

The task submitting terminal 21 may configure information related to creating the task, such as routing information for the task, at the configuration center 24. Specifically, the routing information may be a correspondence between task types and task queues, where the correspondence is used to indicate that one task queue stores tasks of one task type.

Similarly, the task processing terminal 23 may also configure information related to task processing in the configuration center 24, such as a correspondence between the task processing terminal 23 and a task queue, where the correspondence is used to instruct one task processing terminal 23 to acquire a task from the task queue corresponding to the task processing terminal.

The task submitting terminal 21 obtains the task created by the user and sends the task to the task queue according to the routing information. And the task processing terminal 23 acquires and processes the task from the corresponding task queue according to the configured information. In addition, the task submitting terminal 21 and the task processing terminal 23 may also record task snapshot information of the task in the task event library 25, where the task snapshot information includes a task state.

The following describes in detail the task creation and task sending by the task submitting end based on the system architecture diagram shown in fig. 2.

Fig. 3 is a flowchart of a processing method for task sending according to an embodiment of the present application, and as shown in fig. 3, the processing method for task sending according to the present embodiment includes the following steps:

s301, a task to be processed is obtained, the task is an order task to be supplemented with order information or a task to be added with article information, and the task comprises a task type.

The main execution body of the method of the embodiment is the task submitting end shown in fig. 2.

Fig. 4 is a logic block diagram of a task transmission processing method and a task processing method implemented based on the system architecture of fig. 2. The following describes in detail a processing method for task sending according to an embodiment of the present application with reference to fig. 2 in fig. 4:

in this embodiment, the task to be processed may be a task created by a user through a task submitting end.

As shown in fig. 4, the task submitting side includes a plurality of task submitting terminals, and each user creates a task through one task submitting terminal. Specifically, each task submitting end is provided with a task processing application, and the task processing application is used for providing functions of creating a task and sending the created task for a user. Furthermore, each application may further include a service code and a task framework sending end, where the service code is used to provide a service processing logic for creating a task for a user, and the task framework sending end is used to send the task created by the user.

The order task of the order information to be supplemented may be a task of supplementing the order information required for taking out the order from the warehouse, or a task of supplementing the order information required for charging according to the order.

Application a and application B in fig. 4 are distributed on different task submitting ends. Application a and application B may provide functionality for different users to create a task and send the task.

Taking the task of supplementing order information as an example, application a and application B may be used to create a task of supplementing order information to different orders, respectively. For example, application a is used to create a task that supplements order information to order a, and application B is used to create a task that supplements order information to order B.

Application a and application B may also be used separately to create tasks that supplement the same order with different order information. For example, the application a is used to create a task for supplementing the first order information in the order a, the application B is used to create a task for supplementing the second order information in the order B, and the first order information and the second order information are different order information.

S302, determining a task queue corresponding to the task according to the task type of the task and the corresponding relation between the preset task type and the task queue.

The database server in fig. 2 is provided with a plurality of task queues, each for processing tasks of one task type. In order to avoid the problem that the database server is over stressed because different types of tasks are concentrated in one task queue and all task processing ends acquire the tasks from the same task queue, different task types can be bound with the task queues, so that one task queue is used for receiving the tasks of one task type. Specifically, before executing step S302 or step S301, a user at the task submitting end needs to configure a corresponding relationship between the task type and the task queue in the configuration center, so that after the device at the task submitting end is started, the corresponding relationship is periodically and polled to obtain the corresponding relationship from the configuration center, so as to implement subsequent task sending.

And S303, sending the task to the task queue corresponding to the task.

The step S302 and the step S303 are embodiments of sending the task by the task submitting end, and may specifically refer to a logic of sending the task by the task framework sending end in fig. 4, where the logic includes: determining a task queue corresponding to the task according to the task type of the task; establishing communication connection with the determined task queue; and sending the task to the determined task queue so that the task processing end acquires the task from the task queue and processes the task.

As shown in fig. 4, the task queue a corresponds to tasks of one task type, 0 to 17 in the task queue a represent 18 tasks, and the tasks sent by the task submitting end are numbered in sequence according to the above numbers and stored in the task queue a, where the smaller the number of the task is, the shorter the storage time of the task in the task queue is, and conversely, the larger the number of the task is, the longer the storage time of the task in the task queue is.

The task queue in this embodiment is a distributed first-in first-out queue. The task queue may employ a queue implemented based on a List structure in Redis. Referring to fig. 4, in this embodiment, a function of selecting a corresponding queue according to a specific Service scenario and customizing implementation of a method required by the queue in an SPI may be implemented through a Service Provider Interface (SPI) in fig. 4.

On the basis of the embodiment shown in fig. 3, fig. 5 is a first flowchart of a task processing method provided in the embodiment of the present application, and as shown in fig. 5, the task processing method further includes the following steps:

s501, if the task transmission fails, the task is transmitted to the task queue again according to the retry transmission strategy.

Wherein the retry sending strategy comprises: under the condition that the retry times are less than the preset times, repeatedly executing the operation of sending the tasks to the task queue; and sending an abnormal operation prompt under the condition that the retry number reaches a preset number.

Before executing step S501, the task submitting side needs to configure a time interval between the retry number (preset number) of the task and the retry operation of two adjacent tasks at the configuration center.

In the process that a task submitting end fails to send a task to a task queue for the first time and sends the task to the task queue again, an event for sending the task to the task queue again needs to be written into a task event library after sending the task to the task queue again each time, the event for sending the task to the task queue again comprises the time for sending the task again corresponding to the current event, the task event library adds 1 to the retry number of the task according to the event for sending the task to the task queue again, and the time for sending the task again next time is determined according to the time for sending the task again corresponding to the current event and the preset time interval between the retry operations of two adjacent tasks.

Specifically, the operation of repeatedly executing the task of sending the task to the task queue includes: determining whether the retry times are less than the preset times or not according to the preset times which are configured in advance and the retry times of the tasks recorded in the task event library; under the condition that the retry times are less than the preset times, repeatedly executing the operation of sending the tasks to the task queue; and sending an abnormal operation prompt under the condition that the retry number reaches a preset number.

In the embodiment, when the task is failed to be sent to the task queue corresponding to the task, the task is sent to the task queue again according to the retry sending strategy, and when the retry number is smaller than the preset number, the task is sent to the task queue repeatedly; and sending an abnormal operation prompt under the condition that the retry number reaches a preset number. The retry sending strategy can enable the task to realize automatic retry sending, and the success rate of task sending is increased.

On the basis of the above embodiment, after the task submitting end sends the task to the task queue for the first time, the task submitting end needs to write first task snapshot information of the task to the task event library, where the first task snapshot information includes: the task number, the number of the service corresponding to the task, the task name, the data of the service corresponding to the task, the IP address of the task submitting end, the task submitting time, the task state and the like. And after the task is sent to the task queue again every time, writing second task snapshot information of the task into the task event library, wherein the second task information comprises the retry times of sending the task to the task queue and the next resending time of the task.

The task number is used for uniquely identifying the task, and the number of the service corresponding to the task is used for uniquely identifying the task. The task name can be used for determining the task type, and the data of the service corresponding to the task specifically can be data such as an order number. And after the task submitting end sends the task to the task queue, the task processing end is in an unprocessed state when the task processing end does not process the task.

For each writing event of the task snapshot information, if the writing of the task snapshot information fails, the task snapshot information of the task is written into the task event library again according to a re-writing strategy; wherein the rewrite strategy includes: under the condition that the rewriting times are less than the preset writing times, repeatedly executing the operation of writing task snapshot information of the task into the task event library; and under the condition that the rewriting times reach the preset writing times, discarding the task snapshot information of the task.

Similarly, the task submitting end needs to configure the preset writing times of the task snapshot information in the configuration center, and determines whether the task snapshot information can be continuously written into the task event library according to the preset writing times and the current writing times.

In this embodiment, after sending a task to a task queue for the first time, first task snapshot information of the task is written to a task event library, where the first task snapshot information includes: and data of the service corresponding to the task. The task is equivalent to the backup of the task in the task event library, so that even if the task queue in the database fails, the task pulling is not influenced, and the normal processing of the task is ensured. In addition, the task snapshot information is rewritten into the task event library according to the rewriting strategy, so that the task snapshot information can be automatically rewritten again, and the success rate of writing the task snapshot information is increased.

The following describes in detail the task processing performed by the task processing side based on the system architecture diagram shown in fig. 2.

Fig. 6 is a second flowchart of a task processing method provided in the embodiment of the present application, and as shown in fig. 6, the task processing method of the present embodiment includes the following steps:

s601, obtaining the task to be processed in the task queue.

If the task is successfully sent, the task processing end can acquire the task to be processed in the task queue.

Different task types and task queues need to be bound with the task submitting end, so that one task queue is used for receiving tasks of one task type, and the task processing end also needs to bind the task submitting end with the task type. Specifically, before executing step S601, a user at the task submitting end needs to configure the corresponding relationship between the identifier of the task submitting end and the task queue in the configuration center, so that after the device at the task submitting end is started, the corresponding relationship is periodically and polled to obtain the task from the task queue for processing.

In some embodiments, step S601 includes: determining a task queue corresponding to the identifier of the task submitting end according to the identifier of the task submitting end and the corresponding relation between the identifier of the task submitting end and the task queue; establishing communication connection with the determined task queue; monitoring tasks to be processed in the determined task queue; and if the task to be processed in the determined task queue is monitored, acquiring the task to be processed from the determined task queue.

S602, calling a first task processing thread established in advance to process the task, and determining whether the task is successfully processed.

In addition, the task processing end needs to pre-configure configuration information for processing the tasks in the task queue in the configuration center, which specifically includes the number of first task processing threads, the pull interval of the tasks in the task queue, a re-processing policy of the tasks in the task queue, a failure policy of the tasks in the task queue, the number of delay threads, a monitoring configuration of the tasks in the task queue, a current limiting policy of the tasks in the task queue, a data serialization converter, and a task failure processor. Step S602 includes: establishing a corresponding number of first task processing threads according to the number of the first task processing threads configured in advance; each first task processing thread pulls a task from the task queue according to the pulling interval of the task; each first task processing thread processes the task pulled from the task queue.

Fig. 4 is a logic block diagram of a task processing method implemented based on the system architecture of fig. 2. The following describes in detail a task processing method according to an embodiment of the present application with reference to fig. 2 in fig. 4:

with continued reference to fig. 4, each task processing thread processes the pulled task and determines whether the task was successfully processed, including: inputting the service data corresponding to the task into a data serialization converter for deserializing to obtain a deserializing result of the service data; inputting the deserialization result of the service data into a task preprocessor for preprocessing to obtain a preprocessing result; inputting the pre-processing result into a main task processor for processing to obtain an intermediate processing result; inputting the intermediate processing result into a task post processor for post processing to obtain a post processing result; and determining whether the task is successfully processed according to the post-processing result.

Wherein, the monitoring configuration of the task is as follows: the tasks are allowed to be monitored and buried, and monitoring information is collected and displayed by means of an external monitoring platform.

Task current limiting: the task handler may set a maximum processing speed to limit the task processing, such as whether to turn on the limit, maximum number of task processing per second.

A task failure processor: for processing the failed task.

Optionally, when the task processing fails, determining whether a re-processing condition is met by default, and if so, submitting the task queue for re-processing; if the condition is not met, alarming is carried out, and a prompt is sent, wherein the prompt comprises but is not limited to the forms of sending mails, short messages and the like.

Referring to fig. 4, the first task processing thread in this embodiment may be a single consuming thread, or may be multiple consuming threads, and multiple consuming threads may process the task in parallel, for example, one consuming thread processes one task.

And S603, if the task is determined to be successfully processed, sending the processing result of the task to a downstream system of the service corresponding to the task.

And aiming at the order task of supplementing the order information required by ex-warehouse, if the order information required by ex-warehouse of the order is successfully supplemented, sending the supplemented order to the ex-warehouse system.

And aiming at the order task of supplementing the order information required by charging, if the order information required by charging of the order is successfully supplemented, sending the supplemented order to a charging system.

Of course, the supplemented order may also be stored in the database server in fig. 2 and recalled from the database server by the ex-warehouse system or the billing system.

In this embodiment, if it is determined that the task is successfully processed, the task state of the task in the task event library may be updated to be successful.

And S604, if the task processing is determined to fail, re-processing the task according to the re-processing strategy.

Specifically, the reprocessing of the task according to the reprocessing policy includes:

a1, determining whether the current re-processing time is less than the preset processing time according to the preset processing time.

a2, when the number of times of reprocessing is less than the preset number of times of processing, repeatedly executing the operation of processing the task.

In this embodiment, the task processing end needs to configure configuration information related to task processing, such as a maximum retry number (a preset processing number), a maximum retry interval, a retry interval, an exponential increase interval time, an exponential coefficient, and an expiration time, in the configuration center.

The maximum retry number is the maximum number of times that the task can be processed again.

The maximum retry interval is a unit of a maximum time interval for reprocessing a task, that is, a time interval between two adjacent reprocessing cannot exceed the maximum retry interval.

The retry interval is the minimum time interval unit between two consecutive re-processes of a task.

The exponential increase interval time is a time interval increased on the basis of a minimum time interval unit between two adjacent retreatments.

The exponential coefficient is the degree of change in the exponential increase interval time.

The expiration time is the latest processing time of the task.

Specifically, determining the next reprocessing time of the task includes: determining a time interval between two adjacent reprocessing times according to the retry interval, the exponential increase interval time and the exponential coefficient; and increasing the determined time interval between two adjacent reprocessing times on the basis of the current reprocessing time to obtain the next reprocessing time.

The reprocessing strategy is described below by way of example:

assuming that the retry interval is 1 second, the exponential increase interval is 1 second, the exponential coefficient is 2, the maximum retry interval is 5 seconds, and the first reprocessing time of the task is 12:00:00, if the first reprocessing fails, the second reprocessing time is calculated as: if the index increment interval is 1 second and the index coefficient is 2, the time interval between the second reprocessing time and the first reprocessing time is 1 second by 1 second according to the retry interval, the index increment interval and the index coefficient; adding 1 second on the basis of 12:00:00, wherein the second reprocessing time is 12:00:01, and in the case of failure of the second reprocessing, the time interval between the third reprocessing time and the second reprocessing time is 1 second 2 second according to the retry interval, the index increment interval time and the index coefficient; 2 seconds are added on the basis of 12:00:01, and the third reprocessing time is 12:00: 03.

And if the task is determined to be successfully processed, the task state of the task in the task event library still needs to be updated to be successful.

Specifically, if the task corresponds to the next reprocessing time, the operation of processing the task in step a2 is repeatedly executed, including:

a21, if the time after the current repeated processing reaches the next re-processing time of the task, sending the task to the head or the tail of the task queue.

a22, if the time after the current repeated processing does not reach the next re-processing time of the task, sending the task to the delay queue, and transferring the task in the delay queue to the head or the tail of the task queue under the condition that the time after the current repeated processing reaches the next re-processing time of the task.

The delay queue comprises a plurality of delay threads, the next reprocessing time of the task, namely the time interval from the current task execution time to the next execution time, is calculated when the task is reprocessed, then the plurality of delay threads are established locally, and the delay processing is carried out on the tasks which are not executed next time according to the time interval.

a3, when the number of times of reprocessing reaches the preset processing number, sending the task to the failure queue for processing.

In step a3, the task status of the task in the task event library also needs to be updated to failure.

In the embodiment, the task to be processed in the task queue is obtained, and the task is an order task to be supplemented with order information or a task to be added with article information; and calling a first task processing thread established in advance to process the task, and sending a processing result of the task to a downstream system of a service corresponding to the task under the condition of determining that the task is successfully processed. In the embodiment, the tasks are acquired from the task queue and are processed through the first task processing thread, and the tasks in the task queue do not exist in the task queue after being dequeued, so that the number of the tasks in the task queue can be basically kept constant, and the problem that the database pressure is too large due to frequent query of the database task table by the timed tasks can be avoided. In addition, the task processing efficiency can be improved by performing the task processing in parallel by the plurality of task processing threads. In addition, the reprocessing strategy can improve the success rate of task processing.

On the basis of the embodiment shown in fig. 6, fig. 7 is a flowchart three of a task processing method provided in the embodiment of the present application, and as shown in fig. 7, when the number of times of reprocessing reaches a preset number of times of processing, a task is sent to a failure queue for processing, including the following steps:

s701, acquiring the tasks in the failure queue.

In this embodiment, a user of a task submitting end needs to configure, in a configuration center, a corresponding relationship between an identifier of the task submitting end and a failure queue, so that after a device of the task submitting end is started, the corresponding relationship between the identifier of the task submitting end and the failure queue is periodically and polled from the configuration center to obtain a task from the failure queue for processing.

In some embodiments, step S701 includes: determining a failure queue corresponding to the identifier of the task submitting end according to the identifier of the task submitting end and the corresponding relation between the identifier of the task submitting end and the failure queue; establishing communication connection with the determined failure queue; monitoring tasks to be processed in the determined failure queue; and if the task to be processed in the determined failure queue is monitored, acquiring the task to be processed from the determined failure queue.

S702, calling a second task processing thread which is established in advance to process the tasks in the failure queue, and determining whether the tasks in the failure queue are processed successfully.

In addition, the task processing end needs to pre-configure configuration information for processing the tasks in the failure queue in the configuration center, which specifically includes the number of second task processing threads, the pull interval of the tasks in the failure queue, a reprocessing strategy of the tasks in the failure queue, a failure strategy, the number of delay threads, task monitoring configuration, a current limiting strategy, and a task failure processor. Step S702 includes: establishing a corresponding number of second task processing threads according to the number of the second task processing threads configured in advance; each second task processing thread pulls the task from the failure queue according to the pulling interval of the task in the failure queue; each second task processing thread processes the task pulled from the failure queue.

And S703, if the task in the failure queue is successfully processed, sending the processing result of the task in the failure queue to a downstream system of the service corresponding to the task.

In some embodiments, for an order task for supplementing order information required for ex-warehouse, if it is determined that the order information required for ex-warehouse of an order is successfully supplemented, the supplemented order is sent to an ex-warehouse system.

In some other embodiments, for an order task that supplements order information required for charging, if it is determined that the order information required for charging the order is successfully supplemented, the supplemented order is sent to the charging system.

Of course, the supplemented order may also be stored in the database server in fig. 2 and recalled from the database server by the ex-warehouse system or the billing system.

S704, if the task processing in the failure queue fails, determining whether the repeated processing times reach preset repeated times;

s705, under the condition that the repeated processing times are less than the preset repeated times, the operation of calling a second task processing thread established in advance to process the tasks in the failure queue is executed again.

And S706, under the condition that the repeated processing times reach the preset repeated times, setting the task state of the tasks in the failure queue in the task event library as a failure state.

Similarly, the task processing end needs to pre-configure a preset number of times of repetition in the configuration center, and re-process the task according to the failure policy when the task processing in the failure queue fails. Specifically, the task is reprocessed according to the failure policy, which includes step S704 and step S705.

In the embodiment, the task which fails to be processed in the task queue is continuously processed through the failure queue, so that the success rate of task processing can be further improved.

Referring to fig. 4, the task compensation part in fig. 4 shows a task compensation strategy, which specifically refers to regularly pulling the task that is not processed due to timeout in the task event library to perform task processing, so as to avoid the situation of timeout caused by long-time non-processing of the task. The specific implementation process is as follows:

on the basis of the embodiments shown in fig. 6 and fig. 7, fig. 8 is a fourth flowchart of a task processing method provided in the embodiments of the present application, and as shown in fig. 8, the task processing method further includes the following steps:

s801, determining the task with the task state being unprocessed in the task event library.

Specifically, in this embodiment, a task in which the task state in the task event library is an unprocessed state is determined according to the task state in the task snapshot information of the task written in the task event library by the task submitting end.

S802, determining a target task in the tasks in the unprocessed state, wherein the target task is a task of which the time difference between the expiration time of the task and the current time is less than or equal to the preset time.

In this embodiment, the task processing end needs to pre-configure configuration information related to task compensation processing, such as the expiration time and the preset time of the task, in the configuration center, and determine, as a target task, a task in which the time difference between the expiration time of the task and the current time is smaller than or equal to the preset time, among tasks in the unprocessed state, where the target task is an overtime unprocessed task.

And S803, acquiring the target task according to the preset task acquisition information.

In this embodiment, the task processing end needs to pre-configure task acquisition information such as an access start time processor (default is minus 1, no limitation is set), an access end time processor (default is current time minus 10 minutes, that is, a task submitted before 10 minutes), and the number of tasks pulled each time in the configuration center.

Step S803 includes: calculating a start time according to the access start time processor; calculating the end time according to the access end time processor; acquiring target tasks which are within the range of the starting time and the ending time and the number of which does not exceed the number of the tasks pulled each time according to the starting time, the ending time and the number of the tasks pulled each time; the method steps of the task processing in the embodiments shown in fig. 6 and 7 are executed for the target task, and if the task processing is successful, the task state of the task in the task event library is updated to be successful, and if the task processing is failed, the task state of the task in the task event library is updated to be failed.

And S804, processing the acquired target task according to a pre-established third task processing thread.

In this embodiment, the task processing end needs to pre-configure configuration information related to task compensation processing, such as the number of third task processing threads, in the configuration center. Step S805 includes: establishing a corresponding number of third task processing threads according to the number of the preconfigured third task processing threads; each third task processing thread pulls the target task from the task event library according to the pulling interval of the target task; and each third task processing thread processes the target task pulled from the task event library.

With continued reference to fig. 4, in some alternative embodiments, a synchronization service may also be provided, where the synchronization service is configured to provide a function for a user of the task processing end to manually invoke the task compensation service to retry a single or multiple timed-out unprocessed tasks.

Optionally, before task compensation, the following configuration needs to be performed in a configuration center:

and (3) configuring an ES library: when the task is compensated, the configuration related to the connection with the ES database, such as a Uniform Resource Locator (URL), a user name, a password, and other connection information, is performed.

Task list: the configuration is similar to the configuration used by the task submitting end and the task processing end for finding the corresponding task queue, and the difference is that the task submitting end and the task processing end find the corresponding queue according to the task type, and the task compensation is to find the task information in the ES library according to the task type.

CRON: is a general expression, such as: 00/2? Indicating that task compensation is performed every 2 minutes. The access start time processor and the access end time processor are used for inquiring the task to be compensated when the task is compensated, and the CRON is used for configuring the execution frequency of the inquiry task.

In the embodiment, the task in the task event library is determined to be in the unprocessed state, the target task in which the time difference between the expiry time of the task and the current time is less than or equal to the preset time is determined to be in the unprocessed state, the target task is obtained according to the preset time period, and the obtained target task is processed according to the pre-established third task processing thread. In the embodiment, the task processing is performed on the target task in which the time difference between the expiration time of the task and the current time is less than or equal to the preset time in the unprocessed task, so that the situation that the task is unprocessed for a long time can be reduced, the task processing timeliness is guaranteed, the task processing efficiency is improved, and the order for supplementing the order information can be rapidly issued to the downstream system in the order scene.

On the basis of the above task processing method embodiment, fig. 9 is a schematic structural diagram of a processing device for task transmission according to an embodiment of the present application. As shown in fig. 9, the processing device for task transmission includes: an obtaining module 91, a determining module 92 and a sending module 93; the acquiring module 91 is configured to acquire a task to be processed, where the task is an order task to be supplemented with order information or a task to be added with item information, and the task includes a task type; a determining module 92, configured to determine a task queue corresponding to the task according to the task type of the task and a correspondence between a preset task type and the task queue; and a sending module 93, configured to send the task to a task queue corresponding to the task.

In one possible design, the apparatus further includes: the sending module 93 is further configured to perform the following operations: if the task transmission fails, retransmitting the task to the task queue according to a retry transmission strategy; wherein the retry sending policy comprises: under the condition that the retry times are less than the preset times, repeatedly executing the operation of sending the task to the task queue; and sending an abnormal operation prompt under the condition that the retry times reach the preset times.

In one possible design, the task further includes task snapshot information, and the task snapshot information includes first task snapshot information and second task snapshot information, and the apparatus further includes: a write module 94 for performing the following operations: writing first task snapshot information for a task to a task event repository, the first task snapshot information including at least one of: writing a task number, a number of a service corresponding to the task, a task name, data of the service corresponding to the task, an IP address of a task submitting end, task submitting time and a task state into a task event library; and after the task is sent to the task queue again every time, writing second task snapshot information of the task into the task event library, wherein the second task information comprises the retry times of sending the task to the task queue and the next resending time of the task.

In one possible design, the write module 94 is further configured to: if the writing of the task snapshot information fails, rewriting the task snapshot information of the task into the task event library according to a rewriting strategy; wherein the rewrite strategy includes: under the condition that the rewriting times are less than the preset writing times, repeatedly executing the operation of writing the task snapshot information of the task into the task event library; and under the condition that the rewriting times reach the preset writing times, discarding the task snapshot information of the task.

The processing device for task sending provided in the embodiment of the present application may be used to implement the technical solution of the processing method for task sending in the above embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.

On the basis of the above task processing method embodiment, fig. 10 is a schematic structural diagram of a task processing device according to an embodiment of the present application. As shown in fig. 10, the task processing device includes:

an obtaining module 101, configured to obtain a task to be processed in the task queue, where the task is an order task to be supplemented with order information or a task to be added with item information;

the processing module 102 is configured to invoke a first task processing thread established in advance to process the task, and determine whether the task is successfully processed;

a sending module 103, configured to send a processing result of the task to a downstream system of a service corresponding to the task if it is determined that the task is successfully processed.

In one possible design, the processing module 102 is further configured to perform the following operations: if the task processing fails, processing the task again according to a reprocessing strategy; wherein the reprocessing strategy comprises: if the reprocessing times are less than the preset processing times, repeatedly executing the operation of processing the task; and if the reprocessing times reach the preset processing times, sending the task to a failure queue for processing.

In a possible design, the task corresponds to a next reprocessing time, and the sending module 103 is further configured to: if the time after the current repeated processing reaches the next repeated processing time of the task, sending the task to the head or the tail of the task queue; and if the time after the current repeated processing does not reach the next re-processing time of the task, sending the task to a delay queue, and transferring the task in the delay queue to the head or the tail of the task queue under the condition that the time after the current repeated processing reaches the next re-processing time of the task.

In a possible design, the obtaining module 101 is further configured to obtain the tasks in the failure queue; the processing module 102 is further configured to invoke a second task processing thread established in advance to process the tasks in the failure queue, and determine whether the tasks in the failure queue are successfully processed; the sending module 103 is further configured to send a processing result of the task in the failure queue to a downstream system of a service corresponding to the task if the task in the failure queue is successfully processed; if the task processing in the failure queue fails, under the condition that the repeated processing times are less than the preset repeated times, re-executing the operation of calling a pre-established second task processing thread to process the task in the failure queue; and if the task processing in the failure queue fails, setting the task state of the task in the failure queue in the task event library as a failure state under the condition that the repeated processing times reach the preset repeated times.

In one possible design, the apparatus further includes: a determination module 104;

the acquiring module 101 is further configured to acquire a task in which a task state in the task event library is an unprocessed state;

a determining module 104, configured to determine a target task in the unprocessed tasks, where the target task is a task whose time difference between an expiration time of the task and a current time is less than or equal to a preset time;

the obtaining module 101 is further configured to obtain the target task according to preset task obtaining information;

the processing module 102 is further configured to process the acquired target task according to a pre-established third task processing thread.

The task processing device provided in the embodiment of the present application may be used to implement the technical solution of the task processing method in the foregoing embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.

It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the processing module 102 may be a separate processing element, or may be integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a processing element of the apparatus calls and executes the functions of the processing module 102. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element here may be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.

Fig. 11 is a schematic structural diagram of a computer device according to an embodiment of the present application. As shown in fig. 11, the computer apparatus may include: a processor 111, a memory 112, and a transceiver 113.

The processor 111 executes computer-executable instructions stored in the memory, causing the processor 111 to perform the aspects of the embodiments described above. The processor 111 may be a general-purpose processor including a central processing unit CPU, a Network Processor (NP), and the like; but also a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components.

A memory 112 is coupled to the processor 111 via the system bus and is in communication with each other, the memory 112 being used to store computer program instructions.

The transceiver 113 may be used to retrieve the task to be processed.

The system bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The transceiver is used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). The memory may include Random Access Memory (RAM) and may also include non-volatile memory (non-volatile memory).

The computer device provided in the embodiment of the present application may be used to execute the processing method for task sending or the technical scheme of the task processing method in the foregoing embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.

The embodiment of the application also provides a chip for running the instructions, and the chip is used for executing the processing method for sending the task or the technical scheme of the task processing method in the embodiment.

An embodiment of the present application further provides a computer-readable storage medium, where a computer instruction is stored in the computer-readable storage medium, and when the computer instruction runs on a computer, the computer is enabled to execute the processing method for sending the task or the technical solution of the task processing method in the foregoing embodiment.

The embodiment of the present application further provides a computer program product, where the computer program product includes a computer program, the computer program is stored in a computer-readable storage medium, at least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the at least one processor can implement the processing method for task transmission or the technical solution of the task processing method in the foregoing embodiments.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims (15)

1. A processing method for task sending is applied to a task submitting end, and the method comprises the following steps:

acquiring a task to be processed, wherein the task is an order task to be supplemented with order information or a task to be added with article information, and the task comprises a task type;

determining a task queue corresponding to the task according to the task type of the task and the corresponding relation between the preset task type and the task queue;

and sending the task to a task queue corresponding to the task.

2. The method of claim 1, further comprising:

if the task transmission fails, retransmitting the task to the task queue according to a retry transmission strategy; wherein the retry sending policy comprises:

under the condition that the retry times are less than the preset times, repeatedly executing the operation of sending the task to the task queue;

and sending an abnormal operation prompt under the condition that the retry times reach the preset times.

3. The method of claim 2, wherein the task further includes task snapshot information, the task snapshot information including first task snapshot information and second task snapshot information, and after the task is sent to a task queue corresponding to the task, the method further comprises:

writing first task snapshot information for a task to a task event repository, the first task snapshot information including at least one of: writing a task number, a number of a service corresponding to the task, a task name, data of the service corresponding to the task, an IP address of a task submitting end, task submitting time and a task state into a task event library;

and the number of the first and second groups,

and after the task is retransmitted to the task queue every time, writing second task snapshot information of the task into the task event library, wherein the second task information comprises the retry times of the task transmitted to the task queue and the next retransmission time of the task.

4. The method of claim 3, further comprising:

if the writing of the task snapshot information fails, rewriting the task snapshot information of the task into the task event library according to a rewriting strategy;

wherein the rewrite strategy includes:

under the condition that the rewriting times are less than the preset writing times, repeatedly executing the operation of writing the task snapshot information of the task into the task event library;

and under the condition that the rewriting times reach the preset writing times, discarding the task snapshot information of the task.

5. A task processing method is applied to a task processing end, wherein the task processing end corresponds to a task queue, and the method comprises the following steps:

acquiring a task to be processed in the task queue, wherein the task is an order task to be supplemented with order information or a task to be added with article information;

calling a first task processing thread established in advance to process the task, and determining whether the task is successfully processed;

and if the task is successfully processed, sending the processing result of the task to a downstream system of the service corresponding to the task.

6. The method of claim 5, further comprising:

if the task processing fails, processing the task again according to a reprocessing strategy;

wherein the reprocessing strategy comprises:

if the reprocessing times are less than the preset processing times, repeatedly executing the operation of processing the task;

and if the reprocessing times reach the preset processing times, sending the task to a failure queue for processing.

7. The method of claim 6, wherein the task corresponds to a next reprocessing time, and the repeatedly performing the operation of processing the task comprises:

if the time after the current repeated processing reaches the next repeated processing time of the task, sending the task to the head or the tail of the task queue;

and if the time after the current repeated processing does not reach the next re-processing time of the task, sending the task to a delay queue, and transferring the task in the delay queue to the head or the tail of the task queue under the condition that the time after the current repeated processing reaches the next re-processing time of the task.

8. The method of claim 7, further comprising:

acquiring tasks in the failure queue;

calling a pre-established second task processing thread to process the tasks in the failure queue and determining whether the tasks in the failure queue are successfully processed;

if the task in the failure queue is successfully processed, the processing result of the task in the failure queue is sent to a downstream system of the service corresponding to the task;

if the task processing in the failure queue fails, under the condition that the repeated processing times are less than the preset repeated times, re-executing the operation of calling a pre-established second task processing thread to process the task in the failure queue;

and if the task processing in the failure queue fails, setting the task state of the task in the failure queue in the task event library as a failure state under the condition that the repeated processing times reach the preset repeated times.

9. The method according to any one of claims 5-8, further comprising:

acquiring a task with an unprocessed task state in a task event library;

determining a target task in the tasks in the unprocessed state, wherein the target task is a task of which the time difference between the expiration time of the task and the current time is less than or equal to the preset time;

acquiring the target task according to preset task acquisition information;

and processing the acquired target task according to a pre-established third task processing thread.

10. A task transmission processing apparatus, comprising:

the acquisition module is used for acquiring a task to be processed, wherein the task is an order task to be supplemented with order information or a task to be added with article information, and the task comprises a task type;

the determining module is used for determining a task queue corresponding to the task according to the task type of the task and the corresponding relation between the preset task type and the task queue;

and the sending module is used for sending the task to a task queue corresponding to the task.

11. A task processing device, wherein the task processing device corresponds to a task queue, comprising:

the acquisition module is used for acquiring a task to be processed in the task queue, wherein the task is an order task to be supplemented with order information or a task to be added with article information;

the processing module is used for calling a first task processing thread established in advance to process the task and determining whether the task is successfully processed;

and the sending module is used for sending the processing result of the task to a downstream system of the service corresponding to the task if the task is successfully processed.

12. A task processing system, comprising:

processing means for task delivery as set forth in claim 10;

the task processing device of claim 11;

the task queue database is used for storing a plurality of task queues;

the configuration center database is used for storing the processing device sent by the task and task configuration information of the task processing device, wherein the task configuration information of the processing device sent by the task comprises a corresponding relation between a task type and a task queue, and the task configuration information of the task processing device comprises a corresponding relation between an identifier of the task processing device and the task queue;

and the task event information database is used for storing task snapshot information of the task.

13. A computer device, comprising: a memory, a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to implement the method of any one of claims 1 to 9.

14. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor, are configured to implement the method of any one of claims 1 to 9.

15. A computer program product, characterized in that it comprises a computer program which, when executed by a processor, implements the method of any one of claims 1 to 9.

Images