CN113064661B - Batch task memory processing method, device, electronic device and storage medium - Google Patents
Batch task memory processing method, device, electronic device and storage medium Download PDFInfo
- Publication number
- CN113064661B CN113064661B CN201911425296.0A CN201911425296A CN113064661B CN 113064661 B CN113064661 B CN 113064661B CN 201911425296 A CN201911425296 A CN 201911425296A CN 113064661 B CN113064661 B CN 113064661B
- Authority
- CN
- China
- Prior art keywords
- task
- subtasks
- callback
- memory
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44557—Code layout in executable memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
本发明涉及数据处理技术领域，提供批量任务的内存处理方法、装置、电子设备及存储介质，所述方法包括步骤：任务逻辑处理区对目标批量任务进行创建，并初始化所述目标批量任务，所述目标批量任务包括多个子任务；注册所述目标批量任务的子任务，并添加所述子任务的监听参数以及回调参数；在任务响应区中通过内存执行所述子任务对应的动作，并根据所述监听参数以及回调参数对所述子任务对应的动作进行状态监听以及状态回调；通过所述子任务的状态监听以及状态回调判断所述目标批量任务是否完成，并在完成后对所述目标批量任务进行存档。本发明实施例能够降低任务处理过程中的耦合性，提高工作效率。
The present invention relates to the field of data processing technology, and provides a memory processing method, device, electronic device and storage medium for batch tasks, wherein the method comprises the following steps: a task logic processing area creates a target batch task and initializes the target batch task, wherein the target batch task comprises a plurality of subtasks; registers the subtasks of the target batch task, and adds monitoring parameters and callback parameters of the subtasks; executes the actions corresponding to the subtasks through memory in the task response area, and performs status monitoring and status callback on the actions corresponding to the subtasks according to the monitoring parameters and callback parameters; determines whether the target batch task is completed through the status monitoring and status callback of the subtasks, and archives the target batch task after completion. The embodiments of the present invention can reduce the coupling in the task processing process and improve work efficiency.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and apparatus for processing a memory of a batch task, an electronic device, and a storage medium.
Background
Currently, batch processing tasks such as batch file uploading, excel batch processing tasks and the like are often required to be performed. In the existing processing mode, the logic processing of the task and the business codes are mutually coupled, and the logic processing of the task and the business codes can be independently realized by utilizing the middleware, but the complexity and the management difficulty are increased when the use of the middleware is small in the case of processing the task amount. And the high coupling can cause the interaction between the logic processing of the task and the service code. Therefore, the prior art has the problems of high coupling and low working efficiency.
Disclosure of Invention
The embodiment of the invention provides a memory processing method for batch tasks, which can reduce the coupling in the task processing process and improve the working efficiency.
In a first aspect, an embodiment of the present invention provides a method for processing a memory of a batch task, where the method includes the following steps:
creating a target batch task in a task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks;
registering subtasks of the target batch task, and adding monitoring parameters and callback parameters of the subtasks;
executing actions corresponding to the subtasks through the memory in a task response area, and performing state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and callback parameters;
judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after finishing.
In a second aspect, an embodiment of the present invention further provides a memory processing device for batch tasks, including:
The creating module is used for creating a target batch task in the task logic processing area and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks;
The registration module is used for registering subtasks of the target batch task and adding monitoring parameters and callback parameters of the subtasks;
The execution module is used for executing the actions corresponding to the subtasks through the memory, and carrying out state monitoring and state callback on the actions corresponding to the subtasks in the task logic processing area according to the monitoring parameters and the callback parameters;
And the detection module is used for judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after the completion.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements steps in a memory processing method of batch tasks provided by the embodiment when the processor executes the computer program.
In a fourth aspect, a computer readable storage medium has a computer program stored thereon, which when executed by a processor, implements the steps in the memory processing method for batch tasks provided by the embodiments.
In the embodiment of the invention, a target batch task is created in a task logic processing area, the target batch task is initialized, the target batch task comprises a plurality of subtasks, the subtasks of the target batch task are registered, monitoring parameters and callback parameters of the subtasks are added, actions corresponding to the subtasks are executed in a task response area through a memory, state monitoring and state callback are carried out on the actions corresponding to the subtasks according to the monitoring parameters and the callback parameters, whether the target batch task is completed or not is judged through the state monitoring and the state callback of the subtasks, and the target batch task is archived after the completion. The embodiment of the invention creates the target batch task, initializes the target batch task, registers the subtask in the target batch task and adds the monitoring parameter and the callback parameter in the task logic processing area by creating the task logic processing area and the task response area, namely, after completing the logic processing, the subtask is transmitted to the task response area, the action corresponding to the subtask is executed through the memory, and the service code processing is completed through the monitoring process of the monitoring parameter and the feedback is carried out through the callback parameter, thus, the coupling performance in the task processing process can be reduced, the mutual interference between the logic processing and the service code processing is avoided, and the working efficiency of processing the batch task is favorably improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a memory processing method for batch tasks according to an embodiment of the present invention;
FIG. 2 is a flowchart of another method for processing a memory of a batch task according to an embodiment of the present invention;
FIG. 3 is a flowchart of another method for processing a memory of a batch task according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a memory processing device for batch tasks according to an embodiment of the present invention;
Fig. 5 is a schematic diagram of the structure of the creation module 401 in fig. 4;
fig. 6 is a schematic diagram of the registration module 402 in fig. 4;
FIG. 7 is a schematic diagram of the execution module 403 in FIG. 4;
FIG. 8 is another schematic diagram of the execution module 403 of FIG. 4;
FIG. 9 is a schematic diagram of another memory processing device for batch tasks according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of the structure of the detection module 404 in FIG. 4;
Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "comprising" and "having" and any variations thereof in the description and claims of the application and in the description of the drawings are intended to cover a non-exclusive inclusion. The terms first, second and the like in the description and in the claims or drawings are used for distinguishing between different objects and not for describing a particular sequential order. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
As shown in fig. 1, fig. 1 is a flowchart of a memory processing method for a batch task according to an embodiment of the present invention, where the memory processing method for a batch task includes the following steps:
101. creating a target batch task in a task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks.
In this embodiment, the electronic device on which the memory processing method for batch tasks is running may acquire information such as tasks through a wired connection manner or a wireless connection manner. It should be noted that the Wireless connection may include, but is not limited to, a 3G/4G connection, a WiFi (Wireless-Fidelity) connection, a bluetooth connection, wiMAX (Worldwide Interoperability for Microwave Access) connection, a Zigbee (low power lan protocol, also known as the purple peak protocol) connection, UWB (ultra wideband) connection, and other now known or later developed Wireless connection methods.
The task logic processing area may be a plug-in unit for performing logic processing on the task. The task logic processing area can be configured with a task interface, the task interface can be a network interface or a local interface, the task interface can receive a request task sent by a user through the electronic equipment, and the final processing result can be transmitted back to the electronic equipment which sends out the task request to the user through the task interface for use by the user.
The embodiment of the invention can carry out batch processing of tasks aiming at the condition of smaller data volume, and can preset a data volume threshold of the data volume received by the task interface for judging the size of the data volume, so long as the data volume threshold is met, the tasks can be received. The target batch task can be that after the batch task request is obtained, a corresponding target batch task can be created according to the batch task request, and the creation task is convenient for carrying out logic processing and data calculation on the task request contained in the task in a task mode.
The target batch task includes a plurality of subtasks, and the initializing the target batch task may mean counting data amounts of the plurality of subtasks included in the target batch task, may further include performing a slicing process on the target batch task to decompose the target batch task into a plurality of subtasks, and may further include transmitting the initialized subtasks to a subtask portion for registering the target batch task.
102. Registering subtasks of the target batch task, and adding monitoring parameters and callback parameters of the subtasks.
The subtasks of the registration target batch task can be expressed as registering a task identifier for each subtask, and subtasks corresponding to the task identifiers can be directly searched through searching the task identifiers. The above-mentioned monitoring parameters may be used to monitor the task state of each subtask, and the task monitoring parameters may be registered in the task monitor. The Callback parameter may be a Callback Stub (Callback Stub), and the addition Callback parameter may perform state Callback.
103. And executing actions corresponding to the subtasks through the memory in the task response area, and performing state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and the callback parameters.
The task response area is specially used for responding to the subtasks, and data calculation is carried out on the subtasks through the service codes. And, the action corresponding to the execution subtask can be executed in the memory of the task response area. The corresponding action of executing the subtask is to perform corresponding data calculation on the subtask, for example, if the subtask request is to acquire a picture, the process of executing the data calculation may be to acquire the picture first, then analyze the characteristics of the picture, and so on.
The state of executing the subtasks can be monitored through monitoring parameters, and the monitoring can be timing monitoring or real-time monitoring. The current process state can be obtained at any time through monitoring the monitoring parameters, and the monitoring parameters are in one-to-one correspondence with the subtasks, namely one subtask adds one monitoring parameter. When the current state of the subtask is monitored to be the completed state, the Callback parameter can be called to call the current completed state through a Callback entry Callback, and the Callback is returned through the Callback entry Callback. And corresponding one callback parameter to each subtask.
104. Judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after finishing.
In the task response area, each time a subtask is acquired, a monitoring parameter corresponding to the subtask is called to perform state monitoring, and meanwhile, a callback parameter is called to acquire a processing result after the action corresponding to the subtask is completed to perform state callback. If the monitor parameter does not continue to work, it may indicate that all subtasks in the target batch task have been fully processed. At this time, all the subtasks that have been processed may be stored in the memory.
In the embodiment of the invention, a target batch task is created in a task logic processing area and initialized, the target batch task comprises a plurality of subtasks, the subtasks of the target batch task are registered, monitoring parameters and callback parameters of the subtasks are added, actions corresponding to the subtasks are executed through a memory in a task response area, state monitoring and state callback are carried out on actions corresponding to the subtasks according to the monitoring parameters and the callback parameters, whether the target batch task is completed or not is judged through the state monitoring and the state callback of the subtasks, and the target batch task is archived after the completion. The embodiment of the invention creates the target batch task, initializes the target batch task, registers the subtask in the target batch task and adds the monitoring parameter and the callback parameter in the task logic processing area by creating the task logic processing area and the task response area, namely, after completing the logic processing, the subtask is transmitted to the task response area, the action corresponding to the subtask is executed through the memory, and the service code processing is completed through the monitoring process of the monitoring parameter and the feedback is carried out through the callback parameter, thus, the coupling performance in the task processing process can be reduced, the mutual interference between the logic processing and the service code processing is avoided, and the working efficiency of processing the batch task is favorably improved.
As shown in fig. 2, fig. 2 is a flowchart of another method provided in an embodiment of the present invention, including the following steps:
201. and creating the batch tasks in the task logic processing area to obtain the target batch tasks.
The batch task may represent that a task interface in the service logic processing area receives a large number of task requests at the same time, for example, task requests respectively sent by a plurality of users to the task interface or a plurality of task requests sent by a user to the task interface. In order to perform orderly and rapid processing on task requests in batches, different types of task requests can be classified and marked, the classification and marking can be distinguished according to the task identifications, task requests belonging to the same data type in all task identifications are clustered, for example, the data type is a picture acquisition type, the requests belonging to the picture acquisition in the task requests can be clustered into the picture acquisition type, the data type is a data structuring processing type, and the request types needing to perform structuring processing on data in all task requests are clustered into the data structuring processing type. By classifying the task requests, corresponding tasks can be created corresponding to the task requests in each data type, and after all tasks are created, target batch tasks can be extracted from the created tasks according to time sequence.
202. Initializing statistical parameters of the target batch task and initializing finishing post-processing of the target batch task, wherein the target batch task comprises a plurality of subtasks.
After the target batch task is obtained, the target batch task can be subjected to slicing processing, namely, one target batch task is respectively divided into a plurality of subtasks according to different processing objects in the process to be executed, so that a pipeline of subtasks corresponding to the target batch task is formed. Each subtask in the pipeline may be processed sequentially in the task response area.
The statistical parameter may represent the data amount of the subtasks, and after decomposing the target batch task into a plurality of subtasks, the data amount of the subtasks may be counted, for example, the data amount of the subtasks is 10, 20, etc. The post-completion processing of the initialization target batch task may mean that after the sub-tasks are segmented into a plurality of sub-tasks, the sub-tasks are sequentially transferred from the task logic processing area to the memory for storage.
203. Registering subtasks of the target batch task, and adding monitoring parameters and callback parameters of the subtasks.
204. And executing actions corresponding to the subtasks through the memory in the task response area, and performing state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and the callback parameters.
205. Judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after finishing.
Optionally, the step 203 includes:
each subtask in the target batch task is registered respectively.
The registering may be represented by registering a task identifier for each subtask, and distinguishing the subtasks by the task identifier, for example, subtask-01, subtask-02, subtask-03.
And adding the monitoring parameters and callback parameters which are corresponding to the number of each subtask.
After registering each subtask, a task identifier of each subtask may be obtained, and a corresponding monitoring parameter and a callback parameter are added for the subtask corresponding to each task identifier, for example, the subtask is a subtask-01, the corresponding monitoring parameter is a monitor-01, the corresponding callback parameter is a callback-01, the subtask is a subtask-02, the corresponding monitoring parameter is a monitor-02, and the corresponding callback parameter is a callback-02. By registering corresponding task identifiers for each subtask and adding one-to-one monitoring parameters and callback parameters according to each task identifier, the state of the subtask is convenient to track and count, and the whole processing process can be orderly executed.
Optionally, the step 204 includes:
And analyzing the subtasks to obtain task data in the subtasks.
The task response area can actively remove the stored subtasks from the memory, and analyze the subtasks. The analysis of the subtasks can obtain the actions to be executed in the subtasks, namely the task data, for example, the task data is acquisition pictures, or the task data is target detection, or the task data is characteristic value extraction, or the task data is attribute analysis, and the like.
And executing data calculation corresponding to the task data in the memory of the task response area according to the task data.
When task data is analyzed, data calculation can be directly carried out on the task data in the memory, wherein the data calculation can comprise extracting features in sub-task data and calculating corresponding results, for example, if the task data is a search image, the images in the task data can be extracted, and the extracted information is compared with image information stored in the memory in advance to obtain a comparison result, and the comparison result is used as a response result of the sub-task corresponding to the task data.
In the embodiment of the invention, the task logic processing area and the task response area are created, the target batch task is created, the initialization is carried out on the target batch task, the subtasks in the target batch task are registered, and the monitoring parameters and the callback parameters are added, namely, after the logic processing is finished, the subtasks are transmitted to the task response area, the actions corresponding to the subtasks are executed through the memory, the monitoring process of the monitoring parameters is carried out, and the feedback is carried out through the callback parameters, so that the service code processing is finished, and therefore, the coupling performance in the task processing process can be reduced, the mutual interference between the logic processing and the service code processing is avoided, and the work efficiency of processing the batch task is improved.
As shown in fig. 3, fig. 3 is a flowchart of another memory processing method for batch tasks according to the embodiment of the present invention, which specifically includes the following steps:
301. Creating a target batch task in a task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks.
302. Registering subtasks of the target batch task, and adding monitoring parameters and callback parameters of the subtasks.
303. And executing actions corresponding to the subtasks through the memory in the task response area, and monitoring the states of the actions corresponding to the subtasks executed by the memory in the task response area according to the monitoring parameters.
The monitoring parameters are used for monitoring the action states corresponding to the execution subtasks in real time, so that the current state is monitored in real time, and whether the action corresponding to the subtask is completed or not, or what subtask is executed corresponding to the execution, and the like are judged. In the monitoring process, if the data calculation of the subtasks is started in the memory, the monitoring parameters generate corresponding execution information, and when the data calculation of the subtasks is monitored to be completed, the corresponding ending information is also generated.
304. If the action corresponding to the execution sub-task of the memory is monitored, the completed state is subjected to state callback through the callback parameter, and the completed state refers to the action corresponding to the execution sub-task of the memory.
When the action corresponding to the execution of the subtask in the memory is monitored, the memory is in a completed state, and the response to the current subtask which is processed is not continued, but the next subtask is continued to be responded. When the subtask is in the completed state, a Callback parameter can be called to Callback the completed state through a Callback entry Callback.
305. If the memory is monitored to be in a state of executing the action corresponding to the subtask, continuing monitoring according to the monitoring parameter, and not starting the callback parameter.
If the monitoring function monitors that the sub-task is in a state of executing the action of the corresponding function of the sub-task, the monitoring function continues to monitor, and corresponding response information is not generated before the sub-task is not processed, so that the callback function is not started.
306. Judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after finishing.
Optionally, after step 302, the method may further include:
and adding a data statistics table, wherein the data statistics table is used for counting the data quantity of the state callback and the data quantity of the subtasks.
The data statistics table can be added while the monitoring parameters and callback parameters of the subtasks are added, and the data statistics table can be attached to a preset statistics monitor, so that the statistics monitor is called to dynamically update the data volume as long as the data volume changes. The data statistics table can be used for counting the data quantity of all subtasks, and can be placed in the data statistics table after callback parameters callback the completed state. The subtask data quantity and the state callback data quantity can be respectively stored in different areas in the data statistics table, so that confusion is avoided.
And if the callback parameter is detected to carry out state callback on the completed state, updating the data quantity of the state callback in the data statistics table.
When each state is called back to the data statistics table, the data quantity of the subtasks in the data statistics table is updated in sequence until the data quantity of the subtasks in the task data table is remained to be 0, which means that data processing is completed for all the subtasks in the target batch task. The data size of the subtasks may be the remaining unprocessed data size of the subtasks, for example, the data size of the subtasks is 10, if a state callback enters the data statistics table, the data size of the subtasks is reduced by one to 9, and the subtasks are successively decremented until the data size of the subtasks is decremented to 0.
Optionally, the step 306 includes:
and detecting whether the sub-task data volume in the data statistics table reaches the target data volume.
The target data amount may be 0 data amount of the subtask in the data statistics table. Detecting whether the data volume of the subtasks reaches the target data volume may be used to determine whether complete all responses to all subtasks in the target batch task.
And if the data volume of the subtasks reaches the target data volume, generating response information which is processed on the target batch task.
If the data amount of the subtask is detected to reach the target data amount, the data amount of the subtask in the data statistics table is 0. At this time, response information may be generated in the memory, and the response information includes information that the processing of the target batch task has been completed, for example, the response information confirms that the processing has been completed for the target batch task.
And triggering a preset function to send the response information according to the response information, and archiving the target batch task.
When the generation of the response information is detected, a preset function can be triggered immediately to send the response information so as to inform a calling party that the processing of the target batch task is completed, and the target batch task can be stored in a memory. In addition, there may be a case where a request timeout occurs when an action corresponding to a subtask is executed, for which a corresponding preset function may be triggered to issue information of the request timeout, etc. The preset function may be a Hook function Hook, and the preset function may be triggered correspondingly according to specific situations, for example, the received response information is the information that the Hook function Hook is triggered to send and will be the information that the response is successful, and if the situation of delay request occurs, the Hook function Hook triggered correspondingly will be the information that the request is delayed or the request is failed.
In the embodiment of the method, the task logic processing area and the task response area are created, the target batch task is initialized, the subtasks in the target batch task are registered, and the monitoring parameters and the callback parameters are added, namely, after the logic processing is finished, the subtasks are transmitted to the task response area, the actions corresponding to the subtasks are executed through the memory, the monitoring process is carried out through the monitoring parameters, and the feedback is carried out through the callback parameters, so that the business code processing is finished, and therefore, the coupling performance in the task processing process can be reduced, the mutual interference between the logic processing and the business code processing is avoided, and the work efficiency of processing the batch task is improved.
As shown in fig. 4, fig. 4 is a schematic structural diagram of a memory processing device for batch tasks according to an embodiment of the present invention, where a memory processing device 400 for batch tasks includes:
the creation module 401 is configured to create a target batch task in the task logic processing area, and initialize the target batch task, where the target batch task includes a plurality of subtasks;
the registration module 402 is configured to register sub-tasks of the target batch task, and add monitoring parameters and callback parameters of the sub-tasks;
the execution module 403 is configured to execute an action corresponding to the subtask through the memory, and perform state monitoring and state callback on the action corresponding to the subtask in the task logic processing area according to the monitoring parameter and the callback parameter;
and the detection module 404 is used for judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after the completion.
Optionally, as shown in fig. 5, fig. 5 is a schematic structural diagram of the creation module 401 in fig. 4, where the creation module 401 includes:
The creating unit 4011 is used for creating batch tasks in the task logic processing area to obtain target batch tasks;
The initialization unit 4012 is used for initializing statistical parameters of the target batch task and performing post-completion processing of the target batch task.
Optionally, as shown in fig. 6, fig. 6 is a schematic structural diagram of the registration module 402 in fig. 4, where the registration module 402 includes:
A registration unit 4021 configured to register each subtask in the target batch task respectively;
the adding unit 4022 is configured to add and add a number of listening parameters and callback parameters corresponding to each subtask.
Optionally, as shown in fig. 7, fig. 7 is a schematic structural diagram of the execution module 403 in fig. 4, where the execution module 403 includes:
The parsing unit 4031 is configured to parse the subtasks to obtain task data in the subtasks;
the calculating unit 4032 is configured to perform data calculation corresponding to the task data in the memory of the task response area according to the task data.
Optionally, as shown in fig. 8, fig. 8 is another schematic structural diagram of the execution module 403 in fig. 4, where the execution module 403 further includes:
the execution unit 4033 is configured to monitor, according to the monitoring parameter, a state of an action corresponding to the execution subtask of the memory in the task response area;
The callback unit 4034 is configured to, if it is monitored that the memory has completed executing the action corresponding to the subtask, perform a state callback on the completed state through the callback parameter, where the completed state refers to the action corresponding to the completed execution of the subtask by the memory;
and the monitoring unit 4034 is configured to, if it is monitored that the memory is in a state of executing an action corresponding to the subtask, continue monitoring according to the monitoring parameter, and not enable the callback parameter.
Optionally, as shown in fig. 9, fig. 9 is a schematic structural diagram of another memory processing device for batch tasks according to an embodiment of the present invention, where the device 400 further includes:
The adding module 405 is configured to add a data statistics table, where the data statistics table is used to count the data amount of the state callback and the subtask data amount;
and the updating module 406 is configured to update the data amount of the state callback in the data statistics table if it is detected that the callback parameter performs the state callback on the completed state.
Alternatively, as shown in fig. 10, fig. 10 is a schematic structural diagram of the detection module 404 in fig. 4, where the detection module 404 includes:
a detection unit 4041, configured to detect whether the sub-task data amount in the data statistics table reaches the target data amount;
A generating unit 4042, configured to generate response information that the processing of the target batch task is completed if the subtask data volume reaches the target data volume;
And the storage unit 4043 is used for triggering a preset function to send the response information according to the response information and archiving the target batch task.
The memory processing device for batch tasks provided by the embodiment of the invention can realize each process realized by the memory processing method for batch tasks in the embodiment of the method and can achieve the same beneficial effects, and in order to avoid repetition, the description is omitted.
As shown in fig. 11, fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device 1100 includes a memory 1102, a processor 1101, a network interface 1103, and a computer program stored in the memory 1102 and capable of running on the processor 1101, and when the processor 1101 executes the computer program, the steps in the memory processing method for implementing batch tasks provided by the embodiment are implemented.
Specifically, the processor 1101 is configured to perform the following steps:
creating a target batch task in a task logic processing area, and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks;
Registering subtasks of the target batch task, and adding monitoring parameters and callback parameters of the subtasks;
Executing actions corresponding to the subtasks through the memory in the task response area, and performing state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and the callback parameters;
Judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after finishing.
Optionally, the steps executed by the processor 1101 to create a target batch task in the task logic processing zone and initialize the target batch task include:
Creating batch tasks in a task logic processing area to obtain target batch tasks;
Initializing statistical parameters of the target batch task and performing post-completion processing on the initialized target batch task.
Optionally, the steps of registering the subtasks of the target batch task and adding the monitoring parameters and callback parameters of the subtasks performed by the processor 1101 include:
registering each subtask in the target batch task respectively;
and adding the monitoring parameters and callback parameters which are corresponding to the number of each subtask.
Optionally, the steps executed by the processor 1101 to execute the actions corresponding to the subtasks in the task response area through the memory include:
analyzing the subtasks to obtain task data in the subtasks;
And executing data calculation corresponding to the task data in the memory of the task response area according to the task data.
Optionally, the steps performed by the processor 1101 to perform state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and the callback parameters include:
Monitoring the state of the action corresponding to the execution subtask of the memory in the task response area according to the monitoring parameter;
if the action corresponding to the execution sub-task of the memory is monitored, carrying out state callback on the completed state through callback parameters, wherein the completed state refers to the action corresponding to the execution sub-task of the memory;
If the memory is monitored to be in a state of executing the action corresponding to the subtask, continuing monitoring according to the monitoring parameter, and not starting the callback parameter.
Optionally, after registering the subtasks of the target batch task, the processor 1101 is further configured to perform:
adding a data statistics table, wherein the data statistics table is used for counting the data quantity of the state callback and the subtask data quantity;
and if the callback parameter is detected to carry out state callback on the completed state, updating the data quantity of the state callback in the data statistics table.
Optionally, the step of determining, by the state monitor and the state callback of the subtask, whether the target batch task is completed and archiving the target batch task after completion, which is performed by the processor 1101 includes:
Detecting whether the sub-task data volume in the data statistics table reaches the target data volume;
if the subtask data volume reaches the target data volume, generating response information which is processed on the target batch task;
And triggering a preset function to send the response information according to the response information, and archiving the target batch task.
The electronic device 1100 provided by the embodiment of the present invention can implement each implementation manner in the memory processing method embodiment of the batch task, and the corresponding beneficial effects, so that repetition is avoided, and no redundant description is provided herein.
It should be noted that only 1101-1103 with components are shown in the figures, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the electronic device 1100 herein is a device capable of automatically performing numerical calculations and/or information processing according to predetermined or stored instructions, and the hardware thereof includes, but is not limited to, microprocessors, application SPECIFIC INTEGRATED Circuits (ASICs), programmable gate arrays (Field-Programmable GATEARRAY, FPGA), digital processors (DIGITAL SIGNAL processors, DSPs), embedded devices, etc.
The electronic device 1100 may be a computing device such as a desktop computer, a notebook computer, or a palm top computer. The electronic device 1100 may interact with a user by way of a keyboard, mouse, remote control, touch pad, or voice control device.
Memory 1102 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 1102 may be an internal storage unit of the electronic device 1100, such as a hard disk or memory of the electronic device 1100. In other embodiments, the memory 1102 may also be an external storage device of the electronic device 1100, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. that are provided on the electronic device 1100. Of course, memory 1102 may also include both internal storage units and external storage devices of electronic device 1100. In this embodiment, the memory 1102 is typically used to store an operating system installed on the electronic device 1100 and various application software, such as program codes of a memory processing method for batch tasks. In addition, the memory 1102 can also be used to temporarily store various types of data that have been output or are to be output.
The processor 1101 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 1101 is generally used to control the overall operation of the electronic device 1100. In this embodiment, the processor 1101 is configured to execute program codes stored in the memory 1102 or process data, such as program codes of a memory processing method for executing batch tasks.
The network interface 1103 may include a wireless network interface or a wired network interface, the network interface 1103 typically being used to establish communication connections between the electronic device 1100 and other electronic devices.
The embodiment of the present invention further provides a computer readable storage medium, on which a computer program is stored, where each process in the memory processing method for batch tasks provided in the embodiment is implemented when the computer program is executed by the processor 1101, and the same technical effects can be achieved, so that repetition is avoided, and no redundant description is provided herein.
Those skilled in the art will appreciate that all or part of the processes in implementing the methods of the embodiments may be implemented by a computer program for instructing the relevant hardware, and the program may be stored in a computer readable storage medium, and the program may include processes as embodiments of the methods when executed. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory 1102 (Random Access Memory, RAM) or the like.
The foregoing disclosure is illustrative of the present invention and is not to be construed as limiting the scope of the invention, which is defined by the appended claims.
Claims (9)
1. The memory processing method for batch tasks is characterized by comprising the following steps:
the task logic processing area is used for creating a target batch task and initializing the target batch task, wherein the target batch task comprises a plurality of subtasks, the data size of the target batch task is smaller than a preset data size threshold, and the task logic processing area is a plug-in unit for performing logic processing on the task;
registering subtasks of the target batch task, and adding monitoring parameters and callback parameters of the subtasks;
Executing actions corresponding to the subtasks through the memory in a task response area, and performing state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and the callback parameters, wherein the task response area is specially used for responding to the subtasks, and the memory is the memory in the task response area;
Judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after finishing;
The step of performing state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and the callback parameters comprises the following steps:
monitoring the state of the memory in the task response area for executing the action corresponding to the subtask according to the monitoring parameter;
If the fact that the memory finishes executing the actions corresponding to the subtasks is monitored, carrying out state callback on the finished state through the callback parameters, wherein the finished state refers to the fact that the memory finishes executing the actions corresponding to the subtasks;
If the memory is monitored to be in a state of executing the action corresponding to the subtask, continuing monitoring according to the monitoring parameter, and not starting the callback parameter.
2. The method of claim 1, wherein the steps of creating a target batch task at a task logic processing zone and initializing the target batch task comprise:
creating batch tasks in a task logic processing area to obtain target batch tasks;
Initializing statistical parameters of the target batch task and initializing finishing post-processing of the target batch task.
3. The method of claim 1, wherein registering the subtasks of the target batch task and adding the listening parameters and callback parameters of the subtasks comprises:
registering each subtask in the target batch task respectively;
and adding the monitoring parameters and callback parameters which are corresponding to the number of each subtask.
4. The memory processing method of batch tasks according to claim 1, wherein the step of executing the actions corresponding to the subtasks in the task response area through the memory comprises:
Analyzing the subtasks to obtain task data in the subtasks;
And executing data calculation corresponding to the task data in the memory of the task response area according to the task data.
5. The memory processing method of batch task according to claim 1, wherein after registering the subtasks of the target batch task, the method further comprises:
Adding a data statistics table, wherein the data statistics table is used for counting the data quantity of the state callback and the data quantity of the subtask;
And if the callback parameter is detected to carry out state callback on the completed state, updating the data quantity of the state callback in the data statistics table.
6. The method for processing the memory of the batch task according to claim 5, wherein the step of judging whether the target batch task is completed or not through the state monitoring and the state callback of the subtask, and archiving the target batch task after completion comprises:
detecting whether the subtask data volume in the data statistics table reaches a target data volume or not;
If the data volume of the subtasks reaches the target data volume, generating response information which is processed by the target batch task;
and triggering a preset function to send the response information according to the response information, and archiving the target batch task.
7. A memory processing device for batch tasks, comprising:
The system comprises a creation module, a task logic processing area and a processing module, wherein the creation module is used for creating a target batch task in the task logic processing area and initializing the target batch task, the target batch task comprises a plurality of subtasks, the data volume of the target batch task is smaller than a preset data volume threshold, and the task logic processing area is a plug-in unit used for performing logic processing on the task;
The registration module is used for registering subtasks of the target batch task and adding monitoring parameters and callback parameters of the subtasks;
The execution module is used for executing the actions corresponding to the subtasks through the memory, and carrying out state monitoring and state callback on the actions corresponding to the subtasks according to the monitoring parameters and callback parameters, wherein the task response area is specially used for responding to the subtasks, and the memory is the memory in the task response area;
the detection module is used for judging whether the target batch task is finished or not through state monitoring and state callback of the subtasks, and archiving the target batch task after the completion;
the execution module comprises:
The execution unit is used for monitoring the state of the memory in the task response area for executing the action corresponding to the subtask according to the monitoring parameter;
The callback unit is used for carrying out state callback on the completed state through the callback parameters if the fact that the memory completes the action corresponding to the subtask is monitored, wherein the completed state refers to the fact that the memory completes the action corresponding to the subtask;
and the monitoring unit is used for continuing monitoring according to the monitoring parameters and not starting callback parameters if the memory is monitored to be in a state of executing actions corresponding to the subtasks.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the memory processing method of batch tasks as claimed in any one of claims 1 to 6 when the computer program is executed.
9. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the memory processing method of batch tasks according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911425296.0A CN113064661B (en) | 2019-12-31 | 2019-12-31 | Batch task memory processing method, device, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911425296.0A CN113064661B (en) | 2019-12-31 | 2019-12-31 | Batch task memory processing method, device, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113064661A CN113064661A (en) | 2021-07-02 |
| CN113064661B true CN113064661B (en) | 2025-03-07 |
Family
ID=76557934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911425296.0A Active CN113064661B (en) | 2019-12-31 | 2019-12-31 | Batch task memory processing method, device, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113064661B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113485808B (en) * | 2021-07-06 | 2024-06-28 | 中国工商银行股份有限公司 | System time callback method and device |
| CN113986379B (en) * | 2021-10-26 | 2023-12-15 | 抖音视界有限公司 | Application starting method and device, computer equipment and storage medium |
| CN114331382A (en) * | 2021-12-31 | 2022-04-12 | 中国邮政储蓄银行股份有限公司 | Batch file processing method, processing device, processor and batch processing system |
| CN115495252B (en) * | 2022-11-18 | 2023-03-03 | 苏州成科自控设备有限公司 | Design method of MCU background task pool |
| CN118093147B (en) * | 2024-04-29 | 2024-08-02 | 浙江农商数字科技有限责任公司 | Massive data summarizing method and system based on task chain and divide-and-conquer method |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109918184A (en) * | 2019-03-01 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Picture processing system, method and relevant apparatus and equipment |
Family Cites Families (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8726278B1 (en) * | 2004-07-21 | 2014-05-13 | The Mathworks, Inc. | Methods and system for registering callbacks and distributing tasks to technical computing works |
| US8990812B2 (en) * | 2008-07-07 | 2015-03-24 | Infosys Limited | Task decomposition with throttled message processing in a heterogeneous environment |
| US8261274B2 (en) * | 2009-06-29 | 2012-09-04 | Oracle America, Inc. | Method and system for managing a task |
| CN103049164B (en) * | 2011-10-14 | 2017-09-19 | 上海可鲁系统软件有限公司 | The method and its control device of a kind of dynamic vector graphics quick response operation |
| US8650280B2 (en) * | 2011-12-28 | 2014-02-11 | Sybase, Inc. | Monitoring distributed task execution using a chained text messaging system |
| CN102707993B (en) * | 2012-04-11 | 2016-09-21 | 招商银行股份有限公司 | Business datum centralized processing system and method |
| WO2014068950A1 (en) * | 2012-10-31 | 2014-05-08 | 日本電気株式会社 | Data processing system, data processing method, and program |
| CN104899093B (en) * | 2014-03-04 | 2018-06-05 | 华为技术有限公司 | Data processing method, device and system |
| CN104133724B (en) * | 2014-04-03 | 2015-08-19 | 腾讯科技（深圳）有限公司 | Concurrent tasks dispatching method and device |
| CN104991821B (en) * | 2015-06-29 | 2019-12-06 | 北京奇虎科技有限公司 | method and device for processing monitoring tasks in batches |
| CN105700958B (en) * | 2016-01-07 | 2019-05-03 | 北京京东尚科信息技术有限公司 | The method and its system that a kind of task is split automatically and subtask executes parallel |
| US10025625B2 (en) * | 2016-03-31 | 2018-07-17 | Microsoft Technology Licensing, Llc | Batched tasks |
| CN107729139B (en) * | 2017-09-18 | 2021-02-26 | 北京京东尚科信息技术有限公司 | Method and device for concurrently acquiring resources |
| CN109587272B (en) * | 2018-12-30 | 2021-10-08 | 深圳云天励飞技术有限公司 | Method and device for optimizing microservice and electronic equipment |
| CN110471718B (en) * | 2019-07-04 | 2023-07-18 | 深圳市共进电子股份有限公司 | Task processing method and device |
| CN110287018B (en) * | 2019-07-04 | 2021-08-13 | 中国工商银行股份有限公司 | Batch task arranging method and device |
-
2019
- 2019-12-31 CN CN201911425296.0A patent/CN113064661B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109918184A (en) * | 2019-03-01 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Picture processing system, method and relevant apparatus and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113064661A (en) | 2021-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113064661B (en) | Batch task memory processing method, device, electronic device and storage medium | |
| US10838838B2 (en) | Method and apparatus for dealing with abnormality of application program and storage medium | |
| CN108984388B (en) | Method and terminal equipment for generating automatic test case | |
| US9372776B2 (en) | Monitoring user activity and performance of computerized devices | |
| WO2017114152A1 (en) | Service dial testing method, apparatus and system | |
| CN112070416B (en) | AI-based RPA flow generation method, apparatus, device and medium | |
| WO2023103390A1 (en) | Task processing method, task processing apparatus, electronic device and storage medium | |
| CN110232022A (en) | Network environment test method, device and terminal device | |
| CN110119307B (en) | Data processing request processing method and device, storage medium and electronic device | |
| CN110765288A (en) | Image information synchronization method, device and system and storage medium | |
| CN108090352B (en) | Detection system and detection method | |
| CN113238815B (en) | Interface access control method, device, equipment and storage medium | |
| CN111914793A (en) | A kind of early warning method, device and equipment based on area population | |
| US11005797B2 (en) | Method, system and server for removing alerts | |
| CN113254248B (en) | Fault diagnosis method, system and computing device | |
| US8918406B2 (en) | Intelligent analysis queue construction | |
| EP4102361A2 (en) | Method and apparatus for managing functions of a query engine | |
| CN113515743B (en) | Identification method and device for rebound shell process call chain and electronic device | |
| CN111376255A (en) | Robot data acquisition method and device and terminal equipment | |
| CN116126719A (en) | Interface testing method and device, electronic equipment and storage medium | |
| CN113407331B (en) | A task processing method, device and storage medium | |
| CN115061907A (en) | Test method and test system, electronic equipment, storage medium | |
| AU2020221855B2 (en) | Activity detection in web applications | |
| CN114048467A (en) | Model data processing method, device, equipment and storage medium based on intrusion detection system | |
| US20220171664A1 (en) | Parallel execution controller for partitioned segments of a data model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |