CN114897296A - RPA flow labeling method, execution process playback method and storage medium - Google Patents

Abstract

The invention discloses an RPA process labeling method, an execution process playback method and a storage medium. The labeling method includes: using an RPA process labeling model to insert a business label in an RPA process to be labelled, and generating an RPA process containing the business label. The execution process playback method includes: obtaining the business flow chart and the RPA flow chart; obtaining the execution time stamp, screen recording and event log of the RPA process through the RPA process execution process; associating the business flow chart, the RPA flow chart, the screen recording and the event log ; Play the associated business flow chart, RPA flow chart, screen recording and event log synchronously according to the execution time stamp. The present invention adds business labels that are convenient for business personnel to understand for the RPA process, and at the same time displays the execution process of the RPA process in an intuitive and convenient form, reduces the difficulty of understanding the RPA process, and improves the positioning efficiency of the process playback.

Description

RPA process annotation method, execution process playback method and storage medium

technical field

The invention relates to an RPA process labeling method, an execution process playback method and a storage medium, belonging to the technical field of RPA process.

Background technique

RPA is an automated process technology. After the developer writes the RPA process, the computer/robot can automatically and repeatedly execute the process, instead of manually performing specific operations. Because RPA is efficient and accurate, the use of RPA process can achieve the purpose of saving manpower, improving efficiency and employee satisfaction, so it has been widely used. In order to ensure that the RPA process execution process is consistent with the expectations of developers and business personnel, developers and business personnel need to check the specific execution process of RPA after the RPA process is written to ensure the accuracy of the execution process and results.

The general way to check the RPA execution process is to record the complete screen content when the RPA process is executed, and then view it by video playback. This method is relatively simple and intuitive, but there are still some problems: 1. There is a problem that it is difficult to locate the specific process during playback. When the operation steps of the RPA process are very many or very complex, it is impossible to accurately locate the corresponding picture through video playback. For specific execution steps, there may be missing or multiple execution steps, which is not conducive to inspection. When there is an error in the process, some error messages will appear in the process log, but these error messages cannot be quickly matched with the video images. Understand the error situation more intuitively; 2. The technical requirements for inspectors are high. When business personnel without programming ability are responsible for inspecting the RPA execution process, it is difficult for business personnel to intuitively correspond the video screen to the RPA process steps and their logs. It is difficult to carry out the inspection work smoothly and accurately, and if the developers are all allowed to perform the inspection operation, the labor cost is high.

SUMMARY OF THE INVENTION

Aiming at the problems of inconvenient positioning and incomprehension of the RPA execution process playback method in the prior art, the present invention proposes an RPA process labeling method, an execution process playback method and a storage medium. Fully synchronized screen recordings, RPA flowcharts, business flowcharts and event logs realize the playback of the RPA process execution process, which is convenient for analyzing specific process steps and locating errors during the playback process. Technical requirements for the RPA process execution process.

In order to solve the above-mentioned technical problems, the present invention adopts the following technical means:

In the first aspect, the present invention proposes a labeling method for the RPA process, comprising the following steps:

Obtain the RPA process to be marked;

Use the trained RPA process labeling model to insert business labels into the RPA process to be labeled to generate an RPA process with business labels;

Wherein, the RPA process labeling model adopts a neural network model based on a machine learning algorithm.

Combined with the first aspect, further, the training method of the RPA process annotation model is:

Obtain multiple sets of RPA processes containing manually-labeled business labels to form labeling datasets;

The RPA process annotation model is trained by using the annotation dataset, and the trained RPA process annotation model is obtained.

With reference to the first aspect, further, there is a mapping relationship between a service tag and one or more steps in the RPA process, and the service tag is inserted before one or more steps of the mapping.

In combination with the first aspect, further, the service tag includes tag number, tag insertion position, and service information; the service tag is not executable and will not affect the execution of the RPA process.

In a second aspect, the present invention proposes a method for replaying an RPA process execution process, comprising the following steps:

Insert a business label into the RPA process by using the labeling method described in the first aspect;

According to the RPA process and its business label, obtain the RPA flow chart and business flow chart with the mapping relationship;

Obtain the execution timestamp, screen recording and event log of the RPA process through the RPA process execution process;

According to the mapping relationship between execution timestamp, RPA flow chart and business flow chart, associate business flow chart, RPA flow chart, screen recording and event log on the timeline;

Play the associated business flow chart, RPA flow chart, screen recording and event log synchronously according to the execution time stamp.

Combined with the second aspect, further, use the service tag as a node to form a business flow chart, and use each step of the RPA process as a node to form an RPA flow chart; the business flow chart and screen recording are played in the business mode page, and the RPA flow chart and screen Recordings and event logs are played on the development mode page.

Combined with the second aspect, further, in the synchronous playback process, in response to the playback jump command, the target time after the jump is obtained, and the playback pictures of the business flow chart, the RPA flow chart, the screen recording and the event log are all jumped to. target time, and highlight the content of business process diagrams, RPA process diagrams, and event logs at the target time.

In conjunction with the second aspect, further, the playback jump command includes a playback jump command based on a playback progress bar and a playback jump command based on keyword retrieval, wherein the keywords include business process keywords, RPA process keywords and Event log keyword.

In a third aspect, the present invention provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the method described in the first aspect of the present invention.

In a fourth aspect, the present invention provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the method described in the second aspect of the present invention.

After adopting the above technical means, the following advantages can be obtained:

The present invention provides an RPA process labeling method, an execution process playback method and a storage medium. First, a service label is inserted into the RPA process through the RPA process labeling method, and one or more steps in the RPA process are summarized into a service label. The complexity of the process is reduced, and the business label information is relatively simple and easy to understand, which is convenient for business personnel to understand the operation process of the RPA process and reduces the difficulty of understanding the RPA process. Secondly, according to the RPA process with business tags, the screen recording and event log of the RPA process execution process are synchronized with the RPA flow chart and business flow chart and then played back, which can not only conveniently and intuitively view the screen when each step of the RPA process is executed. It can also view the event log, RPA process steps and business steps summarized by the upper level corresponding to the screen image at the same time, which improves the positioning efficiency of process playback and is conducive to accurate analysis of problems and positioning errors during playback. Finally, the present invention provides two playback modes according to the different needs of developers and business personnel. In the business playback mode, a simplified business flow chart is used to display the content of the RPA process, which can facilitate business personnel to understand the RPA process and reduce the need to check the RPA process. The technical requirements of the execution process, in turn, reduce the labor cost of checking the RPA execution process.

In addition, the present invention can also change the playback content by dragging the video progress bar or log search, quickly locate the execution process of the RPA process according to user requirements, further simplify the playback operation of the RPA process execution process, and improve the use efficiency of the RPA process.

Description of drawings

Fig. 1 is the step flow chart of a kind of labeling method for RPA process of the present invention;

Fig. 2 is the step flow chart of a kind of RPA process execution process playback method of the present invention;

Fig. 3 is the playback schematic diagram under the business mode page in the embodiment of the present invention;

4 is a schematic diagram of adjusting the display size of a business flow chart on a business mode page in an embodiment of the present invention;

FIG. 5 is a schematic diagram of playback under a development mode page in an embodiment of the present invention.

Detailed ways

Below in conjunction with accompanying drawing, technical scheme of the present invention is further described:

Example 1:

The present invention proposes a labeling method for the RPA process, as shown in Figure 1, which specifically includes the following steps:

Step 1. Establish and train the RPA process annotation model.

In the embodiment of the present invention, the RPA process labeling model adopts a neural network model based on a machine learning algorithm, and specifically, a sequence labeling model LSTM+CRF may be used. The specific steps for training the RPA process annotation model are:

Step 101: Obtain multiple sets of RPA processes containing manually labeled business labels to form a labeled data set, and the labeled data set includes a training set, a test set, and a verification set.

In the embodiment of the present invention, the service tag can be directly inserted into the prepared RPA process, specifically inserted between the steps of the RPA process, and becomes a piece of information in the RPA process. Each business label can summarize one or more steps in the RPA process, that is, a business label has a mapping relationship with one or more steps in the RPA process. Each business label is inserted before one or more steps of its mapping. Therefore, the RPA process step between two business labels is associated with the business label at the front. After the business label is inserted, the corresponding RPA process is searched backwards. Steps. In the annotation training set, the insertion position of the business process is determined by the business personnel according to the business content.

The business label mainly includes label number, label insertion position, and business information. Among them, the label number is used to distinguish different business labels and mark the sequence of multiple business labels; the label insertion position refers to the specific position where the business label is inserted into the RPA process. Since the label insertion position is related to the label mapping relationship, the label insertion position The insertion position can also indicate the relationship between the business label and the RPA process steps; business information is the information inserted by the business label and displayed in the RPA process. "Confirm order", "End", etc. In addition to the business label with specific meaning, there is also a default business label, which is represented by "O", which means that this operation does not belong to any business.

In the embodiment of the present invention, the specific operation steps in the RPA process may change during execution, and also include a time stamp. The service tag is an empty operation, which cannot be executed, and will not have any impact on the execution of the RPA process. The business tag is mainly used during the playback of the RPA process execution process.

Step 102: Convert each RPA process in the labeled data set into sequence labeled data.

The original RPA process operation steps are represented by json, and the RPA process operation steps are converted into numbers through a fixed mapping to form a sequence. The simplest number mapping method is to select multiple fields from the RPA process, and then determine whether multiple fields belong to the same event. If they belong to the same event, map multiple fields to the same number, and record the RPA process field and number. The mapping relationship for subsequent RPA process restoration. Suppose an RPA process includes: action='click', name='submit', parent='window_xxx', xpath=' //div/p/a', select three fields: action, name, xpath, if they are three If the values are the same, they are mapped to a number s1. At the same time, the business label can also be represented by a number, and the number can directly follow the label number of the business label.

Assuming that the sequence data of the RPA process is: s1, s2, s3, s4, s5, s4, s6, the sequence labeling data of the RPA process after manually labeling the business label is: <t1>s1, s2, <t2>, s3, s4, s5, <t3>, s4, s6, can be expressed as: <s1, t1>, <s2, t1>, <s3, t2>, <s4, t2>, <s5, t2>, <s4, t3 >, <s6, t3> , where <t1>, <t2>, and <t3> are the numbers of the business tags in the sequence annotation data. The business tag <t1> summarizes the operations s1 and s2, and the business tag <t2> Operations s3, s4, and s5 are summarized, and the business tag <t3> summarizes operations s4 and s6.

Step 103: Use the sequence labeling data converted in step 102 to train the RPA process labeling model, and continuously update the model parameters until the trained RPA process labeling model is obtained. The input of the RPA process annotation model is the sequence data of the RPA process, for example, s1, s3, s4, s5, and the output of the RPA process annotation model is the sequence after adding a label to each operation, for example, <s1, t1>, < s3, t2>, <s4, t2>, <s5, t2>, compare the model prediction results with the sequence labeling data of the RPA process after manually labeling the business label, and then the model can learn automatically.

Step 2. Obtain the RPA process to be labeled. The RPA process to be labeled has only the operation steps written by the developer, and there is no business label.

Step 3. Use the trained RPA process labeling model to insert business labels into the RPA process to be labelled to generate an RPA process with business labels. Specifically, the RPA process to be labeled is first converted into sequence data, and the mapping relationship between the serial number in the sequence data and the operation steps of the RPA process is recorded, and then the sequence data is input into the RPA process annotation model for annotation, and the RPA process annotation model is obtained. Finally, according to the mapping relationship between the serial number in the sequence annotation data and the RP process operation steps or business tags, the sequence annotation data is converted into an RPA process with business tags.

Example 2:

The present invention proposes an RPA process execution process playback method, as shown in Figure 2, which specifically includes the following steps:

Step 1. Obtain an RPA process, and insert a business tag for the RPA process by using the labeling method in Embodiment 1. Install RPA process executors, screen recorders, log recorders, and more.

Step 2, according to the RPA process and its business label, obtain the RPA flow chart and the business flow chart with the mapping relationship.

The service flow chart is formed by using the service label as a node, and specifically, the service information in the service label is used as the step content in the service flow chart. Each step of the RPA process is used as a node to form an RPA flowchart, and the content of the steps in the RPA flowchart is the content edited by the programmer. Since the business label is a summary of one or more steps in the RPA process, and the business information belongs to the text content that is easy to understand, the business flow diagram has fewer steps and better understanding than the RPA flow diagram.

Step 3. Obtain the execution time stamp, screen recording and event log of the RPA process through the RPA process execution process.

Call the RPA process executor, run the RPA process, and start the screen recorder and log recorder at the same time. When the RPA process executor runs, it automatically records the start and end time of each step in the RPA process, generates RPA process execution data, and executes the RPA process. The data can be represented as <timestamp, step, rect>, where timestamp is the timestamp, step is the ID of the corresponding step in the process, rect is the box where the corresponding step in the flowchart is located, and the RPA process execution data is obtained from the RPA process execution data. The execution time stamp corresponding to the process step; the screen recorder is used to automatically record the screen content. Since the screen recorder and the RPA process execution program start running synchronously, a screen recording corresponding to the execution time stamp can be generated, and the screen recording data can be expressed as <timestamp, frame>, where frame is a video frame; the log recorder is used to record all events that occur during the execution of the RPA process, and each event contains a timestamp, and the event log data can be represented as <timestamp, line>, where , line is the event line, each event can have multiple lines, and each line has a timestamp.

Step 4. Correlate the business flow chart, the RPA flow chart, the screen recording and the event log on the timeline according to the execution time stamp, the mapping relationship between the RPA flow chart and the business flow chart.

Since there is a mapping relationship between service tags and RPA processes, and the execution timestamp records the start and end time of each step, the corresponding start and end times of each service tag can be determined according to one or more steps mapped to each service tag. , and there is also time in the screen recording and event log, so the present invention can associate all the information by executing the time stamp,

Step 5. Load the player, and synchronously play the associated business flow chart, RPA flow chart, screen recording and event log according to the execution time stamp.

The player page is shown in Figures 3-5. The business flow chart and screen recording are played in the business mode page, and the RPA flow chart, screen recording and event log are played in the development mode page. By default, the left column of the player page is the general information of the current RPA process, including the operator, operation start, end time, number of steps and other information; the main window of the player page includes two parts under the business model page , the middle is the screen recording, the lower right corner is the animation of the business flow chart; the main window of the player page includes three parts under the development mode page, the middle is the event log corresponding to the current operation, the upper right corner is the screen recording, the lower right corner is It is the animation of the RPA flowchart; the bottom of the player page is the playback controller, which can control the playback progress of log highlighting, screen recording, and flowchart animation at the same time. In the present invention, the user can freely switch the playback mode according to the requirements, and can adjust the position and size of each window in the page according to the requirements.

During the synchronous playback process, the flow chart animation and the highlight display of the event log are simultaneously promoted. In addition, in order to facilitate the user to view the execution process of the RPA process, the user can skip the playback progress through the playback jump command. In response to the playback jump command, obtain the target time after the jump, jump the playback screens of the business flow chart, RPA flow chart, screen recording and event log to the target time, and highlight the business flow chart and RPA flow chart and the contents of the event log at the target time. The playback jump command includes the playback jump command based on the playback progress bar (that is, the user directly drags the progress bar to jump) and the playback jump command based on keyword retrieval (that is, the user enters a keyword in the page for retrieval, and then Search structure jump), where the keywords include business process keywords, RPA process keywords, and event log keywords.

Example 3:

The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the RPA process labeling method in Embodiment 1.

Example 4:

The present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the RPA process execution process playback method in Embodiment 2.

Compared with the prior art, the labeling method of the present invention can insert a business label into the RPA process, thereby reducing the complexity of the process, and at the same time, the business label information is relatively simple and easy to understand, which is convenient for business personnel to understand the operation process of the RPA process, and reduces the complexity of the RPA process. Difficulty of understanding. The playback method of the present invention adds RPA flow chart, business flow chart and event log on the basis of recording and playing back the screen recording of the RPA execution process, and synchronizes the above data, so as to facilitate the understanding of the specific details corresponding to the video picture when the playback execution process is performed. Steps and log information can improve the positioning efficiency of process steps, facilitate analysis of problems, and locate errors. At the same time, the technical requirements for checking the execution of the RPA process are reduced through the business flow chart, thereby reducing the labor cost of checking the execution of the RPA process. In addition, the present invention can also change the playback content by dragging the video progress bar or log search, quickly locate the execution process of the RPA process according to user requirements, further simplify the playback operation of the RPA process execution process, and improve the use efficiency of the RPA process.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowcharts and/or block diagrams, and combinations of flows and/or blocks in the flowcharts and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in one or more of the flowcharts and/or one or more blocks of the block diagrams.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions An apparatus implements the functions specified in a flow or flows of the flowcharts and/or a block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in one or more of the flowcharts and/or one or more blocks of the block diagrams.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the technical principle of the present invention, several improvements and modifications can also be made. These improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (10)

1. a labeling method for RPA process, is characterized in that, comprises the steps: Obtain the RPA process to be marked; Use the trained RPA process labeling model to insert business labels into the RPA process to be labeled to generate an RPA process with business labels; Wherein, the RPA process labeling model adopts a neural network model based on a machine learning algorithm. 2. a kind of labeling method that is used for RPA process flow according to claim 1, is characterized in that, the training method of RPA process labeling model is: Obtain multiple sets of RPA processes containing manually-labeled business labels to form labeling datasets; The RPA process annotation model is trained by using the annotation dataset, and the trained RPA process annotation model is obtained. 3. a kind of labeling method that is used for RPA process flow according to claim 1, is characterized in that, a business label and a step or multiple steps in RPA flow process exist mapping relationship, and business label is inserted into a step of its mapping or multiple steps before. 4. a kind of labeling method for RPA process according to claim 1 and 3, is characterized in that, business label comprises label number, label insertion position, business information; Execution makes a difference. 5. an RPA process execution process playback method, is characterized in that, comprises the steps: Insert a business label for the RPA process by the labeling method described in claims 1 to 4; According to the RPA process and its business label, obtain the RPA flow chart and business flow chart with the mapping relationship; Obtain the execution timestamp, screen recording and event log of the RPA process through the RPA process execution process; According to the mapping relationship between execution timestamp, RPA flow chart and business flow chart, associate business flow chart, RPA flow chart, screen recording and event log on the timeline; Play the associated business flow chart, RPA flow chart, screen recording and event log synchronously according to the execution time stamp. 6. a kind of RPA flow process execution process playback method according to claim 5, is characterized in that, utilizes business label to form business flow chart as node, utilizes each step of RPA flow to form RPA flow chart as node; Business flow chart and The screen recording is played in the business mode page, and the RPA flowchart, screen recording and event log are played in the development mode page. 7. a kind of RPA process execution process playback method according to claim 5, is characterized in that, in the synchronous playback process, in response to playing the jump command, obtain the target time after the jump, by the service flow chart, the RPA process The playback screen of graphs, screen recordings, and event logs jumps to the target time, and highlights the content of business flowcharts, RPA flowcharts, and event logs at the target time. 8. a kind of RPA process execution process playback method according to claim 7, is characterized in that, described play jump command comprises the play jump command based on playback progress bar and the play jump command based on keyword retrieval, wherein , the keywords include business process keywords, RPA process keywords, and event log keywords. 9. A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented. 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the steps of the method according to any one of claims 5 to 8 are implemented.

Images