WO2024127582A1

WO2024127582A1 - Operation assistance device, operation assistance method, and operation assistance program

Info

Publication number: WO2024127582A1
Application number: PCT/JP2022/046190
Authority: WO
Inventors: 泰輔若杉; 晴夫大石; 公雄土川; 諒内田
Original assignee: 日本電信電話株式会社
Priority date: 2022-12-15
Filing date: 2022-12-15
Publication date: 2024-06-20

Abstract

An operation assistance device according to one embodiment of the present invention assists, in accordance with preset rules, operation of a web screen serving as the graphical user interface (GUI) of an application to be operated and comprises an identification unit and a correction unit. When the web screen has been changed, the identification unit identifies the changed content at changed locations on the web screen on the basis of HyperText Markup Language (HTML) and Cascading Style Sheet (CSS) files that are the source of the web screen. The correction unit corrects the rules on the basis of the changed content at the changed locations identified by the identification unit.

Description

Operation support device, operation support method, and operation support program

One aspect of the present invention relates to an operation assistance device, an operation assistance method, and an operation assistance program.

Conventionally, there are applications that are equipped with a GUI (Graphical User Interface) suitable for operator operation in order to restrict and support operator operations. However, there are various operators and operation methods for applications, and these may change over time. For example, the content that must be entered into the application may change when a new service is launched, or new input restrictions may be added. There may also be cases where new staff who are unfamiliar with the application must operate the application. In such cases, it is necessary to modify the application itself to ensure that operators comply with the application's operating rules and to prevent operating errors.

However, modifying the application itself can be very costly and time-consuming. In particular, if the application is a business application, the cost and time required for modification is likely to be enormous.

As a result, UI (User Interface) extension technology (see, for example, Patent Document 1) is being used that can assist the user in operating the application without modifying the application itself. This UI extension technology adds new extension parts (extended UI) for functional extension to a web screen without modifying the GUI parts (existing UI) that are originally present on the web screen, which is the GUI screen, and enables various functional extensions by setting the operation of the link between the existing UI and the extended UI. For example, when an extended UI (button) is clicked, a value can be automatically entered into an existing UI (text box).

As another method for supporting operator operations, RPA (Robot Process Automation) technology (see, for example, Non-Patent Document 1) is also being used, in which software-based robots take over and automate routine tasks.

JP 2017-72872 A

On the other hand, due to changes in the specifications of the application's web system, the display position of an existing UI on the web screen, the label name related to the existing UI, the identification name (ID name) added to the input information to the existing UI, etc. may change. In such a case, the extended UI that was applied to the previous web screen can no longer be applied to the changed web screen. Therefore, in order to apply the extended UI to the changed web screen, the operator must visually check and change the rule file that stores the rule definitions for the extended UI's position, operation functions, etc. each time.

The same applies to RPA scenarios, which are the rules for a robot's operation.

This invention was made in light of the above-mentioned circumstances, and its purpose is to provide technology that makes it possible to automatically modify rules such as extended UI rules and RPA scenarios in response to changes to web screens.

In order to solve the above problem, an operation assistance device according to one embodiment of the present invention is an operation assistance device that assists operations on a web screen as a GUI in an application to be operated in accordance with preset rules, and includes an identification unit and a correction unit. When the web screen is changed, the identification unit identifies the changes in the changed parts of the web screen based on the HTML (HyperText Markup Language)/CSS (Cascading Style Sheets) file that is the source of the web screen. The correction unit corrects the rules based on the changes in the changed parts identified by the identification unit.

According to one aspect of the present invention, it is possible to provide a technology that enables automatic modification of rules in response to a changed web screen.

FIG. 1 is a block diagram showing an example of the configuration of an information processing apparatus to which an operation support device according to a first embodiment of the present invention is applied. FIG. 2 is a block diagram showing the software configuration of the operation assistance device according to the first embodiment. FIG. 3 is a schematic diagram for explaining an example of a Web screen on which GUI components are arranged. FIG. 4 is a schematic diagram for explaining an example of overlaying an extension part on a GUI part on the Web screen of FIG. FIG. 5 is a flowchart showing an example of an operation related to the modification of an extension part in response to a change in a Web screen of the operation assistance device according to the first embodiment. FIG. 6 is a schematic diagram for explaining an example of GUI parts on a Web screen before and after a change. FIG. 7 is a schematic diagram for explaining an example of extension parts for a Web screen before and after a change. FIG. 8 is a block diagram showing the software configuration of the operation assistance device according to the second embodiment.

Below, an embodiment of the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 is a block diagram showing an example of the configuration of an information processing device 10 to which an operation assistance device according to a first embodiment of the present invention is applied.

The information processing device 10 is a computer such as a PC (Personal Computer) or a server computer. The information processing device 10 has a processor 11A such as a CPU (Central Processing Unit). The processor 11A may be multi-core/multi-threaded and can execute multiple processes in parallel. The information processing device 10 has a program memory 11B, a data memory 12, an input/output interface 13, and a communication interface 14 connected to the processor 11A via a bus 15.

The program memory 11B is a storage medium that combines a non-volatile memory that can be written to and read from at any time, such as a HDD (Hard Disk Drive) or SSD (Solid State Drive), with a non-volatile memory such as a ROM (Read Only Memory). The program memory 11B stores programs necessary for the processor 11A to execute various processes. The programs include the OS (Operating System) and various applications, as well as the operation assistance program according to the first embodiment.

The data memory 12 is a storage device that uses a combination of non-volatile memory, such as a HDD or SSD, which can be written to and read from at any time, and volatile memory, such as a RAM (Random Access Memory), as a storage medium. The data memory 12 is used to store data acquired and created in the course of performing various processes.

The input/output interface 13 is an interface with the input device 16 and the output device 17. The input device 16 includes a keyboard, pointing device, etc., which allow the operator of this information processing device 10 to input instructions to the processor 11A. Furthermore, the input device 16 may include a reader for reading files and data to be stored in the data memory 12 from a memory medium such as a USB memory, or a disk device for reading such files and data from a disk medium. The output device 17 includes, for example, a display device such as a display that displays output data from the processor 11A. Furthermore, the output device 17 may include a speaker that outputs voice data and music data.

The communication interface 14 is a wired and/or wireless communication unit for connecting to a network. This communication interface 14 can exchange information with other devices via the network. For example, if the information processing device 10 is a PC, the processor 11A can use this communication interface 14 to send data generated by an application in response to an operator's operation to a server, and receive programs and data from the server. Also, if the information processing device 10 is a server, the processor 11A can use this communication interface 14 to receive operation information from a terminal that does not have an information processing function and is equipped with an input device and an output device, and send display information to the terminal.

FIG. 2 is a block diagram showing the software configuration of the operation assistance device according to the first embodiment in relation to the hardware configuration of the information processing device 10 shown in FIG. 1.

The processing unit 11 is composed of the processor 11A and the program memory 11B, and includes, as software-based processing function units, an operation target application execution unit 111, an extended UI creation instruction acquisition unit 112, an extended UI definition unit 113, an application information collection unit 114, an extended UI execution unit 115, and an HTML/CSS structure comparison unit 116. All of these processing function units are realized by having the processor 11A execute the operation assistance program according to the first embodiment stored in the program memory 11B. The processing unit 11 may also be realized in a variety of other forms, including integrated circuits such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (field-programmable gate array), and a GPU (Graphics Processing Unit).

The storage area of the data memory 12 also includes an extended UI part information storage unit 121 and an HTML/CSS structure storage unit 122.

The operation target application execution unit 111 includes an operation target application stored in the program memory 11B. The operation target application execution unit 111 executes the operation target application in accordance with a start instruction from an operator who is an "information processing device administrator" who manages the information processing device 10, and causes the output device 17 to display a web screen as a GUI for the operator who is an "application operator" who actually operates the operation target application.

FIG. 3 is a schematic diagram for explaining an example of a Web screen 171 on which a GUI component (existing UI) is arranged. The operation target application execution unit 111 generates the Web screen 171 based on target GUI component identification information described in the HTML/CSS file 11B11 that is the source of the Web screen 171 and that is included in the operation target application 11B1 stored in the program memory 11B. The target GUI component identification information is information that indicates the HTML/CSS structure of the GUI components on the Web screen 171 of the operation target application 11B1. For example, the target GUI component identification information includes information indicating the presence or absence of a target GUI component on the Web screen 171 of the operation target application 11B1, information indicating the display position and display size of the target GUI component, information obtained by the target GUI component, and the like. The operation target application execution unit 111 can generate a Web screen 171 including two

input forms

172 and 173, which are GUI parts (existing UI), according to the target GUI part identification information (HTML/CSS structure), as shown in the example of FIG. 3, and display it on the output device 17. The input form 172 is located in the vicinity of the right side of the label name "customer name". When the operator, who is the "application operator", inputs character string information from the input device 16, the identification name "customerName" is added to the character string information, which is then passed to the operation target application 11B1 and reflected in the operation of the operation target application 11B1. The input form 173 is located in the vicinity of the right side of the label name "customer ID". When the operator inputs character string information from the input device 16, the identification name "customerID" is added to the character string information, which is then passed to the operation target application 11B1 and reflected in the operation of the operation target application 11B1.

The extended UI creation instruction acquisition unit 112 acquires an extended UI creation instruction from an operator who is an "extension component creation editor" who creates and edits an extended UI, which is an extension component to be displayed on the Web screen 171 of the operation target application 11B1, and application rules for the extended UI. This extended UI creation instruction is an instruction to create an extended UI, which is an extension component to be overlaid on a GUI component. The extended UI creation instruction acquisition unit 112 can acquire an extended UI creation instruction input by the operator operating the input device 16, or may acquire an extended UI creation instruction input by the operator operating another device and transmitted via a network using the communication interface 14. In response to acquiring the extended UI creation instruction, the extended UI creation instruction acquisition unit 112 causes the extended UI definition unit 113 to start creating and editing the extended UI and application rules.

The extended UI definition unit 113 creates and edits an extended UI for each GUI part (input forms 172, 173) on the web screen 171 of the target application 11B1, and creates and edits rules to be applied to the extended UI. For example, the extended UI definition unit 113 creates an extended UI and creates and edits rules to be applied to the extended UI based on an extended UI creation instruction from an extended part creation editor.

When creating an extended UI for a target GUI part, the extended UI definition unit 113 acquires information (GUI part identification information) required to identify the target GUI part from the application information collection unit 114. That is, the application information collection unit 114 acquires, from the operation target application execution unit 111, the target GUI part identification information described in the HTML/CSS file 11B11 of the operation target application 11B1 included in the operation target application execution unit 111. The extended UI definition unit 113 uses this target GUI part identification information to create and edit the extended UI, which is an extended part of the target GUI part. There are various methods for creating the extended UI in the extended UI definition unit 113. For example, a method of creating the extended UI using a known GUI application creation environment as the extended UI definition unit 113 is conceivable. Note that the extended UI definition unit 113 may cooperate with the extended UI execution unit 115 to allow the operation of the extended UI to be actually confirmed while editing the extended UI. This can improve the editing efficiency of the extended UI.

The extended UI definition unit 113 also stores rule information that associates "GUI part identification information" for identifying the target GUI part of the created extended UI, "extended UI display information" that defines how the extended UI is to be displayed, and "extended UI rule information" that defines what input rules are to be set for the displayed extended UI, as a rule file in the extended UI part information storage unit 121.

In this way, the extended UI part information storage unit 121 stores rule information related to the extended UI created and edited by the extended UI definition unit 113, that is, a rule file containing rule information that associates the GUI part specific information, extended UI display information, and extended UI rule information of the extended UI for each extended UI for each GUI part.

As described above, the application information collection unit 114 acquires target GUI part identification information (HTML/CSS structure) in the operated target application 11B1 contained in the operated target application execution unit 111 from the operated target application execution unit 111. The application information collection unit 114 also collects target GUI part identification information described in the HTML/CSS file 11B11 of the operated target application 11B1 contained in the operated target application execution unit 111 at a predetermined collection timing, and stores the collected target GUI part identification information in the HTML/CSS structure holding unit 122. As a result, the HTML/CSS structure related to the GUI parts that make up the web screen 171 is saved in chronological order in the HTML/CSS structure holding unit 122. The collection timing may be, for example, when the operation assistance device is started, at a set time each day, or at any time when an operator who is an "information processing device administrator" who manages the information processing device 10 instructs collection.

The extended UI execution unit 115 displays the extended UI on the output device 17 and determines the input rules for the input information from the input device 16 to the extended UI in accordance with the definitions in the rule information contained in the rule file for each extended UI stored in the extended UI part information storage unit 121.

Specifically, the extended UI execution unit 115 generates an extended UI for each target GUI part based on the target GUI part identification information acquired by the application information collection unit 114 and the GUI part identification information stored in the extended UI part information storage unit 121. The extended UI execution unit 115 then displays the generated extended UI on the target GUI part according to the extended UI display information stored in the extended UI part information storage unit 121. FIG. 4 is a schematic diagram for explaining an example of overlaying extended

UIs

174 and 175, which are extended parts, on

input forms

172 and 173, which are GUI parts (existing UIs) in the Web screen 171 shown in FIG. 3. In this way, the extended UI execution unit 115 can display extended

UIs

174 and 175 according to the definition in the rule information included in the rule file 1211 for each extended UI stored in the extended UI part information storage unit 121.

The extended UI execution unit 115 also detects input to the extended UI by an operator who is an "application operator". The extended UI execution unit 115 then determines whether or not the information input to the extended UI is consistent with the rules defined in the extended UI rule information of the extended UI, which is stored in the extended UI part information storage unit 121. For example, the extended UI execution unit 115 determines whether or not the information input to the extended UI is consistent with the input rules in the rule information for the extended UI. Here, if it is determined that the information input to the extended UI is consistent with the rules of the extended UI, the extended UI execution unit 115 transmits the input information to the operation target application execution unit 111 and reflects it in the execution of the operation target application 11B1. On the other hand, if it is determined that the information input to the extended UI is not consistent with the rules of the extended UI, the extended UI execution unit 115 displays this on the Web screen 171 of the operation target application 11B1. In this way, the extended UI execution unit 115 can execute input rule determination for information input to an extended UI (e.g., extended UIs 174 and 175) according to the definitions in the rule information contained in the rule file 1211 for each extended UI stored in the extended UI part information storage unit 121.

The HTML/CSS structure comparison unit 116 detects whether or not the Web screen 171 has been changed. If there is a change in the specifications of the Web system of the application, the display position of the GUI parts (existing UI) on the Web screen 171, the label names related to the existing UI, the identification names (ID names) added to the input information to the existing UI, etc. may be changed by an operator who is the "information processing device administrator" who manages the information processing device 10. The HTML/CSS structure comparison unit 116 detects whether or not the Web screen 171 has been changed based on the HTML/CSS structures related to the GUI parts that make up the Web screen 171, which are stored in chronological order in the HTML/CSS structure storage unit 122. When a change to the Web screen 171 is detected, the HTML/CSS structure comparison unit 116 identifies the change content at the change location of the Web screen 171 based on the time-series HTML/CSS structures, and transmits the change content to the extended UI definition unit 113. Based on the change location and change content, the extended UI definition unit 113 modifies the rule definition in the rule information contained in the rule file 1211 for the extended UI of the target GUI part on the Web screen 171 before the change, which is stored in the extended UI part information storage unit 121, so that it matches the target GUI part on the Web screen 171 after the change. The extended UI definition unit 113 updates the rule file 1211 containing the rule information for the extended UI of the target GUI part stored in the extended UI part information storage unit 121 to one containing the modified rule information.

Next, the operation of the operation support device configured as above will be described.
Here, the operations involved in creating and editing an extended UI and application rules in response to an extended UI creation instruction from an operator who is an "extension component creation editor," and the operations involved in inputting input information by an operator who is an "application operator" using the extended UI are the same as in the past, so their explanation will be omitted.

Below, the operation relating to the modification of the extended UI in response to changes in the web screen 171 will be described. FIG. 5 is a flowchart showing an example of the operation relating to the modification of the extended UI, which is an extended component, in response to changes in the web screen 171 of the operation assistance device. The processor 11A of the information processing device 10 can perform the processing shown in this flowchart by, for example, executing an operation assistance program stored in advance in the program memory 11B. For example, the processor 11A can start the operation shown in this flowchart when a start instruction is input from the input device 16, or when a control signal instructing a start is received from another device such as a server via the communication interface 14.

First, the processor 11A operates as the application information collection unit 114 and determines whether it is time to collect target GUI part identification information (step S1). If it is determined that it is not time to collect target GUI part identification information, the processor 11A proceeds to the processing operation of step S1. In this way, the processor 11A waits for the collection timing to collect target GUI part identification information.

If it is determined in step S1 above that it is time to collect target GUI part identification information, the processor 11A collects HTML/CSS structures (step S2). Specifically, the processor 11A collects the HTML/CSS structures described there as target GUI part identification information from the HTML/CSS file 11B11 of the operation target application 11B1 contained in the operation target application execution unit 111, i.e., stored in the program memory 11B.

Then, the processor 11A stores the collected HTML/CSS structures in the HTML/CSS structure storage unit 122 of the data memory 12 (step S3).

Then, processor 11A operates as HTML/CSS structure comparison unit 116 and compares the HTML/CSS structures stored in HTML/CSS structure storage unit 122 (step S4). Specifically, processor 11A compares the current, i.e., latest HTML/CSS structure among the chronological HTML/CSS structures stored in HTML/CSS structure storage unit 122 with past HTML/CSS structures. The past HTML/CSS structure to be compared may be the oldest HTML/CSS structure, or the HTML/CSS structure from one point before the latest.

Then, processor 11A determines whether or not there is a difference between the two as a result of the comparison (step S5). Note that if only one HTML/CSS structure is stored in HTML/CSS structure storage unit 122, it is determined that there is no difference.

If it is determined that there is no difference, the processor 11A deletes one of the HTML/CSS structures stored in the HTML/CSS structure storage unit 122 (step S6). The HTML/CSS structure to be deleted may be, for example, the oldest HTML/CSS structure. The processor 11A then proceeds to the processing operation of step S1 above. By deleting the HTML/CSS structure in this way, it is possible to prevent the storage capacity of the HTML/CSS structure storage unit 122 from becoming bloated. Note that if only one HTML/CSS structure is stored in the HTML/CSS structure storage unit 122, this step S6 is skipped.

If it is determined in step S5 that there is a difference between the two HTML/CSS structures, the processor 11A extracts information about the changed parts (step S7). The information about the changed parts includes the position of the changed target GUI component and the changed content.

Figure 6 is a schematic diagram for explaining an example of GUI parts in web screen 171 before and after the change. Processor 11A operating as application information collection unit 114 collects HTML/CSS structure 1221 described as target GUI part specific information in HTML/CSS file 11B11 that is the source of web screen 171 of operation target application 11B1 before the change, and stores it in HTML/CSS structure holding unit 122 of data memory 12. This HTML/CSS structure 1221 for web screen 171 includes information on the position and size of input form 172, which is a GUI part (existing UI), information on the label name "Customer Name", information on the identification name added to input information "customerName", etc. Additionally, this HTML/CSS structure 1221 includes information on the position and size of the input form 173, which is a GUI component (existing UI), information on the label name "Customer ID", information on the identification name to be added to the input information "customer ID", etc.

Then, at some point, when the operation target application 11B1 is changed to the operation target application 11B1' so as to change the existing UI on the web screen 171 due to a change in the specifications of the web system of the operation target application, the processor 11A operating as the application information collection unit 114 at the collection timing for collecting target GUI part identification information collects the HTML-CSS structure 1221 described as target GUI part identification information in the HTML-CSS file 11B11' that is the source of the web screen 171' of the operation target application 11B1' after the change, and stores it in the HTML-CSS structure holding unit 122 of the data memory 12. The HTML-CSS structure 1221 for the changed web screen 171' includes information on the position and size of the input form 172' which is a GUI part (existing UI), information on the label name "user name", and information on the identification name to be added to the input information "userName". Additionally, this HTML/CSS structure 1221 includes information about the position and size of the input form 173', which is a GUI component (existing UI), information about the label name "User No.", information about the identification name to be added to the input information "Use No.", etc.

The processor 11A extracts information about the changes, including the position (and size) of the target GUI part (existing UI) that has been changed and the changed content (label name information, identification name information, etc.).

Then, the processor 11A deletes all of the HTML/CSS structures 1221 stored in the HTML/CSS structure storage unit 122 except for the latest HTML/CSS structure 1221 (step S8). This is because the HTML/CSS structure 1221 after the change will be the one that will be compared from now on, and the HTML/CSS structure 1221 that was previously obtained and stored will no longer be necessary.

Then, processor 11A operates as extended UI definition unit 113 and updates the rule information of the extended UI based on the information of the changed parts extracted in step S7 above (step S9). Specifically, processor 11A modifies the rule definition in the rule information contained in rule file 1211 for the extended UI of the target GUI part on web screen 171 before the change stored in extended UI part information storage unit 121 based on the information of the changed parts extracted, so that it matches the target GUI part on web screen 171' after the change. For example, processor 11A modifies the "GUI part identification information" for identifying the target GUI part of the extended UI from input form 172 to input form 172', modifies the "extended UI display information" that defines how the extended UI is to be displayed from a position (coordinates) corresponding to input form 172 to a position (coordinates) corresponding to input form 172', and modifies the "extended UI rule information" that defines what input rules are to be set for the displayed extended UI from adding the identification name of "customerName" to the input information to adding the identification name of "userName" to the input information.

Then, the processor 11A stores this modified rule information in the extended UI part information storage unit 121 (step S10). Specifically, the processor 11A stores in the extended UI part information storage unit 121 a rule file including this modified rule information for the extended UI of the target GUI part (input forms 172', 173') on the modified Web screen 171'. At this time, the processor 11A may also delete the rule file 1211 including the rule information for the extended UI of the target GUI part on the Web screen 171 before the change that was stored in the extended UI part information storage unit 121. The processor 11A then proceeds to the processing operation of step S1 above.

FIG. 7 is a schematic diagram for explaining an example of extended

UIs

174, 175 and extended UIs 174', 175' which are extended components for Web screens 171, 171' before and after the change. As shown in FIG. 7, the extended UI component information storage unit 121 stores a rule file 1211 in which rule information for extended

UIs

174, 175 is defined so as to correspond to input

forms

172, 173 which are GUI components of Web screen 171 before the change. Here, when Web screen 171 is changed, the rule definition in the rule information is automatically corrected according to the changed input forms 172', 173' on Web screen 171' after the change, and is stored in extended UI component information storage unit 121 as changed rule file 1211'. Therefore, when processor 11A operates as extended UI execution unit 115, it displays extended UIs 174', 175' corresponding to input forms 172', 173', which are modified GUI components, based on the definitions in the rule information contained in this automatically corrected updated rule file 1211', and determines whether the information entered into extended UIs 174', 175' is consistent with the input rules.

As described above in detail, the operation support device according to the first embodiment of the present invention is an operation support device that overlays the extension parts such as the

extension UIs

174 and 175 of the GUI parts according to a preset rule so as to overlap with the GUI parts such as the input forms 172 and 173 arranged on the Web screen 171 as the GUI in the operation target application 11B1, and reflects the input information to the extension parts in the operation target application 11B1 as information input to the GUI parts. Then, when the Web screen 171 is changed to the Web screen 171', the HTML/CSS structure comparison unit 116 as the identification unit identifies the change contents at the change part of the Web screen 171' based on the HTML/CSS files 11B11 and 11B11'. The extension UI definition unit 113 as the correction unit corrects the rules for the extension parts based on the change contents at the identified change part.
In this way, in the first embodiment, by comparing the HTML/CSS files 11B11, 11B11' of the Web screens 171, 171', the affected parts of the GUI parts due to the change are identified, and the rules for the extended parts are automatically corrected according to the changes in the changed parts. Therefore, it is possible to provide a technology that enables the rules of the extended UIs 174', 175', which are extended parts, to be automatically corrected according to the changed Web screen 171' without the intervention of an operator.

In addition, in the first embodiment, the HTML/CSS structure comparison unit 116, which serves as a change detection unit, detects whether or not the web screen has been changed based on the HTML/CSS files 11B11 and 11B11'. Therefore, changes to the web screen can be automatically detected without operator intervention, and revision of rules for extended parts can be started.

In the first embodiment, the application information collection unit 114 as a collection unit collects HTML/CSS structures 1221 related to GUI components constituting the web screens 171, 171' from the HTML/CSS files 11B11, 11B11' at a predetermined collection timing, and the HTML/CSS structure holding unit 122 stores the collected HTML/CSS structures 1221. Therefore, the HTML/CSS structures 1221 collected in chronological order can be stored in the HTML/CSS structure holding unit 122.

In the first embodiment, the HTML/CSS structure comparison unit 116, which serves as a change detection unit, detects whether or not the web screen has been changed by comparing the latest HTML/CSS structure 1221 with past HTML/CSS structures 1221, each of which is stored in the HTML/CSS structure storage unit 122. Therefore, by comparing the HTML/CSS structures 1221 collected in chronological order, it is possible to easily detect whether or not the web screen has been changed.

In the first embodiment, the application information collection unit 114 as a collection unit collects HTML/CSS structures 1221 related to GUI components constituting the web screens 171, 171' from the HTML/CSS files 11B11, 11B11' at a predetermined collection timing, and the HTML/CSS structure holding unit 122 stores the collected HTML/CSS structures 1221. The HTML/CSS structure comparison unit 116 as an identification unit compares the latest HTML/CSS structures 1221 and the past HTML/CSS structures 1221 stored in the HTML/CSS structure holding unit 122, respectively, to identify the changes in the changed parts of the web screen 171'. Therefore, the HTML/CSS structure storage unit 122 can store HTML/CSS structures 1221 collected in chronological order, and by comparing the HTML/CSS structures 1221 collected in chronological order, it is possible to easily identify the changes and the contents of those changes.

In the first embodiment, the changes can include at least one of the positions of the input forms 172', 173', which are GUI components, on the web screen 171' and the identification name to be added to the input information to the GUI components. When the changes include the positions of the input forms 172', 173', which are GUI components, on the web screen 171', the extended UI definition unit 113 as a correction unit corrects the rules for the extended UIs 174', 175', which are extended components, regarding the positions of the extended components on the web screen 171'. When the changes include the identification name to be added to the input information to the input forms 172', 173', which are GUI components, the extended UI definition unit 113 as a correction unit corrects the rules for the extended components, regarding the identification name to be added to the input information to the extended components. Therefore, if the position of the GUI component on the web screen 171' is changed, the rules regarding the position of the extended components, extended UIs 174' and 175', can be modified. Also, if the identification name added to the input information to the GUI components, input forms 172' and 173', is changed, the rules regarding the identification name added to the input information to the extended components can be modified.

[Second embodiment]
The above first embodiment was an example of changing the rules of an extended UI in accordance with a changed web screen, but in a similar manner, it is also possible to change the operation rules of a robot in an RPA scenario in accordance with a changed web screen.

FIG. 8 is a block diagram showing the software configuration of an information processing device 10 to which an operation assistance device according to a second embodiment of the present invention is applied. In this second embodiment, the processing unit 11 includes, as software-based processing function units, an RPA creation instruction acquisition unit 117, an RPA definition unit 118, and an RPA execution unit 119, instead of the extended UI creation instruction acquisition unit 112, extended UI definition unit 113, and extended UI execution unit 115 in the first embodiment. Also, the data memory 12 includes an RPA scenario information storage unit 123, instead of the extended UI part information storage unit 121 in the first embodiment.

The RPA creation instruction acquisition unit 117 acquires an RPA scenario creation instruction from an operator who is an "RPA scenario creation editor" who creates and edits an RPA scenario related to the web screen 171 of the target application 11B1.

The RPA definition unit 118 creates and edits an RPA scenario based on an RPA scenario creation instruction from an RPA scenario creation editor. There are various possible methods for creating an RPA scenario in this RPA definition unit 118, such as using a known RPA scenario creation environment as the RPA definition unit 118. The RPA definition unit 118 may cooperate with the RPA execution unit 119 to allow the actual operation of the RPA to be confirmed while editing the RPA scenario. In this way, the efficiency of editing the RPA scenario can be improved. The RPA definition unit 118 then saves the created RPA scenario, i.e., the rule information for the robot operation, as a rule file in the RPA scenario information storage unit 123.

The RPA execution unit 119 executes operations on the target application executed by the target application execution unit according to the definitions in the rule information contained in the rule file of each RPA scenario stored in the RPA scenario information storage unit 123.

In addition, when the Web screen 171 is changed by an operator who is an "information processing device administrator" managing the information processing device 10 in response to changes in the specifications of the Web system of the target application, as described in the first embodiment, the HTML/CSS structure comparison unit 116 identifies the location and content of the change and transmits them to the RPA definition unit 118. Based on the location and content of the change, the RPA definition unit 118 modifies the RPA scenario for the Web screen 171 before the change, which is stored in the RPA scenario information storage unit 123, so that it matches the Web screen 171 after the change, and updates the old RPA scenario stored in the RPA scenario information storage unit 123 to this modified RPA scenario.

Accordingly, according to the information processing device 10 to which the operation assistance device according to the second embodiment of the present invention is applied, the RPA scenario can be automatically modified in response to changes in the web screen 171. Therefore, it is possible to provide a technology that enables the rules of the robot's operation to be automatically modified in response to the modified web screen 171' without the intervention of an operator.

[Other embodiments]
It should be noted that the present invention is not limited to the above-described embodiment.

For example, in the above embodiment, the presence or absence of changes to the web screen is detected by comparing the HTML/CSS structures 1221 acquired and saved in chronological order, but it is also possible to detect whether a change has occurred using other methods.

Furthermore, it is not necessary to detect whether or not the web screen has been changed. For example, when an operator who is an "information processing device administrator" who manages the information processing device 10 makes a change to the target application, the operator can input information indicating that the change has been made, and in response to that input, automatic modification of the rules for the extended UI, which is an extended component, can be started.

Furthermore, the flow of each process described with reference to the flowchart is not limited to the procedure described. In each process, the order of some steps may be changed, some steps may be performed simultaneously in parallel, and the processing content of some steps may be modified.

The operation assistance programs according to the first and second embodiments and the methods described in the other embodiments can be stored as processing programs (software means) that can be executed by a calculator (computer) on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, MO, etc.), semiconductor memory (ROM, RAM, flash memory, etc.), and can also be distributed by transmitting them via a communication medium. The programs stored on the medium also include a setting program that configures the software means (including not only execution programs but also tables and data structures) that the calculator executes within the computer. The computer that realizes this device reads the programs recorded on the recording medium, and in some cases, configures the software means using the setting program, and executes the above-mentioned processing by controlling the operation of the software means. Note that the recording medium referred to in this specification is not limited to a storage medium for distribution, but also includes storage media such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

In short, this invention is not limited to the above-mentioned embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

LIST OF SYMBOLS 10... INFORMATION PROCESSING APPARATUS 11... PROCESSING UNIT 11A... PROCESSOR 11B... PROGRAM MEMORY 11B1, 11B1'... OPERATION TARGET APPLICATION 11B11, 11B11'... HTML/CSS FILE 12... DATA MEMORY 13... I/O INTERFACE 14... COMMUNICATION INTERFACE 15... BUS 16... INPUT DEVICE 17... OUTPUT DEVICE 111... OPERATION TARGET APPLICATION EXECUTION UNIT 112... EXTENDED UI CREATION INSTRUCTION ACQUISITION UNIT 113... EXTENDED UI DEFINITION UNIT 114... APPLICATION INFORMATION COLLECTION UNIT 115... EXTENDED UI EXECUTION UNIT 116... HTML/CSS STRUCTURE COMPARISON UNIT 117... RPA CREATION INSTRUCTION ACQUISITION UNIT 118... RPA DEFINITION UNIT 119... RPA EXECUTION UNIT 121... EXTENDED UI PARTS INFORMATION RETAINING UNIT 122... HTML/CSS STRUCTURE RETAINING UNIT 123... RPA SCENARIO INFORMATION RETAINING UNIT 171, 171'...

WEB SCREEN

172, 172', 173, 173'...

Input form

174, 174', 175, 175'... Extended UI
1211, 1211'... Rule file 1221... HTML/CSS structure

Claims

An operation assistance device that assists an operation on a web screen as a GUI (Graphical User Interface) in an operation target application in accordance with a preset rule,
an identifying unit that identifies, when the web screen is changed, a change content in a changed portion of the web screen based on an HTML (HyperText Markup Language)/CSS (Cascading Style Sheets) file that is a source of the web screen;
a modification unit that modifies the rule based on the change content in the change location identified by the identification unit;
An operation assistance device comprising:
The operation support device according to claim 1, further comprising a change detection unit that detects whether the web screen has been changed based on the HTML/CSS file.
a collection unit that collects HTML/CSS structures related to GUI components that configure the Web screen from the HTML/CSS files at a predetermined collection timing;
a storage unit for storing the HTML/CSS structures collected by the collection unit;
a change detection unit that detects whether or not the Web screen has been changed by comparing the latest HTML/CSS structure with the past HTML/CSS structure, each of which is stored by the storage unit;
The operation support device according to claim 1 , further comprising:
a collection unit that collects HTML/CSS structures related to GUI components that configure the Web screen from the HTML/CSS files at a predetermined collection timing;
a storage unit for storing the HTML/CSS structures collected by the collection unit;
Further comprising:
The operation support device according to claim 1 , wherein the identification unit identifies the change content in the changed portion of the Web screen by comparing the latest HTML/CSS structure with the past HTML/CSS structure each stored by the storage unit.
The operation support device according to any one of claims 1 to 4, wherein the supported operation is an operation of automatically inputting input information into a GUI component arranged on the web screen according to the preset rules.
The operation support device according to any one of claims 1 to 4, wherein the supported operation is an operation of automatically inputting input information into a GUI component arranged on the web screen according to the preset rules.
An operation assistance method executed by an operation assistance device that includes a processor and a memory and that assists an operation on a web screen as a UI of an operation target application in accordance with a preset rule, the method comprising:
When the web screen is changed, the processor identifies the change content in the changed portion of the web screen based on an HTML/CSS file that is the source of the web screen;
the processor modifies the rule based on the change content in the identified change portion, and stores the modified rule in the memory;
An operation assistance method comprising:
2. An operation assistance program that causes a processor included in the operation assistance device to execute processes performed by each unit of the operation assistance device according to claim 1.