CN109800135B

CN109800135B - Information processing method and terminal

Info

Publication number: CN109800135B
Application number: CN201711148742.9A
Authority: CN
Inventors: 张元超; 林孟光; 邓衍
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2017-11-17
Filing date: 2017-11-17
Publication date: 2022-01-07
Anticipated expiration: 2037-11-17
Also published as: CN109800135A

Abstract

The embodiment of the invention discloses an information processing method, which comprises the following steps: when the recording function of the target application is started, adding a first display window on a current display interface, wherein the first display window and a second display window of the current display interface are in a mapping relation; receiving a first touch instruction at a first display window, wherein the first touch instruction is used for indicating to trigger a function operation corresponding to a first control on a current display interface; acquiring first operation area information corresponding to the first control according to the first touch instruction; determining second operation area information of the current display interface according to the first operation area information and the mapping relation between the first display window and the second display window, and recording the second operation area information until the recording is finished; and when the target application is triggered to play the recording task, triggering the first control according to the second operation area information, and realizing the functional operation corresponding to the first control. The embodiment of the invention also provides a terminal for executing the information processing method.

Description

Information processing method and terminal

Technical Field

The present invention relates to application processing technologies in the field of internet, and in particular, to an information processing method and a terminal.

Background

With the rapid development of electronic information technology, the variety of electronic products is increasing, and users also enjoy various conveniences brought by the development of science and technology. At present, users can realize different application functions through various types of electronic equipment, terminals and the like, and enjoy comfortable life brought along with the development of science and technology.

At present, for a terminal installed with an application, in order to facilitate a user to operate a common function, a script is executed to automatically execute a certain function on the terminal application. In detail, a script for executing a certain function step needs to be set in advance, for example, a click position and an order for realizing the certain function are set, then the set script is transmitted or set at a computer end, when the terminal is connected with the computer end, each operation in the script is analyzed by the computer end, and finally each analyzed operation is acted on the terminal by using an Android Debug Bridge (ADB, Android Debug Bridge) command mode (i.e., a mode of opening a Universal Serial Bus (USB) for debugging, and then connecting the computer through a data line) so as to realize that the terminal automatically realizes the certain function. Or, the terminal may implement the application function for the already root application on the terminal by running the set script. Wherein the application that has been root refers to an application with a high level of authority, which can access and modify almost all files in the terminal.

However, the above ways of automatically implementing the application function all require setting and compiling scripts, and the complexity of compiling scripts is high, and the scripts are not easy to implement universally; meanwhile, on the basis of the written script, the script can be realized only by using a computer terminal or performing root on the terminal, and the realization condition is complex.

Disclosure of Invention

In order to solve the foregoing technical problems, embodiments of the present invention are directed to providing an information processing method and a terminal, which can reduce implementation complexity on the basis of automatically implementing an application function.

The technical scheme of the invention is realized as follows:

the embodiment of the invention provides an information processing method, which comprises the following steps:

when the recording function of the target application is started, adding a first display window on a current display interface, wherein the first display window and a second display window of the current display interface are in a mapping relation;

receiving a first touch instruction at the first display window, wherein the first touch instruction is used for indicating to trigger a function operation corresponding to a first control on the current display interface;

acquiring first operation area information corresponding to the first control according to the first touch instruction;

determining second operation area information of the current display interface according to the first operation area information and the mapping relation between the first display window and the second display window, and recording the second operation area information until the recording is finished;

and when the target application is triggered to play and record the task, triggering the first control according to the second operation area information, and realizing the function operation corresponding to the first control.

An embodiment of the present invention provides a terminal, including:

the device comprises an adding unit, a recording unit and a processing unit, wherein the adding unit is used for adding a first display window on a current display interface when the recording function of a target application is started, and the first display window and a second display window of the current display interface are in a mapping relation;

the receiving unit is used for receiving a first touch instruction in the first display window, wherein the first touch instruction is used for indicating to trigger a function operation corresponding to a first control on the current display interface;

the obtaining unit is used for obtaining first operation area information corresponding to the first control according to the first touch instruction;

a determining unit, configured to determine second operation region information of the current display interface according to the first operation region information and a mapping relationship between the first display window and the second display window,

the recording unit is used for recording the second operation area information until the recording is finished;

and the triggering unit is used for triggering the first control according to the second operation area information when the target application is triggered to play the recording task, so as to realize the function operation corresponding to the first control.

An embodiment of the present invention further provides a terminal, including: a processor, a receiver, a display, and a storage medium, the receiver, the display, and the storage medium in communication with the processor through a communication bus;

the storage medium is used for storing executable instructions;

the receiver is used for receiving executable instructions;

the processor is used for executing executable instructions stored in the storage medium, and the executable instructions are configured to execute the information processing method;

and the display is used for displaying the information after the executable instruction is executed.

The embodiment of the invention provides a computer storage medium which is applied to a terminal and stores machine instructions, and when the machine instructions are executed by one or more processors, the processors execute the information processing method.

The embodiment of the invention provides an information processing method and a terminal, when a recording function of a target application is started, a first display window is added in a current display interface, and the first display window and a second display window of the current display interface are in a mapping relation; receiving a first touch instruction at a first display window, wherein the first touch instruction is used for indicating to trigger a function operation corresponding to a first control on a current display interface; acquiring first operation area information corresponding to the first control according to the first touch instruction; determining second operation area information of the current display interface according to the first operation area information and the mapping relation between the first display window and the second display window, and recording the second operation area information until the recording is finished; and when the target application is triggered to play the recording task, triggering the first control according to the second operation area information, and realizing the functional operation corresponding to the first control. By adopting the technical implementation scheme, the terminal can perform the pre-recording task on the commonly used operation (namely, the functional operation realized by triggering the first control) of the commonly used target application, so that the terminal can directly play the recording task by triggering each time the functional operation is realized, and the functional operation can be automatically realized. Moreover, in the embodiment of the present invention, the method for recording the task by the terminal may implement the function operation of calling the corresponding operation area information of the second display window of the target application by monitoring the operation area information of the touch operation of the first display window mapped with the second display window of the current display interface, and implement the recording of the function operation.

Drawings

Fig. 1 is an architecture diagram of information interaction between a terminal and a server when implementing an application function according to an embodiment of the present invention;

fig. 2 is a first flowchart of an information processing method according to an embodiment of the present invention;

fig. 3 is a first diagram illustrating a mapping relationship between a first display window and a second display window according to an exemplary embodiment of the present invention;

fig. 4A is a first schematic interface diagram of an exemplary method for processing information according to an embodiment of the present invention;

fig. 4B is a schematic interface diagram of an exemplary information processing method according to an embodiment of the present invention;

fig. 4C is a third schematic interface diagram of an exemplary information processing method according to an embodiment of the present invention;

fig. 5 is a fourth schematic interface diagram of an exemplary method for processing information according to an embodiment of the present invention;

fig. 6 is a second exemplary mapping relationship structure diagram of a first display window and a second display window according to an embodiment of the present invention;

fig. 7 is a fifth schematic interface diagram of an exemplary method for processing information according to an embodiment of the present invention;

fig. 8 is a sixth schematic interface diagram of an exemplary method for processing information according to an embodiment of the present invention;

fig. 9 is a second flowchart of an information processing method according to an embodiment of the present invention;

fig. 10 is a seventh schematic interface diagram illustrating an exemplary method for processing information according to an embodiment of the present invention;

fig. 11 is an eighth schematic interface diagram of an exemplary method for processing information according to an embodiment of the present invention;

fig. 12 is a third flowchart of an information processing method according to an embodiment of the present invention;

fig. 13 is a ninth interface schematic diagram of an exemplary method for processing information according to an embodiment of the present invention;

fig. 14 is a first schematic structural diagram of a terminal according to an embodiment of the present invention;

fig. 15 is a schematic structural diagram of a terminal according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.

Barrier-free auxiliary services: the accessibility service function provided on the Android system helps the users to operate the equipment or the terminal more easily, and developers can make the programs better used by using the accessibility service.

ADB: the method is a client-server program, wherein the client is a computer for operation, the server is Android equipment, the ADB of the Android system is called Android Debug Bridge, and the Android Debug Bridge plays a role of a debugging Bridge. Operating the handset using a command mode on the computer: restarting, entering recovery (data backup), entering fastboot (fast start), pushing file functions and the like.

Root: refers to the system Administrator privileges of a Unix-like operating system (including Linux), similar to administerors in Windows systems. The Android root authority can access and modify almost all files in the terminal (Android system files and user files, not including read-only memory image ROM (image used for flash)), which may be data that manufacturers do not want to modify, because users can delete applications pre-installed by manufacturers or operators through the root authority.

Suspension window: in particular to a view control (namely a Window) which is added to a screen of the terminal through a Window Manager and is positioned above other applications on the android system.

Fig. 1 is a schematic diagram of various hardware entities in an information processing system architecture where a terminal for performing information processing according to an embodiment of the present invention is located, where fig. 1 includes: one or more servers 2-1 to 2-n, terminals, and a network 31, where the network 31 includes routers, gateways, etc., and is not shown in fig. 1. The terminal 1 performs information interaction with the servers 2-1 to 2-n through a wired network or a wireless network, so that the terminal 1 acquires access data from the servers 2-1 to 2-n when realizing certain functional operation, and data access of the functional operation is realized. The types of the terminal are shown in fig. 1 and include mobile phone (1-1), tablet computer, PDA (1-2), desktop computer (1-3), PC (1-4) or all-in-one machine (1-5) and the like. The terminal 1 is installed with various application function modules required by a user, such as applications with an entertainment function (e.g., a video application, an audio playing application, a game application, reading software, and a news application), applications with a service function (e.g., a map navigation application, a group buying application, a shooting application, and the like), and system functions such as setting applications, and the embodiments of the present invention are not limited.

Based on the hardware entity shown in fig. 1, when an application function (function operation) of any application is implemented in the terminal, information interaction may be performed with the background server of the application, so as to complete a certain application function, that is, the server provides information interaction of data access and processing for the background server of the application implementing the application function in the terminal, and the terminal implements the application function.

The above example of fig. 1 is only an example of a system architecture for implementing the embodiment of the present invention, and the embodiment of the present invention is not limited to the system architecture described in the above fig. 1, and various embodiments of the present invention are proposed based on the system architecture.

An embodiment of the present invention provides an information processing method, as shown in fig. 2, the method may include:

s101, when the recording function of the target application is started, adding a first display window on a current display interface, wherein the first display window and a second display window of the current display interface are in a mapping relation.

It should be noted that, on the premise that the terminal implements the information processing method provided in the embodiment of the present invention, a plug-in or an application for implementing the information processing method in the embodiment of the present invention is set or installed on the terminal, and may be referred to as a custom operation application here, and the embodiment of the present invention does not limit the presentation form of the application.

In the embodiment of the present invention, the process of implementing the information processing method is made into the custom operation application, so that the information processing method can be implemented by operating in the custom operation application in the process of processing by the terminal.

The information processing method in the embodiment of the present invention is described as an example in a form of a custom operation application.

In this way, the terminal in the embodiment of the present invention may be an electronic device, such as a mobile phone, a tablet, a computer, and the like, in which a functional application and a custom operation application are installed.

In the user-defined operation application, a function key for recording the user-defined operation is arranged to realize the recording function and perform the recording task of the user-defined operation, so that when a user uses a terminal to perform the user-defined operation, the user-defined operation (namely, the function operation, which can refer to the function operation performed on the first control in the embodiment of the invention) can be automatically performed by playing the recording task.

It should be noted that, in the embodiment of the present invention, the customization operation is directed to a function operation implemented by an application installed on the terminal, and therefore, what the terminal implements by the customization operation application is a recording task of the application.

Therefore, any application installed in the terminal is selected as the target application, and the recording task of the user-defined operation can be realized through the recording function of the user-defined operation application.

In the embodiment of the invention, when the recording function of the target application is started, the terminal adds the first display window on the current display interface, and because the current display interface of the terminal is displayed on the second display window, when the terminal detects that the recording function of the target application is started, the terminal adds the first display window which is mapped with the second display window one by one on the second display window.

It should be noted that, in the embodiment of the present invention, the first display window may be a floating window floating on the second display window, and the display areas of the first display window and the second display window are in one-to-one correspondence and have the same size, that is, the first display window and the second display window are in a mapping relationship.

In the embodiment of the present invention, the first display Window is a view control (i.e. Window) added on the terminal screen above the second display Window of the current display interface through the Window Manager. The first display window in the embodiment of the invention is a full-screen, transparent and invisible floating window, so that the terminal basically cannot feel that one more display window is added when displaying, and can completely and clearly display the current display interface.

Illustratively, the first display window and the second display window are in a one-to-one mapping relationship, and what is embodied on the terminal is that the first display window is suspended on the second display window, that is, the mapping relationship between the first display window and the second display window is shown in fig. 3.

For example, since the first display window may be full screen, transparent and invisible, assuming that the current display interface on the mobile phone 1 (terminal) is as shown in fig. 4A, when one first display window is added to the second display window of the current display interface of the mobile phone 1 through the mapping relationship as shown in fig. 4B, the display effect on the mobile phone 1 is as shown in fig. 4C, and the addition of the first display window in the embodiment of the present invention does not affect the original display effect and definition of the mobile phone 1.

In the embodiment of the invention, the current display interface refers to the content currently displayed by the terminal, and the current display interface can display different contents along with the triggering operation, but the current display interface is the current display content. Since the recording function of the target application is started in the user-defined operation application, that is, the current display interface now displays the current recording interface of the target application, and the content of the specific current display interface is determined by the specific operation, the embodiment of the present invention is not limited.

S102, receiving a first touch instruction in a first display window, wherein the first touch instruction is used for indicating and triggering a function operation corresponding to a first control on a current display interface.

After the terminal adds the first display window in the current display interface, because the first display window is a display window set by the terminal in the user-defined operation application, the terminal can monitor the touch operation or the touch event on the first display window. Therefore, in the embodiment of the present invention, when the terminal performs the touch operation on the target application in the first display window, the terminal can acquire the operation area information of the touch operation in the target application. The implementation of the touch operation of the target application by the user is represented by that the user performs the touch operation on the first control on the first display window through a gesture operation, so that the touch operation acting on the terminal generates a touch instruction, and then the terminal receives the first touch instruction on the first display window, and the first touch instruction is used for indicating to trigger the functional operation corresponding to the first control on the current display interface.

In the embodiment of the present invention, the first control is any one of function controls displayed on the current display interface, and the first control may be in various forms such as a function button, a function frame, or a function link.

In the embodiment of the present invention, the gesture operation may include operations such as clicking, sliding, or dragging, which are specifically determined according to a manner of triggering the first control, and the embodiment of the present invention is not limited.

It should be noted that, in the embodiment of the present invention, although the first touch instruction is an operation on the first control performed on the first display window, since the first control is a control in the current display interface and actually performs a function operation on the first control only by acting on the second display window, the first touch instruction received on the first display window in the embodiment of the present invention is generated for the touch operation on the first control, but the terminal cannot truly respond to the first touch instruction to directly implement the function operation on the first control. That is to say, in the terminal in the embodiment of the present invention, because the first display window and the second display window are mapped one by one, the position of the first control on the second display window is consistent with the position of the first control mapped on the first display window, so that the terminal can perform a touch operation with respect to the position of the first control mapped on the first display window, that is, the terminal receives a first touch instruction on the first display window, where the first touch instruction is used to instruct to perform a function operation on the first control on the current display interface, and the first control is not really triggered to implement the function operation.

For example, as shown in fig. 5, it is assumed that the terminal is a mobile phone 1, the mobile phone 1 is triggered to start a video recording function of a news application 2, a current display interface is a recording task setting interface 3 of the news application, an icon and a recording function button 4 of the news application 2 are displayed in the recording interface 3, the recording function button 4 is clicked to perform a recording interface 5, at this time, if a function operation of opening the news application 2 is performed, a user clicks a position mapped by the icon of the news application 2 in a first display window so as to implement an opening function, and thus the mobile phone 1 receives an opening instruction (a first touch instruction) for the news application 2.

S103, acquiring first operation area information corresponding to the first control according to the first touch instruction.

After the terminal receives the first touch instruction in the first display window, the terminal may monitor first operation area information on the first display window for the first control according to the first touch instruction.

In this embodiment of the present invention, the first operation region information may include a touch region or an action region of the first touch instruction (i.e., first position information of the first control on the first display window), first interface information of the current display interface, and a first operation trajectory of the gesture operation.

It should be noted that, because the size of the first display window in the embodiment of the present invention is consistent with the full size of the current display interface, the content of the current display interface may be proportionally mapped on the first display window, and thus, the first position information may represent the second position information of the first control in the second display window, and may be represented by using the coordinate information in the first display window; the first interface information can represent the interface to which the first control belongs, such as webpage information or page information; the first operation track represents a trigger trace generated by gesture operation, and may be composed of elements such as a trigger starting position, an end position, trigger time, a trigger direction and the like.

It can be understood that, since the terminal can obtain the first position information, the first interface information, and the first operation trajectory corresponding to the first control through the added first display window through the user-defined operation application, what the user wants to record the user-defined operation is determined.

It should be noted that, an information processing method provided in the embodiment of the present invention is directed to a series of operations corresponding to a commonly used target application, and a functional operation that has already performed a recording task may be directly realized by one key through playing the recording task.

And S104, determining second operation area information of the current display interface according to the first operation area information and the mapping relation between the first display window and the second display window, and recording the second operation area information until the recording is finished.

After the terminal acquires the first operation area information corresponding to the first control according to the first touch instruction, due to the mapping relationship between the terminal or the first display window and the second display window, the terminal can determine the second operation area information on the current display interface, which has the mapping relationship with the first operation area information, according to the first operation area information on the first display window and the mapping relationship between the first display window and the second display window, wherein the second operation area information is the operation area information on the second display window.

In the embodiment of the present invention, in order to implement a recording function for a target application, the terminal needs to acquire a series of function triggering operations, function controls, and contents of interface display changes for implementing the function operations of the target application, and any one target application does not allow the terminal to arbitrarily monitor the information.

For example, as shown in fig. 6, when a touch operation corresponding to a first control is received on a first display window, an active area (i.e., first operation area information) a' of the first control is determined based on a mapping structure shown in fig. 6, and an active area a (i.e., second operation area information) on a second display window is determined.

It should be noted that, because the information processing method provided in the embodiment of the present invention is a process of performing a broadcast recording task by performing a function operation of a recording target application, a terminal needs to record the record of the operation area information when the control is triggered each time until the recording is completed, which is called a recording task.

In the embodiment of the present invention, the terminal may implement a recording task of one functional operation of the target application through the custom operation application, and may also implement a recording task of a series of functional operations of the target application.

However, when a recording task of the target application is used for recording a series of functional operations, different pages or different controls of the interface need to be triggered once and again, and the triggering of each interface or control in a page is time-sequential. In this way, the terminal needs to determine the second operation area information of the second display window of the target application and record the second operation area information until the recording is completed by acquiring the first operation area information of the first display window each time.

In the embodiment of the present invention, the recording completion may be triggered by a recording completion function key, and the specific implementation of the recording completion function key may be in the form of a recording completion function control, a recording completion function button, or a recording completion icon, which is not limited in the embodiment of the present invention.

It should be noted that, in the process of recording the user-defined operation of the target application, the current display interface displays the interface content or the page content of the target application, and the recording completion function key is arranged on the current display interface, so that no matter how many functional operations need to be triggered by the terminal to complete a recording task, a recording completion instruction can be generated by triggering the recording completion function key when the terminal wants to end recording, and the recording function is ended according to the recording completion instruction.

Furthermore, after the terminal completes the recording task, the terminal can return to the recording task setting interface by setting a return function key on the current display interface, and the implementation manner of the return function key is not limited in the embodiment of the invention, so that the terminal can return to the recording task setting interface from the recording interface of the current display interface by triggering the return function key, wherein the recording task setting interface is a setting interface when the terminal performs the recording task of the target application, and information such as the target application and the recording task which need to be recorded is displayed.

For example, as shown in fig. 7, it is assumed that the terminal is a mobile phone 1, the target application is a news application, when the mobile phone 1 triggers the opening function operation of the target application, the current display interface displays news content of the news application, a stop key 3 (i.e., a recording completion function key) is arranged in the area 2, when the user triggers the stop key 3 on the mobile phone 1, recording is finished, that is, the mobile phone 1 finishes acquiring operation area information of the control; as shown in fig. 7, a return button 4 may be further disposed in the area 2, and when the user triggers the return button 4 on the mobile phone 1, the recording task is returned to the recording task setting interface from the recording interface 5 of the current display interface.

In the embodiment of the invention, the recording task setting completion control is arranged in the recording task setting interface, and the terminal can complete the setting completion of the recording task of the target application through the task setting completion control, namely the terminal completes the self-defined operation setting of the target application.

For example, as shown in fig. 8, it is assumed that the terminal is a mobile phone 1, a recording task setting interface 2 is provided, a task setting completion control 3 may be provided in the recording task setting interface 2, and when a user triggers the task setting completion control 3 on the mobile phone 1, a custom operation of a news application (target application) is completed.

And S105, when the target application is triggered to play the recording task, triggering the first control according to the second operation area information, and realizing the function operation corresponding to the first control.

The terminal determines the second operation area information of the current display interface according to the first operation area information and the mapping relation between the first display window and the second display window, records the second operation area information until the recording is completed, and represents that the terminal completes a recording task to the target application. Here, when the target application is triggered to play the recording task, the terminal determines an action area (second operation area information) of the first control to be triggered, so that the terminal automatically simulates and triggers the first control according to the second operation area information, and the functional operation corresponding to the first control is realized.

It should be noted that, in the embodiment of the present invention, the target application implements recording of the custom operation, and generates the recording task, and the recording task is a function that can be implemented by triggering the custom operation with one key, and the recording task can be triggered by the recording task control, and the specific representation form is not limited in the embodiment of the present invention.

In the embodiment of the present invention, since the functions of the target application are hierarchical, when a series of custom operations of the target application are recorded, each step of the custom operation has a trigger sequence, that is, a trigger time sequence, so that when the terminal plays (that is, starts) a recording task, the terminal needs to send a trigger event for the second operation area information (that is, start or trigger the second operation area information) under a corresponding page or web page according to the trigger sequence during recording. In other words, in the embodiment of the present invention, when the target application is triggered to play the recording task through the recording task control, the trigger event of the second operation area information is sent according to the trigger sequence, so as to implement the simulated triggering of the first control and complete the functional operation corresponding to the first control.

It should be noted that the triggering of the first control by the terminal is not realized by the actual gesture operation of the user, but is simulated by the terminal when the user-defined operation of the first control of the target application is started. In the embodiment of the present invention, the access availability Service (i.e., barrier-free auxiliary Service) is adopted. The process of simulating the user-defined operation by the terminal can be that the terminal simulates and clicks a first region corresponding to the first operation region information on a first display window, at this time, the terminal simulates and triggers a second region corresponding to the second operation region information through an access related interface, and the second region corresponds to a first control, so that the triggering operation of triggering the first control on the current display interface through the access related interface is realized, and the functional operation of the first control is realized.

The self-defined operation application in the embodiment of the present invention is compatible with accessitivervice, the accessitivervice runs in the background, and can receive some events (accessitiveevent, these events represent a series of state changes of the user interface) sent by the operating system of the terminal, such as a focus change, an input content change// button is clicked, and the like, and the service can request to acquire a current active window (i.e., a current display interface) and search the content therein. In other words, any change in the current display interface will generate a time and will be notified to the accessitivyservice by the system. This is like a monitor monitoring the behavior of the interface and immediately issuing an alarm once the interface changes.

It should be noted that, in the embodiment of the present invention, the main interface of the accessibility Service is a class accessibility Service, and the accessibility Service is a sub-class of Service, and this class may be inherited and an abstract method thereof may be implemented to monitor a change of an element state of a current display interface of an application, such as a focus change, a button being clicked, and the like. When there are these changes, the operating system of the terminal will encapsulate these information in the accessibivitev, call back the onassibivitev (accessibivitev) method of the accessibiviteservice, and implement the onassibivitev to process these accessibivitev. The main processes may include:

1. creating a service class; writing a service class needs to inherit an accessitivyservice class, wherein two important methods of onassitiveevent (accessitiveevent) and onnterruput () are to be realized. Wherein onaccestibiteyEvent (accessibiityEvent) is a callback function related to an accessibiityEvent event, and the operating system continuously sends the accessibiyEvent to the event through sendAccessabiliteyEvent ().

2. Declaring a service; like other Service services, it needs to be declared in android manifest. (1) Configuring < intent-filter >, wherein the name of the < intent-filter > is fixed android. (2) And declare BIND _ access _ SERVICE rights so that the system can BIND the SERVICE (post-release requirement 4.1).

Note that: at any point of the configuration error, the system cannot detect the service, so its fixed configuration is as follows:

3. configuring a service class; xml declares the service, which can be configured to accept events of a specified type, listen to a specified package, retrieve window contents, obtain the time of the event type, etc., then some parameter settings need to be made for the service. There are currently two methods of configuration: (1) 4.0 later provides that it can be configured with < meta-data > tag; (2) and configured by setServiceInfo ().

4. Starting service; after the operation is finished, the app can be installed in the mobile phone, after the app is installed successfully, the service can be found in the set-auxiliary function, and the service is in a closed state by default and needs to be started manually.

In the embodiment of the present invention, the service is an accessibility service, and the application having the accessibility service may be exemplarily a custom operation application.

5. Acquiring event information; onaccessibivityevent (accessibivityevent event) is a core method of the service, wherein parameter events encapsulate information from interface-related events, for example, we can obtain the event type of the event, and then select different processing modes according to the type:

6. acquiring window content; it is not sufficient to know only the information of the event, by which the information of the event (event source) is obtained, such as the text of the Button when it is clicked.

The window content here may be operation area information in the embodiment of the present invention.

7. The life cycle of the service; it is to be understood that the life cycle of the service in this case only needs to remember the following three points: (1) the service is completely managed by the system and follows the existing service period; (2) opening a service can only be opened by the user in the setting, and closing can only be closed by the user in the setting or the service itself is closed by the dialeself () method (of course, some third playing software can also force to close the type of service); (3) after the system binds the service, it calls the onServiceConnected () method, which can be rewritten, where some initialization operations can be done.

8. It is detected whether the service is open. (1) The method is judged by a service manager, namely, the access availability manager, but the method cannot detect the service started by the app; (2) most system attributes are set in settings, such as Wi-Fi and bluetooth statuses, and these information are mainly stored in a database (system table and secure table) corresponding to the settings, and whether related services are opened or not can also be judged by directly reading the settings.

It should be noted that, in the embodiment of the present invention, to use the accessitivyservice, the server must be turned on to detect whether the server is turned on, so as to implement the simulation operation.

The following sees a step-by-step use example:

here, by using an example of converting text into voice in ApiDemo, after configuring a server class, a configuration declaration of accessitivyservice needs to be made, and specifically, the declaration needs to be stated in a maniest file:

it should be noted that after the service class is configured and the service is started, the event processing flow may be:

the Accessibility is a message capable of acquiring the control, and the Accessibility is started from View, requested to ViewRootImpl in a reverse recursion way, then subjected to Accessibility Managerservice management core and finally sent to Access Accessibility service.

Events are related to View, which starts from View, which implements an interface called Access availability EventSource, which includes two functions:

public void sendAccessibilityEvent (int eventType); and the number of the first and second groups,

public void sendAccessibilityEventUnchecked(AccessibilityEvent event)；

the two functions may be used to send an accessibityevent, such as when a click event is received by View, when the Focus status of View changes, etc., View calls the two interfaces to start sending the accessibityevent. In addition, the ViewGroup also realizes a ViewParent interface, the ViewParent has an interface function requestSendsessabiliteEvent, in a child View, the sendAccessabiliteEvent method finally calls getParent (). requestSendsessabiliteEvent layer by layer, calls up layer by layer, and finally sends the accessiteEvent to the accessiltyManagerService by using a sendACCessabiliteEvent method binder mechanism of the accessiltyManager in the ViewRotoitllpl. The accessitimymanagervice is initialized in the SystemServer and added to the ServiceManager. The accessitimimenagervice updates the accessituterservicebinding when there is a packet change (install, uninstall), as shown by the following code:

when a new accessitivervice appears, the accessitivervice will be bound to obtain a binding result iacceserviceclient (actually, the Proxy of iacceserviceclientwrapper returned by onBinder in the accessitivervice), and information related to communication with the accessitivervice is stored in an accessitivemnagervice. After the AccessabilityManagerService receives the Event, all Services are traversed, and the Event is sent to the AccessabilityService through the IAaccessitivyServiceClient, so that the generation of the AccessabilityEvent Event and the sending of the Event to the AccessabilityService are realized.

Further, as shown in fig. 9, before S101, an information processing method provided in an embodiment of the present invention may further include: S106-S107. The following were used:

and S106, establishing a recording task of the target application, wherein the recording task is realized through a recording function.

The terminal can realize the recording function aiming at any application installed on the terminal, namely, a plurality of target applications can be provided in the embodiment of the invention, and the terminal can establish the recording task of each target application aiming at each target application, thereby distinguishing the recording function of a plurality of target applications and the realization of custom operation among different target applications. Therefore, before the information processing method of the target application is implemented, that is, before the terminal adds the first display window on the current display interface, the terminal needs to establish a recording task of the target application, wherein the recording task is implemented by using a recording function, that is, the task item of the recording task is established first, and finally a series of task settings for completing the recording task settings need to be performed.

It should be noted that, in the embodiment of the present invention, the information processing method is described by taking the custom operation application as an example, so that when the custom operation application is started, the custom operation application displays a task interface, so that a user can play a recording task or establish a new recording task, and a specific implementation manner is not limited in the embodiment of the present invention.

Here, the terminal determines a target application among the installed applications on the task interface; and the terminal triggers and adds a new event, adds the target application as an object of the recording task and completes the establishment of the recording task.

It should be noted that, in the embodiment of the present invention, a control capable of playing an old recording task and a control instructing to add a new recording task are displayed on a task interface, and when a recording task of a new target application needs to be implemented, the addition of the target application needs to be performed through the control of an additive recording task, so that a terminal displays what applications are installed on the task interface, and thus, a user can select a target application that is desired to perform a custom operation in the installed applications, and then add the recording task to the selected target application by triggering to add a new event, thereby completing the establishment of the recording task of the target application.

And S107, triggering a recording control corresponding to the target application and starting recording.

After the terminal establishes the recording task of the target application, the recording task is realized through the recording function, so that the realization of the specific recording task needs to trigger the recording control corresponding to the target application, and the trigger of the sub-space is used for starting recording (namely starting the recording function and starting recording). That is to say, the terminal needs to set the recording task of the target application to implement a specific user-defined operation task in the recording task, the recording control is a start key for setting the recording task, and the recording function is a means for implementing setting the recording task.

It should be noted that, in the embodiment of the present invention, after the recording task of the target application is established, the recording task setting interface is entered, and the recording control of the target application is arranged on the recording task setting interface, so that when the user clicks the recording control on the recording task setting interface, the recording function is started, and recording is started.

Exemplarily, as shown in fig. 10, in a task interface 2 of a mobile phone 1 (terminal), a news application 5 installed on the mobile phone 1 is selected as a target application, a "new event addition" is clicked to establish a recording task of the news application, the recording task setting interface 3 shown in fig. 10 is skipped to, the task setting is performed, a recording control 4 is arranged on the recording task setting interface 3, and after the recording task of the news application is established, a recording function can be entered by clicking the recording control 4, so that the recording of the user-defined operation of the target application can be started.

Further, as shown in fig. 9, after S104, the information processing method provided in the embodiment of the present invention may further include: S108-S109. The following were used:

and S108, closing the first display window and returning to the recording task setting interface.

And S109, setting the completion of the recording task on a recording task setting interface, and displaying the recording task of the target application.

After the terminal triggers the first control according to the second operation area information and realizes the functional operation corresponding to the first control, because the monitoring of the operation area information of the target application by the terminal is realized through the first display window, after the recording of the user-defined operation of the target application is realized, the monitoring of the operation area information is not needed, and then the terminal can close the first display window and return to the recording task setting interface to realize the confirmation of the completion of the recording task. The user can trigger or click the task setting completion control to complete the set recording task, so that the recording task of the target application is displayed on the task interface, and the user can play the recording task of the target application in a user-defined mode through the displayed recording task.

For example, as shown in fig. 11, after completing recording of a recording task, a mobile phone 1 (terminal) returns to a recording task setting interface 2, a task setting completion control 3 is arranged on the recording task setting interface 2, and the setting of the recording task of a news application 4 (target application) is completed through the task setting completion control 3, so that the user jumps to a task interface 5, and a recording task 6 of the news application 4 (for example, a news application one-key opening task and the like) can be displayed in the task interface 5, so that the user can play the recording task by one-key to realize the functional operation of the first control.

In the embodiment of the invention, the terminal can perform the pre-recording task on the commonly used operation (namely, the functional operation realized by triggering the first control) of the commonly used target application, so that the terminal can directly realize the functional operation by triggering the playing and recording task each time the functional operation is realized. Moreover, in the embodiment of the present invention, the method for recording the task by the terminal may implement the function operation of calling the corresponding operation area information of the second display window of the target application by monitoring the operation area information of the touch operation of the first display window mapped with the second display window of the current display interface, and implement the recording of the function operation.

For a case that a custom operation is implemented as a series of operations of a target application, based on the implementation of the foregoing embodiment, an information processing method provided in an embodiment of the present invention is based on fig. 2, and as shown in fig. 12, a process of S104 may include: S201-S203, as follows:

s201, determining second operation area information of a current display interface according to the first operation area information and the mapping relation between the first display window and the second display window, and recording the second operation area information;

s202, simulating function operation corresponding to the second operation area information, and updating the current display interface;

and S203, performing the next information processing flow on the updated current display interface until the recording is finished.

After the terminal acquires the first operation area information corresponding to the first control according to the first touch instruction, due to the mapping relationship between the terminal or the first display window and the second display window, the terminal can determine the second operation area information on the current display interface, which has the mapping relationship with the first operation area information, according to the first operation area information on the first display window and the mapping relationship between the first display window and the second display window, wherein the second operation area information is the operation area information on the second display window. Therefore, the terminal can simulate to perform touch operation on the second area corresponding to the second operation area information through the function or Service of the Accessibility Service, and jump to a new display interface, that is, the current display interface is updated, and since the touch operation can be the first or non-last user-defined operation of the terminal for the target application, the terminal may need to continue the next user-defined operation on the target application on the updated current display interface, so as to realize the next information processing flow (i.e., the process from S102 to S105 in the previous information processing flow is repeated) until the recording is completed, and thus, the recording of a series of user-defined operations for one target application is realized.

For example, as shown in fig. 13, after a touch operation is performed on a current display interface 2 of a news application (target application) on a mobile phone 1 (terminal) for a quick report 3, a quick report interface 4 shown in fig. 13 is skipped to, where the quick report interface 4 is an updated current display interface, and a trigger operation on any one quick report sub-control 5 can be implemented in the quick report interface 4, and determination and recording of second operation area information on the quick report interface 4 are implemented again until a recording completion control is triggered and recording is finished.

In the embodiment of the invention, the terminal can monitor the operation area information aiming at the target application through the first display window, acquire the operation area information on the target application through mapping, and realize the touch operation on the target application through simulation, so that the terminal can record a series of self-defined operations, further realize the realization of the series of self-defined operations of the subsequent one-key touch target application, and reduce the complexity of the automatic realization operation of the target application.

Based on the same inventive concept of the foregoing embodiments, as shown in fig. 14, an embodiment of the present invention provides a terminal, including:

the adding unit 10 is configured to add a first display window on a current display interface when a recording function of a target application is started, where the first display window and a second display window of the current display interface are in a mapping relationship;

a receiving unit 11, configured to receive a first touch instruction in the first display window, where the first touch instruction is used to instruct to trigger a function operation corresponding to a first control on the current display interface;

the obtaining unit 12 is configured to obtain first operation area information corresponding to the first control according to the first touch instruction;

a determining unit 13, configured to determine second operation region information of the current display interface according to the first operation region information and a mapping relationship between the first display window and the second display window,

a recording unit 14, configured to record the second operation area information until recording is completed;

and the triggering unit 15 is configured to trigger the first control according to the second operation area information when the target application is triggered to play the recording task, so as to implement a function operation corresponding to the first control.

Optionally, the terminal further includes: an update unit 16;

the determining unit 13 is specifically configured to determine the second operation area information of the current display interface according to the first operation area information and the mapping relationship between the first display window and the second display window,

the recording unit 14 is specifically configured to record the second operation area information;

the updating unit 16 is configured to simulate a function operation corresponding to the second operation area information, and update the current display interface; and carrying out the next information processing flow on the updated current display interface until the recording is finished.

Optionally, the terminal further includes: a building unit 17;

the establishing unit 17 is configured to establish the recording task of the target application before adding the first display window on the current display interface, where the recording task is implemented by a recording function;

optionally, the establishing unit 17 is specifically configured to determine the target application in the applications installed on the task interface; and triggering and adding a new event, and adding the target application as an object of the recording task to complete the establishment of the recording task.

Optionally, the triggering unit 15 is further configured to trigger a recording control corresponding to the target application after the recording task of the target application is established, and start recording.

Optionally, the terminal further includes: a closing unit 18 and a display unit 19;

the closing unit 18 is configured to determine second operation area information of the current display interface according to the first operation area information and the mapping relationship between the first display window and the second display window, record the second operation area information until recording is completed, close the first display window, and return to the task interface;

the display unit 19 is configured to set the recording task to be completed in the task interface, and display the recording task of the target application.

Optionally, the triggering unit 15 is specifically configured to send the triggering event of the second operation area information according to the triggering sequence when the target application is triggered to play the recording task, so as to implement simulated triggering on the first control and complete the functional operation corresponding to the first control.

As shown in fig. 15, an embodiment of the present invention further provides a terminal, including: a processor 110, a receiver 111, a display 112, and a storage medium 113, the receiver 111, the display 112, and the storage medium 113 communicating with the processor 110 through a communication bus 114;

the storage medium 113 for storing executable instructions;

the receiver 111, configured to receive an executable instruction;

the processor 110 is configured to execute executable instructions stored in the storage medium 113, and the executable instructions are configured to execute the information processing method described in any one of the foregoing embodiments;

the display 112 is used for displaying information after the executable instructions are executed.

In detail, the processor 110 is configured to add a first display window on a current display interface when a recording function of a target application is started, where the first display window and a second display window of the current display interface are in a mapping relationship;

the receiver 111 is configured to receive a first touch instruction in the first display window, where the first touch instruction is used to instruct to trigger a function operation corresponding to a first control on the current display interface;

the processor 110 is further configured to obtain first operation area information corresponding to the first control according to the first touch instruction; determining second operation area information of the current display interface according to the first operation area information and the mapping relation between the first display window and the second display window, and recording the second operation area information until the recording is finished; and when the target application is triggered to play and record the task, triggering the first control according to the second operation area information, and realizing the function operation corresponding to the first control.

Optionally, the processor 110 is specifically configured to determine the second operation area information of the current display interface according to the first operation area information and the mapping relationship between the first display window and the second display window, and record the second operation area information; simulating the function operation corresponding to the second operation area information, and updating the current display interface; and carrying out the next information processing flow on the updated current display interface until the recording is finished.

Optionally, the processor 110 is further configured to establish the recording task of the target application before adding the first display window on the current display interface, where the recording task is implemented by a recording function; determining the target application in the installed applications on the task interface; and triggering and adding a new event, and adding the target application as an object of the recording task to complete the establishment of the recording task.

Optionally, the processor 110 is further configured to trigger a recording control corresponding to the target application after the recording task of the target application is established, and start recording.

Optionally, the processor 110 is further configured to determine second operation area information of the current display interface according to the first operation area information and the mapping relationship between the first display window and the second display window, record the second operation area information until the recording is completed, close the first display window, and return to the task interface;

the display 112 is configured to set the recording task to be completed in the task interface, and display the recording task of the target application.

Optionally, the processor 110 is further specifically configured to send the trigger event of the second operation area information according to a trigger sequence when the target application is triggered to play the recording task, so as to implement simulated triggering on the first control and complete the functional operation corresponding to the first control.

It should be noted that, in practical applications, the various components in the terminal are coupled together by a communication bus 114. It is understood that the communication bus 114 is used to enable connective communication between these components. The communication bus 114 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for clarity of illustration the various buses are labeled in figure 15 as communication bus 114.

It can be understood that, since the terminal can perform the pre-recording task on the commonly used operation (i.e., the functional operation implemented by triggering the first control) of the commonly used target application, the terminal can directly implement the functional operation by triggering the play-recording task each time the functional operation is implemented. Moreover, in the embodiment of the present invention, the method for recording the task by the terminal may implement the function operation of calling the corresponding operation area information of the second display window of the target application by monitoring the operation area information of the touch operation of the first display window mapped with the second display window of the current display interface, and implement the recording of the function operation.

The embodiment of the invention provides a computer storage medium, which is applied to a terminal and stores machine instructions, and when the machine instructions are executed by one or more processors, the processors execute the information processing method in any one of the preceding embodiments.

The computer-readable storage medium may be a magnetic random access Memory (FRAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM), among others.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Claims

1. An information processing method characterized by comprising:

when the recording function of the target application is started, adding a transparent and invisible first display window with the size consistent with that of the screen of the current display interface on the current display interface, wherein the first display window and a second display window of the current display interface are in a mapping relation;

determining the operation area information actually operated on the first control on the second display window according to the first operation area information and the mapping relation between the first display window and the second display window to obtain the second operation area information of the current display interface, and recording the second operation area information until the recording is finished;

when the target application is triggered to play the recording task, a trigger event aiming at the second operation area information is sent according to the trigger sequence of each step of self-defined operation when the recording task is recorded, and the first control is triggered in a simulated mode, so that the function operation corresponding to the first control is realized.

2. The method according to claim 1, wherein the determining, according to the first operation region information and the mapping relationship between the first display window and the second display window, the operation region information actually operated on the first control on the second display window to obtain the second operation region information of the current display interface, and recording the second operation region information until the recording is completed comprises:

determining operation area information, actually operating the first control, on the second display window according to the first operation area information and the mapping relationship between the first display window and the second display window to obtain the second operation area information of the current display interface, and recording the second operation area information;

simulating the function operation corresponding to the second operation area information, and updating the current display interface;

and carrying out the next information processing flow on the updated current display interface until the recording is finished.

3. The method according to claim 1, wherein before the current display interface adds the first display window which is consistent with the screen size of the current display interface, transparent and invisible, the method further comprises:

and establishing the recording task of the target application, wherein the recording task is realized through a recording function.

4. The method of claim 3, wherein the establishing the recording task for the target application comprises:

determining the target application in the applications installed on the task interface;

and triggering and adding a new event, and adding the target application as an object of the recording task to complete the establishment of the recording task.

5. The method of claim 3 or 4, wherein after the recording task of the target application is established, the method further comprises:

and triggering a recording control corresponding to the target application, and starting recording.

6. The method according to claim 1, wherein the operation region information actually operated on the first control on the second display window is determined according to the first operation region information and the mapping relationship between the first display window and the second display window to obtain the second operation region information of the current display interface, and the second operation region information is recorded until after the recording is completed, the method further comprising:

closing the first display window and returning to a task interface;

and setting the recording task completion on the task interface, and displaying the recording task of the target application.

7. The method according to claim 1, wherein when triggering the target application to play a recording task, sending a trigger event for the second operation area information according to a trigger sequence of each step of a custom operation during recording of the recording task, and simulating triggering of the first control to implement a functional operation corresponding to the first control includes:

when the target application is triggered to play the recording task, according to the triggering sequence of each step of the custom operation during recording of the recording task, sending a triggering event aiming at the second operation area information under a corresponding page or webpage, realizing the simulated triggering of the first control and completing the functional operation corresponding to the first control.

8. A terminal, comprising:

the device comprises an adding unit, a processing unit and a processing unit, wherein the adding unit is used for adding a transparent and invisible first display window with the same size as the screen of a current display interface on the current display interface when the recording function of a target application is started, and the first display window and a second display window of the current display interface are in a mapping relation;

a determining unit, configured to determine, according to the first operation region information and the mapping relationship between the first display window and the second display window, operation region information, which is actually performed on the first control on the second display window, to obtain second operation region information of the current display interface,

and the triggering unit is used for sending a triggering event aiming at the second operation area information according to the triggering sequence of each step of self-defined operation when the target application is triggered to play the recording task, simulating and triggering the first control, and realizing the functional operation corresponding to the first control.

9. The terminal of claim 8, wherein the terminal further comprises: an update unit;

the determining unit is specifically configured to determine, according to the first operation region information and the mapping relationship between the first display window and the second display window, operation region information, which is actually performed on the first control on the second display window, to obtain the second operation region information of the current display interface,

the recording unit is specifically configured to record the second operation area information;

the updating unit is used for simulating the function operation corresponding to the second operation area information and updating the current display interface; and carrying out the next information processing flow on the updated current display interface until the recording is finished.

10. The terminal of claim 8, wherein the terminal further comprises: a building unit;

the establishing unit is used for establishing the recording task of the target application before adding a first display window on the current display interface, and the recording task is realized through a recording function;

the establishing unit is specifically used for determining the target application in the applications installed on the task interface; and triggering and adding a new event, and adding the target application as an object of the recording task to complete the establishment of the recording task.

11. The terminal of claim 10,

the triggering unit is further configured to trigger a recording control corresponding to the target application to start recording after the recording task of the target application is established.

12. The terminal of claim 8, wherein the terminal further comprises: a closing unit and a display unit;

the closing unit is configured to determine second operation area information of the current display interface according to the first operation area information and the mapping relationship between the first display window and the second display window, record the second operation area information until recording is completed, close the first display window, and return to the task interface;

and the display unit is used for setting the recording task completion on the task interface and displaying the recording task of the target application.

13. The terminal of claim 8,

and the triggering unit is specifically configured to, when the target application is triggered to play a recording task, send a triggering event for the second operation area information under a corresponding page or web page according to the triggering sequence of each step of the custom operation when the recording task is recorded, so as to implement simulated triggering on the first control and complete a functional operation corresponding to the first control.

14. A terminal, comprising: a processor, a receiver, a display, and a storage medium, the receiver, the display, and the storage medium in communication with the processor through a communication bus;

the storage medium is used for storing executable instructions;

the receiver is used for receiving executable instructions;

the processor, configured to execute executable instructions stored in a storage medium, the executable instructions being configured to perform the information processing method of any one of the above claims 1 to 7;

15. A computer storage medium applied to a terminal and storing machine instructions, wherein when the machine instructions are executed by one or more processors, the processors execute the information processing method according to any one of claims 1 to 7.