WO2021047135A1 - Page clicking method and apparatus, and clicking device - Google Patents

Abstract

Provided are a page clicking method and apparatus, and a clicking device. According to one embodiment of the method, during a page clicking process, a running device generates an operation request according to a position needing to be clicked and a clicking mode, and sends the operation request to the clicking device; the clicking device converts the operation request into an operation event and feeds back same; and a response is then given to the operation event, so as to complete the process of clicking on a page to be clicked on, wherein sending the operation request to the clicking device and receiving the operation event from the clicking device are completed by means of different communication channels, and the communication channels for sending the operation event to the running device by means of the clicking device are realized by means of an input special interface of the running device. Therefore, an operation event is converted into an operation event of a system process for a target program to give a response. According to the flow, page clicking in a running device is automatically completed with assistance from a clicking device, such that the effectiveness of page clicking can be improved.

Description

Method, device and click device for page click

This application claims the priority of a Chinese patent application filed with the Patent Office of the State Intellectual Property Office of China, the application number is 201910858821.1, the title of the invention is "page click method, device and click device" on September 11, 2019. The entire content of the patent application is approved. The reference is incorporated in this application.

Technical field

One or more embodiments of this specification relate to the field of computer technology, and in particular to a method and device for automatically clicking a page on a computer through a clicking device.

Background technique

With the development of automation technology, more and more scenarios can reduce manual participation. For scenarios involving page clicks, there may also be automation requirements. For example, in the field of software testing, it is expected that the testing process will be automated. In the case of interaction of multiple pages, these pages can be jumped by clicking on hyperlinks, buttons, and the like. In conventional technology, the automatic click to jump between mobile terminal pages can usually be achieved in the following ways:

First, the software issues a click command without hardware assistance. In terms of function, this can meet all the click requirements, but this kind of click usually requires root permission or connected to a computer device to be controlled from the computer side, making this method may be limited in deployment methods. If you don't have root privileges, this kind of click cannot be applied to the system pop-up dialog box. When the system pop-up dialog box, it may interrupt the automated process.

Second, the hardware cooperates to realize the click of a fixed position, such as the click head used for scrolling the rankings. When the click position on the page is inconsistent, the correct click may not be realized automatically.

Summary of the invention

One or more embodiments of this specification describe a page click method, device, and click device, which can realize the automatic click operation at any position on the page through the cooperation of the running device and the click device, thereby improving the effectiveness of the automatic page click .

According to the first aspect, a page click method is provided. A page click method is applied to a running device that is connected to a pointing device in advance. The running device is used to run a target program, and the pointing device passes the first The communication module and the second communication module respectively establish a first communication channel and a second communication channel with the operating device, and the second communication channel is established based on the input dedicated interface of the operating device; the method includes: in the target program running process In the case of a page to be clicked in, an operation request is sent to the pointing device through the first communication channel, so that the pointing device can convert the operation request into an operation event, and perform the operation through the second communication channel Feedback, the operation event is used to describe the offset operation and/or the click operation indicated in the operation request; respond to the operation event to complete the click on the page to be clicked.

In a possible design, the sending the operation request to the pointing device through the first communication channel includes: generating a positioning page and a first operation request, where the positioning page is used to cover the page to be clicked; And give priority to responding to the operation event sent by the pointing device, the page to be clicked corresponds to at least the first target position to be clicked and the first click mode for the first target position; and the first click mode is sent to the pointing device. An operation request for the pointing device to generate a first click event according to the first operation request, the first click event is used to describe a click operation performed by the pointing device according to a preset click mode; the response The operation event to complete the click on the page to be clicked includes: responding to the first click event through the positioning page, and determining the first click position of the current click; based on the first click position and the current click position; The comparison of the first target position is described, and the click on the page to be clicked is completed.

According to a further implementation manner, when the first click position is consistent with the first target position, the responding to the operation event to complete the click on the page to be clicked further includes: canceling the click position. The positioning page; sending a second operation request to the pointing device through the first communication channel, so that the pointing device converts the second operation request into a second operation event for feedback through the second communication channel The second operation event is used to describe the click event corresponding to the first click mode; the second operation event is responded to through the page to be clicked, so as to complete the click on the page to be clicked.

According to another further implementation manner, in a case where the first click position and the first target position are inconsistent, the responding to the operation event to complete the click on the page to be clicked further includes: respectively Acquire the coordinates of the first click position and the first target position in the same page coordinate system; determine from the first click position to the first click position according to the coordinates of the first click position and the first target position The coordinate offset of the first target position; a third operation request is generated based at least on the coordinate offset, and sent to the pointing device, so that the pointing device generates a third operation event according to the third operation request ; In response to the third operation event, complete the click on the page to be clicked.

In an optional embodiment, the generating the third operation request based at least on the coordinate offset includes: determining the offset corresponding to the pointing device according to the coordinate offset and a predetermined conversion rule, Wherein, the conversion rule is used to describe the mapping relationship between the movement distance corresponding to the clicking device and the offset distance of the click position corresponding to the clicking device on the screen; according to the offset corresponding to the clicking device and The first click mode generates the third operation request, so that the clicking device generates an offset corresponding to the movement of the clicking device according to the third operation request, and performs the first click according to the first click mode. Three operation events; the completion of clicking on the page to be clicked in response to the third operation event further includes: canceling the positioning page, and responding to the third operation event through the page to be clicked.

In another optional embodiment, the generating a third operation request based at least on the coordinate offset includes: using the coordinate offset as an offset corresponding to the pointing device to generate a third operation request , So that the clicking device generates a third operation event offset according to the offset; responds to the three operation events through the positioning page, and determines the second click position generated by the third operation event; Whether the second click position is consistent with the first target position, and a fourth operation request is generated according to the detection result and sent to the clicking device, and so on, until the current click position is consistent with the first target position.

According to a second aspect, a click method is provided, which is applied to a click device that is connected to a running device in advance, and the running device is used to run a target program. The device has established a first communication channel and a second communication channel, and the second communication channel is established based on the input dedicated interface of the operating device; the method includes: receiving an operation sent by the operating device through the first communication channel Request, wherein the operation request is generated by the running device when a page to be clicked appears during the operation of the target program; converted into an operation event of the clicking device according to the operation request, wherein the operation event is Describe the offset operation and/or click operation indicated in the operation request; and send the operation event to the operating device.

In an embodiment, the operation event includes one of the following: a click event used to describe the click event performed by the clicking device according to a preset click mode; a click event used to describe the movement of the clicking device according to a predetermined offset A mobile event, and a click event performed by the clicking device according to a preset click mode.

According to a third aspect, there is provided a page click device, which is provided in a running device that runs a target program, and the device includes: a sending unit configured to communicate through a first communication when a page to be clicked appears during the running of the target program The channel sends an operation request to the pointing device for the pointing device to convert the operation request into an operation event for feedback through the second communication channel, and the operation event is used to describe the offset indicated in the operation request Operation and/or click operation, wherein the pointing device establishes a first communication channel and a second communication channel with the running device in advance through a first communication module and a second communication module, and the second communication channel is based on the running device. The input dedicated interface of the device is established; the response unit is configured to respond to the operation event to complete the click on the page to be clicked.

According to a fourth aspect, there is provided a pointing device for clicking a page on a running device with a pre-established connection; the device includes: a first communication module configured to establish a first communication channel with the running device and pass the The first communication channel receives an operation request sent by the running device, where the operation request is generated by the running device when a page to be clicked appears during the running of the target program; the conversion module is configured to follow the operation The request is converted into an operation event of the clicking device, where the operation event is used to describe the offset operation and/or the click operation indicated in the operation request; the second communication module is configured to be dedicated to the input of the operation device The interface establishes a second communication channel with the operating device, and sends the operation event to the operating device through the second communication channel.

According to a fifth aspect, there is provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect or the second aspect.

According to a sixth aspect, there is provided a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the first aspect or the first aspect is implemented. Two-sided approach.

Through the method and device provided by the embodiments of this specification, during the page click process, the operating device generates an operation request according to the location and click mode that needs to be clicked, and sends it to the clicking device. The clicking device converts the operation request into an operation event and gives feedback. Then, respond to the operation event to complete the click on the page to be clicked. Among them, sending operation requests to the clicking device and receiving operation events from the clicking device are completed through different communication channels, and the communication channel for the clicking device to send operation events to the running device is realized through the input dedicated interface of the running device, thereby converting the operation event The operation event into the system process can be responded to by the target program. The above process is assisted by a click device to automatically complete page clicks in the running device, which can improve the effectiveness of page clicks.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

Figure 1 shows a schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

Fig. 2 shows a flowchart of a method for page click according to an embodiment;

Fig. 3 shows a flow chart of a click method according to an embodiment;

Figure 4 shows a specific example of a schematic diagram of the interaction process between the running device and the clicking device to complete a page click;

Fig. 5 shows a schematic block diagram of a page click device according to an embodiment;

Fig. 6 shows a schematic block diagram of a clicking device according to an embodiment.

detailed description

The following describes the solutions provided in this specification with reference to the accompanying drawings.

For ease of description, description will be given in conjunction with a specific application scenario of the embodiment of this specification shown in FIG. 1. Fig. 1 is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. In this implementation scenario, operating equipment and click equipment are included. The operating device here can be a mobile terminal such as a mobile phone, a tablet computer, or a notebook computer, a personal computer, etc.

The operating equipment can include system programs, test modules, and click control modules. Among them, the system program is usually the basic program of the computer, and the system program can serve various applications or modules loaded in the system. The test module and the click control module are application modules loaded in the computer system. The test module can be used to run the target program. The click control module is used to send an operation request to the clicking device, thereby controlling the generating corresponding operation event of the clicking device. The click control module can be integrated with the test module, or it can be a data processing module independent of the test module, which is not limited here. In the case that the click control module and the test module are integrated, the click control module can be implemented through SDK. At this time, the test module can trigger the operation logic of the click control module by calling. When the click control module and the test module are independent of each other, the test module can trigger the operation logic of the click control module by sending a notification message.

Click devices can be used to generate click events. The pointing device may be a mouse, or a device similar to a mouse function, for example. The structure of the pointing device may be different from that of a mouse. For example, compared to a mouse, the hardware composition of the pointing device can be simplified, and it may not include a button, a scroll wheel, and other structures. The pointing device in the embodiment of this specification includes at least two communication modules, and the two communication devices communicate with the running device through different connection channels. Assuming that the two communication modules are the first communication module and the second communication module, and the corresponding two connection channels are the first communication channel and the second communication channel respectively, the first communication channel can be used to run the device to send information to the pointing device , The second communication channel can be used for the click device to transfer operation events to the running device. It is worth noting that the second communication channel is established based on the input dedicated interface of the operating device and can be identified as a system-level communication channel. Among them, the first and second here only indicate the corresponding relationship, and do not limit the order or name.

In the process of testing the target program, the tested target program may jump to each other by clicking between pages, and display different content by clicking positioning within the current page range, and so on. These pages can all be called pages to be clicked. When a page to be clicked appears during the running of the test target program, the test module can trigger the click control module to send a click request to the click device by sending a notification or invoking, so as to complete the automatic clicking process of the page to be clicked. The page to be clicked may at least correspond to the first target position to be clicked and the first click mode for the first target position. The click control module can obtain the information of the first target position and the first click mode.

In response to the situation where the page to be clicked appears in the target program, the click control module can send an operation request to the click device through the aforementioned first communication channel to notify the click device of what operation is required. The click device can generate corresponding operation events according to the operation request, such as the offset event and click event corresponding to the click device. Taking a mouse as an example, a mouse pointer corresponding to the mouse may be provided on the operating device that establishes a connection with the mouse. When the mouse moves, a relative movement distance is generated. The relative movement can be packaged as an offset event and sent to the electronic device. The operating device converts the relative displacement of the clicking device into the corresponding offset distance of the mouse pointer on the screen by analyzing the offset event. When the operation event also includes a click event, the running device can respond to the relevant click event at the position where the mouse pointer finally moves.

In an optional implementation manner, the operating device may also first generate a positioning page. The positioning page can be overlaid on the page to be clicked, and give priority to responding to click events received by the system program. After the positioning page is generated, or at the same time, the click control module may send an operation request to the click device. The operation request may include a preset click mode, such as a left click. The clicking device can generate a click event according to the click pattern included in the first click request, and send it to the running device. After the system program of the running device receives the click event, it responds to the click event through the positioning page and determines the current first click position. Further, the click control module can compare the first click position with the first target position to be clicked on the page to be clicked, and according to the comparison result, determine whether it is necessary to adjust the setting of the clicking device based on the first click position of the current click. Click on the location. Further, an operation request is generated accordingly, so that the clicking device adjusts the offset distance and/or the click mode according to the operation request, and generates a new click event to complete the click operation on the page to be clicked.

In this way, in the implementation scenario shown in Figure 1, the click control module transmits the required operation information to the click device, and the click device directly generates the corresponding operation event according to the operation information without generating real and physical operations, which can improve Effectiveness of page auto-click.

The following describes the technical ideas of this specification from the perspective of the operating device and the point of view of the clicking device.

Figure 2 shows a page click method of an embodiment. The execution subject of the method shown in FIG. 2 may be an operating device (the operating device shown in FIG. 1), that is, a device that runs the target program. Further, the method shown in FIG. 2 may be executed by a module in the running device, such as the click control module in FIG. 1. The module can be implemented by software or hardware, which is not limited here. This step can be performed by a module in the running device, for example, the click control module shown in FIG. 1.

The running device can establish a connection with the click device in advance. Among them, the running device and the pointing device can establish a connection through two mutually independent communication channels. Specifically, the running device may establish a first communication channel with the first communication module in the pointing device, and establish a second communication channel with the second communication module in the pointing device. Understandably, the communication channel here is a more vivid description method. In actual implementation, it may be in a wired manner or in a wireless manner, which is not limited here. The wireless method may be Bluetooth, infrared, etc., for example. In one embodiment, the first communication module is a Bluetooth module, and the second communication module is an infrared connection module. Accordingly, the first communication channel may be in a Bluetooth connection mode, and the second communication channel may be in an infrared connection mode. In the case where the first communication module and the second communication module are the same type of communication modules, the first communication channel and the second communication channel may also have different channel identifiers. For example, the channel ID of the first communication channel is "4089", the channel ID of the second communication channel is "1078" and so on.

It is worth noting that computer equipment, such as desktop computers, mobile terminals, etc., usually reserve dedicated input device interfaces for the equipment. Devices connected through a dedicated interface are recognized as system-level input devices. In the implementation architecture of this specification, the second communication channel may be established based on the input dedicated interface of the operating device. In this way, the information sent by the clicking device to the running device can be received and recognized by the system process.

The method shown in FIG. 2 includes the following steps: Step 201, when a page to be clicked appears during the running of the target program, an operation request is sent to the pointing device through the first communication channel, so that the clicking device can convert the operation request into an operation event , And feedback through the second communication channel, the operation event is used to describe the offset operation and/or click operation indicated in the operation request; step 202, respond to the above operation event to complete the click on the page to be clicked.

First, in step 201, when a page to be clicked appears during the running of the target program, an operation request is sent to the pointing device through the first communication channel.

Here, the target program is determined according to the scene. For example, in an automated test scenario, the target program is the program under test. In scenarios such as automatic verification, the target program can be various scenarios that include verification information. The target program can be run on the running device. During the running of the target program, a page jump may occur. The page jump can be realized automatically through the h5 page, or through clickable elements such as hyperlinks and buttons in the current page to be clicked. It can be understood that the page to be clicked appears during the running of the target program, which means that the current page displayed by the target program contains clickable elements, such as hyperlinks, buttons, and so on.

It can be understood that the element to be clicked in the page to be clicked is the object to be clicked. Each object to be clicked usually corresponds to its position on the current page and its click mode. Among them, the position in the current page can be expressed by coordinates in the screen coordinate system. The screen coordinate system can be the system default coordinate system. Optionally, the coordinates in the screen coordinate system can be represented by pixels. For example, (1, 2) can represent a pixel that is one pixel away from the origin of the coordinate in the horizontal direction and 2 pixels away from the origin of the coordinate in the vertical direction. The click mode may be left-click, left-click double-click, right-click, etc., for example.

When there are multiple click objects on the page to be clicked, the current process can target one of the to-be-clicked positions and the corresponding click mode. In this specification, for the convenience of description, it can be assumed that the currently targeted object to be clicked is the first object to be clicked, and the corresponding position and click mode of the first object to be clicked can be recorded as the first target position and the first target position accordingly. The first click mode.

When a page to be clicked appears during the running of the target program, a corresponding operation request can be generated and sent to the pointing device through the first communication channel. The corresponding offset and/or click requirement may be indicated in the operation request. It can be understood that after the connection between the clicking device and the running device is established, any operation of the clicking device will generate a corresponding operation event and feedback to the running device. Take the mouse as an example, the relative position offset of the mouse will be reflected by the mouse pointer of the device connected with the mouse. The relative movement of the mouse and the movement distance of the mouse pointer on the screen may have a certain mapping relationship. The various click modes of the mouse will also respond to the current position of the mouse pointer. Under the technical concept of this specification, the pointing device may not physically move and click, but it needs to generate an operation event similar to a mouse, that is, an event corresponding to an offset operation and/or a click operation. The specific content of these events (such as how to offset, how to click) can be indicated by the operation request sent by the running device to the pointing device.

The click device can convert the operation request sent by the operation device into a corresponding operation event by analyzing the operation request sent by the operation device, and use the second communication module to feed back to the operation device through the second communication channel. It can be understood that the operation event sent by the clicking device to the running device may be system-level, and the system may respond to the corresponding operation event according to the priority level of each application currently running. Therefore, in step 202, the click operation on the page to be clicked can be completed on the basis of responding to the above-mentioned operation event.

It can be understood that, in practice, when the operation event generated by the clicking device includes an offset event, the offset in the offset event is usually the relative offset of the clicking device. In the technical concept of this specification, the relative offset corresponding to the pointing device is a virtual relative offset, that is, the pointing device does not generate a physical offset, but it has the same meaning as the relative offset generated by the pointing device. The unit of the relative offset of the clicking device and the offset of the click component on the screen corresponding to the clicking device in the running device may be the same or different. For example, the relative offset of a common pointing device mouse may be calculated in units of dimensions such as millimeters and centimeters, while the cursor may be calculated in units of pixels. In an operational event, if the passed data does not have the same measurement standard, a conversion rule is required between the two. In the manual operation process, this conversion rule can be ignored because it is visible. However, for the automatic click scenario, the operating device needs to indicate the corresponding offset. For a variety of different situations, when generating an operation request, there may be different implementation manners, and there may be different corresponding ways to the operation event.

According to one embodiment, the position of the corresponding positioning pointer (such as the mouse pointer) of the pointing device in the running device cannot be determined, and it needs to be positioned first. Then, the operation request sent to the pointing device in step 201 may be a click request. The purpose of this click is to determine the current click position, that is, to locate the current position of the pointer. Therefore, the current operation can only include click operations in any click mode (without offset). For the convenience of description, the operation request generated in this case may be referred to as the first operation request. The operating device may generate the first operation request according to a predetermined click pattern, or may randomly designate a click pattern to generate the first operation request, which is not limited here. The click device generates the first click event according to the first operation request and provides feedback.

It can be understood that if the first click event is responded to through the current page to be clicked, since the click position corresponding to the first click event cannot be determined in advance, it is assumed that the current to be clicked is the first target position on the page to be clicked, and the click mode is first Click mode, then, directly respond to the operation event through the page to be clicked, the possible click position may be the second target position to be clicked other than the first target position, etc., which may easily cause accidental clicks. In order to prevent accidental clicks, the positioning page may be generated before or at the same time when the first operation request is generated. As the name implies, the positioning page is an auxiliary page used to locate the click location. The function of the positioning page is to determine the current click position. Therefore, the positioning page can cover the page to be clicked and give priority to responding to the operation event sent by the clicking device. In this way, the current click position can be obtained by positioning the page. Here, the click position generated by a click event can be called the first click position.

Then, based on the comparison between the first click position and the first target position, the click on the page to be clicked can be completed. Wherein, the comparison between the first click position and the first target position can be realized by their respective coordinates in the screen coordinate system, which will not be repeated here.

Further, in one embodiment, if the first click position is consistent with the first target position, the positioning page can be cancelled, and the second operation request is sent to the pointing device through the first communication channel. The second operation request here indicates the first click mode. In this way, the click event generated by the click device is to perform a click operation in situ according to the first click mode, which is referred to as the second operation event here. At this time, by responding to the second operation event through the page to be clicked, clicking on the first target location according to the first click mode can be completed.

In another embodiment, if the first click position and the first target position are inconsistent, the click position needs to be adjusted from the first click position to the first target position. At this time, the coordinates of the first click position and the first target position in the screen coordinate system can be obtained separately, and the coordinate offset from the first click position to the first target position is determined according to the coordinates of the first click position and the first target position And generate a third operation request based on at least the coordinate offset. The third operation request is sent to the clicking device, so that the clicking device generates a third operation event according to the third operation request, responds to the third operation event, and completes the click on the page to be clicked.

It can be understood that, in practice, the offset corresponding to the click device and the offset of the click position (ie, the offset of the click component in the running device) may be stored in advance, and may have a predetermined conversion rule. For example, when the translation ratio is 1:2, that is, when the offset corresponding to the click device is 1 unit (such as millimeter), the offset of the click position is 2 units (such as pixels). In this embodiment, the offset of the click position is known, that is, the coordinate offset from the first click position to the first target position. In this way, in an optional implementation manner, the offset corresponding to the clicking device can be determined according to the coordinate offset and a predetermined conversion rule. It is worth noting that in the conventional technology, the offset corresponding to the clicking device is the relative movement of the clicking device, and in this embodiment, the offset corresponding to the clicking device converted by the operation request is a virtual offset. , Generate an offset event consistent with the corresponding offset of the actual click device movement.

As a simple example, suppose that the coordinate offset between the first click position and the first target position is (200, 400), and the conversion rule is: the offset corresponding to the click device and the offset of the click position (ie The translation ratio between coordinate offset) is 1:2, then the offset corresponding to the click device is (100, 200).

After generating the third operation request according to the aforementioned coordinate offset and the first clicking mode, and sending it to the clicking device, the clicking device can convert it into a corresponding operation event for feedback. That is, in the foregoing example, the operation request includes a horizontal offset of 100 pixels, a vertical offset of 200 pixels, and the first click mode corresponding to the first target position. The operation information generated by the click device includes an offset of 100 units in the horizontal direction, an offset of 200 units in the vertical direction, and a click event performed through the first click mode. At this time, the positioning page can be cancelled, and the click on the first target position on the page to be clicked is completed by responding to the corresponding operation event through the page to be clicked.

In practice, if the conversion rule between the offset corresponding to the clicking device and the offset of the click position (that is, the offset of the positioning pointer in the running device) is not stored in advance, accurate operations cannot be directly indicated. Under the inventive concept of this specification, the pointing device does not actually have an offset, but the operation request sent by the operating device is received through the first communication module and converted into a corresponding click event. The offset of the clicking device is virtual in the click event. In this way, for the actual click effect of the click event, a click test is required.

In an alternative implementation manner, the idea of manual operation can be used for reference, and the click position is changed iteratively until the click position is consistent with the first target position. Specifically, the coordinate offset is used as the offset corresponding to the pointing device, and the third operation request is generated. The third operation request may also indicate any of the aforementioned click events or click events of a predetermined click mode. In this way, the pointing device can generate a third operation event offset according to the offset according to the third operation request. Respond to the three operation events through the positioning page, and determine the second click position generated by the third operation event, detect whether the second click position is consistent with the first target position, and generate a fourth operation request based on the detection result and send it to the clicking device. By analogy, until the click position corresponding to the operation event sent by the click device is consistent with the first target position.

In another alternative implementation, the click position obtained by clicking again can be used to determine the conversion between the offset corresponding to the click device and the offset of the click position (ie the offset of the positioning pointer in the running device) Rules, and click in accordance with the obtained conversion rules. Specifically, an arbitrary offset or a coordinate offset between the first click position and the first target position is used as the offset corresponding to the clicking device, and the third operation request is generated. The third operation request may also indicate a click event of a predetermined click mode. In this way, the pointing device can generate a third operation event offset according to the offset according to the third operation request. Respond to the third operation event by positioning the page, determine the second click position generated by the third operation event, and detect the coordinate offset between the second click position and the first click position. According to the relationship between the offset in the third operation request and the coordinate offset of the second click position relative to the first click position, it can be determined between the offset corresponding to the click device and the offset of the click position Conversion rules. Afterwards, the corresponding operation event can be generated according to the conversion rule, which will not be repeated here.

Among them, in each of the foregoing embodiments, each time the operating device sends an operation request to the clicking device and responds to the operation event sent by the clicking device, it can be regarded as a specific implementation process of the flow in FIG. 2 and will not be repeated here. In each of the above-mentioned embodiments, the process shown in FIG. 2 has been executed at least twice.

According to another embodiment, the coordinates of the final click position in the previous operation event may also be recorded in advance. At this point, when the current operation event is generated, it is no longer necessary to locate the page, but according to the coordinate offset from the first click position to the first target position, the offset corresponding to the click device and the offset of the click position The conversion rule and the first click mode generate an operation event, and the click on the first target location according to the first click mode is completed directly according to the operation event feedback from the clicking device.

Next, please refer to FIG. 3, which shows a click method, which is applied to a click device that is connected to a running device in advance. The running device is used to run the target program, and the pointing device establishes a first communication channel and a second communication channel with the running device through the first communication module and the second communication module, respectively. The second communication channel is established based on the input dedicated interface of the operating device, so that the operating device responds to the operation event of the click device through the system process.

The method shown in FIG. 3 includes the following steps: step 301, receiving an operation request sent by a running device through a first communication channel, where the operation request is generated by the running device when a page to be clicked appears during the running of the target program; step 302 , The operation event is converted into the operation event of the click device according to the operation request, where the operation event is used to describe the offset operation and/or the click operation indicated in the operation request; step 303, the operation event is sent to the operation device.

First, in step 301, the operation request sent by the operating device is received through the first communication channel. Here, the first communication channel is a channel through which the first communication module communicates with the operating equipment. The communication channel is a more vivid speech, and its essence is the pre-appointed communication mode between the first communication module and the operating device. Taking Bluetooth as an example, the first communication module and the operating device may have a dedicated connection channel. The connection channel can be indicated by a channel identifier such as a channel number, or can be distinguished from other communication channels by an agreed specific format. The operation request sent by the running device can carry an agreed channel identifier or a specific format. In this way, the pointing device can recognize that it is information that needs to be received by itself through the agreed channel identifier or a specific format. According to the channel identifier agreed with the first communication module or the information in a specific format, the second communication module does not perform identification and processing. This is equivalent to establishing a dedicated first communication channel between the first communication module and the operating device.

In the automatic click scenario, the operation request here can be generated by the operating device when the page to be clicked appears during the running of the target program, and indicates the offset operation and/or click operation corresponding to the clicking device.

Then, in step 302, the operation event of the click device is converted according to the operation request, where the operation event is used to describe the offset operation and/or the click operation indicated in the operation request. In this step, the information in the operation request can be parsed, and information to be transmitted through the second communication channel can be generated. Wherein, similar to the first communication module, the second communication module and the operating device may have a dedicated connection channel, which is the second communication channel. The second communication channel may also be indicated by a channel identifier such as a channel number, or be distinguished from other communication channels by an agreed specific format. The operation event sent by the click device can have an agreed channel identifier or a specific format. In this way, only operating equipment can receive and recognize operating events.

As a specific example, in the case that the first communication channel and the second communication channel are both wireless communication channels, suppose the operating device sends out the broadcast information "4089#0,0#$1$", indicating that the channel ID is 4089. For the request of the communication module, #0, 0# means horizontal movement 0 pixels, vertical movement 0 pixels, in $1$, 1 means left-click, ## and $$ are all special formats, then click in the device The first communication module recognizes the information sent to its own channel according to the information header "4089" of the broadcast information, and can receive and parse the information. Then, the package information according to the information sending format of the second communication module is: "1078% 0, 0, 1", where 1078 is the channel identifier of the second communication module,% is the special format of the agreed operation event, and the package information is "1078 "%0,0,1" is the information that expresses the operation event. It can be seen that in the conversion process, relative to the operation request, the offset and/or click information contained in the operation event remains unchanged, but only the packaging form has changed.

Step 303: Send an operation event to the running device. For example, the second communication module broadcasts the above package information "1078% 0, 0, 1". At this time, the operating device can identify the information as a click event through the 1078 identifier.

From the above process, it can be seen that in the flow of the click method shown in FIG. 3, the function of the click device is to transparently transmit data. That is, the data received through the first communication module is propagated through the second communication module. The reason for this is that in conventional technology, some terminals have a sandbox mechanism, and non-system programs have no authority to control the position and click operations of positioning components (such as mouse pointers). Therefore, a click device is needed to transfer the corresponding operation requirements through clicking The equipment is converted into an operation event and forwarded to the operating equipment. It can be understood that the connection between the pointing device and the running device through the second communication module is similar to the connection between the mouse and the computer in the conventional technology. In this way, the second communication module and the running device can communicate at the system level, that is, the information sent to the running device through the second communication module can be received by the running device through the system process, so that it can be oriented to various applications running on it. .

It is worth noting that the embodiments described in FIG. 2 and FIG. 3 describe the interaction cycle from the running device and the clicking device respectively. In each interaction cycle, the operating device is the active party, which can generate different operation information according to requirements, and the clicking device is the follower, which is converted into corresponding operation events according to the received operation request. Each of the specific implementation manners described in FIG. 2 and FIG. 3 can be coordinated and adapted to each other.

In order to more clearly describe the interaction and adaptation between the flow executed by the running device and the flow executed by the clicking device, please refer to FIG. 4, which shows an interactive schematic diagram of a specific embodiment.

As shown in Figure 4, first, in S401, a page to be clicked appears in the target program running on the running device. The page to be clicked may include at least the first target position to be clicked and the first click mode for the first target position, such as single click, double click, and so on.

It is worth noting that, in the case that the execution subject of the page click method is a module in the running device, the module can be embedded in the target program, or it can be a module independent of the target program, which is not limited here. When the module is embedded in the target program, it can be encapsulated by means such as SDK. When the page to be clicked appears in the target program, the first target position and the first click mode are passed as parameters, and the method of calling the module to execute the page click . When the module is independent of the target program, the target program can notify the module in the form of an event when the page to be clicked appears, so that the module can trigger the method of executing the page click through the event.

Next, in S402, a positioning page is generated. The positioning page can be overlaid on the current page, giving priority to responding to click events. In order to facilitate positioning, each position on the positioning page and the current page can be represented by the screen coordinate system. When the positioning page and the current page are located in different coordinate systems, the position relationship is converted through coordinate system conversion.

While S402 is being performed, or afterwards, in S403, an operation request is generated and sent to the pointing device. The operation request here is used to instruct the clicking device to generate the first click on the current page to be clicked (for the current page to be clicked, it is not necessarily on the current page to be clicked). The operation request is also an operation request sent for the first time for the current page to be clicked, and it can indicate a simple click event.

The purpose of this click is to determine the current click position. Therefore, the current click can be a click in any click mode. The operating device can send a click request to the pointing device according to a predetermined click mode, or it can randomly designate a click mode to send a click request to the pointing device, which is not limited here.

Further, in S404, the pointing device receives the operation request sent by the running device through the first communication module, and converts the operation request into the first click event. It can be understood that a traditional clicking device, such as a mouse, generates a click event through a physical relative offset and/or key press action. Under the inventive concept of this specification, the pointing device may not perform relative offset and/or keystroke actions, but directly convert into operation events according to the operation request of the running device, instead of the relative offset and/or physical displacement of the pointing device. Key action.

In this step, the relative offset and/or key action included in the first click request can be converted into the format of the click event, and then the channel identifier corresponding to the second communication module or the agreed specific format can be added.

Then, in S405, the pointing device may feed back the first click event to the running device through the second communication module. The communication mode between the second communication module and the operating device has been described above, and will not be repeated here.

Then, in S406, the running device may respond to the first click event through the positioning page, and locate the current first click position. Since the operating device generates a positioning page in S402 and responds to the click event first, the first click event received by the system process of the operating device here can be responded to through the positioning page. Here, instead of responding directly to the page to be clicked, the click position generated by the current click event can be avoided. If there are other buttons or hyperlinks and other elements that can respond to clicks, accidental clicks can be caused, which makes the page uncontrollable jump. Cause unnecessary confusion.

Next, in S407, it is possible to compare whether the first click position is consistent with the first target position, and generate a new operation request according to the comparison result. In this step, it is possible to compare whether the coordinates of the first click position and the first target position in the screen coordinate system are consistent. If the first click position and the first target position are represented by coordinates in different coordinate systems, the coordinates can also be converted into coordinate representations in the same coordinate system before comparison.

It can be understood that when the two are consistent, the new operation request may only indicate a click operation at the current click position. When the two are inconsistent, the new operation request can indicate the offset operation and the click operation. Here, the offset operation may be the offset from the first click position to the first target position, expressed by the offset amount, and the click operation may be expressed by the first click mode.

In S408, the running device sends the new operation request to the pointing device through the first communication channel.

Then, in S409, the click device receives a new operation request, generates a new operation event accordingly, and sends the new operation information to the operating device through S410. When the new operation request only indicates the sequential click operation, the new operation event only includes the click event, and the new operation request indicates the offset operation and the click operation, the new operation event can include the offset corresponding to the click device Events and click events.

Further, in S411, the positioning page is cancelled, and the click on the first target position is completed according to the new operation request.

Figure 4 shows the interaction process, showing two interaction processes. It can be seen that in each interactive process, the operating device can generate different operation requests according to different needs. And perform other auxiliary operations according to the operation request, such as generating a positioning page, canceling a positioning page, and so on. The click device performs an information conversion operation in each process, and converts what information is received, which is equivalent to assisting the click process in the running device.

Recalling the above process, during the page click process, the operating device generates an operation request according to the location and click mode that needs to be clicked, and sends it to the clicking device. The clicking device converts the operation request into an operation event and gives feedback, and then responds to the operation event to Finish clicking on the page to be clicked. Among them, sending operation requests to the clicking device and receiving operation events from the clicking device are completed through different communication channels, and the communication channel for the clicking device to send operation events to the running device is realized through the input dedicated interface of the running device, thereby converting the operation event The operation event into the system process can be responded to by the target program. The above process is assisted by a click device to automatically complete page clicks in the running device, which can improve the effectiveness of page clicks.

According to another embodiment, there is also provided a page click device. Fig. 5 shows a schematic block diagram of a page click device according to an embodiment. The device 500 may be installed in a running device that runs the target program. The apparatus 500 includes a sending unit 51 configured to send an operation request to the pointing device through the first communication channel when the page to be clicked appears during the running of the target program, so that the pointing device can convert the operation request into an operation event through the second communication channel. The communication channel provides feedback, and the operation event is used to describe the offset operation and/or click operation indicated in the operation request. The clicking device has established a first communication channel with the operating device through the first communication module and the second communication module in advance. And the second communication channel, the second communication channel is established based on the input dedicated interface of the operating device; the response unit 52 is configured to respond to the operation event to complete the click on the page to be clicked.

It is worth noting that the apparatus 500 shown in FIG. 5 is an apparatus embodiment corresponding to the method embodiment shown in FIG. 2, and the corresponding description in the method embodiment shown in FIG. 2 is also applicable to the apparatus 500. Go into details again.

According to another embodiment, a pointing device is also provided. Fig. 6 shows a schematic block diagram of a pointing device according to an embodiment. The click device 600 is used to click on a page on a running device that is connected in advance. The pointing device 600 includes: a first communication module 61, configured to establish a first communication channel with the running device, and to receive an operation request sent by the running device through the first communication channel, wherein the operation request appears during the running of the target program by the running device Generated in the case of the page to be clicked; the conversion module 62 is configured to convert the operation event of the click device according to the operation request, wherein the operation event is used to describe the offset operation and/or the click operation indicated in the operation request; second communication The module 63 is configured to establish a second communication channel with the running device through the input dedicated interface of the running device, and send operation events to the running device through the second communication channel.

In a possible design, the pointing device can be implemented with a mouse.

It is worth noting that the device 600 shown in FIG. 6 is a device embodiment corresponding to the method embodiment shown in FIG. 3, and the corresponding description in the method embodiment shown in FIG. 3 is also applicable to the device 600. Go into details again.

According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method described in conjunction with FIG. 2 or FIG. 3.

According to an embodiment of still another aspect, there is also provided a computing device, including a memory and a processor, the memory stores executable code, and when the processor executes the executable code, a combination of FIG. 2 or FIG. 3 is implemented. The method described.

Those skilled in the art should be aware that, in one or more of the foregoing examples, the functions described in the embodiments of this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on the computer-readable medium.

The specific implementations described above further describe the purpose, technical solutions, and beneficial effects of the technical concept of this specification in further detail. It should be understood that the above are only specific implementations of the technical concept of this specification, and It is not used to limit the protection scope of the technical concept of this specification. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the embodiments of this specification shall be included in the protection scope of the technical concept of this specification within.

Claims (10)

A method for page clicking is applied to a running device that is connected to a pointing device in advance. The running device is used to run a target program. The pointing device establishes a first communication module with the running device through a first communication module and a second communication module. A communication channel and a second communication channel, the second communication channel being established based on the input dedicated interface of the operating device; the method includes: When a page to be clicked appears during the running of the target program, an operation request is sent to the pointing device through the first communication channel, so that the pointing device can convert the operation request into an operation event, and pass the The second communication channel provides feedback, and the operation event is used to describe the offset operation and/or click operation indicated in the operation request; Respond to the operation event to complete the click on the page to be clicked. The method according to claim 1, wherein the sending the operation request to the pointing device through the first communication channel comprises: Generate a positioning page and a first operation request, the positioning page is used to cover the page to be clicked, and to give priority to responding to the operation event sent by the clicking device, the page to be clicked at least corresponds to the first target position to be clicked and the target The first click mode of the first target position; The first operation request is sent to the pointing device, so that the pointing device generates a first click event according to the first operation request, and the first click event is used to describe that the pointing device is in accordance with a preset Click operation in click mode; The responding to the operation event to complete a click on the page to be clicked includes: Responding to the first click event through the positioning page, and determining the first click position of the current click; Based on the comparison between the first click position and the first target position, the click on the page to be clicked is completed. The method according to claim 2, wherein, when the first click position is consistent with the first target position, the responding to the operation event to complete the click return on the page to be clicked include: Cancel the positioning page; Send a second operation request to the pointing device through the first communication channel, so that the pointing device can convert the second operation request into a second operation event and feedback through the second communication channel. 2. The operation event is used to describe the click event corresponding to the first click mode; Respond to the second operation event through the page to be clicked, thereby completing the click on the page to be clicked. The method according to claim 2, wherein, in a case where the first click position and the first target position are inconsistent, the responding to the operation event to complete the click return on the page to be clicked include: Acquiring the coordinates of the first click position and the first target position in the screen coordinate system respectively; Determining a coordinate offset from the first click position to the first target position according to the coordinates of the first click position and the first target position; Generating a third operation request based on at least the coordinate offset, and sending it to the pointing device, so that the pointing device can generate a third operation event according to the third operation request; In response to the third operation event, the click on the page to be clicked is completed. The method according to claim 4, wherein the generating a third operation request based on at least the coordinate offset comprises: Determine the offset corresponding to the pointing device according to the coordinate offset and a predetermined conversion rule, where the conversion rule is used to describe the moving distance corresponding to the pointing device and the screen corresponding to the pointing device The mapping relationship between the offset distance of the click position; The third operation request is generated according to the offset corresponding to the pointing device and the first click mode, so that the pointing device generates the offset corresponding to the movement of the pointing device according to the third operation request and A third operation event of clicking according to the first click mode; The completion of clicking on the page to be clicked in response to the third operation event further includes: Cancel the positioning page, and respond to the third operation event through the page to be clicked. The method according to claim 4, wherein the generating a third operation request based on at least the coordinate offset comprises: Using the coordinate offset as the offset corresponding to the pointing device to generate a third operation request for the pointing device to generate a third operation event offset according to the offset; Responding to the third operation event through the positioning page, and determining a second click position generated by the third operation event; It is detected whether the second click position is consistent with the first target position, and a fourth operation request is generated according to the detection result and sent to the clicking device, and so on, until the click position generated by the operation event fed back by the clicking device Consistent with the first target position. A click method, applied to a click device that is connected to a running device in advance, the running device is used to run a target program, and the pointing device establishes first communication with the running device through a first communication module and a second communication module, respectively Channel and a second communication channel, the second communication channel is established based on the input dedicated interface of the operating device; the method includes: Receiving an operation request sent by the operating device through the first communication channel, where the operating request is generated by the operating device when a page to be clicked appears during the operation of the target program; Converting into an operation event of the clicking device according to the operation request, wherein the operation event is used to describe the offset operation and/or the click operation indicated in the operation request; Sending the operation event to the operating device. The method according to claim 7, wherein the operation event includes one of the following: Used to describe the click event performed by the clicking device according to a preset click mode; It is used to describe a movement event in which the clicking device moves according to a predetermined offset, and a click event in which the clicking device moves according to a preset click mode. A page click device, which is set in a running device that runs a target program, and the device includes: The sending unit is configured to send an operation request to the pointing device through the first communication channel when the page to be clicked appears during the running of the target program, so that the pointing device can convert the operation request into an operation event through the first communication channel. The second communication channel provides feedback, and the operation event is used to describe the offset operation and/or the click operation indicated in the operation request, wherein the click device is connected to the operation through the first communication module and the second communication module in advance. The device has established a first communication channel and a second communication channel, and the second communication channel is established based on the input dedicated interface of the operating device; The response unit is configured to respond to the operation event to complete the click on the page to be clicked. A point-and-click device, which is used to click on a page on a running device that has a pre-established connection; the point-and-click device includes: The first communication module is configured to establish a first communication channel with a running device, and to receive an operation request sent by the running device through the first communication channel, wherein the operation request is performed by the running device in the target program running process Generated when there is a page to be clicked in; A conversion module configured to convert into an operation event of the click device according to the operation request, wherein the operation event is used to describe the offset operation and/or the click operation indicated in the operation request; The second communication module is configured to establish a second communication channel with the operating device through the input dedicated interface of the operating device, and send the operation event to the operating device through the second communication channel.

Images